Not too long ago, William Wulf had what he calls an “inter-ocular event—a 2x4 between the eyes.” After returning to academia from the trenches of Washington, D.C., Wulf noted that colleagues focused almost exclusively, and serenely, on analysis. In his remarks, Wulf urges his audience to remember, “Every engineering project…occurs in a holistic, noisy, messy social environment.”

What are the implications of this reality as engineering designs become evermore complex? Wulf believes that new ethical problems are emerging for the profession. We are building systems, whether for computer networks or space travel, whose future actions are literally impossible to predict. Wulf asks, “How does one ethically build a system when you know there will be unintended, possibly catastrophic behaviors as a result—how do you act right under those circumstances?” He says this problem has already “come back to bite us in simple ways.” We often attribute security flaws in computers to viruses. Instead, Wulf cites a study where at least half of the breaches resulted from the system “performing exactly per specifications. But the specifications didn’t foresee a way they could be used to defeat the system.” One conclusion: “If you can’t get the specs right, better not assume the output will be right.”