Unnaturally Long Attention Span

AvatarA blog about Grad School at Stanford, Working in a Silicon Valley Internet Company, and Statistical Machine Learning. mike AT ai.stanford.edu

Engineering Software

Many of my technical readers out there have job titles that match the regular expression "(Sr.|Jr.)? Software Engineer (I)*". For the non-geek readers, that means we call ourselves "Software Engineers" :-). But, what would you consider the difference is between someone who is a "Software Engineer" and someone who's title is a "Computer Programer"? They seem like similar lines of work, yet one seems to imply a higher level of education, perhaps at least a 4-year college instead of a technical trade school. Historically speaking, there has been a huge difference between an engineer and a programmer. During the second World War, "computers" where actually women that caculated artilliary projections using desk calculators. These women became the first computer programmers when they were assigned to program the ENIAC, a room-sized computer with 18,000 vacuum tubes. Engineering, on the other hand, implied not the people that operated the machines, but those that designed the system and solved the larger problems.

Most major universities separate the School of Engineering from the School of Sciences. Engineering includes departments like chemical, mechanical, nuclear, bio and electrical engineering. Science includes departments like chemistry, physics, biology and computer science. Although, it seems like there's a lot of duplication here with the sciences, supposedly, this is because engineering has some common skillset. Engineering cirricula require a certain type of maths. Engineers usually study topics like design, tolerances, robustness, production processes, and technical writing.

However, let's return to what is software engineering? One of the most popular degrees that Software Engineers graduate with is Computer Science, which is not an engineering degree. What I'd like to argue--and this point has been made before--is that software engineering, is not only not taught in formal education, but is consequently different from the other types of engineering. This is why large companies recruiters complain about fresh college grads knowing nothing about debugging, why there are so many software internships, why most software engineering jobs require a few to several years of prior experience--its because that's when you actually learn some "software engineering"!.

However, I say software engineering is fundamentally different from other types of engineering, because the field itself is still in its prenatal state. Maybe that is one of the reasons why it hasn't been developed yet in formal education; we don't really know yet what are the best practices and fundamental formulas in software engineering. The level of engineering that goes into building even the largest software projects is nothing like the level of engineering that goes into building something like a car. It's more like the amount required to build a snowman. Even when I was working at Microsoft on the Windows source code, the Hoover Dam of software engineering, there was very little engineering in place to manage the complexity and uncertainty. To get a sense of this, compare the reliability of your office buiding to Microsoft Office. If your plumbing system or electric system failed as much as my MS Outlook or Firefox crashes, you would be a very unhappy camper. Yet for information workers, both things are equally important to their day.

Software engineering is a newer discipline than mechanical engineering, or even electrical engineering. Obviously, the software world is undergoing a very rapid change right now. We haven't had time yet to sit back and understand the principles and fundamental formulae that govern software. There are lots of well-specified problems, where there is no agreement on what is the best algorithm. Sure, there are small groups of people every studying problems like software reliability, static source code analysis to identify software weaknesses, and theoretic guarantees for software correctness and performance. I think these efforts will become increasingly important.

I've explained in my mind what the distinction between software and other forms of engineering are, but why do I think this is an important issue? There's no problem with working in a field that is largely unstructured, complex, and ad-hoc. That's part of the excitement of being in a brand new field. Life is great as a software engineer. The problem is that software is increasingly replacing the function of physical objects and electrical components. That is, computers are used to replace other things. Your typewriter has been replaced by your word processor. The control center of your car has been replaced by a small computer running an embedded operating system. Your telephone has been replaced by a small computer which emulates the phones functions. The stock market itself has been infused with tons of small programs, trading trillions of your dollars. Take the typewriter as an example, the mechanical engineer that designed it knows that unless the few joints between the key and the hammer fail, your keystroke will translate into a mark on the paper. The materials in the product have been carefully chosen with respect to their well known structural flexibility, strength, and mass. I won't even begin to explain all the things that could go wrong between the time you hit a key on your computer and see a letter appear on screen. In this "design", the components involved were chosen because they seem to work. It's crazy talk to try to estimate how reliable this design is. Yet, this fundamental unreliability is what we entrust to keep our airplanes in the air, our cars on the highway, our bank accounts and financial markets secure. It's just a matter of time before a catastrophic software failure occurs (many major ones already have), or we decide to design responsible software. Software that works as reliably as a toaster. Software that just doesn't break, no matter what the user does.

Most Popular Posts:
Ranking Colleges Using Google and Open Source


7:51 PM, March 12, 2006 shanguelei manguelei said...

your latest blog entry makes me want to get into software engineering. Then reality set in and I realized that I am irrational

12:15 AM, May 15, 2006 Anonymous said...

I find it weird how one person hate microsoft but love linux?I mean it seem to me it so emtional the hate,I ahve tried firefox and I dislike that my favorite places is gone.So back I am with bill gate :) new microsoft operating systemhttp://www.downloadcomputergames.com/

6:56 AM, June 02, 2006 Vladimir Misev said...

... Software that works as reliably as a toaster. Software that just doesn't break, no matter what the user does.

And what we have now is quite opposite:
NetBSD Toaster

NetBSD at LinuxWorld 2005

~cvk - how to script toaster

anyway, very interesting article...

7:38 AM, June 02, 2006 Steve Steiner said...

One need only read about the Therac-25 disaster to agree that there is great risk to replacing physical components with software.

However it is good to move beyond the knee jerk reaction of lumping all software into the high risk category.

It simply isn't the case that a failure in my online scrapbook is as bad as a failure in my air traffic control software.

Beyond a certain point, I'm far more concerned about how fun my game software is rather than how reliable it is.

1:35 PM, June 02, 2006 Mike said...

Of course, you're right. Not all software needs to be bulletproof--our expectations have been kept pretty low. But the technology and tools aren't there for applictions that really do need to be bulletproof. Is it really acceptable that credit card companies and other major institutions keep "losing" our personal information?

9:59 PM, May 28, 2009 control valves said...

I believe construction of such projects requires knowledge of engineering and management principles and business procedures, economics, and human behavior.

1:09 AM, July 16, 2009 Anonymous said...

Welcome to lineage 2 adena Our WoW Gold, wow power leveling Online Maple Story mesos Store wow gold for World Guild Wars Gold Of Warcraft power leveling, Cheap 2moon dil WoW Gold, FFXI Gil World Of lotro gold Warcraft Gold, final fantasy gil warcraft goldlord of the rings gold