Unnaturally Long Attention Span

As I mentioned before, I'm launching a new website called Diffbot on April 1st. Diffbot is a cool new kind of web-based RSS reader and bookmark manager. Have a handful of sites that you read daily? Diffbot lets you know when those sites have updated and only shows you the portion that changed. It's also just a convenient place to put your bookmarks so that they are accessible wherever you go. This is still very much a work in progress, so we'd really appreciate any feedback on how we could do better. You can now signup to get an invite when it's ready!

Many of my technical readers out there have job titles that match the regular expression "(Sr.|Jr.)? Software Engineer (I)*". For the non-geek readers, that means we call ourselves "Software Engineers" :-). But, what would you consider the difference is between someone who is a "Software Engineer" and someone who's title is a "Computer Programer"? They seem like similar lines of work, yet one seems to imply a higher level of education, perhaps at least a 4-year college instead of a technical trade school. Historically speaking, there has been a huge difference between an engineer and a programmer. During the second World War, "computers" where actually women that caculated artilliary projections using desk calculators. These women became the first computer programmers when they were assigned to program the ENIAC, a room-sized computer with 18,000 vacuum tubes. Engineering, on the other hand, implied not the people that operated the machines, but those that designed the system and solved the larger problems.

Most major universities separate the School of Engineering from the School of Sciences. Engineering includes departments like chemical, mechanical, nuclear, bio and electrical engineering. Science includes departments like chemistry, physics, biology and computer science. Although, it seems like there's a lot of duplication here with the sciences, supposedly, this is because engineering has some common skillset. Engineering cirricula require a certain type of maths. Engineers usually study topics like design, tolerances, robustness, production processes, and technical writing.

However, let's return to what is software engineering? One of the most popular degrees that Software Engineers graduate with is Computer Science, which is not an engineering degree. What I'd like to argue--and this point has been made before--is that software engineering, is not only not taught in formal education, but is consequently different from the other types of engineering. This is why large companies recruiters complain about fresh college grads knowing nothing about debugging, why there are so many software internships, why most software engineering jobs require a few to several years of prior experience--its because that's when you actually learn some "software engineering"!.

However, I say software engineering is fundamentally different from other types of engineering, because the field itself is still in its prenatal state. Maybe that is one of the reasons why it hasn't been developed yet in formal education; we don't really know yet what are the best practices and fundamental formulas in software engineering. The level of engineering that goes into building even the largest software projects is nothing like the level of engineering that goes into building something like a car. It's more like the amount required to build a snowman. Even when I was working at Microsoft on the Windows source code, the Hoover Dam of software engineering, there was very little engineering in place to manage the complexity and uncertainty. To get a sense of this, compare the reliability of your office buiding to Microsoft Office. If your plumbing system or electric system failed as much as my MS Outlook or Firefox crashes, you would be a very unhappy camper. Yet for information workers, both things are equally important to their day.

Software engineering is a newer discipline than mechanical engineering, or even electrical engineering. Obviously, the software world is undergoing a very rapid change right now. We haven't had time yet to sit back and understand the principles and fundamental formulae that govern software. There are lots of well-specified problems, where there is no agreement on what is the best algorithm. Sure, there are small groups of people every studying problems like software reliability, static source code analysis to identify software weaknesses, and theoretic guarantees for software correctness and performance. I think these efforts will become increasingly important.

I've explained in my mind what the distinction between software and other forms of engineering are, but why do I think this is an important issue? There's no problem with working in a field that is largely unstructured, complex, and ad-hoc. That's part of the excitement of being in a brand new field. Life is great as a software engineer. The problem is that software is increasingly replacing the function of physical objects and electrical components. That is, computers are used to replace other things. Your typewriter has been replaced by your word processor. The control center of your car has been replaced by a small computer running an embedded operating system. Your telephone has been replaced by a small computer which emulates the phones functions. The stock market itself has been infused with tons of small programs, trading trillions of your dollars. Take the typewriter as an example, the mechanical engineer that designed it knows that unless the few joints between the key and the hammer fail, your keystroke will translate into a mark on the paper. The materials in the product have been carefully chosen with respect to their well known structural flexibility, strength, and mass. I won't even begin to explain all the things that could go wrong between the time you hit a key on your computer and see a letter appear on screen. In this "design", the components involved were chosen because they seem to work. It's crazy talk to try to estimate how reliable this design is. Yet, this fundamental unreliability is what we entrust to keep our airplanes in the air, our cars on the highway, our bank accounts and financial markets secure. It's just a matter of time before a catastrophic software failure occurs (many major ones already have), or we decide to design responsible software. Software that works as reliably as a toaster. Software that just doesn't break, no matter what the user does.

Most Popular Posts:
Ranking Colleges Using Google and Open Source