One cannot talk about the problems of the software industry without talking about software education. Years ago I was working on a process improvement activity. The goal was to improve our interviewing techniques for June Grads. We did a typical matrix of qualities we wanted versus interview techniques. Part of the exercise was to rank the desired qualities in order of importance. (We then focused on the interview techniques that yielded the best weighted scores.) The surprising thing that came out of that exercise was that things like coding skills, knowledge of data structures, and knowledge of algorithms ranked very low while things like communications skills, teamwork, and problem solving ranked much higher. IOW, we were interested in things that the CS departments were not teaching.
IME, most CS departments focus too much on details and elegance. They teach students how to code, but not how to design. They teach students LISP and Haskell because they are elegant and interesting while not mentioning that they are useless for commercial applications because they are too slow and/or virtually unmaintainable for large applications. They teach behemoth infrastructures like CORBA when any communications can be handled much more efficiently and reliably with simple messages having by-value data packets. Worst of all, they teach the mechanics software development activities, but not the overall process of software development. IOW, the CS departments are producing code hackers instead of software engineers.
Another of my pet peeves is textbooks where the author delivers the content in Sermon on the Mount mode. The author provides a vast bundle of tehcniques, practices, and guidelines as the proper way to do development without explaining why those are good things to do. They don’t talk about how to apply a few fundamental principles; instead they provide a plethora of cookbook tools derived from those principles. That leaves the student up a creek when a problem where the guidelines don’t quite work is encountered. Nor does it help when different guidelines enable plausible alternatives from which the student must choose. (Shameless plug: in my book I focus so much on explaining how one should think about design that I sometimes deliberately lead the reader to a poor design just to prove that the methodology is self-correcting, so that one will realize it is a poor design by thinking about basic principles.)
A corollary problem is that quite often authors don’t talk about things that they should talk about because they don’t want to confuse their readers with “unnecessary complexity”. A classic example of this occurs in most OOA/D books today. You could count on one hand the number of such books that mention Normal Form. In fact, every Class Diagram should be normalized just like a relational database. That’s because both are firmly grounded in the same set theory where Normal Form is an elegant way to ensure consistency and, more importantly, to avoid ambiguity. Similarly, you could count on one hand the number of OO books that discuss using a variation on design-by-contract (DbC) to resolve state machine interactions correctly. It is tedious, so experienced developers only use it when things are really confusing, but it is still an invaluable tool to ensure correctness in complex situations.
For that matter, try finding a methodology book outside R-T/E that describes how to use finite state machines. Why? The IT community thinks using state machines requires too much skill. It is true that getting state machines to work initially is tricky. But once they do work, they tend to be very reliable and robust in the face of volatile requirements. However, the real reason I think that attitude is, at best, condescending is because it is mired in a view of IT from the ‘60s when financial institutions had acres of entry-level COBOL programmers churning out MIS reports. Today IT is a very different beast. With multitasking, online processing, interoperability, concurrent processing, asynchronous real-time input, ubiquitous networking, and multi-user systems, IT is beginning to look a whole lot like R-T/E. It is time the IT industry moved into the 21st century and accepted that (A) minimal skills sets don’t cut it anymore and (B) you can’t hide complexity behind massive infrastructures without paying the tolls in performance, reliability, and maintainability.