Facts and Fallacies of Software Engineering

Rating:
Summary: (Not so) common wisdom about building software
Review: Bob Glass is a programmers' programmer -- he's at the other end of the scale from software engineering gurus. This collection of fifty-five software facts and ten fallacies is distilled from his forty-five years of programming experience. Much of it is the kind of thing your grandmother could have told you -- if your grandmother was a programmer -- but some items will surprise and others annoy some people.

Take the following two items as examples:

"The 60/60 rule: 60 percent of software's dollar is spent on maintenance, and 60 percent of that maintenance is enhancement."

"Understanding the existing product consumes roughly 30 percent of the total maintenance time."

That implies that one of the most valuable skills you can teach a Computer Science student is how to *read* code. But as Glass points out (and he taught graduate students for a while) CS courses only teach students to *write* programs, and then they don't often grade the code on readability.

The section on design, which Glass describes as the most intellectual phase of a software project, is the best description of how software designers actually work that I've ever read. He claims that top-ranked designers routinely ignore or subvert the methodologies used by their shops in order to do the job the right way. He has an especial warning for anyone trying to develop anything other than the most trivial program using XP.

The ten fallacies may have taken some well-known quotes out of context. Surely Eric Raymond of the open source movement didn't really mean, "Given enough eyes all bugs are shallow", to be taken literally. Either way, the point Glass is trying to make is that, without proof, we shouldn't *assume* that open source code is less buggy than proprietary code.

Facts and Fallacies of Software Engineering is aimed at two audiences: those without Glass' 45 years of programming experience, and those who have it but, like all good professionals, want to refresh the basics. Since some of the wisdom came with useful statistics that I've never seen published before, I'd recommend the book to anyone involved in developing or maintaining software.

Rating:
Summary: Lots of good starting points
Review: Glass is an opinionated old coot, not afraid to speak his mind no matter what the source of the silliness he sees around him. These aren't the deepest thoughts on the business, craft, and engineering of software, but they cover a lot of important ground. Best, these aphorisms come from hard-won experience in many areas. I don't disagree with any of them, but I have to qualify my agreement in a few places.

For example, #32: "It is nearly impossible to test software at the level of 100% of its logic paths." I'd go farther and say it's usually dangerous even to try. If some pointy-haired manager decides that more coverage is better, programmers will dutifully optimize that metric. Error recovery is hard to test, so they'll edit out the error recovery. Integrity checks may be hard to trigger, so the integrity checks have to go. Trust me, this is not an improvement.

Or #51: "Residual errors will always persist." No argument there. "The goal should be to minimize or eliminate severe errors." That's one good goal, but there are lots of systems where the goal should be to continue service in the presence of errors, via error recovery, some fail-soft state, or other means. Even perfect software may violate its contract in the presence of hardware errors: given a one-in-a-billion error rate and 3GHz clock, errors happen three times per second. Even at one-per-trllion, they happen every five minutes or so.

Also, #46, a prioritized list of "-ilities" that define quality. Of course I disagree with his or any order. Heck, in one system I worked on, four different subsystems had four different lists. Reliability was paramount in one area - software failure would have to be repaired by unsoldering and replacing a chip. Other subsystems variously emphasized speed, compiled code size, and user-friendliness. In the bulk of the system, maintainability was at the top of the list (a fifth different priority list).

Although I generally like the content and style, a few things might have strengthened the book. Glass' favorite author, based on citation counts, is Glass. He's good, but so are other authors. This book isn't really self-contained. Lots of discussions are cursory (and he sometimes points that out), requiring a lot more elaboration of his part or a lot of reference-chasing time on the reader's part, assuming the reader can find the references at all.

Still, this is a welcome comment on the current state of affairs in software development. His one point - if there is only one - is that it's the same state we were in back in the 70s. Only the buzzwords have really changed. That point I agree with.

//wiredweird

Rating:
Summary: A valuable and easy to read summary of the state of the art
Review: I have read a fair number of software engineering books, and this is one of the more enjoyable books that I have read. When I first heard about it, I thought the concept of a sort of summary of the state of the art sounded really interesting. Although I haven't read any of the author's previous books, I have read and enjoyed his columns in IEEE Software and Communications of the ACM, so I had high hopes about this book. And I wasn't disappointed.

Facts and Fallacies of Software Engineering is divided into 55 facts and 10 fallacies. Each fact and fallacy is presented in the same way. There is a headline/slogan that summarizes it, usually one or two pages of Discussion giving more details, then a Controversy section describing what (if anything) people disagree about and finally Sources and References.

The 55 Facts are divided into the following sections and sub-sections: Management (People, Tools and Techniques, Estimation, Reuse, Complexity), Life Cycle (Requirements, Design, Coding, Error Removal, Testing, Reviews and Inspections, Maintenance), Quality (Quality, Reliability, Efficiency) and Research.

The 10 Fallacies are divided into Management, Life Cycle and Education.

This way of organizing the material works really well, and makes the book very accessible and easy to read. It also allows you to jump around and read what interests you the most first (which is what I did, although I eventually read all of it).

Many of the facts are well known (for example Fact 3 "Adding people to a late project makes it later", Fact 16 "Reuse-in-the-large remains a mostly unsolved problem" and Fact 24 "Requirement errors are the most expensive to fix during production"), but that doesn't matter. It is actually good to be reminded of these facts even if you already know them, and the author does a very good job of summarizing them.

Another thing I like about the book is the Sources and Reference parts (although I think they might as well have been combined into just one Reference section). Often there are references to research papers where the original fact was presented. It is nice to know that what is presented as a fact is indeed a fact that has been validated by research, and not just the opinion of the author (although there is certainly room for opinions in a lot of places as well).

There are also lots of references to other books on software engineering, and a lot of the classic books (like The Mythical Man-Month, Peopleware and Design Patterns) are referenced. So there is plenty of leads in case you want to find out more about a certain fact.

Among the facts I liked the most were Fact 12, Fact 21 and Fact 26.

Fact 12: "Since estimates are so faulty, there is little reason to be concerned when software projects do not meet estimated targets. But everyone is concerned anyway". This fact and the related ones simply state that when a project is late, it is very likely because the estimated time to complete it was unrealistic. Very true.

Fact 21: "For every 25 percent increase in problem complexity, there is a 100 percent increase in complexity of the software solution." I had never seen this fact before, but it does ring true to me. And as the author writes, it explains a lot of the other facts in the book as well.

Fact 26: "Explicit requirements 'explode' as implicit (design) requirements for a solution evolve". In other words, each explicit requirement leads to many more implicit requirements (by a factor of up to 50). This too I had never seen stated, but I definitely recognize this from my own experience.

The Fallacies section list ten common misunderstandings in software engineering, and this is where I disagree on a couple of points. Fallacy 7 states "Random test input is a good way to optimize testing". I agree that it can not be the only test approach, but since he also writes "It may or may not survive as a test approach", he is skeptical to it in general. My own experience is that it is an invaluable complement that helps flush out numerous what I call "timing dependent" bugs caused by the nature of asynchronous events.

I also don't think all his arguments in Fallacy 8 are valid. I agree that since there is no data demonstrating the truth of "Given enough eyeballs, all bugs are shallow", we should not just accept it as truth. But I think he misses the point when he refers to research showing that inspections don't get much extra benefit beyond 2 to 4 participants. My interpretation is that the "enough eyeballs" are not so much inspecting the software in question as running it and debugging it when there is a problem. And the "all bugs are shallow" should not be interpreted too literally. Of course the bugs may still be difficult, but if many people look at it, the chances of someone solving it fairly quickly increases.

Those two examples notwithstanding, I did find myself nodding my head and saying "yes, I agree with that" almost all of the time reading this book.

There are many more interesting facts that I have not commented on, and if you are interested in software engineering I highly recommend this book.

Rating:
Summary: On Computational Feasibility
Review: I recommend this book to any software person who wants to find out why most software projects that fail, fail.

But my specific interest revolves around writing software in large engineering (hard engineering) companies, in that very peculiar environment. Specifically, the hard engineering environment in which knowledge of the programming language is considered the most advanced software theory needed to produce a product, and in which Electrical Engineers without any formal education in advanced data structures, algorithms to manipulate them, search strategies, and heuristics are writing most of the code.

I am seeing HUGE software projects fall FAR short of their schedulled goals, because those doing the coding have simply used up all their computational cycles (I'm talking about real-time software). In these situations, managers seem to imply
1. they were blindsided
2. there are no ways to forsee such computational box canyons
3. no one could have done better, anyway.

On all 3 points, managers are wrong. And the substance on which they are so badly wrong, would make for another 10 fallacies in Glass' book. For example:

Fallacy N: Hard engineers produce advanced software algorithms.
The average hard engineer is writing Freshman undergraduate code, and making basic errors in design.

Fallacy N+1: Simple code (meaning no Computer Science theory) is much more efficient than highly tailored software algorithms. Actually, it is often orders of magnitude less efficient.

Fallacy N+2: Analytical mathematical solutions are the only ones worthy of respect. Actually, with the incredibly complex problems in modern engineering, closed form mathematical solns may never be found. Many such solns are so computationally expensive, that they never could be practical solutions.

Fallacy N+3: There are no analytical methods for identifying algorithms with prohibitive computational complexity--you must just write the software and try it out. Actually, there is a whole field of C.S. that does this.

Fallacy N+4: Modern Engineering problems are unique. Actually, the tar pits of computational complexity remain pretty stable, and Artificial Intelligence has amply failed in most of them (such as the Frame problem, and Automated Reasoning). Managers are just ignorant of the tar pits.

Fallacy N+5: Next year we'll get a new processor which is 60% faster, and our problems will go away. Actually, the mathheads that keep saying this have still not figured out that the disastrous software algorithms have computational growth rates that are exponential, and linear growth in processor speed does not define a solution to this problem.

Fallacy N+6: We write modular software. Actually, "modular" to a hard engineer and "modular" to a software engineer are radically different things. I regularly see "modular" software that has no defined interface, and no behavioral contract, so does not meet even basic requirements for reuse. Glass does not address the radically different semantics given the same words, by software and hard engineers. "Algorithm" is another word with very different meanings.

Fallacy N+7: Rule-Based software will, of course, solve our problem. Actually, automated reasoning has large undecidable and intractable areas. Engineering companies are just beginning to step in this tar pit, 25 years behind Artificial Intelligence.

Fallacy N+8: Knowledge of the syntax of a computer language, is the most valuable asset in software design. Actually, this is a trivial skill, relatively. The ability to represent symbolically complex reality in code structures, and complex manipulations of those structures efficiently, is orders of magnitude more valuable. Knowledge of just language syntax leads to literalistic algorithms, which tend to be brute force.

To his credit, Glass did mention...
Fallacy: Real-time code optimizers will fix any slowness in execution. Actually, you may gain 10-18% in speed this way, but these failing projects are orders of magnitude slower then needed. Better get rid of the real-time optimizers, and hire highly educated software designers.

I REALLY, REALLY appreciate Glass. But someone with a modern Masters degree in C.S. would realize that there is no reason to be blind-sided by most of these software disasters, and there are proven ways of evaluating these risks of failure while still in the design stage. Most VERY VERY bright hard engineers don't have the basic theory to detect computational risk, and don't even know where they should acquire it, if they wanted to improve their software algorithm design ability.

The problem is not just that (as Glass mentions) managers of software projects are out of touch with the technical people who write software. The disconnect is rather worse: most of the people who write software in big engineering companies are hard engineers, with no theoretical background in software algorithm design. There is an unspoken "amateur ethic" which condemns formal design theory. Technical leads of these "programmers" are often as uneducated, and could not identify sound software designs if their career depended on it. (So they are continually promoting Public Relations designs, and being blind-sided by failures.)

The situation in software design in America remains "tragic," and anti-intellectual, and the amateur (KISS) ethic continues to produce software projects which are computationally orders of magnitude outside of feasibility. Process does not begin to address this problem (Glass alludes to this). Nor does the strategy to redouble expectations toward the 'amateur coding ethic' and stop being such a nay-sayer. The truth remains that most technical leads in engineering companies, cannot even recognize what a software design is, and have no theoretical tools to analyze computational feasibility.

I wish that someone would address this problem in basic "Facts and Fallacies" books.

Stephen Wuest, Raytheon, M.S. in C.S. and A.I., 1999.

Rating:
Summary: Maybe the most important book you will ever read
Review: Once again, Glass has proven that he belongs in the software engineering pantheon along with Tom DeMarco, Gerry Weinberg, and Steve McConnell.

This book will open your eyes. If you work in the field, you'll never think about your livelihood the same way again.

If you take only one thing away from this book, remember this: don't blindly trust what the advocates of the latest methodology are saying, whether it be OO, XP, RUP, or UML, without some substantive evaluative research backing them up. Glass makes compelling arguments as to why the software industry has fallen easy prey to the hucksters and snake-oil salesmen.

Rating:
Summary: Maybe the most important book you will ever read
Review: Once again, Glass has proven that he belongs in the software engineering pantheon along with Tom DeMarco, Gerry Weinberg, and Steve McConnell.

This book will open your eyes. If you work in the field, you'll never think about your livelihood the same way again.

If you take only one thing away from this book, remember this: don't blindly trust what the advocates of the latest methodology are saying, whether it be OO, XP, RUP, or UML, without some substantive evaluative research backing them up. Glass makes compelling arguments as to why the software industry has fallen easy prey to the hucksters and snake-oil salesmen.

Rating:
Summary: Full of Amazing "Facts"!!!
Review: Some of the more interesting and amazing "facts":

Fact 18: Reusable components are 3 times as hard to build and should be tried out in 3 different settings.

Fact 19: Modification of reused code is particularly error-prone. If more than 20 to 25% of a component is to be revised, it is more efficient and effective to rewrite it from scratch.

Fact 21: For every 25% increase in problem complexity, there is a 100% increase in the complexity of the software solution. That's not a condition to try to change (even though reducing complexity is always a desirable thing to do); that's just the way it is.

- Why is reuse-in-the-large unsuccessful? (Because complexity magnifies diversity.)

- Why are there so many different correct approaches to designing the solution to a problem? (Because the solution space is so complex.)

Fact 26: When moving from requirements to design, there is an explosion of "derived requirements" (the requirements for a particular design solution) caused by the complexity of the solution process. The list of these design requirements is often 50 times longer than the list of original requirements.

Fact 30: COBOL is a very bad language, but all the others (for business data processing) are so much worse.

Fact 33: Even if 100% test coverage were possible, that is not sufficient criterion for testing. Roughly 35% of software defects emerge from missing logic paths (missing requirements), and another 40% from the execution of a unique combination of logic paths (Combinatorics). They will not be caught by 100% coverage.

Fact 42: Enhancement is responsible for roughly 60% of software maintenance costs. Error correction is roughly 17%. Therefore, software maintenance is largely about adding new capability to old software, not fixing it.

- The 60/60 rule: 60% of software's dollar is spent on maintenance, and 60% of that maintenance is enhancement. Enhancing old software is, therefore, a big deal.

Rating:
Summary: Good for Quoting
Review: The book is an excellent source for quoting. I love to send small bits of it around the office, to make me look wiese.
Besides from this, it contains only old software wisdom. Nothing new and no added value. So, if you read several books about software development, you will allready know the contents of this one.

Rating:
Summary: Insightful To The New Manager/Team Leader
Review: The other reviewers have done a fine job of covering the content of the book. I will comment about its usefulness. In short, this book is truly valuable to the developer who has recently been promoted to team leader. While developers would benefit greatly from this book, the reality is that most developers would rather read books like "Effective C++", "Design Patterns", "Expert One on One Oracle", etc. To the new manager, though, this book is a gem. The book talks about specific management issues as well as the development life cycle and quality. In short, the book focuses exactly on what the team leader does and the team leader's team. In addition to the material presented in the book, the author gives a great number of sources and reference for further reading.

Rating:
Summary: Insightful To The New Manager/Team Leader
Review: The other reviewers have done a fine job of covering the content of the book. I will comment about its usefulness. In short, this book is truly valuable to the developer who has recently been promoted to team leader. While developers would benefit greatly from this book, the reality is that most developers would rather read books like "Effective C++", "Design Patterns", "Expert One on One Oracle", etc. To the new manager, though, this book is a gem. The book talks about specific management issues as well as the development life cycle and quality. In short, the book focuses exactly on what the team leader does and the team leader's team. In addition to the material presented in the book, the author gives a great number of sources and reference for further reading.