'Opt-In' Changes and the Importance of Testing Suites

A development mistake I see all the time is when a developer fails to make changes to core classes 'Opt-In' or backwards compatible. For example, suppose you find a bug in your table/grid widget that causes one of the several instances of that widget to fail mysteriously. You come up with a solution that involves an extra method call or two at a certain point, or changing an option/property to another value. Your first instinct might be to make that change the default behavior, but in most cases, that's wrong.

Minimize Affect

Summary: How many instances of that widget do you have in your application? Are they all failing? Why not? Were they functioning properly before? If they ain't broke, why fix 'em?

Most likely the change you are going to make really should have been the default behavior all along, however, you now have working code that was developed and presumably tested against the old behavior. If you change that behavior now, even if it is now the 'right' behavior and was previously the 'wrong' behavior, you now need to test every single instance and subclass of your widget in all possible scenarios to ensure you haven't introduced a bug. Do you have a comprehensive automated testing suite? If you do, great, but you probably don't.

Even if you do have a good test suite, why fix something that isn't broken? Suppose, instead of making that change the default, you instead add a useFooHack property to your base class. It defaults to false, but if it's true, you use your new behavior. When you discover that other instances have a problem caused by the old behavior, you just change the flag to enable Foo, and it's fixed. Easy as that. If you discover that every last instance has the problem, you can remove the useFooHack=true line from your instances and if(useFooHack) from your base class. Simple. The alternative is to risk introducing bugs into every instance that you thought was working.

Testing is a waste of time

Now some people will learn of problems like this and say you should have done more testing before committing. You know, you should have gone through all the major paths of your application to ensure that they're working before committing. Most of the time those people are managers who don't have to waste time doing the testing themselves, but expect you to do it.

Suppose a developer spends X time testing before a check-in and finds some bugs that take Y time to fix, it has cost him X+Y time. However if he instead verifies that his code does what he intended it to do and checks in, but someone discovers the bugs, emails him, and he spends Y time fixing it, it only cost him Y time. Now if X is small, say 30 seconds, because all the developer had to do was kick off the automated testing suite, then the developer will do the testing to avoid getting angry emails. However, if testing takes more than a few minutes, most people will realize they can get more work done by letting someone else hunt for bugs. You can argue about it all you want, you're not going to change human nature, and the optimal choice most of the time is to skimp on the testing.

In short, you have to work with people the way they are. No amount of training or lecturing will keep people doing things that are not in their own best interest. If your project's big problem is that people keep breaking the build, make not breaking the build the easy path. You may have to sacrifice some hours to developing and maintaining a test suite, but you will generally save more hours of lost productivity from broken builds.