The value of “programmer tests” is far from being a consensus. I am not even talking about Test-Driven Development here, just the practice of programmers building up a suite of automated tests that validate a piece of software that they are working on.

By the way, I am totally on the programmer test bandwagon, have been since I read the “Test infected” article by Beck and Gamma back in 2001. When I see luminaries in the software development field dissing programmer tests, I am totally baffled.

Let me try an analogy. I wonder if they would also question the value of a good set of test fixtures for a language implementation (be it a compiler, interpreter, virtual machine, what have you).

…when I see some household names in the software development field dissing programmer tests, I am totally baffled.

Anyone that has ever set eyes on the codebase for a compiler or interpreter for any programming language must have noticed a large set of test programs that the compiler development team maintains to ensure the compiler is correct, exploring how the compiler or interpreter handles multiple variations of features in the source language. Often the tests not only look for pass/fail outcomes, but inspect the result to ensure the expected output was produced. This is not a modern practice – I am going to guess that it has been the common practice since compilers existed (that is more than 60 years now), because it is just too obvious and technologically trivial to have been missed even in the first years of Fortran.

The way I (and many others) see it, any program for any domain is implementing a language. This language is made of concepts that exist in the domain at hand, and defines the valid ways you can deal with those concepts. For instance, for an ecommerce application, you have products, and a shopping cart. You can add products to a shopping cart, you can change the quantity for a specific item or remove it, you can empty the shopping cart, you can proceed to checkout (if your cart is not empty). That is a language, and the application implements that language, just like a compiler, or interpreter.

In other words, there is no fundamental difference in testing compilers and business applications.

Automated tests are analogous to test programs that you submit to a compiler or interpreter to test it. You play with some feature and see if it produces the expected results. You try variations that you suspect could be tricky for the language implementation to handle. You look for interactions between features that you suspect may result in some weirdness. You do that for every feature you can think of, from the most important, to the most trivial, and you can only be sure the language implementation is complete when, for every language feature, you have at least one instance of a test input that exposes that feature.

What you don’t do is tell your QA teammates to take your compiler for a spin and see if it works, or ship and hope your users will do it, and wait for bug reports for the many silly mistakes you will for sure have made.

If you are a responsible developer, you just don’t ship code to the poor users if you haven’t yourself submitted the code to a decent battery of test cases that ensure that, at least in the situations you and your team could think of, the program being tested performs as expected.

Also, writing programmer tests is a sign of empathy towards your fellow developers. When you add test coverage for a feature you implemented, you are also providing a safety net for your fellow developer teammates, which will be making changes to the codebase. By having a solid test suite, they don’t have to manually ensure not only the feature they are working on is correct, but that they haven’t broken any other feature ever implemented by the team.

…writing programmer tests is a sign of empathy towards your fellow developers.

Of course, programmer tests are not a guarantee that there will never be bugs lurking somewhere in the codebase being tested. Even the most used compilers for programming languages have bugs found by their users. You may still want QA in the team. And you will still get bugs reported by users. But QA, and of course, users, finding bugs should be the exception, and limited to non-obvious scenarios. [Of course, when you get a bug reported, you write a test case that reproduces that bug, fix the bug so that test - and all existing tests - now pass, and you just made your test suite a little better.]