I've been stuck in testing hell the past few weeks, what with the changing [1] of the [2] old guard [3], and the new [DELETED-guards-DELETED] crew are heavy into the whole “unit test über alles” religion. I understand the reason behind testing, and unit testing in particular. The few times I've tried unit testing, it has been helpful, but …
That last bit might sound a bit glib, but it is a real concern. Even Knuth is quoted as saying, “beware of bugs in the above code; I have only proved it correct, not tried it.” I even joked about having to test our tests with both my current (but soon-to-retire) manager and his (soon-to-me-my) manager, because our current regression test has 15,852 individual tests (not all of them might be valid—hard to say, since the tests are automatically generated).
“Project: Lumbergh [4]” was not written with unit tests in mind. Or more specifically, it is a unit. The whole thing. And it's complicated because not only does it implement our business logic (which over ten years has gotten quite complex) but because it has to query multiple databases at the same time across the network (“at the same time” because it has to do its job as a phone call is being made). How does one deterministically test delayed (and/or dropped) queries?
Another issue—“Project: Lumbergh” sends a message to “Project: Cleese [5]” in some circumstances (“Project: Cleese” handles an HTTP (HyperText Transport Protocol) notification on behalf of “Project: Lumbergh” because trying to add a TCP (Transmission Control Protocol) connection in a program that deals with UDP (User Datagram Protocol) would have taken too much engineering time) and we need to check if “Project: Cleese” was notified. I solved that one by mocking the HTTP endpoint that “Project: Cleese” talks to with two interfaces, one for “Project: Cleese” and one for the regression test to query. The issue there is the regression test might ask the mock if it received a request from “Project: Cleese” before “Project: Cleese” gets the notification from “Project: Lumbergh” (a classic race condition). I got it working reliably, but now the regression tests takes over twenty minutes to run, instead of the two-plus minutes (even on the happy path, and I'm not entirely sure why).
It seems like management is more concerned about the tests rather than the product.