There is definitely a culture clash at work with regard to testing [1]. I was involved with a four hour discussion about our current regression test with the newer members to the department. The new manager, AG, was saying the regression test should test all the paths in the code. I was saying, yes, that is possible, but many of the new tests would slow the regression test down to a crawl. Then CZ would chime in saying it should be possible to test without slowing down to a crawl. Both AG and I countered that some of the tests involve a timeout, because how would one otherwise test a negative? CZ said we could check the logs, since the logs are logged in a particular order, but I countered that our previous regression test (which runs with the Protocol Stack From Hell [2]) takes over five hours because it checks the logs to ensure we don't miss a log. Using the logs to check to see if something happened isn't deterministic, because the check might come before the action is even logged, and you are still trying to test a negative (in this case, something that should not happen, did not happen, across a distributed, multi-process system) and that slows down the regression test.
We had to leave it with an agreement to disagree on the details for the time being.
Also discussed was a reversion of code in “Project: Lumbergh [3].” I felt the reversion reverted too much and involved parts that weren't part of “Project: Lumbergh.” CZ felt that was fine, because the changes to the other parts were a part of “Project: Lumbergh” if only indirectly. I countered that in that case, he should have reverted the changes in a different repository that the regression test lives in because that too, indirectly relates to “Project: Lumbergh.” CZ said no, it's in a different repository and thus, shouldn't be reverted. I retorted that the changes he reverted weren't part of “Project: Lumbergh” directly, just as the regression test isn't part of “Project: Lumbergh” directly, and it's only a historical artifact of how our stuff was developed that “Project: Lumbergh,” “Project: Wolowizard [4],” and “ Project: Clean-Socks [5]” (plus a few others I haven't mentioned) all share a single repository, and that our other repository contains “Project: Sippy-Cup [6],” “Project: Cleese [7],” “Project: Seymore [8]” and our current regression test.
We had to leave it with an agreement to disagree on the details for the time being.
Also related to the repositories, we also had a discussion about the versioning software we use. I mentioned I prefer git [9] but we're stuck with SVN [10] for the moment, with the only thing I like about SVN is the ability to checkout a subdirectory in a repository—it's certainly not because branches are easy, or SVN tags make any sense to me. CZ likes SVN for it's ability to easily branch and tags and hates git, except for git's ability to stash files (we still haven't moved from SVN to git [11]).
We had to leave it with an agreement to disagree on the details for the time being.
I have a feeling this is going to be a rough few months.