« The metric system: a case study in technical standard setting | Main | 6 Kg of Vegemite »

December 06, 2004

The BileBlog, without the Bile

The last week I have been corresponding with Hani, the writer of the BileBlog, without realising it was him. The BileBlogger seemed to know a lot of people that I know so I thought:

• a) he must be based in Europe, or travel to Europe a lot and
• b) would in turn be known to my European contacts.

Though the name Hani is not that rare, I made the connection, checked with some colleagues and then realised my email correspondent Hani Suleiman and the BileBlogger were one and the same. His emails were courteous and well thought out.

I read the Bile Blog and find myself in broad agreement with much that is said. So, after discussing it with him first, I am blogging about the last three issues in his BileBlog, without the bile. The purpose is to create some discussion and amplify Hani's points, which I believe have some merit.

So I will do away with the profanity and abuse, and try to distil the issues into a technical discussion that is hopefully more accessible. Where relevant, I'll also mention some positives and helpful suggestions for some of the issues, both of which are considered 'offtopic' on the bileblog.

Issue 1: Developers Vs English - or why documentation is important

I agree with this blog. Code is about precise communication with a machine. Documentation should be the same, but with your users. Who are your users?

For a business application, like I get paid to write by day, it is the business employees. They are generally communicated with via training.

For a web app to be used by the public, your communication needs to be done via Usability. We have a Usability Engineer who conducts usability research. It is so they can use the web app without needing documentation. The interface must be extremely intuitive.

With a Java library, like for example ehcache, the users are other developers. Communication with them can happen in the following ways:

• The code itself, which should be eminently readable. A tool like checkstyle can help with that
• The sample code, or tests, if it comes with these. These demonstrate how to use it.
• The JavaDoc. The quick view CTRL-Q in IntelliJ will explain just what a method is about.
• The documentation.

Hani's point is that the usefulness of a library is dependent on your ability to use it. Your ability to use it is dependent on how well the author communicated with the users.

For these reasons, I take documentation for the open source projects I maintain very seriously. I am still not happy with the quality, so I keep going back and adding stuff.

I also agree with Hani about precision in naming. I find that if something is named counter intuitively, I cannot understand it.

Issue 2: Tending to Mediocrity - or why open source quality is generally low

The thesis here is that a low quality open source project comes about as follows:

1. There is a low barrier to entry to create the project in the first place
2. Democratic management of the project leads to a jumble of voices
3. Patches and suggestions get accepted far too readily
4. The result is a low quality mish mash

I think a successful project needs a benevolent dictator. The project should not try to do much and not try to please everyone. I do this with mine. The successful projects out there, like Linux, do this.

The other reason why open source projects can lack quality is the developers are motivated by personal interests, which may not coincide with making a project better. I asked Jon Tirsen why he did Damage Control, rather than just making Cruise Control better. He answered that he was not paid for it, so why would he want to work on someone else's project when he could work on his own? When the famous urge to 'scratch an itch' that fuels open source manifests itself in useless duplication it is symptomatic of a ego-driven 'NIH (not invented here)' disorder. (Though I am using Jon to illustrate my point, in the case of Damage Control I know that it was also done to do something in Ruby and create a build tool that did more than Java).

Issue 3: JUnit bible thumpers - or why the dogma?

Hani's point is that before JUnit and XP, quite a few projects got done. Successfully. An industry existed that knew how to build quality software. They used testers, or QA Engineers.

Look at my open source projects, and you will see that each comes with a comprehensive JUnit test suite. I think it is a useful approach.

My problem is with the JUnit zealots out there. I think that often they don't understand testing at all. We have a tester at work who has just gone through training. He has a wall chart which defines about 100 different types of test. A Unit, or Component test is just one of these. But most JUnit developers use the term loosely; they really mean a Unit Isolation test.

Any other type of test, such as a Unit Integration Test, is anathema to them. Because it is hard to do Unit Isolation tests in J2EE, they say, let's get rid of J2EE. Don't use any frameworks that get in the way. To me, that is throwing the baby, and the bathtub, out with the water.

I was trained in statistics and auditing. Testers are after a similar thing to auditors or statisticians. Trying to prove assertions based on a sample size less than 100% of the population of phenomena. In the case of testing, it is the lack of flaws, of many different types, in the system. The tester today talked about doing an Installation test. That is where we do a dry run release, including all the database scripts, to see if there are any release problems. Makes sense, right? But it is not a JUnit Isolation test, so to JUnit zealots it is meaningless. I am quite frustrated with the dogmatic responses I get to this type of argument. I am currently refusing to debate testing, including Unit testing, unless those involved have read a testing textbook. I have a great one on hand, Lessons Learned in Software Testing: A Context Driven Approach by Brett Pettichord. Incidentally, I have met Brett. Brett's book, and Brett, are a welcome breath of fresh air.

We should be trying to produce quality software for a reasonable cost. So that the client gets maximum Net Present Value. (Yes, NPV is a Business Finance term, but it models very well how business's think and what they value).

So, in testing, Unit Isolation White Box testing, is one of about 100 tools in our arsenal. Treating it as the only tool is wrong. As the saying goes: "When all you have is a hammer, every problem looks like a nail".

A comment in defense of JUnit. A suite of automatic regression tests, with high coverage, support safe refactoring of the code base. This benefit is less about testing in the sense of producing a high quality output, than in enabling developers to make changes without fear. In this sense tools like JUnit are very valuable.

I am also likewise concerned about the state of JUnit. JUnit has been inactive for 2 years. A few months ago I got together a coalition of about ten developers willing to work through the reported bugs and patch list to bring JUnit up to date. I volunteered to take over maintenance of the project, along with the team. Eric Gamma, Kent Beck and Eric Meade considered this, but in the end decided they could continue to commit energy to the project. A look at JUnit CVS shows that they have been busy working on release 3.8.2 for the last few months. Great news!

Update:Some people asked where the estimate of 100 testing types came from. Have a look at these two testing glossaries. Looking through one of them I counted 68 different types of testing, one of which was Unit Isolation Testing.

• http://www.testingstandards.co.uk/living_glossary.htm
• http://www.aptest.com/glossary.html

Thanks to Jules Barnes, our lead tester, for these links.

In Summary

1. Documentation is vital and should be treated seriously
2. Most Open Source projects are of low quality.
3. JUnit is a tool for Unit Isolation White Box testing, one of many testing techniques which can all produce quality software. It does have a special role supporting refactoring though.

Posted by gluck at December 6, 2004 08:33 PM

Comments

Posted by: Cedric at December 7, 2004 10:27 AM

Posted by: jfc at December 8, 2004 01:01 AM

Posted by: Robert McIntosh at December 8, 2004 02:08 AM