And it is a goal that we had at the beginning of our project.
It didn't happen, or certainly not as I would have liked.
So then, what did I learn from our so called failure, what are the mandatory requirements for making unit testing a sucess...
- Infrastructure to automatically run unit tests. In order to make unit testing work, you must have some facility to run the unit tests regularly. Preferably these should run fast and should be run on every checkin. It is not enough to rely on the developer to run the unit tests before they checkin their changes.
- A univeral commitment to quality. Your whole development team needs to be fully and categorically commited to quality, and a working unit testing system is a necessary requirement to get there. For a unit testing system to work it means that if tests fail, developers need to care - as soon as one unit test fails, your system is immediately less effective. The unit test frameworks do not work when even one test is failing. You can't have just one developer, say the lead developer, telling developers to fix their unit tests.
- Mangement buy in. The problem we had with unit tests and getting unit tests fixed is that developers always made some excuse about how they could fix it now, they just had this bug to fix or this use case to finish, so management need to realise that a reason for the use case being delayed is because I had unit tests to fix is as good a reason as, there was a power failure and I couldn't use my PC.
The benefits of unit tests are proven, but I can't prove those benefits to you, here. You have to do it, 100%, pay the cost - and it's true, there is a cost, but the pay off from paying that price worth it, you'll probably gain 3 times as much you put in. But don't bother, unless you've ticked the 3 boxes outlined above.
12 comments:
4. You have to do it from the start. I have never seen unit test work if they were added on when the system is half way implemented and already in trouble. Unit Tests will keep you out of trouble, but it will not get you out off trouble.
I would agree, but is it _mandatory_ that you unit test from the beginning? Is it not a nice to have, rather than a necessary requirement?
If you start later you face some immediate problems:
1) code coverage will be bad.
2) tests written after the fact tend to be rubbish (pointless simple tests)
3) you will need to start a cultural change amongst the devs and most devs like to do things the same way they always did. If you were testing from the start no-one will want to be the first one not to write a test. If there are currently not tests it is easy to quickly add something without a test.
agreed, you will have those problems.
Mmmmm...
Expected more of a response...
Are you looking for a rant?
When I see sense I cannot help but agree.
If you don't start them initially you will have those problems, but it is possible to start them late and they'll be less valuable, as you pointed out.
If you don't have a commitment to quality, management buy in and an infrastructure to run them don't even bother starting, even if it is at the beginning.
I've got a response for Dirk :)
You can do it either in the beginning, during, or even at the end.
You do need the commitment for quality (but you already have this if you've decided to do unit testing), and some buy in from management, but it's not a must.
Example: Your ending the release, the product is almost out. You don't dare touch the code. Then QA finds a bug. You build a test for that bug only.
Example 2: Your in the middle of the project. There's a new component being developed - start writing unit tests now.
Anti-Example 3: Your in the beginning of the project. It's the time to learn new stuff. You invest time and learn about unit testing. You decide you want it. You write unit tests, but then there's a big change in requirements. Deadline has not changed. So you put time first, and not write unit test.
Obviously, case 3 breaks the commitment for quality thing, but it happens a lot, you need to compromise.
My take - Do what you can, use the best tools for the job, automate everything, and stick to it.
Gil Zilberfeld
Typemock
What you say does sound right, and I used to say the same, but in my experience it does not work that way. It may work if all of the devs are into TDD and have done it before. But as much as I would like to agree I can not. Historically starting late just never worked for me. The problem might be me.
I strongly agree with example 1. Test per bug is the best and most useful tests you can have.
Unit tests are awesome! The confidence you have to commit code after you just run the tests that gives 100% useful coverage is great. 10% coverage just does not give me that same confidence. If the coverage is only 10% I will make doubly sure and recheck my code before I commit.
In the end I think you guys may be right. It is probably better to try than to not try. One test is better than no tests.
Gil...
you said
You do need the commitment for quality (but you already have this if you've decided to do unit testing), and some buy in from management, but it's not a must.
So how does the scenario go down when the management wants faster development because they want more features but development says no, we need to do unit tests...
I suppose in cases where the time frames set by management _do_ allow for unit testing, or development quotes times, factoring in unit testing that management accept, in those cases you don't have management's direct buy in, but you do have their tacit buy in :)
I also understand that it is a myth that unit testing requires more time. I think it _is_ true that unit testing takes more time _at first_, but once you have a large suite of unit tests and good coverage unit testing will start to speed up development because of improving code quality and the ability to refactor with confidence (less time spent debugging). A side effect of unit testing will also be a strongly decoupled system, which will further accelerate the progress rate.
I think it much depends on what we as the development team decides is acceptable practice or not. If we offer management the opportunity to say no to unit testing by singling it out, it is the same as offering them an opportunity to say no to doing design by singling it out. And we do not do that. (Ok, not in most cases! We do sometimes offer the VW Polo instead of the bicycle or the SUV. We don't ever get to build the Bentley...) What I am trying to say is that if it NOT NEGOTIABLE in our own minds, we will not make it negotiable to management. We are all good at compromising the best solution to the most pragmatic, barely enough solution, but one's professionalism stops you from going beyond that.
As long as unit testing is not something a development team sees as an essential part of being professional, that long it will remain an optional that (short-sighted) management will cut out.
In my experience the biggest obstacle is not so much management - they will ALWAYS try and squeeze blood out of a stone, even if it is out of ignorance - but because developers don't believe in the value of unit testing. Through a lack of evidence, mostly. You have to have done it to know how valuable it is. The top guys are always easy to convince - I do not know a single good developer who has tried it and not stuck with it - it is the less experienced developers that needs encouragement and convincing.
IMHO it is not up to management to say whether you do unit tests or not. None of their business, thank you very much! Like it has NOTHING to do with management whether you do refactoring or use patterns or frameworks or what your error handling strategy looks like. There is a boundary. Development commits to delivering a robust, maintainable, error-free solution according to an agreed set of specs, how they do that has NOTHING to do with management. And as long as you consider and include unit testing as an essential part of that job, you are OK.
If your management does not support that, wellllll.... Clients understand if you explain this to them, so if your management doesn't there is a real problem...
All of this coming from somebody who is not currently writing code. But who has fought valiantly on a number of projects to get unit testing introduced.
Because it makes a HUGE difference. Measurable difference.
Now don't get me started on pair programming... ;-)
@Antoine
The reasons why management got involved in the unit testing debate in the past has always been the same: A dev got them involved.
1)The slacker dev who is just not doing his job blames unit testing for missing a deadline.
2)A dev who is against unit testing says something like: "It is all finished and working perfectly, but I can't put it in the next build or give it to you because i have to write these stupid tests."
Management does not know or care how we do things just that it gets done, but the above exposes unit testing and makes it appear like a problem to management. You should always deal with this problem very quickly and decisively. Managers like to remove obstacles so do not allow anyone to make UT appear as one.
Management tend to love UT, but you have to win them over with results first.
I think that unit testing does need management support, for the same reasons as Dirk has said: the management must be in agreement or when a dev gives them the "I would have been finished by now but I have to write tests" excuse, they will be drawn into the debate. Unless they are convinced tests are fundamental they will wonder why the developer is being forced to write them and "waste his time". I've seen this attitude to tests many, many times - until the results prove the practice you're going to be fighting for the time to get up to speed with unit tests.
Meszaros' book has a graph that shows that unit testing does require more effort up front, but properly managed and refactored it decreases effort in the long run as your code base is much more maintainable. I would say his book is required reading if you want to do automated testing right, or implement it practically.
Post a Comment