Commit comments... more is more

First off I must say that it has been a long time since I last posted - I have just been swamped with work on www.sanlamconnect.co.za.

The other day the issue of what is a good commit comment came up... and there are a number of approaches to this issue.

The big question is, Should it stand alone? IOW should it make sense by itself, or should it be taken in conjunction with diff. If you apply the former then the comment will repeat what the developer can ascertain from the diff and typically add more information. If you apply the latter then the developer is required to look at the diff to make sense of what is going on.

So the other day a fellow developer accused me of being too terse in my commit comment, and I was a little indignant at his criticism because I regarded my commit comments as being adequate. I felt why must I rehash what the user can see from the diff. The comment as far as I remember was "fixing bug"...

I reasoned, if the user has any sense they can look at the diff and they can figure out what the bug was and what I did to fix it.

But would it have killed me to just put down maybe another 10 words to indicate more details, like "fixed bug in execute target, the file extension was placed incorrectly". Is that too much to ask? Another 10 words that would have taken me another second to write that is going to save the next person at least the time it takes to open the diff, notwithstanding the time it's going to take them to make sense of it.

So, the conclusion is, write detailed commit comments. Definitely a paragraph is not necessary, but at least 10 words is a good start.

What about those developers that would have taken another minute to write those 10 words because they can't touch type... to them I say, tough, it's time you learnt.

The Double Importance of Unit Testing when more than one developer is involved

So now, you have a team of 10 developers working on a project, I know it's a lot, but some projects have that number...

And so a solution is required to a problem and both developers require a solution to the problem. For any number of reasons, both developers end up solving the problem, not in exactly the same way mind you.

The result is that you have two solutions to the same problem.

Now what? You have 2 options...

1. You could leave the code as it is. This is surprisingly enough, a viable option, since it's done often enough! But it's clearly not ideal as it leaves you with duplicate code and duplicate functionality. And if you consistently apply this approach then before long you'll have many many instances where this occurs. Maybe inside that duplicate code there is another facility that is required and thus duplicated. And if you consider that with 10 developers involved, you can have many many instances where developers are unfamiliar with what other developers have done, so you have exponentially more instances where functionality is duplicated. The more developers you have, the more duplication. Each time a developer is added, the potential for duplication is exponentially higher based on the number of connections. With 2 developers, only 1 piece of communication is requried per piece of work, with 3, 2 pieces etc... each time you add another developer, the probability of duplication is significantly increased. So you inevitable end up with multiple ways of solving the same problem with code that differs in its quality (different developers wrote each one)

2. You could however, refactor, such that both parties use the same functionality. A more appropriate solution; but easier said than done right? It sounds good in theory but it is not practiced. Why not? What if it doesn't work? What if there was one little consideration the one version used and depended on that the other version didn't have or do? Then the implementations are not exactly the same and thus cannot simply be amalgamated. How do I know the implementation that now depends on new code is going to work. Unless you have some facility to validate the changed process you don't really know. You could of course start up the application and exercise the functionality from both points of view to validate they both work. A clunky and time consuming process that will hold you back from making the change in the first place. That's where unit tests come in.

Unit tests let you remove duplication and refactor with confidence allowing you to improve quality, conciseness, readability and delivery times.

Open source software is self service software...

Jeff Attwood at Coding Horror makes an interesting point about open source software being "self service" software. That immediately struck a chord with me as a I saw his point straight away. 

But he didn't quite mean what I expected him to say. What he meant was that participating on software projects is a self service enterprise and thus leaders of open source projects need to make their projects as easy as possible to serve yourself on. There are particular issues that go with any kind of self service.

But the same rings true for _users_ of open source software, and those users are exactly that, self service users. With open source software the users help themselves, they fix their own bugs, get their own help and basically make their own way. They do not rely on a commercially driven entity in order gain value from the software they are using.

And I think companies thus need to take this into account when they're considering whether so use open source software. At cinema's in South Africa they have self service kiosks for buying movie tickets (I don't know if they have them elsewhere in the world) - and I really appreciate these. I do not have to wait in the queue, I also get to have my pick of where I want to sit. 

However, it is only particular people that are able to use these kiosks. They are not everyone's cup of tea. You need to be technologically savy, and need to be willing and able to back yourself. iow, to try new things and have the confidence and where withall to see it through.

But let me say that not all open source is as self service as "self service". Some open source initiatives are supported by commercial entities, and some open source applications look and behave like the closed source equivalents - OpenOffice and Apache Web server being two simple examples.

In general though, using open source is analogous to using the self service option, when presented, and seeing things in that light help to shed light on some the challenges and value adds of open source software.

Hibernate: What does update actually mean?

I work on a large enterprise application that uses hibernate for it's persistence.

I regularly in the code base find the equivalent of...

Person person = personDao.loadById(id);

person.setLastname("newlastname");

person.setFirstname("newFirstname");

personDao.update(person);

The update call in the above code snippet is unncessary and useless.

Even after being on the project for more than 2 years I still find developers not understanding the purpose of update and I think I understand why. The obvious thought process is, I've changed a stored object, I now need to call update to save the change; sounds reasonable enough? That however, is not the purpose of update.
This particular use of update belies a misunderstanding of the nature of hibernate, and that is that it is a "transparent" persistence framework. What that means is that persistence is made to be transparent to the developer. You should not have to worry about the persistence issues. You load objects from data store, you change the objects, and the changes are saved, without any explicit save instructions. Imagine for a minute, if you had to call update whenever you made a change the complexity upgrade would be significant as it would mean having to track all the objects you do in fact change.

So then, if update is not for that, then what is it for?

The update method could have been named "attachAndPersistChanges" - that is a more accurate name, but probably a little unwieldy. It could also be described by the sentence "update this object in the data store". In other words, update is designed for objects that are not currently persistent.

The update method makes objects that are not persistent, persistent and updates the persistent object with any changes present in the not persistent (detached) object. The object must not already be available in the level one (session bound) cache otherwise an exception will be thrown (for this situation, use merge).

It does nothing for objects that are already persistent. I do not know if there is a performance penalty with running update on an already persistent object, if there is, it's probably minimal. It does clutter up the code with needless lines however.

Then when should I use it?

My current project is a large enterprise system with java on the back end, .net on the front end and web services in between. The front end is fairly light and thus does not know about the data model. For that reason the front end does not operate directly on the data objects. If it did, there might be a case for using update. i.e. I've received an object from the front end with changes that were made by the user. and I need to persist those changes so I call update on the object. The pattern we follow however is the one outlined in the code snippet. Load an object from store, make changes to it based on data received from front end - changes are automatically persisted, no update call.

But there is one case that I've encountered where we do need to use update and it illustrates an important thing to remember about hibernate...

Consider the code... 

transactionController.startTransaction();

Person person personDao.loadPersonById(personId);

person.setLastName("newLastname");

person.setFirstName("newFirstname");

transactionController.endTransaction();

transactionController.startTransacton();

person.setLastName("secondLastname");

transactionController.endTransaction();

I know it's a silly piece of code and you wouldn't find it like this in reality, but it illustrates the case simply. It is setting up a case where there are two transactions and one transaction is interacting with an object which another transaction touched.

What will the value of person.getLastname on the database? Believe it or not, the value will be unaffected by the second setLastname operation. It will be "newLastname". Why, what's going on?

The thing to remember here is that as far as hibernate is concerned, from the perspective of the second transaction, that person object is transient. The session that it was connected to is now closed and any change that is made to it is made as if it's a normal pojo with no knowledge of the persistence store even though a new transaction has been started. That object does not know about the new transaction.

And that is where update comes in. An update needs to be called on the person object in the second transaction to persist the change to lastname. This as we explained earlier, will attach the object to the new session and apply any required updates. 

transactionController.startTransacton();

person.setLastName("secondLastname");

personDao.update(person);

transactionController.endTransaction();

On our project, that is the only legitimate context where update is appropriate and necessary.

When to inherit, when to compose?

I suppose if you're reading this you've read some of the material out there on when to inherit, when to compose. You've also probably familiar with the "is-a", "has-a" principles...

My brother Peter, who is a dotNet programmer commented on the issue and came up with a useful principle...

Don't do it.

Or more politely... When you think you need to inherit, you pobably don't.

That is probably a good best practice approach - I need to post about best practices and why I say it's a good best practice approach. The problem is that inheritance is abused and the naive rule of "is-a", "has-a" is responsible for to many situations where inheritance is not the answer - composition would have been more than adequate and without the constraints and complexity that inheritance introduces.

It's a little like threading. If you can do it in one thread, then do it in one thread - you don't want the concurrency problems.

Developers do what they see...

This is probably the first really big project that I have been on and I have learnt a lot. One of the key lessons I have learnt is that the average developer is a sheep. He (or she) will do what they see and very seldom will they go against that.

In fact, even if it's wrong, they will do what they see.

It is has been quite remarkable.

So changing habits is not easy, especially if you're on a complex enterprise project which has been going for 18 months with a very fast development cycle. There are _many_ different ways of doing almost anything in the code so the poor developer has no idea what approach to follow.

It also means that changing habits is not easy, is has thus been quite encouraging to see some habits changing - I introduced commons collections into the code base, used it in my code, and showed a few dev's what it offers - and I'm now finding it is used more and more.

The other good thing about the way developers do what they see is that if what they see is good and right, those habits are persisted. I guess the opposite also applies, if what they see is low quality, they will extend that low quality.

Hibernate best practices...

I have been on a very large enterprise project using hibernate for the persistence. Our data model is upwards of 200 objects. It is a clustered web application and thus requires high performance so we've implemented caching and have all our objects and relationship set to lazy load. The typical database interaction is lots of reads and few rights. The data of one user does not affect the data of another.

Here are some of my best practices which I would have loved to have had in place from the beginning. They would have saved a lot of time and reduced bugs...

Use id based Equality
This is contrary to a lot of the typical recommendations but I don't see a problem with it. It means that you have a working equality method without any further effort. Furthermore, you have a guaranteed equality, the same equality that the database uses. There are some issues with this when it comes to one-to-one relationships, but those can be resolved by adjusting the equals method. The other issue is that there are instances where there is no business based unique key.

Don't use auto generated Keys
So in your model classes set the id. This will mean you need another field as a persistence marker. The fundamental reason for this, coupled with the fact that you're using the id for equals, is that you have immediate equality, both prior to persisting the object and in your tests. By using auto generated keys you you have to wait until the object is persisted before its equals method works. You cannot thus use it in a set prior to persistence, and when you run tests you have to manually set the id every time you create the object.

Use instrumentation
Hibernate has an issue (a big issue IMHO) that if you execute the following code (Person is set to be Lazy loaded).

          Person p = session.load(Person.class, id);
      ContactDetails cd = p.getContactDetails();

And on the database that particular object (contact details) is actually a subclass of ContactDetails called AddressDetails, the real type of the variable cd will be ContactDetails and _not_ Address Details as you would have expected. In order to get the real type you have to do first ask hibernate for the real type of the object and then call load again with that real type. So in order for the variable cd to be the correct type you would require the following code:

          Person p = session.load(Person.class, id);

      ContactDetails cd = p.getContactDetails();
      cd = (ContactDetails)load(id, Hibernate.getClass(cd));

An extra line is required.

Instrumentation however injects code into the compiled class of ContactDetails and intercepts the call to getContactDetails to add the bit to load the real type. It's a small little ant job that runs within the context of our IDE and sorts it all out.

Use Field access over method access
Unfortunately, I only discovered that hibernate had the ability to directly act on the fields rather than go via methods too late to make use of this ability. It would have enabled me to setup some nice validation and/or side effects on the setting of a value in a hibernate managed object. I'm sorry I didn't do it earlier. This is a practice which I have thus not been able to test, it does sound like a good idea but maybe there are other issues I do not know about.

Think hard about your Cascades
Hibernate is not good at saving a whole object tree in one go. The problem is that as soon as any sql is executed on the database, the sql is validated against the state of the database. So any constraint violations are immediately raised. If you think hard on the cascades the ability so save whole trees is significantly improved though it is not totally enabled. One obvious practice is that if it has a not-null constraint the cascade must be set. If this is not set unless you save the foreign object before the local one you will get a constraint violation. The objects go together (thus the not-null) so the cascade should be on.

The above recommendations are fairly generic to any project that uses hibernate in the real world. There's a few that I've learnt on this project that are specific to a high performance and high concurrency web application...

Prefer trawling the object model over running queries
The kind of application we have is one where a user would login and probably send on average 10 - 20 minutes on the web site. Thus we would typically have a lot of the objects they require in cache already. Thus when you need to find some data for the person concerned it is a lot better to simply trawl the objects rather than use a hibernate query. The reason for this is twofold..

• Querying in hibernate always causes a flush. Data is therefore written to the database at a time when it shouldn't necessarily be. Even though you know the query does not touch any of the pending writes, the flush will still happen.
• A data connection is made. On our application we gained significant performance concurrency improvements by removing queries - the objects were already in the cache in any case. Where we had deadlocks we simply removed the query from the equation (changed to trawling the model) and the deadlocks were significantly reduced.

Make everying lazy
On our application where isn't a single relationship which is not lazy. We can't think of a case where having lazy off would be useful to us. Yes, I understand that it means the first read will be slow, but after that it'll be in the cache so it will be fast. If lazy is off even if the object is in the cache it will still load that object from the cache

In other words, if you have an object A which has a reference to object B and the reference is specified as non lazy. When you load A it will _always_ load B even if you don't go near B. Even if B is in the cache it will still load B from the cache. On our application we have a lot of static data. This static data was initially set to be lazy disabled (prefetching). Then we would preload all this static data. We found that the preload was very slow - even though it was coming from the cache - because it would have to load the root object plus all the non lazy relationship stemming from that object.

What Every Development Shop cannot do without!

For those of you who have not heard of hudson, where have you been. It is the new kid on the block in the Continuous Integration space, and it (IMO) stands head and shoulders above the competition.

It has added an incalculable amount of value to our development environment, enough that we depend on it as much as we depend on our IDE. Furthermore, it has single handedly raised the quality of the code such that I am so disappointed we did not have it in since the beginning of our project.

Why do I say all of that?

To my mind, the killer feature that hudson has is that it supports plugins. This is a feature what sets it apart.

And the killer plugin, which has contributed to the code quality is a fairly simple plugin which allows the results of various code quality metrics to be summarised and tracked in hudson.

We now have continuous metrics for the following...

• Checkstyle - this is a tool which examines source code against a number of rules. It checks for instance, the formatting is correct, cyclomatic complexity and npath as well as simple class/method lenghts. It flags as a warning if the rule goes outside of the allowable scope.
• FindBugs - this is a class level code checking tool. It can find things like equals being executed against different types
• Cpd (Copy-Paste) - a cool plugin that checks for where code is the same and thus probably was copied and pasted.
• Pmd - another class level checker

Now I could have run these code checking tools without hudson. Hudson however, allows me to track changes over time and to know when new violations have been added etc. And the running of these tools has already avoided many potential bugs.

So if you haven't checked out Hudson and its Plugins, it's never too late. The longer you leave it the worse your code is going to get.

If you want to know more, then let me know.

JDK 1.4 is being retired...

RIP ... JDK 1.4

If you haven't heard, JDK 1.4 is to officially entier End of Service Life. Over at dzone, Alex Miller comments about and what I find most interesting is that the fact that it's going to be retired is not going to make much difference to the software community. They are going to happily continue using it. If they're still using 1.4 now, I doubt whether it reaching end of Service is going to push them over the edge.

For one thing, software shops which use Websphere are typically still on 1.4 - this is because the big bear IBM is controlling the VM version of those shops and it's going to be a long time before WPS on Java 5 sees the light of day - so though WebSphere 6.1 supports Java 5, Websphere Process Server 6.0 does not. So spare a though for people like us who are still stuck on Java 1.4 and the news that it is going to be retired will be a non even in our lives.

Testng is Cool but flawed

A few months ago we took a closer look at our testing strategy. We assessed testng and noticed it had a number of features we thought we'd find useful. e.g. data driven testing, ability to run only the tests the failed, configurable testing in xml based on annotations and/or xdoclet tags.

Based on the features and a little prototyping we decided to use testng as a standard test framework. We setup the tests to run on our CI server (hudson) and we were good to go...

And things moved a long smoothly, we used a lot of the extra features of testng and also found that having an xml file that indicated what was a test was quite useful.

But then issues started to surface...

1. Testng cannot run each test on a different VM. The makers of testng do not give you the option to run your tests in a totally new VM so you can potentially run into issues when doing integration like tests requiring the hibernate session factory and database. In junit I have the option of runnning each test case in a totally new VM.

2. Testng doesn't really work with 1.4 xdoclet annotations. Unfortunately we are still on 1.4 and thus had to use the xdoclet annotation mechanism for tagging a test. This was problematic almost all the time. I wouldn't be surprised if the issues we had with testng would be a lot less if we were able to use class annotations.

3. Testng tries to skip tests when the setup fails, i.e. it doesn't fail fase.This was the major issue. The problem was that when a test's setup does fail and thus the test is skipped, testng struggles to identify how many tests were supposed to run. Thus in our CI server we found that the number of tests kept on changing. Permanently. Thus it because very difficult to get predictable results and a good idea of what is going on in the tests.

4. The Eclipse plugin for testng is not as good as junit's. The developers in my shop were unhappy partly because of this. When I announced that we were moving back to junit they were happy. Some of them had continued to subclass TestCase because they felt the assertions provided were a lot richer when this is done (and they are). With Testng, because it is not required that you subclass when creating a test, you generally use the java keyword "assert".

So now we are moving off testng back onto the old faithful, junit. There are however tests which are built with the data provider and/or parameter subsystems which testng offers and these will stay testng tests. We have no plans to move these off testng. And if our developers really need the extra functionality provided by testng, they can use it, within reason - our testng CI run has not totally gone away.

The approach now is, if the tests only requires the basic features which junit3 has, then use junit. If you need testng functionality, get permission for it before you use it.

The Developments at SpringSource

The blogosphere has been abuzz with the news that SpringSource is going to be charging for support, and for maintenance releases. SpringSource is becoming a more traditional, more mainstream enterprise. Rod Johnson, one of the progenitor's of Spring has seen Spring become like hibernate in that though it is not a standard it might as well be.

Now they SpringSource, the entity which focuses on Spring development are looking at ways to extract value out of that popularity, but also, I think, take Spring to the next level.

And the developments are not unusual, or should not have been totally unexpected. How many other open source applications have gone the similar route. Redhat linux, JBoss, to a lesser extent hibernate... What other way do you monetize and open source application apart from selling support etc...

From all that I've read, the change to the release system is that "official" releases will only be available to non paying customers for three months after an official release. So 2.6.2 will only be made freely available if it is released in the 3 month period after version 2.6.

The bug fixes that go into any release will still be put into the source code and the licensing of the source code has not changed. So, then, why can't some developer checkout the source code, build it, and then make that freely available for all the other developers. I guess the only draw back is that it's not an "official" release. It's like a new car is only a new car if you drive it off the show room floor.

Personally, I don't have a major issue with where Spring is going. I think it's an acceptable way to build a viable business model out of their operation. Though it may serve to alienate purists, it might also cause people on the fringes to be brought in.

Programming is Hard

I can remember my brother once remarking that "Programming is Hard", and I'm not sure he realised the importance and relevance of what he was saying.

On our large project - about 20 Java devs - I monitor the checkins and do some basic QA on them. The developers in this environment, who are apparently fairly good, still make school boy errors.

Fortunately we have hudson setup to do all the checks possible - checkstyle, pmd, findbugs and cpd - and so I can find most of the basic mistakes, like copy and pasted code for instance. But then often I find more complex issues, just bad design for example, and I've often thought about this issue.

I also maintain that computer programming is engineering. There is debate about this, but I think there are enough similiarities with "classic" engineering for it to be labelled engineering (I think it helps quality if you call it that).

Why then does computer programming not have the same aura that classic engineering has, and also the same obsession about quality...

Two reasons...

The first one is safety. When building a computer programming you're not building a bridge and thus you do not have to worry about people dieing if it fails. A "bug" in a bridge could prove very costly, not only in terms of money, but more importantly in terms of people's lives. There have been many instances when bridges have failed and their failure is put down the human error. Something that would have been called a "bug" in a computer program.

The second one is that the programming world is abstract. The result of this is that the world is a lot less limited. You're not limited by something physical when building a computer program. The result of this is that the barrier to entry is a lot lower, it's much easier to get into and also that it makes programming "easier" than other engineering. Easier in quotes because I think the problem we have with quality when building computing systems is because people think it is easier.

Programming with Quality is IMHO every bit as hard as building a bridge successfully or desigining and building an alarm system. If we want to produce quality we need to put as much effort into it as an qualified and certified engineer would put into building a bridge. We cannot think that our often mediocre, slap-dash efforts measure up.

Problem is, if you're reading this, you probably do see yourself as on a par with an Engineer.

Spring and JEE

A helpful article I found on the value add of Spring...

Why do J2EE applications need Spring? The core of Spring explained...

It says most of the things I would say and doesn't "get ahead of itself".

I would strongly agree with points 1 and 2. Point 3, though it is a feature, is not something that I think, is used as much as people think. The reason I say that is how often do you actually have two implementations of the same interface, and switch between them? In our application we have loads of DAO objects, loads of service interfaces and yet we without fail have a one to one mapping between the interface and its implementation. The interfaces, IMO are useless fuzz. What I would prefer in fact would be for them all to be accessed directly (no interface) and then if and interface is required, then I can quickly and easily with the tools we have today, extract an interface and it's now abstracted. I only have the fuzz where it is absolutely necessary.

Though I must admit that we do use this feature and it does add value. I just don't think it is is the killer feature of the Spring Framework.

One of these days I think I should write a "why I like Spring Article..."

JEE take up could be better...

I work on a fairly large Jave Enterprise Edition application, deployed onto Websphere. The company is a large corporate and the application is a publicly available web site, involving a .net front end, java back end and a database.

The application runs on websphere and is deployed by a team of people who manage the server environment. There are lots of similar applications which they manage.

We have our own "dev" application servers - these are little more than developer spec boxes running linux. They are used for basic testing and to validate the application before it gets promoted. And, because we manage these environments, have carte blanche access.

The amazing thing is that the application always works just fine in this environment, but when promoted, it suddenly no longer works. And it is no coincidence that the application does not work in environments we have no access to.

Now, when the application does not work, we get blamed. The onus always falls on us, the developers to fix the problem even though we have little to no access to their environments the application is deployed on. The fact that the applications work in the environments we do manage means that the problem lies in the fact that we cannot configure these environments ourselves.

What does this cause?

It causes us developers to build our applications in such a way as they have as little depedency as possible on the environment they're running in. Thus we use as little of the JEE spec as possible because those are the bits that need to be configured on the running application server.

I am convinced that if we had better access to the test servers, we would be more inclined to use the rich functionality provided by the JEE specification.

Setting Coding Practices

Let's face it, anyone that has worked in development for any length of time will know that there is always change, and the "quality" of the code is a key area where this change is seen.

When you start on a project, you're not too familiar with the technology, you're not familiar with the problem space and you have a lot to learn. So you code in a certain way, and you use particular techniques to achieve your goals in code. Over time you learn better techniques, but the problem is you don't retrofit your old code to use the new techniques.

Recently I had to work on code that was written right at the beginning of my current project. Wow! Was it legacy.

This change and development is a good thing in the sense that it is a sign that things are improving, however it presents a challenge in that developers are very good copy cats. They will typically do things the way they see them done (even if that way is wrong) and don't as a rule go against that. In a sense that is also a good thing because we don't want a group of cowboys all doing their own thing.

The challenge is how do we accept this change and improvement, allow for it and even encourage it, and at the same time communicate the best practice approach to doing things. Furthermore, at some point we need to limit improvements (there's always trade offs), because any change that is made, needs to be tested. In our code base we still have all the various ways that things have been done from the beginning until now. When you have a project that has been going for two years, this is inevitable.

The Continuous Integration Game!

If you use hudson, then you really should take a look at the CI game plugin. It's actually hilarious!

It stems from an idea originally (I think) by Daren Cummins described here.

The way the game works in hudson is that you get points according to your build activity. Plus points if you do something positive, like check in on a successful builds and minus points if you do something negative, like checkin to break a build.

Other examples of ways to score (or not) are if you change test failure/success, add or remove code checking violations (checkstyle, pmd etc).

It's been a great source of humour and a little bit or rivalry amongst our devs. Obviously you can't read too much into it, because it's easy to game, but it has added some value.

The plugin home page is here.

Firefox 3 Rocks!

If you haven't downloading and are using firefox 3 you are missing out BIG time.

It is unbelievably fast - the whole browsing experience is super fast, and the javascript engine is about about 5 - 10 X faster. Even faster than Opera's.

The memory leak issues have also been fixed. With 17 tabs open, Task manager reports 237 Megs Used. I don't think I could have functioned with 17 tabs with firefox 2.

An Improved Skill level indicator

With reference to my previous post on skill level indicators, I have come up with a new version... which I think is better...

It works in terms of "problems", since when employers ask for skill what they're really asking is, how useful are you to me with this technology?

1. Problems, what problems? (iow, I haven't had any problems because I haven't used it enough)
2. Somebody else solves my problems
3. I solve my problems but have to sometime ask someone else to help
4. I solve other people's problems, sometimes someone comes with a new problem
5. I solve other people's problems and there is never a problem that I don't know how to solve immediately

A working definition of the skills scale

Have you ever done a "skills matrix"?

It entails writing down all your "skills", i.e. API's, technologies, languages, whatever might be relevant and putting a number from 1 to 5 next to them, and maybe a time length in years of your experience on the topic.

It is to get a prospective employer and idea of how valuable you could be.

The problem however, has always been, what does "1 to 5" mean. What is the reference point? You might regard yourself as an expert in Swing because you've built some funky table structure where you can edit the cells and render the cells according to their contents, you might put down 4, maybe if you're brave 5. But then what if one of the interviewers is a contributer to the swing API? In comparison to them you're only a 2...

I have thus always been skeptical of someone who puts down 4's and 5's. I'm going to make sure they're backing that up with something real. In fact, I'd love to interview someone who puts 4 or 5 for hibernate, it could be fun to show up their lack of skills.

Now what about if we come up with a more absolute definition of what the numbers 1 to 5 actually mean. So they're more than just mere numbers. Here is my stab at it...

1 – only read about and played a little with. I have indirect knowledge or experience (was on a project where they used it but I was not involved).

2 – shallow understanding, worked with by duplicating other work.

3 – becoming familiar, able to work with and beginning to understand what is going on. Extensive experience programming with.

4 – Understand the inner workings of the technology. Thus can solve problems where they appear. Extensive deep experience in working with.

5 – Extensive understanding of the whole technology. Totally knowledgeable and experienced with everything about the technology. Possibly spotted and fixed bugs in the API if applicable.

Is this helpful, or relevant? If you apply this grading system to some of the technologies you know would that change your score? I don't think any ranking method would be perfect, maybe people can comment on how it can be improved. Maybe there's a totally different solution.

Technorati Tags: employment, grading

Open Source and the cost you don't see

The story on the register about Sun moving towards charging customers for certain enterprise features included the following quote from research house "The Standish Group"

"Open Source software is raising havoc throughout the software market. It is the ultimate in disruptive technology, and while it is only 6 per cent of estimated trillion dollars IT budgeted annually, it represents a real loss of $60bn in annual revenues to software companies."®

The full report is available here.

This supports something I've been saying for a long time and that is that companies who use open source should not, in fact, see it as a free ride. They are getting a significant amount of value out of the open source software they use, $60bn worth. Personally, I think that is under what the real value is - think how many versions of apache are running out there (compare the price of an IIS license and its features).

I'm not demanding however, that companies start paying for the open source software they use - that would be like the good guy (open source) becoming the bad guy, just not on the outside. The open source community is not in it for the money.

Companies should give back to the open source community, they could donate cash if they so wish, but a better idea is for them to let their developers work on open source projects, on company time. Those very companies are using the open source software and will thus benefit from the work their developers do on it, because the developers will work on features that they require, and on bugs they have found.

The company at large will not see money going out to open source projects, even though value is being added to the open source project. There will thus be no nominal effect on the "bottom line". They will have a happier developer (what developer does not want to work on an open source project), they will gain a lot more street cred and the open source software they use, will be improved.

It's a win-win situation.