As cameron has indicated Australia has joined the ranks of countries explicitly asking their government departments to consider open source in their "purchasing" decisions.
I have been gathering material for a course on this topic; as the kind of selection criteria considered when evaluating open source projects reflects a different set of risks then a procurement department is perhaps used to.
PopularityA couple of months ago I went over the amount of email some of our open source leaders put out. While this does indicate amazing dedication to our our OSGeo community; it perhaps not the best metric of productivity.
I also have been following the various web processing service implementation; and the amount of email is a real indication of popularity and activity.
(The above graph is just a reflection of activity. Both deegree and GeoServer are blips as not all of the activity reflects "wps" work. Some of the other projects just seem to have one email list for both development and user questions. Projects that successfully turn users into customers will also reduce email volume handling support offline. Figuring out how to deal with all this is one reason why you should take the course!)
ProductivityThe other aspect to this is using the open nature of open source to evaluate those offering support level agreements. This is a case were we can really look into the productivity of an organisation and look into their experience and track record.
(The above diagram is for the GeoTools project, issues resolved in the last quarter. GeoTools developers sell very few direct support contracts - instead focusing on the support of downstream products. Making for a nice graph I can safely publish without ruffling any feathers)
This level of visibility, combined with support being an area heated competition, provides a procurement department with more leverage (and responsibility) than simply talking to a sales representative.
OpenMeasuring how open a project is actually one of the top mandates of the OSGeo foundation incubation process.
Any project emerging from the incubation process has:
- A public email list (although sometimes that is not the only email list!) - allowing you to produce graphs such as the above
- A public issue tracker - allowing you to produce graphs such as the above
- Open development procedures; allowing you determine how much effort is requried to participate and perform common tasks (such as a release)
- An open review of the legal status of the codebase - giving you some confidence the code won't be pulled over a legal reason leaving you high and dry
There is of course more to measure: recent requests to summarise quality assurance procedures for example. In my experience the "open development procedures" ends up providing a great view on the QA controls for a project. Putting that together into a table for review and comparison is just a matter of research.