Using processes means that we can gather information (metrics) to help us improve and justify our initiatives. Metrics is the one of the many impressive terms used by consultants (like me) to make something simple sound clever, mysterious and worthy of large fees. It is true that implementing sophisticated IT dashboards requires a lot of knowledge and experience, however for small development teams the simpler the information presented the more useful it is. Metrics are a set of consistently presented numbers that tell use on a regular basis how well we are steering the ship. i.e. Are things getting better or worse? Are we delivering what we committed to? The Pareto1 principle tells us that 80% of our problems come from 20% of the system we are working on, with metrics we can find out which 20% so that we can do something about it
Metrics are used to for two purposes
- Team improvement initiatives:
- Communication with the customer
So what are useful some metrics and how can they be used? Here are some example suggestions:
- Rework rate and costs — by documenting each issue via a ticket system (more on that later) we can get some idea of how much we are spending on rework. In particular we can capture information to tell us:
- Where we are finding issues (in user acceptance testing, regression testing or production). This may tell us that specific parts of our processes need attention (e.g. UAT or Unit Testing)
- Which sub-systems require most rework. This might imply that some parts of the system are fragile and need to be re-architected.
- Classification of root causes (e.g. Poor requirements/user stories, poor unit testing, etc).
- Process instrumentation. By looking at the amount of time spent in each process we can find efficiencies and cost savings. These means that as each step in our process is complete we need to record dates and time. Again a ticket system, with status tracking, can help to automate this.
- Trend analysis, which allows us to spot potential problems before they become too large and test our assumptions about where we should deploy our effort. It is possible to trend all sorts of data, provided that the information is initially captured in our system (e.g. Severity, cost to fix, time to fix, business feature impacted, time to test etc.). Some useful trends are production rework rate (number of bugs reported in production divided by the number of changes deployed in the last release), testing effectiveness (simple of count of number of bugs discovered in Unit Test, UAT, Regression and Production over time), ratio of new features to bug fixes implemented and so on.
Other tools, as well as using a ticket system:
- Introduce a time sheet system: Complicated if done from scratch but you may access to an existing system you can use. However you require access and tools to analyse the data which may be difficult
- Ask people to fill on project work sheets i.e. a time sheet for a project, rather than for an individual. It should be filled as each project work product is delivered (e.g. domain model, fully dressed use cases, test cases, etc.). The sheets should be simple i.e. the deliverable, the project name, the project phase, dates and a summary total of effort and the number of people involved.
You will probably have an intuitive feel for the major issue with your current approach. Metrics allows you to test that assumption and satisfy yourself that you have done something positive about it. The core principal is that metrics should be simple and useful. Note that as our processes improve and we do more projects , which metric is useful will change and it will be necessary to re-tool the metric reports. Make sure you capture that in a ticket as well!