Implementing the changes
One of the most difficult tasks that any development manager has to face is changing the development process that is in place. There will almost certainly be pressures from both developers and management.Developers are a lot more conservative about things than they would like to think – changing tools and processes always runs the dangers of treading on the toes of somebody’s favourite tool or approach. As developers are the most significant resource in the development process it is important to ensure that they understand that the process is designed to help them produce better code rather than being some kind of management espionage tool.
Management frequently find it difficult to devote resources to process improvement. Process improvement has a cost that cannot always be offset against an obvious immediate benefit. In the long-term there will always be a benefit but it is difficult to quantify in advance and sometimes even to measure retrospectively. In the engineering world outside of software engineering the benefits have been recognised. This is probably because these environments have tangible objects and visible effects. For example in ‘Just In Time’ production environments the parts store is smaller and the loading bays always have trucks in them. The software world only has reams of code. This is where measurement can help – it makes the software visible as numbers that arrive on somebody’s desk. The few minutes saved by each programmer trying to identify a bug because an improvement in code quality makes the task easier is not easily seen, nor is the bug that was found before the customer found it. In neither case can you say for sure that the measurement of particular features of your code saved you money but the cumulative effect will be lower costs and better code quality. Better code quality will improve your product’s standing in the market and mean that developers spend more time developing new code than maintaining existing code.
Let's think about what we are trying to do. We are trying to inroduce a rigorous evidence-based quality process are. The main features of such a process are –
Discipline is the overriding feature of the system and it is achieved through the other three features. Discipline is also required to ensure that the other three features happen.
There are two cultures that need to change to introduce quality systems -
The business culture
The software development culture
Changing the business culture
At the business level quality is in everybody’s interest but nobody’s budget. There is a cost to developing a quality culture and that cost will need to be recovered. The problem is that the cost is tangible but the benefit is frequently intangible.
A sales team might spend six months working with a customer to achieve a sale – there is a cost to this – but the result and the cost are equally visible. Most importantly the cost and the result occur in the same phase and are accounted for in the same budget.
A development team might spend six months improving software quality – the cost of this will be visible but the benefit might not be. However the sales team might not have made the sale if the quality of the software was not so high – but this is only a ‘might’ – we can never prove it. The result has also occurred in a different phase, and more importantly, is accounted for in a different budget.
Equally the cost of putting six months of development effort into quality might be that the six months was not used to add new features to the product. And again the sales team might protest that they lost a sale because the product did not have a particular feature that rival products had. So the business case for quality improvement is a hard sell.
To successfully introduce an evidence based quality process the business must be modified to account for quality and to reward the act of improving quality. The business must think of quality improvement as an investment and act in the same way as it would in respect of any investment i.e. measuring the cost and the return on the investment.
Changing the software development culture
When we come to the software development culture we have a different problem – it’s social rather than budgetary. It’s hard to understand why people trained in a technical discipline would not use readily available technical tools to measure and improve the quality of their output and yet this frequently happens.
As a contrast let’s look at standard ‘physical’ engineering disciplines. In the classic widget factory a design engineer designs the product, a test engineer tests it with a test team, a production engineer then takes that product and tools up a factory to produce it. Assuming this process goes well the design engineer has lost control of his ‘baby’ quite early in the process and is off to design the next ‘new thing’. The test and production engineer has less emotional investment in the product as their role is primarily process. As the product is tangible errors are frequently obvious and only a small number of redesign, build and test cycles are feasible due to cost. This far greater cost of iteration has created a desire for ‘right first time’ design. Another important factor is that the cost of a change is very obvious to the customer.
In the software development process there is more frequent iteration over the design, test, fix phases in the period between initial and final delivery. There is less separation between these phases when compared to the ‘physical’ process and they will frequently overlap. The developer’s involvement is much more integral to the process. They are also more likely to be put under pressure to make functional changes to the product as well as fixing errors caught in the test cycle. Unlike the process of producing a physical item the cost of making a change is not immediately apparent to the customer. If faced with a late request from a customer for a major change in a software project I have frequently pointed out that if you are building a house you know not to ask the builder to move the staircase to the opposite side of the house two days before you want to move in (unless of course you are willing to pay the cost and take the delay).
Although the creation of physical products and software products both have a considerable emotional investment the longer involvement of the software development team makes that involvement deeper. Because you can make changes late in the software development process you tend to do so and therefore you force an unrealistic expectation of perfection on yourself and, possibly, in the mind of the customer. This considerable emotional investment can lead to behaviour that discourages the detection of bugs. This is the main reason that developers should not test their own code. Inexperienced developers frequently ‘ignore’ bugs in their own code even though they know they are there, In general developers are defensive of their code and treat the discovery of bugs or poor design as a personal slight. As a result they are (consciously or subconsciously) unwilling to participate in processes that will ruthlessly expose these bugs.
To successfully introduce these processes we need to encourage developers to believe that -
- it is better for bugs to be found internally rather than by the customer.
- ‘right first time’ means less maintenance and therefore more time can be spent on developing new products.
- the processes that we introduce will achieve these aims.
What kind of change do we want to make?
In essence we are looking for a quality process. In many ways it doesn’t matter what the process is called or who originated as long as it is evidence-based, iterative and automated and as a result encourages a discipline on the participants that rewards them when they do the right thing and warns them if they are doing the wrong thing.
There are some valuable resources that relate to the introduction of quality processes. You may find them useful when evaluating possible approaches. It is important that before you set out to create a quality process that you have some kind of development process in place.
A good starting point is the Capability Maturity Model (CMM) – this provides a roadmap for the incremental adoption of a development process. It can also allow you to judge the level that your current process is at and what measures you might sensibly adopt to move to the next level. The levels are described below in terms of the development artefacts present at each level –
- Level 0: No process. Single body of code on a single machine. Continuous development
- Level 1: Performed Informally. Code base separated and stored after each release. Individual programmers using tools e.g. JUnit, Checkstyle, JHawk. Automated build and release
- Level 2: Planned and Tracked. The programmer is responsible for writing unit tests. Version control. Automated Build and release incorporates quality checks.Code not released until Unit tests run
- Level 3: Well-Defined. IT best practices are documented and performed throughout the enterprise.
IT deliverables are repeatable AND transferable across the company.
- Level 4: Qualitatively Controlled.
Measurable process goals for each defined process. These measurements are collected and analysed quantitatively.
Future IT implementation performance is predictable.
- Level 5: Continuously Improving.
Enterprise has quantitative (measurement) and qualitative (quality) understanding of each IT process.
The company understands how each IT process is related to the overall business strategies and goals of the corporation.
Many software companies reach levels 2 and 3. One of the problems is that that is ‘good enough’ – the software that these companies produce is ‘as good as everybody else’s’ and therefore the customers are reasonably satisfied. I liken this to the products of the 1970’s car industry. The cars were good enough for short commutes, on a long journey in summer they would probably overheat, after 2 years they started to use a lot of oil, after 5 you really needed to get rid of it to somebody who couldn’t really afford a car and after 8 it was only fit for the scrap heap. Since all cars were the same everybody put up with that.
In 30 years that scenario had completely changed. The main reason was the implementation of quality control mechanisms. Many of these mechanisms had been around for a long time but hadn’t been introduced in the car industry as they were viewed as ‘too complex’ to be amenable to the kind of rigorous testing that that a small engineering product could be subjected too. There was also the issue that cars frequently contained parts that were made by suppliers – most companies preferred their suppliers to compete on price rather than quality. There had to be a minimum level of quality of course but price was the primary driver.
The most famous of these quality control systems is the Toyota Production System (TPS). TPS had a very long history but by the late 1970s Toyota had implemented it extensively and people began to notice – unfortunately for the European and US car industries the customers noticed before the car manufacturers.
In the 1970s TPS was already at CMM Level 5. Not only was Toyota using the TPS in its production facilities it was also imposing it on its suppliers. If you wanted to be a supplier to Toyota you needed to have either TPS or an acceptable equivalent running in your plant. Very few of Toyotas suppliers had such systems – as a result Toyota actually started assigning its staff to train its suppliers in TPS. The cost of doing so was borne by Toyota, who were obviously so convinced of the value of the approach that they were willing to do so.
Why is it that we don’t succeed?
It has proved to be very difficult to get software companies to implement these quality initiatives in their development process. There must be reasons why this is the case.
- Pressures generated by the work environment – these have always been there – the Agile Manifesto seeks to embrace change and bring it into the process. I believe that the idea of this was that we should not be alarmed by change. You don’t have to be in the software business long to realise that virtually every deadline presented to you is unachievable. This is because there will be no compromise on the deliverable – so we end up saying that it will be ready when it’s ready. You can only estimate that it will take to provide a solution to a software problem when you can consistently reproduce the problem. If you can locate the task of reproducing the problem with a group other than the development group then you can help. TPS ensures that there is capacity to solve problems and that a problem becomes visible. There are ways you can do this in a development environment that doesn’t involve having a meeting every day – you can put a page up on a Wiki, a whiteboard in the middle of the office or a printed list beside the water cooler. Often in a development group there is someone who has encountered and solved the problem that you are struggling with – without letting everybody in the group know you cannot be sure of getting hold of the right person.
- We don’t learn. When we do something to fix a problem, when we assess a set of potential solutions to an architectural problem and choose one of them, when we find a pattern, when we find an anti-pattern we mustn’t bury our results in the code - we need to document them. We need to document them in a way that allows them to be found again. Personally I favour using a document for each project that is kept throughout the project and is updated with useful information every time you find something.
- We don’t believe in efficiency. We somehow think that ruthless efficiency takes the ‘art’ out of programming. It doesn’t – it reduces the need for maintenance and increases the time available for ‘art’ – that final polish that distinguishes our product from the herd. Instead of trying to just get the damn thing to work for the two weeks leading up to the deadline we can be reducing the memory footprint, improving the speed, making the user interface really snappy.
- Short term thinking conflicts with long term goals. Often the issue raised is the cost of introducing quality processes – but what if you have to send somebody at short notice at business class prices to the other side of the planet? What is the long term effect of employees in a continuous state of crisis – do your best people leave because they can get better conditions? What is the cost of having senior management, even the CEO, sit in meetings with disgruntled customers? What is the cost of loss of reputation and future business due to having a product that is just ‘good enough’?
How can we make change happen?
There are a number of ways that the necessary cultural change can happen -
- Top down. This almost never happens. If the person who runs your company is a person who truly cares about quality the culture will either be there or will be under development already. I really think that for this to succeed the instigator has to be the founder – you have to be in an owner-managed company for this to happen. Equally if your owner-manager didn’t build quality in from the start they are unlikely to start doing after some kind of Pauline conversion. Bought in management is usually there because the original founder has sold the company on and the new management have a whole raft of other issues that they view as more important.
- Middle down. Middle management is seldom the agent of change. They are in the power vacuum between the top and the bottom. Once the decision to change has been made they can be effective as the implementers of that change – but they are unlikely to have been the initiators.
- Bought in consultancy. Reading the Jeffrey Likers book on TPS (Jeffrey Liker – The Toyota Way, McGraw Hill 2004) is depressing. The number of times Toyota sent their guys in to a supplier, effected the change to their methodology, watched it succeed, left thinking ‘job well done’ and came back a few months later to find the whole thing in ruins is just beyond belief. Unless you can leave a champion who is in a position to maintain the standards you are wasting your time. And not only do you have to leave a champion - you have to leave a changed mind set in the company management which ensures that they continue to operate in that way
- Hired evangelist. This is the one I have seen work. You hire an employee who really, really believes in the approach – you put them right in the middle of your developers and you let them get to work. The issue here is personality – many technology evangelists are a bit like their religious equivalents -–they have a ‘my way or the highway’ aspect to them. That just won’t work – your evangelist has to sit there and teach by example – their code always works, they are the only one sitting at the Friday development meeting without a sad story to tell. They have done their work while everyone else is holding up the project – therefore they have time to go out and teach. They are showing people how to write JUnit tests, how to automate the things they are doing laboriously by hand, how to install and use tools to check the quality of their work.
- Internal working group. In this scenario a group of people go off and learn about metrics then come back and spread the word amongst the masses – perhaps through internal training. Having been part of an effort like this I think that if the external training is good some people in the group will become at least slightly ‘evangelical’ and if they are well respected within the company they will help to ‘convert’ those who respect them. This approach can work if there is already a majority in the development team who favour the technology and who are looking for a common approach and a set of guidelines. This is usually more successful than the lone internal evangelist (see below) but it does require a majority of the development team to be at least ambivalent to the new technology.
- Internal Evangelist. You always feel that this should be the most effective way but it has the same problem as the ‘Top Down’ approach above. If you have somebody in your organization who is an effective evangelist they will already have promoted or be promoting their ideas. Saying to someone ‘Off you go and learn about metrics then be an evangelist for them in the company’ just doesn’t work – for a true evangelist it has to come from the heart – people see that very easily.
- Sheer Force. Essentially ‘learn it and do it or there won’t be a place in the company for you'.Doing this is just desperation. If you feel you have to do this either you have the wrong approach or you have employed the wrong people for what you are trying to do.
If you want to find out more about Java metrics you can read our full tutorial here or you can read the individual sections on System and Package Level Metrics, Class Level Metrics and Method Level Metrics.