[Back] [Contents] [Forward]
2 The Five Levels of Software Process Maturity
Continuous process improvement is based on many small, evolutionary steps rather than revolutionary innovations [Imai86]. The CMM provides a framework for organizing these evolutionary steps into five maturity levels that lay successive foundations for continuous process improvement. These five maturity levels define an ordinal scale for measuring the maturity of an organization's software process and for evaluating its software process capability. The levels also help an organization prioritize its improvement efforts.
A maturity level is a well-defined evolutionary plateau toward achieving a mature software process. Each maturity level provides a layer in the foundation for continuous process improvement. Each level comprises a set of process goals that, when satisfied, stabilize an important component of the software process. Achieving each level of the maturity framework establishes a different component in the software process, resulting in an increase in the process capability of the organization.
Organizing the CMM into the five levels shown in Figure 2.1 prioritizes improvement actions for increasing software process maturity. The labeled arrows in Figure 2.1 indicate the type of process capability being institutionalized by the organization at each step of the maturity framework.
Figure 2.1 The Five Levels of Software Process Maturity
The following characterizations of the five maturity levels highlight the primary process changes made at each level:
1) Initial The software process is characterized as ad hoc, and occasionally even chaotic. Few processes are defined, and success depends on individual effort.
2) Repeatable Basic project management processes are established to track cost, schedule, and functionality. The necessary process discipline is in place to repeat earlier successes on projects with similar applications.
3) Defined The software process for both management and engineering activities is documented, standardized, and integrated into a standard software process for the organization. All projects use an approved, tailored version of the organization's standard software process for developing and maintaining software.
4) Managed Detailed measures of the software process and product quality are collected. Both the software process and products are quantitatively understood and controlled.
5) Optimizing Continuous process improvement is enabled by quantitative feedback from the process and from piloting innovative ideas and technologies.
2.1 Behavioral Characterization of the Maturity Levels
Maturity Levels 2 through 5 can be characterized through the activities performed by the organization to establish or improve the software process, by activities performed on each project, and by the resulting process capability across projects. A behavioral characterization of Level 1 is included to establish a base of comparison for process improvements at higher maturity levels.
2.1.1 Level 1 - The Initial Level
At the Initial Level, the organization typically does not provide a stable environment for developing and maintaining software. When an organization lacks sound management practices, the benefits of good software engineering practices are undermined by ineffective planning and reaction-driven commitment systems.
During a crisis, projects typically abandon planned procedures and revert to coding and testing. Success depends entirely on having an exceptional manager and a seasoned and effective software team. Occasionally, capable and forceful software managers can withstand the pressures to take shortcuts in the software process; but when they leave the project, their stabilizing influence leaves with them. Even a strong engineering process cannot overcome the instability created by the absence of sound management practices.
The software process capability of Level 1 organizations is unpredictable because the software process is constantly changed or modified as the work progresses (i.e., the process is ad hoc). Schedules, budgets, functionality, and product quality are generally unpredictable. Performance depends on the capabilities of individuals and varies with their innate skills, knowledge, and motivations. There are few stable software processes in evidence, and performance can be predicted only by individual rather than organizational capability.
2.1.2 Level 2 - The Repeatable Level
At the Repeatable Level, policies for managing a software project and procedures to implement those policies are established. Planning and managing new projects is based on experience with similar projects. An objective in achieving Level 2 is to institutionalize effective management processes for software projects, which allow organizations to repeat successful practices developed on earlier projects, although the specific processes implemented by the projects may differ. An effective process can be characterized as practiced, documented, enforced, trained, measured, and able to improve.
Projects in Level 2 organizations have installed basic software management controls. Realistic project commitments are based on the results observed on previous projects and on the requirements of the current project. The software managers for a project track software costs, schedules, and functionality; problems in meeting commitments are identified when they arise. Software requirements and the work products developed to satisfy them are baselined, and their integrity is controlled. Software project standards are defined, and the organization ensures they are faithfully followed. The software project works with its subcontractors, if any, to establish a strong customer-supplier relationship.
The software process capability of Level 2 organizations can be summarized as disciplined because planning and tracking of the software project is stable and earlier successes can be repeated. The project's process is under the effective control of a project management system, following realistic plans based on the performance of previous projects.
2.1.3 Level 3 - The Defined Level
At the Defined Level, the standard process for developing and maintaining software across the organization is documented, including both software engineering and management processes, and these processes are integrated into a coherent whole. This standard process is referred to throughout the CMM as the organization's standard software process. Processes established at Level 3 are used (and changed, as appropriate) to help the software managers and technical staff perform more effectively. The organization exploits effective software engineering practices when standardizing its software processes. There is a group that is responsible for the organization's software process activities, e.g., a software engineering process group, or SEPG [Fowler90]. An organization-wide training program is implemented to ensure that the staff and managers have the knowledge and skills required to fulfill their assigned roles.
Projects tailor the organization's standard software process to develop their own defined software process, which accounts for the unique characteristics of the project. This tailored process is referred to in the CMM as the project's defined software process. A defined software process contains a coherent, integrated set of well-defined software engineering and management processes. A well-defined process can be characterized as including readiness criteria, inputs, standards and procedures for performing the work, verification mechanisms (such as peer reviews), outputs, and completion criteria. Because the software process is well defined, management has good insight into technical progress on all projects.
The software process capability of Level 3 organizations can be summarized as standard and consistent because both software engineering and management activities are stable and repeatable. Within established product lines, cost, schedule, and functionality are under control, and software quality is tracked. This process capability is based on a common, organization-wide understanding of the activities, roles, and responsibilities in a defined software process.
2.1.4 Level 4 - The Managed Level
At the Managed Level, the organization sets quantitative quality goals for both software products and processes. Productivity and quality are measured for important software process activities across all projects as part of an organizational measurement program. An organization-wide software process database is used to collect and analyze the data available from the projects' defined software processes. Software processes are instrumented with well-defined and consistent measurements at Level 4. These measurements establish the quantitative foundation for evaluating the projects' software processes and products.
Projects achieve control over their products and processes by narrowing the variation in their process performance to fall within acceptable quantitative boundaries. Meaningful variations in process performance can be distinguished from random variation (noise), particularly within established product lines. The risks involved in moving up the learning curve of a new application domain are known and carefully managed.
The software process capability of Level 4 organizations can be summarized as predictable because the process is measured and operates within measurable limits. This level of process capability allows an organization to predict trends in process and product quality within the quantitative bounds of these limits. When these limits are exceeded, action is taken to correct the situation. Software products are of predictably high quality.
2.1.5 Level 5 - The Optimizing Level
At the Optimizing Level, the entire organization is focused on continuous process improvement. The organization has the means to identify weaknesses and strengthen the process proactively, with the goal of preventing the occurrence of defects. Data on the effectiveness of the software process is used to perform cost benefit analyses of new technologies and proposed changes to the organization's software process. Innovations that exploit the best software engineering practices are identified and transferred throughout the organization.
Software project teams in Level 5 organizations analyze defects to determine their causes. Software processes are evaluated to prevent known types of defects from recurring, and lessons learned are disseminated to other projects.
The software process capability of Level 5 organizations can be characterized as continuously improving because Level 5 organizations are continuously striving to improve the range of their process capability, thereby improving the process performance of their projects. Improvement occurs both by incremental advancements in the existing process and by innovations using new technologies and methods.
2.2 Understanding the Maturity Levels
The CMM is a descriptive model in the sense that it describes essential (or key) attributes that would be expected to characterize an organization at a particular maturity level. It is a normative model in the sense that the detailed practices characterize the normal types of behavior that would be expected in an organization doing large-scale projects in a government contracting context. The intent is that the CMM is at a sufficient level of abstraction that it does not unduly constrain how the software process is implemented by an organization; it simply describes what the essential attributes of a software process would normally be expected to be.
In any context in which the CMM is applied, a reasonable interpretation of the practices should be used. The CMM must be appropriately interpreted, using informed professional judgment, when the business environment of the organization differs significantly from that of a large contracting organization.
The CMM is not prescriptive; it does not tell an organization how to improve. The CMM describes an organization at each maturity level without prescribing the specific means for getting there. It can take several years to move from Level 1 to Level 2, and moving between the other levels will usually take on the order of two years.
Software process improvement occurs within the context of the organization's strategic plans and business objectives, its organizational structure, the technologies in use, its social culture, and its management system. The CMM focuses on the process aspects of a Total Quality Management effort; successful process improvement implies that aspects outside the scope of software process are also addressed (e.g., the people issues involved with changing the organizational culture that enable the process improvements to be implemented and institutionalized).
2.2.1 Understanding the Initial Level
Although Level 1 organizations are frequently characterized as having ad hoc, even chaotic, processes, they frequently develop products that work, even though they may be over the budget and schedule. Success in Level 1 organizations depends on the competence and heroics of the people in the organization. Selecting, hiring, developing, and/or retaining competent people are significant issues for organizations at all levels of maturity, but they are largely outside the scope of the CMM.
2.2.2 Understanding the Repeatable and Defined Levels
As projects grow in size and complexity, attention shifts from technical issues to organizational and managerial issues -- the focus of process maturity [Siegel90, DoD87, GAO-92-48]. Process enables people to work more effectively by incorporating the lessons learned by the best staff into documented processes, building the skills needed to perform those processes effectively (usually via training), and continually improving by learning from the people performing the job.
To achieve Level 2, management must focus on its own processes to achieve a disciplined software process. Level 2 provides the foundation for Level 3 because the focus is on management acting to improve its processes before tackling technical and organizational issues at Level 3. Management establishes a leadership position in achieving Level 2 by documenting and following project management processes.
Processes may differ between projects in a Level 2 organization; the organizational requirement for achieving Level 2 is that there are policies that guide the projects in establishing the appropriate management processes. Documented procedures provide the foundation for consistent processes that can be institutionalized across the organization, with the aid of training and software quality assurance.
Level 3 builds on this project management foundation by defining, integrating, and documenting the entire software process. Integration in this case means that the outputs of one task flow smoothly into the inputs of the next task. When there are mismatches between tasks, they are identified and addressed in the planning stages of the software process, rather than when they are encountered while enacting the process. One of the challenges of Level 3 is to build processes that empower the individuals doing the work without being overly rigid [Humphrey 91b].
2.2.3 Understanding the Managed and Optimizing Levels
Maturity Levels 4 and 5 are relatively unknown territory for the software industry. There are only a few examples of Level 4 and 5 software projects and organizations [Humphrey91a, Kitson92]. There are too few to draw general conclusions about the characteristics of Level 4 and 5 organizations. The characteristics of these levels have been defined by analogy with other industries and the few examples in the software industry exhibiting this level of process capability.
Many characteristics of Levels 4 and 5 are based on the concepts of statistical process control as exemplified in Figure 2.2. The Juran Trilogy Diagram [Juran88] illustrates the primary objectives of process management.
Figure 2.2 The Juran Trilogy Diagram: Quality Planning,
Quality Control, and Quality Improvement
Juran breaks quality management into three basic managerial processes [Juran88]. The purpose of quality planning is to provide the operating forces, i.e., the software producers, with the means of producing products that can meet customer needs. The operating forces produce the product, but some rework must be done because of quality deficiencies. This waste is chronic because the process was planned that way; quality control is carried out to prevent things from getting worse. Sporadic spikes in the process, as shown in Figure 2.2, represent fire fighting activities. Chronic waste provides an opportunity for improvement; seizing that opportunity is referred to as quality improvement.
The first responsibility, and the focus of Level 4, is process control. The software process is managed so that it operates stably within a zone of quality control. There is inevitably some chronic waste, and there may be spikes in the measured results that need to be controlled, but the system is generally stable overall. This is where the concept of controlling special causes of variation comes into play. Because the process is both stable and measured, when some exceptional circumstance occurs, the "special cause" of the variation can be identified and addressed.
The second responsibility, and the focus of Level 5, is continuous process improvement. The software process is changed to improve quality, and the zone of quality control moves. A new baseline for performance is established that reduces chronic waste. The lessons learned in improving such a process are applied in planning future processes. This is where the concept of addressing common causes of variation comes to the fore. There is chronic waste, in the form of rework, in any system simply due to random variation. Waste is unacceptable; organized efforts to remove waste result in changing the system, i.e., improving the process by changing "common causes" of inefficiency to prevent the waste from occurring.
It is anticipated that organizations reaching the highest maturity levels of the CMM would have a process that is capable of producing extremely reliable software within predictable cost and schedule limits. As understanding of the higher maturity levels grows, the existing key process areas will be refined, and others may be added to the model. The CMM is derived from ideas about process that were inspired in manufacturing, but software processes are not dominated by replication issues like a manufacturing process is. The software process is dominated by design issues and is a knowledge-intensive activity [Curtis88].
2.3 Visibility Into the Software Process
Software engineers have detailed insight into the state of a project because they have first-hand information on project status and performance. However, on large projects their insight usually is drawn only from their personal experience in their area of responsibility. Those outside the project without first-hand exposure, such as senior managers, lack visibility into the project's processes and rely on periodic reviews for the information they require to monitor progress. Figure 2.3 illustrates the level of visibility into project status and performance afforded to management at each level of process maturity. Each succeeding maturity level incrementally provides better visibility into the software process.
Figure 2.3 A Management View of Visibility Into
the Software Process at Each Maturity Level
At Level 1, the software process is an amorphous entity -- a black box -- and visibility into the project's processes is limited. Since the staging of activities is poorly defined, managers have an extremely difficult time establishing the status of the project's progress and activities.[ Begin Footnote ] --- This leads to the Ninety-Ninety Rule: 90% of the project is complete 90% of the time.--- [ End Footnote ] Requirements flow into the software process in an uncontrolled manner, and a product results. Software development is frequently viewed as black magic, especially by managers who are unfamiliar with software.
At Level 2, the customer requirements and work products are controlled, and basic project management practices have been established. These management controls allow visibility into the project on defined occasions. The process of building the software can be viewed as a succession of black boxes that allows management visibility at transition points as activity flows between boxes (project milestones). Even though management may not know the details of what is happening in the box, the products of the process and checkpoints for confirming that the process is working are identified and known. Management reacts to problems as they occur.
At Level 3, the internal structure of the boxes, i.e., the tasks in the project's defined software process, is visible. The internal structure represents the way the organization's standard software process has been applied to specific projects. Both managers and engineers understand their roles and responsibilities within the process and how their activities interact at the appropriate level of detail. Management proactively prepares for risks that may arise. Individuals external to the project can obtain accurate and rapid status updates because defined processes afford great visibility into project activities.
At Level 4, the defined software processes are instrumented and controlled quantitatively. Managers are able to measure progress and problems. They have an objective, quantitative basis for making decisions. Their ability to predict outcomes grows steadily more precise as the variability in the process grows smaller.
At Level 5, new and improved ways of building the software are continually tried, in a controlled manner, to improve productivity and quality. Disciplined change is a way of life as inefficient or defect-prone activities are identified and replaced or revised. Insight extends beyond existing processes and into the effects of potential changes to processes. Managers are able to estimate and then track quantitatively the impact and effectiveness of change.
2.4 Process Capability and the Prediction of Performance
The maturity of an organization's software process helps to predict a project's ability to meet its goals. Projects in Level 1 organizations experience wide variations in achieving cost, schedule, functionality, and quality targets. As illustrated in Figure 2.4, three improvements in meeting targeted goals are observed as the organization's software process matures.
First, as maturity increases, the difference between targeted results and actual results decreases across projects. For instance, if ten projects of the same size were targeted to be delivered on May 1, then the average date of their delivery would move closer to May 1 as the organization matures. Level 1 organizations often miss their originally scheduled delivery dates by a wide margin, whereas Level 5 organizations should be able to meet targeted dates with considerable accuracy. This is because Level 5 organizations use a carefully constructed software process operating within known parameters, and the selection of the target date is based on the extensive data they possess about their process and on their performance in applying it. (This is illustrated in Figure 2.4 by how much of the area under the curve lies to the right of the target line.)
Figure 2.4 Process Capability as Indicated by Maturity Level
Second, as maturity increases, the variability of actual results around targeted results decreases. For instance, in Level 1 organizations delivery dates for projects of similar size are unpredictable and vary widely. Similar projects in a Level 5 organization, however, will be delivered within a much smaller range. This narrowed variation occurs at the highest maturity levels because virtually all projects are performing within controlled parameters approaching the organization's process capability for cost, schedule, functionality, and quality. (This is illustrated in Figure 2.4 by how much of the area under the curve is concentrated near the target line.)
Third, targeted results improve as the maturity of the organization increases. That is, as a software organization matures, costs decrease, development time becomes shorter, and productivity and quality increase. In a Level 1 organization, development time can be quite long because of the amount of rework that must be performed to correct mistakes. In contrast, Level 5 organizations use continuous process improvement and defect prevention techniques to increase process efficiency and eliminate costly rework, allowing development time to be shortened. (This is illustrated in Figure 2.4 by the horizontal displacement of the target line from the origin.)
The improvements in predicting a project's results represented in Figure 2.4 assume that the software project's outcomes become more predictable as noise, often in the form of rework, is removed from the software process. Unprecedented systems complicate the picture since new technologies and applications lower the process capability by increasing variability. Even in the case of unprecedented systems, the management and engineering practices characteristic of more mature organizations help identify and address problems earlier in the development cycle than they would have been detected in less mature organizations. Earlier detection of defects contributes to project stability and performance by eliminating the rework during later phases. Risk management is an integral part of project management in a mature process. In some cases a mature process means that "failed" projects are identified early in the software life cycle and investment in a lost cause is minimized.
2.5 Skipping Maturity Levels
The maturity levels in the CMM describe the characteristics of an organization at a maturity level. Each level builds a foundation for succeeding levels to leverage for implementing processes effectively and efficiently. Organizations can, however, profitably use processes described at a higher maturity level than they are. Engineering processes, such as requirements analysis, design, code, and test, are not discussed in the CMM until Level 3, yet even Level 1 organizations must perform these activities. A Level 1 or Level 2 organization may be able to perform peer reviews (Level 3), do Pareto analysis (Level 4), or pilot new technologies (Level 5) profitably. When prescribing what steps an organization should take to move from Level 1 to Level 2, frequently one of the recommendations is to establish a software engineering process group, which is an attribute of Level 3 organizations. Although measurement is the focus of Level 4, it is also an integral part of the lower maturity levels.
These processes cannot reach their full potential, however, until the proper foundation is laid. Peer reviews cannot be fully effective, for example, unless they are consistently implemented, even when fires threaten the project. The maturity levels describe the problems that predominate at a level. The dominant problems of a Level 1 organization are managerial; other problems tend to be masked by the difficulties in planning and managing software projects.
Skipping levels is counterproductive because each level forms a necessary foundation from which to achieve the next level. The CMM identifies the levels through which an organization must evolve to establish a culture of software engineering excellence. Processes without the proper foundation fail at the very point they are needed most -- under stress -- and they provide no basis for future improvement.
A Level 1 organization that is trying to implement a defined process (Level 3) before it has established a repeatable process (Level 2) is usually unsuccessful because project managers are overwhelmed by schedule and cost pressures. This is the fundamental reason for focusing on management processes before engineering processes. It may seem easier to define and implement an engineering process than a management process (especially in the eyes of technical people), but without management discipline, the engineering process is sacrificed to schedule and cost pressures [Humphrey88].
An organization that is trying to implement a managed process (Level 4) without the foundation of a defined process is usually unsuccessful because there is no common basis for interpreting measurements without defined processes. While data can be collected for individual projects, few of the measurements have significant meaning across projects, and they do not significantly increase organizational understanding of the software process. It is difficult to identify meaningful measurements in the absence of defined processes because of the variation in the processes being measured.
An organization that is trying to implement an optimizing process (Level 5) without the foundation of a managed process (Level 4) is likely to fail because of a lack of understanding of the impact of process changes. Without controlling the process within statistically narrow boundaries (small variations in process measures), there is too much noise in the data to determine objectively whether a specific process improvement has an effect. Decisions can degenerate into religious wars because little quantitative foundation exists for making rational, informed decisions.
The process improvement effort should focus on the needs of the organization in the context of its business environment. The ability to implement processes from higher maturity levels does not imply that maturity levels can be skipped.
[Back] [Contents] [Forward]
[Главная страница сайта]