The CMM establishes a set of public in available criteria describing the characteristics of mature software organizations. These criteria can be used by organizations to improve their processes for developing and maintaining software, or by government or commercial organizations to evaluate the risks of contracting a software project to a particular company.
This chapter focuses on two SEI-developed methods for appraising the maturity of an organization's execution of the software process: software process assessment and software capability evaluation.
This overview is not sufficient by itself for readers to conduct either an assessment or evaluation. Anyone wishing to apply the CMM through these methods should request further information on assessment and evaluation training.
The CMM is a common foundation for both software process assessments and software capability evaluations. The purpose of the methods are quite different, however, and there are significant differences in the specific methods used. Both are based on the model and the products derived from it. The CMM, in conjunction with the CMM-based products, enables the methods to be applied reliably and consistently.
4.1 Software Process Assessment and Software Capability Evaluation Methods
Software process assessments focus on identifying improvement priorities within an organization's own software process. Assessment teams use the CMM to guide them in identifying and prioritizing findings. These findings, along with guidance provided by the key practices in the CMM, are used (by a software engineering process group, for example) to plan an improvement strategy for the organization.
Software capability evaluations are focused on identifying the risks associated with a particular project or contract for building high-quality software on schedule and within budget. During the acquisition process, software capability evaluations may be performed on bidders. The findings of the evaluation, as structured by the CMM, may be used to identify the risks in selecting a particular contractor. Evaluations may also be performed on existing contracts to monitor their process performance, with the intent of identifying potential improvements in the software process of the contractor.
The CMM establishes a common frame of reference for performing software process assessments and software capability evaluations. Although the two methods differ in purpose, the methods use the CMM as a foundation for appraising software process maturity. Figure 4.1 provides a summary description of the common steps in assessments and evaluations.
Figure 4.1 Common Steps in Software Process Assessments
and Software Capability Evaluations
The first step in is to select a team. This team should be trained in the fundamental concepts of the CMM as well as the specifics of the assessment or evaluation method. The members of the team should be professionals knowledgeable in software engineering and management.
The second step is to have representatives from the site to be assessed or evaluated complete the maturity questionnaire and other diagnostic instruments. Once this activity is completed, the assessment or evaluation team performs a response analysis (step 3), which tallies the responses to the questions and identifies those areas where further exploration is warranted. The areas to be investigated correspond to the CMM key process areas.
The team is now ready to visit the site being assessed or evaluated (step 4). Beginning with the results of the response analysis, the team conducts interviews and reviews documentation to gain an understanding of the software process followed by the site. The key process areas and key practices in the CMM guide the team members in questioning, listening, reviewing, and synthesizing the information received from the interviews and documents. The team applies professional judgment in deciding whether the site's implementation of the key process areas satisfies the relevant key process area goals. [ Begin Footnote ] --- These judgements may have to take place without complete information when company proprietary or security issues may be involved.--- [ End Footnote ] When there are clear differences between the key practices in the CMM and the site's practices, the team must document its rationale for judging that key process area.
At the end of the on-site period, the team produces a list of findings (step 5) that identifies the strengths and weaknesses of the organization's software process. In a software process assessment, the findings become the basis for recommendations for process improvement; in a software capability evaluation, the findings become part of the risk analysis performed by the acquisition agency.
Finally, the team prepares a key process area profile (step 6) that shows the areas where the organization has, and has not, satisfied the goals of the key process areas. A key process area can be satisfied and still have associated findings, provided the findings do not identify major problems that inhibit achieving any goals of the key process areas.
In summary, the software process assessment and software capability evaluation methods both:
4.2 Differences Between Software Process Assessments and Software Capability Evaluations
In spite of these similarities, the results of a software process assessment or software capability evaluation may differ, even on successive applications of the same method. One reason is that the scope of the assessment or evaluation may vary. First, the organization being investigated must be determined. For a large company, several different definitions for "organization" are possible. The scope may be based on common senior management, common geographical location, designation as a profit and loss center, common application domain, or other considerations. Second, even in what appears to be the same organization, the sample of projects selected may affect the scope. A division within a company may be assessed, with the team arriving at findings based on a division-wide scope. Later, a product line in that division may be evaluated, with that team arriving at its findings based on a much narrower scope. Comparison between the results without understanding the context is problematic.
Software process assessments and software capability evaluations differ in motivation, objective, outcome, and ownership of the results. These factors lead to substantive differences in the dynamics of interviews, the scope of inquiry, the information gathered, and the formulation of the outcome. The assessment and evaluation methods are quite different when the detailed procedures employed are examined. Assessment training does not prepare a team to perform evaluations, or vice versa.
Software process assessments are performed in an open, collaborative environment. Their success depends on a commitment from both management and the professional staff to improve the organization. The objective is to surface problems and help managers and engineers improve their organization. While the questionnaire is valuable in focusing the assessment team on maturity level issues, the emphasis is on structured and unstructured interviews as tools for understanding the organization's software process. Aside from identifying the software process issues facing the organization, the buy-in to improvement, the organization-wide focus on process, and the motivation and enthusiasm in executing an action plan are the most valuable outcomes of an assessment.
Software capability evaluations, on the other hand, are performed in a more audit-oriented environment. The objective is tied to monetary considerations, since the team's recommendations will help select contractors or set award fees. The emphasis is on a documented audit trail that reveals the software process actually implemented by the organization.
This does not mean, however, that the results of software process assessments and software capability evaluations should not be comparable. Since both methods are CMM-based, the points of comparison and difference should be evident and explainable.
4.3 Other Uses of the CMM in Process Improvement
For software engineering process groups or others trying to improve their software process, the CMM has specific value in the areas of action planning, implementing action plans, and defining processes. During action planning, the members of the software engineering process group, equipped with knowledge of their software process issues and business environment, can compare their current practices against the goals of the key process areas in the CMM. The key practices should be examined in relation to corporate goals, management priorities, the level of performance of the practice, the value of implementing each practice to the organization, and the ability of the organization to implement a practice in light of its culture.
The software engineering process group must next determine which process improvements are needed, how to effect the change, and obtain the necessary buy-in. The CMM aids this activity by providing a starting point for discussion about process improvement and by helping to surface disparate assumptions about commonly accepted software engineering practices. In implementing the action plan, the CMM and the key practices can be used by the process groups to construct parts of the operational action plan and to define the software process.