The Impact of Capability Maturity Model Integration on Return on Investment in IT Industry: An Exploratory Case Study

Capability Maturity Model Integration (CMMI) is a framework known for improving the quality of processes on consistent basis. This paper makes an attempt to explore, analyze and describe the impact of CMMI in terms of return on investment (ROI) for the IT industry. An IT organization who has acquired a maturity level of CMMI 2 is taken as case study. At first, in reference to the level of applied CMMI, a thorough study is completed. Afterwards the luxuries and limitations are studied and explored with reference to organizations applying the monetary terms using ROI framework. Finally, the presented research highlights the key benefits and difficulties in switching to CMMI from conventional quality assurance. Keywords-CMMI; return on investment; ROI; IT; quality assurance


INTRODUCTION
IT industry today has to deal with increased direct customer interaction as more services of an organization are exposed to the customer directly.This is one of the biggest factors driving IT towards business goals such as customer satisfaction, performance, increasing security and decreasing time to market their products.In order to deliver such objectives, organizations are rapidly reviving themselves by spending more on Quality Assurance (QA) and testing.As of 2016-2017 organizations all over the world are spending 31% of their budget in addressing the quality problems [1].In fact this year was proved to be different from the proceeding ones as the budget dropped from 35% to 31%.Before that, the budget of QA and testing was raised from 18% to 35%.However despite this year's reduction, there is an overall prediction that QA and Testing budget will increase to 40% in 2019 [1].To improve the quality of processes and control the cost of QA and testing, the organizations are adopting relative certifications and trainings.
The Capability Maturity Model Integration (CMMI) is one the most popular training and certification programs adopted by more than 5,000 businesses from over 70 countries including United States, Germany, China, Italy, India, Australia, Pakistan etc.Many international organizations and companies demand CMMI maturity to grant contracts as CMMI makes sure that organization processes are of international standard, hence CMMI helps an organization to compete on global scale.Some of the salient advantages of CMMI are consistency, cost reduction, self-improvement and process improvement.The organizations that decide to improve the processes have to pay a significant investment of time and money.A variety of concerns are addressed by CMMI by ensuring process improvement through trustworthy manners.One of the better ways to present financial value is through Return on Investment (ROI) [2,3].This research highlights how to improve and map efforts to the corporate objectives and expectations regarding the effects of CMMI adoption.The ROI will be tracked on an ongoing basis.At the same time intermediate indicators like customer satisfaction, etc. will be tracked to determine the impact of improvement activities.ROI calculation is used along with other approaches to develop a business case for a given proposal.
The key objective of this research is to present a report containing results about the impact of CMMI and also to explain the continuity and future work in this spot.This study specifies and signifies that executing CMMI practices for quality over conventional quality practices is beneficial and the cost spent on obtaining/applying CMMI has break-even with benefits.Furthermore, such information can be made part of quantitative management and be used as an input for improving processes and technologies.The training materials, appraisal methods and process improvement models used in this research are collected via Product Suite of CMMI.Benchmark versions of the models and appraisal methods have been introduced.The use of CMM for scientific data management has been also proposed [5,6].

II. LITERATURE REVIEW
Several researches have shown numerous advantages of process maturity of an organization by using different assessment approaches [7][8][9][10][11].An empirical study was carried out by authors in [12] to find out the effects of CMM on project performance and software quality in Information Systems (IS).The authors claimed that IS implementation strategies are related to CMM levels and higher CMM levels relate to higher software quality and project performance.Authors in [13] explored benefits of CMM such as rework, schedule, productivity, defects, error defection effectiveness, and ROI from the review of seventeen published articles.They concluded that investment in CMM programs leads to enhanced software development and maintenance.Process quality can be improved by adopting CMMI models [14].In [15], author isolated the effort of process maturity effects versus other factors effects through using 161-project samples.Software development efforts can be reduced from 4% to 11% by raising organization to the next maturity level, but the percentage of reduction varies in each level of CMMI.Return on Investment [16], is an output observation when an organization implements some processes.The cost savings is realized by comparing it with the invested cost.The amount saved as a result of process improvement divided by the cost of process improvement is defined as improvement benefit index.The cost of improvement also caters the cost of personnel within a process improvement method such as consultant fees, purchasing tools, and the activities for improving management.The customer satisfaction is not easily quantified within the cost.With the similar procedure, the ROI can be calculated against each Key Process Area (KPA) of CMMI.According to authors in [17], business managers make their decisions on the basis of ROI as it is a highly accepted metric representing adaptability and simplicity.ROI can be calculated by using the following relation: ROI= (GI-COI)/COI, where GI= Gain from investment

COI=Cost of investment
If an investment has a negative ROI then usually it is not used.According to [18] in the IT sector, the ROI measures for evaluation are difficult to calculate by the management.Only 20% of organizations have formal ROI measures while 25% have plans to adopt it in the next 12 months.Literature for cost reduction and impact on schedule using CMM is discussed below.
According to Boeing, Australia [19] the average cost to fix defects was reduced to 33%.Reduction in unit software cost was 20% according to Lockheed Martin [19].10% reduction in overall cost per maturity occurred according to Northrop Grumman [19].Boeing reported 50% reduction in the release, in rework following test up to 60% reduction.General Motors reported a rise from 50% to 95% when number of milestones met.25-30% productivity raised within three years [20].Lockheed Martin reported 50% reduction of software defects and a 55% increase in award fees.
Implementing CMMI process improves schedule and budget predictability, cycle time, productivity, quality, customer satisfaction, and ROI.CMMI process improvement may be established in various ways.For adopting CMMI Product Suite, several organizations have integrated, changed and updated the current processes with the CMMI processes, while some have established new processes.Some organizations have expanded the scope of their improvement efforts by integrating software, hardware and related disciplines with the CMMI model to their existing models.In some cases, few organizations do not adopt CMMI.However, they have established a process which is emphasized differently than the Software Capability Maturity Model (SWCMM), but the outcome shows the significance of the finest practices that are articulated in CMMI models.However, these organizations cannot be accredited as CMMI adopted.Similarly, the impact of CMMI-based process improvement can be used to demonstrate many possible measures of performance.A highlevel description of the CMMI impact based process improvement is shown in Figure 1.In the upper left, costs of process improvement are illustrated.In the center, box process capability and organizational maturity are depicted, and the benefits of process improvement are illustrated in the box on the right.As shown, in order to calculate ROI or related measures the costs and benefits are combined [21].
III. LIMITATIONS Consistently measuring ROI is tough since organizations tend to refrain from sharing their financial data.The study concentrates on one specific software organization in Pakistan, so the facts provided are specific to that organization.High-Level Model of CMMI Impact IV.METHODOLOGY CMMI is one of the ways to implement the quality plans and is generally used as a standard for software quality.The best way to plan quality is to implement CMMI.The quality is achieved by following and implementing CMMI's defined rules and processes.Based on CMMI standard, the Center for Advanced Research in Engineering (CARE) needs to create a Quality Plan that fulfills the requirement of CMMI.

A. Introduction of Organization
An engineering organization is taken as a case study.It has a history of good reputation by receiving acknowledgements in the form of international awards.It is a certified CMMI compliant organization with an engineering team of more than fifty employees and more than fifty employees as supporting staff.In the areas of emerging technologies, the organization offers an extensive list of products, expertise and design services.It stands at the forefront of the technology by focusing on research and development.The organization got flourished through a well-trained, creative, diverse and outstanding team responsible for outstanding products and system design portfolio.The organization achieved CMMI level 2 by implementing all 7 KPAs of Level 2 in two years.A brief summary of CMMI related implementations are:  Process definition by Quality Engineering department.
 Training for managers and engineers regarding CMMI.
 Developed in-house software for configuration management, which also worked with Concurrent Versions System (CVS) and Visual Source Safe (VSS).
 Developed in-house software for reporting.
 Hired people for configuration management.
 Deployed special focus on documentation and task trackers.
V. RESULTS

A. Cost
This category covers the areas where organization reports reduction in the cost of the work product, process and general savings to model-based improvement at various stages, intermediate and final.The summary is shown in Table I.

B. Schedule
The two important aspects i.e., schedule predictability improvement and reduction in time to complete tasks, are covered under this category.Productivity is determined by the quantity of work done over a period of time.The summary is shown in Table II.

C.
Quality The quality improvement is measured with reference to the reduction in the number of defects.Pre-test rework is decreased throughout the duration of a project, which helps in satisfying the customer while improving the project quality.The main objective of the organization is to satisfy its customers by minimizing the defects through their identification at early stages.To achieve this goal, formal inspections of CMMI are quite helpful.The summary of benefits and impact of using CMMI standard with reference to quality is presented in Table III.

E. Return on Investment (ROI)
Return on Investment (ROI) may be calculated through a number of ways yet none of them is defined as official standard.Some of these are as:

•
Benefit -cost ratio (BCR) • Net present value (NPV) • Internal rate of return (IRR) • Payback period (PP), breakeven point (BEP) ratio ROI of an investment (capital) is the ratio of "Total Earnings before interest and taxes" and "average invested capital".However, for calculating ROI for a process improvement project, the computation used is: Return on Investment for a Process Improvement Project = (Benefits from Process Improvement -Cost (investment) incurred towards undertaking process improvement).

2)
CMMI Impact Initial investment: 40,000,000, net profit: 75,000,000.ROI:18% F. Benefits and Cost ratio ROI can be calculated as simply the ratio of enjoyed benefits by a certain effort to the deserved cost.Figure 4 shows the rework have gone out of boundaries at periods 1, 3, 4, 5, and 6, while at other points everything is well managed.This report validates that an organization benefits from CMMI, if it works hard at process improvement.It is hoped that from the current review, the organizations wishing to adopt CMMI can get useful information regarding its impact on ROI.

VII. RECOMMENDATION
For more improvement in the organizational processes and longtime success through customer satisfaction, organizations should take steps towards level 3, 4 and 5. OPD an OPF will provide a base to organization improving, better managing their processes and aligning their organizational vision with the work.Upgrading their work to a higher level will be most helpful for them.


Provides ways to compare the projects Cons  Time value of money (TVM) is not incorporated in ROI  No approach to identify the dollar enormity of ROI  Cannot predict the time to achieve the desired results.

TABLE I .
SUMMARY OF BENEFITS AND IMPACT: RETURN ON COST

TABLE V .
SUMMARY OF BENEFITS AND IMPACT: ROI

TABLE VI .
SUMMARY OF BENEFITS AND IMPACT OF CMMI