UNIVERSITY OF BOTSWANA MSc.

(COMPUTER SCIENCE)

RESEARCH PROPOSAL

TOPIC: SPI-BASED EVALUATION OF THE STATE OF SOFTWARE DEVELOPMENT PRACTICE IN BOTSWANA

STUDENT NAME: M. MOTLHALA

STUDENT NUMBER: 9703761

Abstract Software is today a very important tool in every aspect of our life. However acquisition of quality software that meets the customers requirements has been a challenge. Software project failure has been a worldwide phenomenon, and still, today there are software development projects having cost over runs, running out of schedule, producing products not meeting user expectations or those failing to be operational at all. In every development process when there is failure, one should want to know why there is failure in development or why the product produced is of low quality (does not meet the expected standards). It is through this concerns that the concerned will seek solutions to the problem. In the software development world, to deal with this software development problems researchers came up with models, frameworks and approaches that can be used to assess software development processes, in a way to improving software processes because there is a wide spread belief that quality of a software product is directly dependent on the quality of the process that produced it (Zarour, et al., n.d.) (Jarvinen, 2000) (Basri & O'Connor, 2010).That is why is so pertinent as a software producing company, to study software processes in order to identify strengths and weakness, and perform improvements were there is a need. The objective of this study is to compare the software development practices in Botswana software development companies against best practices in the software development industry. This exercise will help in identifying weaknesses in software development processes employed by software development companies in Botswana. It is from the weaknesses that will be found that improvements towards best practices can be made. To archive the objective of the study a review of the literature on frameworks or models used for software development process assessment will be done to make a conceptual comparison of the framework or model that can be applied in a Botswana situation. Most of the research done on software development process assessment frameworks or models is biased towards frameworks that are most applicable in big organizations that have more than 50 employees, and there is not much of literature on software process assessment of small and medium sized organizations similar to which are found in Botswana. Most importantly, there is lack of literature related to the topic that has been done in Africa, and there is nothing found that Page | 2

has been done in Botswana all together. These goes to show that the state of software development practice or adoption of best practices in software development companies of Botswana is unknown. To further try to achieve the objective of the study, a survey pertaining to software development practice will be carried out on Botswana companies. The survey will be based on best practices, to help in measuring or assessing level of adoption of best practices by software development companies in Botswana. Keywords: Botswana, Software Development Processes, Best Practices, Assessment

Page | 3

Table of Contents
Abstract...................................................................................................................... 2 Keywords: Botswana, Software Development Processes, Best Practices, Assessment. 3 Table of Contents........................................................................................................ 4 Introduction................................................................................................................. 4 Literature Review...................................................................................................... 14 Scope........................................................................................................................23 Project Plan...............................................................................................................23 Reference.................................................................................................................. 23

Introduction Software today is so dominant in our lives more than it was during yesteryears. Software is a very important aspect of our lives today. It is used in each and every industry; in schools, health facilities, car manufacturing industries, hotels, airports, army, telecommunication, mining, and many other fields. Software is used to operate different equipment or gadgets today. Imagine Page | 4

software used in a life supporting equipment in a hospital, what do you think will happen if the software fails? I suppose any person will think death in that case. This is how important software can be today; its use in very critical systems. The vast usage of software in different industries goes to show the high demand of software in the entire world. It is in that case that companies are competing to supply software products to different organizations and individuals in the industry, and if the supply surpasses demand the best product will be chosen (Haase, 1996). The best product will supposedly be the one that meet user requirements, developed within schedule, and is developed within budget, or it might be the one that has been developed using good software development practices. So, high quality software that meets user requirements, and is developed within budget and schedule is what software development organizations are striving for today. Despite the importance of software and a need for it to be of high quality, there are still problems that are evident in the software development industry. The software sometimes do not meet user requirements, are delivered late, they exceed the budget, and some even fail to be operational at all. This is highlighted by the chaos report (by Standish Group) of 2004 (Preuss, 2006). The following graphs show the results of the Chaos 2004 Survey of the Standish Group. Only 29% of the projects were successful in 2004. This is so worrisome because the figure is far much lower than half.

Page | 5

A. Rate of Failure or Success of Projects in 2004

B. Rate of Projects per year from 1994 to 2004

Failure or Success of

Page | 6

B. Average percent of cost overrun per year from 1994 to 2004

D. Average percent of time overrun per year from 1994 to 2004

Figure 1: Results from Chaos Survey of 2004

Page | 7

The study of Standish Group (1995) showed that 31.1% of all information technology development projects are cancelled before completion and on average only 16.2% of software projects are completed on-time and on budget (Niazi, et al., 2003). One can see that from 1995 to 2004 successful projects increased by 12.8% from 16.2% registered in 1995. Though the Chaos report does not cover all the companies, small and large, in the entire world, at least it is highlighting the problem of software project failure that exists in the software development industry. Research has shown that one way to address this failure problem is to assess the processes involved in the development of software in order to find out where there are weaknesses and then plan for improvements needed. That is when you find out whether best practice in software development are followed, and try to close the gap between the current situation of failures and what the best practice suggest should be done in helping to produce quality software. This is stressed by (Chevers & Duggan, 2007) when they indicated that among the factors that influence IS quality the most pivotal is software production process. It is therefore wise for software development organizations to strive for refinement and improvement of their development practices to maintain and increase competitive advantages, which will be brought about by improved quality of development process and finally resulting in quality products. There is wide spread believe that software product quality is positively related to software development process and thus calling for software process assessment and improvement (Chevers & Duggan, 2007) (Dyba & Skogstad, 1997) (Zarour, et al., n.d.) (Jarvinen, 2000) (Basri & O'Connor, 2010). Dyba & Skogstad (1997) are therefore saying the role of process in that case is to tie together the people developing the software and the technology used, as well as the product complexity and environmental characteristics (illustration is on figure 2).The need for software process assessment and improvement led to the adoption of the Software Process Improvement (SPI) concept which has received a lot of attention from researchers in the area of software engineering. Software process improvement is said to be a critical research and business priority aiming at achieving competitive advantage and its intents among others are (Dyba & Skogstad, 1997): Improving software product quality Increasing productivity Decreasing the lead time for product development

Page | 8

Figure 2: Process as an integrating factor In order to help in improving the quality of software development and quality of software products, and eliminate the problem of budget schedule overruns some research based organizations developed some process capability models/frameworks known as software process improvement (SPI) models/frameworks. These models can be refered to as SPI best practice models according to (O'Connor & Laporte, 2011). This has been asserted by (Halvorsen & Conradi, n.d.); that a number of Software Process Improvement (SPI) frameworks have evolved from this principle. Software Engineering Institute (SEI) developed the Capability Maturity Model (CMM) which is the widely used model worldwide. CMM Integration (CMMI) is another model after CMM developed by SEI. The International Organization for Standardization (ISO) came up with the ISO/IEC 15504 model; also know as SPICE (Software Process Improvement and Capability dEtermination). These two models are the core or bases of software process improvement. Other models which followed after CMM and SPICE include Trillium and BOOTSTRAP. These SPI models are for defining and measuring processes and practices that can be used by software developing companies (Staples, et al., 2007). The introduction of the models outlined above did not come without challenges though. There is a wide spread believe that all the models outlined above are biased towards large companies, and some research has been done to justify that thought or belief. Chevers and Duggan (2007) in their Page | 9

study carried out in Jamaican software companies also stressed this belief when they said the overwhelming majority of research undertaken in the software process improvement domain has evaluated the impact of SPI initiatives on software quality concerns within organizations in developed countries and less than a handful have focused on SPI initiatives in developing countries. That would mean small software companies would not benefit from these core SPI models as much as large software companies would benefit. This became a cause for concern to different quarters of software development research looking at the part small software companies play in the economies of different countries. Research has shown that the software development industry is dominated by small companies, and that is stressed by (Richardson & von Wangenheim, 2007) by indicating that 85% of software organizations in US, Brazil, Canada, China, India, Finland, Ireland, Hungary, and many other countries are small organizations. One main consideration should be; though the focus on large software organizations by the SPI model outlined above, Richardson and von Wangenheim (2007) assert that small and large organizations face similar software engineering challenges, and thus the need to assess and improve their software processes just like it can be done in large software companies. Also, small software companies just like large organizations need to deal with rapid technology changes, maintain their products, operate in a global software environment, and sustain their organizations through growth. Chevers and Duggan (2007) also indicated that small and large organizations face similar pressures from clients to produce high quality systems and other software products, but have additional concerns related to size and the availability of resources. However, they often require different approaches because of specific business models and goals, market niche, size, availability of resources (financial and human), process and management capability, and organizational differences, among other things Richardson and von Wangenheim (2007). O'Connor and Laporte (2011) supported Richard and von Wangenheim; saying all software companies are not the same and vary according to factors including size, market sector, time in business, management style, product range and geographical location. OConnor and Larpote continued to stress that those who develop software process and process improvement models should then be aware of the fact that software companies are different. OConnor and Larporte again raised an important point that, to be widely adopted by the software industry, any process or process improvement model should be capable of handling the differences in the operational contexts of the companies making up that industry. As highlighted above the CMM, Page | 10

ISO/IEC, TRILLIUM, and BOOTSTRAP are not suitable for small organizations (Anacleto, von Wangenheim, Salviano, & Savi, 2004) because they require a lot of financial resources to use which small companies cant afford, they require specialized personnel which small companies normally dont have or cannot afford, they require a lot of time to carry out, and that is not favored by small companies, also, it takes time for companies to realize the benefits of using these models, and this is also not favored by small organizations. These models are also said to be cumbersome. According to O'Connor and Laporte (2011), it has been reported that small software companies find it difficult to relate standards (SPI models) to their business needs and to justify the application of the international standards in their operations. Regardless of the contribution that small software organizations make in the economies of countries and having predominant SPI models which are not favorable to them, there is still little research done on SPI models intended for the improvement of processes and software product quality of small software organizations. It is however important to note that there are some efforts made in some quarters in order to address this problem faced by small software organizations. A few organizations in different countries have developed models which are more localized than being international. That is, these models unlike CMM and ISO/IEC 15504 have not been used in different countries. Some of these models include FAME, RAPID, SPIRE, and MARES. These small organizations models where based on the original models, CMM and ISO/IEC 15504. Because of this phenomenon of little research concerning SPI in small organizations especially in Africa, the author has found it wanting to make a contribution towards software development research by conducting an SPI based research in small software companies of Botswana. In addition, lack of usage or testing of these existing small organizations models in different environments, also motivated the author to propose the assessment of software development practices in Botswana where there is little or nothing (to the best of my knowledge) that has been documented or published about software development or SPI initiatives. It would also be important to highlight that, software development organizations in Botswana fall under the small category which is most predominant in developing countries. The author therefore intends to use one of the existing SPI models developed for small organization to evaluate the software development process of organizations in Botswana. The model will be chosen using the criteria that will be determined from the literature.

Page | 11

Definition of Terms A software process is a set of partially ordered process steps, with sets of related products, human and computerized resources, organizational structures and constraints, intended to produce and maintain the requested software products (Lonchamp, 1993). A software process assessment is the means whereby organizations can identify and assess their strengths, weaknesses, existing improvement activities, and key areas for further improvement (Lane, 2012). Main Objective The author proposes to carry out a study to assess the extent of adherence to software development best practice in software development companies of Botswana. Specific Objectives

To identify an SPI model that can be used to assess the software processes of software companies in Botswana To highlight the strengths and weaknesses of software development in Botswana, in line with software processes To add to software process improvement area of research in Africa, particularly in Botswana

1.1. Project Problem Statement Though there is an improvement in terms of software projects success as evidenced by yearly Chaos results on software projects failure or success, the problem of project failure is far from over and it still need serious attention. The importance of software in our lives is to a big extent clear as evidenced by its usefulness in all aspects of industry. Its importance is increasing even further. This increasing importance of software in our lives calls for adoption of best practice by practitioners so as to improve the quality of the processes in use, and therefore achieve targets relating to time, budget and quality (Cater-Steel, 2004). Adoption of best practice would also help companies to be competitive in the world market of software development. There have been some efforts towards improving software development processes of some software companies in some countries outside Africa (e.g. Brazil, Mexico, Finland, Jamaica, Australia, USA, and etc). However all this efforts outside Africa, it seems like Africa has not yet joined bandwagon of this SPI research which aimed Page | 12

at improving software development processes and eventual. On top of the evident software projects failure, there is no or little research that has been done in Africa, particularly Botswana that is targeting the improvement of software processes in local software companies. This lack of SPI research and a need to pick the standard or quality of software products developed by local companies, has pushed the author to propose the assessment of the state of software development practice in Botswana to fill this clear gap and improve the state of practice in Botswanas software companies. Currently, the state of software development practice in software development companies of Botswana is unknown. There is therefore a need as stated above to assess the state of software development practice in Botswana to set a foundation from which improvements can be made as strengths or weaknesses will be identified from the assessment. Thus the research problem addressed in this study is: To what extent are software organizations in Botswana adhering to software development best practice?

The Purpose of Study The purpose of this study will be to find the weaknesses in the practices of software development companies of Botswana which help in identifying areas of improvement. This study also aims at recommending a SPI framework for use evaluation and improvement of software development practices of software companies in Botswana. This study would like to lay the foundation for future improvement of software development practices in Botswana which will in turn improve the software quality produced by these companies.

Page | 13

Source: http://www.npd-solutions.com/pdbpa.html

Literature Review The purpose of this chapter will be to discuss the core or main SPI models on which the other models for small companies are based, together with their origin. This chapter will also discuss the SPI models developed in different countries of the world, and other theories surrounding software process improvement like benefits of SPI and factors that influence success of SPI. Since the more than a decade back the software process engineering area saw the emergence of various software process models whose main goal was to provide a way to assess and improve software development processes in the industry. Among these models are the Capability Maturity Model (CMM) and Capability Maturity Model Integration (CMMI) from the Software Engineering Institute (SEI), Software Process Improvement and Capability dEtermination (SPICE) also known as ISO 15504, BOOTSTRAP, Trillium, Tickit, COBIT, and OWPL. 2.1. Main SPI Models

Page | 14

Capability Maturity Model (CMM) The CMM model might appear to be the first and the best known and most widely used model world-wide. CMM came about as a response to the need of the Department of Defense (DoD) because most of their projects were over time and budget. So, the DoD asked the SEI to provide leadership in assisting software organizations to develop and continuously improve their capability to identify, adopt, and use sound management and technical practices (A Framework for Software Process Model Selection and Model Tailoring, Simon Alexandre). (Haase, 1996) claims that CMM is the Father of all maturity based instruments. Many sources from literature have also indicated that it is the leading quality improvement standard in North America for software development. Haase indicated that CMM is useful in large units of more than 200 persons, and almost useless for small enterprises of less than 30 people. CMM for Software specifically describes the principles and practices underlying process maturity and is intended to help software organizations improve the maturity of their software processes. The CMM for Software is organized into five maturity levels, which represents less mature processes to highly mature processes. Each maturity level, except level 1, is decomposed into several key process areas that indicate the areas organization should focus on to improve its software process.

BOOTSTRAP BOOTSTRAP assessment and improvement methodology is the product of the ESPRIT project also named BOOTSTRAP. The project took the Software Engineering Institutes (SEI) Capability Maturity Model (CMM) as the basic reference and extended it in several ways. The BOOTSTRAP assessment methodology describes the assessment process, determines where an organization stands (maturity level), identifies its strengths and weaknesses (capability), and offers improvement guidelines (action plans). The BOOTSTRAP Assessment is performed separately at both the organization and project levels. This kind of assessment is said to be the main advantage of the BOOTSTRAP methodology (Kuvaja & Bicego, 1994). The models and standards that are used as a basis for BOOTSTRAP methodology include: CMM, ISO 9001 and ESA-PSS 005. The maturity levels used by the BOOTSTRAP methodology follow an identical algorithm as that used for CMM. BOOTSTRAP use the quality profiles of the processes to reflect the strengths and weaknesses. The improvement plan is also derived from the capability profiles. Page | 15

Unlike CMM, BOOTSTRAP methodology can be applied to small and medium sized companies (Haase & Messnarz, 1994). There are important improvements made to the traditional maturity level philosophy of process improvement in the BOOTSTRAP methodology. The enhancements include: Capability may also be measured between maturity levels on the basis of quartile scale dividing each maturity level into four quartiles, The capacity results may be focused on preselected major functions of the process areas and expressed with process attributes, The capability is determined both for the Software Producing Unit and its projects, which then can be compared with one another, The capability profiles can also be compared with the European rolling means taken from the BOOTSTRAP database BOOTSTRAP Assessment Process

In the BOOTSTRAP methodology it is recommended that assessments be performed with assistance from an external assessor. This can be a problem to small and medium sized enterprises as they might not afford services of external assessors.

The experiment from the BOOTSTRAP project shows that self-assessment does not have the same effectiveness as assisted assessment. (Pasi Kuvaja and Adriana Bicego) Assisted assessment is preferred because it the methodology is applied more rigorously and without any personal tendency to influence the results of the evaluation. Also, the external assessor has wide experience gained from earlier assessments performed, thus it assures reliable results than self-assessment.

The assessment process includes three major phases; Preparation phase, assessment execution, and actions plan derivation.

ISO/IEC 15504 (SPICE) ISO/IEC 15504 is an international standard for assessing software processes. A prime motivation for developing this standard has been the perceived need for an internationally recognized software process assessment framework that pulls together the existing public and proprietary models and Page | 16

methods, so is more like an umbrella model that harmonizes the concepts of other existing models. ISO/IEC 15504 is composed of a document set with nine parts, and the set contains an assessment model known as the exemplar assessment model (Part 5). Its purposes are: Continuous process improvement, and Capability determination. Its scope include: acquisition, supply, development, operation, maintenance and support processes. The ISO 15504 standard went through some trials through an effort know as SPICE, and that was an international empirical study. That was partially to address concerns within the software engineering community with the lack of evidence supporting software engineering standards (El Emam & Jung, 2001). There was a need for an empirical basis that would show that the standards indeed represent good practices.

Assessment Using ISO/IEC 15504 The reference model, which is Part 2 of the document set of ISO/IEC 15504, acts as a basis for performing assessments of software process capability. The architecture of ISO/IEC 15504 which is defined in this reference model (Part 2) is two dimensional. One dimension consists of processes that are actually assessed. These processes are grouped into five categories. The second dimension is the capabilities dimension, which consists of the capability scale that is used to evaluate the process capability. This capability scale is used across all the processes. Having processes in the dimension doesnt mean that all the processes will be assessed for every dimension. An organization can scope an assessment to cover only the subset of processes that are relevant for its business objectives. This might mean the assessment can be tailored for small or medium sized software producing organization to include processes that are relevant for their business objectives. The output from a process assessment is a set of process profiles, one for each instance of each process within the scope of the assessment. Each process profile consists of one or more process attribute ratings for an assessed process, and each attribute rating represents a judgment by the assessor of the extent to which the attribute is achieved. You can aggregate attributes ratings to produce a capability level. As the basis for conducting reliable and consistent assessments of process capability, the reference model, containing descriptions of process purpose and process attributes need to be supported with comprehensive sets of indicators of process performance and capability, and that is catered for in the Exemplar model. The exemplar model expands the Page | 17

reference model by adding the definition and use of assessment indicators. Assessment indicators are objective attributes or characteristics of a practice or work product that support an assessors judgment of the performance or capability of an implemented process. The indicators that are found in this model are in two classes; indicators for process performance, and indicators of process capability. The indicators for process performance relate to base practices defined for the process dimension, and indicators for process capability relate to the management practices for the capability dimension. TRILLIUM Trillium is a product of Bell Canada, and its main focus is telecommunication but is also applicable to other domains including software development. The goal of the model is to provide a means to initiate and guide a continuous improvement program. The model is used in a variety of ways: To benchmark an organizations product development and support process capability against best practices in the industry In self-assessment mode, to help identify opportunities for improvement within a product development organization In pre-contractual negotiations to assist in selecting a supplier The Trillium Model provides key industry practices which can be used to improve an existing process or life cycle. The practices in the Trillium Model are derived from a benchmarking exercise which focused on all practices that would contribute to an organizations product development and support capability. The Trillium model is based on CMM. Additional sources are ISO 9001 and ISO 9000 3. Trillium uses the Capability Evaluation and Capability JointAssessment methods for evaluating an organizations product development and support process capability. A Capability Evaluation is the evaluation of a supplier by a second party, typically the customer. A Capability Joint-Evaluation assumes an effective partnership relationship exists between the customer and supplier. The Capability Self-Assessment and the Continuous Improvement (CI) program are linked, in that often the first will initiate the latter. Both are conducted internal to the organization (i.e. first party). The goal is to: Identify opportunities for improvement, and Deploy a robust management mechanism to continuously identify and act on improvement opportunities Page | 18

The application of Trillium is most effective if it is used uniformly across the organization, involving all departments or personnel affecting product development and support. The ultimate objective of the improvement programs initiated as a result of a Trillium assessment is increased customer and shareholder satisfaction, rather than rigid conformance to the standards. Even though there are opinions that small companies are condemned to fail when implementing these frameworks, there are some assertions in the literature that small companies are able to implement software process improvement as effectively as large companies (Shen & Ruan, 2008). 2.2. SPI Models for Small Software Companies Because of the challenges faced by small software companies in applying SPI models applied in large companies that are mostly found in developed countries, Chevers and Duggan (2007) proposed a modified framework for assessing software development processes in small software producing companies of Jamaica. The modification was based on the Capability Maturity Model for Software (SW - CMM). The selection of the CMM as the basis for modification was based on the following reasons as stated by Chevers and Duggan: (1) it is claimed that it is the Father of all maturity based assessment models, (2) it is widely know and well documented, (3) it is acclaimed as the leading quality improvement standard in North America for software development, (4) it is not only an assessment instrument but it is also a framework that describes the key elements of an effective software process, and (5) it prescribes an evolutionary improvement path that includes five maturity levels on a continuum from an ad-hoc and chaotic process to a fully mature and disciplined one. The modified model included only the first three maturity levels of the original CMM model. Some key process areas (KPAs) in the three levels were eliminated and some were combined to form one KPA. The model also added one important and applicable KPA from the top eliminated levels to one of the included levels. However, Chevers and Duggan say one limitation to their approach is that they used their own judgment and knowledge of the software development environment in Jamaica as the basis for modifying the SW CMM structure and instrument. In another study, (Anacleto, von Wangenheim, Salviano, & Savi, 2004) worked on customizing the ISO/IEC 15504 model/framework so that it is applicable in small companies. The study was based Page | 19

on small companies of Brazil. The customization of the ISO/IEC model took Part 5 of the document set (the exemplar model) as the basis for modification or customization. The resultant assessment methodology was the MARES. MARES is composed of; a process assessment model based on Part 5 of the ISO/IEC 1554 document set (the exemplar model), including a process reference model and a measurement framework, and a context-process model and a process-risk model. The MARES process assessment model adopted levels 0 to 3 of the capability dimension as they are from the exemplar model. Just like in Chever and Duggans (2007) study in Jamaica several processes of the exemplar assessment model have been excluded from the MARES as being irrelevant for small companies. The exclusion was due to the specific characteristics of small software companies. Some processes were unified into one process. The context-process model of MARES helps the competent assessors to adapt the standard appropriately to a specific organization. The MARES assessment method include; planning, contextualization, execution, monitoring and control, and post-mortem. The ISO/IEC 15504 was chosen because comparably it is flexible, and that facilitates the adaptation of the standard to a specific context, such as, for example, small software companies. A study was carried out in Australia to evaluate the effectiveness of process improvement programs, and an assessment-based SPI program was carried out in 22 small software development firms in Australia. The process improvement program used the Rapid Assessment for Process Improvement for software Development (RAPID) model. The RAPID method is based on the Technical Report (TR) version of the ISO/IEC 15504. However, unlike the SPICE model, the RAPID assessment model only goes up to level three (established), not level five. RAPID method collects evidence only by interview, and reference to documents may be done to illustrate some issues under discussion (Cater-Steel & Rout). In this evaluation, a pool of nine qualified SPICE assessors was deployed. A set procedures and templates was used including a demographic questionnaire, assessment plan, assessment instrument, assessment report, feedback form, followup meeting and final report. A four point scale (not achieved; partially achieved; largely achieved; and fully achieved) was used to assess the processes. That is, to determine the extent to which the process attributes have been achieved. It was from there that the capability level (0, 1, 2, and 3) for each of the eight processes adopted for RAPID was determined. A report compiled for each

Page | 20

organization included strengths, weaknesses, process attributes ratings and capability levels, and recommendations for improvement. Another model/method discussed in the literature, that can be used for assessment of software development processes in small software companies is the FAME (Fraunhofer IESE Assessment Method) method. The FAME method unlike other models has some approaches that can be followed to select processes that can be assessed. That is, its approach is to systematically determine the processes to improve by taking the business focus into consideration. There is belief that by doing so, no time will be wasted unnecessarily on assessing processes which will not benefit the organization. FAME uses a common framework of best software engineering practices; that is ISO/IEC TR 15504. As is evidenced from the literature ISO/IEC TR 15504 has been recognized and used around the world. One good thing about the ISO/IEC 15504 framework is that, through the SPICE trial, it has been internationally validated to prove its usefulness for performing assessments. Established methods like BOOTSTRAP and CMMi already use this framework. FAME contains four added value elements namely; Business Focus, Efficiency, Reliability and Benchmarking. The approach that FAME use to determine processes to be assessed or to improve is Product-Process Dependency (PPD) Modelling, Study on Influential Processes, and Experience based Heuristics. 2.3. Comparison of SMEs Specific Models and Choosing of one Many people who work in the SPI domain have chosen one SPI framework as their favorite. This choice is mostly subjective and seldom based on objective evidence of appropriateness. A reason for this is that SPI frameworks are difficult to compare due to their comprehensiveness (Halvorsen & Conradi, n.d.). The frameworks also differ in basic assumptions such as how the terms software process and software quality should be interpreted, making the comparison task even harder. The four classes of comparison methods according to Halvorsen & Conradi include;

Characteristics Comparing attributes (as discussed in Ares et al. 2000, Cattaneo et al. 2001, Haase 1996, and Srumgrd 1997) Framework mapping - Comparing kernel concepts (as discussed in Emam et al. 1997, Paulk 1995, Paulk 1995;2, Software Engineering Institute 1998, and Tingey 1997)

Page | 21

Bilateral comparison - Comparing textual phrases (as discussed in Emam et al. 1997, and Paulk 1995) Needs mapping - Comparing user needs vs. framework properties

Methodology The underlying research paradigm of this study will be a positivist and the research approach is both descriptive and evaluative. The extent of adoption of best practice by software development small software companies will be evaluated. This study conforms to the nomothetic research style as it uses a survey and field experiments to test hypotheses within the scientific tradition. Quantitative analysis, focusing on statistical analysis of numerical data, and also qualitative analysis of textual and numerical data, are employed. As detailed in Chapter 3, this approach is consistent with the traditional research approach adopted by software engineering researchers. A survey is considered to be a feasible means of providing data for any study investigating the state of practice (Wilson, Petocz & Roiter 1995). In this case, the survey provides a broad industry-wide snapshot of the adoption of best practice techniques in use throughout software development groups in Queensland. The mail survey is complemented by the field experiments, which provide an indepth analysis of the outcomes of a software process improvement program involving 22 small software development firms. This type of multi-method approach is strongly advocated by Morrison and George (1995) and other software engineering and information systems researchers (for example Gable 1994; Gallivan 1997; Groves et al. 2000). It is claimed that a superior piece of research can be achieved by combining the main strength of survey research, generalisability, with the main strength of experimentation (Gutek 1991). The multi-method approach used in this study provides richer data and the opportunity to compare the best practice survey results with the results from the process improvement program. The statistical methods used to analyse the data are based on descriptive and correlational analysis, including both parametric and non-parametric methods appropriate for the types of variables in the hypotheses, and the distribution of data.

Page | 22

Scope According to Chevers and Duggan, the SPI programs that some organizations have adapted to help improve software quality are usually implemented in two generic stages: (1) an introspection stage, which they say is when the capability and effectiveness of existing software production processes are assessed and (2) a remedial stage, in which the results of the assessment are used to prescribe improvement initiatives. Project Plan

Reference Basri, S. B., & O'Connor, R. V. (2010). Organizational Commitment Towards Software Process Improvement. IEEE , 10. Cater-Steel, A. P. (2004). An Evaluation of Software Development Practice and Assessment-Based Process Improvement in Small Software Development Firms. Chevers, D., & Duggan, E. W. (2007). A Modified Capability Framework for Improving Software Production Processes in Jamaican Organizations. Electronic Journal on Information Systems in Developing Countries (EJISDC) , 1 - 18. Dyba, T., & Skogstad, O. (1997). Measurement-based Software Process Improvement. Telektronikk , 73 - 82. El Emam, K., & Jung, H. (2001). The Journal of Systems and Software (ELSEVIER). An empirical evaluation of the ISO/IEC 15504 assessment model , 23 - 41. Haase, V. H. (1996). Software Process Assessment Concepts. Journal of Systems Architecture (ELSEVIER) , 621 - 631. Halvorsen, C. P., & Conradi, R. (n.d.). A Taxonomy to Compare SPI Frameworks. Jarvinen, J. (2000). Measurement Based Continuous Assessment of Software Engineering Processes. VVT Technical Research Center of Finland. Page | 23

Lane, S. (2012, April 25). Software Process Assessment - S-Cube - European Network of Excellence. Retrieved June 5, 2012, from S-Cube: http://www.s-cubenetwork.eu/km/terms/s/software-process-assessment Lonchamp, J. (1993). A structured conceptual and terminological framework for software process engineering. IEEE , 41 -53. Niazi, M., Wilson, D., & Zowghi, D. (2005). A framework for assisting the design of effective softwareprocess improvement implementation strategies. The Journal of Systems and Software , 78, 204 - 222. Niazi, M., Wilson, D., & Zowghi, D. (2003). A maturity model for the implementation of software process improvement: an empirical study. The Journal of Systems and Software . Preuss, D. H. (2006, August 25). Interview: Jim Johnson of the Standish Group. Retrieved May 31, 2012, from InfoQ: http://www.infoq.com/articles/Interview-Johnson-Standish-CHAOS Richardson, I., & von Wangenheim, C. G. (2007). why are Small Software Organizations Different? IEEE , 18 - 22. Schwening, C., & Thiry, M. (2010). A method to define process capability profiles from the characteristics of very small software enterprises. CLEI ELECTRONIC JOURNAL , 13 (1). Shen, B., & Ruan, T. (2008). A Case Study of Software Process Improvement in a Chinese Small Company. Sivashankar, M., Kalpana, A. M., & Ebenezer Jeyakumar, A. (2010). A Framework Approach Using CMMI for SPI to Indian SMES. IEEE , 1 - 5. Staples, M., Niazi, M., Jeffery, R., Abrahams, A., Byatt, P., & Murphy, R. (2007). An exploratory study of why Organizations do not adopt CMMI. Journal of Systems and Sotware , 883 - 895. Zarour, M., Desharnais, J.-M., & Abran, A. (n.d.). A Framework to Compare Software Process Assessment Methods Dedicated to Small and Very Small Organizations. Retrieved May 15, 2012, from DocStoc.com: http://www.docstoc.com/docs/44281165/A-Framework-to-Compare-SoftwareProcess-Assessment-Methods

Page | 24