Vous êtes sur la page 1sur 10

19th Australian Conference on Software Engineering

Construction by Configuration: Challenges for Software Engineering


Research and Practice

Ian Sommerville
School of Computer Science, University of St Andrews, St Andrews, Scotland
ifs@cs.st-andrews.ac.uk

Abstract configuring appropriate modules. Alternatively, a


generic application (COTS) designed for a specific
The past ten years have seen a radical shift in business application area (such as library management) may be
application software development. Rather than bought and adapted to the customer’s business.
developing software from scratch using a conventional We know that this approach to application
programming language, the majority of commercial development has had a major impact over the past 10
software is now developed through reuse – the years. While specific figures are hard to come by, a
adaptation and configuration of existing software Gartner report suggested that 95% of the top 2000 US
systems to meet specific organizational requirements. companies use ERP systems. In a study published in
The most widespread form of reuse is through the use 2000, Everdingen et al. [1] predicted that more than
of generic systems, such as ERP and COTS systems, 50% of European midsize companies would use ERP
that are configured to meet specific organizational systems by 2000 and the figure is almost certainly
requirements. In this paper, I discuss the implications much higher now. Penetration is significantly less in
of software construction by configuration (CbC) for smaller organizations because of the high initial costs
software engineering. Based on our experience with of these systems but, in these organizations, COTS
systems for medical records and university systems have had a significant effect. My own
administration, I highlight some of the issues and experience suggests that the preferred procurement
problems that can arise in ‘construction by strategy for information systems in most organizations
configuration’. I discuss problems that arise in CbC is now to ‘buy and configure’ rather than ‘specify and
projects and identify a number of challenges for program’.
research and practice to improve this approach to Academic research has reflected this change in a
software engineering. limited way. There is an international conference series
on COTS-based software (ICCBSS) but the majority of
1. Introduction software engineering research remains focused on
‘specify and construct’ development. For example, in
Software engineering in industry has changed the proceedings of the 2007 International Conference
dramatically since the mid-1990s. Prompted by factors on Software Engineering (ICSE), there was a single
such as the need to upgrade legacy systems to cope paper on COTS-based systems. In ASWEC 2007, there
with the Year 2000 problem, business pressures for were 2 papers on COTS-based development.
faster system deployment and dissatisfaction with I believe that this is an area where there is a
existing business processes, many companies and pressing need for research. The initial promise of rapid
government organizations have switched their primary system deployment for a relatively low cost is often
systems development strategy from ‘specify and unfulfilled. There are horror stories of problems with
construct’ to ‘buy and configure’. ERP systems development. It seems that delays in
Rather than defining the requirements for a new delivery and problems in use are as common as with
systems and then developing that system using a systems developed using a ‘specify and construct’
conventional programming language such as Java or approach. Deployed systems often fail to deliver the
C#, systems are developed by buying an off-the-shelf cost reductions expected and users are frequently
generic application system and configuring this system unhappy with the system performance and
to create a specific organizational application. This functionality.
may involve buying a general-purpose ERP system and

1530-0803/08 $25.00 © 2008 IEEE 3


DOI 10.1109/ASWEC.2008.75
In this paper, I will focus on the problems of different backgrounds. These systems may be
‘programming’ COTS systems where ‘programming’ constructed using either modular generic systems or
means configuring generic modules using the support COTS assemblies. They may be organised as ‘thin
provided by the system. I call this ‘construction by client’ systems where all processing takes place on
configuration’ (CbC). I will discuss some of the servers and access to applications is through a web
problems that we observed during long-term, detailed browser. Alternatively, a ‘thick client’ approach may
studies of configuration and deployment of 2 large- be used where applications are installed on client
scale information systems, as well less detailed computers and communicate with a server-based
analyses of several other systems. I propose a research database. Each of these architectures results in
agenda to extend our understanding of and support for different types of problem.
this mode of systems development. I draw on examples Thin-client systems, where all functionality is
from a patient information system for mental health implemented as services running on a server, simplifies
care installed in a large hospital and on a university- system management. The system servers are
wide student administration system being deployed in responsible for all processing, with functionality
a large UK university with more than 20, 000 students. accessed through a web browser. This approach avoids
the problems of user re-configuration discussed below
2. Configurable systems and reduces installation costs. However, it brings with
it its own set of problems:
While all systems require some configuration when 1. Significant investment in the server infrastructure
they are deployed, I am concerned here with software may be required to cope with periods of peak
that is designed to be generic but which can be tailored loading and to provide redundancy in the event of
and adapted to specific organizational requirements. server failure. This is a particular problem when
The configurable generic systems that are intended to periods of peak loading are relatively short but
be the basis for business-specific applications are of critical.
three kinds: 2. There has to be close cooperation between staff
1. Single PC-based applications with built-in responsible for configuration and staff responsible
programming capability. The best-known example for infrastructure management. Large organizations
of such an application is probably MS Excel. often have heterogeneous computing infrastructures
Starting with basic spreadsheet functionality, Excel with different operating systems and browsers in
can be extensively reprogrammed to create specific use in different parts of the organization. Changes
applications. Applications that can be extended to the infrastructure (browers and operating
using plug-in capabilities (e.g. the Firefox browser) systems) may lead to problems and
can also be thought of as configurable systems. incompatibilities.
2. Modular generic systems. These are systems, such 3. Security problems with browsers are well-known.
as ERP systems, where the supplier has a range of Where the information maintained is critical (e.g.
functional modules (manufacturing, CRM, etc.) and medical records), the risks involved in using a
the purchaser decides which modules they will brower-based systems may be considered to be too
include in their system. These individual modules high.
are then integrated and configured. 4. The system is inflexible. The server-based system
3. COTS assemblies. In this case, the application is cannot easily be tailored to groups of users within
constructed by integrating several off-the-shelf an organization who have non-standard needs, such
systems. For example, we have studied a bank’s e- as emergency care or maternity in a hospital.
procurement system, which was created by The thick-client approach avoids some of these
integrating an invoicing system and a web-based e- problems. In this approach, the server is solely
commerce system. responsible for data management and a special-purpose
In all cases, the developed system may also make application which is responsible for local processing is
use of other desktop applications such as MS Word or installed on each client. The benefits of this approach
MS Excel to provide additional functionality. are that server load and network traffic are
significantly reduced and that off-line operation is
2.1 System architecture possible when servers or network access are not
available. This approach, of course, has its own
Our work on configurable systems has been mostly problems:
concerned with organizational information systems, 1. Higher management overhead as the applications
with a shared database and multiple users from deployed on each client have to be updated to
reflect changing requirements.

4
2. Excessive flexibility allowing individuals and Business process configuration is particularly
groups to reconfigure the system locally thus important. All ERP systems and COTS embed their
increasing the costs and problems of system own generic model of business processes. For these
evolution. systems to work effectively, it is essential that business
3. Security problems as sensitive data is cached on processes are adapted to conform to this model [2].
local machines with possible access to that data Otherwise, it is extremely difficult to make effective
without conformance to the system security use of these systems.
procedures. While the requirement to change process might be
Of course, the architecture of the system will seen as a barrier to the adoption of these systems, the
largely depend on design decisions made by the opportunity for change is actually welcomed by many
developers of the COTS system that is being organizations. They see this is a way of standardizing
configured. If a different architecture is required, processes across the organization. Control of the
serious problems can arise. We observed this in the process is removed from individual units and retained
university administrative system where, to force by central management. I discuss some of the
process conformance, it was required that a thin-client difficulties that this can cause later in the paper.
architecture should be used.
However, there were no COTS systems available 3. The CbC process
with this architecture so, after a decision was made on
a system, the system provider was paid to implement a The CbC process obviously varies depending on the
web-based interface. After these changes were made type of system being developed and the facilities
and the browser interface implemented, it was offered by the COTS system that is being configured.
discovered that the performance of the delivered In the systems that we have observed, the following
system was so poor that critical organizational process activities have been common:
deadlines could not be met. 1. System selection. A system is procured based on
high-level business rather than operational
3. System configuration requirements.
2. Requirements analysis. Existing business process
The deployment phase for all software systems stakeholders are consulted and their ‘requirements’
inevitably involves some system configuration. This are elicited. However, these are seen as providing
may simply involve providing some data about the advice to the development team rather than real
operating environment or its users. However, for system requirements.
generic COTS systems, this configuration is much 3. Business process redesign. The business processes
more extensive and may involve adapting the system to are redesigned to fit the processes assumed by the
reflect some or all of the following: COTS or ERP system.
1. The specific needs of a customer (e.g. a 4. System configuration. The system is configured to
hospital). reflect the new business processes and, as far as
2. The requirements of a group of users within the possible, the user requirements.
organization (e.g. a maternity unit). 5. System testing. The configured system is tested to
3. The required interactions with other systems in some extent although, as I discuss below, this is a
its environment. particularly problematic stage of the process.
4. The characteristics of the platform on which the 6. Deployment and evolution. The system is deployed,
system operates. put into use and evolved rapidly to reflect user
To meet these needs, many different configuration problems and difficulties.
activities may be involved in adapting a COTS system In the system developments that we observed, there
to its environment. These include: were several process issues that led to problems with
• Selecting the required system modules the deployed system:
• Defining a data model 1. A 2-stage requirements process. In the first stage,
• Defining business rules there was minimal user involvement and
• Defining workflows management defined the requirements that were
• Defining external interactions used to select the COTS product to be purchased.
• Defining the user interface The 2nd stage took place after the procurement
decision had been made and the requirements were
• Defining the reports to be produced
constrained by the facilities available in the
• Setting platform parameters
purchased system.
• Re-defining business processes

5
2. Co-design of software and process. The ability of danger to themselves and others. However, there are
rapidly reconfigure the system meant there was strict legal safeguards associated with such detention
scope for user involvement in the development of and often time limits as to when assessments are
both the process and the software. Therefore, the carried out. It was observed that the purchased system
requirements evolved during the development and was being used effectively to support this process
the system at any one time reflected the elsewhere and this was an important factor in the
requirements of the most influential group of procurement decision.
system users. However, the systems that were demonstrated were
3. Ad hoc testing. There were no systematic in use in England, which has a different legal system
approaches adopted to testing, partly because the from Scotland where the system was to be deployed. In
system rarely ‘failed’. Each configuration setting particular, there was a more recent mental Health Act
led to a working system. However, the lack of in Scotland with quite different processes for assessing
detailed requirements made it very difficult to test patients for involuntary hospitalisation. It proved to be
the system for suitability before deployment. impossible to reconfigure the system to reflect these
4. Lack of supporting processes. Conventional processes. Thus, the system did not meet one of its
software engineering includes supporting processes principal requirements – the existing manual systems
such as configuration and quality management. It had to be retained and there were no cost savings from
was found to be impossible to import these introducing the new system.
processes into a CbC development process. In reality, it is not always possible to ‘do it the
Morisio et al. [3] found comparable problems in system’s way’. Existing business processes have
studies of COTS-based development processes in evolved for good reasons to ensure that work is done
NASA. effectively and efficiently. In complex organizations,
The 2-stage requirements process led to the such as a hospital, where different parts of the
selection of a system based on the high-level organization work in different ways (e.g. maternity and
requirements of managers who did not have sufficient emergency medicine), standardized processes imposed
knowledge of current processes in their organization. by management who do not understand operational
While a systematic approach was used for software systems use may simply not be good enough.
selection, the systems that were ultimately procured The co-design of requirements, software and
were found to be lacking in a number of important process is, in many respects, a good thing. Users were
respects when a more detailed understanding of the actively involved in the process and their comments
requirements emerged. were fed back quickly to the development team.
These problems largely arose because the system However, the process did not properly recognise the
providers had ‘hard-wired’ assumptions into their important differences between processes in different
generic software, with limited ability to reconfigure parts of the organization and, critically, that there is no
these assumptions. The buyers of the system did not real benefit for users in spending time communicating
question these assumptions and only discovered with other users in different departments. Of course,
problems when the system was being configured. this is not just a problem for CbC – it is also a serious
An example of an area where this can arise is in problem for agile development processes.
assumptions about the legal and regulatory framework During the system development, the major problem
in which the system will be used. Many systems have that this caused was the impossibility of maintaining a
to conform to regulatory requirements (e.g. in the US stable system specification – it changed daily. When
the Sarbanes-Oxley accounting rules). Typically, a this was combined with the fact that the system users
system will first be developed and sold in a home and other stakeholders did not communicate
market and the regulations that apply to that market effectively, it was practically impossible for anyone to
will be implemented in the generic system. However, understand the actually delivery schedule for the
when these systems are sold in another country with features in the system.
different laws and regulations, it may be difficult to This co-design activity did not stop when the initial
reconfigure the system to conform to these regulations. version of the system was deployed. It continued
When one important reason for procuring a system throughout the ‘bedding in’ period where the new
is to reduce the costs of compliance, this can be processes and the new system was brought into
particularly problematic. The patient information operational use. Here, we observed a new problem.
system we observed was used to manage the The stakeholder group with the greatest political
information about patients with mental health influence drove the changes to the system, without due
problems. It is sometimes necessary to detain such regard for the requirements of other stakeholders. For
patients against their will because they constitute a example, in the hospital system, this manifested itself

6
as changes to the user interface influenced by the likely to be involved in the configuration process. This
hospital management and senior doctors. Nurses and was the case in the systems that we observed where the
junior administrators using the systems were not system owners set up an internal project team to
consulted. develop and deploy the systems.
Testing was observed to be a serious problem. We discovered that there were three principal
System failures did not, by and large, manifest sources of difficulty faced by CbC developers:
themselves in ways that were obvious to developers • Understanding the configuration options
because the lack of a detailed specification meant that • Understanding the configuration semantics
they did not really know what the system was • Understanding how a system is configured
supposed to do. Systems rarely failed in the sense that Most configurable systems offer a range of different
they crashed or produced clearly incorrect output. configuration options with, sometimes, subtle and
Rather, the failures could only be detected by users difficult to understand interactions between these
who understood their local processes and who could options. Sometimes, these options are obscure and
identify where system support was inadequate. In poorly documented and there is rarely information
addition, regression testing was found to be impossible available about how different options may interact.
as the COTS systems were not designed to run For confidentiality reasons, we cannot discuss the
automated test suites. specifics of the applications that we have studied but
While a period of user testing was scheduled before the same problems also arise in PC software. For
the system was deployed, schedule slippage meant that example, my version of MS Word offers at least 8
the time for this was compressed. This meant that, different ways to configure the system:
rather than spending a few hours per week on testing, • Preferences screen
the developers expected the users to be available for • Customisation screen
several days before deployment. As the users were • Organiser screen
very busy people, this was practically impossible so
• Definition of templates
there was very little testing carried out. The systems
• Definition of styles
were deployed so that testing and usage were one and
• Definition of macros
the same thing.
By and large, good change and configuration • Inclusion of add-ins (e.g. Endnote)
management practice has evolved in development Even with 20 years experience of using Word in
environments where an important requirement was many different versions, I would find it hard to explain
source code control. Existing source code control what each of these does and how they interact.
systems cannot be used alongside the COTS systems to Once a developer has discovered the different ways
manage the evolving configuration data, because the to configure an application, he or she is then faced with
configuration is done using specialized support built the problem of deciding which options to use.
into the system. The configuration data cannot be However, the semantics of the configuration options
separated from this system itself. This made it are rarely explicitly defined and developers have to
practically impossible to revert to previous system rely on (often minimal) documentation and examples
versions when problems were discovered. In general, to try to understand them. The configuration options
quality management was a serious issue as there was may reflect the underlying semantics of the system
no shared perception of what was meant by a ‘high being configured and so developers have to infer these
quality system’. Good practice, such as inspections and to understand the configuration possibilities.
reviews, were not carried out partly because of the When you are developing a software system, it is
problems of configuration visibility that I discuss in the useful to be able to predict the consequences of
next section of the paper. changes to that system. Unlike a conventional
programming language where the programmers
understanding of the language semantics is used to
4. Configuration problems decide how to implement changes, changes to
configurations are typically experimental. The meaning
Large-scale ERP systems are so complex to
of the configuration is defined by the underlying
configure that development is usually the responsibility
system, which is a black-box. Over time, gurus emerge
of the system supplier. User-configuration is
who can make things happen with a system but cannot
practically impossible because the learning time for the
explain why these happen and cannot effectively
system is so long. However, for smaller-scale ERP transmit their knowledge to others.
developments (e.g. using open source ERP [4]) or for The problem of understanding the current
COTS-based development, the system owners are more configuration of a system is a critical one for the

7
development and maintenance of a system. We have However, things are rarely so simple. In almost 20
never yet seen any configurable system where there is years of studies with end-user organizations, we have
a simple way of displaying the current configuration. never seen a process that has not been adapted to suit
This means that building an overall picture of a the local circumstances. The individuals involved in
configuration is very difficult. This situation is made enacting processes always modify these processes to
worse by the fact that configurations cannot be make them more suitable for the way in which they
maintained in a version management system. There is work. If they are forced to work with a standardized
therefore no single description of a configuration system, they will simply add on activities outside that
available. system. For example, in a system that generated PDF
This causes problems where changes to the system reports, we observed users making use of an MS Word
are proposed and have to be costed and implemented. add-in that converted PDF to Word because they
Assessing the impact of a change is generally very needed to add additional information to the report.
difficult, even when system design documentation and However, if they have a configurable system,
the code of the system programs is available. It is even enterprising users will discover how to do the
more difficult in configurable systems for two reasons: configuration and will make local changes to suit their
1. The co-design process where system requirements own requirements. We observed this in the patient
are developed alongside the implementation means information system that was deployed across a number
there is no specification of the system. This, of of geographically dispersed clinics. The doctors in
course, is also a problem with some agile charge of these clinics had evolved their own way of
development approaches but there, at least, the working and of keeping patient records and they asked
code is available to define the system. In local IT staff to modify the deployed system to reflect
configurable systems, there is no single description this. Of course, this caused problems when the
of either what should be implemented and what has information from different clinics was integrated to
actually been implemented. create management reports.
2. It is not enough simply to understand the It might be argued that this was a management
configuration to assess the impact of a change. It is failure and that system managers should have retained
also necessary to understand the underlying COTS control over the changes to the system. However, there
system. During development, where consultants are real practical difficulties here because of the
from the system supplier are available, this is less distribution of power and influence in an organization.
of a problem. However, after the system is handed If a senior manager (or in this case, a senior doctor)
over, gaining access to system information may be asks a relatively junior member of the IT staff to make
much more difficult. changes to a system, it is very difficult for them to
The consequences of this are that the change costs refuse to do so, whatever the organizational policy.
may be much higher than expected and may take much The possibility of relatively simple re-configuration
longer than expected. For example, in the patient can also lead to situations where configuration
information systems that we observed, a small change decisions are challenged and changed to reflect
to the user interface that was originally predicted to changing political power and influence in an
take 2 or 3 days ended up taking 6 weeks to organization. The patient information system that we
implement. observed was originally chosen and deployed because
it had the capabilities to produce a set of reports that
4.1 Process configuration were required by the healthcare management board.
These reported patient statistics under a number of
As I have discussed, it is generally accepted that different headings which the managers assumed were
ERP and COTS system developments can only be consistent with clinical record keeping.
successful if the business processes are configured to However, the initial trial deployment of the system
match the process model that is assumed by the caused major problems because the categories under
software. Typically, therefore, the deployment of a which the clinicians recorded patient information were
new system involves process change. This is often quite different from those used by the management
welcomed by the organizational management who see reports. They then insisted that the system be
it as an opportunity to improve and control of business reconfigured so that clinical categories where used and
processes and to ensure these are standardized across that additional software was developed to convert these
the organization. New processes may allow more to the management categories. This proved to be
effective use of new IT systems. impossible as the clinicians themselves did not have a
standard method of recording patient information.

8
Some compromise was therefore sought where This lack of involvement does, of course, cause
some management information was added to normal credibility problems as it makes it much more difficult
clinical recording. The amount of this information to explain the value of software engineering research.
varied over time as different individuals were involved It also means that those involved in the CbC process
in the development of the system and as political find it impossible to relate their work to good software
power and influence changed. What should have been engineering practice. Methods and tools are reinvented
a 6-week process of configuring the user interface for and mistakes are repeated.
the system, ended up taking almost a year before a I believe that there are intellectually challenging
stable interface was agreed. software engineering problems in this mode of
The adoption of a thin-client approach where all software engineering that the SE research community
interaction is with a central system makes end-user re- should tackle. These relate to both the COTS systems
configuration impossible. However, it does not change that are being configured and the configuration
the political realities where senior managers or engineering processes.
professionals argue that (sometimes rightly) that their
requirements are different from the rest of the 5.1 Design for configuration
organization and they need special support. In such
situations, parts of the organization may simply play In the configurable systems that we have observed,
lip service to the new system but will actually develop it seems that little attention has been paid to the
their own parallel system to carry out their work. problem of design for configuration. What I mean by
We observed this in the university administrative this is that the designers of COTS systems should
system where the system assumed that all departments recognize that configuring a system is time-consuming
sent standard letters to students. In fact, departments and expensive and that the generic system should be
which had problems in meeting their student designed to simplify the configuration process and to
recruitment targets (such as engineering and computer reduce the probability of configuration errors. As
science) sent personalized letters because they believed researchers, we need to explore the notion of
this would attract students to them. As personalization configurability and to establish design principles and
was impossible in the university system, they simply guidelines for developers of configurable systems.
maintained a parallel system to meet their real There are at least 3 areas where research is required:
requirements. • Design principles for configurability. What
principles should be applied when designing the
5. Research challenges configuration options in a system?
• Configuration visibility. What do users require
Construction by configuration is a well-established when trying to understand a configuration and how
development method for organizational systems. is configuration information best presented to
However, as I have discussed, this approach can be as them?
problematic as other approaches to software • Configuration description. How can we move away
development yet it has received very little attention from low-level configuration to configurations that
from the software engineering research community. are a better reflection of business policies?
This has occurred for two reasons: Good software design principles (such as
1. Many software engineering researchers are simply information hiding, low coupling, etc.) have been
unaware of the scale of the change that has taken established over many years but it is not clear how
place. The changes to development practice have these apply to configurable systems. I believe that there
been driven by business rather than technical is scope for research examining how these fundamental
considerations and have had very little publicity in design principles can be applied in configurable
technical literature such as the Communications of systems and reflected in the configuration support
the ACM, IEEE Software, etc. tools. Examples of design principles that might apply
2. Application systems are difficult to study in a to configurable systems are:
university environment. These systems are 1. Manageability. In every system we have observed,
expensive to procure and can only be used with it has been impossible to get an overview of the
business knowledge which is lacking in a configuration then drill down through this overview
university. However, open source middleware is to configuration details. In fact, it has usually been
widely available (e.g. Apache Axis) and this often impossible to generate a complete view of the
has the same configuration problems. whole configuration. I believe that COTS systems
should be structured so that the configuration is

9
maintained as a separate, structured entity that can commands from this. Of course, this relies on
be managed as a unit. As I discuss below, facilities configurations being separate from the application
should be provided to allow the configuration to be being configured
viewed and navigated by configuration engineers.
2. Minimisation. A serious problem with configurable 5.2 Configuration engineering
systems is that there are often several ways to
implement a configuration. It is difficult for As well as research focusing on design for
designers to understand which of these is best and configuration, I believe that there is also scope for
how they interact. The principle of minimization is research concerned with ‘configuration engineering’.
that the number of configuration options that allow This should focus on the process differences between
the same feature to be implemented should be code-based development and construction by
minimized. Ideally, there should be only one way configuration. Possible research areas include:
to implement a feature of a system. • Knowledge management
3. Separation. Problems sometimes arise because • Configuration modelling
configurable systems may not separate the • System testing
configuration of the system for platform • Supporting processes
characteristics from the configuration of Most of the problems that arose when configuring
functionality? There is a need to identify distinct and deploying configurable systems were a
types of configuration and examine how these consequence of poor knowledge management. In all
should be supported. cases, someone in the organization knew of and
4. Independence. Following the general design understood the problem but this knowledge was simply
principle of low coupling, the interactions between not transmitted to managers, the development team
the different parts of the configuration should be members or other users. All to often, we heard remarks
minimized. Required interactions should be explicit like ‘I knew this would be a problem’ and ‘we have
and clearly documented. that problem too’. Users, in particular, were very poor
As I have already discussed, configurable systems at sharing their knowledge.
do not usually make it possible for developers to view I believe that the most effective way to tackle this
the configuration ‘as a whole’. Rather, they must use problem is to develop more effective systems for
the different configuration options in turn to examine knowledge management that makes it easier to capture,
what has been configured. Relationships that exist classify and share knowledge about assumptions, the
between different parts of the configuration are not organization and the system itself. Such a system could
usually made explicit in these views. also help integrate supporting processes such as change
There is a need for tools that allow engineers to see and quality management.
the ‘configuration state’ of a system and to explore The modelling of software systems to remove
dependencies across that state. Software visualization inessential detail is accepted as good software
research has focused on viewing program entities and engineering practice [6]. While system configuration
their relationships [5]. Perhaps this can be developed makes use of business process models, we do not have
and extended to cover configurations and the methods and techniques for modelling other aspects of
relationships between configurations and the the configuration. Starting with a model of the generic
underlying system components? system to be configured, is it possible to add detail to
Configuration of systems is an error-prone and ad- this to define the specific configuration. The extent to
hoc process because the configuration is defined in which this is possible with closed soruce systems is an
terms of the underlying system entities rather than the open question but it is an area that I believe is worthy
organizational requirements and policies that must be of further research.
supported by the system. We can see an example of System testing during CbC is a major problem
this in security configuration where high-level security because tests are to validate the system rather than
policies have to be translated into detailed commands verify it against a defined specification. This validation
to manage access control lists, etc. is difficult because of the new processes that are
One area of possible research, which is linked to the introduced alongside the system. Problems may be
issue of configuration modelling, discussed in the process rather than software related.
following section, is to explore whether or not higher- There is scope here, perhaps to explore test-first
level policy languages can be used as a basis for development as practiced in some agile methods [6]
defining configurations. That is, the organization sets and to investigate whether the requirements
out what it wants to do in some policy language and an engineering processes should be oriented towards the
automated translator creates the configuration

10
definition of tests that can be applied by a development are unusable in most industrial settings (e.g. formal
team. Other research issues in testing include the methods).
problems of providing test coverage and automated Construction by configuration has immense
regression testing tools. economic significance. As a research area, it offers
Finally, the supporting processes of configuration, new challenges and opportunities to the software
change and quality management are different. We need engineering research community. I believe that it is
to understand what are the quality attributes that should now time to embrace these challenges and to
apply to a configuration. User-led change will remain demonstrate the relevance of software engineering
an issue for many configurable systems and there is a research to modern software development.
need for change and configuration management
support that is user-accessible and that can be 10. References
integrated with more general knowledge management
support. [1] Everdingen, Y.V., Hillegersberg, J.V., Waarts, E. ‘ERP
adoption by European midsize companies’, Communication
6. Conclusions of the ACM, Vol. 43 No.4, April 2000, pp. 27-31.

[2] Law, C.C.H. and Ngai, E.W.T. ‘ERP systems adoption:


Construction by configuration is now perhaps the An exploratory study of the organizational factors and
most widely used development technique for large- impacts of ERP success’. Information & Management.
scale organizational information systems. However, Volume 44, Issue 4, June 2007, pp 418-432.
problems with this approach are well-known. Delays in
delivery, unmet organizational expectations and user [3] Morisio M., Seaman C.B., Basili V.R., Parr, A.T., Kraft
dissatisfaction are common. However, academic S.E., Condon S.E. ‘COTS-based software development:
research into these problems and possible solutions is Processes and open issues’. J. of Systems and Software,
relatively limited and few researchers have explored Volume 61, Number 3, 1 April 2002 , pp. 189-199.
how good software engineering practices can be [4] Serrano, N.; Sarriei, J.M. ‘Open source software ERPs: a
adapted to support construction by configuration. new alternative for an old need’, IEEE
To research effectively in this area, we need to Software, Volume 23, Issue 3, May-June 2006 pp 94 – 97.
come out of the laboratory environment and interact
directly with the businesses and organizations [5] Gracanin, D., Matkovic, K., and Eltoweissy, M.
developing these systems. Approaches such as ‘Software visualization’, Innovations in Systems and
ethnographic studies [7] and action research [8] are Software Engineering: A NASA Journal, Volume 1, 2005, pp
required to develop an understanding of the problems 221-230.
and issues faced by CbC developers and to investigate
[5] ‘Model driven engineering’. IEEE Computer, Special
the real utility of new tools and technologies. Issue, Volume 39, Issue 2, February 2006.
Of course, there are very real difficulties in
interacting with industry and studying systems as they [6] Beck, K. Extreme Programming Explained: Embrace
are, rather than as researchers might like them to be. Change, 2nd Edition, 2004. Addison-Wesley.
Industrial interaction takes a lot of time and
confidentiality issues may limit the possibilities for [7] Hughes, J.A. and Shapiro, D. ‘From Ethnographic record
research dissemination. to system design: some experiences in the field’. Computer
It is always tempting to use more manageable Supported Collaborative Work Volume 1 Issue 3, June 1993,
laboratory examples and to test new techniques on pp 123-147.
simplified systems. However, as we have seen from 30
years of software engineering research, this approach is [8] Avison, D.F., Myers, M. D. and Peter Axel Nielson,
often unsuccessful. The complex reality of systems in P.A.. ‘Action Research’, Communications of the ACM, Vol.
42, No. 1, January 1999, pp. 94-97.
use makes it impossible to use techniques that may be
technically superior but which, for practical reasons,

11
Ian Sommerville is Professor of Computer Science at St Andrews University in Scotland.
He has worked for many years with social scientists and was amongst the first computer
scientists to explore how information from ethnographic studies of work could be used to
inform system specification and design. This has led to his current research in the
dependability of socio-technical systems where he is working on modelling
responsibilities across organisations, organisational memory and coping with systems
failure. Ian is the author of a widely used textbook on software engineering which was
first published in 1982 and which is now in its 8th edition.

12