Computers in Industry 84 (2017) 2538

Contents lists available at ScienceDirect

Computers in Industry
journal homepage: www.elsevier.com/locate/compind

Impact analysis of ERP post-implementation modications: Design,

tool support and evaluation
Minou Parhizkara , Marco Comuzzib,*
a
Department of Computer Science, City University London, Northampton Square, EC1V 0HB London, United Kingdom
b
School of Management Engineering, Ulsan National Institute of Science and Technology (UNIST), 50 UNIST-gil, Ulju-gun, Ulsan 44919, Republic of Korea

A R T I C L E I N F O A B S T R A C T

Article history:
Received 10 June 2016 ERP systems evolve in the post-implementation phase because of changing business requirements. Post-
Received in revised form 25 October 2016 implementation changes are likely to decrease the quality of ERP systems and of the data that they use,
Accepted 8 November 2016 which negatively impacts organisational performance. We propose a framework for impact analysis of
Available online xxx ERP post-implementation modications. Our framework allows mapping dependencies among ERP
system components and, based on these dependencies, automatically assessing the impact of a proposed
Keywords: change on both the design-time structure and run-time landscape of the system through a novel set of
ERP impact metrics. The framework also provides semi-automatic support to safely terminating the running
Impact analysis
process instances affected by change. The framework is evaluated with expert users in two pseudo-real
ERP-post-implementation
ERP system implementations.
ERP change
Enterprise systems evolution 2016 Elsevier B.V. All rights reserved.

1. Introduction by the system. It degrades, for instance, when different functions

ERP systems are packaged software systems supporting most
operations of organisations, such as nance, logistics, human
resource management, and manufacturing [5]. Most medium to
large organisations already have gone through at least one cycle of
ERP software implementation and nd themselves in the so-called
post-implementation phase [7].
ERP systems inevitably change in the post-implementation
phase to align the ERP functionality to changing business
requirements [30,40]. Post-implementation changes may be
driven by internal reasons, e.g., a new and more efcient
warehouse management policy is suggested by the management
and must be reected in the ERP system, or by external factors,
such as suppliers, technology providers or regulatory agencies [15].
For instance, an ERP system may require updates to accommodate
new regulations devised by nancial authorities to counter
emerging money laundering practices.
The literature on ERP post-implementation focuses mainly on
post-implementation CSF (Critical Success Factors), e.g.,
[15,28,33,29]. ERP system and data quality typically are
considered CSF of the post-implementation phase [16,36]. ERP
system quality refers to the quality of the data processing enabled
* corresponding author.
E-mail addresses: minou.parhizkar.1@city.ac.uk (M. Parhizkar),
mcomuzzi@unist.ac.kr (M. Comuzzi).
manipulating the same object in an ERP system are not compatible,
that is, using function A makes a data object unusable for function
B. ERP data quality refers to standard data quality dimensions of
the ERP database, e.g., the accuracy or currency of customer
information [17]. Poorly managed post-implementation modi-
cations reduce the ERP system and data quality, leading to chaotic
systems that are hard to manage [18,40,45]. Therefore, there is a
need to support stakeholders, such as business analysts and
developers, in assessing the impact of ERP post-implementation
change and to guide them during the implementation of proposed
modications. This need is currently only partially answered by the
existing literature on ERP systems modelling and design.
There is a large body of literature, in fact, dealing with advanced
methods and tools to model ERP system components [31,36], align
ERP system functionality to business requirements [27,37] and
optimise ERP systems installations [34,44]. This literature,
however, focuses on the ERP implementation and early usage
phases. In these phases, designers have more freedom in
conguring and possibly customising the ERP system to address
possibly conicting requirements. In the post-implementation
phase, the existing design-time structure of an ERP system and its
currently instantiated business operations is crystallised and may
limit the options available to designers. A change of the process
handling purchase requisitions, for instance, may affect all the
open purchase cases with suppliers. Hence, the impact of this
change on the ERP system should be carefully analysed and,

http://dx.doi.org/10.1016/j.compind.2016.11.003
0166-3615/ 2016 Elsevier B.V. All rights reserved.
26 M. Parhizkar, M. Comuzzi / Computers in Industry 84 (2017) 2538
furthermore, currently running cases affected by a change should
be allowed to terminate safely before a change can be imple-
mented.
To close the research gap identied above, this paper presents a
framework, i.e., a set of methods and a tool, to support
stakeholders in the impact analysis of ERP post-implementation
changes. Our framework allows mapping the dependencies among
design-time and run-time entities of an ERP system and
automatically detecting dependencies among them relevant for
a given change request. Based on the identied dependencies, the
framework supports the user in analysing the impact of a proposed
change through a semi-automated impact analysis mechanism.
The scope and depth of a proposed change can also be analysed
through a novel set of ERP post-implementation modication
impact assessment metrics.
The implemented tool, which shows the feasibility of our
framework, has been evaluated by ERP experts in hands-on
sessions with different change management scenarios to evaluate
usability and t for purpose of our framework in ERP post-
implementation change management.
The paper is organised as follows. Related work is reviewed in
the next section. The design of the framework is presented in
Section 3. Section 4 discusses the tool implementation. The
evaluation with ERP experts is discussed in Section 5 and
conclusions are nally drawn in Section 6.
2. Related work
Business-IT strategic alignment is a long standing issue in the
enterprise systems literature [23]. More recently, process align-
ment has emerged as a research discipline in which organisational
business processes are considered the main vehicle to reach this
alignment [8,38,39,41]. By embedding organisational business
processes, ERP systems can be seen as one of the main technical
artefact to reach organisational process and strategic alignment
[44]. Managing post-implementation changes is a way of
preserving process alignment. However, while the literature on
process alignment mainly analyse ex-post how process alignment
impacts organisational performance [8,39], in this paper we take a
design perspective, aiming at providing solutions to facilitate the
management of post-implementation changes.
Even though research about improving design solutions for ERP
systems is often limited by the lack of access to real ERP
implementations and by the complexity of real world ERP
installations, literature contributions in the areas of ERP concep-
tual modelling and ERP risk management are abundant.
Soffer et al. [36] propose an ERP modelling language for ERP
systems, which considers business processes and business objects
as the main conceptual constituents of an ERP system. Dependen-
cies among processes and objects can also be captured. An
algorithmic approach to align ERP systems capabilities with
enterprise requirements using this modelling language also has
been proposed by the same authors [37]. Both works focus on the
ERP implementation phase and the proposed language does not
model run-time entities and their dependencies with design-time
business processes and objects. A model of ERP systems compris-
ing business processes, functions and business objects is also
adopted by Manuel and AlGhamdi [25] and Park and Kusiak [31] in
the context of designing ERP systems for improved process
integration.
A large stream of research can be found in the area of risk
assessment for ERP maintenance [24,22]. From the point of view
of the target of change, these works consider planned maintenance
activities, such as system upgrades, rather than ad-hoc modica-
tions to support emerging business requirements. Moreover, the
impact of maintenance activities has to be considered limited,
since maintenance packages are normally designed by software
vendors with the aim of minimising the impact on existing
installations.
As far as the impact of enterprise systems change on
operational aspects of an organisation is concerned, de Boer
et al. [11], Krishna et al. [20] and Dam et al. [10] have proposed
approaches to change identication and propagation in enterprise
architectures (EA). While de Boer et al. and Krishna et al.s work
focus on change impact analysis, Dam et al.s work also considers
repairing dependencies in an EA that have been disrupted by
changes. These works consider a methodological approach similar
to the one presented in this paper, identifying mapping of
dependencies among EA elements, taxonomy of possible changes,
and mechanism to assess change propagation as main constitu-
ents. None of them considers dening metrics to quantify the
impact of proposed changes.
EA change impact analysis presents two limitations. Firstly, the
scope of EA is much broader than ERP or, more generally, enterprise
systems. Therefore, EA change impact analysis tend to consider ERP
systems as black boxes in the organisational software landscape,
failing to focus in depth on the internal structure of these systems.
Secondly, EA typically focuses on design-time concerns. This
means that EA change impact analysis fails to address the impact of
change on ongoing business operations, such as long-running
instances of business processes that are not yet completed at the
moment of change.
From the standpoint of engineering change, ERP systems are
perceived as complex software products embedded in a complex
organisational environment. Software systems change impact
analysis is a discipline of software engineering focusing on
understanding, predicting and possibly quantifying the impact
of source code modications [6,16]. While most works in this area
focus on source code change impact analysis, there are also efforts
considering impact analysis on models of the software behaviour,
such as UML diagrams [19,43]. Various techniques have also been
proposed to deal with inconsistencies and evolution in the
requirements engineering phase [20,42]. In the context of ERP
systems, Dor et al. [13] propose a technique for change impact
analysis and propagation analysis based on program slicing. The
proposed technique is tested extensively on the SAP codebase.
Software systems impact analysis focuses on the impact of
proposed changes on the existing source code, while our approach
focuses on the impact of proposed changes at the operational level,
that is, on the conceptual entities dening an ERP system and the
currently running business processes. As such, we argue that
software systems impact analysis should follow impact analysis
based on our framework. That is, once the operational impact of a
proposed change has been analysed, the next step is to analyse the
impact of the change implementation on the existing source code.
Note, in fact, that changes to an ERP system may not necessarily
involve modication of the source code, but they can be
implemented through conguration of existing options or bolt-
on to functionality provided by an external third party [7].
3. Framework design
The design of our framework is divided into the following three
parts:

Design of the ERP dependency meta-model (Section 3.1): this
meta-model allows dening dependencies among the entities of
an ERP system both at the levels of design-time static structure
and run-time operations;

Design of the impact analysis mechanisms (Section 3.2): given a
proposed post-implementation change modication, this mech-
anism allows to automatically identify the ERP components
 M. Parhizkar, M. Comuzzi / Computers in Industry 84 (2017) 2538
Fig. 1. ERP dependencies meta-model.
affected by change and to semi-automatically support the user in
safely terminating the currently running process instances
affected by the change;

Design of the impact assessment metrics (Section 3.3): this
metrics quantify the impact of a proposed change based on the
results of impact analysis.
3.1. ERP dependency meta-model
The meta-model of dependencies among entities constituting
an ERP system is shown in Fig. 1.
The model integrates common perspectives of different ERP
models proposed in the literature. The design-time part of our
meta-model follows from considering a traditional 3-tier archi-
tecture of ERP systems of user interface, application and data
layers. The application layer in the middle is further decomposed
into business processes realised as the orchestration of business
functions. Similar decompositions have been considered for
instance by Manuel and AlGhamdi [25] and Park and Kusiak
[31]. Strnadl [38] also has proposed a model of organisational IT
based on processes, services (functions) and business information
(data objects). Our meta-model allows for hierarchical aggregation
of processes, that is, functions in business processes may refer to
the execution of sub-processes. Such a hierarchical aggregation is
typical of processes as service-based compositions [26].
As far as the run-time is concerned, process instances represent
individual execution of business processes, e.g., the processing of a
specic purchase order. Process instances create instances of
individual functions, which may require the instantiation of
specic data objects, or documents, for their execution.
27
Post-implementation changes concern changes to align the ERP
system to changing business requirements [7,15]. These changes
usually concern the static design structure of an ERP system. Based
on this assumption, the set of possible post-implementation
modications is derived by combining the primitive operations
create, update, delete with the design-time elements of the meta-
model, i.e., the Changeable Entities. Note, in fact, that post-
implementation change management is not concerned with ad-
hoc interventions on running business process instances, but it
rather focuses on permanent changes of the design-time entities
identied by the meta-model.
Besides involving, i.e., having an impact, on other design-time
entities, a proposed modication may also affect one or more
running process instances. For each instance, it is possible to
identify a critical point of execution in respect of a proposed
change. This critical point is a particular point in the process
execution beyond which the execution of a process instance can be
safely terminated after the implementation of the proposed
change. A more precise characterisation of this critical point is
provided in the next section while discussing the impact analysis
mechanisms.
Table 1 presents a set-theoretic representation of the model of
Fig. 1, which is used to support the presentation of the impact
analysis mechanism and the formalisation of the metrics.
In the remainder of the paper we consider a running example of
typical sales and procurement processes integrated by an ERP
system in a traditional manufacturing rm. Fig. 2 shows a BPMN-
based representation of a possible design-time scenario in the
running example. Functions are represented as BPMN activities,
whereas business objects are represented as BPMN data objects. In
the example, the sales process starts with a customer inquiry,
28 M. Parhizkar, M. Comuzzi / Computers in Industry 84 (2017) 2538

Table 1
Notation summary.

Element Notation Related predicates

D E
ERP system ERPSYS PRO; FUN; OBJ; PIN; FIN; DOC
 
Business processes PRO prop ; p 1; . . . ; P
 
Business functions FUN f unf ; f 1; . . . ; F use pro fun 
PRO  FUN
Business objects
OBJ fobjo g; o 1; . . . ; O use fun obj 
FUN  OBJ
 
Active instances of a process prop PINp pinp;i ; i 1; . . . ; I called by  PIN  PIN
PIN [p1 PINp
P instance of
pro  PRO  PIN
 
Active instances of a function f unf FINf f inf ;j ; j 1; . . . ; J instance of fun  PIN  FIN
FIN [f 1 FINf
F
 
Active documents instances of an object objo DOC o doco;k ; k 1; . . . ; K instance of obj  FIN  OBJ
DOC [o1 DOC o
O

Set of changeable entities CHE PRO [ FUN[
OBJ fcheg
Post-implementation change chc cheOLD ; cheNEW ; update
chc cheOLD ; delete
chc cheNEW ; create
Set of post-implementation changes CH fchc g; c 1; . . . ; C involves  CH  CHE
affects  CH  PIN
involveschc ; che; che 2 CHE
affectschc ; pin; pin 2 PIN
Critical point of process instance pinp;i cp : CH  PIN7!FIN
D E
Impact of a change chc an ERP system ERPSYS (at design- IMPchc ; ERPSYS DTI; RTI RTIchc fcheeCHEjinvolveschc ; chegDTIchc fpinePINjaf f ectschc ; ping
timeDTI, and run-timeRTI).

Fig. 2. Running example (design-time scenario).

which may eventually result in a purchase order. If the ordered
goods are not in stock, then they have to be manufactured on
demand. This involves procuring the required raw materials from
supplier(s) (procurement sub-process) and manufacturing the
goods (production sub-process). Once the goods are sent to the
customer, an invoice is issued and the process terminates when the
payment is received.
In the procurement sub-process a request for quotation is rst
issued to the supplier and processed, e.g., compared to previous
quotations, until an agreement on the supplier bid is reached. This
triggers the creation of a purchase order. The process terminates
when the invoice received from the supplier is paid and registered
as an accounting entry. Note that the Supplier PO business object is
required by both the receive goods and verify supplier invoice
functions for consistency verication purpose.
3.2. Impact analysis mechanism
In our framework, given an ERP system ERPSYS and a proposed
change ch, impact analysis is the problem of determining which
predicates involves (ch, che) and affects (ch, pin) evaluate to true,
that is, to dene the design- and run-time impact DTIch and
RTIch, and to support the safe termination of the process
instances affected by change in RTIch.
In the remainder of this paper we focus on the update type of
ERP post-implementation changes, which covers also the create
and delete types of changes. Any update modication, in fact, can be
modelled as the sequential combination of deleting an existing ERP
entity and creating a new entity as a substitute.
 M. Parhizkar, M. Comuzzi / Computers in Industry 84 (2017) 2538
Fig. 3. Impact analysis mechanism.
In our running example, let us consider the following change ch:
D E
ch SupplierPOOLD ; SupplierPONEW ; update ; Safely Complete and Migrate Run-Time Instances phase in Fig. 3).
that is, an update of the Supplier PO business object. This change
may be required by a new directive from a nancial authority to
show tax information and prices including tax calculated based on
the nancial regime of the suppliers country.
Fig. 3 shows a process-based view of impact analysis in our
framework using the BPMN notation. Phases that can be fully
automated are modelled as BPMN service tasks (see the cog icon
in the top left corner of the task symbol), whereas phases that may
require human intervention are modelled as BPMPN user tasks (see
the user icon) In the remainder of this section, we discuss in
detail each phase of the mechanism.
3.2.1. Phase: calculate dependencies
In this phase, the design-time and run-time dependencies
DTIch and RTIch of the proposed change ch are calculated based
on the ERP dependency model.
Regarding design-time dependencies, let us consider the case in
which the change ch involves a data object obj (in our running
example, obj = SupplierPO). In this case, all the functions f unf using
obj as either input or output are affected by the change. Moreover,
all the processes prop using the functions f unf affected and,
recursively, all the processes using the affected processes as
sub-processes at some point are also affected by the change ch. In
other words, identifying the design-time impact DTIch implies to
navigate the relations use pro fun, use fun obj and is sub process
in the meta-model of Fig. 1. In particular, the sub-process relations
allows to take into account transitive dependencies among
processes using affected functions.
In our running example (see Fig. 2), the change ch
SupplierPOOLD ; SupplierPONEW ; update affects the functions Create
Purchase Order, Receive Goods and Verify Supplier Invoice and the
processes procurement and sales.
As far as run-time dependencies are concerned, RTIch
comprises all the currently running, i.e., started but not yet
terminated, instances of the processes identied as part of DTIch.
29
These process instances have to be safely brought to completion
minimising the disruption introduced by the change (see the
3.2.2. Phase: assess design-time compatibility
Not all changes are critical for a given ERP system. Some
changes, in fact, may be perfectly backward compatible with the
entities affected by the change and, therefore, with the existing
ERP system. For example, a compatible update of the Supplier PO
data object in our running example may concern adding a new eld
to capture more details about the supplier identity. The new
Supplier PO is able to replace the old version, since the processes
and functions using it can simply ignore the new eld. In other
words, we say in this case that new version chenew of an entity is
compatible with the old version cheold .
The notion of compatibility of proposed changes with an
existing system is a well-known issue in the literature about
change management [10,13]. Assessing compatibility between
data objects [9], functions or services [32], and business processes
[12] implies to verify the syntactic and semantic similarity
between two versions of the same entity. Syntactic similarity
refers to similarity among the specications of different entities,
e.g., the matching among the signatures of two functions or the
syntactic matching among activity labels in a process models.
Semantic similarity refers to similarity of the meaning of two
entities, e.g., two business functions are semantically similar if
they implement the same function between input and output
parameters, or two process are semantically similar if they produce
the same results for the client.
Compatibility checking in our framework is a semi-automated
activity. While, in fact, some tasks related to compatibility
assessment may be fully automated, e.g., checking the syntactic
similarity between the schema of two data objects or two process
models, other tasks require human intervention, e.g., judging
whether the execution of two functions or processes creates the
same result for the user.
Based on whether a proposed change ch is compatible with the
existing ERP system, our framework involves two different paths
(see Fig. 3). The trivial case is the one of compatibility of the old
30 M. Parhizkar, M. Comuzzi / Computers in Industry 84 (2017) 2538

version of an entity cheold with the new version chenew . This case
involves the implementation of the change and the migration of
affected running instances to the new version of the entity, after
which the old version can be deleted. The other case is the one of
non-compatibility, for which the old version of an entity can be
deleted only after the process instances affected by change have
been brought to a safe state in respect of the proposed change.
3.2.3. Phase: implement change (compatibility and non-compatibility
case)
This task implies that the new entity in a proposed change ch is
implemented in the ERP system, e.g., a new version of the Supplier
PO business object is congured or a new process for customer
orders is designed and deployed in our running example.
Note that, in practice, implementing a new entity in an ERP
system may range from changing a few parameters in the system
conguration, which can be done very quickly, even automatically,
to developing new code or customising the existing code base of
the ERP system (see [7] for a taxonomy of possible ERP change
implementation strategies).
3.2.4. Phase: migrate run-time instances (compatibility case)
If a new entity chenew is fully compatible with the ERP system
ERPSYS, or at least semantically and syntactically similar to the old
version cheold to a certain acceptable degree, then we assume that it
is able to seamlessly substitute the existing entity without any
further specic impact.
Migrating a design-time entity, e.g., a business process, to a new
entity, e.g., a business object, means to make sure that an entity
uses the new entity in any future instance. For example, a
compatible update of the Supplier PO business object in our
running example implies that all running instances of the

Fig. 4. Running example (run-time scenario).

 M. Parhizkar, M. Comuzzi / Computers in Industry 84 (2017) 2538 31

procurement process can start seamlessly using the new version of
the Supplier PO and that all future instances utilise the new Supplier
PO version. A compatible update of the Supplier PO may concern,
for instance, adding a new eld to capture more details about the
supplier identity. Running instances are not likely to be affected by
this change, since they can simply ignore the new eld, while new
instances are able to use the new version of the Supplier PO without
any further issue.
In case of non-compatibility, the critical phase in Fig. 3 is the
one regarding the safe termination of currently affected running
instances:
3.2.5. Safely complete and migrate run-time instances (non-
compatibility case)
Non-compatibility of a change ch with the existing system
ERPSYS implies that currently running instances affected by a
change ch may not be able to terminate their execution because
unable to use a new version chenew of an entity.
In this case, for each affected running instance, we dene a
critical point cp of execution with respect to the proposed change
ch. This critical point cp is dened as the last activity in a process
instance that uses the entity che involved in the change ch.
We say that a process instance affected by change is safe if its
execution is currently beyond the critical point cp determined by
the change ch, while it is not safe (or critical) otherwise. The
purpose of the Safely complete and migrate run-time instances
phase is to identify critical process instances and to bring them to a
safe completed execution. This is achieved by ensuring that the
entity cheOLD is not deleted until all critical process instances have
reached past their respective critical point cp.
Fig. 4 shows a possible run-time scenario in the running
example. There are 4 running instances of the procurement process,
pinp;1 to pinp;4 . The last function in this process using, as either
input or output, the entity Supplier PO involved in the change ch is
the function Verify Supplier Invoice. Therefore, according to the
denition given above, the activity Verify Supplier Invoice is the
critical point for all the instances of the process procurement.
Instances of the process procurement are safe if their execution is
beyond the Verify Supplier Invoice activity, and critical otherwise.
Fig. 4a shows the case of individual process instances, that is,
instances that have not been created because of sub-processes of
other processes. Instance pinp;2 is safe, since its execution has
already passed the verify supplier invoice function. Instance pinp;1 ,
on the contrary, is not safe. Its critical point is then set to the Verify
Supplier Invoice function, which still has to be executed.
Instances pinp;3 and pinp;4 exemplify the case of hierarchical
process integration. For the sake of simplicity, we consider only
one process in the hierarchy in this example, that is, instances of
the procurement process pinp;3 and pinp;4 are generated by
instances of the sales process, i.e., pins;1 and pins;2 , respectively.

Fig. 5. Duplication of calculate critical points.

Fig. 6. Timeline of safely migrating critical instances.

32 M. Parhizkar, M. Comuzzi / Computers in Industry 84 (2017) 2538

In the case of hierarchical process integration (see Fig. 4b), been qualitatively appraised by ERP domain expert during our
determining the safeness of a process instance in respect of change evaluation, it is not the purpose of this paper to propose validated
ch may be more elaborated. Specically, two cases may occur. measurement scales for these metrics. We only aim at showing
In the rst case, the execution of the called process instance, i.e., that our framework enables the denition of useful metrics and we
pinp;3 in Fig. 4b, is already beyond the critical point. In this case, the leave the validation of their measurement scale and their
called instance and all the instances calling it in the hierarchy are application for advanced risk management in ERP post-imple-
safe. mentation to future work.
In the second case, e.g., instance pinp;4 , the execution of the We distinguish three level of impact of a given change ch, i.e.,
called process is not beyond the critical point. In this case, the system, phase, and entity.
instance is not safe and a critical point has to be set for it and, At the entity level the impact of a change ch is broken down into
recursively, for all the instances in the process integration the percentage of processes, function, objects, process instances,
hierarchy that use this instance. In our example, therefore, the function instances or documents that are potentially affected by ch.
instance pinp;4 is not safe. Its critical point is set to the Verify That is, we dene the following metrics at the component level:
Supplier Invoice activity, whereas the critical point of pins;2 , which jfpro 2 PROjinvolvesch; progj
represents in this case the process integration hierarchy, is set to IMPP ch
jPROj
the procurement sub-process.
Note that, in case a calling process instance also utilises the
entity che involved in the change at some point, then its critical jff un 2 FUNjinvolvesch; f ungj
point is set, following the denition, as the last point in the process IMPF ch
jFUNj
using che. Fig. 5 shows a modied version of our running example
referring to this case. The critical point of the calling instance pins is
set to the function Pick and Pack Goods, since it is the last one to
jfobj 2 OBJjinvolvesch; objgj
utilise the Supplier PO object in the process. IMPO ch
jOBJj
As a result, a list of critical affected process instances is
identied with a respective critical point. While the safe process
instances can be immediately migrated to the version of the entity
jfpin 2 PINjaf f ectsch; pingj
chenew involved in the change ch, the critical affected instances can IMPPIN ch
jPINj
be migrated to the new entity chenew only after their execution has
passed their respective critical point. While determining the
safeness or criticality of an affected process instance is done
jff in 2 FINjaf f ectsch; f ingj
automatically, determining whether the execution of a process IMPFIN ch
jFINj
instance has passed the critical point requires human intervention.
Fig. 6 shows a timeline of the safe completion phase in our
running example. Let us refer to t0 as the time instant in which the
jfdoc 2 DOCjaf f ectsch; docgj
assessment of the safeness or criticality of affected process IMPDOC ch
jDOCj
instance is completed. The safe instances pinp;2 , pinp;3 and pins;1
can be immediately migrated to the new entity chenew. For the Metrics at the entity level are aggregated using SAW (Simple
critical process instances pinp;1 , pinp;4 and pins;2 , we have to wait for Additive Weighting) [21]. The rst level of aggregation is
their execution goes beyond the identied critical point before associated to the phase dimension, that is, we distinguish among
migrating. Let us assume that this happens at times tp;1 , tp;4 , ts;2 , impact at design-time (IMPDT ) and run-time (IMPRT ) as follows:
respectively. The maximum of this time instants, i.e., tp;1 in our X
IMPDT ch vl  IMPl ch
example, determines the instant at which the old entity cheold can
l2fP;F;Og
be deleted. After tp;1, in fact, the entity cheold is not needed by any
remaining critical process instances to safely terminate their
execution. X
IMPRT ch vm  IMPm ch
m2fPIN;FIN;OBJ g
3.2.6. Phase: delete old components (compatibility and non-
compatibility cases) where weights vl and vm capture the relative importance of
Finally, once the change has been implemented and all the impact of different classes of entities in determining impact in a
affected running instances are migrated to new versions of the specic phase.
entities, then the old entities can be deleted from the system. Impact assessment metrics can be further aggregated at the
Similarly to the case of implementation, this task may range for system level. The global impact of a proposed change on an ERP
changing a few parameters in the ERP system conguration, to a system can be expressed as:
complex operation affecting the code base of the ERP system.
IMPSYS ch vDT  IMPDT ch vRT  IMP RT ch
3.3. Impact assessment metrics where the weights vDT and vRT capture the relative importance of
a change impact on the design-time and run-time entities in an
Decisions about change implementation are eventually taken ERP system, respectively.
by project managers and executives [14] who may not have specic Values of the weights characterising the impact assessment
knowledge about the internal structure and the runtime landscape metrics can be determined and validated by surveying domain
of an ERP system. As such, information obtained from impact experts or by the decision makers responsible for the impact
analysis should be synthesised into a compact set of impact analysis. The former case applies in the case of standard change
assessment metrics for decision makers. In this regard, our scenarios, which may be relevant to most organisations using an
framework denes a set of hierarchical metrics to quantify the ERP system, whereas the latter method is preferable when the
impact of a proposed change ch. Even though these metrics have
 M. Parhizkar, M. Comuzzi / Computers in Industry 84 (2017) 2538 33

Table 2
Impact metrics values in running example.

Metric value Notes

P
IMP ch 2=3 0:66 The change does not affect the production process

IMPF ch 3=16 0:19 3 functions of the procurement process are affected by change because using Supplier PO (create purchase
order, receive goods, verify supplier invoice)
O
IMP ch 1=14 0:07 The Supplier PO business object is the only one involved in the change

IMPPIN ch 6=6 1:0 All process instances are affected by the change ch
FIN
IMP ch 12=46 0:26 3 function instances in each of the 4 procurement instances are affected by change

IMPDOC ch 3=38 0:08 Only the Supplier PO documents are involved in the change

IMP DT ch 0:7  0:66 0:2  0:19 0:1  0:07 0:507 Decision makers give more importance to impact at the process layer vp 0:7 as compared to functions
and business objects (vf 0:2 and vo 0:1)
IMP RT ch 0:45  1:0 0:45  0:26 0:1  0:08 0:575 Decision makers give more importance to run-time impact of process and function instances
(vPIN vFIN 0:45) as opposed to documents (vDOC 0:1)
IMPSYS ch 0:3  0:507 0:7  0:575 0:5546 Decision makers give more importance to the impact on ongoing operations (vRT 0:7) as opposed to the
structure of the ERP system (vDT 0:3)

Fig. 7. Architecture of software tool.

perception of change impact is driven by organisation specic
concerns, e.g., industry, type of processes or products.
In our running example, the design-time scenario (see Fig. 3)
involves 3 business processes, i.e., sales, procurement and produc-
tion, 14 business objects and 16 functions (not including the
functions of the production process, which is not expanded in
Fig. 3). In the run time scenario (see Fig. 4), there are 6 process
instances, 46 function instances and 38 documents involved,
assuming that each business object is instantiated once by process
instances. The values of the metrics for the change ch
SupplierPOOLD ; SupplierPONEW ; update are calculated in Table 2
4. Tool support
To show the feasibility of the approach, our framework has been
embedded in a software tool, i.e., a decision support system
supporting business analysts in the assessment and implementa-
tion of ERP post-implementation changes.
The architecture of the tool is shown in Fig. 7.
The tool interacts with two types of external software packages,
i.e., the ERP system and external analytic tools. As far as the ERP
system is concerned, as previously remarked, we adopt a non-
invasive approach to impact analysis, whereby we cannot assume
to have control over the internal implementation of the ERP
system. The requirements imposed by our framework on the ERP
system are (i) a reective interface to provide information about
design-time components, e.g., list of business processes, functions
and data objects and (ii) a management interface allowing to query
the status of execution of currently running process instances in
the ERP system. The former facilitates the denition of the ERP
dependency model, whereas the latter is required by the impact
analysis mechanism (see Section 3) to determine the state of
execution of running instances affected by change when deter-
mining the safeness of currently affected process instances.
External business intelligence tools can access information about
impact analysis reports and metrics to run more advanced impact
analytics.
The tool is implemented using a state of the art model driven
application development platform for enterprise system integra-
tion.1 The tool has been instantiated for the pseudo-real ERP
systems SAP ERP 6.0 for GBI Inc. and Microsoft Dynamics NAV R2
2013 for Cronus Inc. Both companies have been developed for years
by the respective software vendor for professional training
purposes. Although ctitious, their ERP instantiations contain a
sufcient level of detail, in terms for instance of number and type
of completed and open orders, products, clients, suppliers, and
processes, to simulate the operations of a complex medium to large
organisation at a given instant in time. GBI Inc. is a typical large-
sized manufacturing company, assembling products from spare
parts, whereas Cronus Inc. is a medium-sized company trading a
variety of products. Since we did not have access to the ERP APIs,
the mapping of the two case onto our tool has been performed
manually. In both instantiations, we consider only sequential
business processes, that is, processes constituted by a sequence of
1
www.mendix.com.
34 M. Parhizkar, M. Comuzzi / Computers in Industry 84 (2017) 2538

Fig. 8. Snapshots of software tool.

function invocations. This simplifying assumption facilitates the
calculation of critical points during impact analysis and has
allowed quicker understanding of the framework functioning by
practitioners during the evaluation.
Fig. 8 shows snapshots of the implemented tool in the case of
SAP ERP 6.0 GBI Inc. Fig. 8a shows the interface listing all post-
implementation modications implemented so far and the input
interface for a new proposed change. Fig. 8b shows the results of
applying the impact analysis mechanisms, i.e., the list of entities
affected (run-time entities are shown on the right-hand side) and
values of assessment metrics (left-hand side).
5. Evaluation
We provide two different types of evaluation of our tool. First, to
show its feasibility in practice, we evaluate to what extent our tool
 M. Parhizkar, M. Comuzzi / Computers in Industry 84 (2017) 2538 35

Table 3
SAP coverage analysis.

Enterprise SAP Features Manageable by impact Comments

Level analysis framework?
Organizational Authorization/Roles and X Out of the scope of analysis
Layer Responsibilities
Organizational Level (Client, X Out of the scope of analysis
Company Code, Plant, etc.)
Presentation Graphical User Interface X Out of the scope of analysis.
Layer Transaction Data Header/Item
Business Logic Module U Framework can identify which modules are affected by change as a result of modications
Layer of functions constituting a module.
(design- Business Processes U Framework can identify which business processes are affected by change and can handle
time) Sub-process Integration U sub-process integration.
Business Function/Transaction U Framework can identify which business transactions are affected by change.
Business object Master Data U Framework can identify the impact of change of master data as business objects
Layer Organisational Data X Out of the scope of analysis
(design- Transaction Data X Out of the scope of analysis (post-implementation changes do not deal with ad-hoc
time) changes to running process instances, which manipulate transaction data)
Attributes of Data Item X Framework cannot differentiate the change impact of individual attributes of a business
object.
Operation Active Business Process U Framework can identify which process instances are affected by change.
Layer Active Business Transaction U Business transactions affected by change can be identied by the framework based on the
(run-time) process instances affected by change.
Active Transaction Data U Transaction data affected by change can be identied by the framework based on the
process instances affected by change.

Table 4
MS Dynamics NAV coverage analysis.

Enterprise MS Dynamics NAV features Manageable by Comments

Level impact analysis
framework?
Organizational Authorization/Roles and Responsibilities X Out of the scope of analysis
Layer
Presentation Graphical User Interface X Out of the scope of analysis
Layer Fact Box/Sorting/Navigation/Field
Business Logic Department U Framework can identify which modules are affected by
Layer change as a result of modications of functions
(design- belonging to a Department.
time) Business Processes U Framework can identify which business processes are
Business Process Integration U affected by change and can handle sub-process
integration.
Business Function/Action U Framework can identify which business actions are
affected by change.
Business Master Data: U Framework can identify the impact of change on master
Object Layer data as business objects
(design- - Chart of Account
time) - Cards (Vendor, Customer, Item)
- Orders (sales, Production, Purchase)

Organisational Data X Out of the Scope of analysis

Journals X Framework cannot specify what journal will be
(General, Sales, Purchase, Item) impacted by modication of data.
Attribute of Card X Framework cannot differentiate the change impact of
individual attributes of a business object (or card).
Operation Active Business Process U Framework can identify which process instances are
Layer affected by change.
(run-time) Active Business Transaction U Business transactions affected by change can be
identied by the framework based on the process
instances affected by change.
Active Transaction Data U Transaction data affected by change can be identied by
the framework based on the process instances affected
by change.
Posted Documents U Framework can identify posted documents affected by
change, since posted is only a particular state of a
business object.

is able to handle standard functionality provided by commercial
ERP packages. This evaluation activity has involved a panel of 7 ERP
professionals with average 10.7 years experience with different
types of ERP systems (SAP, Oracle/JD Edwards, and Microsoft
Dynamics) in different industries (banking, higher education, ERP
consulting). Then, we present the results of an experiment with
ERP users to assess the t for purpose and usability of our tool in
practice. This evaluation activity has involved the panel of 7 ERP
professional and 12 master students in Business Systems Design
36 M. Parhizkar, M. Comuzzi / Computers in Industry 84 (2017) 2538

who recently completed an introductory course on business
process management and enterprise systems.
As far as the coverage analysis for SAP and MS Dynamics NAV is
concerned, for each system, we rst have identied a set of
standard features. Then, we have evaluated whether our tool is
able to handle changes of the identied features. This process has
been executed following a customised Delphi method, whereby
the list of ERP package features and the matching with the
functionality in our framework have been identied rst by the
authors and then discussed with the panel of ERP professionals to
achieve substantial agreement.
Tables 3 and 4 show the results of the coverage analysis. The
coverage of the two systems, i.e. SAP and MS Dynamics NAV,
appear to be aligned. More specically, our framework is able to
deal with change in the core features of the application logic of an
ERP system, that is, business processes, related functions and
business objects. The framework is not able to address change in
the presentation layer or at the level or roles and responsibilities in

Table 5
Results of experiment. (For interpretation of the references to color in the table, the reader is referred to the web version of this article.)
 M. Parhizkar, M. Comuzzi / Computers in Industry 84 (2017) 2538
the organisational layer. We argue that change in the presentation
layer is always present when the core functionality of an ERP
system. It is related to issues of usability and acceptance [15], but it
does not affect the running operations of an organisation. Impact of
change on organisational governance, i.e., roles and responsibili-
ties, is subject of our ongoing work.
As far as the business object layer is concerned, for both
systems, our framework covers post-implementation changes of
master data, while organisational and transactional data are not
covered. Master data are the ones more likely to evolve in the post-
implementation phase, as they capture the concepts relevant for an
organisations daily operations, such as customers, suppliers,
materials, and products. Organisational and transactional data are
not covered because they are of out of the scope of our analysis. In
particular, we argue that the organisational structure is likely to be
stable during the lifetime of an ERP system or, at least, it changes
much less frequently than master data. Transactional data are out
of scope since, as discussed before, impact analysis is not focused
on ad-hoc changes to running processes and their data, but rather
on changes to the static structure of the ERP system.
The experiment with ERP professionals and students has been
executed in individual working sessions. In each session we rst
have presented the functionality of the tool using a running
example of change scenarios, then asked the participant to run two
change scenarios independently, and then collected feedback from
the participant using a structured questionnaire. The items of this
questionnaire relate to typical dimensions of technology accep-
tance and success evaluation, i.e., system usefulness, ease of use,
ease of learning and satisfaction with the system [1,24]. All items
are evaluated on a 5 point Likert scale from strongly disagree (1) to
strongly agree (5).
While we selected ERP professionals because they represent the
natural type of users of our tool in the real world, the student group
has been selected because less likely to suffer from attribution bias
during the evaluation. We argue, in fact, that the personal
experience of ERP professionals in managing post-implementation
may have biased their understanding and utilisation of our
framework. The students, on the other hand, possess a sufcient
knowledge background to understand the issues associated to ERP
post-implementation change, but they do not have direct
experience with these in the real world that may bias their
experience with our tool.
Table 5 shows the results of the evaluation. The feedback
obtained from both groups of participants is overall positive. The
few unsatisfactory results are found in the evaluation of ease of
use. Because of its proof of concept nature, our prototype was not
developed considering usability as primary non-functional
requirements. Therefore, lower scores for the ease of use criteria
have been expected.
It is remarkable that the usefulness evaluation (in particular
items C4C6) and the satisfaction with the tool (items C17C20)
are generally high for both students and professionals. This signals
that, on the one hand, the tool addresses a relevant problem in
practice and, on the other hand, it provides an effective solution to
it.
The student group has provided generally higher evaluations.
This can be explained by their relative lack of experience with
utilising ERP systems in the real world, which has prevented them
to realise the nuances of the issues that can be encountered in ERP
post-implementation change management in real world scenarios.
In this regard, the expert participants stressed that the tool could
be extended with more advanced functionality to assess the
organisational impact of ERP changes and the nancial impact of
proposed changes based on different possible implementation
strategies.
37
6. Conclusions
This paper has presented a framework, i.e., a set of methods and
a tool, to support stakeholders such as business analysts in
assessing the impact of post-implementation changes to an ERP
system. The framework denes a meta-model of the dependencies
among ERP entities, a mechanism to assess the impact of proposed
changes and a set of metrics to synthetically quantify the scope and
depth of ERP changes. The implemented tool has been instantiated
in the case of two commercial ERP software packages, i.e., SAP ERP
6.0 and Microsoft Dynamics NAV 2013. Based on the instantiations,
our framework is able to assess the impact of changes of the main
entities of an ERP system, such as business processes and functions,
but it still fails to capture change at the level of roles and
responsibilities in the organisational. The evaluation of the tool
with ERP experts has revealed that our framework provides an
effective solution to an issue that is relevant for practice.
As suggested by the participants to the evaluation, our
framework can be extended by considering impact of post-
implementation changes at the organisational level and the
nancial implications of a change. Change at the organisational
level, in particular, entails analysing the impact of change on the
ERP responsibility and authorisation policies, which are closely
knitted to the issue of security of an ERP system [35].
From a methodological point of view, while in this work we took
a model-driven approach to impact analysis, future work may
investigate ERP post-implementation impact analysis from an
evidence-based perspective. Rather than modelling explicitly ERP
entities and their dependencies, these may be learned from the
execution logs of ERP systems. In this way, impact analysis can be
based on actual dependencies learnt from process execution data.
Based on execution frequency and other possible information in
execution logs, such as resource or cost information, impact
analysis can be also characterised more in depth by dening other
measures of dependencies among, such as strength or cost.
Acknowledgment
This work was supported by the 0000 Research Fund (Project
number 1.160044.01) of UNIST (Ulsan National Institute of Science
and Technology).
References
[1] Y.M. Abgaz, Change Impact Analysis for Evolving Ontology-based Content
Management, Dublin City University, 2013 (PhD Dissertation).
[2] D. Aloini, R. Dulmin, V. Mininno, Risk management in ERP project
introduction: review of the literature, Inf. Manage. 44 (6) (2007) 547567.
[3] S. Ahmadi, E. Papageorgiou, C.H. Yeh, R. Martin, Managing readiness-relevant
activities for the organizational dimension of ERP implementation, Comp. Ind.
68 (2015) 89104.
[4] R.C. Beatty, C.D. Williams, ERP II: best practices for successfully implementing
an ERP upgrade, Commun. ACM 49 (3) (2006) 105109.
[5] P.J. Bhattacharya, P.B. Seddon, R. Scheepers, Enabling strategic transformations
with enterprise systems: beyond operational efciency, Proc. Int. Conf. Inf.
Syst. 2010 (2010) 5567.
[6] S.A. Bohner, Extending software change impact analysis into cots components,
Proceedings of the 2002 Software Engineering Workshop, in Conjunction with
27th Annual NASA Goddard/IEEE Conference (2002) 175182.
[7] L. Brehm, A. Heinzl, M.L. Markus, Tailoring ERP systems: a spectrum of choices
and their implications, Proceedings of the 34th Annual Hawaii International
Conference on System Sciences, January (2001) 19.
[8] A. Cataldo, R.J. McQueen, J. Hardings, Comparing strategic IT alignment versus
process IT alignment in SMEs, J. Res. Pract. Inf. Technol. 44 (1) (2012) 4355.
[9] Roberto Da Silva, Raquel Stasiu, Viviane Moreira Orengo, Carlos A. Heuser,
Measuring quality of similarity functions in approximate data matching, J.
Infometrics 1 (1) (2007) 3546.
[10] H.K. Dam, L.-S. L, A. Ghose, Managing changes in the enterprise architecture
modelling context, Enterp. Inf. Syst. 10 (6) (2016) 131.
[11] F.S. de Boer, M.M. Bonsangue, L.P.J. Groenewegen, A.W. Stam, L. van der Torre,
Change impact analysis of enterprise architectures, Proceedings of the 2005
38 M. Parhizkar, M. Comuzzi / Computers in Industry 84 (2017) 2538
IEEE International Conference on Information Reuse and Integration (2005)
177181.
[12] R. Dijkman, M. Dumas, B. Van Dongen, R. Krik, J. Mendling, Similarity of
business process models: metrics and evaluation, Inf. Syst. 36 (2) (2011) 498
516.
[13] N. Dor, T. Lev-Ami, S. Litvak, M. Sagiv, D. Weiss, Customization change impact
analysis for ERP professionals via program slicing, Proceedings of the 2008
International Symposium on Software Testing and Analysis (2008) 97108.
[14] W.W. Eckerson, Performance Dashboards: Measuring, Monitoring, and
Managing Your Business, John Wiley & Sons, 2010.
[15] Y.M. Ha, H.J. Ahn, Factors affecting the performance of enterprise resource
planning (ERP) systems in the post-implementation stage, Behav. Inf. Technol.
33 (10) (2014) 10651081.
[16] A.E. Hassan, R.C. Holt, Predicting change propagation in software systems,
Proceedings of the 20th IEEE International Conference on Software
Maintenance, September (2004) 284293.
[17] A. Haug, J. Stentoft Arlbjrn, A. Pedersen, A classication model of ERP system
data quality, Ind. Manage. Data Syst. 109 (8) (2009) 10531068.
[18] P. Inedo, B. Rapp, A. Inedo, K. Sundberg, Relationships among ERP post-
implementation success constructs: an analysis at the organizational level,
Comp. Hum. Behav. 26 (5) (2010) 11361148.
[19] I. Ivkovic, K. Kontogiannis, Tracing evolution changes of software artifacts
through model synchronization, Proceedings of the 20th IEEE International
Conference on Software Maintenance (2004) 252261.
[20] A. Krishna, S.A. Vilkomir, A.K. Ghose, Consistency preserving co-evolution of
formal specications and agent-oriented conceptual models, Inf. Software
Technol. 51 (2) (2009) 478496.
[21] C.L. Hwang, K. Yoon, Multiple Attribute Decision Making: Methods and
Applications a State-of-the-art Survey, Springer Science & Business Media,
2012.
[22] C. Lopez, J.L. Salmeron, Dynamic risks modelling in ERP maintenance projects
with FCM, Inf. Sci. 256 (2014) 2545.
[23] J. Luftman, R. Papp, T. Brier, Enablers and inhibitors of business-IT alignment,
Commun. AIS 1 (1999) (Article 11).
[24] Arnold M. Lund, Measuring Usability with the USE Questionnaire12. 2001
Retrieved on May 1st 2016 from: http://garyperlman.com/quest/quest.cgi?
form=USE.
[25] P.D. Manuel, J. AlGhamdi, A data-centric design for n-tier architecture? Inf. Sci.
150 (3) (2003) 195206.
[26] N. Milanovic, M. Malek, Current solutions for web service composition, IEEE
Internet Comput. 8 (6) (2004) 5159.
[27] P.A. Millet, Toward a model-driven, alignment-oriented ERP methodology,
Comp. Ind. 64 (4) (2013) 402411.
[28] M. Moalagh, A.Z. Ravasan, Developing a practical framework for assessing ERP
post-implementation success using fuzzy analytic network process, Int. J.
Prod. Res. 51 (4) (2013) 12361257.
[29] C.S.P. Ng, G.G. Gable, T. Chan, An ERP-client benet-oriented maintenance
taxonomy, J. Systems and Software 64 (2) (2002) 87109.
[30] T. Oseni, M.M. Rahim, S.P. Smith, S. Foster, An initial empirical evaluation of the
inuence of ERP post-implementation modications on business process
optimisation, Proceedings of the 2014 European Conference on Information
Systems (2014) (Retrieved from http://aisel.aisnet.org/ecis2014/proceedings/
track06/4).
[31] K. Park, A. Kusiak, Enterprise resource planning (ERP) operations support
system for maintaining process integration, Int. J. Prod. Res. 43 (19) (2005)
39593982.
[32] P. Plebani, B. Pernici, URBE: web service retrieval based on similarity
evaluation, IEEE Trans. Knowl. Data Eng. 21 (11) (2009) 16291642.
[33] J. Ram, D. Corkindale, M.L. Wu, Implementation critical success factors (CSFs)
for ERP: do they contribute to implementation success and post-
implementation performance? Int. J. Prod. Econ. 144 (1) (2013) 157174.
[34] P. Samaranayake, Business process integration, automation, and optimization
in ERP, Bus. Process Manage. J. 15 (4) (2009) 504526.
[35] W. She, B. Thuraisingham, Security for enterprise resource planning systems,
Inf. Syst. Secur. 16 (3) (2007) 152163.
[36] P. Soffer, B. Golany, D. Dori, ERP modeling: a comprehensive approach, Inf. Syst.
28 (6) (2003) 673690.
[37] P. Soffer, B. Golany, D. Dori, Aligning an ERP system with enterprise
requirements: an object-process based approach, Comp. Ind. 56 (6) (2005)
639662.
[38] C.F. Strnadl, Aligning business and it: the process-driven architecture model,
Inf. Syst. Manage. 23 (4) (2006) 6777.
[39] P.P. Tallon, A process-oriented perspective on the alignment of information
technology and business strategy, J. Manage. Inf. Syst. 24 (3) (2014) 227268.
[40] M. Themistocleous, Z. Irani, R.M. O'Keefe, R. Paul, ERP problems and
application integration issues: an empirical survey, Proceedings of the 34th
Annual Hawaii International Conference on System Sciences (2001) 1018.
[41] R. Yu-Yuan Hung, T. Chung, B. Ya-Hui Lien, Organizational process alignment
and dynamic capabilities in high-tech industry, Total Qual. Manage. Bus.
Excellence 18 (9) (2007) 10231034.
[42] A. Van Lamsweerde, R. Darimont, E. Letier, Managing conicts in goal-driven
requirements engineering, IEEE Tran. Software Eng. 24 (11) (1998) 908926.
[43] E. Visser, J. Warmer, A. Van Deursen, A. Van Deursen, Model-driven software
evolution: a research agenda, Proceedings of the International Workshop on
Model-Driven Software Evolution in Conjunction with the 11th European
Conference on Software Maintenance and Reengineering (2007) 715.
[44] J.H. Worley, K.A. Chatha, R.H. Weston, O. Aguirre, B. Grabot, Implementation
and optimisation of ERP systems: a better integration of processes, roles,
knowledge and user competencies, Comp. Ind. 56 (6) (2005) 620638.
[45] Y. Zhu, Y. Li, W. Wang, J. Chen, What leads to post-implementation success of
ERP? An empirical study of the Chinese retail industry, Int. J. Inf. Manage. 30 (3)
(2010) 265276.