UNIVERSITY OF WATERLOO

Masters of Accounting

CONTINUOUS AUDITING FRAMEWORKS AND IMPLEMENTATION

ACC 626
Professor Malik Datardina
University of Waterloo
Waterloo, Ontario

Prepared by:
Caroline SC Ziebart
June 24, 2012

Table of Contents

1.0 Introduction ................................................................................................................................. 1

2.0 Continuous Auditing Frameworks and Models ........................................................................... 1
2.1 Difference in Audit Approach ......................................................................................... 1
2.2 Benchmark Continuous Auditing Framework ................................................................. 2
2.3 Risk Indicator Continuous Assurance Framework ......................................................... 3
2.4 Independence Issues ..................................................................................................... 3
3.0 Integration of IT Systems ............................................................................................................ 4
3.1 IT Architecture ................................................................................................................ 4
3.2 Integrating the CA System into the Current Environment .............................................. 5
4.0 Practical Implementation of Continuous Auditing ....................................................................... 6
4.1 Siemens ......................................................................................................................... 6
4.2 AT&T .............................................................................................................................. 7
4.3 HSP ................................................................................................................................ 8
4.4 Third Party Solutions ...................................................................................................... 8
4.5 The Market for Continuous Auditing .............................................................................. 9
5.0 Future Initiatives and Conclusion .............................................................................................. 10
Appendix I Traditional Audit Approach .............................................................................................. 12
Appendix II Continuous Auditing Integrated Audit Approach ............................................................ 13
Appendix III Chan and Vasarhelyi 7-Step Continuous Auditing Paradigm ....................................... 14
Appendix IV Risk Indicator Continuous Assurance (RICA) in an Audit .......................................... 15
Appendix V IIPCA Model ................................................................................................................. 16
Appendix VI Third Party Solutions to Continuous Auditing ............................................................... 17
Appendix VII Continuous Auditing Implementation Stages ...............................................................18
Works Cited ...................................................................................................................................... 20
Annotated Bibliography ..................................................................................................................... 21

1.0 Introduction
The terms continuous auditing and continuous monitoring are often misused. Continuous auditing (CA)
is a tool used by auditors, and defined by the ISACA Standards Board (ISACA) as a methodology used
by auditors, typically assisted by technology, to perform audit procedures and issue assurance on a
continuous basis (e.g., weekly, monthly). On the other hand, continuous monitoring is a management
tool, and defined by ISACA as a process put in place by management, usually automated, to determine
on a recurring and repetitive basis (e.g., weekly, monthly) if activities are in compliance with policies and
1

procedures implemented by management This is more of a quality assurance function.

This paper focuses on continuous auditing frameworks and implementation of a CA system.

2.0 Continuous Auditing Frameworks and Models

Continuous auditing (CA) systems can be used by businesses in several different ways, such as in
governance, risk management and compliance (GRC) departments and used to assist external auditors.
These systems are expected to be increasing prevalent in the future because they offer the ability to more
closely monitor and more frequently audit the higher risk areas of the business. The following section will
explore two different frameworks; the first has been informally called the benchmark continuous auditing
framework for the purposes of this paper, and the second is the risk indicator continuous assurance
approach. Then, IT architecture alternatives are discussed which leads to a discussion of integrating a CA
system with the business current IT environment.

2.1 Difference in Audit Approach

Regardless of the framework being used, reliance on a continuous auditing system fundamentally
changes the audit approach. This will be further discussed throughout this paper, but it is worthwhile to
examine a few key differences upfront. In a traditional audit, risk assessment is done on an annual basis
(see Appendix I). This risk assessment is used to determine the audit approach, which drives the nature,
timing and extent of fieldwork performed. The results of the prior years audit, including the nature and
quantity of errors or exceptions found, largely determine the assessment of risks for the current years
audit. This changes when a continuous auditing system is implemented (see Appendix II). Since the audit
process is now continuous, risk assessment must also be an ongoing process. This is beneficial because
key risks are continuously monitored and proactive approaches can be taken to mitigate these risks when
necessary. Because risks could change at any given time, and will change significantly during the life of
the company, audit approaches need to be continuously updated to reflect new or changing risks.
Periodic fieldwork is still performed during a continuous audit, especially in an external audit, since
substantive procedures are required even when relying heavily on controls (CAS 330 The auditors

(Jill Joseph Daigle, 2008)

responses to assessed risk). However, the nature of fieldwork becomes more analytical in nature rather
than performing tests of details. Refer to the next section for more detail.

2.2 Benchmark Continuous Auditing Framework

Continuous auditing introduces an entirely new audit framework. Rather than periodic audits being
performed on past transactions, continuous auditing takes a proactive approach and tests the data and/or
systems more regularly. There are four key steps that need to be addressed when developing a
continuous auditing system:
1. Automate current audit procedures - Continuous auditing does not need to be applied to the
company as a whole. Rather, key business processes are identified and considered for CA. A key
consideration is the access and availability of input data used in the identified business process.
2. Benchmarks are developed Benchmarks need to be developed to determine what is considered
a pass and a fail of an audit procedure. These are developed based on current internal
controls in place, and developed using historical data, estimation techniques, association with
other input data, or clustering techniques. The benchmarks developed should be tested against
historical data. It is important to run these tests on a separate server, and not to use live data.
Note that benchmarks do not have to be based on key risk indicators in this approach, but can be
based on the process as a whole and largely leveraged from the current controls in place. As in
any audit, risks still need to be identified since audit testing is performed with these key risks in
mind, but the specific tests themselves are not necessarily mapped to specific risks. This is in
contrast to the key risk indicator approach discussed below.
3. Exception report clearing Data analytics are performed in the CA system by running the
benchmark against the actual transactional data. A fail against the benchmark is considered an
internal control violation and an exception. For each test run, an exception report is produced,
which is the output of the CA system. A drawback of CA systems is the risk of false alarms, since
there could be several exception reports generated by the system, many of which may not be true
errors or exceptions. Significant manual labour may be required to clear the exception reports.
4. Audit reporting If there are no exceptions found, then a clean audit opinion can be given. Any
exceptions found need to be manually addressed by the auditor and clearing of exceptions
requires significant judgment.

In order to help managers understand CA systems, a 7-step paradigm was developed by Chan and
Vasarhelyi (see Appendix III). A key difference is that the audit model becomes proactive under CA,
rather than the traditional reactive model. This is because rather than auditing historical transactions
many months after they take place, continuous auditing theoretically provides real-time assurance. In
practice, this may take place minutes, hours or days after the transaction. However, the misstatements
2

This reference applies to this entire section. (David Y Chan, 2011)

are detected before they make it to the financial statements, making a CA system proactive. The role of
the auditor also changes drastically. Internal auditors are required to primarily exercise judgment in
clearing exception reports, whereas external auditors time will be spent on certifying the continuous
auditing system (see Section 2.4 Independence Issues). Audit procedures in a CA system tend to be
analytical and control-based, as opposed to the test of details frequently done in traditional audits. The
entire population can be tested on a continual basis in a CA system. Therefore, audit reporting is done
more frequently, and if there are no exception reports or significant areas requiring judgment, audit
reporting can theoretically become continuous as well.

2.3 Risk Indicator Continuous Assurance Framework

An alternative to the approach described above is to think of continuous auditing in terms of risk
indicators. Risk Indicator Continuous Assurance (RICA) uses risk indicators to measure the control
effectiveness within a set of activities or operations. This is used in identifying audit risks, which is part of
the planning stage of the audit process (see Appendix IV). First, the risk indicators (RI) need to be
identified. This is the risk that is mitigated by the control (for example, user access to systems). Then, the
risk indicator metric must be computed. The RI metric quantifies the effectiveness of the control (for
example, [number of obsolete user accounts]/[total number of accounts per system]). Thresholds must be
developed in order to assess the magnitude of the RI metric and evaluate the overall control effectiveness
(for example, >3% of obsolete user accounts means that the control is ineffective).

In order for the risk indicator approach to be effective, auditors must have direct access to internal
systems. Coming up with appropriate RIs and RI metrics is the most difficult task, and it is important that
these metrics are comprehensive enough so material omissions are not made, yet simple enough to
interpret. It is also important for the thresholds and RI metrics to be reviewed and updated regularly. This
means that risk assessment in the audit changes from being done during the planning phase to being an
ongoing process.

2.4 Independence Issues

Several scholarly journals cite continuous auditing (CA) independence issues as a major hurdle to
overcome. Since continuous monitoring is a management tool (refer to section 1.0 Introduction for the
definition), impairment of independence is not an issue. The ISACA Standards Board (ISACA) provides
a six-step approach to assist companies in solving CA independent issues:
1. Understand why the issue is an independence issue based on the specific facts and
circumstances. Audit procedures are frequently developed by external auditors, but the business
is heavily involved in implementing a continuous audit system, meaning the audit procedures
could become transparent to the company. Also, external auditors are heavily involved in
3

This reference applies to this entire section. (Dale Johnstone, 2009)

developing a CA system which puts the auditors at risk of acting in an advisory role. There are
also certain instances where external auditors develop code for the CA system, but management
wants to use that code for the CM system.
2. The ethical issues are then identified.
3. Stakeholders are identified.
4. Identify ethical principles to adhere to. These can come from Generally Accepted Auditing
Standards (GAAS), ISACA, IIA, Institute of Management Accountants (IMA), American
Institute of Certified Public Accountants (AICPA) or IESBA.
5. Come up with possible solutions and consequences.
6. Use judgment to determine the best possible alternative.
The exact solution will vary greatly depending on the companys specific CA system. However, signing a
policy clearly stating the role of external auditors, internal auditors and management in development of
the system is a good first step. Agreements such as these help solve issues surrounding the ownership of
code.

Since continuous auditing systems are developed by both the external auditor and internal management,
there is considerable uncertainty with regards to legal liability if the systems fail to detect material
5

misstatements. This is an issue that has not yet been resolved and therefore requires further research
and guidance from regulatory bodies.

3.0 Integration of IT systems

When adopting a continuous auditing system, it is vitally important to consider both the architecture of the
new CA system and how the system will be integrated into the current IT environment.

3.1 IT Architecture

There are two different IT architecture alternatives for continuous auditing systems. The first is Embedded
Audit Models (EAM), which are built into the system to provide assurance over certain types of activities.
Usually, external software is purchased and then it is customized with continuous auditing procedures
such as matching certain key documents or comparing transactions against an audit threshold. EAM
ghosting is similar to EAM, but rather than running the continuous auditing functions on the live business
systems, it is run on a copy of the systems. This mitigates the risk that live data will be affected by the
audit process. However, the downside to this approach is that it requires significantly more computing
power and storage, since copies of the system must be made and retained.

(Jill Joseph Daigle, 2008)

(John R Khun Jr, 2010)
6
This reference applies to this entire section. (Dale Johnstone, 2009)
5

Monitoring Control Layer (MCL) systems are those which the continuous auditing system is external to
the business processing system. This contrasts with EAM systems in which the CA system is embedded
into the business software. Data is transmitted to the CA system at pre-specified internals, and audit
procedures are run with that data.

There are several limitations to these approaches, however. A lot of processing power is demanded by
CA systems, which could be costly to acquire and maintain. With regards to EAM, purchased software
must be customized and EAM must be built into the enterprise resource planning (ERP) systems.
Therefore, EAM must be tested at the ERP system level as well, which could be problematic for large
companies operating many ERP systems.

3.2 Integrating the CA System into the Current Environment

So far, this paper has discussed continuous auditing systems in isolation, but it is important to realize that
this is only the first step in developing a full CA system. After the CA model is built, it must be integrated
with the current business model. Then, it is crucial that practitioners receive the proper training so they
know how to use the CA system to its full potential.

Traditionally, the internal audit function supported the management, operations and information systems
processes within a company. In continuous auditing systems, these business process systems must be
integrated with the electronic audit evidence. This brings up the issue of who controls the data, since it is
imperative that auditors remain independent and the business cannot control, change or tamper with the
electronic audit evidence in any way.

Researchers developed a full power continuous auditing system called intra/inter process continuous
auditing (IIPCA) to help integrate the CA system into the current IT environment (see Appendix V). This
is the first model to jointly consider CM, internal audit and electronic audit evidence. It is critical that CA
models include internal control testing and testing transactions. Any business rules or policies must also
be followed. This testing should be performed before the CA system is integrated with other existing
systems.

The inter-process auditing function ensures that data goes through each required process in the business
and is not lost along the way. At each process level, the data is ticketed with the phase of business
process and date. Inter-process auditing can be used in operations management as a way to measure
efficiency, since the time between data input and outflow is tracked. The intra-process auditing function
ensures that all tasks within a specific process are completed. Again, the data is ticketed with the activity
performed on or with the data and time. The electronic audit evidence function defines all processes
7

This reference applies to this entire section. (Munir Majdalawieh, 2012)

surrounding data analysis, retention and disposition. Not only do the tickets on the data need to be
stored, but any alarms triggered and exception reports need to be kept as well. Since there are effectively
two levels of audit (process level and transactional level), there is the potential for huge amounts of data
to be generated. Businesses need to ensure the appropriate databases are in place to store this data.

4.0 Practical Implementation of Continuous Auditing

Several companies are making the shift towards continuous auditing and/or continuous monitoring
systems. In practice, companies implementing a current auditing system should examine what has been
done at other companies, especially those in a similar industry, to gain an understanding of what
benchmarks are typically used. The challenges faced by other companies during the implementation
process also serve as a key learning tool for future adoption of continuous auditing systems.

4.1 Siemens
A continuous monitoring of business process controls (CMBPC) system was implemented at Siemens.
This is a critical pilot implementation that has been used as a guideline for implementing continuous
auditing systems in other companies.
The internal audit group documents audit action sheets (AAS), which were created to assess any
8

configurable process controls that could be automated. Types of controls are: a) verify by testing a
specific control for existence, correctness and functioning of the control, b) verify by ensuring a prohibited
behaviour cannot happen, c) verify any automatic control settings in the ERP system. Siemens used the
9

monitoring and control layer approach as opposed to EAM. Refer to section 3.1 of this paper for a
description of MCL and EAM. Since the continuous auditing system is external to the rest of the systems
used in the company, there is less intra-department co-ordination required. Another reason why MCL was
chosen over EAM is that the physical and logical access separation from the rest of the entity means that
the auditing system will be less susceptible to manipulation by employees.

10

First, it was decided what AASs should be automated. Both automation of the control and assessment of
the effectiveness of the control elements were considered. The challenge was not only making the audit
procedures machine readable but also machine understandable. This required the internal audit team
11

to re-engineer some audit processes. The degree of effectiveness of each control was assessed by the
system based on a 0-4 rating.

(Michael Alles G. B., 2006)

(Michael Alles A. K., 2008)
10
(Michael Alles G. B., 2006)
11
(Michael Alles A. K., 2008)
9

Each control has several control elements related to it. For example, password controls contains control
elements relating to password length, expiry date, log in attempts. An overall rating based is assigned to
the control based on the aggregate rating of related control elements. This can either be automated (ex
using a weighted average of the control elements) or assessed by the auditors judgment. Some kind of a
control exception hierarchy is also required. It is inefficient and unrealistic to change the entire audit
process, since large companies like Siemens have legacy systems and understaffed audit departments.
Some re-engineering is unavoidable due to the need to separate out formalizable and automatable
controls.

12

The continuous auditing system collected data and ran tests every 10 seconds. However, the pilot
implementation at Siemens used a simple MS Access database, which worked for the pilot testing done,
but could not handle the large volume of data being continuously monitored and retained as support in an
entity-wide application of the system. Throughout the implementation process, it was found that the
volume of data retained can be greatly reduced by only retaining the data if the system found control
exceptions. It is critical to have a good database that retains these exceptions.

13

An interface where the

auditor can see the 0-4 score achieved by the control and related controls is also required. It was also
particularly difficult to implement compensating controls in the system, and ensuring that alarm floods did
not overwhelm the human auditor. In hindsight, a parallel alarm classification hierarchy should have been
implemented to assess the materiality of the control exceptions. The Siemens team learned that a clear
change management plan needed to be developed before such large changes are made to the auditing
and IT environment.

14

4.2 AT&T Bell Laboratories

15

In the late 1980s, AT&T became one of the first companies to adopt a continuous auditing-type system
was AT&T. A Continuous Process Audit Methodology (CPAM) was developed to implement Continuous
Process Audit Systems (CPAS). This was a challenge since at the time because corporations generally
used a main database system with other databases connected to it. This means that auditors had to audit
both the system and the reconciliation between the systems, which is not the most efficient way to audit
since these types of procedures do not address certain key issues such as the timeliness of addressing
errors (since audits are only done once a year). Data is only as reliable as the system that generated it so
real-time assurance over system controls was determined to be useful.

CPAS uses two different ways to obtain continuous assurance: 1) data flowing through the system is
continuously tested based on auditor-defined rules, and 2) data is tested indirectly by looking at specific
12

(Michael Alles G. B., 2006)

(Michael Alles G. B., 2006)
14
(Michael Alles A. K., 2008)
15
This reference applies to this entire section. (Miklos A Vasarhelyi, 1991)
13

occurrences of errors or individual results. Different types of data can also feed into the system. Data can
be pulled from either the standard application reports, the raw data that feeds to these application reports,
or direct monitoring data. Direct monitoring data is the output data from a monitoring system. Any errors
or exceptions found trigger an alarm, similar to that at Siemens as described in section 4.1. CPAS uses a
hierarchy of alarms to determine the priority of exception report clearing. This makes the audit process far
more efficient since critical errors are examined first.

The team thought it was important for audit work should be done in a separate and independent location,
which is why a MCL system was used (refer to section 3.1 for description). The auditors work is broken
down into two phases. The first is the start-up phase, where the auditor works with the business to gain
an understanding of the control environment, which helps the auditor develop an audit plan and
appropriate procedures. The auditor developed a series of flow charts that were communicated to
computer engineers, IT staff, and management. After this is completed, the auditor can actually use the
system to perform audit work.

The system developed at AT&T primarily used a series of metrics, analysis and alarms. First, metrics are
developed to measure the expected outcome. This is similar to the benchmark continuous auditing
framework described in section 2.2 of this paper. Analysis is broken down into three sub-categories:
functional/natural flow, logical/key data interaction and empirical/observational. The algebraic structure of
the code that tests the control against metric is determined. However, the metric for a specific control
could vary in different situations or across time, so these contingencies that determine the numeric value
of the output also need to be factored into the model. Lastly, certain industry- or company-wide rules of
thumb can be used as a benchmark as well. Whenever the metric or benchmark is not met, an alarm is
generated.

The following hierarchy of alarms was developed at AT&T to prioritize alarm error clearance:
-

Type 1: minor issue dealing with functionality of auditing system

Type 2: low-level alarm dealing with minor operational issues

Type 3: high-level alarm dealing with issues that must be investigated by the auditor (ex suspense file
becomes too large)

Type 4: serious crisis

4.3 HSP
The project started by improving the supply chain. HSP had primarily legacy systems, which meant that a
different approach had to be taken. At HSP, the auditor had unrestricted access to raw data, so
benchmarks were determined to test the validity of this data. First, data validity tests were applied to
individual transactions. Then, continuity equations (CE), which used probability models to calculate the

expected value and variance of a business process metric, were used to provide additional assurance
over the data. This required coordination between mathematicians, IT personnel, and auditors. The CEs
addressed the risk that the data was not examined in aggregate, but the degree of aggregation needs to
be determined by the accountant.

16

4.4 Third-Party Solutions to Continuous Auditing

17

So far, the continuous auditing systems that have been discussed in this paper were built in-house. While
there are some clear advantages to building a system in-house, such as the ability to tailor the system to
meet the companys auditing, risk management and individual internal reporting needs, many companies
do not have the time, expertise or financial capability to build their own CA system. Over the past few
years, third-party solutions to continuous auditing have been developed.
Refer to Appendix VI for a comparison of various third-party continuous auditing solutions. These thirdparty solutions are all based on the MCL architecture (refer to appendix 3.1 for a discussion of MCL
systems). This is because it is easier to sell an add-on continuous auditing program rather as opposed to
an EAM system integrated with the ERP environment.

4.5 The Market for Continuous Auditing

As previously mentioned, there are many benefits to continuous auditing. The ability to provide real time
assurance increases the effectiveness and efficiency of the audit since material misstatements are dealt
with on a proactive rather than reactive basis.

18

However, a 2010 KPMG survey of 112 respondents in a

variety of industries found that half of respondents are either unconvinced on the benefits of a CA or CM
system or are failing to move forward with implementing such a system, and only 20% of respondents
either currently have a CA/CM system in place or are planning on implementing one. The majority of
companies surveyed use a mix of manual and automated checks.

There are still significant challenges to overcome before the market fully accepts and adopt continuous
auditing. A 2011 PWC survey found that continuous auditing systems are being implemented because
globalization is increasing the complexity and breadth of risks. Automation of the assurance process
allows the company to achieve efficiencies and reducing the cost of compliance, while at the same time
increasing the efficiency of the assurance process by managing compliance more effectively.

19

Stakeholders perceive continuous auditing systems as a way to realign the audit process to focus on
anomalies. It is also seen as a more cost effective way to automate issue analysis, reporting, and
documentation.

16

(Michael Alles A. K., 2008)

This reference applies to this entire section. (John R Khun Jr, 2010)
18
(David Y Chan, 2011)
19
(The Path Forward for Data Analysis and Continuous Auditing, 2011)
17

The greatest perceived benefit is that greater assurance can be obtained by a continuous auditing
system, which was stated as a benefit by 61% of respondents, but nearly half of respondents believe their
20

organization is effectively controlled with manual checks. This may explain why 61% of respondents
believe that implementation of such a system will not reduce costs in their organization. This report has
provided evidence that continuous auditing provides a higher quality of assurance, but nearly 40% of
respondents are unaware of this. Therefore, there needs to be greater education and awareness
surrounding the numerous benefits of continuous auditing systems. Another reason why CA systems
have not been widely adopted is the concern over potential independence issues, as previously
discussed in section 2.4 of this report.

There is currently demand for continuous auditing systems, but current demand is not very high
compared to potential demand. As more companies adopt CA systems, market momentum will be
created, thereby enticing even more companies to adopt for fear of being left behind.

More recently, a small-scale study was conducted to determine the acceptance of continuous auditing
systems by internal auditors and the degree of adoption. Companies found that CA systems greatly
assisted with SOX compliance requirements.

21

Therefore, the companies most likely to start adopting a

continuous auditing system are large public companies that need to comply with SOX requirements.
Management acceptance of CA systems is mixed and the most common drawback cited is the perceived
cost of the system. The degree of adoption varied per industry, with most companies being in the
emerging stage, meaning the continuous auditing system is in the process of being implemented but is
not yet used on a company-wide scale (see Appendix VII). This means there is significant opportunity for
growth in the CA market, even in companies that have already begun the adoption process, since full
implementation of CA systems is not prevalent.

22

5.0 Future Initiatives and Conclusion

There are many areas for research in the field of continuous auditing. With regards to the IT architecture,
there has not been much research surrounding EAM, especially with regards to independence and public
perception of the auditors role. There has also not been much research done regarding both the external
and internal auditors legal liability of a material misstatement is found in the financial statements. This
paper discussed several frameworks and methods on how to use continuous auditing systems to obtain
assurance, but there needs to be more work done around how to integrate continuous auditing systems
with management decision making.

23

20

(Continuous auditing and monitoring: Are promised benets now being realised?, 2010)
(Miklos A. Vasarhelyi, 2011)
22
(Continuous auditing and monitoring: Are promised benets now being realised?, 2010)
23
(John R Khun Jr, 2010)
21

10

In terms of the actual continuous auditing process, this paper gave a few examples of exception reporting
and alarm hierarchies (for example, those used at Siemens and AT&T). However, there is still more
research to be done around how to best implement alarms and how to structure alarm hierarchies. For
example, it has yet to be determined the degree to which artificial intelligence can assist in continuous
auditing.

24

Lastly, there has not been much work done surrounding the types of organizations that implement
continuous auditing systems. This report suggested that SOX compliance may play a role in which
companies adopt a CA system, but there are also cheaper third-party alternatives that can be adopted by
smaller entities. It has not been determined whether implementation of such a system is driven by the
personalities of management or by the nature of the organization.

25

There is significant promise in the area of continuous auditing. Several companies have successfully
implemented such a system, and surveys suggest that potential demand for continuous auditing is quite
high. However, more research needs to be done in the field and managers need to be better educated on
the benefits of CA systems before continuous auditing becomes widespread.

24
25

(John R Khun Jr, 2010)

(John R Khun Jr, 2010)

11

Appendix I Traditional Audit Approach

26

Purpose: The purpose of this appendix is to describe the traditional audit process as a basis for
comparison against the continuous auditing process.

26

(The Path Forward for Data Analysis and Continuous Auditing, 2011)

12

Appendix II Continuous Auditing Integrated Audit Approach

27

Purpose: The purpose of this appendix is to describe the auditing framework when a continuous auditing
system is in place.

KRI: Key Risk Indicators

27

(The Path Forward for Data Analysis and Continuous Auditing, 2011)

13

Appendix III Chan and Vasarhelyi 7-Step Continuous Auditing Paradigm

28

Purpose: This 7-step continuous auditing paradigm shows the 7 key differences between traditional
auditing and continuous auditing. It was developed to be used by researchers and practitioners as a basis
for the current understanding of continuous auditing. Practitioners can use this as a first resort when
trying to implement a continuous auditing system. Researchers can use the paradigm to further the
current understanding of continuous auditing.

28

(David Y Chan, 2011)

14

Appendix IV Risk Indicator Continuous Assurance (RICA) in an Audit

29

Purpose: The purpose of this diagram is to show how RICA is integrated into a typical audit.

29

(Dale Johnstone, 2009)

15

Appendix V IIPCA Model

30

Purpose: This is a model developed to integrate a continuous auditing model into the existing enterprise
business processes.

Electronic audit evidence functions:

-

IM: inventory management

CO: confirmation by authenticated documents

ED: external documentation

RP: re-performance

ID: internal documentation

AP: analytical procedures

IC: inquiries

Ticket number: at each phase of the business cycle, the data is stamped with the time. The tickets also
indicate whether there was a flag or alarm triggered in the data.

30

(Munir Majdalawieh, 2012)

16

Appendix VI Third Party Solutions to Continuous Auditing

31

Purpose: The purpose of this appendix is to compare the functionality of various third-party continuous
auditing solutions available.

31

(John R Khun Jr, 2010)

17

Appendix VII Continuous Auditing Implementation Stages

32

Purpose: The purpose of this appendix is to show the degree of adoption among 9 companies that have
started adopting CA systems.

Level of adoption:
-

Traditional: traditional audit methods are used (see Appendix I) but investments in research and
development have been made

Emerging: early adoption of a CA system but IT audit still works independently

Maturing: growth stage with coordination between intra-company departments, and beginning to rely
on benchmarks/key performance indicators

Full continuous: complete CA system including data warehouse, benchmarking history, error history,
and complete integration with risk management department

Adoption metric:
-

Objectives: the degree to which the objective of internal audit is to provide continuous assurance

Approach: the degree to which continuous audit alarms have been implemented. A maturing
continuous auditing system is one in which the alarms are effectively utilized.

IT/Data access: the degree to which data access is automated and integrated into the CA system. A
traditional approach relies on manual data extraction.

32

Audit automation: the degree to which audit processes and alarms are automated

(Miklos A. Vasarhelyi, 2011)

18

Audit and management sharing: the degree to which management has facilitated the implementation
of a CA system and data sharing among departments

Management of audit function: the degree to which systems have been implemented to manage the
CA system

Analytical methods: the usage of analytical methods in a CA system at the transactional level

19

Works Cited

Continuous auditing and monitoring: Are promised benets now being realised? (2010). Retrieved June 24, 2012,
from KPMG Advisory:
http://www.kpmg.com/BE/en/IssuesAndInsights/ArticlesPublications/Documents/Continuous%20Auditing%20and
%20Monitoring.pdf
Dale Johnstone, E. C. (2009). Achieving Continuous IT Auditing: RICA. ISACA , 5.
David Y Chan, M. A. (2011). Innovation and Practice of Continuous Auditing. International Journal of Accounting
Information Systems , 9.
Jill Joseph Daigle, R. J. (2008). Auditor Ethics for Continuous Auditing and Continuous Monitoring. ISACA , 4.
John R Khun Jr, S. G. (2010). Continuous Auditing in ERP System Environments: The Current State and Future
Directions. Journal of Information Systems , 23.
Michael Alles, A. K. (2008). Putting Continuous Auditing Theory into Practice: Lessons from Two Pilot
Implementations. Journal of Information Systems , 21.
Michael Alles, G. B. (2006). Continuous monitoring of business process controls: A pilot implementation of a
continuous auditing system at Siemens. International Journal of Accounting Information Systems , 25.
Miklos A Vasarhelyi, F. B. (1991). The Continuous Audit of Online Systems. Auditing: A Journal of Practice & Theory
, 17.
Miklos A. Vasarhelyi, M. A. (2011, August). The Acceptance and Adoption of Continuous Auditing by Internal
Auditors: A Micro Analysis . Retrieved June 24, 2012, from JEBCL: http://jebcl.com/symposium/wpcontent/uploads/2011/08/Continuous-Auditing-Implementation-Study.pdf
Munir Majdalawieh, S. S. (2012). Intra/inter process continuous auditing (IIPCA), integrating CA within an
enterprise system environment. Business Process Journal Management , 24.
The Path Forward for Data Analysis and Continuous Auditing. (2011, May). Retrieved June 24, 2012, from ISACA:
http://www.isaca-kc.org/Chapter%20Meetings/20110512%20Continuous%20Auditing.pdf

20

Annotated Bibliography
Author

Title of Article

Periodical/
website

Vol. / No. /
Edition

Year
Pages
published

Date
accessed

Location, data
base, website,
link

Added in the
final paper?

Michael Alles,
Gerard Brennan,
Alexander
Kogan, Miklos A.
Vasarhelyi

Continuous
monitoring of
business process
controls: A pilot
implementation of
a continuous
auditing system at
Siemens

International
Journal of
Accounting
Information
Systems

July 2006

2006

May 16,
2012

ABI Inform

Yes

25

Annotation
The authors of the paper implemented a continuous monitoring of business process controls (CMBPC) system at Siemens.
The paper addresses several issues with regards to the implementation of such a continuous auditing system. Audit action
sheets (AAS) are documents that were created to assess any configurable process controls that could be automated. First, it
was decided what AASs should be automated. Both automation of the control and assessment of the effectiveness of the
control elements were considered. The degree of effectiveness of each control was assessed by the system based on a 0-4
rating. There were some problems with the data base not being able to handle the volume of data being continuously monitored
and retained. Since the data is being continuously monitored (every 10 seconds), the volume of data retained can be greatly
reduced by only retaining the data if the system found control exceptions. However, a good database is still required to retain
these exceptions. An interface where the auditor can see the 0-4 score achieved by the control and related controls is also
required. An overall rating based on the aggregate of related control elements (ex all password controls, including those relating
to password length, expiry date, log in attempts) then use their judgment in assigning an overall rating to that control (ex
passwords). This can either be automated (ex using a weighted average of the control elements) or assessed by the auditors
judgment. Some kind of a control exception hierarchy is also required. It is inefficient and unrealistic to change the entire audit
process, since large companies like Siemens have legacy systems and understaffed audit departments. Some re-engineering is
unavoidable due to the need to separate out formalizable and automatable controls.
David Y. Chan,
Miklos A.
Vasarhelyi

Innovation and
practice of
continuous
auditing

International
Journal of
Accounting
Information
Systems

December 2011 2011

Annotation

21

February
29, 2012

ABI Inform

Yes

Traditional auditing is outdated in the fast-paced real-time world we currently live in. The following continuous auditing paradigm
can be used by researchers and practitioners in developing and furthering the current understanding continuous auditing:
1. Continuous audits rather than periodic audits, implemented in the following 4 stages
2. Audit model becomes proactive, meaning misstatements are detected before they make it to the financial statements,
rather than reactive
3. Several audit procedures are automated
4. Internal auditors primarily test those controls requiring judgment and handle exception reports, whereas external
auditors will need to certify the continuous auditing system
5. Nature of audit procedures: Types of controls are continuous data assurance and monitoring, as opposed to the
analytical procedures and test of details that are frequently done in traditional audits
Timing of audit procedures: Testing occurs on a continual basis
Extent of audit procedures: Entire population is considered in testing
6. Much of the testing is performed by the system, such as analytics and modeling
7. Audit reporting also becomes more frequent, and if there are no exception reports or significant areas requiring
judgment, audit reporting can theoretically become continuous as well
CA systems can be implemented using the following 4 steps:
1. Automation of audit procedures
2. Data modeling and benchmarks developed
3. Data analytics and exception report clearing
4. Audit reporting
This paper suggests that continuous auditing will replace traditional auditing.
Michael Alles,
Alexander
Kogan, Miklos A.
Vasarhelyi

Putting
Journal of
Continuous
Information
Auditing Theory
Systems
into Practice:
Lessons from Two
Pilot
Implementations

Vol 22, No 2 Fall 2008

2008

21

February
29, 2012

ABI Inform

Yes

Annotation
This paper discusses the implementation of continuous auditing (CA) systems at Siemens and HSP.
Siemens: First the internal audit group needs to determine what control procedures are being followed. Types of controls are: a)
verify by testing a specific control for existence, correctness and functioning of the control, b) verify by ensuring a prohibited
behaviour cannot happen, c) verify any automatic control settings in the ERP system. Then, the team determined what controls
can be automated. Siemens used the monitoring and control approach as opposed to EAM. The challenge was making e-audit
output not only machine readable but also machine understandable so reengineering of audit processes took place. The team

22

had particularly difficulty implementing compensating controls in the system, and ensuring that alarm floods did not overwhelm
the human auditor. In hindsight, the team should have implemented a parallel alarm classification hierarchy which assesses the
materiality of the control exceptions. Also, the team learned that a clear change management plan needed to be developed
before such large changes are made to the auditing and IT environment.
HSP: The project started by improving the supply chain. HSP had primarily legacy systems, which meant that a different
approach had to be taken. At HSP, the auditor had unrestricted access to raw data, so benchmarks were determined to test the
validity of this data. First, data validity tests were applied to individual transactions. Then, continuity equations (CE), which
used probability models to calculate the expected value and variance of a business process metric, were used to provide
additional assurance over the data. The CEs addressed the risk that the data was not examined in aggregate, but the degree of
aggregation needs to be determined by the accountant.
John R. Kuhn,
Jr., Steve G.
Sutton

Continuous
Journal of
Auditing in ERP
Information
System
Systems
Environments:
The Current State
and Future
Directions

Vol 24, No 1
Spring 2010

2010

23

February
29, 2012

ABI Inform

Yes

Annotation
This paper looks at the IT architecture alternatives for continuous auditing systems. Continuous auditing systems are key in
many governance, risk management and compliance (GRC) departments and used to assist external auditors. Two types of
systems are:
1) Embedded Audit Modules (EAM): built into the system to provide continuous assurance over certain types of activities, but
is typically added onto the purchased accounting system (ex SAP) afterwards and customized for that particular business. This
includes modules that match certain key documents (ex purchase order to invoice) or compare transactions against some key
audit threshold. Reports over the functionality of these controls are also available on a real-time basis.
Alternatives to EAM:

EAM ghosting: run the EAM functions on a copy of the EAM system on a separate and external system, so as to
minimize the risk of live data being affected by the EAM system
Can also have separate (copies of the EAM) systems for the development, quality assurance, and production of the
EAM systems, which further reduces risk of live data being changed by the IT department
2) Monitoring Control Layer (MCL): The continuous auditing (CA) system is external to the processing systems. The CA
system receives data at pre-specified intervals and runs audit procedures with that data.
Limitations:

23

Processing requirements to implement the system, could be costly

Need to customize purchased software
EAM must be built into the ERP system, and therefore needs to be tested, designed, etc at this level as well; many
larger companies operate several ERP systems (ex Siemens has over 20)
Could have large volume of error reports, risk of false alarms
If external auditor designs the system, possible independence issues
Uncertainty regarding legal liability of systems fail to detect material misstatements
These limitations provide a basis for future research.
James E.
Hunton, Jacob
M. Rose

21st Century
Auditing:
Advancing
Decision Support
Systems to
Achieve
Continuous
Auditing

Accounting
Horizons

Vol 24, No 2 2010

2010

17

March 1,
2012

ABI Inform

No

Annotation
Decision support systems are defined as any system intended to help improve the information available for decision making
purposes, and are used in both external audits and internally. Previous studies have shown that DSS systems are not fully or
properly utilized. However, this paper realizes that DSS systems can be useful in implementing continuous auditing systems.
Data mining, text mining, and other data analysis techniques (and in some cases, artificial intelligence) can be leveraged in
continuous auditing (CA) systems. Therefore, it is the researchers opinion that DSS systems are a necessary precursor to CA
systems. Dynamic auditing systems that adapt to new and unusual information should also be considered in CA systems. The
authors stress that auditors need to be appropriately trained in these systems. This paper also suggests that continuous
auditing/monitoring is inevitable due to increasing pressure on auditors to assess economic, fraud and other financial statement
risk factors (ex valuation of complex derivatives).
Sridhar
Ramamoorti,
Michael P.
Cangemi,
William M.
Sinnett,

The Benefits of
Continuous
Monitoring

Financial
Executives
Research
Foundation

August 2011

2011

98

May 2,
2012

ABI Inform

Annotation
This is a piece of research done in order to help executives practically implement a continuous monitoring or continuous

24

auditing system. The paper examined 11 companies that had already implemented such a system, and learned the following:
-

Munir
Majdalawieh,
Sofiane
Sahraoui, Reza
Barkhi

CM can be a way to achieve better performance across the entire company not just through a return on the investment,
but indirectly through operational effectiveness and risk reduction
- A continuous monitoring program needs a CM champion, which is someone who will be held responsible for the
program and allocating the internal resources
- Internal auditors play a critical role in moving towards a CM system
- Using externally developed software can help in cost management
- Companies that initially launch a CM program in one division are rapidly working to expand it to all of their businesses
- Companies also want to learn how CM programs are implemented elsewhere with the hope of improving their own CM
system
Intra/inter process Business
May 17,
Vol 18, No 2
2012
24
ABI Inform
Yes
continuous
Process
2012
auditing (IIPCA), Management
integrating CA
Journal
within an
enterprise system
environment
Annotation
This paper outlines how a company can develop a full continuous auditing (CA) system. There are 3 objectives to developing
the CA system: 1) building the CA model, 2) integrate CA model with business model, 3) ensure practitioners know how to use
the CA system to its full potential. Several approaches have already been introduced, and they fall into one of the two
categories: using CA as a quality assurance too, or automate the auditing process from a monitoring perspective. Critical
success factors in CA models include internal control testing, testing transactions, ensuring business rules are followed, and
three main requirements in systems are: continuous control monitoring, continuous data assurance and continuous risk
monitoring assessment. The paper proposes the following IICPA model:

25

Electronic audit evidence functions:

- IM: inventory management
- CO: confirmation by authenticated documents
- ED: external documentation
- RP: re-performance
- ID: internal documentation
- AP: analytical procedures
- IC: inquiries
Ticket number: at each phase of the business cycle, the data is stamped with the time. The tickets also indicate whether there
was a flag or alarm triggered in the data.
Miklos A.
The Continuous
Vasarhelyi, Fern Audit of Online

Auditing: A
Journal of

Vol 10, No 1

1991

26

17

May 17,

ABI Inform

Yes

B. Halper

Systems

Practice &
Theory

2012

Annotation
This is a critical piece of literature in the area of continuous auditing since it was the first paper to truly explore the idea of a
Continuous Process Audit Methodology (CPAM) to implement Continuous Process Audit Systems (CPAS). At the time,
corporations generally used a main database system with other databases connected to it. Therefore, auditors have to audit
both the system and the reconciliation between the systems. However, these types of procedures do not address certain key
issues such as the timeliness of addressing errors (since audits are only done once a year), and data is only as reliable as the
system that generated it so real-time assurance over system controls is useful. There are two different ways to obtain
continuous assurance: 1) data flowing through the system is continuously tested based on auditor-defined rules, and 2) data is
tested indirectly by looking at specific occurrences. The CPAS can pull the data from either the standard application reports, the
raw data that feeds to these application reports, or direct monitoring data. Audit work should be done in a separate, independent
location. The paper also describes a hierarchy of alarms that can be used, and the types of controls that can be implemented.
Dale Johnstone
and Ellis Chung
Yee Wong,
CISA, CFE,
CISSP

Achieving
Continuous IT
Auditing: RICA

ISACA

2009, Volume 6 2009

May 25,
2012

ISACA:
http://www.isaca.
org/Journal/PastIssues/2009/Volu
me6/Pages/Achievin
g-Continuous-ITAuditingRICA1.aspx

Annotation
There are several challenges to continuous IT auditing that the article discusses. Achieving auditor independence is a challenge
because the auditor is involved in both developing and auditing the CA system. CA systems are also effectively controls on
controls, so it is critical that the controls get sufficient depth of assurance. Sufficient and appropriate evidence should also be
obtained, which support the auditors scope and objective.
This report suggests an alternative CA approach, called risk indicator continuous assurance (RICA), which focuses on the
identification of risk indicators (RI) which are metrics that measure control effectiveness. There are 3 steps to a RICA
approach:
1. Identify the RIs
2. Compute the RI metric that will be used (ex a ratio of errors/total population)
3. Develop thresholds to evaluate the RIs
The paper also outlines the three critical success factors to this approach: auditors have direct access to internal systems,

27

critical points and risk indicators are reviewed and updated regularly, risk indicators are appropriately defined based on the
population of inputs and are simple enough to interpret.
Jill Joseph
Daigle, CISA,
CIA, CISSP,
Ronald J. Daigle,
Ph.D., CPA, and
James C.
Lampe, Ph.D.,
CPA

Auditor Ethics for

Continuous
Auditing and
Continuous
Monitoring

ISACA

2008, Volume 3 2008

May 25,
2012

ISACA:
Yes
http://www.isaca.
org/Journal/PastIssues/2008/Volu
me3/Pages/AuditorEthics-forContinuousAuditing-andContinuousMonitoring1.aspx

Annotation
It is important to recognize that continuous auditing (CA) is not the same as continuous monitoring (CM). In both CA and CM,
auditor independence is a critical issue which was especially heightened after SOX 404 was implemented. CA is defined by the
ISACA Standards Board as a methodology used by auditors, typically assisted by technology, to perform audit procedures and
issue assurance on a continuous basis (e.g., weekly, monthly) and CM is defined as a process put in place by management,
usually automated, to determine on a recurring and repetitive basis (e.g., weekly, monthly) if activities are in compliance with
policies and procedures implemented by management. Therefore CA is an auditing tool used by auditors, and CM is a
management tool. Independence impairment affects CM. The paper suggests the following 6-step approach to solve
independence problems:
1. Understand why the issue is an independence issue with the specific facts and circumstances (ex consider integration
of CA and CM systems and sharing of data and code within the company)
2. Identify ethical issues
3. Identify stakeholders
4. Identify ethical principles to adhere to
5. Identify possible solutions and consequences
6. Determine the best alternative by using judgment
The exact solution will vary greatly by the companys specific CA and CM system.

28