Sana Adi Searching for strategy: Value for

Patrice Dutil
Money (VFM) audit choice in the
new public management era
Abstract: Value for Money (VFM) audits are now critical aspects of accountability in
Westminster systems, based on the assumption that ministries are systematically
examined. This study explores the VFM inquiries of the Auditor General of Ontario
(AGO) between 1997 and 2014 as a case study to uncover trends of consistency of
approach under the flourishing “new public management” philosophy of cutting
costs and raising efficiencies. It asks three broad questions: does the frequency and
intensity of VFM audits correlate with the financial importance of the ministries?
Do political controversies affect the choice of what will be subject to a VFM audit?
Finally, do Auditors General make a personal difference in what ministries will be
audited through a VFM framework? The findings demonstrate a pattern of over-
auditing of some ministries while important ministries are consistently under-
audited.

Sommaire : Les verifications de l’optimisation des ressources (VOR) sont devenues

des aspects cruciaux de la reddition de comptes au sein des systèmes de type
Westminster, qui se fondent sur l’hypothèse voulant que les ministères soient
examines de façon systematique. Cette etude se penche sur les enqu^etes de VOR
menees par le verificateur general de l’Ontario entre 1997 et 2014, comme etude de
cas afin de reveler les tendances d’approche uniforme selon la philosophie en plein
essor de la « Nouvelle gestion publique » qui consiste a reduire les co^ uts et a
accro^ıtre l’efficacite. L’etude aborde trois grandes questions : La frequence et
l’intensite des VOR correspondent-elles a l’importance financière des ministères?
Les controverses politiques ont-elles une incidence sur le choix de ce qui fera l’objet
d’une VOR? Et enfin, les verificateurs generaux font-ils une difference personnelle
entre les ministères qui seront soumis a une verification dans le cadre d’une VOR?
Les conclusions indiquent l’existence d’un schema selon lequel certains ministères
subissent des verifications excessives, alors que des ministères importants sont
constamment sous-verifies.

Auditors in the Westminster system long ago slipped the bonds of limiting
their work to verifying and reporting on the accuracy of public accounts.
In the 1970s, their mandates were extended to measuring the managerial

Sana Adi is a graduate of Ryerson University’s Master of Public Policy and Administration
program, class of 2015. Patrice Dutil is Professor in the Department of Politics and Public
Administration, Ryerson University, Toronto (http://patricedutil.com). The authors thank
the journal’s editor and anonymous reviewers for their helpful comments.

CANADIAN PUBLIC ADMINISTRATION / ADMINISTRATION PUBLIQUE DU CANADA

VOLUME 61, NO. 1 (MARCH/MARS 2018), PP. 91–108
C The Institute of Public Administration of Canada/L’Institut d’administration publique du Canada 2018
V
92 SANA ADI, PATRICE DUTIL

performance of government ministries through Value for Money (VFM)

audits or “performance audits.” Audit has become an important subject of
inquiry but remarkably, the actual targets of VFM audits have never been
documented. This study explores the range of VFM inquiries, taking the
work of the Auditor General of Ontario (AGO) as a case in point. The web-
site of the AGO states that:
Each year, the Office audits selected ministry or agency programs and activities, with the
major ones generally audited every five to seven years. In deciding which areas to audit,
the Office considers such factors as the results of previous audits, the total revenues or
expenditures at risk, the impact of the program or activity on the public and the costs of
performing the audit in relation to the benefits (Office of the AGO 2018).

This study explores the VFM inquiries of the AGO between 1997 and
2014 as a case to uncover trends of consistency of approach under the flour-
ishing “new public management” philosophy of cutting costs and raising
efficiencies. It asks three broad questions: does the frequency and intensity
of VFM audits correlate with the financial importance of the ministries? Do
political controversies affect the choice of what will be subject to a VFM
audit? Finally, do Auditors General make a personal difference in what
ministries will be audited through a VFM framework? The period
1997–2014 was chosen for review because it covers the first audits under
the Progressive Conservative government led by Premier Mike Harris,
elected in late 1995, and the end of the Liberal government under Premier
Dalton McGuinty. It also corresponds to the unequivocal arrival of the
“new public management” philosophy of cutting costs and raising efficien-
cies in the administration of the state (Evans and Smith 2015; Gendron,
Cooper, and Townley 2001; Richards and Smith 2006) which made VFM all
the more relevant to governance.
This study begins by reviewing the literature on VFM and provides
background on Ontario’s Auditor General’s approach to VFM. It then
reports on the findings of our empirical analysis, which a pattern of over-
auditing of some ministries while important ministries are consistently
under-audited, which challenge the notion that the audits are strategically
chosen depending on risk, on size, or on impact. The study then considers
other possible explanations and outlines a research agenda which should
be of interest to scholars and practitioners alike.

Perspectives on VFM auditing

Remarkably few scholars have sought to track the workings of Auditors
General, though there have been indications that this data could be helpful
to a deeper analysis of the work of this important agent of Parliament.
Most of those who research and write on audit have devoted studies on
VFM AUDIT CHOICE IN THE NEW PUBLIC MANAGEMENT ERA 93

the impact of VFM far more than on its actual practice. Scholars of VFM
have divided into three broad camps. The first and earliest either contested
or defended the legitimacy of VFM either as instruments to improve the
management of public affairs or as a control on democratic government.
The second group, more contemporary, examines VFM through an impact
lens. The third camp specializes in better understanding the technical
aspects of VFM.

VFM and legitimacy

Michael Power, in his landmark The Audit Society: Rituals of Verification
(1997; see also 2000), argued that modern society increasingly demanded
that auditors function as an institutional check on those entrusted with
decision-making and spending. In terms of politics, he pointed to the late
1980s/early 1990s when government VFM audits in the United Kingdom
were applied with increasing frequency as demand to verify government
action and practices grew louder. Power distinguished three core character-
istics that made VFM distinct from other forms of assessment. First, there
had to be a certain level of independence from the subject under investiga-
tion to avoid bias, methodologically collected evidence, and a series of rec-
ommendations based on the findings. Second, the results of an audit had to
be based on solid financial information to legitimize any advice on possible
reforms. Third, audits had to ensure accountability: they had to identify
exactly which parts of the organization had not lived up to their mandates.
For all his attention to method, however, Power did not test his characteris-
tics on which ministries were actually covered by the AG. Admittedly, this
was never his focus: he likely assumed that the process was consistent.
Power’s claims of the spread of audit were challenged by a few authors
(Bowerman, Raby, and Humphrey 2000; Humphrey and Owen 2000; Lin-
deberg 2007.) on the grounds that the practice was not sufficiently wide-
spread to justify his claims. The issue of the comprehensiveness of the VFM
is compelling, and given the concerns expressed at the beginning of this
century, must be tested: to what degree are government operations actually
audited for VFM? Given the general concern about government efficiency,
why are government ministries not systematically audited on their perfor-
mance? What is the logic of auditing some and not others?

VFM as power tool

In the Canadian context, the nature of the Auditor General attracted a funda-
mental debate among political scientists. Donald Savoie (1995), Sharon Suther-
land (1986) and Denis Saint-Martin (2004) considered VFM audits as political
activities that essentially challenged democratic government. David Good
94 SANA ADI, PATRICE DUTIL

labelled the federal Auditor General as a “watchdog,” overseeing the political

government spenders and guardians (Good 2014). Because of this oversight
responsibility, Good considered the Auditor General as independently power-
ful, easily misconstrued as separate from Parliament (Good 2014).

The issue of the comprehensiveness of the VFM is com-

pelling, and given the concerns expressed at the begin-
ning of this century, must be tested: to what degree are
government operations actually audited for VFM?

Hatherly (1999) argued that audits typically fell into two models,
accountability or hierarchical, that also raised political choices. In the
accountability model, the relationship between auditors, the audited and
stakeholders could be seen as lateral, based on an exchange between the
auditor and the group being audited. The power dynamic in this model
allows for feedback between the auditor and the audited, but leaves the
stakeholders as external actors since there are few mechanisms for interac-
tion or dialogue, especially regarding feedback on quality (Hatherly 1999).
The hierarchical model was based on regulatory control stemming from
the stakeholder’s lack of willingness or ability to oversee the group in ques-
tion. Pressures from external standards set expectations of the audited,
while internal procedures regulate its actions. Hatherly argued that the
real-world application of the horizontal accountability model tended to
naturally evolve into the hierarchical model. Internal regulations became
increasingly influenced by the external standards set by the auditor, not the
equitable, bilateral relationship that governs the accountability model. As
this happened, stakeholders became increasingly removed from the pro-
cess, which increased the power awarded to the auditor to ensure account-
ability and oversight of the audited. The audit took on a more corporatist
approach that sought to establish auditee compliance through report publi-
cation rather than an assessment of auditee performance and function.

VFM in technical areas

The third school has focused on trying to theorize the process of VFM
audits. Jeppesen et al. (2017), for instance, have identified four strategic
options in their study of four Nordic national audit offices: a performance
auditing strategy; a financial auditing strategy; a portfolio strategy; and a
hybrid strategy and found that most had adopted a portfolio strategy. This
study was based on interviews with principals, and did not document the
subjects of VFM audit.
VFM AUDIT CHOICE IN THE NEW PUBLIC MANAGEMENT ERA 95

Very early on, Roberts and Politt (1994) argued that VFM enhanced
democratic accountability, but fell short of the standards expected in a pro-
gram evaluation or review. Keen (1999) posited a model that showed how
auditors mixed intuitive and analytical thinking, empirical data, and
behavioural observations in audits. Otto Brodtrick’s 2004 study of Auditors
examined tangentially what they examined and depicted VFM choices
mainly as the product of routine and rationality. Based on interviews with
auditors and VFM auditees, Alwardat, Benamraoui, and Rieple (2015)
pointed out that while audit procedures in the UK did lead to better mana-
gerial practices, the VFM processes were not well institutionalized, which
they felt could lead to a number of conflicts. Also in the Canadian context,
Mayne (2004, 2006, 2010) has published provocative articles challenging
various aspects of VFM in practice.
In her many surveys of line ministry managers, Danielle Morin shed
light on the role of auditors. She showed that VFM audits improved public
organization and that the process was necessary for effective and efficient
governance (Morin 2004, 2014; Morin and Hazgui 2016). The Auditor Gen-
eral was seen as a necessary check on government that could ultimately
streamline and improve organizational processes (Morin 2004). Moreover,
through the VFM, the Auditor General was seen as playing a fundamental
role in alerting the public to weaknesses in the management of public
organizations (Morin 2003; Van Loocke and Put 2011). Morin’s study con-
firmed findings in Scotland, as well as local governments in Finland and
Norway, which revealed that VFM audits are regarded by the auditees as
an important and efficient tool for controlling and reforming the public sec-
tor (Morin 2008; Lapsley and Pong 2000; Johnsen et al. 2001). In the Cana-
dian context Taft (2016) concluded that recommendations by Auditors
have been more frequently accepted and implemented over the past twenty
years. Still, the assumption was that VFM audits rigorously covered all
aspects of spending. In another original contribution on the work of Audi-
tors General in the Westminster system, Talbot and Wiggan (2010) exam-
ined how public sector audit offices self-reported on their performances.
They found that while audit offices make useful contributions in analysing
their work, they do not report a concern for what ministry or program is
actually audited (see also Leclerc 1996; Sharma 2007; Sterck 2007; Leeuw,
2011). In sum, the literature is heavily engaged in formulating theories and
scrutinizing practices. Yet no study has documented exactly what minis-
tries are audited in VFM approaches.

VFM audit in Ontario

Typical of other provinces and the government of Canada, Ontario was an
early adopter of VFM audits (Morin 2003; Desautels 2001). It conducts
96 SANA ADI, PATRICE DUTIL

VFM audits to report on “whether government programs are being well

managed and whether they provide good value for the money spent on
them” (Office of the AGO 2015). Its reports cast judgment on whether gov-
ernment programs are administered with “due regard for economy, effi-
ciency, and environmental impact” and if effective performance measures
are in place to establish program accountability (McGlashan 2011). The
Auditor General Act allows the Office of the Auditor General to go beyond
government ministries and audit broader public sector organizations,
crown corporations, municipalities and agencies receiving provincial
funds. The Office employs approximately a hundred people and spends an
annual budget of around $16 million (Office of the AGO 2015).
Such a large mandate would justify the assumption that all ministries
are systematically audited to determine the degree to which they deliver
value for money. There exists no evidence to support this supposition.
Moreover, the AGO does not justify its audit choices beyond what it
declares on its website.
Three hypotheses can be identified to test the scope of the AGO’s audit-
ing strategy. First, it would be logical to assume that the AGO is more
intensely focused on larger ministries that spend greater proportions of
public funds. Thus the Ministries of Health and Long-Term Care and Edu-
cation, which collectively made up sixty-four per cent of the 2014 provin-
cial budget, would be expected to proportionately receive the most audit
scrutiny, in terms of both frequency and intensity. A competing hypothesis
would be that the AGO makes choices as a response to political pressure
for special studies to be conducted, in other words where risk has been per-
ceived by exogenous actors. These can emanate from media revelations,
the Public Accounts Committee of the Ontario legislature or from the gov-
ernment itself (legislation allows the premier of Ontario or the cabinet to
ask the Auditor General to conduct special audits). If this is the case, VFM
audits would be expected to spike in sectors with spending controversies.
A third hypothesis would be that VFM audit choices are influenced by
individual auditors, driven by their own intellectual and professional inter-
ests and perceptions of risk. In the period under study between 1997 and
2014, the AGO was led by two men with strikingly similar backgrounds:
Erik Peters (1993–2004) and Jim McCarter (2004–2013). Bonnie Lysyk began
her tenure as Auditor General in late 2013, so only the final year covered in
this study involves her mandate. All three were chartered accountants who
had worked in both the public and private sectors; McCarter and Lysyk
also completed an MBA.
To identify the targets of VFM audits, AGO reports were compiled by
ministry and categorized into eight sector categories: health, transportation,
justice, community and social services, finance, education, energy and envi-
ronment, and “other” for the remaining audits (“other” is a large catch-all
VFM AUDIT CHOICE IN THE NEW PUBLIC MANAGEMENT ERA 97

category of one-off studies on ministries, crown corporations, colleges and

universities, school boards and children’s aid societies that do not reappear
consistently). These groupings were chosen in order to correspond with
budget categories in the province. In each sector category, the aim was to
gauge VFM audits based on the frequency (number) of scrutiny during the
selected period, the comprehensiveness of audits per sector, and policy rec-
ommendations. For each year under inspection, ministerial expenditures
were compiled into the eight categories applied to the VFM publications.
To control for inflation and fluctuating budgets, the proportion of alloca-
tions for each category was calculated, giving a percentage figure. Audit
frequency was measured as a percentage of the overall yearly VFM efforts.
Audit intensity (or comprehensiveness) was measured by the length (in
page numbers) of each audit report and recorded. The operating assump-
tion is that programs and services typically receiving greater page alloca-
tions were audited in more detail; certainly a longer report would reflect
the depth and detail of an audit investigation. Another dimension of this
analysis required a deconstruction of the VFM report contents in order to
reveal the kinds of objectives provided in notable publications. Thus a con-
tent analysis was applied to the AGO’s scope to determine if there were
any recurring elements in their evaluation criteria, or if risk was of particu-
lar concern.
Media scans were conducted on VFM coverage for each sector category
to test media influence and determine whether there was a relationship
between the AGO’s interests and what was topical in mainstream news sto-
ries. The assumption was that if the AGO was influenced by political trends
in the media coverage, there would be a corresponding increase in VFM
reports in that sector. A limitation to this method comes from the sole reli-
ance on online media sources. Internet use by the press was more limited
in the mid-to-late 1990s, possibly resulting in fewer online uploads by
mainstream news organizations. As a result, this study is likely to capture
more stories found covering the AGO as the years progressed.
Finally, to measure the influence of individual Auditors General in
terms of audit choices, the results were divided between the tenures of
Peters and McCarter.

Results
The number of yearly VFM audits remained relatively consistent through-
out the eighteen-year period under study (Table 1), ranging from a high of
38 in 2013 to a low of 26 in 2001, with an average of 29 in the Peters years and
33 in the relatively larger ministry led by McCarter. The health category
received the most attention from the AGO’s VFM reports under both Auditors,
with an overall average of 6.4 reports a year, typically constituting more than
98 SANA ADI, PATRICE DUTIL

Table 1. Number of VFM Audits by Auditor

Peters McCarter Lysyk
Year Audits Year Audits Year Audits
1997 32 2005 29 2014 37
1998 30 2006 28
1999 28 2007 36
2000 28 2008 37
2001 26 2009 33
2002 32 2010 35
2003 22 2011 34
2004 33 2012 28
2013 38
Total 231 298 37
Average 28.9 33.1 37
Total 566
Average 31.5

twenty per cent of the AGO’s VFM output per year (Table 2). The aggrega-
tion of “other” categories that fell outside the eight broad budget categories
used by the government of Ontario consistently came in second place, taking
over eighteen per cent of the VFM audit output. Next in prevalence was
community and social services with four reports per year. It constituted an
average of over thirteen per cent of the output over the years examined.
Transportation received the least attention, averaging less than two VFM
audits a year, or just above five per cent of the output (Table 2). The second
lowest frequency of VFM audits was the massive ministry of education,
receiving less than three reports a year, less than ten per cent of the output.

Hypothesis 1: budget weight

The results demonstrate that the AGO’s interests are not determined by
funding allocations. The health sector reflected this clearly: it consumed an
average of 37.8 per cent of the provincial budget during this period, but
was the object of 20.3 per cent of the yearly VFM reports (Table 2). In con-
trast, there seemed to be more intensity in the reports produced by the AGs
on health care ministries. The average report over the period under study
was 92.6 pages long, reaching a high of 171.5 pages in 2008 and only 40
pages in 1997 (Table 3). These significant fluctuations in report length mini-
mize any obvious trends and do not support the budget weight hypothesis.
 Table 2. Proportion of Annual Budget (%$) and Proportion of Total VFM Reports (%A) 1997–2014 Per Category
Social Env. &
Health Trans. Justice Services Finance Education Energy Other
Year %$ %A %$ %A %$ %A %$ %A %$ %A %$ %A %$ %A %$ %A
1997 31.8 15.6 – 6.3 3.1 9.4 16.5 25.0 15.3 9.4 15.5 3.1 11.1 9.4 6.9 21.9
1998 33.6 23.3 4.9 3.3 3.3 6.7 19.5 20.0 17.2 6.7 17.3 10.0 1.5 13.3 3.0 16.7
1999 34.2 17.9 2.6 7.1 3.7 10.7 16.8 25.0 18.7 7.1 20.0 7.1 2.0 7.1 3.2 17.9
2000 35.6 21.4 2.3 3.6 3.9 14.3 15.5 14.3 18.1 7.1 20.5 7.1 3.2 17.9 2.7 14.3
2001 37.3 18.5 2.3 6.9 4.2 17.2 15.4 20.7 18.6 10.3 19.8 6.9 2.0 0.0 3.6 10.3
2002 38.0 21.9 2.3 3.1 4.3 15.6 14.2 6.3 16.7 9.4 18.5 6.3 2.1 15.6 3.9 21.9
2003 39.6 13.6 2.4 4.5 4.4 18.2 13.4 18.2 15.6 9.1 19.8 13.6 1.1 4.5 3.8 18.2
2004 39.6 18.2 2.2 3.0 4.0 12.1 13.0 9.1 17.4 12.1 18.8 3.0 1.4 15.2 3.7 27.3
Avg./year 36.2 18.8 2.7 4.7 3.9 13.0 15.5 17.3 17.2 8.9 18.8 7.1 3.1 10.4 3.9 18.6
2005 40.0 17.2 2.3 6.9 3.7 6.9 13.0 10.3 15.8 6.9 19.5 10.3 1.3 6.9 4.7 34.5
2006 39.1 17.9 4.1 3.6 3.6 7.1 13.2 10.7 14.0 7.1 19.4 7.1 1.3 21.4 5.1 26.9
2007 40.5 22.2 3.1 11.1 3.6 13.9 12.0 5.6 12.7 13.9 19.8 5.6 1.5 11.1 5.5 16.7
VFM AUDIT CHOICE IN THE NEW PUBLIC MANAGEMENT ERA

2008 39.5 24.3 3.6 2.7 3.8 13.5 12.6 16.2 13.3 2.7 19.8 13.5 1.5 13.5 5.9 13.5
2009 42.1 24.2 2.1 12.1 3.9 6.1 13.3 12.1 11.8 12.1 19.9 9.1 1.5 9.1 5.4 15.2
2010 37.5 28.6 2.0 2.9 3.3 11.4 12.6 11.4 13.2 14.3 20.0 14.3 1.3 5.7 6.2 11.4
2011 37.1 14.7 1.9 5.9 3.5 5.9 12.0 14.7 10.1 17.6 24.6 11.8 1.8 14.7 9.3 20.6
2012 37.9 32.1 1.9 3.6 3.2 14.3 11.7 3.6 12.6 17.9 25.0 14.3 2.4 7.1 5.4 3.6
2013 38.8 18.4 2.0 5.3 3.2 5.3 12.4 10.5 11.5 5.3 24.5 13.2 2.2 23.7 5.3 15.8
Avg./year 39.2 22.2 2.6 6.0 3.5 9.4 12.5 10.6 12.8 10.9 21.4 11.0 1.6 12.6 5.9 17.6
2014 38.7 16.2 2.2 2.7 3.3 18.9 12.2 5.4 11.2 13.5 25.3 10.8 2.1 10.8 4.9 21.6
Avg. 37.8 20.3 2.6 5.3 3.7 11.5 13.9 13.3 14.7 10.1 20.4 9.3 2.3 11.5 4.9 18.2
99
 Table 3. New/Special (N/S) VFM Reports and Follow-Up (F) VFM Reports 1997–2014 Per Category
Social Env. & 100
Health Trans. Justice Services Finance Edu. Energy Other
Year N/S F N/S F N/S F N/S F N/S F N/S F N/S F N/S F
1997 2 3 1 1 1 2 4 4 1 2 1 0 2 1 2 5
1998 2 5 0 1 2 0 3 3 1 1 1 2 3 1 1 4
1999 3 2 1 1 2 1 3 4 1 1 1 1 0 2 3 2
2000 5 1 1 0 2 2 2 2 1 1 1 1 3 2 4 0
2001 2 3 1 1 3 2 3 3 2 1 1 1 0 0 1 2
2002 3 4 0 1 3 2 1 1 2 1 1 1 3 2 3 4
2003 2 1 0 1 1 3 2 2 0 2 2 1 1 0 3 1
2004 3 3 1 0 1 3 2 1 2 2 0 1 2 3 7 2
Total 22 22 5 6 15 15 20 20 10 11 8 8 14 11 24 20
Avg./year 2.8 2.8 0.6 0.8 1.9 1.9 2.5 2.5 1.3 1.4 1.0 1.0 1.8 1.4 3.0 2.5
2005 4 1 2 0 1 1 1 2 2 0 1 2 1 1 7 3
2006 3 2 0 1 1 1 2 1 0 2 2 0 4 2 1 6
2007 4 4 2 2 3 1 1 1 3 2 1 1 3 1 2 4
2008 6 3 1 0 4 1 4 2 1 0 3 2 1 4 4 1
2009 4 4 2 2 0 2 3 1 3 1 2 1 1 2 4 1
2010 5 5 0 1 0 4 1 3 4 1 2 3 1 1 1 3
2011 2 3 0 2 2 0 2 3 4 2 2 2 4 1 3 4
2012 5 4 1 0 4 0 0 1 2 3 2 2 1 1 0 1
2013 5 2 2 0 0 2 2 2 0 2 3 2 5 4 3 3
Total 38 28 10 8 15 12 16 16 19 13 18 15 21 17 25 26
Avg./year 4.2 3.1 1.1 0.9 1.7 1.3 1.8 1.8 2.1 1.4 2.0 1.7 2.3 1.9 2.8 2.9
2014 2 4 0 1 3 4 2 0 4 1 2 2 3 1 8 0
Grand Total 62 54 15 15 33 31 38 36 33 25 28 25 38 29 57 46
SANA ADI, PATRICE DUTIL

Avg. 3.4 3.0 0.8 0.8 1.8 1.7 2.1 2.0 1.8 1.4 1.6 1.4 2.1 1.6 3.2 2.6
VFM AUDIT CHOICE IN THE NEW PUBLIC MANAGEMENT ERA 101

Table 4. Total Page Count Per VFM Report 1997–2014 Per Category
Social Env. &
Year Health Trans. Justice Services Finance Education Energy Other
1997 40 31 19 76.5 11.5 18 35.5 31
1998 51.5 1 35.5 65.5 17 23.5 59 20
1999 75 22.5 42.5 58 15 26.5 5 22.5
2000 67 7 41 38 15 19 50 53
2001 76 34 76.5 63.5 39.5 32 0 33.5
2002 116.5 2 81.5 45.5 69.5 34 115.5 116.5
2003 48 10 79 51.5 14 55 19.5 86.5
2004 96.5 25 43.5 65.5 55 9.5 85.5 149
Total 570.5 132.5 418.5 464 236.5 217.5 370 512
Avg./year 71.3 16.6 52.3 58 29.6 27.2 46.3 64
2005 72 47 25.5 28.5 35 26 27.5 147
2006 85 6 25 53 7.5 21 89 50
2007 11.5 59 65 25.5 77.5 21 82 69
2008 171.5 24.5 119 98.5 17 78.5 54 111
2009 154 47 10.5 64 60.5 35.5 33.5 90.5
2010 167.5 9 37 36.5 74.5 54 33 37
2011 58 14 36 51 70.5 51 89 79
2012 150.5 25 96 9.5 50 51 31.5 6.5
2013 128.5 47 13.5 71 6.5 75.5 139 87.5
Total 998.5 278.5 427.5 437.5 399 413.5 578.5 677.5
Avg./year 110.9 30.9 47.5 48.6 44.3 45.9 64.3 75.3
2014 97 11.5 129 96 148.5 69 101.5 287.5
Grand Total 1666 422.5 975 997.5 784 700 1050 1477
Avg. 92.6 23.5 54.2 55.4 43.6 38.9 58.3 82.1

On average, VFM reports in all ministries over the period under study
were 56 pages long.
In striking contrast to the health sector, the justice sector received far
more attention from the AGO than its budget would justify. It averaged
11.5 of the total reports, yet only received an average of 3.7 per cent of the
budget (Table 2). Interestingly, the VFM audits were fairly comprehensive
of this sector, totalling approximately 54 cumulative pages per year
(Table 4). Similarly, the environment, energy and natural resources sector
was audited five times more per year (11.5 per cent) than would be
expected by a 2.3 per cent budget allocation. Audit comprehensiveness was
also quite high at an average of 58 pages. Conversely, the finance sector,
which made up 14.7 per cent of the budget, comprised only 10.1 per cent of
102 SANA ADI, PATRICE DUTIL

the yearly VFM reports. These publications were also less comprehensive,
with an average page count of 44 pages. One of the more surprising results
was the audit count of the education sector which was expected to garner
significant attention to coincide with its 20.4 per cent annual
budget allocation. VFM reports averaged to half as many, hovering at
around 9.3 per cent. These reports were lacking the comprehensiveness of
larger, more detailed studies, and averaged 39 pages. Thus, the logical
assumption that ministries with greater funding would receive a propor-
tionate amount of regulatory attention is not consistent with the evidence.
This study’s findings do not support the hypothesis that there is a relation-
ship between budget and VFM frequency per sector as there does not
appear to be a correlation between the two.
The closest relationship between VFM audit frequency and the propor-
tion of the budget was evident in the community and social services sector.
It received 13.3 per cent of the VFM audits, as well as receiving an average
of 13.9 per cent of the average provincial budget.
If the AGO’s choice to audit programs was based on financial impor-
tance, then it would be expected that the proportion of VFM audits should
more closely match the budget allocation for that sector. While it is clear
that healthcare, as the largest expenditure in the government of Ontario,
would have commanded more attention than other ministries, it did not do
so in proportion to its weight in government. The health care sector is sig-
nificantly under-audited.
It is apparent that the choice of VFM audits is not driven by how much it
costs the treasury to fund a program. Other factors must drive the AGO’s
decision. Is risk a consideration? Possibly, but it stretches credulity. Are the
ministries of the Environment, Energy and Natural Resources inherently
more risky? Strikingly, few programs in these ministries are concerned
with direct-to-client services. More research is needed in determining how
choices are made if indeed they are based on risk, and this is more likely to
be realized following a sustained research program of interviews.

Hypothesis 2: the impact of political

controversy
The influence of political controversies on the AGO’s selection process is
evident in the division between new/special and follow-up reports. This
division is consistently equal in all sector categories except for health,
finance, and the “other” aggregate. There was an average of 3.4 (53 per
cent) new/special reports on the health sector and 3.0 (47 per cent) follow-
up reports out of a total average of 6.4 yearly audits. The remaining five
sector categories showed an equal division. The finance and “other” sector
categories had the same split between new/special reports (56 per cent)
VFM AUDIT CHOICE IN THE NEW PUBLIC MANAGEMENT ERA 103

and follow-ups (44 per cent). It is possible that, due to the high rate of pro-
gram output by the health and finance sectors, the AGO is limited in its
resources to conduct more follow-up reports. However, this does not
explain why the education sector has an even division despite having a
similarly high policy output. Interestingly, the “other” aggregate category
had more new/special reports than follow-ups. This may simply be due to
the AGO’s reduced interest in smaller ministries and programs, so it pro-
duces new/special reports more often than follow-ups.
The impact of political controversy may shed light on the perceptions
of risk of the AGO. It seems that issues in the media drive at least a por-
tion of AGO VFM audits, certainly far more that those triggered by the
perceived administrative risks. There was no shortage of controversies in
this period affecting a number of ministries: Government Services (IT pro-
ject cost-overuns); Ministry of Transport (highway 407); the Ministry of
Energy (the Mississauga and Oakville Power Plants; costly Green Energy
initiatives); the Ministry of Health (the ORNGE and E-Health controver-
sies) to name but the more prominent. Government debt was driven from
$101.9B in 1995-96 to $268B in 2013-14, making Ontario the world’s most
indebted sub-sovereign borrower. But there again, in terms of audits, con-
troversies hardly seemed to move the AGO significantly in terms of its
audit choices.
Looking at the neglected sectors can help reveal patterns in the AGO’s
interests. Several ministries were altogether ignored. For example, there
were no VFM reports conducted on the energy sector and its affiliated min-
istries and programs in 1999, 2001, 2003, 2010, and 2012, or nearly thirty
per cent of the period under investigation. Similarly, there were no audits
of the ministry of the Environment in 1998, 2001, and 2013, while Natural
Resources was also neglected in 1999, 2001, 2003, 2010, and 2012. These
omissions are remarkable given the important programs at work through-
out this time.
The media influence on the AGO’s selection process was also evident in
another way. The difference between new/special reports and follow-ups
may reveal a propensity for the AGO to quickly shift its focus to high-
profile issues rather than to take stock of previous recommendations to
track their progress. If there is a bias toward new/special reports, then the
VFM is actively pursuing policy issues raised by the media rather than
responding to political pressures to review established programs and poli-
cies. When VFM reports precede news coverage, one could assume that
this reflects the AGO’s favoured position in the media as a source of sala-
cious news stories regarding government “waste of tax dollars.” However,
when the reverse is true, it may reveal the influence that the media has on
the AGO’s selection process for audits, contributing a novel explanation for
the AGO’s scope unseen in the literature.
104 SANA ADI, PATRICE DUTIL

By favouring high profile ministries, the AGO has secured its role as the
most credible player in assessing programs and expenditures (Hatherly
1999). Coupled with its strong media presence, the office wields significant
power not only to highlight issues in accountability, but in some ways to
also enforce its message through public backing. For example, the Novem-
ber 2012 report on the Education Sector Collective Agreement for
2012–2014 had a significant impact on the media’s spin of the conflict. With
the AGO ultimately deciding that the government had acted in accordance
with managerial and efficiency standards, public opinion shifted against
the teacher’s union (Hammer and Alphonso 2013).
An impact such as this one should not be taken lightly. The AGO acts a
public watchdog for pointing out government shortcomings and areas
requiring increased accountability, and the media is quick to tout its sup-
port for this role. In the news scans conducted for this research paper, no
negative media reports on the AGO were identified. All the same, this link
between political controversy and VFM choice needs a great deal more
attention from researchers in a comparative framework. The evidence
would seem to show that the AGO is far more reactive in making its choice
of audits (and therefore more political, in a limited sense) than it is proac-
tive in anticipating risk.

Hypothesis 3: the impact of the individual

Auditor General
There seems to be little substantial difference based on the tenure of each
Auditor General as to the choices made. For instance, VFM audits in the
years of Erik Peters featured 5.5 reports each on the health sector and the
mixed bag of “others.” The same trend could be observed in the work per-
formed by the Auditor General’s office under Jim McCarter. However, the
education category attracted relatively more attention under McCarter, as
did environment and energy. Erik Peters paid relatively more attention to
social services and justice.
Given the general consistency, there is little evidence that personal
Auditor General choice was a factor. The small differences are more driven
by controversies or government requests for special inquiries. For instance,
the ministry of Child Services was particularly under fire at the turn of the
century while energy issues were more prevalent at the end of the period
under study (the issue of gas-generated energy plants, in particular, drove
a number of AGO studies). This snapshot is limited because there were
only two individuals at the helm of the AGO’s office for most of this period.
All the same, the study invites comparison with other jurisdiction over a
lengthier period.
VFM AUDIT CHOICE IN THE NEW PUBLIC MANAGEMENT ERA 105

Conclusion
As decentralization of government progressed, the AGO’s role as the pre-
eminent accountability watchdog has become practically uncontested.
Audit choices thus become a key concern and the problem must be asked:
what motivates the AGO’s choice of targets in conducting VFMs? Are they
made based on strategic management concerns, as the AGO claims on its
website? Using the data available, this study posited three key hypotheses
as the bases for choices: the professional views of the individual AGOs; the
budgetary importance of the various ministries/agencies; or triggered by
political controversies. The evidence shows no purely rational incentive to
audit ministries based on budget allocations as VFM audit proportions did
not align with budget proportions. Moreover, there is no evidence that a
change in the chief Auditor had a material impact in the choices made.

The evidence shows no purely rational incentive to audit

ministries based on budget allocations as VFM audit
proportions did not align with budget proportions.
Moreover, there is no evidence that a change in the chief
Auditor had a material impact in the choices made

There is some evidence that political controversies did influence the

AGO’s selection process when it comes to ministries with high policy turn-
over, but the relationship is more correlational than causational. The health,
finance, environment and energy, and “other” aggregate sector categories
were unevenly reviewed between new/special and follow-up reports.
Lastly, the influence of the media on which programs were scrutinized
more intensely is not consistent across all sectors, but does present itself
sporadically, such as the VFM report on ORNGE Ambulance Air Services
in 2012. There is evidence to support the hypotheses that the AGO selects
programs and ministries on which to conduct its VFM audits based on
political and incidental factors, but it is limited to a few sector categories.
These are reaction-audits, certainly not based on anticipated risk.
This study leaves open the possibility that other hypotheses could
explain audit choice. It could be, for instance, that the AGO has focused on
issues other than programs (for example, governing issues, regulatory ini-
tiatives, outsourcing, and partnerships) that may explain the profile of aud-
ited ministries. Audits may also have been triggered by internal concerns
that have not been captured by the media or by AGO reports. Finally, it
could be argued that VFM audit selections are more driven by a history of
concerns (that is, that some ministries or programs are more vulnerable to
failure and thus a regular concern of Auditor General of the province). If
106 SANA ADI, PATRICE DUTIL

so, these concerns have not been articulated by the Auditor General of the
province, nor by the Public Accounts Committee of the Ontario legislature.
Finally, the “other” category may need to be refined for a closer examina-
tion and other distinctions.
Regardless, the enormous gaps in what gets audited show that Westmin-
ster, at least in the Ontario case, is still far from realizing the comprehen-
siveness of government audit, let alone an “audit society.” This policy
conundrum may require a case study approach to document and analyse
the sectors under investigation by the AGO in order to reveal trends. Of the
three central hypotheses of this paper, the literature appears to only sup-
port the rational argument for the AGO’s interests as something of a risk-
reducing mechanism for government (Power 1997; Hopwood and Miller
1994). Furthermore, any links between media coverage and the AGO’s
reports have proven to be minimal and tend to reflect the perspective that
the AGO influences mainstream discourse, not vice versa. In the words of
Sharon Sutherland, the media is “following the auditors’ script” (Suther-
land 1986). While this may often be true in the case of revealed “scandals”
in misappropriations of money, there is little evidence that auditors follow
media leads unless a major scandal is uncovered.

Regardless, the enormous gaps in what gets audited

show that Westminster, at least in the Ontario case, is
still far from realizing the comprehensiveness of gov-
ernment audit, let alone an “audit society”

This study reveals that the Auditor’s Office in Ontario has not followed
a particular risk-based or administrative-innovation strategy in its focus on
VFM except to track its operations on a fixed formula that has been consis-
tent for the entire period under study. The Auditor’s Office in Ontario
sticks to “what’s been done before” of what ministries will be examined,
and to what degree. It has proven to be remarkably path-dependent and
generally unresponsive to the budget importance of various ministries, or
to their news-worthiness. As such, many ministries are left relatively
under-evaluated, while others are systematically over-examined.
These findings should interest both practitioners of audit as well as schol-
ars (see Funkhouser 2011). At a minimum, it calls on auditors to justify the
reasons for their choices of audit and to demonstrate that, over time, they are
indeed contributing to making audit as universal as possible. For scholars
the propositions of this article call for more research. How far back does the
pattern identified in this relatively narrow study go? How does it compare
with other jurisdictions? Many new lines of inquiry on the office of Auditors
General in the Westminster system are thus urgently needed.
VFM AUDIT CHOICE IN THE NEW PUBLIC MANAGEMENT ERA 107

References
Alwardat, Y. A., A. Benamraoui, and A. Rieple. 2015. “Value for money and audit practice
in the UK public sector.” International Journal of Auditing 19: 206–217.
Bowerman, M., H. Raby, and C. Humphrey. 2000. “In search of the audit society: Some evi-
dence from health care, police and schools.” International Journal of Auditing 4: 71–100.
Brodtrick, O. 2004. “How does the Auditor General’s office see itself?” Canadian Public
Administration 47 (2): 225–42.
Desautels, D. 2001. Reflections on a Decade of Serving Parliament. Report of the Auditor General
of Canada to the House of Commons: Ottawa, Canada.
Evans, Bryan, and Charles W. Smith. 2015. “The transformation of Ontario politics: The long
ascent of neoliberalism.” In Transforming Provincial Politics: The Political Economy of Cana-
da’s Provinces and Territories in the Neoliberal Era, edited by Bryan M. Evans and Charles
W. Smith. Toronto, Canada: University of Toronto Press, pp. 3–18.
Funkhouser, M. 2011. “Accountability, performance and performance auditing: reconciling
the views of scholars and auditors.” In Performance Auditing: Contributing to Accountability
in Democratic Government, edited by Jeremy Lonsdale, Peter Wilkins and Tom Ling. Chel-
tenham: Edward Elgar Publishing Limited.
Gendron, Y., D.J. Cooper, and B. Townley. 2001. “In the name of accountability: State audit-
ing, independence and new public management.” Accounting, Auditing and Accountability
Journal 14 (3): 278–310.
Good, D.A. 2014. The Politics of Public Money. Toronto, Canada: University of Toronto Press.
Hammer, Kate, and Caroline Alphonso. 2013. “Ontario teachers divided over return to
extracurriculars.” The Globe and Mail, 24 February. https://www.theglobeandmail.com/
news/national/education/ontario-teachers-divided-over-return-to-extracurriculars/
article9013649/
Hatherly, D.J. 1999. “The future of auditing: The debate in the UK.” The European Accounting
Review 8 (10): 51–65.
Hopwood, A. G., and P. Miller. 1994. Accounting as social and institutional practice (Vol. 24).
Cambridge, UK: Cambridge University Press.
Humphrey, C., and D. Owen. 2000. “Debating the ‘power’ of audit.” International Journal of
Auditing 4: 29–50
Jeppesen, K.K., T. Carrington, B. Catas us, Å. Johnsen, K. Reichborn-Kjennerud, and J.
Vakkuri. 2017. “The strategic options of supreme audit institutions: The case of four Nor-
dic countries.” Financial Accountability and Management 33: 146–170.
Johnsen, A., P. Meklin, L. Oulasvirta, and J. Vakkuri. 2001. “Performance auditing in local
government: An exploratory study of perceived efficiency of municipal value for money
auditing in Finland and Norway.” The European Accounting Review 10: 583–99.
Keen, J. 1999. “On the nature of audit judgements: The case of value for money studies.”
Public Administration 77: 509–525.
Lapsley, I., and C.K.M. Pong. 2000. “Modernization versus problematization: Value-for-
money audit in public services.” European Accounting Review 9 (4): 541–67.
Leclerc, G. 1996. Accountability, performance Reporting, Comprehensive Audit - An Integrated Per-
spective. Ottawa, Canada: Canadian Comprehensive Audit Foundation.
Leeuw, F. 2011. “What the critics say. . .on the effects, the lack of effects, and perverse effects
of performance audits.” In Performance Auditing: Contributing to Accountability in Demo-
cratic Government, edited by Jeremy Lonsdale, Peter Wilkins and Tom Ling. Piscataway,
NJ: Transaction Publishers. 231–247.
Lindeberg, Tobias. 2007. “The ambiguous identity of auditing.” Financial Accountability and
Management 23 (3): 337–50.
Mayne, J. 2004. “Reporting on outcomes: Setting performance expectations and telling per-
formance stories.” Canadian Journal of Program Evaluation 19 (1): 31–60.
108 SANA ADI, PATRICE DUTIL

—— 2006. “Audit and evaluation in public management: challenges, reforms, and different
roles.” The Canadian Journal of Program Evaluation 21(1), 11–45.
—— 2010. “Performance Auditing: Cozy, comfortable and in need of challenge.” Optimum
Online 40(3), 1–6.
McGlashan, L. 2011. Performance Auditing at The Office of the Auditor General of Canada: Beyond
Bean Counting. Ottawa, Canada: Library of Parliament.
Morin, D. 2003. “Controllers or catalysts for change and improvement: Would the real value
for money auditors please stand up?” Managerial Auditing Journal, 18(1), 19–30.
—— 2004. “Measuring the impact of value-for-money audits: A model for surveying aud-
ited managers.” Canadian Public Administration 47 (2): 141–64.
—— 2008. “Auditors General’s universe revisited – An exploratory study of the influence
they exert on public administration through their Value for Money audits.” Managerial
Auditing Journal 23 (7): 697–720.
—— 2014. “Auditors general’s impact on Administrations: A pan-Canadian study
2001–2011.” Managerial Auditing Journal 29 (5): 395–426.
Morin, D, and M. Hazgui. 2016. “We are much more than watchdogs: The dual identity of
auditors at the UK national audit office.” Journal of Accounting and Organizational Change,
12 (4), 568–89.
Office of the Auditor General of Ontario. 2015. History of Our Office. Toronto, Canada.Office
of the Auditor General of Ontario.
—— 2018. “What We Do” http://www.auditor.on.ca/en/content/aboutus/whatwedo.html
Last verified on 30 January 2018.
Power, M. 1997. The Audit Society: Rituals of Verification. New York: Oxford University Press.
—— 2000. The Audit Society—Second thoughts. International Journal of Auditing 4 (1),
111–19.
Richards, D., and M. Smith. 2006. “The tensions of political control and administrative
autonomy: from NPM to a reconstituted Westminster model.” In Autonomy and control:
Coping with agencies in the modern state, edited by Tom Christensen and Per Laegreid. Lon-
don: Edward Elgar Publishing, pp. 181–202.
Roberts, S., and C. Politt. 1994. “Audit or evaluation? A National Audit Office VFM study.”
Public Administration 72: 527–549.
Saint-Martin, D. 2004. “Managerialist advocate or ‘control freak’? The Janus-faced Office of
the Auditor General.” Canadian Public Administration 47 (2), 121–40.
Savoie, D.J. 1995. “Who is auditing the Auditor-General?” The Globe and Mail, 25 August.
Sharma, N. 2007. “Interactions and interrogations: Negotiating and performing value for
money reports. Financial Accountability and Management 23: 289–311.
Sterck, M. 2007. “Impact of performance audit.” In Budgeting, Accounting and
Auditing for Results – Towards Integrated Financial Management, edited by V. Conings, M.
Sterck and G. Bouckaert. Leuven: Steunpunt Bestuurlijke Organisatie Vlaanderen. http://
steunpuntbov.be/eng/pub/s0405014.htm
Sutherland, S.L.1986. “The politics of audit: The federal Office of the Auditor General in
comparative perspective.” Canadian Public Administration 29 (1): 118–48.
Taft, J. 2016. “From change to stability: Investigating Canada’s Office of the Auditor Gener-
al.” Canadian Public Administration 59 (3): 467–485.
Talbot, C., and J. Wiggan. 2010. “The public value of the National Audit Office.” Interna-
tional Journal of Public Sector Management 23 (1): 54–70.
Van Loocke, E., and V. Put. 2011. “The Impact of Performance Audits: A Review of the
Existing Evidence.” In Performance Auditing: Contributing to Accountability in Democratic
Government, edited by Jeremy Lonsdale, Peter Wilkins and Tom Ling. London: Edward
Elgar Publishing, pp. 175–208.