Executive Summary

The Global Case Management System (GCMS) is a multi-year project that will replace
several existing systems at Citizenship and Immigration Canada (CIC) and the
Canada Border Services Agency (CBSA). The goal of the project, which began in
fiscal 200001, is to provide CIC with an automated, highly integrated case
management tool to support its global business network and provide enhanced endto-end client services. When complete, GCMS will support more than 7,000 users at
CIC and the CBSA in some 125 points of service in Canada (offices, call centres,
processing centres, citizenship courts and ports of entry) and more than 100
missions abroad.
In September 2004, the first phase of the GCMS project was implemented with the
go live of the Citizenship component (Deployment 1). Although this component was
successful from a technological standpoint, some initial operational difficulties were
encountered, but they have largely been resolved. Consequently, when the project
sought to amend the Effective Project Approval (EPA) in the spring of 2005, an audit
of the GCMS project was requested as part of the process.
Our audit sought to examine the progress made since GCMS Deployment 1 and the
next phase of GCMS. We examined the project management processes and controls
for ensuring both data integrity and the protection of sensitive information. Our
observations cover the period from September 2004 to October 2005.
Our overall conclusions are that the GCMS project management processes are sound
and that they will increase the likelihood of success by ensuring the existence of an
effective management control framework. Furthermore, the audit team found that
adequate controls are in place to support data integrity and protect sensitive
information assets.
The following report provides the audit teams observations and recommendations,
as well as the GCMS project teams responses and action plan to address these
recommendations.

1. Introduction
The Internal Audit and Disclosures (IAD) Branch of Citizenship and Immigration
Canada (CIC) was asked to conduct a focused system-under-development audit of
the Global Case Management System (GCMS) project. The IAD audited the project
management processes and the control measures in place for data integrity and
sensitive information assets, to assess their adequacy for the successful
implementation of the project.
The audit was conducted in accordance with the Government of Canadas policy on
internal audits, as well as auditing standards prescribed by the Institute of Internal
Auditors.

1.1 Background

GCMS is a multi-year project that will replace several of the business systems of CIC
and the Canada Border Services Agency (CBSA) with an integrated, case
management-based set of applications and components that will support both
organizations client operations.
The GCMS project received Preliminary Project Approval on March 1, 2001, with an
initial budget of approximately $195 million (T.B. 828697). In view of the assessed
level of risk and the scale of the project, CIC was instructed to manage GCMS as a
Major Crown Project and to confirm the viability of using a commercial off-the-shelf
(COTS), Client Relationship Management package to provide much of the required
case management functionality.
Because of a variety of factors, many of which were outside the control of CIC, the
selection of a vendor and the awarding and signing of the contract were delayed by
nine months. As a result of this delay, it was April 2003 before the project team had
access to the COTS product and technical experts required to build and deploy
GCMS. Later in 2003, an amended Effective Project Approval was approved by
Treasury Board ministers (T.B. 830875) and the budget increased to approximately
$200 million.
Since the project was granted preliminary approval, it has had to deal with a number
of external and internal events that have directly affected the project and that have
necessitated adjustments to the project implementation strategy. These events
include the following:

the events of September 2001, which forced countries to reconsider the tools
they were using to deal with threats while, at the same time, not unduly
impeding the legitimate movement of people;

the continued and growing interest in Canada as a destination for immigrants

and temporary residents (tourists, foreign students, temporary workers);
the implementation of the most comprehensive reforms to the immigration
legislation in more than 30 years, including the introduction of the secure
permanent resident card; and
the creation of the Canada Border Services Agency.

The project has conducted several independent readiness reviews (July 2000, June
2001 and December 2002) as well as two Independent Verification and Validation
reviews (August 2002 and July 2003) to address various issues and concerns about
the project. The results of these reviews were provided to the Treasury Board
Secretariat (TBS) through formal reporting (September 2002) and in regular monthly
reports.
Treasury Board ministers also requested that the Chief Information Officer Branch
(CIOB) of TBS engage the services of an independent third-party monitor to report
on the progress of the project, with particular attention to the vendor relationship.
Two monitors were engaged in October 2003. In January 2004, they provided their
report to TBS. A subsequent follow-up report was provided in March 2005.
Reports prepared by the external Treasury Board monitor highlighted the fact that
senior management at both CIC and the CBSA enthusiastically and unequivocally
supported the GCMS project. The report also noted that there was a high degree of

awareness and engagement within CIC and the CBSA user communities, as
demonstrated in periodic surveys designed to take the pulse of the two
organizations.
The project has also served as a catalyst for business transformation. In identifying,
defining and formalizing business requirements, the project has had to re-examine
existing business processes and to develop consensus among departmental users.
The result is a more consistent, standardized set of practices aimed at promoting
greater uniformity, increased efficiency and improved client-centred service across
both organizations.
CICs business includes over 1.5 million client transactions a year, approximately
two-thirds of which occur overseas in environments that can be characterized as
challenging in terms of their complexity and the local infrastructure.
When completed, GCMS will support more than 7,000 users at CIC and the CBSA in
some 125 points of service in Canada (offices, call centres, processing centres,
citizenship courts and ports of entry) and more than 100 missions abroad.
GCMS is a critical element of a larger, client-centred service and business
transformation vision that seeks to maximize the benefits associated with its
citizenship and immigration programs in the years to come. By integrating the
business functionality and information that now reside within more than a dozen of
CICs legacy systems built over the last three decades, the project aims to eliminate
redundant data, improve the consistency and quality of that data and contribute to
streamlining operations. It should also allow both CIC and the CBSA to further
improve the delivery of their programs and services.
In September 2004, the first phase of the GCMS project was implemented with the
go live of the Citizenship component (Deployment 1). The project continues to
meet the expectations of both users and IT experts and has proven to be a powerful
tool that can be customized to meet the specific requirements of CIC and the CBSA.
Although the Citizenship component was successful from a technological standpoint,
some initial operational difficulties were encountered. These were attributable to data
conversion problems, a steep learning curve for users, and a higher than anticipated
number of system defects. These problems are being addressed as the project
moves toward the next phase of implementation. The system has become more
stable, and production is approaching predeployment levels.
As part of Deployment 1, the project team engaged in several lessons-learned
exercises. These exercises identified and documented areas for improvement such as
accountabilities, requirements development, communications, project planning and
scheduling, resource planning, and project management control and tracking.
Learning from these exercises, the GCMS project team prepared detailed action plans
and began to adopt new practices.
In the spring of 2005, the GCMS project team began preparing a submission to
Treasury Board ministers through TBS for amendments to the Effective Project
Approval (EPA) for the GCMS project. As part of this process, the TBS expressed a
need for an audit of the project.

The type of audit that TBS considered most suitable was a system-underdevelopment (SUD) audit. In general, TBS was looking for a focused SUD audit that
would provide reasonable assurance that the GCMS project was on a solid foundation
as it moved forward.
In June 2005, the terms of reference were developed in preparation for this audit,
which began in July 2005.

2. Audit Objectives
The audit had two objectives:

to assess the soundness of the project management processes; and

to determine the appropriateness of controls to ensure data integrity and

protect sensitive information assets.

3. Audit Scope
The audit scope covered the progress made since GCMS Deployment 1. We
examined the project management processes and controls for ensuring both data
integrity and the protection of sensitive information. Our observations cover the
period from September 2004 to October 2005.
We did not examine processes or controls in effect prior to September 2004.
Furthermore, an examination of the contract between the Government of Canada
and the principal vendor for the GCMS project was outside the scope of this audit.

4. Audit Criteria
We derived our assessment criteria from a number of reasonable and credible
standards of performance and control relevant to this engagement. The sources of
criteria for this audit included the following:

Project Management Body of Knowledge (PMBOK) the Project Management

Institutes (PMI) standards document for project management knowledge and
practices;

Control Objectives for Information and Related Technology (COBIT) the IT

Governance Institutes framework for IT practices;
Various Treasury Board Secretariat policies (e.g., TBS Enhanced Management
Framework);
Various CIC departmental policies; and
The Management Control Framework for CIC Audit Activities.

The criteria for the two audit objectives were as follows:

The project management processes increase the likelihood that the project
will succeed by ensuring that an effective management control framework
exists.

Project planning techniques are sound.

Project monitoring, tracking and control techniques are appropriate.
There are processes to enable the project to achieve its overall
business objectives.
o There are improvements to the project management processes applied
for Release 2.
o The contract and subcontract management processes are adequate.
o Project requirements management techniques are adequate.
o Change management procedures are adequate.
o Techniques for integration management applied with respect to the
project management offices (PMO) linkages to project authorities
(e.g., Treasury Board) are sound.
The controls to ensure data integrity and the protection of sensitive
information are adequate.
o Procedures to support data conversion and to verify and maintain the
integrity of converted data are adequate.
o Procedures for protecting sensitive information have been incorporated
into the project and are adequate.
o
o
o

5. Methodology
The audit began in July 2005, and we carried out our examination from July to
October 2005. This involved the following activities.

interviews;

reviewing documentation;
reviewing best practices against the practices of recognized organizations
such as the PMI s PMBOK and the IT Governance Institutes COBIT;
a detailed examination of selected project management processes; and
audit testing.

As part of the audit, we interviewed 44 people, including key executives, GCMS

project members and other key stakeholders. We also obtained relevant
documentation to validate the information gathered during interviews (see appendix
A for a list of interviewees).
The audit team also examined documentation and conducted detailed testing to
determine whether specific processes or controls were in place. In addition, the team
conducted a literature review of readily available best practices to identify any gaps
in the project management processes or controls under review.
Detailed testing included the following identified areas:

Detailed master schedule;

Exception reporting;
Change management;
Risk management;
Dashboard reporting;
Task authorization/task authorization requests validation;

NON-COTS contracts;
Asset purchases; and
Security clearances.

6. Conclusions
We have concluded that the GCMS project management processes are sound, and
that they will increase the likelihood of the projects success by ensuring that an
effective management control framework exists. Furthermore, the audit team found
that adequate controls were in place to ensure data integrity and to protect sensitive
information.

7. Observations and Recommendations

7.1 Project management processes
The audit objective was to assess the soundness of the project management
processes in place and their contribution to the projects success by ensuring that an
effective management control framework exists. The project management processes
that we examined as part of this audit included the following areas: project planning;
project monitoring, tracking and control; contract and subcontract management;
requirements management; integration management; and change management. The
following paragraphs present our observations and, as applicable, recommendations
for each area.

7.1.1 Project planning

In the context of project planning, the audit team reviewed a number of elements.
Our observations in this area are as follows.

The business case

Under project planning, we expected to find that the GCMS project team had
developed a business case that was comprehensive and complete, and that identified
and justified the GCMS project and indicated how it related to CIC and governmentwide priorities. At a minimum, a business case should identify:

the opportunities for improvement;

the benefits of such an undertaking;

the technical solution at a high level;
indicators against which to measure any improvement in program
performance;
costs (up-front, direct ongoing, indirect and potential user/client costs); and
the risks, and steps to be taken to manage those risks.

The audit team found that the GCMS business case had covered all significant
elements.

Project charter
We found that the GCMS project team had a project charter that defined the scope of
the project. It also set out the overall project management framework and standards
to be used. The project charter also clearly defined who the internal and external
stakeholders were, described the roles and responsibilities of the members of the
project team and established the overall project governance structure (e.g.,
accountabilities and project authorities). Furthermore, we found that the project
charter had set out the management commitments (human and financial resources,
time frame and deliverables).
The team found that the GCMS project charter had been continuously updated
throughout the life cycle of the project to reflect various project changes, and that all
significant elements had been addressed in the project charter.

Project approvals
The audit team expected to find that the project had received the appropriate
approvals from not only CIC, but TBS as well. The TBS preliminary project approval
was granted on March 1, 2001 (T.B. 828697). Effective project approval was granted
on January 31, 2002 (T.B. 829502) and amended on October 9, 2003 (T.B. 830875).
During this audit, CIC sought and obtained a further amendment to the EPA. Finally,
we noted that the GCMS project senior management within CIC has demonstrated,
and continues to demonstrate, strong support for the project.

Project plan
The audit team expected to find a project plan that included:

how system requirements would be managed;

how negotiation of resource commitments would be handled;

a software life cycle with predefined stages;
the key processes and deliverables for the project;
authorization to start work and key team members assigned;
estimate of the size and complexity of the software;
estimate of product resource requirements;
detailed schedules;
commitment and approvals from within CIC; and
reporting requirements.

The master project schedule (MPS) is the primary mechanism used for project
planning within GCMS. Our examination showed that the MPS contained all the
details listed above and that it reflected the GCMS project charter.
The MPS is maintained by the PMO with input from individual project teams who are
responsible for preparing subteam schedules that feed into the MPS. Weekly
meetings are also held with the project team's project control officers to discuss the
projects progress as it applies to their subschedules.

To validate the information contained within the MPS, the audit team conducted the
following audit tests (using the MPS dated July 15):

tests to validate the internal consistency of reported dates in subteam

schedules; and

tests to map tasks contained in the MPS back to the subteam schedules and
compare reported information.

We noted that the level of detail and consistency varied between the different
subteam schedules. Consequently, the audit team had difficulty mapping tasks
contained in the MPS back to subteam schedules and found that the level of difficulty
in mapping tasks varied between subschedules. Without a better link between the
MPS and the subschedules, the project cannot be sure that all material items are
included in the MPS. Given the MPS importance in planning, we are concerned that
stronger links do not exist. The project is aware of these facts and was in the process
of revalidating all schedules at the time of the audit. We were informed that efforts
are also being made to realign the schedules to improve the links between the MPS
and its subschedules.
In our view, the project needs to ensure that all project team members commit to
uniformly applying the project management planning standards. We believe that
doing so will improve the quality and consistency of the information feeding into the
MPS. Strengthening of the MPS should also strengthen the timeliness of information
and decision making within the project.
Recommendation 1
The GCMS project should ensure the consistent application of project management
planning standards (GCMS Schedule Management and Development Standards)
developed by the PMO.
GCMS management response
Agreed.

Project reviews
The audit team expected to find a process for reviewing and revalidating the major
planning deliverables (the business case, project charter and plan) for the project.
The project team has completed a thorough review and analysis of Deployment 1
activities and has applied lessons learned to develop a revised approach and plan for
completing the GCMS.
The project has carried out the following reviews:

Independent Readiness Reviews (July 2000, June 2001 and December 2002);

Independent Verification and Validation Reviews (August 2002 and July

2003);
Independent TBS Monitor Reviews (February 2004 and March 2005);

GCMS Project TeamGates 1 and 2: Lessons Learned (November 2003 and

December 2004); and
GCMS Review of Project Response to Deployment 1: Lessons Learned (June
2005).

The audit team found that the level of project review was generally adequate for the
size and scope of the project.

Governance and organizational structure

The project has the following governance committees:

Change Control Board (CCB)

Design Review Board (DRB)

Design Review Team (DRT)
Executive Committee (ExCOM)
Executive Management Team (EMT)
Group of Approvers (GA)
Integrated Management Committee (IMC)
Operational Validation and Implementation Team (OVIT)
Project Management Board (PMB)
Risk Management Board (RMB)
Senior Project Advisory Council (SPAC)
Senior Level Advisory Committee (SLAC)

We examined the terms of reference for each of the governance committees and
found them to be complete. A review of agendas, supporting materials and records
of decisions indicated that they had met the requirements of government policies and
CIC standards. The audit team attended (at random) various governance meetings
(e.g., SLAC, RMB, CCB, PMB and EMT) and found that each was chaired at the right
level and that the composition of the committees was appropriate, given their terms
of reference.
We also found that the project had an appropriate organizational structure in place.
Key positions have been identified with clearly documented and communicated roles
and responsibilities. As one of its primary objectives, the IMC is dedicated to
identifying and resolving staffing issues for the project in a timely fashion.
Overall, we found the governance and organizational structure of GCMS to be
comprehensive and in compliance with applicable government directives (e.g., the
Major Crown Projects Policy and the TBS Enhanced Management Framework).

Process plans
The audit team expected to find process plans for the following:

quality assurance;

risk management;
communications;

performance management;
HR management;
procurement management;
requirements management;
IM/IT;
testing;
roll-out; and
training.

Overall, the audit team found that plans had been completed for each of the areas
noted above. Process plans are continually being improved as demonstrated by the
incorporation of lessons learned from Deployment 1. Significant strides have been
made in bolstering areas identified in lessons-learned exercises. In particular, these
include:

a formal accountability framework (approved April 14, 2005);

the appointment of a deputy project manager;

tightened project management processes (e.g., for HR management);
improved management reporting (e.g., improvements to the GCMS
dashboard); and
stronger control over project resourcing (e.g., IMC process).

7.1.2 Project monitoring, tracking and control

In the area of project monitoring, tracking and control, the audit team reviewed a
number of elements, as noted below.

Data collection and tracking requirements

The audit team expected to find that baseline estimates for activities set out in the
project plan (including milestones) had been established and accepted by the project
team. The GCMS project team has identified, defined and documented major
baseline activities, milestones and data. Updates to the project plan approach and
associated deliverables reflect the intent of the business case. The evolving project
charter is consistent with these adjustments to baseline activities.
The audit team found the project tracking and oversight process to be consistent
with TBS Enhanced Management Framework principles.

Planned vs. actual performance

Processes have been identified to:

track actual performance;

compare actual with planned performance (e.g., track deviations from plan);
identify when and what adjustments will be needed; and
determine the impacts of any adjustments and the subsequent changes
needed (e.g., additional resources) to close the gap.

The audit team completed a review of the minutes of the steering committee
(discussed below) and other meetings to determine if performance was reviewed
regularly and adjustments made as required. The audit team found that planned vs.
actual performance was reviewed regularly and that the reasons for any deviations
from planned performance were documented and explained concisely. However,
ownership of these deviations is weak. This is discussed in greater detail in section
7.1.3 (Monitoring of service delivery by a third party).

Financial tracking
The audit team expected to find that financial tracking was being conducted and that
the results were reported. In particular, we expected the following:

Budgeted vs. actual and forecast is analysed, documented and reported on a

regular basis.

Financial data are accurate, complete and produced in a timely fashion.

High-level financial tracking information from the PMO is consistent with
information at the project team level.
Financial coding structure is consistent with CIC standards.
Financial authorities as established in the project charter and CIC are applied.

Dedicated PMO staff who work in collaboration with the CIC Finance Branch track and
control GCMS financial information. The audit team conducted a sample testing of
asset acquisitions to test for compliance with the Financial Administration Act. Our
examination in this area found that the processes were functioning as intended.
Furthermore, a review of detailed financial data and supporting documentation for
the month of July 2005 found that internal financial controls were adequate.

Tracking performance and results

The audit team expected to find that performance and results were documented and
communicated within the PMO and were continuously adjusted to reflect project
improvements and the tracking of all issues through to completion. In particular, we
expected to find the following:

Project logs and status reports are prepared and updated regularly.

A distribution list of concerned parties is maintained.

Issues and problems are identified, documented and tracked to closure.
There is a clearly understood process for resolving problems when they arise.

The audit team reviewed various logs (e.g., issue, decision, action and risk) and
found them to be adequate.
In addition, as part of its audit, the audit team tested one periods GCMS dashboard
(May 2005) to assess the accuracy of the information it contained. Our examination
revealed that the dashboard reflected the information available within the project.

Risk management

The audit team tested the risk management tracking processes (MayJune 2005) to
ensure that the process worked as intended and was in accordance with generally
accepted practices.
The audit team expected to find a process for ensuring that project risks were
identified, assessed and documented and, in particular, that the following elements
were in place for the GCMS project:

a systematic risk assessment framework;

a risk assessment approach that allows for regular updates;

risk assessment procedures that identify both external and internal risk
factors;
procedures for monitoring changes in risks;
risk assessment documentation;
risk mitigation strategies identified and documented; and
involvement of stakeholders in identifying project risks.

We found that the GCMS project had mechanisms that identified, documented and
tracked risks. An initial risk assessment was completed in 2001 and has been
regularly updated throughout the life of the project. A comprehensive Threat and
Risk Management Assessment was completed in December 2004. Overall, we found
that the monitoring of risks associated with the project was well documented and
tracked.

Meetings
The audit team expected to find that meetings were held on a regular basis to make
decisions and to review the projects status, performance and risks.
The audit team found that since Deployment 1, team meetings have been focusing
on providing a better understanding of the status, costs, time frames and risks
associated with the project. A review of agendas, minutes and records of decisions
(ROD) for ExCOM, SLAC, SPAC, PMB and EMT conducted by the audit team found
meetings were taking place, major activities were being updated and risks were
being tracked and followed up on.

Steering committee
The audit team expected to find that a steering committee had been established and
was operating and, in particular, that individuals at the appropriate levels had been
identified and were participating in the committee.
Our audit found that the steering committee was functioning as intended. ExCOM is
the senior departmental forum for monitoring progress, reviewing issues and making
decisions regarding alternative approaches and priorities for the GCMS project. As
such, it serves as GCMS steering committee. ExCOM meets on a monthly basis.
Steering committee members include:

Deputy Minister (Chair)

ADM, Policy and Program Development

ADM, Centralized Services Delivery and Corporate Services

ADM, Operations
ADM, Strategic Directions and Communications
Assistant Deputy Attorney General
DG, Human Resources

The GCMS project team has also formed the Senior Level Advisory Committee to
provide additional perspective and practical advice to the GCMS Executive
Management Team. SLAC meets on a monthly basis. Its members are:

Project Sponsor ADM, Policy and Program Development

ADM, Centralized Services Delivery and Corporate Services

ADM, Operations
Project Leader, DG, Business Solutions
DG, IMTB
Executive Director, IM/IT
Executive Director, PMO
Executive Director, Business Requirements and Application Functional Design
Executive Director, Deployment, User Acceptance Testing (UAT) and Training
Delivery Manager
Chief, GCMS Finance
IndependentmMonitor, TBS
VP, Enforcement, CBSA
Director, Science Innovation and Technology, CBSA

The audit team reviewed the minutes and presentations of ExCOM and SLAC and
found that they had documented key decisions and directions. Roles and
responsibilities, together with terms of reference, have been clearly defined and
documented for each committee.

Communication strategy/plan
We expected to find that a communication strategy/plan had been developed and
implemented to communicate the projects results and progress to external
stakeholders.
We found that the GCMS project team used the following key communication tools to
reach its various audiences:

the GCMS electronic newsletter Spotlight on GCMS;

Taking the Pulse employee surveys;

CICs internal website CIC Explore;
GCMS portal;
regional visits;
executive staff meeting: weekly GCMS topical briefings;
regular status updates to ExCOM, SLAC, SPAC;
TBS Monthly Operational Reports (MOR); and
GCMS dashboard.

The project regularly provides timely updates to all stakeholders (internal and
external). We understand that the GCMS project team, as part of its activities for
Release 2, has incorporated specific communication tasks into the project
management plan to prepare users for Release 2. The goal of these tasks is to bring
users to a stage where they are committed to the project and ready for the changes
it will bring.

Process for managing documents

The audit team expected that the GCMS project team would have a sound document
management process. We found that the GCMS project maintained records as part of
the projects historical records. Examples include:

versions of documents (e.g., Business Case, Project Charter);

logs (decision logs, issue logs, risk logs, change logs);

risk assessments; and
minutes, RODs and presentations.

The project team uses electronic tools (LiveLink GCMS Portal and ClearCase) to
electronically house documents relating to the project. The audit team found that the
GCMS project team had a sound document management process and that it was
functioning as intended.

7.1.3 Management of contracts and subcontracts

In examining this area, the audit team reviewed software acquisition plans, whether
the experience of staff was adequate to support software acquisitions, the control of
consultants, monitoring of service delivery by a third party, the independence of the
PMO and principal vendor, and the extent to which the use of subcontractors was
documented and controlled. Our observations in these areas are as follows.

Software acquisition plans

Plans for contracting and acquiring software include:

the acquisition strategy;

the method for soliciting bids;

constraints related to acquisition;
acquisition resource requirements;
roles and responsibilities;
cost and schedule estimates;
contract tracking, oversight and evaluation plans;
acquisition risk management; and
life cycle support.

The audit team found that the software acquisition plans for the GCMS project
followed CICs and Public Works and Government Services Canadas (PWGSC)
standard procurement policies. In particular, CIC has a number of software licences
that it has made available to the GCMS project. Documented procedures also exist

for acquiring additional software. The approval process includes the project as well as
the Information Management and Technologies Branch within CIC.
In addition, most of the procurement of hardware and software to establish the
production environment had been completed, and the domestic production
environment was operating in a high-availability, fail-over configuration mode.
Upgrades to Foreign Affairs Canadas (FAC) international network (MITNET) and
server/desktop environment (SIGNET) were proceeding within the parameters
required by GCMS. Project team members continue to work closely with FAC in
efforts to minimize the amount of infrastructure required for the GCMS Release 2.
The original procurement strategy (November 2001) has been updated to reflect the
projects software requirements. In general, the audit team found that the software
acquisition plan included all significant elements.

Staff experience
The audit team expected to find that experienced acquisition, contracting and
management staff were supporting the software acquisition for the project team. In
particular, we expected that key staff would have appropriate knowledge and
expertise in:

evaluating the technical aspects of the proposed system;

CIC information management and information technology (IMIT)

requirements (architecture, interfaces);
federal and CIC contracting requirements, costing methodologies and tools;
and
end-user requirements.

The GCMS project team had a contracting expert from PWGSC assigned to provide
advice. In addition, the project team had support from CIC internal contracting
resources to facilitate the contracting process. IMTB staff with experience in
evaluating and assessing technical solutions have been assigned to the project.
Accordingly, we found that the experience of the acquisition, contracting and
management staff assigned to the GCMS project to support software acquisitions was
adequate.

Control of consultants
The audit team expected to find policies and procedures for controlling the activities
of consultants assigned to the GCMS project in order to protect the organizations
information assets. In particular, we expected that the GCMS project team would
ensure that:

contractual requirements are developed, managed and maintained;

terms and conditions are standardized and reviewed by legal counsel;

contractual commitments are traceable and verifiable;
cost and schedule estimates are independently reviewed; and
security screening of key contractors is performed in conformity with CIC
policy.

The audit team found that over time, the project team had instituted processes (e.g.,
time reporting) to better manage its contractors. In particular, the audit team tested
the exception-reporting process to ensure that overtime reporting for contractors
was complete and that appropriate authorities had approved them. Our examination
in this area indicated that this process was functioning as intended.

Monitoring of service delivery by a third party

The audit team expected to find that a process for monitoring service delivery by a
third party had been set up to ensure that the terms of contractual agreements
continued to be met. In particular, we looked at whether:

actual costs and schedules were compared to planned;

issues were documented and tracked until closure;

services or products were reviewed prior to financial settlements;
acceptance criteria were documented;
periodic reviews were carried out;
an independent audit of contractor operations was done; and
monitoring was consistent with the type of contract awarded.

The audit team tested the contracting process to ensure that it was functioning
according to CIC/PWGSC policies by examining a sample of seven NON-COTS
contracts to determine whether they were sufficiently detailed. Our examination in
this area showed that the process was functioning as intended, but that some areas
needed improvement. For example, we noted instances in which particular goods or
services were to be delivered, but there was no reference to a delivery date beyond
the contract end date. Including delivery dates for deliverables in the contract will
allow the project to better understand and monitor progress toward targets. We
advised the GCMS project team of these weaknesses and were assured that they
were being addressed.
The audit team also expected to find that performance measures would be in place
to assess the performance of the principal vendor. We noted that the main reporting
tool, the GCMS dashboard, had been refocused since Deployment 1 to provide more
information on the status of the project. However, it does not report on one key
aspect of the project: vendor performance. We acknowledge that the project team is
working with the vendor to identify and define appropriate criteria or indicators for
measuring the vendors performance. We support these efforts and make the
following recommendation.
Recommendation 2
Objective and quantifiable performance criteria related to the principal vendor should
be identified and discussed with the principal vendor. These metrics should be chosen
so that they adequately reflect contractor performance.
GCMS management response
Agreed.

Independence of the project management office and principal

vendor
We expected to see the appropriate degree of independence needed to ensure an
arms-length relationship between the PMO and the principal vendor.
Our review of a sample of seven NON-COTS contracts indicated that declarations of
conflict of interest had been completed by all. We also reviewed the roles and
responsibilities of key staff in the PMO to determine whether they were at arms
length with the vendor. The audit team found that the GCMS project management
office was staffed with individuals who were independent of the principal vendor.

Documentation and control of the use of subcontractors

The audit team expected to find the use of subcontractors to be documented and
controlled and, in particular, that documented guidelines or procedures existed for
the evaluation and use of subcontractors for the project.
We carried out a test of task authorization (TA)/task authorization requests (TARs) to
determine whether it was working according to policy. An examination of 65 TA/TARs
showed that COTS/NON-COTS were being monitored for start and end dates, and
that clearly defined roles and responsibilities were set out for each person who joins
the project. Appropriate authorities requested and approved these resources.
Overall, the audit team found that the documentation and control of subcontractors
was functioning as intended.

7.1.4 Requirements management

In examining this area, the audit team reviewed a number of elements, as discussed
below.

System requirements and specifications

The audit team expected to find that the PMO had defined procedures and plans to
ensure close liaison with system users in developing the design specifications, and to
ensure that these specifications would meet the users requirements. In particular,
we looked at whether:

project management had developed and approved policies regarding

requirements management;

policies and procedures ensured that requirements had been documented;

the process for developing requirements had been defined and had involved
users, project management and team members and key IMIT personnel;
business requirements had been clearly defined prior to acquisition;
plans were in place to link requirements to design components, test plans and
contract management procedures; and
requirements had been reviewed and approved by project management, user
groups and senior management.

The audit team found that the GCMS project team had a documented requirements
management plan that had been signed off by the appropriate project authorities.
The user community has been actively involved in developing the business
requirements. In particular, the GCMS project team has actively engaged
representatives from across the Department and the CBSA to define business
requirements and design specifications (e.g., Group of Approvers, OVIT, Design
Review Team, and Subject Matter Experts).
In trying to identify, define and formalize business requirements, it has been
necessary to re-examine and rationalize processes in three major business domains
(i.e., immigration facilitation, immigration control and integration), and to gain
consensus among users in the Department. This approach has resulted in delays in
finalizing the business requirements and obtaining final approvals and sign-offs. The
project team is aware of this and believes that this approach is warranted. It believes
that this approach will result in a more consistent and standardized set of business
practices, including greater uniformity, increased efficiency and improved service
across CIC.

Processes to ensure that requirements are met

Requirements are documented and available to all project members. In addition,
requirements are linked to the overall design of the GCMS solution, including
components, testing and contract management. The audit team expected to find that
requirements had been analysed for completeness, understandability, feasibility and
consistency and, more importantly, that specifications had considered:

the design of the process for importing data from existing systems to GCMS;

the definition and documentation of input requirements;

the definition of interfaces;
user-machine interfaces;
the definition of processing requirements;
the definition of output requirements;
internal control and security requirements; and
functional and operational requirements.

The audit team found that documentation on the requirements was available.
However, it is controlled to ensure that information is safeguarded from unwanted
changes. The final approved documents are provided to team members through the
ClearCase document management tool. The audit team did not determine if the
requirements existing at the time of our audit had been adequately analysed and
linked to design components or test plans. We understand that the GCMS project
team, as part of its activities for Release 2, will develop test plans that will cover
those requirements.

Processes to manage changes to requirements

The audit team expected to find that changes to requirements were handled through
a controlled process whereby user representatives and project team members
negotiated and agreed upon any changes. The audit team found that changes to
requirements had been documented and reviewed by the appropriate functional
areas and user community representatives for completeness and operational

approval. The process for managing changes to requirements was functioning as

intended.

7.1.5 Integration management

In the context of integration management, the audit team reviewed several areas as
discussed below.

Linkage to project authorities

The audit team expected to find that the project management office had formal
reporting structures in place that would provide key decision makers with the
information they need to make informed decisions. We also expected that the project
would have mechanisms to incorporate feedback from key stakeholders such as
other departments and TBS. In particular, we noted the following:

MOR and GCMS dashboards provide details on the technical status of the
project to the Chief Information Officer Branch of the TBS.

An independent TBS monitor provides regular updates on the status of the

project to Treasury Board and CIC executives.
Meetings are held every two weeks with the TBS analyst on the program side.
Quarterly meetings of the SPAC involve cross-departmental representation to
address GCMS issues that may affect other government departments. SPAC
consists of representatives from 16 federal departments and agencies.

7.1.6 Change management

The audit team reviewed a number of elements relating to change management, as
follows.

Change management procedures

Effective change management is a vital aspect of the overall governance process and
is critical to controlling the scope of a project. In the case of GCMS, the change
management process consists of two types of requests:

Change requests changes that result in a change to approved project or

product deliverables; and

Problem reports issues that address a defect or deviation from the approved
functional baseline.

The project team had a clearly defined process for approving change requests that
involved higher levels of management authorization than problem reports. The
procedures for change requests for the GCMS project include:

categorizing and prioritizing requests;

detailing how urgent requests are to be handled;

updating procedures for keeping requestors informed;
allowing for assessment of the effects of the change; and

identifying the authorities required before the changes can be approved.

The audit team found that the process, procedures and change log were functioning
as intended. Documentation indicates that the process is controlled and centralized.

Requests for changes

The audit team expected to find that change requests were monitored and recorded.
In particular, we expected that change-control logs would be used to record all
change requests.
The audit team found that two people managed the process for the GCMS project
team. They are responsible for monitoring the life cycle of all change requests (i.e.,
opening, logging, tracking and closing them). The project team had recently
improved the process to address the closing of change requests within acceptable
time frames. In particular, the project team has focused on identifying high-priority
change requests with an emphasis on addressing and closing these issues.

Documentation of approved changes

Change requests are used as a means of documenting or recording decisions and
ensuring that representatives from not only the GCMS project team but, as
appropriate, other key stakeholders, understand the impact of the potential change.
Such tracking enables GCMS to:

provide members with a way to record the history of changes made to the
GCMS product;

control proposed changes that may affect the scope, schedule or budget;
ensure that all changes adhere to the Configuration Management and
Document Management processes; and
communicate approved changes to the GCMS team as a whole.

The audit team found that the project team documented and controlled change
requests centrally. As part of our testing, the audit team examined the change
request process by examining a random sample of 42 change requests from all
change requests processed as part of Release 2. Our objective was to determine the
degree of compliance with the change request policy. The audit teams examination
revealed some minor errors, but no systematic or material problems. We brought
them to the attention of the GCMS project team and they are currently being
addressed.

7.2 Controls over data integrity and sensitive

information
The objective of the audit for this area was to assess the appropriateness of controls
over data integrity, and those for ensuring that sensitive information is protected.
The areas examined as part of this audit included the appropriateness of procedures
and plans in the following areas: data conversion and integrity of converted data,
and procedures to protect sensitive information. We present our observations and, as
appropriate, recommendations for each area, as follows.

7.2.1 Data conversion and integrity of converted data

With respect to data conversion and the integrity of converted data, the audit team
reviewed a number of elements, as discussed below.

Procedures for ensuring the accuracy of data

During Deployment 1, 18 percent of Citizenship data were converted. As a direct
result of the lessons learned from Deployment 1, the data conversion strategy for the
remaining Citizenship data was revisited and refocused to better determine which
information is to be converted, the amount to be converted and the accessibility of
data not converted, and to ensure that the approach is well integrated with data
conversion plans for Release 2. The project has also indicated that CIC and the CBSA
have embarked on a data cleansing exercise to improve the consistency of
converted data.
The remaining 82 percent of Citizenship data will undergo further analysis to
determine which data elements will be selected for conversion as is to the new
GCMS system. The CIC IMIT maintenance group has assumed overall responsibility
for this activity. The time lines for converting these select summary Citizenship data
are expected to mirror those for Release 2. We understand that any Citizenship data
that are not converted will be accessible through a data warehouse, with summarized
details contained in the new GCMS summary tab.
For Release 2, a revised strategy that both minimizes the amount of data that will be
transferred to GCMS and provides users with alternative mechanisms to access
unconverted legacy data has been developed. Specifically, key data elements for all
clients, and all data related to active cases, will be sourced from those legacy
systems considered to contain the most accurate and up-to-date information, and
imported into GCMS. Enough summary data stemming from inactive cases will also
be imported into GCMS to handle reports, interfaces and real-time decisions. The
general approach is to determine which information to convert, how much to convert,
and how best to access data not converted. As with the Citizenship data noted
above, the intent is to utilize the summary tab format for data elements not
converted.
The remaining legacy system data will be moved to a data warehouse and made
directly accessible to well-trained and mentored power users through ad hoc query
tools. The information needs of other users will be met with custom reports. At this
point in time, no conversion is scheduled to occur for Release 2 data until early 2006.
The plans and controls we reviewed in this area should assist the GCMS project team
as it prepares for Release 2.

Control of access to data

The audit team expected to find that access to data was controlled to avoid risks of
unauthorized changes. Access controls for sensitive information are authorized by
delegated managers on a system-by-system basis (e.g., SAP, SMS, LiveLink and
ClearCase). The audit team did not find clearly documented access procedures or
protocols with respect to GCMS project-sensitive information assets as a whole.

Interviews indicated that the sensitive information was the data. Thus, it is our
understanding that the project team will be developing access controls and detailed
procedures as part of the data conversion process for Release 2 in early 2006. These
activities should assist the GCMS project team in protecting the GCMS data.
As part of our testing, the audit team examined the GCMS project team security
clearance process by examining a random sample of 15 security clearances for
individuals on the project to determine: 1) whether the individuals had the
appropriate security clearance when they began working on the project; and 2)
whether the individuals were still with the project and whether their security
clearance was still valid. Our examination found the process was functioning as
intended.

7.2.2 Protection of sensitive information

In the context of requirements management, the audit team reviewed the following
elements.

Safeguarding of information
A preliminary Privacy Impact Assessment (PIA) for GCMS was completed as part of
the original Effective Project Approval submission of January 2002, and a full PIA was
submitted to the Office of the Privacy Commissioner (OPC) in February 2004. The
GCMS project team had provided three periodic status updates to the OPC: one in
August 2004, another in January 2005, and the third in March 2005.
In addition to the above activities, the project team has clearly defined and
documented the procedures for records management, instructing project team
members on what to keep and what to dispose of. Furthermore, the project team is
actively engaged in discussions with CIC and Archives Canada over policies relating
to the retention and destruction of data.
The audit team found that the GCMS project teams measures for safeguarding
information were adequate.

8. GCMS Project Action Plan

Recommendation
1. The GCMS project
should ensure the
consistent
application of project
management
planning standards
(GCMS Schedule
Management and
Development
Standards)

Response

Action Plan

Agreed.

Reinvigorate
project
management
practices and
enforce consistent
execution and
application of
inputs, outputs and
standards:

Responsibility Completion
Date
Project
Executive
GCMS

January
2006

developed by the
PMO.

Integration
activities
initiated to
confirm plan
linkages and
close gaps
Project
managemen
t planning
and quality
assurance
practices
being
enhanced to
include
validation,
monitoring
and
escalation

2. Objective and
quantifiable
performance criteria
related to the
principal vendor
should be identified
and discussed with
the principal vendor.
These metrics should
be chosen so that
they adequately
reflect contractor
performance.

Agreed.

Improve vendor
management at the
macro level by
implementing
qualitative
measures to
monitor overall
performance, and
at the micro level
by strengthening
contract
management
practices:
In
discussions
with the
principal
vendor to
confirm the
criteria to
provide an
overall
qualitative
assessment
of vendor
performance
In
discussions

Project
Executive
GCMS

January
2006

with PWGSC
to refine
task
authorizatio
ns to
include
identificatio
n of specific
conditions of
satisfaction
and
performance
Train
managers
on the use
of task
authorizatio
ns and
conditions of
satisfaction
and
performance
to enhance
performance
managemen
t

Update
executive
dashboard
reporting to
report on
principal
vendor
relationship
and contract
performance

Appendix: GCMS Audit Interviews List

Assistant Deputy Minister, Policy and Program Development

Assistant Deputy Minister, Centralized Services Delivery and Corporate

Services
GCMS Project Leader
GCMS Associate Director
Executive Director, GCMS IMIT
Executive Director, Business Requirements and Functional Design
Executive Director, GCMS/National Case Management System
R2 Delivery Manager
Executive Director, PMO

GCMS Finance
Project Control Officer, PMO
Engineering Process Manager
Risk and Issue Manager
GCMS Training
Principal vendor, Subcontract Management
CIC, DG, Finance Branch
CIC, Chief Information Officer
Director, B.C./Yukon Region
Vice-President, CBSA
Manager, Information Management, CBSA
Analyst, TBS
Chief Information Officer Branch, TBS
GCMS Independent TBS Monitor
Consultant, GCMS Audit Program
Policy Officer, GCMS OVIT, CPC Sydney
Team 3 Leader, CPC Sydney
Program Support Officer, CPC Sydney
PMO, GCMS Master Project Scheduler
GCMS Change/Problem Management Project Leader
Project Control Officer, UAT
Project Control Officer, IMIT
Procurement and Contracting Officer, CIC Contracting
Personnel Security Supervisor, CIC Security
GCMS Project Planner
Manager, GCMS Data Conversion
GCMS Project Lead, Data Conversion
GCMS COTS Data Transformation Services Specialist, Data Conversion
CIC Information Management, Archiving
GCMS Project Office Manager, PMO
Officer, CIC St. Clair, Toronto Region (D1)
Supervisor, CIC St. Clair, Toronto Region (D1)
A/Operations Manager, GTA West, Toronto Region (R2)
PRRA Coordinator, Toronto Region (R2)