BGH

Assessment of NCTD
Program Management
Framework for Positive Train
Control Program
Subtask 2: Analysis – Gap Analysis
Prepared for:
Brad Hansen, M.S., PMP
Director, PMO Capital Projects
May 2013
© 2013 0 icfi.com/transportation
EXECUTIVE SUMMARY
Subtask Objective
 Identify gaps in the PTC Program project management processes, artifacts,
tools and staff resources required for effective management
 Assess the PTC Program’s Program Management Plan (PMP) compliance with:
– PMI PMBOK
– 49 Code of Federal Regulation (CFR) § 633.25 - Contents of a project
management plan
– ICF’s experience and lessons learned from analogous programs, tailored to the
NCTD PTC Program environment
 Review priority components of the PMP, as requested by NCTD:

– Program Management Organization – the organizational structure and associated
resources
– Requirements Management – the processes and tools NCTD employs to manage
requirements
– Risk Management – the process and tools employed by NCTD to manage
programmatic risk
 Recommend prioritized improvement areas to NCTD’s program management

approach based on expected benefit to the NCTD organization
EXECUTIVE SUMMARY
Key Themes
 The implementation of the PTC System is complex and dynamic,

placing stress on NCTD’s organizational capabilities
 In many instances, the PTC Program’s “current state” activities are

inconsistent with the Program Management Plan’s (PMP)
documented processes, tools, and structures
 Formally documented program-level controls, integrating all PTC-

related activities and components, do not exist or are not being
exercised
– The program-level controls that do exist are largely specific to the HTI
PTC System contract—which represents only 40% of the total program
budget
EXECUTIVE SUMMARY All The color scale reflects the degree of
Most compliance with (i.e. all, most, some,
Gap Analysis Results Some
Few
few) the processes, artifacts and tools
prescribed within the PMBOK
knowledge areas
How well does the PTC PMP stack up to PMBOK? How well does the Current State of the Program follow PMBOK?
NCTD PTC
PMBOK Knowledge Area Ratings Summary
PMP Current State
Integration Management Little evidence of integration processes or tools were observed
The entirety of the PTC Program scope is not defined and

Scope Management
documented
No cost management baseline exists and inconsistencies
Cost Management between PTC Program and NCTD cost management processes
are present
Lack of a schedule baseline or a single integrated master
Schedule Management
schedule
Undefined risk management process, but recently instituted
Risk Management project-level processes will enhance Program level risk
management
An organizational quality capability does not exist, nor is there a
Quality Management
Quality Management System
The current organizational structure is not the most effective
Human Resource Management
approach
Mature organizational procurement capabilities with few needed
Procurement Management
improvements
Level of stakeholder identification, analysis and engagement for
Stakeholder Management
a Program of this complexity is not evident
Strong organizational capability, though ineffectively integrated
Communications Management
into the PTC Program
EXECUTIVE SUMMARY
Recommended Improvements
1 Baseline the Program
2 Enhance program control
3 Re-align program organization
4 Implement a coordinated stakeholder strategy
Dedicate resources to prepare for post-

5 implementation operations and maintenance
Methodology
5
METHODOLOGY
Scope of Gap Analysis
 Evaluation of PTC Program Management Plan
 Assessment of Numerous Program artifacts, Program staff

interviews, PTC Program organization charts, and observations –
all constitute the “current state”
 Standards used :
 A Guide to the Project Management Body of
Knowledge (PMBOK) – Fifth edition (with emphasis
on the ten Knowledge Areas)
 The Standard for Program Management – Third

Edition
METHODOLOGY
FRA§ 633.25 PMP Content Requirements are

addressed by PMI’s PMBOK Knowledge Areas
PMBOK
§ 633.25 PMP Content Requirement
Knowledge Area
(a) description of adequate recipient staff organization, complete with well-defined reporting relationships,
statements of functional responsibilities, job descriptions, and job qualifications
Integration Management
(d) document control procedure and recordkeeping system
(i) plan for internal reporting requirements including cost and schedule control procedures
(e) change order procedure which includes a documented, systematic approach to the handling of
Scope Management
construction change orders
(b) budget covering the project management organization, appropriate consultants, property acquisition, utility
Cost Management relocation, systems demonstration staff, audits, and such miscellaneous costs as the recipient may be
prepared to justify
Schedule Management (c) construction schedule
Risk Management Not required (but recommended, by the assessment team)
(g) quality control and quality assurance programs which define functions, procedures, and responsibilities for
construction and for system installation and integration of system components
Quality Management (h) material testing policies and procedures

(j) criteria and procedures to be used for testing the operational system or its major components
(f) description of organizational structures, management skills, and staffing levels required throughout the
Human Resource Management
construction phase
Procurement Management Not required (but recommended, by the assessment team)
Stakeholder Management Not required (but recommended, by the assessment team)
Communications Management Not required (but recommended, by the assessment team)
METHODOLOGY
Rating
 Our color coded scale provides a rating for each knowledge area:
All The color scale reflects the degree of

Most compliance with (i.e. all, most, some, few) the
Some processes, artifacts and tools prescribed within
Few the PMBOK knowledge areas
 Ratings are provided for the PMP and the “current state”
PMP Current State
Assesses the Program’s “current state”

Assesses the PMP against PMBOK
processes, artifacts and activities
Knowledge Areas
against PMBOK Knowledge Areas
Gap Analysis Results
9
GAP ANALYSIS RESULTS
Integration Management Rating: PMP Current State
“… includes the activities needed to identify, define, combine, unify, and coordinate multiple
components within the program.”
Analysis Identified Gaps

 Several key integration-related artifacts exist  Documented process and tools that integrate
– PMP, last revised on March 22, 2012 Program-level scope, schedule, schedule and
– Project charter budget (i.e., no Integrated Master Schedule,
– Organizational charts Program WBS, or time-phased budget)
– Program management information systems
 Many current state activities do not follow
 Recent project-level standard processes, processes documented in the PMP
templates, and reporting requirements have
been implemented and are contributing to  Process for maintaining the PMP
integration management
 Process for program-level monitoring and
 Inconsistencies between the “current state” control
organizational structure and the structure
defined in the PMP  Program Charter lacks clear articulation of the
purpose, scope, budget, schedule and
roles/responsibilities
 Process for integrated change control (scope,

schedule, cost, requirements, system
configuration
Scope Management Rating: PMP Current State
“… defines the work required to deliver a benefit (major product, service, or result with specified
features and functions) [and] includes all of the activities involved in planning and managing the
program’s scope.”

 System requirements for PTC are provided in  A scope statement that defines the entire PTC
49 Code of Federal Regulation (CFR) § Program
236.1005 – Requirements for Positive Train – (note: an “original Scope Statement” was
Control systems referenced in the materials reviewed during the
– Additional due diligence is required to ensure Discovery subtask, but not found)
PTC Program requirements are compliant
with this CFR  A Program-level Work Breakdown Structure
(WBS) to include ALL Program work (vendor-
 The PMP contains elements of scope implemented and NCTD-implemented
management including, but are delegated to projects and tasks)
HTI:
– Contract WBS  Control of Program scope
– Scope change control process
– Requirements management and traceability  Program-level requirements management and
traceability for all PTC Program projects
Cost Management Rating: PMP Current State
“… identifying the program’s financial sources and resources, integrating the budgets of the
program components, developing the overall budget for the program, and controlling costs
throughout the duration of both the components and the program.”

 Total program budget of $87.3M is funded  Program cost management processes do not
through nine sources reconcile with organizational cost
management practices (15 accounts versus 5
 The program’s contingency fund decreased accounts)
by 70% during the period from July 2011
through January 2013.  Post-implementation operations and
– It is not clear whether the remaining maintenance cost estimates have not been
contingency funds will be sufficient documented - current estimates appear to be
anecdotal and not analytically based
 There are differences in the budget
management practices used by the Program  Cost management process
and the NCTD Finance Division, resulting in
at least $21.2M (~24% of the total program
budget) of funds that cannot be currently
reconciled by the Finance Division
Time Management Rating: PMP Current State
“… includes determining the order in which the individual components are to be implemented, the
roadmap for the program, and the milestones to be measured to keep the overall program on track
and within defined constraints.”

 Activities of individual contracts and certain  A Program-level schedule baseline
Program elements appear in multiple
schedules  A single integrated master schedule that
encompasses ALL Program activities
 Project Managers are assigned to monitor  Dependencies
contractor progress against project-level  Predecessor/successor relationships
schedule baselines  Critical Path
 Schedule hierarchy
 The PMP defines an approach to schedule
management using a three-level hierarchy,  Schedule management process
each with a different level of detail - NCTD is
not executing the approach  Objective basis for NCTD’s ability to achieve
the December 31, 2015 completion date
Risk Management Rating: PMP Current State
“… includes the processes of conducting risk identification, analysis, and response planning in
order to reduce the likelihood of occurrence and/or impact an individual, or series of risks, have
on a program.”

 There are currently four versions of risk  A documented risk management process to
registers within the program, some more ensure risk management consistency within
current than others all Program elements
 Risk management discussion in the PMP and  A single integrated risk register
the Program Risk Management Plan have
major inconsistencies  Risks are currently assessed and prioritized
using only qualitative measures
 Project-level risk management plans and
processes have recently been implemented
Quality Management Rating: PMP Current State
“… includes the activities of the performing organization that determines program quality policies,
objectives, and responsibilities so that the program will be successful.”

 Current quality management processes in the  Program-level quality management processes
PMP are specific to the HTI contract as part of a Quality Management System
 NCTD is required to document “a complete  Clarity as to who is accountable within the

description of the safety assessment and PTC Program or NCTD organizations for
Verification and Validation processes applied quality management
to the PTC system” in an FRA-mandated PTC
Safety Plan (PTCSP) prior to FRA-testing of  In-house staff accountability and capability for
the system program quality management
– Rail Safety Consulting LLC is developing
NCTD’s PTCSP
Human Resource Management Rating: PMP Current State
“… includes the process that organize, manage, and lead the project team—comprised of the
people with assigned roles and responsibilities for completing the project.”

 The program manager has authority to assign  The organizational chart in the PMP does not
priorities, apply resources, and direct the work match 1) the associated narrative description
of individuals assigned to the project or 2) NCTD’s organizational chart
 The PTC Program organization is largely  The PMP’s listing of contractor and consultant
geared to oversee the work of contractors and resources is incomplete and does not match
consultants – is this precluding attention to the contents of the PTC Vendor List
internal work elements?
 Job skills, experience, and qualifications of
 Five Project Managers oversee 15 projects personnel fulfilling positions within the PTC
(contracts) program are not documented
– This makes it difficult to assess whether the
PTC Program has been appropriately and
adequately staffed
Procurement Management Rating: PMP Current State
“… addresses specific procurement needs that are unique to managing the overall program and
the needs of the constituent projects/components.”

 An NCTD Contract and Procurement  Processes associated with formally closing
representative is ‘matrixed’ to the program out each contract after all deliverables and
and provides contracting/procurement support payments have been made are not defined
 Procurement management processes are  The PTC Vendor List does not include “all
integrated with, and leverage, NCTD’s vendors and suppliers of each PTC Program
organizational procurement capabilities system, subsystem, component, and
associated product, and process in use
system-wide.”
Stakeholder Management Rating: PMP Current State
“… includes the activities associated with identifying, categorizing, and tracking all stakeholders
who will interact with the program or be affected by the implementation of the program.”

 PMP identifies three stakeholder groups:  A dedicated PMP section for stakeholder
– Communities management
– Media Outlets
– Government Entities  A complete list of stakeholders to manage
 Preliminary analyses conducted by NCTD’s  Coordination and collaboration between the

Government Affairs and Communications PTC program and NCTD’s Government
Division identified 15 stakeholder groups in Affairs and Communications Division—this
three categories: does not appear to take full advantage of the
– Internal agency’s capabilities
– General Public
– Targeted
Communications Management Rating: PMP Current State
“… includes the activities for facilitating timely and appropriate generation, collection, distribution,
storage, retrieval, and ultimate disposition of program information.”

 The approach outlined in the PMP is largely  The PMP fails to address how
focused towards external communications communications will be managed and
disseminated to internal stakeholders
 NCTD has a robust internal communications regarding the status of the program
capability that has developed key
communication-related elements regarding  Lack of internal communication, resulting in
PTC an insufficient level of understanding and
engagement within the PTC Program among
 Stakeholder management and communication project team members
is a critical aspect of the PTC Program and its
success
Recommendations for Improvement
Prioritized based on expected benefit to the NCTD

organization
20
RECOMMENDATIONS FOR IMPROVEMENT
1 Baseline the Program
2 Enhance program control
3 Re-align program organization
4 Implement a coordinated stakeholder strategy
Dedicate resources to prepare for post-

5 implementation operations and maintenance
 Scope: Identify all work that’s required to be accomplished by the

PTC Program and develop an associated program-level WBS
 Cost: Reconcile current cost estimates and resource requirements

with the newly developed WBS; develop risk-based budget
contingencies
 Schedule: Develop a program-level integrated master schedule

that encompasses all currently-identified tasks
 Evaluate and implement a program management information system

sufficient for the size and complexity of the PTC Program
 Review and update the Risk Management plan (e.g. implement an

integrated master risk register that includes quantitative analyses)
 Revise the PTC’s Quality Management plan so it aligns with the future
PTCSP and newly established Quality function within NCTD
 Identify and define configuration management requirements specific

to the PTC Program
 Implement program-level variance analysis and reporting mechanisms
 Implement a process for maintaining and updating the PMP for the
duration of the program
 Strengthen the organizational relationship between the PTC Program

and NCTD’s Project Management Oversight Office, Capital Projects
 Establish an independent Quality management capability within NCTD
 Formalize cross-organizational representation within PTC Program
 Establish formal job descriptions that include skills, experience, and

qualification requirements for each role within the PTC Program
organization
 Reissue an updated Program Charter to all NCTD staff that clearly

highlights the purpose, objectives, governance structure, and
roadmap for the PTC Program
 Review and expand upon the stakeholder analysis currently drafted

by NCTD’s Government Affairs and Communications Division
 Implement a coordinated internal and external communications

strategy based on the updated stakeholder analysis
 Explore opportunities to link the PTC outreach strategy with Board

of Directors and external project champions engagement
 Document on-going maintenance requirements for PTC-related

components and ensure adequate maintenance plans are
developed, implemented, and the appropriate personnel trained
 Identify additional organizational training needs, for example:

– Locomotive operators: work with Herzog to understand and document initial and
recurrent training plans
– Maintenance of Way: determine which PTC-related components require
familiarity with and understanding of by MOW personnel
 Plan, document, and integrate newly implemented PTC-related

components into NCTD’s configuration management capability
 Develop and document one-time, on-going, and incremental

operating costs related to PTC and implement processes for
reviewing and revising these costs (and assumptions) on a regular
basis.
Next Steps
27
NEXT STEPS
Plan of Action
 To include:
– Implementation plan for the recommendations identified in this task
– Current and proposed project management organization chart

BGH

Transféré par

Informations du document

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

BGH

Transféré par

Droits d'auteur :

Formats disponibles

Assessment of NCTD

 Review priority components of the PMP, as requested by NCTD:

 Recommend prioritized improvement areas to NCTD’s program management

 The implementation of the PTC System is complex and dynamic,

 In many instances, the PTC Program’s “current state” activities are

 Formally documented program-level controls, integrating all PTC-

Gap Analysis Results Some

Integration Management Little evidence of integration processes or tools were observed

The entirety of the PTC Program scope is not defined and

1 Baseline the Program

2 Enhance program control

3 Re-align program organization

4 Implement a coordinated stakeholder strategy

Dedicate resources to prepare for post-

Scope of Gap Analysis

 Evaluation of PTC Program Management Plan

 Assessment of Numerous Program artifacts, Program staff

 The Standard for Program Management – Third

FRA§ 633.25 PMP Content Requirements are

Schedule Management (c) construction schedule

Risk Management Not required (but recommended, by the assessment team)

Quality Management (h) material testing policies and procedures

Procurement Management Not required (but recommended, by the assessment team)

Stakeholder Management Not required (but recommended, by the assessment team)

Communications Management Not required (but recommended, by the assessment team)

All The color scale reflects the degree of

PMP Current State

Assesses the Program’s “current state”

Integration Management Rating: PMP Current State

Analysis Identified Gaps

 Process for integrated change control (scope,

Scope Management Rating: PMP Current State

Analysis Identified Gaps

Cost Management Rating: PMP Current State

Analysis Identified Gaps

Time Management Rating: PMP Current State

Analysis Identified Gaps

Risk Management Rating: PMP Current State

Analysis Identified Gaps

Quality Management Rating: PMP Current State

Analysis Identified Gaps

 NCTD is required to document “a complete  Clarity as to who is accountable within the

Human Resource Management Rating: PMP Current State

Analysis Identified Gaps

Procurement Management Rating: PMP Current State

Analysis Identified Gaps

Stakeholder Management Rating: PMP Current State

Analysis Identified Gaps

 Preliminary analyses conducted by NCTD’s  Coordination and collaboration between the

Communications Management Rating: PMP Current State

Analysis Identified Gaps

Prioritized based on expected benefit to the NCTD

1 Baseline the Program

2 Enhance program control

3 Re-align program organization

4 Implement a coordinated stakeholder strategy

Dedicate resources to prepare for post-

 Scope: Identify all work that’s required to be accomplished by the

 Cost: Reconcile current cost estimates and resource requirements

 Schedule: Develop a program-level integrated master schedule

 Evaluate and implement a program management information system

 Review and update the Risk Management plan (e.g. implement an

 Identify and define configuration management requirements specific