The importance of data for policy

formation and evaluation

Dr. Ted McDonald, UNB

Australia-Canada Roundtable on Foreign
Qualification Recognition March 2013
Data sources in Canadian research

Zietsma (2010): Immigrants working in regulated
occupations (2006 Census)

Goldmann, Sweetman & Warman (2009): The
Economic Return on Immigrants Human Capital:
The Impact of Occupational Matching (LSIC)

Hawthorne (2008): Foreign Credential Recognition
Volume, Canadian Issues (LSIC, Census, surveys)

Three workshop papers from Wednesday (Census)


Advantages and limitations of
Census data
National scope, large sample, widely accessible
Information on labour market outcomes, mobility,
location of training, language fluency,
demographics. Linkable across family members
Very useful for identifying patterns and trends
o E.g., occupation mismatch

Self reported
Major data gaps
Less useful for explaining the *why*
Data for monitoring vs. evaluation
Monitoring
Participation rates, pass rates, completion rates,
costs, durations, complaints, inquiries
o % successfully licensed to practice
o Are occupation targets being met

Evaluation
Does a new program achieve the intended
outcomes?
Are there unintended or unexpected
consequences?
Are there more cost effective ways of achieving the
same target?


Narrow vs. broad evaluation questions
Example (narrow) how effective is offshore
credential assessment at reducing the number of
immigrants who fail to become licensed to work in
their area of training?

Example (medium) how might interprovincial
migration of skilled immigrants be affected by
adoption of national standards for licensure?

Example (broad) how much has economic
performance improved because of reduced skill
shortages in key occupations?

The scale of the data problem
Separate colleges/councils for each regulated
profession in each province/territory
o 500 separate agencies responsible for licensing in Canada

Hundreds of universities, colleges and training facilities

Federal government departments: HRSDC, CIC, Health
Canada

Credential assessment agencies (public and private)

Provincial/state government departments

Settlement agencies, providers of bridging programs

Demographic data, household data, spouse data,
labour market data

Some examples of data sources
Data on doctors
o Medical Council of Canada eligibility and qualifying exam outcomes
o CARMS resident matching service (includes IMGs seeking residencies in
Canada)
o CAPER post-MD education registry
o Provincial colleges of physicians and surgeons (registry)

CIC ICARE/ICAMS data collection on settlement
projects including labour market and language
training

Canadian Information Integration Program records
a wide range of data on participants who use CIIP
support services
First steps towards a data inventory
What data are being collected? [a lot already is]
o Who is collecting it?
o What does it include?
o What time period?
o What documentation exists?

What are the obstacles to sharing it?
o Confidentiality concerns
o Legislation
o Cost
o Proprietary concerns

Ability to link data (common identifiers)
Potential benefits of an inventory
Standardized data collection and reporting

Data preservation, documentation

Data sharing agreements
o Between agencies, colleges, councils
o Across provinces/states
o With research institutions

More researcher input into data collection,
dissemination

Permission to link could be obtained when data are
collected


One example of data sharing
The Ontario Fairness Commissioner receives data
from the college/institute of each regulated
occupation in Ontario every year

Data on applicants for licensure
o Number who have applied
o Status of the applications
o Number of applications approved
o Country of education

Data on licensed members
o Country of initial training
o Membership status

Manitoba is in the process of collecting the same
data


Even simple comparisons (over time, across
occupations, across regions, across countries) can
be very informative and can suggest to us where to
look in more depth.