1

Bridging Refugee Youth and Childrens

Services (BRYCS)
Evaluation Plan, HEA 625

Jasmine May & Jolie Azorlibu

2

Table of Contents

Section 1: Conceptualization of Evaluation .............................................................................. 3

Section 2: Program Description ................................................................................................. 4

Section 3: Evaluation Plan ......................................................................................................... 7

Section 4: Reporting Plan .......................................................................................................... 12

Section 5: Detailed Budget ......................................................................................................... 13

Section 6: Detailed Timeline ...................................................................................................... 15

Reference .................................................................................................................................... 17
 3

Section 1: Conceptualization of Evaluation

Purpose:

The purpose of this evaluation is to understand how the BRYCS bridges the gap between

service provider aiding refugees, refugee communities, and public child welfare professionals in

the United States through cross-service trainings. It is also intended to determine the outcomes of

the program.

Questions to be addressed:

Did collaboration strengthen among child welfare and refugee serving agencies?
Are recipients of services being better served, as a result of cross-service trainings?
Are the refugee serving agencies involved appropriately equipped culturally to provide services?
Are the cross-service trainings effective for the refugee serving agencies?

Key Stakeholders:

The Federal Office of Refugee Resettlement, Lutheran Immigration and Refugee Service, and

the United States Conference of Catholic Bishops are all partners of BRYCS. Refugee youth,

children, their families, and professionals serving refugee youth and children serves as another

extension of key stakeholders.

Assumptions:

Local resettlement agencies, mutual assistance associations, public child welfare agencies, and
mainstream organizations already interact with one another.
There are positive relationships between refugee youth and children and the agencies that serve
them.
Local resettlement agencies implement other programs apart from the ones from BRYCS

Contextual factors:
Public child welfare agencies and refugee-serving agencies may lack information about each
others goals, operating structures, and services to families and children.
The myth within the refugee communities about child protective services taking children away
has a significant impact on refugee parents and can indirectly affect how they perceive their role
as parents in the United States.
New cultures and traditions are enriching cities and neighborhoods, yet are posing new
challenges for service delivery systems established before the arrival of new immigrants.
Refugee and immigrant communities are different, therefore the cross-service trainings are to be
tailored to meet specific local service needs.
Section 2: Program Description
 4

BRYCS is an organization formed in 2001 to provide information hub for empowering

immigrant children and their families. It has one of the nations largest online collection of

resources for immigrants and refugees, which increases information sharing and promotes

collaboration at the local, state, and national levels. The main goal of the organization is to serve

as a bridge between service provider aiding refugees, refugee communities, and public child

welfare professionals in the United States. The aforementioned are the targeted audience for this

evaluation. The results of this goal has been recognized in the improvement of areas such as

child welfare, family strengthening, education, and youth programs targeted towards assisting

refugee resettlement or immigrant serving agencies. Other goals include improving ethnic-based

community organization, and mainstream-service organization like juvenile justice, child

welfare, and the school system. The mission of BRYCS is to broaden the scope of information

and collaboration among service providers across the United States. Therefore, the program is

intended to be implemented and replicated nationwide. BRYCS provides training, consultations,

resource development, and access to up-to-date information through a website (www.brycs.org)

and clearinghouse. For the purpose of this evaluation project, only one program (cross-service

trainings) will be evaluated in order to prevent generalization of data. There are no previous

overall evaluations, but an evaluation of a pilot case study conducted in Saint Louis, Missouri

recommended that making services to newcomer refugee families have to be more accessible,

appropriate, and effective.

Logic Model

Inputs Outputs: Outputs: Short term outcome Long term outcome

Activities Participation

Cross-service Identify lead Refugee serving establish mechanisms of Create and sustain a
 5

trainings agency Agencies ongoing communication and comprehensive

collaboration among all service continuum of care for
providers through cross-service the refugee population
training
Expose refugees to available
resources

Money Define service Resettlement Develop formal and informal develop family and
constellation agencies relationships between agencies community centered,
Mutual assistance and across service systems. and strengths-based
associations ethnic- practice
based
organizations/refuge
e-serving agencies
Public child welfare
agencies
Mainstream
organizations

Staff Develop task Refugee Community Recruit staff members from Build relationships
force culturally and linguistically with policy makers
diverse backgrounds and public agencies to
increase coordination
and collaboration

Time Define roles

The resources needed to implement the BRYCS cross-service trainings are finances, time

allocation, and staff members. Once the aforementioned resources have been secured then

planning of the cross-service trainings can commence. The ultimate purpose of the cross-service

trainings is to connect refugee serving agencies and their resources to better serve their

recipients. In doing so, building a partnership is essential to utilizing and maximizing services

across different agencies. The first step to this partnership is identifying the lead agency among

all of the agencies involved with the project. The lead agency serves to generate interest and

support for local agencies to get involved in the training process and to guide the entire process

of planning and implementing the training. The lead agency also becomes responsible for the
6

allocation of significant staff time and expenses related to the planning process. Next, a service

constellation is identified to determine agencies that can be classified as, resettlement agencies,

mutual assistance associations/ethnic-based organizations/refugee-serving agencies, public child

welfare agencies, and mainstream organizations. It is important to look within the service

constellation to see if agencies already have an existing relationship. Once essential partners are

identified from the service constellation, a task force should be created. The task force serves to

provide ideas, suggestions, and assistance in developing and implementing the cross-service

training. Following the creation of the task force, roles and priorities can be defined. This is

imperative to ensure all agencies involved are clear of the expectations of the task force and

cross-service trainings. The cross-service trainings and its components directly impact the

refugee serving agencies who in turn impact refugee youth and children, and their communities.

The outcomes most proximal to the implementation of the cross-service trainings include the

establishment of mechanisms of ongoing communication and collaboration among all service

providers, exposure of refugees to available resources, the development of informal and formal

relationships between agencies, and the recruitment of staff members that have culturally and

linguistically diverse backgrounds. Long term outcomes include creating and sustaining a

comprehensive continuum of care for the refugee population, development of family and

community centered strength based practices, and to build relationships with policy makers and

public agencies to increase coordination and collaboration. A limitation of this logic model and

cross-service trainings at large is that, agencies may not successfully implement all of the steps

or components of cross service training due to internal barriers.

Section 3: Evaluation Plan

Specific evaluation questions

 7

As a result of the cross-service trainings, did refugee serving agencies collaborate and
consolidate resources more efficiently to serve refugee youth and children?
Are more refugee youth and children being served by the cross-trained agencies?
Are duplication of services decreasing, due to an increase in resource sharing and collaboration
among refugee serving agencies?
Is cultural competency being implemented in the agencies to support culturally appropriate
methods and dissemination?
How was the cross-service trainings effective for refugee communities in comparison to the
communities whose agencies were not trained?

Quasi-experimental Design

Refugee serving agencies have the choice to submit applications for the cross-service

trainings to BRYCS, and this process will help identify treatment groups for this study. BRYCS

has an established database of all agencies who serve refugee communities. This database will

aid in identifying the control group for this study by looking at agencies that have not currently

applied to participate in the trainings or has never been trained in the cross-service trainings. One

of the key ingredient of a quasi-experimental design is that it lacks randomization. Given that the

refugee serving agencies exercise autonomy in applying for cross-service trainings, this offsets

the non-randomization aspect of this study. This offset is achieved by self-selection of treatment

(cross-service trainings) by the agencies. Included with the application for the cross-service

trainings, are pretest interview questions for the agencies to take. Refugee serving agencies who

do not submit an application for the cross-service trainings will be asked to participate and also

provided with pretest interview questions. These pretest interview questions will help in

determining baseline measurements for both groups of their efforts already being taken to serve

the refugee youth and their communities. Five agencies will be present in both the control group

and treatment group.

The interview questions conducted in the pretests phase will be given to the agency

representatives who will receive the cross-service trainings and other members of the agencies.

These interview questions will identify the current programs of the agencies, and how they are
 8

developed, implemented and evaluated. The interviews will also identify if any collaboration is

being done currently or have been done in the past by the agencies. After the interviews have

been given, the treatment group that applied for the cross-serving trainings will begin their

trainings. The control group will not receive the trainings. After the trainings have been

completed, both groups will be given a posttest via surveys and interviews. The youth and

community members will participate in focus groups and answer surveys.

Variables to be measured

The number of agencies in the collaboration

Level of sharing information and ideas within the collaboration
Skills gained from the cross-service trainings
Did the cross-service trainings have an impact on utilization of services by the refugee youth
community
Attitudes and efficacy of the cross-service trainings
The impact of the cross-service trainings

Instrumentation

Interviews
Interviews will be administered in both the pre and posttest phases. Representatives from the
agencies who are receiving the cross-service trainings will be interviewed, in addition to other
members from the agencies. In the control group, members from the agencies will be
interviewed. The pretest interview will focus on observing baseline measurements for both the
treatment and control group. The posttest interview will serve to identify the efficacy of the
cross-service trainings compared to no treatment at all. The posttest interview will also help to
determine attitudes towards the cross-service trainings. Fifty interviews will be administered to
the treatment and control group each.
Surveys
The surveys will be completed by a convenient sample of the refugee youth and the community
members from both the control and treatment group at the pretest and posttest. Out of the refugee
youth and their families or community members that participate in the survey, 100 people will be
randomly selected through a raffle to receive a $10 gift card. 50 surveys will be given to the
agency or organization representatives (50-control, 50-treatment) before and after they attend the
cross-service trainings. For the treatment group, the surveys are mainly to determine change of
attitude towards the cross-service trainings.
Focus groups
The focus groups will mainly be for the refugee youth and their families or communities. The
goal of this is to gather a convenient sample and out of that, select about groups of five at a time
to have a guided discussion about the services they receive. Interpreters will be present if needed
during the focus group meetings.
Observation
Observations will take place through interactions, engagements and participation with the
community members and at the agencies. The goal of these observations is to understand how
the agencies communicate and implement interventions in the refugee communities, and how
they collaborate with other agencies. The observations will take place pretest, posttest and
quarterly (every three months).
 9

BRYCS database
Through an internal database, BRYCS has contact information for all refugee youth and children
serving agencies. This database will help access agencies that are not applying for the cross-
service trainings. The database will help select the five agencies needed for the control group. In
addition to selecting the control group, the internal database will aid in ascertaining data of the
participating agencies that will be pertinent to the evaluation.

Data collection plan

The pretest interviews will accompany the application for the cross-service trainings. For

the control group who will not apply for the cross-service trainings, upon consenting their

participation they will be sent interview questions to respond to. Once the cross-service trainings

have been completed, an immediate posttest will be given to both groups. The aims of the

posttest for the treatment group will be to determine their attitudes about the trainings and its

efficacy. The control group will also receive posttests for comparative and statistical purposes.

Another posttest will be given to both groups post 6 months. This test will be used to determine

the success of cross-service trainings implementation. Surveys will serve as another method of

data collection. Alike the interview questions, surveys will be administered to staff members of

the agencies pre and posttest. Surveys will also be given to members of the refugee youth

community. Focus groups will be conducted with the refugee youth community and their

families. Another valuable source of data collection will be achieved through observations of the

refugee community and the agencies. Observations will be recurring to try to maximize optimal

organic interactions between the two parties. Observations will occur both before

implementation of the cross-service trainings and after. The internal database of BRYCS will

serve as another venue of data collection. The database contains basic information on all of the

agencies that will participate in this study. Included in some of the basic information about the

agencies are mission and vision statements, services that are offered, if the agency has

collaborated with BRYCS before, and projects that the agency have spearheaded or collaborated

on.
 10

Evaluation Crosswalk

Interview Survey Focus groups Observation BRYCS

database

Did collaboration
strengthen among child
welfare and refugee X X X X
serving agencies?
- As a result of the cross-
service trainings, did
refugee serving agencies
collaborate and
consolidate resources
more efficiently to serve
refugee youth and
children?

Are recipients of X X X
services being better
served, as a result of
cross-service trainings?
-Are more refugee youth
and children being
served by the cross-
trained agencies?
- Are duplication of
services decreasing, due
to an increase in
resource sharing and
collaboration among
refugee serving
agencies?

Are the refugee serving X X X

agencies involved
appropriately equipped
culturally to provide
services?
-Is cultural competency
being implemented in
 11

the agencies to support

culturally appropriate
methods and
dissemination?

Are the cross-service X X X X

trainings effective for
the refugee serving
agencies?
-How was the cross-
service trainings
effective for refugee
communities in
comparison to the
communities whose
agencies were not
trained?

Section 4: Reporting Plan

The data collected for this evaluation will be disseminated to key stakeholders and made

available on the BRYCS website. The agencies who directly participated in the evaluation will be

sent an electronic copy of the evaluation report and an executive summary report. The executive

summary will also be made available on the BRYCS website. The refugee youth community and

their families that participated in the evaluation will be provided hard copies of the executive

summary.
12

Section 5: Detailed Budget

Program Director/Principal Investigator: Jasmine May & Jolie Azorlibu

ROLE ON Cal. Acad. Summer INST.BASE SALARY FRINGE

NAME PROJECT Mnths Mnths Mnths SALARY REQUESTED BENEFITS TOTAL

Jasmine May PD/PI 12 $75,000 $18,750 $5000 $23,

Jolie Azorlibu PD/PI 12 $80,000 $20,000 $5000 $25,

Project Coordinator staff 12 $35,000 $8,750 $2,000 $10,

13

Research Assistant staff 9 $17,000 $4,250 $1,000 $5,

Office Assistant staff 12 $28,000 $7,000 $1,500 $8,

Office Assistant staff 10 $28,000 $7,000 $1,500 $8,

SUBTOTALS
$65,750 $16,000 $81,
CONSULTANT COSTS
N/A
EQUIPMENT (Itemize)

N/A

SUPPLIES (Itemize by category)

Participants incentives- $10 for 100 participants= $1000
Paper supplies- $500 $8,000
Office supplies- $500
Interpreter fee- $60/hour (100 hours for duration of evaluation) = $6,000
TRAVEL
Domesti
c
($5000) $5,00
INPATIENT CARE COSTS N/A
OUTPATIENT CARE COSTS N/A
ALTERATIONS AND RENOVATIONS (Itemize by category) N/A

OTHER EXPENSES (Itemize by category)

CONSORTIUM/CONTRACTUAL COSTS DIRECT COSTS

SUBTOTAL DIRECT COSTS FOR INITIAL BUDGET PERIOD (Item 7a, Face Page)
$
CONSORTIUM/CONTRACTUAL COSTS FACILITIES AND ADMINISTRATIVE COSTS

TOTAL DIRECT COSTS FOR INITIAL BUDGET PERIOD

$ 94,
DETAILED BUDGET FOR INITIAL BUDGET PERIOD FROM: Dec 2016
THROUGH: Nov 2017

DIRECT COSTS ONLY

List PERSONNEL (Applicant organization only)

Use Cal, Acad, or Summer to Enter Months Devoted to Project
Enter Dollar Amounts Requested (omit cents) for Salary Requested and Fringe Benefits
14

Section 6: Detailed Timeline

Dec 2016 Jan 2017 Feb 2017 Mar 2017 Apr 2017 May 2017

Post Select 5 Conduct Trainings Post training

applications for agencies for interviews takes place interviews
cross-service treatment
trainings group

Contact non- Observe Observe

service control and control and
15

applicants to treatment treatment

join study agencies agencies

Select 5 Distribute
agencies for Pretest
control group surveys

Jun 2017 Jul 2017 Aug 2017 Sep 2017 Oct 2017 Nov 2017

Observe Posttest
agencies surveys
(control and
treatment)
interaction
with
community

Focus groups

Observe Posttest
collaboration interviews
with other
agencies
(control and
treatment)
16

Reference

Morland, L., Duncan, J., Hoebing, J., Kirschke, J., Schmidt, L. (2005). Bridging Refugee Youth
and Children's Services: A Case Study of Cross-Service Training. Child Welfare, 84(5),
791-812.