Vous êtes sur la page 1sur 8

The Value Template Process: A Participatory

Evaluation Method for Community Health


Partnerships
Alice J. Hausman, Rickie Brawer, Julie Becker, Robin Foster-Drain, Charmaine Sudier, Robin Wiicox,
and Barbara J. Terry
ooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo

T
his article presents the results of a pilot implementation of current and prospective partners and the leadership of
an evaluation process designed to help community health partner organizations in order to be sustainable. Unless
partners see a benefit to their participation, they will
collaboratives obtain relevant information for planning and
eventually stop participating and new partners will be
evaluation. The Value Template Process assists collaboratives to difficult to engage.'
identify performance and impact indicators that are meaningful This article presents the results of a pilot imple-
and measurable with accessible data. The process also mentation of an evaluation process designed to assist
encourages communication and engagement in assessment
community health collaboratives meet their informa-
tion needs. Using principles of participatory evaluation
among collaborative members. The pilot study demonstrated and social capital, the Value Template Process identi-
that the process's underlying assumptions of social capital were fies performance and impact indicators that are mean-
valid and that the process was feasible and useful to the ingful to collaboratives and measurable with accessi-
community heaith collaborative. ble data. The process also encourages communication
among collaborative members and their engagement
in all phases of assessment, planning, and evaluation.
KEY WORDS: community health collaboratives, evaluation,
The pilot study assessed: (1) if the underlying assump-
participatory evaluation
tions of the evaluation model were valid and (2) if the

The authors thank the following members of the Haddington Collaborative Health
Increasingly, public health practice is turning to com- Project for their contributions: Ruth Bazemore, Ruth Campbeil, Patricia Fuitz, and
munity collaborative models to improve population Grace Haywood. Their insights, energy, and willingness to guide us enabled the
health status. These collaboratives are voluntary part- success of this endeavov-^«e are very grateiul.
nerships among public and private organizations that Corresponding author: Aiice J. Hausman, PhD, MPH, Department of Public
join forces and link resources to improve health among Health, Temple University, 304 Vivacqua Hall, Philadelphia, PA 19122 (e-mail:
hausman@temple.edu).
residents of defined communities.^'^ Despite the appeal
of this practice model, partnerships and coalitions are
fragile entities, difficult to maintain in part because they Alice J. Hausman, PhD, MPH, is Professor and Chair, Department of Public Health,
face significant barriers to the use of appropriate infor- Temple University, 304 Vivacqua Hall, Philadelphia, Pennsylvania.
mation for strategic planning.^-'' Often, collaboratives Rickie Brawer, MPH, is Associate Director, Office to Advance Population Health,
do not set clear goals and measurable objectives that Thomas Jetterson University Hospitai, Philadeiphia, Pennsylvania.
members can identify as success, nor do they work in- Julie Becker, PhD, MPH, is a Research Consultant, Public Health Consultants,
Philadelphia, Pennsylvania.
ternally to build the infrastructure necessary for strate-
gic planning and implementation. Collaboratives must Rcbin Foster-Drain, MD, MPH, is President and Executive Director, To Our
Children's Future With Hearth, Phiiabelphia, Pennsylvania.
also be able to assess the impact of their activities and
be able to communicate the benefit of participation to Charmaine Sudier is Director of Community Development and Organization, To Our
Children's Future With Health, Philadelphia, Pennsylvania.
Robin Wiicox, MPA, is Vice President for Community Health, Texas Health
Resources, Dates.
J Public Health Management Practice, 2005,11(1), 65-71 Barbara J. Terry, RN, BSN, MS, NHA, is President and CEO, Institute for Healthy
© 2005 Lippincott Williams & Wiikins, Inc. Communities, Harrisburg, Pennsylvania.
8B I Journal of Public Health Management and Practice

process was feasible and produced results useful to the Research on participatory methods and coUabora-
collaborative. tives demonstrates that when partnerships engage in
assessment and evaluation, they increase their owner-
ship of both methods and results.""" Simply put, the
# The Value Template Process Value Template Process engages partnership members
in a dialogue about why they participate and what their
The Value Template Process was created by the Penn- expectations are for results. They are then asked to op-
sylvania Institute for Health Communities (an affiliate erationalize their expectations by offering indicators of
organization of the Hospital and Health System Asso- success. Table 1 presents an example "template" that
ciation of Pennsylvania) and a work team composed provides a structural framework for organizing and
of representatives from public health, business, health presenting the derived indicators across all partnership
care, health planning, and education. The Value Tem- constituencies for easy communication. The columns
plate Process is built upon principles of social capital reflect the membership of the collaborative, although
and participatory research. The core idea of social cap- the actual groupings or titles will vary according to how
ital is that social relationships affect individual behav- each collaborative defines its structure. The rows are the
ior and social networks influence the productivity of indicators of value identified by the groups through the
organizations.^^ Research has shown that societal fea- Template Process, as it is described below. In each cell
tures such as strong social networks, high levels of trust are examples of indicators derived from the piloting
among community constituents, and civic participation of the process. This format provides a visual way for
(to name a few) are related to improved health indica- partnership constituencies to see similarities and differ-
tors, so the concept has particular relevance to com- ences, and to identify sectors that can be strengthened.
munity health initiatives.^"'" Community health initia- While the indicators themselves are useful to the
tives employ social capital principles in that issues of partnership, the underlying premise of the Value Tem-
trust, information sharing, and strength of inter-agency plate Process is that the communication of what mem-
relationships are critical to maintaining cohesion and bers value and the act of identifying the indicators
generating effective action.''''^'' Here, the Value Tem- of value are most important. The acts of discussion
plate Process applies concepts and principles of social both within and across partnership constituencies fa-
capital to community health partnerships, viewing the cilitate consensus-based evaluation and can help build
network of different agencies as the social unit where an infrastructure for ongoing assessment and strategic
social capital is built and bartered. Coleman's' presen- planning.
tation of social capital as a product of rational, pur- The Value Template Process is different from other
poseful individuals who build social capital to maxi- tools that have been developed to evaluate partnerships
mize their individual opportunities underlies the Value or coalitions because it focuses more on processes of dis-
Template Process. The process does not actually mea- covery and communication rather than the actual data
sure social capital, but rather focuses on demonstrat- collection. Rather than actually measuring or monitor-
ing to members how they benefit both individually ing aspects of collaboratives, it sets the stage for the
and collectively, and how they can further their self- use of methods and tools designed for those purposes.
interest by supporting and sustaining the partnerships. For example, the Community Toolbox is a resource of
The underlying premise based on social capital is that methods and tools community health coalitions can use
the more participating individuals and organizations to chart their activities and progress along a pathway
realize their own self-interests through participation, to community change.'"*'^ Butterfoss, Goodman, and
the more likely they are to continue to participate. Wandersman'* have studied factors that support the

TABLE 1 o Example of \n': cells present examples of specific indicators of value specified by each focus group

Stakeholders

Value indicator Health care/ Small community


domain business/politicians organizations Youth groups Faith-based groups Block captains
Personal benefits Better job performance Si<ill development Life si<ills Spiritual mission met Use and value of
personal experience
Benefit to organization Enlianced reputation Use of evaluation data Inter-group cooperation Expansion of mission Shared resources
Community benefits More/better services Children's programs More youth activities Increased safety More parental involvement
Performance Roie definition Mutual support Networking Networking Shared information
The Value Template Process I 8 7

functioning and effectiveness of coalitions and have de- and social issues in West Philadelphia. The HCHPC cur-
veloped tools for measuring key features of coalitions rently consists of over 40 organizations and 30 commu-
that assist in formal evaluations. The Template Pro- nity residents. Projects include youth literacy programs,
cess facilitates the flow of the information that the tools nutrition programs, health services for the elderly and
generate among partners. The Value Template Process children, and community beautification projects. The
also complements previous efforts to identify indica- collaborative receives its funding from a combination of
tors that may better reflect the way collaboratives work foundation and federal grants as well as through a small
to achieve health improvements'^"^" by generating ad- donor base. The collaborative is organized into sub-
ditional categories of relevant outcomes. committees based upon common backgrounds, such
as health care providers, organizations involved with
youth, and faith-based organizations (see Figure 1).
• Applying the Template Process
The Template Process
The pilot test of the Value Template Process was con-
ducted in one urban community health partnership, Table 2 presents the Value Template Process phases as
the Haddington Community Health Project Collabora- they were implemented in the pilot test. Generally, the
tive (HCHPC). The research team included university- process involves a series of focused discussions and val-
based personnel and members of the collaborative. idation of results. The first phase involves engagement
This partnership was selected for the pilot study be- with the collaborative and defining procedures; the sec-
cause it sustains a high level of organizational stabil- ond phase collects and analyzes focus group data and
ity and community action. We felt that if principles validates the analysis with participants; the third phase
of social capital were relevant to the Value Template brings results back to the larger collaborative and facil-
Process, they would be revealed in our work with the itates planning for follow-up; the last phase involves
Haddington Collaborative. Temple University's Insti- integration of the process into ongoing strategic plan-
tutional Review Board reviewed and approved the re- ning and performance monitoring.
search protocol; participants signed informed consent
forms and gave permission to audiotape. Collaborative
members helped phrase the consent letters and proce- 9 Results
dures for presenting them to focus group participants.
Pbase 1
The pilot test research team was established by drawing
Haddington Community Heaitii Project Collaborative
upon the HCHPC leadership's guidance to ensure full
The Haddington area of West Philadelphia is 93% participation of the collaborative in the Template Pro-
African American with a high level of poverty (42.2% cess. The team included individuals who had worked
live below 150% poverty) and low level of education together before in both practice and academic settings.
(19.3% have less than high school education). Children Engagement of the collaborative was facilitated by the
and youth are at increased risk for pregnancy, sexually pre-existing relationships between the academic re-
transmitted disease, use of drugs and alcohol, and vic- searchers and collaborative members who were com-
timization from violent crime.'^' In an effort to address mitted to expanding the capacity of the collaborative
these issues, the HCHPC was formed in 1995 as a part- in the areas of strategic planning and evaluation. These
nership of a church, a community group, a hospital and nriembers were approached early with the idea of pi-
the Philadelphia Housing Authority to address health loting the Template Process, and subsequent meetings

Haddington Collaborative Health Project

zzb=
(1) Local (1) Health care (1) Politicians (2) Block
businesses providers political groups captains

(3) Faith-based (4) Youth (5) Small


FIGURE 1. Structure of the collaborative organizations groups community-based
and focus group definition. organizations
68 I Journal of Public Health Management and Practice

TABLE 2 o Procedure for using the value template

Phase one Outline partnership structure and constituencies:


• identify accountabiiity relationships (eg, between the board and all other partnership members, or among subcommittees).
• identify different types of activities iini^ed to partnership members;
• Involve partnership in process.
Phase two Conduct focus groups:
• Engage board members in defining and organizing focus groups.
• Group participants by professional affinities and/or activities into focus groups.
• Conduct focus groups using questions designed to identify indicators refiecting their respective interests and concerns.
• Summarize themes of expectations and concerns by group and present to group for vaiidation.
• Reach agreement on indicators of success ("vaiue") for each group.
Phase three Aiign indicators with constituencies using vaiue tempiate:
• Organize all indicators into categories that emerge from aii focus groups.
• Present aii resuits to partnership members with faciiitated dialogue.
• Identify existing data sources, discuss practicaiity, and cogency.
Phase four Coiiect data on indicators:
• Confirm consensus and participation.
• Coiiect, organize and interpret data.
• Review resuits with partnership using faciiitated diaiogue.
• Assess subsequent strategic pianning process.
Sustainability integrate data coiiection and systematic review into partnership process.

with staff from the lead agency of the collaborative (see Table 3). More than 90% of the participants were
and interested HCHPC members built support for par- African American.
ticipation. The pilot project was well-received by the Focus group questions were directed to yield in-
HCHPC membership and 4 additional members were formation in 3 areas: (1) the value of participation in
appointed to be a part of the research team to ensure the partnership, in terms of both the individual and
that the community was appropriately represented in their parent organization, (2) perceived indicators of the
the research design and methodology. collaborative's success, and (3) identification of what
was working well or needed improvement within the
partnership.
Phase 2 The tapes and notes from the 2 note-takers present at
The team worked with the collaborative to identify ap- each focus group were reviewed and abstracted by at
propriate focus group constituencies, questions, and least 2 researchers and 3 members of the research team
procedures for communication of results. HCHPC reviewed the abstracts. Two domains of themes were
members helped to determine the structure and num- searched for within the data. The first set focused on
ber of the focus groups, assisted with the design of the the identification of predetermined categories of value
questions to be asked, and took responsibility for con-
tacting participants and organizing the groups. The fo-
cus groups followed the subcommittee structure, ex- 1ABLE3 O Focus group participation
cept where it was necessary to group like subcommit- No. who No. who
tees to generate a group of sufficient size. Five focus participated participated
groups were held with the following constituencies: No. invited in focus in review
hospitals/businesses/political officials, youth group Groups to participate group group
leaders, block captains (residents elected by their neigh-
bors on their block to serve as liaisons with city agen- Hospital/business/iegislation 9 7 6
cies), faith-based organizations, and small community- Smaii community-based 18 4 5
based organizations (see Figure 1). The focus groups organizations
were held between January and June of 2002 and were Youth group 13 6 8
conducted by a professional facilitator. Participants Faith-based groups 14 9 12
were offered refreshments and given subway tokens Biock captains 30 8 5
as small appreciation gift. Focus groups ranged in size Totai 84 34 36
from 4 people to 8 people for a total of 34 participants Overall participation rate,% 40 43
The Value Template Process I 69

indicators. Specifically, we looked for stated benefits of to measure any of the listed indicators. This process
participation for the individual, their organization, and helped to edit the list from the perspective of practical-
the community served; stated markers or indicators of ity; only those indicators that were likely to be actually
success; and comments about the internal performance measured either from existing data or other data easily
of the collaborative. The second set focused on social obtained using existing collaborative resources stayed
capital: data were read for how concepts of social capital on the list. The result of the editing was used to com-
were expressed in the different groups. Within each of plete a Value Template, creating a matrix of indicators
these domains, constructs were modified as we learned that was disseminated throughout the collaborative. A
more from the focus groups, utilizing techniques from fuller discussion of the indicators and data sources is
grounded theory study.^ presented elsewhere.'^
Next, we provided feedback to each focus group in-
dividually to clarify information, language, and to val-
Phase 4
idate indicators. Constituency members unable to at-
tend the focus group could participate in the validation The final phase of the Template Process is to actually
groups and their participation helped to ensure that collect the data on the specified indicators, analyze re-
information from the focus groups reflected the larger sults, and interpret them through the eyes of the collab-
constituency, particularly in focus groups that had less orative members. The pilot implementation project did
initial participation (see Table 3). We presented the list not carry through this phase, and, as such, the degree
of indicators we had "heard" to the validation group to which results were integrated into ongoing collabo-
and participants then either added more indicators or rative planning was also not assessed.
clarified our language to better reflect their intended
meaning. In none of these validation meetings were in-
dicators removed from the list. Phase Two provided • Discussion
an opportunity for enhanced communication within
the constituencies that comprised each focus group. The pilot test of the Value Template Process focused as-
It encouraged sharing of information and highlighted sessment on (1) the validity of the underlying assump-
commonalities, differences, and expectations of the tions of the evaluation model and (2) the feasibility
individual members within the constituency. Improved and utility of the process from the perspective of the
communication among collaborative members fosters collaborative.
the building of "trust," a major tenet of social capital.
Validation of underiying assumptions
Phase 3
The first assumption made for the Value Template Pro-
The third phase of the Template Process is character- cess was that principles of social capital are applicable
ized by synthesizing the results of the individual fo- to community health partnerships as entities and that
cus groups for presentation to the collaborative as a social capital dynamics of individual self-interest, reci-
whole. Bimonthly, all members of the HCHP are in- procity, and communication were relevant to the collab-
vited to a meeting that functions like a town meeting, orative's internal functioning and achieved result. We
where information is presented from the subcommit- found that social capital was clearly expressed in the in-
tees to the larger collaborative and time is allotted for dicators of value of participation. Individual members
open discussion. A complete list of all the value indi- articulated a number of personal benefits they derived
cators identified from the focus groups was presented from participation, ranging from meeting their spiri-
to over 40 collaborative members during one of these tual needs to improved job performance and satisfac-
open meetings. There was discussion about the indica- tion. Benefits to organizations that sponsored participa-
tors, ideas were shared across constituencies, and the tion of their representatives, such as community health
list was validated with the larger group. Again, some clinics, hospitals, as well as churches, were specified in
additions were made to the list. Importantly, the discus- more economic terms, such as increased market share
sion shifted to looking across the small group results through improved community relations and increased
to identify common concerns and themes. For exam- attendance. Identification of these important expecta-
ple, improved quality of life was a community bene- tions for personal and organizational investment in the
fit that all groups specified. Similarly, the need for in- collaborative sets the stage for providing information
creased communication throughout the collaborative on those factors that will sustain continued participa-
was widely shared. tion. Reciprocity was noted as a key benefit for par-
At this time, collaborative members were asked to ticipation. The value of the collective effort was enu-
identify data they already collect that might be used merated in the faith-based group in terms of resources
70 I Journal of Public Health Management and Practice

that were shared. Other individual members noted how tation. Discussions were Hvely and collaborative mem-
their job-related tasks in the community were facili- bers were supportive of the effort. Whereas the pilot did
tated by the different groups working together. Specifi- not follow the board's actions in subsequent meetings,
cally, sharing information with collaborative members it appeared that "next step" plans were being made
about available resources, upcoming events, and prob- to help move the process from a research exercise to a
lem solving assisted organizations in promoting their component of strategic planning.
services and programs while also improving commu- In many respects, we did expect that HCHPC would
nity access. This was valued by individuals and the be fully engaged in the Template Process because of
organizations they represented. their demonstrated interest and previous experience
The importance of communication both within and in doing evaluation and planning. The pre-existing
across subcommittees in the collaborative was clearly relationships between the academic researchers and
stated in a number of the focus groups. Within groups, HCHPC members also greatly facilitated implementa-
people felt their ideas were heard and applied. How- tion. Furthermore, the early involvement of the collabo-
ever, the need for increased communication across sub- rative minimized the time spent in "learning" about the
committees was identified in a number of different con- structure and activities of the collaborative. We would
stituencies and this was seen as one pathway for help- expect that in some instances, more time would need
ing the collaborative move forward with its mission. to be spent in Phase 1 than was spent in the pilot study.
Improved communication across constituencies helps Even with a well-established and successful collabo-
to create bridging social capital, believed to be cru- rative, the Value Template Process required additional
cial to communities in "getting ahead."^ The format of resources. The use of an external facilitator to conduct
the template itself allowed participants a visual means the focus groups and the review discussions was very
to identify commonalities and differences amongst the successful, and we would recommend this in any repli-
constituency groups. This visual form helped to facili- cation. The process requires sufficient research capac-
tate additional discussions about goals and steps to take ity to conduct preliminary analyses and to establish
towards progress. methods for following through on measuring indica-
The second assumption of the Value Template Pro- tors. Here, the academic members of the research team
cess is that it is fully participatory. Our hope for the conducted the bulk of the focus group analyses. Ac-
process was that collaborative members would become cess to research expertise, either through partnering
engaged in the process, own it, and use the results. with public health professionals with research train-
We found that the process was successful in generat- ing or by having staff with expertise, is an important
ing participation at all levels. The perceived role of the component of the Template Process. A workbook guide
collaborative in the process was aptly characterized by for procedures and links with existing evaluation tools
a HCHP board member when she commented on the for measuring identified indicators would also facili-
importance of participating in the pilot study: "It's up tate replication. The Template Process needs to be repli-
to us to see what's working and what isn't." Partic- cated with newer and/or less organized collaboratives
ipation in the focus groups varied across the differ- to see if implementation is as easy and if it is as help-
ent constituencies, but all were represented. In all, 28 ful with planning and sustainability. Such replication
member organizations participated in the focus groups would also provide further insight into the role of so-
and validation process, approximately 38% of the total cial capital in community health collaboratives and its
collaborative. use for evaluation.

Feasibility, utiiity, and limitations


The pilot implementation of the Value Template Process • Conclusion
demonstrated that it is a feasible process and that it can
provide results that are useful to the collaborative. The The Value Template Process utilizes principles of so-
concepts and procedures of the Value Template Pro- cial capital and holds promise for helping commu-
cess were easily communicated and members quickly nity health partnerships meet their information needs
engaged in the process and organized its implemen- for accountability and sustainability. It can provide
meaningful and measurable indicators of success, fos-
ter communication and trust, and build social capital
The pilot implementation of the Value Template Process within the partnership. It can provide vital informa-
demonstrated that it is a feasible process and that it can tion and, after further refinement, it may assist in fa-
provide results that are useful to the collaborative. cilitating participation and enhance sustainability over
time.
The Value Template Process I 71

REFERENCES patory Action Research. Thousand Oaks, CA: Sage; 1991:19-


55.
1. Green LW, et al. Partnerships and coalitions for community-
13. Chambers R. Whose Reality Counts? London: Intermediate
based research. Public Health Reports. 2001;116(suppl.):20-
Technologies Publications; 1997.
31.
14. Coombe CM. Using empowerment evaluation in commu-
2. Green LW, Mercer SL. Can public health researchers and
nity organizing and community based health initiatives. In:
agencies reconcile the push from funding bodies and the
Minkler M, ed. Community Organizing and Community Build-
pull from communities? American Journal of Public Health.
ing for Health. New Brunswick, NJ: Rutgers University Press;
2001;91(12):1926-1929.
1997:291-307.
3. United Way. Agenci/ Experiences with Outcome Measurement:
15. Francisco VT, et al. A methodology for monitoring and evalu-
Survey Findings [item no. 0196]. Alexandria, VA: United Way
ation community health coalitions. Health Education Research.
of America; 2000.
1993;8:403-416.
4. Hausman A. Implications of evidence-based practice for
16. The Community Tool Box [Web site]. Available at: http:/ /ctb.
community health. American Journal of Community Psychol-
ku.edu/. Accessed April 5, 2004.
ogy. 2002;30(3):453-467.
17. Butterfoss FD, et al. Community coalitions for prevention and
5. Berkman L, et al. From social integration to health:
health promotion: factors predicting satisfaction, participa-
Durkheim in the new millennium. Social Science and Medicine.
tion, and planning. Health Education Quarterly. 1996;23(1):65-
2000;51:843-857.
79.
6. Kawachi I, et al. Social capital and self-rated health: a contex-
18. Cheadle A, et al. Environmental Indicators: A tool for evaluat-
tual analysis. American Journal of Public Health. 1999;89:1187-
ing community-based health-promotion programs. American
1193.
Journal of Preventive Medicine. 1992;8:345-350.
7. Coleman J. Social capital in the creation of human
19. Cohen S, et al. Social Support And Measurement And Interven-
capital. American Journal of Sociology. 1988;94(suppl.):95-
tion: A Guide for Health and Social Scientists. New York: Oxford
120.
University Press; 2000.
8. Putnam R. Bowling Alone: The Collapse and Revival of American
20. Hart M. Guide to Sustainable Measures. Available at: http://
Community. New York: Simon and Schuster; 2002.
www.sustainablemeasures.com/. Accessed December 3,
9. Kreuter M, Levin N. Social Capital Theory: implications
2003.
of community based health promotion. In: DiClemente R,
21. Norris T. Community Initiatives, Inc. [Web site]. Available
Crosby R, Kegler M (eds). Emerging Theories in Health Promo-
at: http://www.communityinitiatives.com/home.html. Ac-
tion Practice and Research. New York: Jossey-Bass; 2002:228-
cessed December 03, 2003.
254.
22. Philadelphia Health Management Corporation. Haddington
10. Kawachi I, et al. Social Capital, Income Inequality and
Collaborative Health Needs Assessment. Philadelphia: Philadel-
Mortality. American Journal of Public Health. 1997;87(9):1491-
phia Health Management Corporation; 1996.
1498.
23. Creswell J. Research Design: Qualitative and Quantitative Ap-
11. Miller R. Healthy Boston and social capital: application, dy-
proaches. Thousand Oaks: Sage Publishers; 1994.
namics, and limitations. National Civic Review. 1997;86(2):157-
24. Hausman A, et al. Identifying Value Indicators and So-
166.
cial Capital in Community Health Partnerships. Paper in
12. Whyte WF, et al. Participatory action research: through prac-
preparation.
tice to science in social research. In: Whyte WF, ed. Partici-

Vous aimerez peut-être aussi