Feedback SU D1.

10 Validation and Impact Assessment

Formal Structure of Task:

WP1. SOCIAL DIMENSIONS: CO-DESIGNING CITIZEN OBSERVATORIES

T1.6. Validation and Impact Assessment of the GT2.0 CO

Develop and implement the methodology for measuring and monitoring progress towards
achieving the GT2.0 objectives according to the identified indicators and outputs. It will
validate the CO against the requirements of various local stakeholders as well as the overall
GT2.0 concept for CO, comparing their functionalities, scales extent of use, types of actor
interactions, etc. against the initial baseline. Moreover, this task will assess the social and
institutional impact per case.

D1.10 Methodology for validation and impact assessment (M3) Report detailing the
methodology for validation and impact assessment, include baseline measure of
stakeholder interactions (T1.6).

MS2. Baseline measure of stakeholder interactions for future impact assessment.

The UNESCO-IHE provide a methodology based in the Logic Model. The monitoring, validation and
impact assessment will provide a comprehensive a feedback tool to inform improvements to the final
citizen observatories and innovate specific aspects of the products, closing the gap between a proven
technology and a commercialized product, as well as for producing timely feedback to enable learning
and adjustments during the course of the GT2.0 project (D.1.10).

Framework to Validate and Assess Results of Ground Truth 2.0

Results: Social
Societal Needs: Added Value Economic
Participation in natural Environmental
resources management for Institutional
sustainable development Impacts
Societal & economic benefits
Global uptake of CO
GT2.0 CO Concept: full feedback-loop in Enable technologies
the information chain from citizen-based
data collection to knowledge sharing for Outcomes
joint decision-making, cooperative Changes in social practices
planning and environmental stewardship of the CO community

Monitoring
Objectives: Activities:
6 general Inputs Co-designing & Upscaling Outputs
Specific CO Demo- Case the observatory

Source: GT2.0 D1.10 Methodology for Validation and Impact Assessment

There are three level to asses results of the GT2.0 CO:

1. Progress monitoring measuring general project-wide progress against the Ground Truth 2.0
objectives according to identified indicators and outputs. (blue box in the above figure)

2. Validation of the Demo Case Observatories will document the observable (evolving) social
practices in the CO communities, and assess the performance in terms of: a) coherence of
observable practice with the requirements of local stakeholders and with the overall Ground
Truth 2.0 concept for citizen observatories; b) effectiveness in achieving results (comparing
results to objectives); and c) efficiency (comparing results to inputs).

3. An Impact assessment will capture social, institutional, economic and environmental changes
that can be attributed to the Ground Truth 2.0 Demo Cases, comparing the situation during
and following the up-scaling of the citizen observatories to an early initial baseline (grey box
in the above figure).

Regarding these three levels, it is necessary to underline:

1. In the progress monitoring only two of the three overarching objectives of GT2.0 are
considered. It means that enabling technologies is a transversal objective, and this is
understandable, because the main output is a technology product, the CO. However, it is very
important that the Swedish demo-case set up its own objectives, which should be related to
the citizen participation in water/soil resources management and how the co-creation and co-
design of the CO enhance the social capital.

2. The validation of the demo-case CO will mainly concentrate the first year on coherence
criteria, it means, the changes in social practices of the CO community to improve its societal
needs and the extent of assimilation of GT2.0 CO concept.

For this reason, is important to do as a first step the next two aspect.

i. Territorial characterization of the community chosen to scale-up the CO. This is a

preliminary deskwork to release the existing conditions and capacities of the
rural/urban territory (for example, doing a PEST assessment to analyse political,
environmental, social and technological factors).

ii. Baseline of the GT2.0 CO demo-case is a fundamental aspect to the whole process of
monitoring, validation and assessment of the results of this project. However, a
methodology or even some basic guidelines are not developed in the document D.1.1
For this reason, it is necessary to:

a. Identify the state of art of the CO demo-cases central challenge

Collection of baseline data through literature review, qualitative and

quantitative analyses, and field programs carried out as part of territorial
planning practices.
A review literature of secondary information on local environmental and
socio-economic conditions from the local authorities to the national agencies.
 Liaison with the GT2.0 Co-Design Team to gather data and to formulate an
understanding of CO activities.
Obtain data on national and regional environmental socio-economic
conditions provided by the State Statistical Institute.

b. Validate the baseline information with stakeholders during the interaction

moments.

Here is important to identify the social necessities of the stakeholders and

identify the elements which may conditionate the participation and
engagement of communities in the co-design and co-creation of the CO demo-
case. That is barriers and incentives. Then, it is possible to select specific
variables to build indicators that directly measure the achievement of specific
case objectives.

c. Relate the baseline information to the variables and indicators propose in the
D1.10 to assess outputs, outcomes and impacts of the CO demo-case.

it worth to underline the next chart which should be considered for the exercises exposed previously:

Assess the Elements of Social Practice

Dimension Case-specific outcomes
Inventory of things, technologies, tangible entities
and input materials required to engage with the Citizen
observatory

Possible distinction between materials produced by the

Materials
project, available as context factor (infrastructure), or
contributed by stakeholders where necessary

Link to intervention logic via analysis of provision

(inputs) or access (I&B, Context analysis) to materials
Measures of skills, know-how and techniques required
to engage in the Citizen Observatories
Possible distinction between knowledge sources
developed and circulated as part of the project and
Competencies competencies and prior experience of local
stakeholders required to receive, and implement it
Link to intervention logic via analysis of generic
knowledge packages (inputs) and contextualization
activities (activities, T1.2)
Representations of ideas, understanding and
aspirations reflected in the design of the Citizen
Observatories
Meanings Possible distinction between status functions and
definitions built into the platforms as part of the
project, and social patterns emerging through user
activities
 Link to intervention logic via analysis of user-defined
visions (objectives), associations and classifications
(activities, T1.2).
Configuration of people engaging in the Citizen
Observatory and thus contributing to its continued
existence

Possible distinction between users actively recruited

Carriers
through implementation of the engagement strategy
and independent activities of local users

Link to intervention logic via analysis of status and

commitment of participants (activities, T1.4)
Source: GT2.0 D1.10 Methodology for Validation and Impact Assessment

3. The impact assessment is limited to the lifetime project (3 years), but it includes the
assessment of impacts which are seen to overtake this limit in time and space based on
assumptions and derivatives. For this reason, it would be necessary to clarify and build specific
indicators to 1) CO-related environmental effects, and 2) cumulative environmental effects.
The first ones are changes to the biophysical or human environment that will be caused by
the CO demo-case or activity arising solely because of the proposed principal works and
activities, as defined by the scope of the CO demo-case. The second ones are changes to the
biophysical or human environment that are caused by an action associated with the CO demo-
case, in combination with other past, present or reasonably foreseeable future projects or
activities that have been or will be carried out.

Also, it would be desirable to organize the environmental impacts by magnitude, extent,

frequency, duration or intensity of the impacts. This specific information allows to identify
where the resources need to be efficiently focalized to mitigate an impact or stimulate the
participation or engagement of a stakeholder/receptor. Not to mention, the input information
for a future study on the level of resilience of the inhabitants. These criteria are described in
the guidance of the CEA Agency (FEARO 1994) and the Final EIA Guidelines (NBENV 2009).

Other aspects:

Supposedly the CO demo-case is not a product, but the final aim is the uptake of the concept
by market. It would be important to engage this stakeholder from the beginning of the scale
co-design process to deal with the future business/commercial premises.

Taken into account the experience of EgovLab in co-design and co-innovation projects, it
would be important to propose some indicators that measure the performance of this
activities according to past experiences (bottleneck, success stories, key challenges, citizen
engagement strategies etc.) and which to strengthen the citizen participation.

Social mapping could be used not just as a participatory activity to gather geographical
information, but as an experiment to measure the closeness of inhabitants to undertake
georeferentiation actions. If this is the case, it would be important to develop a methodology
to use social mapping and indicators to assess related inhabitant skills.
 According to the document, It should the citizens be linked with the Policy and decision
makers to influence policy and decision making processes by improving the response to
monitoring requests and use of a broader knowledge base and better data sets for policy
responses. To provide accurate and opportune information about the condition of the
environment is not sufficient to influence the taking decisions, which have specific times and
spaces to be done. It would be interesting to evaluate the possibility to link the information
derive from the CO demo-case not only with the global marked uptake of the GT2.0. CO
concept, but with technical platforms of public institutes, too. This compatibilization should
be measure and the GT2.0 allow this possibility.

Finally, the Validation and Impact Assessment Methodology makes part of the result-based
management, but it would be important to assess the management of the project itself for
two reasons. Firstly, because this methodology D1.10 propose an adaptive management
process (Plan-Do-Evaluate-Adjust) where adjust management actions and arrangements need
to be done to enhance effectiveness; and secondly, to really integrate elements of the theory
of change (influential factors, strategies, turning points, assumptions, etc.).