Projet de programme / Draft program - c2015 - UFRGS
←
→
Transcription du contenu de la page
Si votre navigateur ne rend pas la page correctement, lisez s'il vous plaît le contenu de la page ci-dessous
c2015
Projet de programme / Draft program
MISE À JOUR / UPDATED 2015-04-08
TABLE DES MATIÈRES TABLE OF CONTENTS
DIMANCHE 24 MAI – INFORMATIONS DÉTAILLÉES SUNDAY, MAY 24 – DETAILS
LUNDI 25 MAI – INFORMATIONS DÉTAILLÉES MONDAY, MAY 25 – DETAILS
MARDI 26 MAI – INFORMATIONS DÉTAILLÉES TUESDAY, MAY 26 – DETAILS
MERCREDI 27 MAI – INFORMATIONS DÉTAILLÉES WEDNESDAY, MAY 27 – DETAILS24 MAI DIMANCHE SUNDAY MAY 24
13h-16h
Approche fondée sur les droits de la personne: Un regard sur l’évaluation axée sur le genre
Jeiran Rahmanian, Vanessa Anastopoulos
ATELIERS/WORKSHOPS
Consulting Challenges
Gail V. Barrington
ATELIERS/WORKSHOPS
Les protocoles d’évaluation faisant appel à des méthodes mixtes
Jacques Bérard
ATELIERS/WORKSHOPS
Participatory Evaluation: Learning for a Change
Linda E. Lee
ATELIERS/WORKSHOPS
Skills for Visual Data and Non-Lethal PowerPoint Presentations
John Burrett
ATELIERS/WORKSHOPS
9h-12h
Évaluer les réglementations pour mieux anticiper leurs effets : Balises pour fiabiliser les Analyses d'Impacts
Réglementaires (AIR)
Eva Anstett
ATELIERS/WORKSHOPS
Facilitation skills for evaluators
Jennifer Birch-Jones
ATELIERS/WORKSHOPS
Shared learning and participatory evaluation: the systematization approach for the assessment of
development interventions
Esteban Tapella, Pablo Rodriguez Bilella
ATELIERS/WORKSHOPS
Understanding and Using Contribution Analysis
Kaireen Chaytor, John Mayne
ATELIERS/WORKSHOPS
Using Microsoft Excel to Enhance Data Analysis in Evaluations
Sandra Sellick
ATELIERS/WORKSHOPS
29h-16h
Qualitative Work in Evaluation: Why? When? and How? For the World We Want
Cindy Tananis
ATELIERS/WORKSHOPS
325 MAI LUNDI MONDAY MAY 25
10h-10h30
Évaluation d'une formation en matière d'agression sexuelle envers les enfants autochtones
Xavier Barsalou Verge, Renée Séguin, Mélanie M. Gagnon, Christian Dagenais
PRÉSENTATION ÉTUDIANTE/STUDENT PRESENTATION
Le centre d'expertise Marie-Vincent offre un programme de formation en matière d'agression sexuelle envers les
enfants aux communautés autochtones du Québec. Tout au long de ce projet échelonné sur deux ans, une démarche
évaluative est menée en parallèle. Les objectif de cette démarche sont multiples, mais ont un but commun :
s'assurer que le programme de formation ait réellement une utilité pratique et qu'il réponde aux besoins de ces
communautés. L'exposé s'attardera à décrire la démarche évaluative qui comprend de s collectes de données
quantitatives, ainsi qu'une étude de cas multiple pour en apprendre plus sur l'utilité pratique de la formation en
contexte autochtone. Il sera aussi question des efforts mis en place pour retransmettre les résultats de l'évaluation
aux participants et à leur communauté. Pour conclure, les enjeux et les leçons tirées de cette évaluation menée
auprès de populations autochtones seront présentés.
10h30-11h15
Is Evaluation Losing Salience?
Andy Rowe
CONFÉRENCE D’EXPERT/EXPERT LECTURE
The connectivity of natural and human systems is demonstrated by important shifts incorporating sustainability into
public policy, the actions of for-profit organisations and public awareness. Climate change and sustainable
development are but two demonstrations of this change. To be salient today analysis and advice must address
sustainability. The premise of this presentation is that evaluation as a field is still strongly focused on the past when
a focus on human systems alone was OK. It is no longer OK, evaluation risks losing salience and falling even further
behind on our use agendas. For evaluation to regain salience it needs to incorporate sustainability into training,
standards, practice and theory, and the stance of evaluation to contemporary siloed go vernance. Suggestions are
offered and audience discussions of the issue and ways forward will be invited.
10h30-11h30
A Participatory Approach for Analysis and Reporting to Enhance Evaluation Knowledge
Shannon Clark Larkin, Nicole Michaud, Michelle Picard-Aitken
EXPOSÉ/PAPER
The use of a streamlined and stakeholder engagement approach for analysing and reporting on evaluation findings
and recommendations for the evaluation of the Canada Excellence Research Chairs Program will be presented. Given
tight timelines, and complex data collection, governance and consultation processes, an approach was implemented
for the analysis and reporting phases that enhanced stakeholder participation and resulted in increased access to
evaluation knowledge across the various stakeholder groups. Details on the preparation and usefulness of a
presentation deck of evaluation findings that was co-produced by the internal and external evaluation team will be
provided. The deck was refined and updated based on feedback from various stakeholder groups during the analysis
phase. This resulted in an evaluation report that contained concise and actionable recommendations that addressed
both accountability requirements and continuous improvement for program management.
Bringing the coaching attributes to evaluation
Kassem El Saddik
4EXPOSÉ/PAPER
The paper intends to bring the co-active coaching attributes and perspectives into the evaluation practice. It will
highlight key similarity between the two practice in terms of principles, ethics, qualities and strategy. It argues that
applying the fundamental elements of co-active coaching in evaluation will take the latter to an extra level by
unveiling new dimensions, revealing innovative theory of change and uncovering unintended outcomes. It sh eds the
light on the co-active coaching cornerstones (holding people naturally creative and resourceful and whole),
attributes (self-management, listening, curiosity and deepening the learning), setting the strategy (designing
alliance) as well as the ethical and paradigm (client-centric, unveiling intended and unintended outcomes).
Developmental Evaluation Learnings from a Girls Empowerment Program
Anne Miller
EXPOSÉ/PAPER
Sirius Girls is a pilot program in partnership with United Way's All in for Girls In itiative and the Women's Leadership
Council. It is an after school program for vulnerable girls in grade ten using a social -emotional and service learning
approach to assist girls to take action with the support of adult allies (women Leaders who volunteer with the
program) to experience mastery and build confidence to take new risks, creating a better community for all girls to
thrive. Since the program is an innovative initiative at the development and pilot stage, a developmental evaluation
approach has been used to help understand the successes, challenges and positive impacts of the program. This
presentation will describe the developmental evaluation approach taken with this program, new measurement tools
(like the VACO survey) that have been explored for use with this population, and learning from the process.
Évaluation des effets de la pratique de l'évaluation d'impact sur la santé (ÉIS)
Jean-Marie Buregeya
PRÉSENTATION ÉTUDIANTE/STUDENT PRESENTATION
L'évaluation d'impact sur la santé (ÉIS) est une démarche évaluative utilisée afin d'analyser les effets sur la santé des
politiques publiques qui ne relèvent pas du secteur de la santé (Harris et Spickett, 2011). J'applique une approche
novatrice (Analyse de contribution) pour l'analyse des effets à un programme complexe, en particulier pour le projet
de revitalisation du Vieux-Sorel et le projet d'urbanisme du centre-ville de Chateauguay afin d'apprécier le potentiel
d'ÉIS pour bonifier cette politique et ses impacts sur la santé et l'équité. Dès lors, il est nécessaire de présenter le
protocole de recherche puisque l'analyse de contribution est indiquée aux cas où les méthodes statistiques, tels que
les essais randomisés et quasi-expérimentaux, ne sont pas les mieux indiquées pour établir la causalité d 'une
intervention aux effets observés (Mayne, 2012), et n'a pas été utilisée pour évaluer les ÉIS.
Evaluation for the world we want: Child Rights Impact Assessment (CRIA) for Business
Tara Collins
EXPOSÉ/PAPER
'Evaluation for the world we want' should include attention to human rights and business in order to support a
better world. The international discourse has done so as evidenced by the United Nations Guiding Principles on
Business and Human Rights (2011) and the Children's Rights and Business Principl es (UNICEF, Save the Children, &
Global Compact (2012)). These documents identify impact assessments to evaluate effects upon human rights. But
how can the child rights impact of business be assessed in practice? The implications of this relationship must be
ascertained through the development and use of child rights impact assessment (CRIA) for business. However, this
activity is emerging with restricted participation. As there are significant challenges to identify and assess processes
and results, other actors including evaluators are needed to support progress. This paper presentation presents a
preliminary CRIA tool to contribute to the evaluation field.
Evidence-Based Principles to Guide Collaborative Approaches to Evaluation
J. Bradley Cousins
CONFÉRENCE D’EXPERT/EXPERT LECTURE
5This expert lecture will focus on practice in collaborative approaches to evaluation with particular emphasis on the
development, validation and ongoing international field testing of evidence -based principles. Featured will be the
work of a collaborative research team headed by Cousins, Whitmore and Shulha that has been unfolding over a 3
year period (see Cousins, Whitmore & Shulha, 2013, 2014; Shulha, Whitmore, Cousins, Al Hudib & Gilbert, 2015).
Specifically, the lecture will: (i) describe a bottom-up, four-phase empirical process to develop and validate
principles involving 320 evaluators from CES, AEA and IDEAS; (ii) introduce the set of eight (8) validated principles
and (iii) report on ongoing international interests in field testing the principles. A goal of the lecture is to engage
with audience members in the identification and mobilization of field testing opportunities.
How to publish in a journal of evaluation?
Robert Schwartz, Kristelle Alunni-Menichini, Lynda Benhadj, Jean-Marie Buregeya, Marie Beausejour
CONFÉRENCE D’EXPERT/EXPERT LECTURE
Dissemination of practice notes, real-life cases addressing challenges, and research and program evaluation findings
in evaluation’s domain have a primordial importance in order to change professional practices and influence
decision making in public organizations. Since the publishing process in the field of evaluation can be complex, Dr.
Robert Schwartz, Editor-in-Chief for the Canadian Journal of Program Evaluation, will be presen ting the specific
characteristics of his journal and guidelines on 'what to do or not to do to publish in evaluation' and this, in
collaboration with EASY research chair’s members. He will also answer frequently asked questions addressing the
publishing process in evaluation. We invite you to participate at the preparation of these questions by sending yours
at kristelle.alunni@gmail.com before April, 1st. This expert lecture is thus an excellent opportunity to learn about
important tips to support publication of evaluation work.
Increasing the Evaluator's influence: techniques and micro-practices
Natalie De Sole, Melanie Hwalek
TABLES RONDES/ROUNDTABLES
Evaluation's influence is conditioned by the stakeholders' receptiveness to listen, make sense, and use f indings. The
door of opportunity often closes after evaluators turn in the final report and meet one last time with our client. The
nonprofit's program director, foundation's program officer, or the governmental policy advisor must champion our
finding's use or file them away. A plethora of models exist on how to engage stakeholders yet discussion is limited
on the nuanced techniques required to bring these models to life. At this roundtable participants will be shown good
practice models related to rules of engaging individual, group and organizational stakeholders to create change.
Participants will be asked to share insights about how they apply engagement models in specific contexts. Together
we will identify techniques and micro practices that increase stakeholder buy-in and the eventual adoption of
evaluation findings.
L'approche participative en évaluation : le cas de Pouvoir Partager/ Pouvoirs Partagés
Joanne Otis, Lyne Massie
EXPOSÉ/PAPER
Pouvoir Partager/ Pouvoirs Partagés (PP/PP) est une intervention conçue pour outiller les femmes vivant avec le VIH
(FVVIH) face à la question du dévoilement du statut séropositif au VIH. S'appuyant sur un devis mixte, PP/PP a été
évalué au Québec et adapté culturellement au Mali (Gundo So) grâce à plusieur s recherches participatives où la
collaboration entre les différents acteurs et l'implication des FVVIH sont essentiels. Réfléchi par une équipe de
chercheuses AVEC et POUR les FVVIH elles-mêmes, PP/PP contribue au renforcement d'un plus grand pouvoir d'ag ir
chez les femmes du Québec et du Mali relativement au dévoilement de son statut séropositif au VIH. Diverses
stratégies de partage des connaissances et de renforcement des capacités ont permis la coproduction d'outils de
pratiques professionnelles disponibles en ligne (www.pouvoirpartager.uqam.ca), contribuant ainsi à améliorer la
qualité de vie d’'un plus grand nombre de femmes.
6L'évaluation pour optimiser le dépistage du VIH pour les hommes gais: le projet SPOT à Montréal
Ludivine Veillette-Bourbeau, Joanne Otis
EXPOSÉ/PAPER
SPOT est une recherche-intervention offrant, depuis 2009, un counseling et un dépistage du VIH rapide, anonyme et
gratuit en site communautaire pour les hommes gais ou bisexuels. En soutien à l'évaluation des effets, une analyse
d'implantation a été menée. Un monitorage du degré d'implantation a permis de voir sa fluctuation et des analyses
de régression linéaires d'identifier les éléments associés à sa variation, le counseling étant adapté au participant. Un
processus d'implantation en plusieurs phases, modulé par des facteurs liés aux motivations des acteurs, à la
complexité des dynamiques partenariales, aux défis de la coordination et à l'organisation de l'équipe terrain a
émergé de l'analyse qualitative. La pérennisation de SPOT soulève des défis: transformation d'une intervention en
contexte de recherche vers un service régulier, maintien des acquis de la recherche et appropriation de la gestion
par les partenaires.
Managing project constraints for the next decade
Judy Lifshitz
CONFÉRENCE D’EXPERT/EXPERT LECTURE
Evaluations are a key source of information to assist in responding to specific questions to support decision making
in a time where resources to conduct them are more and more limited. Thus, it is crucial that evaluatio ns be on
time, on budget and in scope so that the evaluations can be useful for decision making. This presentation will
provide an overview of the international standards used in the field of project management to help to address the
three project constraints of time, budget and scope. Examples and practical tools and techniques for addressing
time, budget and scope will be shared with participants based on the presenter's experience and internationally
recognized best practices. As a PMP, over the last five years, Judy has managed several evaluations and has used
these tools and techniques to ensure that timelines, budgets and scope is managed. She has done previous
presentations on project management at CES and PMI related events.
Reframe QUTs evaluation framework encompasses a multi-faceted approach to meet stakeholder needs
Lyn Alderman
CONFÉRENCE D’EXPERT/EXPERT LECTURE
Since 1984, the Australian government has invited students to provide feedback on their course of study and in the
early 2000s annual student feedback surveys became a government accreditation requirement for universities. As a
result, Queensland University of Technology (QUT) launched online invitational student feedback surveys in 2007. In
2010, stakeholders raised concerns about the mixed purpose of the surveys, the potential for data misuse and how
academics felt evaluation was being done 'to them' rather than 'for them'. In response, QUT introduced Reframe a
five-year project encompassing three phases: stakeholder engagement and discov ery - executive support,
environmental scan, literature review, and theoretical concept; product development; and dissemination and
communication - target audiences, communication plans, professional development and iterative refinement. In
2015, QUT's broad evaluation approach meets the needs of a range of stakeholders.
The experience of a paticipative evaluation: Brazilian QualiSUS-Network Project
Gisela Cardoso, Marly Marques Da Cruz, Aline Duque De Macedo, Ana Cristina Gonçalves Vaz Dos Reis,
Celita Almeida Rosario, Patrícia Pássaro Da Silva Toledo, Solange Kanso
EXPOSÉ/PAPER
The Brazilian QualiSUS Network Project (QNP), a cooperation between the World Bank and the MoH, is a strategy to
promote the qualification of health care assistance and management, and the development of technologies among
Health Care Networks. This intervention was implemented in 15 different and diverse health regions. In order to
support the institutionalization process and use of M&A as a tool for management and decision -making in context,
an implementation evaluation was conducted. The evaluation adopted a participatory approach which included the
development of 15 regional logical models, the identification of potential evaluation users, an agreement on
7indicators and goals; the characterization of the political-organizational contexts; the identification of facilitators
and barriers in the implementation process, and the analysis of the agreed management arrangements and its
contribution to the implementation in the contemplated regions.
The UN Resolution and International Evaluation Year - What's in it for Africa?
Ian Hopwood
EXPOSÉ/PAPER
What explains the upsurge of enthusiasm for evaluation (UN Resolution, International Evaluation Year, etc.)? Will it
last? Can it bring real change? What are the driving forces? The author uses his wide international and African
experience to assess real potential and identify pitfalls. Can evaluation escape the “donor driven image” and
monitoring overhang? Are there pre-conditions for evaluation to thrive and make a difference in Africa? How does
evaluation relate to cultural context and governance systems? The links to policy and planning? And to public
pressure for performance and accountability? Correcting the imbalance between accountability and learning?
Shifting the focus from projects to systems and policies? Getting Africa’s leadership and donors to acknowledge
'failure' and embrace learning approaches? The conclusions highlight new partnership opportunities and capacity
development initiatives to foster African leadership for appropriate practice.
Valuation: Methods for Representing the Value of Program Impacts in Monetary Units
John Gargani
EXPOSÉ/PAPER
Increasingly, evaluators are being asked to perform valuations — measuring the merit, worth, or significance of
program impacts in monetary units. In particular, valuation forms an integral part of recent value -for-money policies,
social return on investment analyses, and pay-for-success initiatives. I provide an overview of valuation, describe
valuation methods currently used by evaluators, and demonstrate how and why these methods sometimes produce
contradictory results. I conclude by describing a system for reporting valuations that may better communicate their
relative trustworthiness, thereby supporting the many uses to which they are put by funders, private investors, and
policy makers. Throughout, I provide examples using data on prisons provided by the UK government. The data and
worked examples will be made available electronically.
Valuing evaluation power and the power of evaluation in “Speaking Truth to Power”
Sandiran (Sandi) Premakanthan
CONFÉRENCE D’EXPERT/EXPERT LECTURE
The main focus of this session is about valuing evaluation power and the power of evaluation in speaking evaluation
truth to power, those who create the demand for it, legislators, the public, heads of government departments and
agencies, program management, non-government organizations and donors. I have defined the terms evaluation
power and the power of evaluation and identified several sources of institutionalized evaluation power. They
include: governments through legislations, authority instruments, and policies, philanthropic foundations, financial
institutions, government aid agencies, United Nations (UN) agencies, non-governmental organizations (NGOs) and
numerous networks: evaluation societies, associations and international networks. The evaluation power of the
government of Canada, termed the “value model” and the creation of the power of evaluation and its use for
informed decision making is discussed.
What Environmental Assessment can Learn from Evaluation
Paul Kishchuk
EXPOSÉ/PAPER
The Yukon Environmental and Socio-economic Effects Assessment Act allows for consideration of social and
economic effects in parallel with environmental effects when resource development projects are scrutinized. Yet,
almost a decade after implementation of the Act, full and proper consideration of social and economic effects still
struggles to gain traction in Yukon's environmental assessment process. Drawing on his experience in the fields of
8both socio-economic effects assessment and program evaluation, Paul will put forward a hypothesis, using the logic
model paradigm, which illustrates why socio-economic effects assessment has not been embraced in small Yukon
communities. The presentation will go on to explore how an evaluative approach to socio -economic effects
assessment can help advance resource development projects in rural and remote communities in Canada.
Whether Evaluations made a difference in the Management of HIV/AIDS Programs in South Asia?
R.S. Goyal
EXPOSÉ/PAPER
In the quest to seek desired impact, cost effectiveness and accountability, funders of HIV/AIDS programs have been
seeking systematic assessment of outcomes and impact of these programs. These evaluations were also expected to
become prime mover for continuation of resource allocation and scaling up of the interventions. This paper presents
a synthesis of evaluations of HIV/AIDS programs in South Asia particularly contributing to the program management
in relation to; communication, stigma and discrimination and, preventing HIV among adolescents and young people.
The broad inclusion criterions were; randomized control/ quasi-experimental/descriptive deign and evidence for
casualty/ what has or has not worked/ provide evidence for up- scaling/replication of interventions. Preliminary
analysis indicates that; curriculum based-teacher led interventions in schools; and youth friendly health services
have made appreciable contribution to HIV/AIDS programming in the region.
Working for a World We All Want: The Journey of a Community Needs Assessment
Denise Belanger, Linda Lee
EXPOSÉ/PAPER
While many practitioners embrace participatory evaluation, they are often faced with questions regarding how to
meaningfully involve participants in processes. What does meaningful participation and consultation look like in
different situations and with different constituents? Drawing from their work over the past few ye ars, Linda and
Denise will discuss collecting the voice of those who live in inner city neighbourhoods. In particular, the presentation
will focus on a community needs assessment undertaken in partnership with a community based agency where the
assessment focused on meaningful involvement of community, in the hope that those involved felt valued and
heard. In addition, capacity building was important to this project, as Denise and Linda worked with staff to
undertake community-based data collection and involve them finding meaning in the information collected.
Leçons apprises de l'évaluation du projet allo info sida
Rosine Flore Chuedo Djoungou
EXPOSÉ/PAPER
La stigmatisation liée au SIDA constitue un problème central pour les porteurs de VIH au Cameroun, car les malades
marginalisés tombent en déprime morale, ce qui complique leur suivi. Le gouvernement tente de résoudre ce
problème, dans le cadre de sa stratégie nationale de lutte contre le SIDA. C’est dans cette perspective que SUNAIDS,
en partenariat avec le CNLS a mis en œuvre le projet Allo Info Sida, dont l’objectif est d’apporter l’information, le
soutien psychologique et moral aux malades du Sida et aux PVVS. Il s’agit d’un dispositif d’appel téléphonique
gratuit accessible à partir du téléphone fixe et du téléphone portable de toutes les régions du Cameroun. Après 2
ans de mise en œuvre du projet, l’évaluation du projet a permis de mesurer l’atteinte des objectifs, d’analyser la
démarche de mise en œuvre du projet , dans l’optique d’envisager des persp ectives d’évolution et les possibilités de
repositionnement du projet face à l’évolution du contexte environnemental.
10h30-11h45
Web-Based Stakeholder Feedback Portals: Allowing Wider Stakeholder Feedback
Makenzie McPherson
EXPOSÉ/PAPER
9Advances in technology provide evaluators with continuously improving, cost-effective methods for the
dissemination of results and opportunities for capturing stakeholder feedback. One of these mediums, a web -based
stakeholder feedback portal, provides evaluators the opportunity to post visual representations (e.g. process flow
maps, logic models, root cause analysis diagrams, etc.) allowing stakeholders to review, edit, provide feedback, or
validation. This presentation will provide a demonstration of an active web -based stakeholder feedback portal,
showing how it can effectively strengthen the evaluation, and how it might be adapted for other uses.
13h15-14h45
An innovative approach to link evidence & practice for practitioners in the employment services area
Susanna Gurr
EXPOSÉ/PAPER
The BC Centre for Employment Excellence aims to improve employment outcomes for job seekers by promoting
innovation and sharing evidence about employment programs with practitioners to enhance knowledge and ensure
best practices are implemented. The Centre has 2 key functions: 1) conduct projects in collaboration with
stakeholders to test innovative approaches 2)share research with practitioners. The Centre was launched in 2012 at
the initiative of the BC Gov't. The Centre emphasizes the role of practitioners in its activities to ensure the
knowledge developed and shared is relevant. Hear from the Managing Director about methods used to connect BC
practitioners to different forms of evidence in various engaging ways and the collaborations a nd projects that have
emerged from the Centre's activities. What are some opportunities to close the gap between evaluation and
knowledge capacity to enhance practice and ultimately lives of job seekers? What are some pitfalls to avoid?
Are user fees exemptions enough to increase the use of healthcare services by the worst off?
Nicole Atchessi, Valéry Ridde, Maria-Victoria Zunzunégui
PRÉSENTATION ÉTUDIANTE/STUDENT PRESENTATION
User fees exemption programs have been implemented in certain low- and middle-income countries to improve the
populations' access to healthcare services. Given the considerable healthcare needs of the worst off, they should,
when exempted from user fees, be using healthcare to a greater extent than others. The aim of this study was to
assess whether user fees exemptions increased healthcare services use among indigents in the Ouargaye district in
Burkina Faso. The indigents' increased healthcare services use was not attributable to user fees exemptions. Some
contamination of the intervention is conceivable. Interventions combining user fees exemptions with actions
targeting other obstacles to healthcare access would probably be more effective in increasing indigents' use of
healthcare centres.
Collaborative evaluation capacity building for social innovation in the public sector through DE
Keiko Kuji-Shikatani, Scott Urquhart, Jennifer McMaster, Kieran McMonagle, Cristina Ilas, Angelika Kerr
PANEL/PANEL
Program evaluation began with the desire to seek information that can be utilize d to improve the human condition.
The panel explores how infusing evaluative thinking through DE is positioned as a responsibility for internal
evaluators and leaders at all levels of the system focused on organizational learning and improvement in its
relentless pursuit of reaching every student, including: collaboratively building relationships and holistically
nurturing the whole person as Four Directions circle of caring adults – school staff, parents, community partners, and
the ministry - so as to very purposefully position the sharing of caring of First Nations Métis and Inuit learners; and
partnering with three school boards to conduct case studies of existing teen parent programs that integrate
academic, life skills and counselling components with additional community services to support teen parents in
finishing high school and transitioning to further education, training or to the workplace.
Demystifying Gender Mainstreaming
Emmanuel Trépanier, Laurentine Mefire
10EXPOSÉ/PAPER
Mainstreaming gender equality in international organizations is a challenging process that requires awareness,
political will, behavioural change, resources, time and, quite often, intensive guidance from specialized consultants,
using complex tools. Universalia's newly developed Gender-Sensitive Institutional and Organizational Performance
Assessment (GSIOA) Model is a user-friendly, time-efficient and free tool intended for users from all walks of life.
Adapted from the Universalia IOA Model, the tool allows users to assess the capacity, performance, external
environment and motivation of international organizations from a gender perspective in order to help decision -
makers improve equity in the workplace and contribute to positive social impacts. This presentation will allow
audience members to experience the GSIOA Model with pragmatic and challenging examples and increase their
acumen for equity.
Evaluating the Humanitarian Projects during Crisis: The Syrian Case
Tarek Daoud
IGNITE (PRÉSENTATION RAPIDE)/IGNITE-STYLE RAPID PRESENTATION
The prolonged crisis in Syria leads to catastrophic humanitarian conditions. According to UNOCHA (2014) report
300000 people are living in under siege. As a response, INGO’s launched a series of humanitarian projects in
coordination with local partners. Humanitarian organizations face challenges in evaluating the impact of the
implemented projects due to the problems involved in overcoming the constraints of physical access and the
obscure limitation of sharing evaluation results due to security purposes. I n this chaos situation, the participatory
evaluation approach found to be the best solution in order to provide an in -depth insight evaluation. Applying the
participatory evaluation improved projects performance, empowered the implementing partners evaluat ion skills,
creates evaluation teams in siege areas, enhanced the organizational learning growth, increased the validity of the
data collected, and strengthen the relation between local staff and external experts.
Evaluation du taux de survie à 12 mois des patients infectés par le VIH au Sénégal
Maguatte Ndoye Ndiaye
IGNITE (PRÉSENTATION RAPIDE)/IGNITE-STYLE RAPID PRESENTATION
Contexte: Le taux de survie constitue un indicateur clé du programme et permet d'évaluer l'impact de la mise sous
ARV des personnes éligibles au traitement. Objectif: Évaluer le taux de survie des patients 12 mois après la DUS mise
sous ARV pour l'année 2012 Méthodologie : la formule de calcul est = (Cohorte actuelle (CA)/cohorte nette (CN) x
100 CN = Cohorte de départ (N) Transfert In (TI) — Transfert Ailleurs (TA) CA = CN — DC — PDV — ABAN — ARR
Résultat: le taux de survie nationale est estimé en 71% en 2012 et 73% en 2011. Conclusion: Disparité observée dans
les régions.
Evaluation for the World We Want? Emerging Lessons from the Global South
Robert Mclean, Mallika Samaranayake, Ziad Moussa, Colleen Duggan
PANEL/PANEL
In the view of many, evaluation can be used to help build the world we want. Canada's IDRC has supported this
vision through multiple avenues including supporting evaluation field -building in regions with identifiably less
capacity to match evaluation demand with local expertise. Several compelling results have emerged, and at
considerable scale. Two specific efforts have been the Community of Evaluators, South Asia (CoE) and the Middle
East and North Africa Evaluators Network (EvalMENA). Current President of the IOCE, Ziad Moussa will reflect on
experiences jointly leading the EvalMENA network. Founding President of the CoE Mallika Samaranayake will
comment on her work enhancing evaluation across South Asia. Insights will be of interest to those wanting to learn
unique lessons about professionalization, capacity-building and networking in evaluation. Especially of interest to
those working to bridge evaluation supply and demand between the global North and South.
Evaluation of Data Quality
Harold Henson
11EXPOSÉ/PAPER
Problems with data have been an ongoing challenge in the field of program evaluation. In 2009, the Auditor General
reported that a significant share of published evaluations were compromised by problems with the available
performance measurement data. There are few reasons to believe that the situation has improved substantially
despite the improvements in technology. In this paper the author argues that the same basic process and
methodology used in program evaluation in general could be applied to the evaluation of data. It is argued that the
standardized evaluation questions and lines of evidence can be modified to assess quality of data generated by
programs for evaluation. At the end of this exercise there will be a report that resembles a program evaluation but
addresses the issue of data quality. In the best case scenario, high quality evaluations will be possible without
expensive surveys.
Evaluation Principles for a Democratic World
Elizabeth Lewis
IGNITE (PRÉSENTATION RAPIDE)/IGNITE-STYLE RAPID PRESENTATION
As an organization with the mission to advance freedom and democracy worldwide, the evaluative efforts of the
International Republican Institute not only seek to adhere to internationally accepted standards for evaluations, but
to reinforce the democratic principles its programs seek to advance. IRI will use an example of a participatory needs
assessment it conducted in Somaliland to show how evaluation can consider six democratic principles: accessibility,
accountability, efficiency & effectiveness, equity & inclusiveness, responsiveness and transparency. Commissioned
by the UK's DFID, this needs assessment informed IRI's development of the Strategy for International
Democratization Support to Somaliland for the international community. The assessment captured input from
representatives of the international donor community, international implementers, subject matter experts and
Somaliland stakeholders from civil society, political parties, the media and government.
How to Use Process Flow Mapping for Continuous Quality Improvement
Ralph Renger, Melissa Rogan, Makenzie McPherson
EXPOSÉ/PAPER
Continuous quality improvement (CQI) is a common evaluation purpose. One potentially useful CQI method is
process flow mapping. Strategies for identifying and contacting potential subject matter experts, the basic steps and
tips in conducting a process flow map (PFM) interview, how to integrate interviews into a single, summary PFM,
methods for validating the PFM, and how efficiency data in conjunction with the PFM can be used to facilitate
corrective actions to improve system efficiency will be shared in the context of conducting CQI in a cardiac system of
care. Implications for using PFM in conducting CQI in other evaluations will be discussed.
How We Make a Difference
Paul Kishchuk
IGNITE (PRÉSENTATION RAPIDE)/IGNITE-STYLE RAPID PRESENTATION
A mash-up of Mark Schacter's 'Keeping Busy...Making a Difference' logic model diagram and the DIKW (data -
information-knowledge-wisdom) paradigm is used to illustrate what the end game of evaluation sho uld be.
Influencing Change: Perspectives from Government, Private Consulting and the University
Donna Smith-Moncrieffe, Celine Pinsent, Mark Nafekh, Pierre Mercier
PANEL/PANEL
Influencing Change: Perspectives from the Government, Private Consultants and th e University A number of
evaluations conducted in Canada are produced by the federal government in collaboration with private consultants,
and educational institutions. It is important for evaluators from both the private and public sector to collaborative ly
reflect on how evaluation findings have had an impact on the communities and policies. This panel will answer
important questions about impact and influence from the federal government, university and evaluation
consultant’s perspective. The presentations will respond to some of the following questions as per the theme of the
12conference: •How does evaluation make a difference in the community? •How has evaluation influenced policy in
the federal government and educational institutions? •What are some of the strategies used to reduce the barriers
to implementing sound evaluations?
Instrumentation de la théorie du programme dans le cadre d'une évaluation participative
Rodrigo Quiroz, Nathalie Bigras, Julie Dion, Karine Doudou
EXPOSÉ/PAPER
La théorie du programme (Chen, 1990, 2005) est une méthode utilisée par les évaluateurs afin de mieux définir
l'intervention à évaluer (Champage et al., 2011), de comprendre les processus à l'ouvre dans son implantation
(Green & MacAllister, 2002), d'identifier ses effets et les mécanismes en jeu dans leur production (Chagnon et al.,
2009), d'accroitre l'utilisation des résultats de l'évaluation (Wholey, 1987) et d'améliorer le programme (Weiss,
1995). Malgré son importance, il y a un manque d'écrits illustrant l'instrumenta tion de la production de la théorie du
programme dans le cadre d'une d'évaluation participative. Cet exposé vise à combler ce vide en présentant les
fondements conceptuels, ainsi que les étapes, les techniques utilisées et les principaux résultats obtenus de
l'évaluation d'un programme destiné à la formation des éducatrices en petite enfance.
L'analyse logique comme levier de changement : innovations et applications en santé publique
Tarik Benmarhnia, Véronique Foley, Jean-Marie Buregeya, Samantha Gontijo-Guerra, Gisela Cardoso,
Kristelle Alunni-Menichini, Marie-Claude Tremblay
PANEL/PANEL
L'utilisation de l'évaluation : ce que la théorie ne mentionne pas
Grégory Häuptli
IGNITE (PRÉSENTATION RAPIDE)/IGNITE-STYLE RAPID PRESENTATION
Il existe une importante littérature sur l'utilisation de l'évaluation et plus particulièrement sur les facteurs internes
(qualité de l'évaluation) et externes (contexte politique et institutionnel) influençant négativement ou positivement
la probabilité que les résultats d'une évaluation soient utilisés par les chargés de programmes ou les décideurs
politiques. Cette présentation confronte cette littérature aux défis, contraintes et opportunités auxquels font face
les évaluateurs dans le but de mettre en avant les facteurs qui sembl ent être peu théorisés, voire ignorés par la
littérature. Il ne s'agit donc pas de remettre en cause la littérature sur l'utilisation de l'évaluation mais plutôt de
questionner pourquoi certaines contraintes vécues ne sont pas ou ne peuvent pas être théori sées.
Measuring the extent of institutionalization of ART programs in health facilities in Uganda
Henry Zakumumpa
PRÉSENTATION ÉTUDIANTE/STUDENT PRESENTATION
In 2004,Uganda commissioned a national antiretroviral therapy (ART) scale -up program with external donor support.
We sought to measure the extent of institutionalization of ART programs in health facilities in Uganda and compare
institutionalization scores by health facility type. Level of Institutionalization (LoIn) scales developed by Goodman, et
al(1993)were used to measure the extent of institutionalization of ART programs at 195 health facilities in 42
districts of Uganda which received donor support between 2004 and 2009 to initiate ART services. The overall mean
institutionalization score for participating health facilities was 3.5(Range, 1-4) and the mean score for niche
saturation, the highest level of institutionalization, was 3.2(Range,1 -4).Private health facilities had the lowest mean
institutionalization score. Programs for enhancing the institutionalization of ART interventions in private health
facilities are recommended. ART program evaluation and supervision are deficient.
Not an Island - Insights from Other Disciplines
Alexandra Dagger, Shelley Borys, Bill Blois, Marie-Josée Dionne-Hebert
PANEL/PANEL
13Evaluation has a long history of integrating knowledge and techniques from other disciplines. Approaching our work
with these other disciplines in mind brings fresh insight and adoption of new ideas and methods as well as benefits
to evaluands. This session will explore learning that evaluators can draw from areas such as performance
measurement, internal audit, risk and communications disciplines. Evaluators have not always tended to turn to
these areas for knowledge to use in our work but they have the po tential to add value. The panelists are senior
members of the federal evaluation community. However, although they come from the public sector, the disciplines
explored and lessons learned highlighted will be relevant to all evaluators.
Performance challenges of Tuberculosis control actions in Brazil: scapegoats, myths & possibilities
Gisela Cardoso, Elizabeth Moreira Dos Santos, Dolores Abreu, Pedro Paulo Chrispim, Gisele Oliveira, Juliana
Borenstein, Amanda Pereira
EXPOSÉ/PAPER
Tuberculosis (TB) is a national health problem for Brazil. The country has different cultural contexts and
epidemiological TB profiles. As a preliminary step of a performance evaluation regarding TB prevention and control
actions, the objective of this study was to conduct a strategic and logical analysis of TB control actions in four
Brazilian municipalities addressing activities at primary health care. To develop the strategic analysis, we focused on
a main question: "What are the potentialities of the national and municipal strategies to respond to TB control
challenges?". The steps of the strategic analysis included the identification of the strategic objectives, goals,
indicators and their classification according to the EGIPPS model domains and alignments. The appraisal of t he
logical modeling involved document review and interview with TB key actors. Results are going to be presented
according to the questions posed for both strategic and logical analysis.
Power, Institutions and Gender Relations: Can Evaluations Transform them?
Ranjani Krishnamurthy
EXPOSÉ/PAPER
This paper examines the perception of women, men, adolescent girls and boys from three Chennai slums, India how
the institutions of marriage, family, local markets and government should function if gender and social relations are
to be equitable. It then examines their perception of actual change in the last five years, what they consider positive
and negative and causality of change. In the process, marginalised people's inter -sectoral assessment of government
programmes also emerges (with recommendations) and this is contrasted with sectoral evaluations of government.
This paper argues that mainstream evaluations of government are sectoral or of one Department, and this does not
match the intersectoral and inter-institutional lived realities of women/girls and men/boys. Evaluations capturing
movement towards the world women/girls want, should give importance to their indicators of change & power,
methodologies, interpretations and recommendations
Program Design and Performance Measurement
Michel Laurendeau, John Burrett
EXPOSÉ/PAPER
While evaluation and performance measurement frameworks have helped, most evaluators still have to conduct
evaluations of public programs where the theory of intervention is uncertain and incomplete, with weak indicators
and poor ongoing performance measurement. Program research and descriptions of good practices often fail to
identify key success factors and support the improvement of program design. Frameworks have also been poor at
linking delivery and outcome indicators. The presenters will discuss two approaches to improved specification of
logic models with the use of: 1) network analysis to identify expected linkages of cause and effect and external
influences; 2) Integration Definition Function (IDEF) modeling techniques to describe delivery processes. They will
demonstrate how these techniques can support the development of improved logic models, a more compelling
analysis of program performance and a better understanding of program economy, efficiency and effectiveness.
14Program science in practice: Lessons, challenges and implications
Sean Rourke, David Seekings
EXPOSÉ/PAPER
Program science is an emerging public health approach that systematically applies scientific knowledge and evidence
to improve the development, implementation and evaluation of programs. By emphasizing the importance of the
context in which interventions occur, program science highlights the essential role evaluation plays in informing
strategic action to address key health challenges. This research project will build a better understanding of what this
approach looks like in practice, using evidence from case studies of community led HIV prevention interventions to
assess the practicalities of making program science work on the ground. The presentation will examine the
challenges and opportunities of using evidence to inform each stage of community -based interventions, from
development to evaluation, highlighting key lessons from the cases. It concludes by discu ssing the broader
application of program science beyond public health to evaluation and public policy more generally.
The Influence of Internal Evaluation on Program Management
Andrea Macdonald
EXPOSÉ/PAPER
Nonprofits are under increased pressure to deliver more for less and to demonstrate results of their work. When the
Clean Foundation, an environmental nonprofit based in Nova Scotia, invested in evaluation training for management
and core program staff, the intent was to increase capacity in internal eva luation and develop frameworks to
evaluate core programs. However, as the group went through the process, the discussion and work often focused
more on questioning the design and delivery of individual programs and provided insight into management and
stakeholder expectations. The value to undertaking the evaluation internally by the management team will be the
focus of the presentation. The work of Arnold Love on the use of internal evaluation will frame the discussion. Clean
Foundation's experience will be used as an example of how internal evaluation influences program management and
the successes and challenges that have resulted.
The Role of African VOPEs in Influencing Gender and Equity Focused Evaluation by Governments
Jennifer Mutua
EXPOSÉ/PAPER
Africa is going through a transformation of hope and despair in equal measure. The landscape is characterized by
improved GDP growth rates coupled with new discoveries of natural wealth in some countries. Conversely, the
continent is rife with alarming increases in poverty and youth unemployment in an environment where economies
are not managed prudently. Waves of insecurity and patriarchal social order compound these. Unprecedented M&E
capacity building, including from gender and equity perspectives have b een offered to individuals and governments
by international development partners. Hence, a critical mass of professionals who believe in evidence-based and
equity focused decision making and implementation is slowly growing. However, national culture and p ractice
remain weak. Equity focused evaluation can influence more prudent public resource management through advocacy
leveraging on 2015 EvalYear spearheaded by Voluntary Organizations for Professional Evaluation (VOPEs).
The Role of Evaluation in the Scale up of Population Health Interventions: The Innovation Strategy
Leslie Payne
EXPOSÉ/PAPER
The Public Health Agency of Canada's Innovation Strategy (IS) supports projects to take action to reduce health
inequalities and effectively address priority complex public health problems and their underlying factors. The IS uses
an intervention research approach, meaning that the program supports the delivery a set of population health
interventions while generating critical knowledge, on (1) how the intervention pr ocess brings about change and (2)
the context in which the intervention worked best and for which populations. The IS uses an innovation 3 phase
model to assess and support the scale up interventions. This presentation will provide an overview of how the I S
model has used evaluation and intervention research to support the scale up effective population health
15interventions in Canada and highlight a new approach to measure scale up readiness among population health
interventions.
To screen or not to screen: the impact of cancer screening letters
Emily Tsoa
EXPOSÉ/PAPER
Cancer Care Ontario's Evaluation & Reporting Team supports the Cancer Screening unit in evaluating the impact of
their three screening programs. These organized screening programs include the O ntario Breast Screening Program,
the Ontario Cervical Screening Program, and the Colon Cancer Check Program. Correspondence letters are currently
being sent to the general population who are due for screening. The impact of cervical screening invitation an d recall
letters, and colon cancer screening invitation letters, have been evaluated. Randomized control groups and historical
control groups were the designs used for these evaluations. Results from these evaluations will inform how future
screening correspondence should be delivered to the residents of Ontario.
Putting a ring on it: Evaluating the long-term engagement of operational staff
Joseph Travers
IGNITE (PRÉSENTATION RAPIDE)/IGNITE-STYLE RAPID PRESENTATION
How does a small evaluation group engage a large operational group of staff and management over a multi-year
project to ensure the goals of the evaluation are met? The Ignite presentation will cover feedback we've received
from staff and management about how we've done so far in doing this, and wh ere we need to improve.
Achievements, failures, and unexpected situations will be covered, as well as how to use this feedback to enhance
the operationalization of this project going forward.
13h15-15h15
Avons-nous réglé la question de l'utilisation des résultats de l'évaluation?
Marie Gervais, Francois Dumaine, Olivier Sossa, Andrealisa Belzer, Courtney Amo, Anne Routhier,
Mirianaud Oswald Agbadome
PANEL/PANEL
Au Canada, des efforts importants ont été investis depuis plus de 10 ans pour améliorer la qualité de l'évaluation et
renforcer la capacité en évaluation aux différents paliers gouvernementaux. Différents chemins ont été pris pour
convaincre les gestionnaires et les décideurs de la valeur ajoutée de l'évaluation à l'élaboration et à la gestion d es
politiques et programmes publics. Toutefois au quotidien, quel sort réserve -t-on aux résultats des évaluations? En
quoi ces résultats viennent-ils véritablement éclairer la prise de décision? L'évaluateur pourrait -il faire mieux dans sa
capacité à mobiliser autour des résultats d'une évaluation? Ce panel réunira des experts provenant de divers
contextes de pratique (gouvernement, secteur privé, milieu académique) et de différentes régions du Canada. Ceux -
ci partageront leurs perspectives sur les enjeux résiduels de cet aspect de la pratique.
14h45-16h15
APPRENDRE ENSEMBLE: L'IMPORTANCE DES COMMUNAUTÉS DE PRATIQUES EN ÉVALUATION
Raïmi B. Osseni, Erika Kaneza
PRÉSENTATION ÉTUDIANTE/STUDENT PRESENTATION
Au printemps 2014, quelques étudiants de l'école nationale d'administration publique décident de se retrouver
régulièrement pour échanger au sujet des pratiques et du secteur de l'évaluation en mettant l'accent sur la région
de l'Outaouais. Peu à peu, cette communauté de pratique va se développer et offr ir à ses membres un espace
permettant d'approfondir leurs apprentissages et de renforcer l'accès et le partage des connaissances. Les cercles
d'apprentissage sont des « petites communautés d'apprenants qui se regroupent intentionnellement, dans le but de
se soutenir dans le processus d'apprentissage » (Collay, Dunlap, Enloe et Gagnon, 1998). La présentation de l'équipe
16d'évalue-action propose d'explorer son modèle de développement professionnel et ceux qui l'ont inspiré, ses
réalisations, les leçons apprises, et de découvrir les avantages de créer de tels cercles au sein de milieux
professionnels.
Building Capacity in Community-Based Networks: Partnership Grants Learning Project
Heidi Schaeffer
EXPOSÉ/PAPER
How can we know more about the difference that investments in building capacity with not-for-profit networks and
partnerships can make? Health Nexus and the Association of Ontario Health Centres, working with the Tamarack
Institute and with leadership from The Mowat Centre for Policy Innovation at the University of Toronto, asked just
this question in a recent evaluation. Come and hear about the Partnership Grants Learning Project that set out to
measure outcomes of a 7.2 million dollar investment in 27 not for profit community organizations by the Onta rio
Ministry of Citizenship and Immigration. An innovative new tool called Outcome Harvesting was used. Learn about
how Outcome Harvesting, inspired by Outcome Mapping, can help practitioners operating in networks and
partnerships to evaluate the social change results they are achieving through a learning oriented participatory
evaluation combining Outcome Harvesting with Most Significant Change and Social Network Analysis.
Causality, Complexity, Critical Realism, Validity: What Do These Terms Have in Common?
France Gagnon
PRÉSENTATION ÉTUDIANTE/STUDENT PRESENTATION
It is a complex world out there. A program's causal pathway to change is rarely linear and simple. Different causal
pathways will exert a differential impact on various members of the target popu lation under similar implementation
conditions, settings, and times. Or, the program cultural, socio -economic, political context may negatively impact
outcomes in one setting but not another. And evaluation findings will have different meaning, relevance, and utility
depending on whether one is a program developer, program intervention manager, staff, community member, policy
analyst at the provincial or federal level or member of the target audience. How is one to produce evaluations that
are more credible and actionable? This presentation will review and present literature on causality, complexity,
critical realism and validity to provide suggestions for improving the construct validity of interpretations.
Contribution d’Avenir d’enfants dans le renforcement des capacités de ses partenaires en évaluation
Julie Rocheleau, Youssef Slimani
EXPOSÉ/PAPER
Depuis 5 ans, Avenir d’enfants (AE) soutient financièrement plus de 130 regroupements locaux de partenaires (RLP)
sur l’'ensemble du territoire québécois. AE est une organisation apprenante qui a le privilège de chapeauter un
projet innovant et mobilisateur en faveur du développement global de la petite enfance. AE note un renforcement
de la culture évaluative auprès des partenaires engagés dans les projets qu ’il soutient. AE a misé sur le renforcement
des capacités de ses partenaires, ce qui a mené petit à petit à une adhésion aux processus évaluatifs. Ceci s ’est
traduit par le développement de nouvelles compétences, et l’intérêt croissant démontré à l’égard de l’évaluation.
Cette présentation mettra en évidence, à travers des expériences du terrain et des exemples au niveau
organisationnel, le chemin qui a été parcouru par AE et ses partenaires dans le but de développer et consolider une
culture évaluative dans le domaine de la petite enfance.
Data collection and their methods for Impact Evaluation
Raed Zahrawi
IGNITE (PRÉSENTATION RAPIDE)/IGNITE-STYLE RAPID PRESENTATION
A five-year Compact ($275 million) is developed to reduce poverty and increase income in Zarqa Governorate of
Jordan, through increases in the supply of water available to households and enterprises through improvements in
the efficiency of water delivery, the extension of wastewater collection, and the expansion of wastewater treatment.
Measuring the impact of the Compact activities on economic and social outcomes was designed to establish a causal
17Vous pouvez aussi lire
