Course:LFS302B

LFS 302B: Summer Field ; Course, Indonesia
LFS 302B
Section:	TBA
Instructor:	TBA
Email:	TBA
Office:	TBA
Office Hours:	TBA
Class Schedule:	TBA
Classroom:	N/A
Important Course Pages
	Syllabus
	Lecture Notes
	Assignments
	Course Discussion

Introduction

This course on the development project monitoring & evaluation for timely responses (METR) will be taught under real-world conditions of learning-by-doing on location, measuring the right things in the right way at the right time. It will increase students':

Practical understanding of one of the major reasons for failures of assistance interventions (development/aid projects and programs), namely inadequate evaluation of impacts
Understanding of how to overcome these failures by identifying objectively verifiable outcomes of development projects and accountability for them.

Students are provided with an analytical tool that will increase their value as employment candidates for:

National and international donor agencies
Consultancy companies
Non-governmental organizations (NGOs) working in the development field
Specialist research agencies exploring improved monitoring and evaluation

Capacity is built for the design and implementation of ex post project and program evaluation (by effectively measuring the right things in the right way at the right time, hence "METR"), thereby:

Increasing the likelihood of lasting beneficial changes in development initiative outcomes
Informing proceses for better design of future projects
Offering informed recommendations for policy reform to provide better enabling conditions for lasting equitable growth and development

Learning Outcomes

Upon completion of this course, with a special focus on assistance interventions through (aid) development projects and programs for natural resource management/health & nutrition/education projects, students will be able to conduct the following.

External Project Evaluation

Independently and credibly evaluate development projects' and programs' relevance, efficiency, effectiveness, and impacts:

within and beyond the project's location
identifying indications of development continuity beyond the lifetime of assistance interventions

Shared Learning Processes

Design shared learning processes to encourage:

beneficial changes in behaviour of the project implementors
improve the design of future projects
promote policy reform that creates more enabling conditions for growth and development

Specific Topics and Approach

Understanding the Nature of Successes and Failures of Conventional Project Monitoring and Evaluation (M&E), primarily:
- over-reliance on internal or semi-independent evaluation of project inputs and outputs
- entrenched interests that are content with inadequate M&E, rather than a commitment to outcome/impact-based and more independent M&E
- potential for strengthening information feedback connections between
  - sources of project funds (e.g., taxpayers, foundations, charities)
  - intended beneficiaries of projects
Team Building Among METR Specialists
- Terms of Reference (ToR) enhanced by METR
- Following the Trajectory of Trust (see below)
METR Engagement with Project
- Clarification of Purpose (derived from ToR)
- Appreciative Inquiry about the Project and METR itself, by asking, "what was your best moment on the project?"
Obtaining Data and Information in the time available
- Types of data and information sources (handling excess/insufficiency)
- Field visits and interviews
Evaluation of the Role of a Broad Portfolio of Development Capital/Assets affected by the Project and the Institutions that influence them
- Financial, physical (human-made), natural (sources and sinks), knowledge (codified and informal/traditional), human and social capital/assets and the institutions that formally and informally manage these assets, as well as de facto and de jure rights, roles, responsibilities, and relationships (R4) of key stakeholders
- To achieve the 3 E's:
  - Economic Efficiency
  - Environmental Management
  - Equity
- Assessment of the degree to which projects face any of the disabling 3 U's:
  - Uncertainty of Tenure
  - Under-evaluation of Natural Resources
  - Under-regulated Negative Externalities
Measuring the Application of Key Good Governance Principles by project implementers, notably:
- Transparency
- Participation/inclusivity
- Accountability
- Timely responsiveness
Assessment of Mutual Stakeholder Understanding among project donors and implementers, as well as partners/beneficiaries to harness social capital through:
- Trajectory of Trust (ToT) before starting to deliver project benefits
  - beginning with mutual understanding (appreciation of stakeholder expectations and concerns (hopes and fears) about the project)
  - in turn, fostering mutual respect
  - laying the foundation for mutual trust
Appraisal of Project Logical Frameworks (log frames) for project design, that should be:
- underpinned by a schematic PROBLEM TREE: symptoms, core problem, proximate & root causes
- for participation modification, including elements such as overall purpose, specific objectives, activities (outputs and inputs), development assumptions and risks, and objectively verifiable indicators for measurements that together effectively
  - "track success and capture failure"
- in the absence of log frames, construction of rapid log frames for the project with recommended improvements
Application of Core Criteria of Project Evaluation
- conceptual integrity or core logic of the project approach
- relevance of specific project objectives/projected end results to purpose
- quality of design of implementation
- adequacy of implementation sequencing (starting with trust building, see #6 above)
- efficiency of delivery of project outputs in terms of human, financial, and time resources
- effectiveness of outputs, comparing with ex ante targets and cost-benefit analysis
- match of outputs-to-purpose
- specific outcomes on targeted beneficiaries/partners and non-targeted neighbours
- specific impacts after the end of the project on project beneficiaries and their neighbours, clearly answering the question:
  - "what will happen on the first day after the last day of the project?"
- validity of fundamental assumptions and risks
- appropriateness of objectively verifiable indicators, OVI (direct and proxies)
Assuring Dynamic Internal Lessons-learned Processes about fundamental assumptions
- - "what we thought we knew but experience taught us otherwise"
- Distinguish between lessons identified, learned/acted on, and remembered. The importance of initiating this process earlier than usual in the project implementation process.
Promoting Shared Learning of Project Outcomes (successes and failures) among relevant institutions, to:
- encourage better future project design
- inform those who exert influence on:
  - policy and decision makers to create enabling conditions for more sustainable development
  - vested interests that oppose policy reform for sustainable development
Presentation of Findings and Recommendations Style
- Analytical
- Accessible
- Readily Replicable

Course Information Resources

METR Criteria and Indicators, including:
- Problem Tree
- Logical Framework Development
Contextual Readings: "Best Practices" for M&E from EuropAID, World Bank, USAID, FAO, IFAD, EU, CIDA, DfID, GIZ, and others
Bullet point reporting style
M&E Case Studies
Development Project Site Reports

Experiential Learning Process

Assuming adequate FRE background,

Introduction of methodology materials
Establishment of student METR teams, consisting of allocated skills roles:
- Economics/Finance/Accounting
- Technology/Science of interventions
- Institutional/Governance
Design of METR for actual development projects
Implementation of METR for on-going development projects, including presentation of findings to stakeholders for feedback
Final METR report for dissemination to project implementers, donors, and other interested parties

Student Evaluations

Student METR Teams (confidential multiplier self & team member assessments, ranging from 0.7 to 1.1)
- METR-compliant M&E of an actual development project - 65%
Individual Student
- Participation performance in the course - 15%
- Brief of policy implications of METR findings - 20%

Student Numbers and Deployment

Student number maximum of 12
Students in teams including 3 skill sets:
- Economic/financial
- Science/technology
- Institutional/governance
Up to 2 students per skill set
Each team is assigned part or all of the development project area

Course Schedule

This course is offered during the summer semester, between May 01 and May 31, with a total of 31 days or approximately 4 week duration.

Week 1

Depart for Indonesia
METR method instruction
METR design
Travel to development project site

Week 2-3

Implementation of work on site evaluating development project

Week 4

Presentation of findings to development project for feedback
Final report
Exit field site