Monitoring and evaluation of Community Based Adaptation: Where we are now?

Developmental practitioners have been monitoring and evaluating (M&E) developmental interventions using diverse set of participatory tools and concepts since time immemorial. Oftentimes it is done by comparing outputs and outcomes with the objectives and targets set by the project in a participatory manner which worked very well in a well understood field of development where boundaries are well defined and baselines stay static for all practical purposes. However, the concept of climate change adaptation is not at the same level of maturity as that of development, neither boundaries are clear, nor the diverse players engaged in climate change adaptation have come to the same terms as their engagement in development. This brings us to a set of well acknowledged challenges for M&E of climate change adaptation which include uncertainty in nature of impacts at time and geographical scales which makes difficult to pinpoint baselines which are not static as in a non-changing climate, long-term nature of climate change adaptation benefits accrued from adaptation projects while most projects are planned and implemented at relatively short periods, and different opinions, concepts, contexts and scales in which adaptation takes place. The overarching questions for M&E of CBA are how far can we use traditional/existing tools for M&Eing adaptation and what additional concepts/tools are needed for M&Eing adaptation? Specific questions are who decides what and how to be M&Ed, should different expectations of stakeholders from adaptation means different M&E frameworks/concepts/tools, and how to communicate and ensure M&E doesn’t become a burden on resources?

The session discussions have revealed that the same participatory tools that have been engaged in developmental planning could be well used for M&E of climate change adaptation (Robin James and Mike Wiggins).  Tools that bring greater integration of different actors, time and three-dimensional space appear to have greater value in understanding vulnerability and resilience (Robin James) and without fail be able to capture the most significant change brought by the project (Mike Wiggins). However, there is a need for stronger conceptual framework within which these tools can work well and communicate the right message from the evaluation outcomes.

Participatory tools have become standard protocol of entry into local level interventions for many agencies, whether for research or development or for both.

The session has clearly brought out that the conceptual frameworks are better developed if they are developed bottom up i.e. by distilling experiences and messages from M&E of on-the-ground projects (i.e. inductive) rather than through deductive means. The Participatory Monitoring, Evaluation, Reflection and Learning (MERL) is one such framework that has evolved bottom up from the experience of Action Research for Community Adaptation in Bangladesh (ARCAB) and other experiences that the IIED-CARE team has considered while developing MERL (Jessica Ayers). MERL employs learning by doing approach where in communities and practitioners are able to track, respond to, and take advantage of changing contexts and surprising events and emphasizes ‘accountability downwards’. Since the framework is based on several on-the-ground experiences, it has been able to identify a common set of indicators that all partners have chosen to evaluate the effectiveness of their actions.

ARCAB is an innovative long-term community based adaptation project that aims to generate scientific evidence for community based adaptation with the involvement of several local, national and international partners (Sumana Tanchangya). Being an innovation in itself, ARCAB has encountered several challenges which include difficulty defining CBA and the wide range of action and research partners working in different communities with different approaches, indicators and frameworks. On the positive side, these differences and commonalities have provided a fertile ground for crystallising M&E framework that was eventually emerged as ARCAB M&E Framework.

One major limitation with the M&E frameworks and approaches appears to be approaching resilience and adaptation ex ante i.e. informing project planners what would work and wouldn’t work with reasonable confidence before the project is designed and implemented. However, the Climate change and Environmental Degradation Risk and adaptation Assessment (CEDRA) pilot program appears to have overcome this limitation to a certain extent by putting emphasis on participative processes leading to identification of adaptation actions (Mike Wiggins). This program also recognizes the fact that communities are not just facing climate change but many other issues and a means to address all these issues in the M&E framework.

The session has clarified that the communities has to be the ones who determines what need to be measured and evaluated as is evident from the work presented in the session. However, it was observed that most often community tend to focus about immediate responses and the participatory adaptation interventions should be able to help them to see that short-term responses won’t help much for climate change adaptation. Frameworks/tools developed only based on ‘common elements’ tends to miss important lesson that differences and reason behind the existence of these differences has to offer. Avoiding tipping points is absolute necessity and it can be done by taking climate risks out of risk category and tracking what people are doing, outcomes around adaptive capacity and vulnerability relative to climate risks. 

Prepared by SVRK Prabhakar (IGES) based on discussions at the session on Monitoring and Evaluation of CBA, 6th International Conference on Community Based Adaptation, Hanoi, Vietnam. 

Bringing Evidence to the Table: Reflections from the 6th International Conference on Community Based Adaptation, Hanoi, Vietnam

Attending 6^th CBA in Hanoi, Vietnam I started realizing that people are getting more comfortable speaking about CBA. They don’t anymore talk about what is CBA and why CBA is needed but rather have moved to a next higher level of discourse i.e. sharing their experiences in a more articulated manner.

On the positive side, considering that this conference is about and for practitioners of CBA and most representation of speakers are from on-going projects,most speakers have spent considerable time explaining about their project and what it aims to achieve. It is evident that more and more projects are putting efforts involving multiple actors into their projects, and surprisingly even government-driven programs talk about involving communities. Most of this change could have come both from self realization that adaptation is all about participation, ownership and empowerment; but partly must also have come from emphasis by NGOs, bi- and multi-lateral agencies such as GEF, World Bank and others.

On the down side [probably a better word would be ‘work in progress’], still very less evidence is being presented on how CBA project have benefited communities and other stakeholders and actually have led to adaptation or climate risk reduction. Probably this could be done by selectively inviting participation from projects that have finished implementation. It is tricky since very less or no resources would be available for project staff to travel and present in conferences after the project is finished.

A plenary from 6th CBA, Hanoi, Vietnam

Presenting empirical evidence would have made lot of difference to the conference gathering and to the CBA in general. Most work presented talks about what communities believe and think and stress on how to involve various stakeholders in CBA in ‘textual’ terms. While these are important operational aspects, we need more refined information being presented in a much more analytic manner. This preference could be just for me, since I recall people saying this is not the venue for presenting analytic work. Nevertheless, analysis results can always be tailor-presented to the audience. One area where CBA literature can improve is presenting reasons and framework/methodology behind identifying practices and policies that projects chose to implement. Why only X and why not Y, how much of the decision to chose X over Y was influenced by community preferences and those of outsiders? There are questions raised about right to adaptation, fairness in decision making, and even more importantly if inequality is being well addressed.

There is still a larger question that lingers in my mind. How much a project planner or even a researcher should weigh in on what communities think and believe [We don’t give second opinion on what a doctor prescribes as a medicine even for cold] and how to cross-check what communities think is good for them is really good for them. Probably this can be done by emphasizing and help converting unfelt needs and wants into felt needs and wants. Probably this is happening on the ground but nothing much has come into presentations and articulations. 

I think CBA has passed the stage of saleability; however this doesnt mean that we stop hiring sales men to sell it [I changed this sentence after hearing the speech of Richard Bossi]. We also need more people who have reached a stage of providing more emphatic, empirical, and analytical evidence (even if it is qualitative), and sharing experiences of CBA working on the ground; clarifying operational overlaps, as against conceptual ones, that exist between development, DRR and CCA; and measured benefits from mainstreaming CCA/CBA/DRR. M&E can help achieve lot of this. It would be interesting to see M&E staff from projects sharing their experiences of M&Eing adaptation and what they expect from M&E researchers or those who tailor M&E tools for adaptation.


More to come...