Looking for Help in Reviewing our Monitoring and Evaluation System

We had a couple of questions from a donor this week about our third-party evaluations. We have done two of them officially. Here  is a slightly edited version of how I put it to the donor:

We have conducted two third-party evaluations, and the results of both can be obtained by emailing me at gabrielle.blocher@nuruinternational.org.

The first evaluation was done with our original M&E system that we subsequently overhauled after this use of it, so there is not much longitudinal data on these indicators. It was conducted by two outside consultants, Sarah Sathaye with Bain Consulting and Crystal Ashton with Choice Humanitarian.

The second evaluation was done using the MPAT (Multi-dimensional Poverty Assessment Tool). We have chosen the MPAT as our tool for assessing the poverty levels of the communities where we work according to the existence of an enabling environment. This evaluation was led by one of the lead authors of the MPAT itself, Alasdair Cohen, with IFAD.

We definitely plan to continue to have third-party input into our work, but we are not sure it will be in the form of the conduct of the MPAT, as we feel we have some good in-house expertise in being able to implement this unique tool. We will continue third-party evaluations though.

We have not changed our programs as a direct result of the evaluations mainly because the two evaluations were not only very different from one another, but also conducted in differing sub-locations. Also, the MPAT is meant to assess the poverty level of the communities where we work, not the success of an actual intervention. The latter is assessed using what we call Program Metrics. Program Metrics are continually assessed and are tools to figure out whether interventions are working. We have used them to change the nature of some of our interventions.

So, that is one way to explain what we are doing. I hope I was able to answer the donor’s questions. The donor was most curious about how often and in what manner we have outsiders come in and take a look at what we are doing. As a former management consultant, I am a firm believer in the utility of the “fresh set of eyes”. Having them around is not without its challenges, though. Outsiders who are on a short project (a month or so) have to be schooled up on as many aspects of Nuru’s entire model as they possibly can so that their input can be useful. If they do not understand the model, their critiques of aspects of it will not be as useful as they could be.

The reason I am thinking about this issue is not just because the donor asked, but also because we would like to have another third-party evaluator come in and simply review our entire M&E system. Both times we conducted evaluations in the past with outsiders, we have asked for some additional help in the form of critiques of our system. In both cases, the evaluators were so consumed with conducting the evaluation that critiques of the system were not easy to spend time on. If you know of someone who is an M&E expert and would like to consult for us, please let us know! We are excited to get some major feedback. The time has once-again come for that.