My name is Mwita Babere and I work with the Monitoring and Evaluation department as a Research and Advising Manager. My major tasks are to create and maintain continuous feedback loops to update programs on the data collection progress and reporting, explaining and advising programs on metrics in a systematic and effective way for decision-making. This includes writing M&E reports for all major assessments the team conducts for our impact programs and leadership program. Recently we’ve been able to share reports on 2012 literacy improvements with the Education team and the schools where we work. Before that I completed a report on our findings for the Agriculture team on both maize harvest yield increases and the change in income it causes. I also create M&E curriculums and training guides and co-facilitate data collection trainings. This helps support our data collection team, prepares our temporary enumerators to conduct field surveys in the area, and ensures that a high quality of data is returned to the office.

Additionally, I conduct research on M&E approaches, techniques and best practices so as to enhance our M&E system. Recently I have conducted additional research on needs assessment strategies, scaling and scouting strategies, and household hunger measurements. Currently I’m working on how to measure household hunger so that we can better understand the impact our Agriculture Program has beyond maize yields. By researching best practices and how other organizations and institutions measure hunger frequency, duration, and severity, we can ensure that Nuru uses proven methods. We are working to adapt these tools as we consider the activities of the Agriculture program and Nuru International Kenya, the harvest seasons and cycles of hunger here in Kuria West, and the cultural context in our communities.

These tasks are challenging, yes, but I am always motivated because every day is a learning experience for me. Constant interactions with people in the community gives me an opportunity to get to know a lot of things about the levels of poverty and its root causes, changing my previous assumptions. It’s amazing what M&E results can depict and working with M&E is such an exciting experience. It goes without mentioning that team cohesiveness and good interpersonal relationships within our team makes the seemingly impossible possible.

Thank you for reading this post and feel free to leave a comment

The post Thoughts from Kenya: M&E Guest Post by Mwita Babere appeared first on .

In Ethiopia, Kristin Lindell is one of our newest Fellows and has joined us to help develop out our M&E systems. With a background developing M&E systems at Global Brigades in Panama, Kristin’s first main task will be to train on and implement a Strengths and Needs Assessment to investigate deeply into the need of hunger in the community where we’ll be working. In addition, she’ll be training on and implementing Logic Model frameworks to provide our new programs with supports for program planning, project management, communication and monitoring and evaluation.  Another main task in Ethiopia will be the implementation of the Multidimensional Poverty Assessment Tool (MPAT) to gather a baseline level of poverty in the community.  We already started translating the MPAT into Amharic and Gamugna (the local mother tongue) with the help of some of our new counterparts in Ethiopia: Bereket Akele and Yohannes Ethiopia. The MPAT will also provide Nuru with key insights into the various components it measures; and it will be interesting to see how these components measure up against the scores at our project in Kenya.

In Kenya, Rogonga Augustine, Kelly Gannon and our M&E team are busy strengthening existing monitoring systems by working across programs to truly understand the ins and outs of each program area. The team has successfully worked with Healthcare (HC) to build a monitoring system that HC Field Officers (FOs) employ while at home visits.  The data we are retrieving includes information like:

• Number of home visits per FO per month
• Number of calls/SMS received and made per FO per month
• Number of pregnant women visited per FO per month
• Number of referrals given per FO per month
• Number of follow-ups made per FO per month
• Commodities sold per FO per month

In addition, we are implementing a Salesforce cloud-based customized database for data storage and reporting.  This has been a difficult process but we are hopeful for the upcoming launch at the end of the month!

In 2013, for Agriculture, we will continue to measure the yield of farmers for the 2013 maize long-rain season.  In addition to this, we are hoping to measure the occurrence of hunger seasons and how this changes over time for our farmers. For our CED program, we will continue to strengthen our systems for a  more thorough understanding of our impact.  Our M&E team is working with CED FOs to measure the percentage of CED members regularly tracking expenses and savings. For HC, we’ll follow up at the end of 2013, from our baseline we collected in 2012, to measure program metrics in accordance with HC’s goal toward the adoption and maintenance of identified healthy behaviors in order to improve maternal and child health. For Education, we gathered a baseline in January to measure the Standard 2 level literacy using the Uwezo tool among 4 intervention schools in Isebania division, 3 schools in Mabera division and 3 intervention schools in Mabera division. We’ll follow up at the end of the school year to determine impact for the Education program.

We are excited to report our data at the end of 2013 to demonstrate the potential impact of Nuru Kenya’s interventions.

The post M&E Update: Measuring in Kenya & Ethiopia appeared first on .

I am getting ready to experience some big changes in my life. I am due to have a baby in about three weeks. It is our first child, so we are wondering what it will be like, experiencing some anxiety, but overall very excited about it. It is a grand new adventure that we have been thinking about for some years, so we are overjoyed that the moment will finally arrive. We will be parents!

While I have been pregnant, I have been thinking hard about my position at Nuru and my future here. I have made the very tough decision to move on to something different career-wise after the baby comes. This brings me back to this foggy morning: I have accepted a job at a startup that is based here in Ohio so that I can travel a bit less and spend a bit more time here in my home. I love the work that Nuru does, but my personal situation for the last couple of years has made travel to Kenya an impossibility, and that has been very difficult. I have been away from the front lines of what Jake calls our fight against extreme poverty, and that makes my job harder than it should be. I can only imagine that having a little squealing infant will make travel to Kenya even less frequent than it is now.
I had the opportunity to talk to a young man who is interested in potentially starting a career in international development yesterday. It was great to talk to him about Nuru’s approach to the problem of extreme poverty, and to describe the lessons we have learned over the last four years. It has been an amazing experience to work in an environment like Nuru’s. We innovate and are “nimble”, but we have structure and we prioritize good management and leadership of our people. As I explore the world of for-profit start-ups, I have learned that it is quite amazing to have both of these aspects of an organizational make-up present in one place! That is what we get for being founded by an entrepreneurial Marine: an organization that chases cool dreams in an organized and structured way. Pretty great.

I will miss many things about Nuru. I know from past experiences of making big life changes that I can’t even define what all I will miss – there will be things I nostalgically think about in the weeks and months to come that I might not now think of fondly. I know for a fact now that I will miss the people, their intelligence and tenacity and fun-ness, I will miss the innovation, and I will miss the exciting global mission.

Nuru truly gave me a totally different view of life on earth for humans that will never go away from my consciousness. To go into the home of someone who lives on another end of a spectrum of access to resources and consumption of them, put his or her baby in your lap, share a pot of chai with them, and discuss life and what they experience on a daily and yearly basis is an amazing lesson in surface-differences, and much more importantly and profoundly, absolute and deep similarities.

These similarities might be what my old meditation teacher would call the capital-T Truth: that there is just one thing on earth.

To learn what that one thing is, one must search far and wide, both within and without.

Nuru gave me an opportunity to conduct that search, and thus changed me forever.

Thank you, Nuru.

The post Thank You Nuru appeared first on .

The M&E team implemented a survey tool called Uwezo meaning ‘capability’ in Kiswahili. Nuru adapted the Uwezo assessment, developed by a regional (Kenya, Tanzania and Uganda) initiative, which is a booklet that consists of reading and comprehension assessments in English and Kiswahili to administer to the children. Standard 2 level literacy is defined by Uwezo as the point at which a child successfully reads a short story and answers two comprehension questions correctly. The assessment was implemented by Nuru M&E staff in 2012 at the beginning of the school year (BOY) and end of the school year (EOY). The Nuru M&E team assessed 100% of the students present for the assessment (1,950 total) in all nine schools.

Results show that while the majority of students are not reading at the Standard 2 literacy level until Standard 6, both male and female students made substantial improvements in Standard 2 literacy during the 2012 school year. In the earlier grades, Standard 2 Kiswahili literacy improvements outpace Standard 2 English literacy improvements, whereas the opposite is true in the upper grades.

Efforts to quantify the impact of the Nuru program will be made during the 2013 school year, by using comparison schools within the community.

The post Education Uwezo Literacy Assessment 2012 appeared first on .

This will be one of the things I leave to Jamie in my absence, though, as I am heading out on maternity leave towards the end of December. Yep, I am nearing the end of a pregnancy. It seems to be taking a very long time, but I guess this is how long it takes normally. This is my first one, so I am learning something new every day. In my absence Jamie will be stepping up as the interim M&E Director. I know she will do a bang-up job. The hard part will just be prioritizing for her. She is very busy as it is, and she will have a few new things on her plate in this bigger role. She and I will work together to prepare for this in the next two months.

Alright, here is a quick update on data for all of our programs:

Agriculture

We finished a full comprehensive impact study this year. This entailed gathering baseline data back in the spring to determine what the starting point for yield for the Nuru farmers with whom we would work was and gathering impact data in the last month or so to determine their new yields after our intervention. We found great news about the impact of our intervention which we will report soon (and can report now to anyone who requests it), but I think the thing the M&E team was most excited about was that we successfully conducted the study. It was a good process and we learned a lot.

We have gathered some other data for Agriculture too in recent weeks related to income. This stuff is a little more complicated so I will not get into it here, but it was another good study.

CED

The biggest task we have been engaging in related to this program has just been working with the folks who are building our database to ensure their tool meets the needs of the CED program. It is complex as the program itself is intensely improving the way it handles all of its data right now. Much more on this later , but we are moving well down a hard path with this program.

Education

Another impact study down! We got baseline literacy data from the schools at the start of this year, and we have gathered literacy data from the schools again recently, after about one year of Nuru work. We have yet to analyze the data, so I don’t know the nature of the news, but we are very excited to have, again, conducted a good study with good data and a 100% sample.

Healthcare

We have at this point gathered perhaps all of the baseline data we plan to gather for this program. It is a lot of data. Kelly and I were just talking today and she told me she was looking at a stack of over 700 household surveys. We are working on analyzing the final chunk of surveys we just gathered, and we will report on them shortly. The more important question we’re still working on answering is how frequently and whom we will be testing in the future to most appropriately determine impact. It might seem like a simple question, but there are a bunch of factors we are trying to take into account that, again, I don’t want to get into here, but we will discuss here in the future!

Leadership

And finally, Leadership. We have nailed down our list of indicators for this program, and we are working on a couple of things. One is sharing said list with the rest of our staff. The other is going ahead and implementing the use of these indicators. Much more on this one later, but for now, we are excited to know what we want to measure for this program.

So, a lot of progress! We are happy with where we are.

The post During my Blog-Writing Break, we Gathered a lot of Data! appeared first on .

• We have gone through a lot of iterations of our approach to M&E as well as our approach to our interventions themselves, such that at this point a lot of time has passed and we are not able to show impact.
• We are not able to quickly answer some of the questions that donors and potential donors have for us about our constituents.

These are both interesting critiques that point out  some of the challenges we have had in the past and continue to have related to M&E.

As to the first point: the M&E model that we started out our Nuru work with was a well-conceived one but not one that worked in execution for what we were trying to do back then. It was based on C.A. Sullivan’s Water Poverty Index. We had a system where we generated five index scores which, together, were meant to represent the poverty level of the communities where we work. There was an index score for each of our five programs at the time: Agriculture, Education, Healthcare, Small and Medium Enterprises (SME, now evolved into CED), and WatSan. These scores ware based on a composite of a bunch of different metrics related to each of the programs. The main problem with this old system is that it didn’t really make sense to measure the effectiveness of programs that changed with the needs of communities. It made sense to paint a pretty vivid picture of the community according to a bunch of different things, but not to tell us about our effectiveness.

When we realized this was a problem, we went back to the drawing board. This was in 2010. We took the whole year to find the best new system we could then we took all of 2011 to incorporate this new system into all the programs’ models.

Now, we are pretty comfortable with this new system, but all that has happened related to it is baseline data gathering. We have gathered baseline data for Agriculture, CED, Education, Health, and even our poverty metric tool, the MPAT. We have not gathered impact data yet, but we will by the end of this year.

The new system is meant to tell us two different things: what is the enabling environment in the communities where we work (the MPAT), and how well are our programs doing. Those two purposes bring me to our second critique that I have listed above. There are a few things that donors want to know that, for us, don’t fall into either of those categories. Examples are education levels of the adults in our communities (for the education program we want to know about literacy levels of children), annual yearly income at the household level (for CED we want to know about savings rates and ability to cope with shocks), and alternate objectives of the IGA program other than profitability (we measure IGA’s success with just one outcome).

We are altering some of our data gathering tools to be able to report these numbers for our potential donors. It is easy enough to do so, so we are trying. We can tell you more about this in the fall.

Critiques are always so useful to us! We are learning every day and trying to get better at all of our work.

The post How to Talk about Monitoring and Evaluation appeared first on .

I am catching up a bit on some blog-reading and I just read this post about how poverty can be and should be defined across varying countries and regions.  As with much in our field, it is a complex question. If a country defines poverty more leniently than an established international standard (like the $1.25-a-day spending power line of demarcation defined by the World Bank), why should the established international standard overtake the country-level standard? Ways of life obviously vary widely throughout the world and throughout regions within countries. Does an international standard really make sense? I am definitely not proposing an answer to this question here. I think the question itself highlights the complexity of the work we are trying to do at Nuru, that is, development work in general.

As I head to Palo Alto, one of the questions we are trying to answer right now is whether we can supply a metric value that we expect many foundations that issue grants to organizations like ours to ask of us: yearly income amounts for our constituents. Similar to purchasing power, income is obviously meant to indicate something about the standard of living and choices available to our constituents. The problem is that we do not assess this value right now as a means of learning about the effectiveness of our interventions. Nor, in fact, do we consider it a great direct measure of what we are trying to do. We are trying to create an enabling environment for our community-members, and we feel that the MPAT is the best means of testing what that is.

For our programs, we want to know how much our constituents are saving and a few other things about their lives that could potentially be used as proxies for income with a little stretch of assumptions, so that may be what we end up deciding to do. We might decide to assume that some major portion of our constituents’ income is from the agricultural yield that they produce as a result of our interventions. As we are already measuring that, we can perhaps make an assumption about income as a result of that.

More on this little dilemma later, but as usual, we would love thoughts on this!

The post Measuring Income — Does it Test Poverty? appeared first on .

Another challenge that Gaby talked about in a previous post is measuring the impact programs, despite their ever changing programmatic shifts and iterations.  We hope that the metrics we’re collecting now for a baseline will still be valid when we follow-up to determine impact.  Even in the upcoming Olympic Games, there are challenges with collecting data.  In this recent Washington Post article, we see that swimsuit technology is changing over time thus leading to faster swim times due to reduced drag in the water. This year in London, we will most likely see a decline in swimming world records due to the decision to return back to a simpler suit.  Go Michael Phelps! And Ryan Lochte!  (I’m excited for the Kenyan runners too!)

Similar to this, the healthcare program is developing a Social Marketing strategy that aims to impact a widespread audience through community-wide campaigns.  This technology will undoubtedly affect the way we measure our programs.  It is with diligence that M&E will continually stay atop these undulations in the program areas, so we can know if the changes are effective in the short and long-term.  Luckily, we don’t have to worry if the interventions get too successful (like in the Olympic Games); but we do have to consider the scalability and sustainability of each program.

The post What can We Do Better to Measure Impact (Part 2) appeared first on .

She has so much that she has to get a handle on right now, but it is going well. She just spent this past week having meetings with each of the impact program teams as well as Leadership. It was a good experience beginning to get on the same page with her colleagues. Some of them brought up some ideas about monitoring and evaluating their programs that we M&E professionals hadn’t even thought of, which is great. I am very excited about what she is doing.

Now that we are comfortable that we have established our M&E approach (we did that way back in 2010) and we have begun to gather data related to every single impact program and are beginning to do so with Leadership, the questions we are asking and trying to answer are a little more detailed than “how can we measure literacy?”. They are more like “how can we measure literacy effectively and ensure that our findings can in some way be attributed to the interventions that our program is conducting?”.

Until the last couple of months, we have not felt that we would be able to effectively conduct comparison-based studies if we tried with the resources we have at hand. As we learn more about how other organizations have done these studies with the same relatively limited resources that we have, we feel more optimistic that we could potentially do them. We will decide how this looks next month when we have Heather Ozhogin onboard. She is our new Data Scientist. Heather, welcome in advance to what is going to be part of your workload!

Another slightly detailed question we are trying to figure out the answer to is this: to what geographic area can we attribute the results of studies we conduct?  We have done a baseline survey for Healthcare in three sublocations. The Healthcare program has started their interventions in those sublocations. They are now scaling to OTHER sublocations in the Kuria District of Kenya. Our specific question about this is: do we need to do new baseline surveys in the new sublocations, or can we just conduct our impact study in the three sublocations where we started? I’ll get into what the answer might be a bit more in my next post. Until then…

The post What Can We Do Better to Measure Impact? appeared first on .

I have spent much of this first month absorbing and observing as much as I can during our transition from FT7 to FT8. It’s been great to connect the work I was doing remotely from the US since February to the project in Kenya. David has been incredible at bringing me up to speed on the operations and strategies of the M&E team on the ground here. While I have learned a lot, one aspect of our work as M&E really stuck out in my first couple weeks.

For me, the questions as to the what, why and how of data have driven (and complicated) a lot of what I do personally, academically, and professionally. In graduate school I struggled with the idea of linking data to human rights in my research projects. Numbers are an extremely powerful tool wielded by donors, governments, diplomats, activists, and development workers to plan policy, set benchmarks and be motivated to act. But what can be observed and quantified never tells the entire story and numbers can easily be manipulated.

As a Monitoring and Evaluation Program Manager, these questions as to the what, why and how need to always be asked at each and every step of program and operational measurements. Its something I need to constantly be reminding myself to do as I am in the transition process here in Kenya. The why and the what are previously what I   concentrated my focus on when analyzing and thinking through ideas. Very quickly after I arrived though, I was reminded of the importance of the how question.

I went with my team to Ngochoni where we are doing data collection on last year’s harvest yield of new farmers who joined Nuru this year. The amount of effort that goes into three data points (GPS coordinates, number of acres farmed in 2011 long rains, and total number of bags of maize harvested) is much more complex then I would have thought from my desk in NYC. Most shambas (farms) are not neat, one acre plots. Some are surrounded by bushes, others have odd shapes with a banana grove in the middle, rows are not straight and evenly spaced, and many plots are far from the home. Furthermore, asking the farmer for the number of bags collected is not a simple answer either. The farmer might not have bagged all the maize in the standard 90kg bag to sell, it might be stored in a homemade granary or collected in large tins.

Despite these challenges and others, I was also extremely inspired and excited about the work I’m a part of. The local staff here is competent and understands the shortfalls of the information collected as well as the strengths. Data is a messy, difficult and challenging tool—but that is part of the reason I like it. If you only spend time looking at neat numbers as they are presented in a report, it is easy to forget how each and every number is so complex. As we finish collecting harvest yield and start to shift to data entry and then data analysis and reporting, we have to remember where the information originated in order to accurately tell the story.

The post What I (Kelly Gannon) have Learned this Month about Data Collection appeared first on .