I am in Palo Alto this week and last to meet with Jake and the rest of the Senior Leadership Team and to work with FT8, the newest group of folks who are heading to Kenya to work at the project site. This group consists of all new people, which is a first for us (other than FT1 way back in 2008). Everyone is a new-hire heading to the field for the first time and becoming part of the larger Nuru team for the first time.

I think it is a great dynamic because almost all relationships are brand new ones (there is one married couple…so that one is not so new). The way the team shapes up and ends up working together will be organic and there will not be one or two members of the team to whom the rest of the members turn to figure out answers to questions about Nuru and its ropes.

Aerie and his team have been working on evolving the nature of the makeup of the teams, and I think that this one will be pretty great. There is a dedicated team leader and there are fellows for each of our current programs.

I need to get used to the fact that M&E is no longer the new kid on the block in the field, and has not been for a long time. A couple of years ago, I made the case for having someone from M&E on each foundation team, and it was frankly an easy sell. Since then, we have always hired someone or gotten a current team member to go on each rotation. Also since then, we have hired our wonderful Kenyan staff, which makes us unbelievably more productive in the field.

The M&E fellow on this next rotation will be Kelly Gannon. Kelly has already been working with Nuru for the last two months or so. She knows her way around our organization. She knows the things we are struggling with and the things we have under control, and later today Jamie and I will have a conversation with her about what we hope to be her main areas of focus when she does this 13-month rotation. Nuru overall, and the M&E program in particular, is in a very good position at this point to really improve upon our work.

We have gathered baseline data for all of the impact programs, and our contractors, Derek Yankoff and Troy Hickerson, are hard at work developing a database to house all of the baseline and impact data that we are gathering for all programs. We have nailed down some draft indicators for our new Leadership program, and we will put them to use in the field soon.

Today when Jamie and I meet with Kelly to discuss her goals, we’ll talk about what to do next. She will focus on continuing to develop our Kenyan staff and hire new people if necessary, She will also focus on continuing to gather data when necessary, but most importantly, she will be working on analyzing the data we already have with the field staff. She will play a very important role in a very transitional and growth-filled time for Nuru. We are so excited about this new team!

Posted from Palo Alto, California, United States.

I spent a couple of days this week creating a deck that draws upon a lot of the knowledge I gained from working at Booz Allen Hamilton and Booz & Company from 2005 through 2008. During that time I was lucky enough to work with a few program management experts like Eric Kronenberg. I learned about creating a schedule by determining tasks and activities, assigning resources to them, and creating dependencies. I learned about cost accounting and budgeting, the critical path, crashing, forward and backward passes, and also risk assessment.

I worked with clients who were manufacturing large items for the military. When I say large, I mean really large, like, submarine-size, or airplane-size. Bigger than a breadbasket. These items were very complicated and required significant quantities of tiny and huge acts of labor and tiny and huge pieces of material to all come together to create something that flew, swam, drove, or otherwise propelled itself across distances. Small schedule slips could make or break the whole process. There were teams of people assigned by our clients to manage the schedules, and they coordinated frequently and significantly with each other.

One of the last clients I worked with was a company trying to take on a new project that was in their industry but outside of their expertise. They were very optimistic about the revenue potential of the new project until they got a month and a half behind-schedule, and that equated to going more than 30 million dollars over their budget. They were not practicing sound project management. Our team was there to train them about what that is.

I have been thinking about applying some of the things I learned back then to the way we do scheduling and budgeting here at Nuru since I have been here, but the work we do is very different from the work that my old clients do. Although it is complex work with many moving parts, the output of our work is incremental change across many factors in human beings’ lives rather than tangible large pieces of equipment for which the indicator of success is binary (they either work or do not work). Also, I should note that there already is a good deal of rigor in the way our work is currently managed. All team-members know how to develop schedules and budgets and manage to them.

That being said, there are a few concepts from my old field of work that Aerie and I feel are worth introducing to the field staff in the next Foundation Team training at the start of May. So, I got to build a deck this week that reminded me of my old work. During the training, we’ll talk about critical path, dependencies, resource loading and just one or two other small topics. Nothing too excessive or crazy, and nothing specific to the world of manufacturing.

I am interested to see how these topics are received by our staff. I anticipate that they will like it. Maybe I can eventually convince my old colleagues to use our success with program management as a good example for people who are trying to build airplanes. Who knows?

 

 

 

 

Posted from Cincinnati, Ohio, United States.

I mentioned a few weeks ago that we have gathered a lot of data related to many of our program metrics. The analysis and QC that we were conducting back then and have continued since have gone very well.

At this point we have finished the first round of analysis and QCing for the baseline data for the Education program’s literacy assessment, which, as a reminder, was done with the Uwezo tool. We have baseline literacy levels for boys and girls by grade level at several of the schools where the education program has been conducting interventions. In time, we will conduct a subsequent test of these students to see whether or not we have, to use one of Jake’s old favorite phrases “moved the needle” on literacy. Continue Reading…

Posted from Cincinnati, Ohio, United States.

The M&E team had a pow-wow with the Education team last night. We discussed a couple of the issues I mentioned in my last post, and how we should most appropriately react to them. We have a couple of ideas, but none of them are set in stone.

One simple thing we might do the next time we test the literacy levels of the children in the community is ask them how long they have been enrolled at their current school. If we do this, we will likely have to provide a couple of options for the surveyed children to choose from just to make analysis easier. If a child has been in the school a significant number of months, consistently, then we will have to decide to assume that they have been exposed to our interventions consistently. These are the children we want to test.

One problem with this approach is the awkwardness of asking that question of children. Will it make them uncomfortable or feel like we are being exclusive in whom we want to test and/or do not want to test? Another problem is whether or not they will understand or know the answer to the question. We deal with self-reported recall data all the time here at Nuru, and, though we have gotten pretty
used to it in terms of data gathering, it is questionably reliable sometimes. We are talking about asking little children to remember some length of time. There will be some error.

Another concern we have about this test is just the timing of it. The longest time ago that we have baseline literacy rates that we are confident with is November of 2011. So that means, at this point, taking into account breaks from school when no interventions are happening, only two solid months of measurable interventions have occurred.

This run-time issue is, to me, the biggest issue we need to deal with before we decide when to test the literacy levels of these children. We need to determine whether we truly believe that our interventions should have affected literacy levels at such time as we decide to test next or should not have. If we believe they should have, we will test, if we believe they should not have, we will have to consider utility
of testing the children weighed against the impact on the schools and their ability to learn.

That is what we are working on now: a more solid perspective on how quickly we expect our interventions to affect literacy levels than what we have now. We hope to have that soon, and once we do, we will determine when to test again and how to do so.

Posted from Cincinnati, Ohio, United States.

Last week I gave you a quick update on what we have been doing related to Program Metrics.

I would like to brag about how we have been analyzing the data we have already gathered.  David and Rogonga have been doing some hard-core analysis of all of the data we gathered for Healthcare and Watsan in December, and they will have some values to share with us in about a month. They have done all the analysis and formula-building at this point, and what is left is just some data cleaning. It takes time because they have to reconcile hard copy surveys with what was entered into our data entry sheet. Jamie, as well, has been conducting complex analysis, in her case of the already extant and built-by-Jennifer Ag model. Last Friday she built a nested-if function that contained seven sub-functions. Those of you who love the puzzle-solving that working with Excel allows you to do will appreciate how fun that was for her. (I’m not kidding, it was fun! She told me.) I myself have gotten to do some excel modeling with the literacy-data we have gathered using the Uwezo tool. These values as well will be reportable once the data has been cleaned a bit.

We actually HAVE data. That is the great news. We are in very good shape, and we will have some to actually report to all of you in the near future.

The main thing that we have been spending the other half of our time on here on the team is Study Design. That is what we are calling determining the when, how, and who of the next and all subsequent iterations of data gathering for each Program Metric. Some things to take into account here are

  1. How long it might take for an activity to actually affect a metric value
  2. How much gathering the data related to the metric value will impact the community
  3. How long it will take and how difficult it will be to gather the data
  4. Whether any extenuating circumstances might affect the value of the metric within the timeframe that we have chosen for assessment

All of these points are important, and my team spent a little bit of time discussing this last point in some detail today.

Studying a community and how it changes and its members change is not like a perfect experiment that you might have learned how to run in High School science classes. There is no way to create a completely immutable environment when the subjects of your experiment are human. A couple of extenuating issues that have come into play with us so far are huge fluctuations in market maize prices, droughts (of course), Somali refugees coming into our communities, government mandates about school-closings and subsequent re-distributions of student populations, government rules about Community Health Workers, and many other things.

So the question we were faced with today was, how should we design our studies such that we are able to measure the impact of our interventions when we know that wildly varying extenuating circumstances are going to come into play for all potential subjects of our study, both comparison groups and standard groups?

Because this post is already a bit long (and because I don’t know the best answer to our question just yet), I’ll get back to you in two weeks with a follow-on post.

To be continued…

Posted from Cincinnati, Ohio, United States.