Results Framework Builder
Mercy Corps is an international humanitarian organization working in over forty countries across a broad range of focus areas including agriculture, cash distribution, emergency response, food security, gender equality, and governance. I work on Mercy Corps’s in-house Monitoring, Evaluation, and Learning (MEL) platform that enables program teams to track progress towards performance targets and report results with supporting evidence.
Program design at Mercy Corps begins with a theory of change and a results framework that explains the logical sequence by which the program expects to achieve its goals. One of my most interesting projects involved designing a tool that allows a program team to define its results framework, so that the system can automatically organize program data according to this hierarchy.
Problem space
I joined the team at the beginning of an intensive development and rollout period, with the ambitious goal of scaling a promising Minimum Viable Product into a global enterprise solution. UX discovery and evaluation of the existing product uncovered a pervasive barrier to adoption that sounded something like this:
It’s not enough to produce lists of indicators and results. The data needs to be organized according to our program’s results framework. If we can’t see which result each indicator is measuring, how can we tell if we’re on track to deliver our intended impacts? It’s great that the software can automatically generate reports, but we can’t really use them or share them with donors as long as they are missing the results framework.
Conceptual design
Before diving into interaction design of a complex feature set, I like to create a task affinity diagram that synthesizes everything I have learned about essential user tasks into a concise design reference.
A results framework with performance indicators linked to most levels (released product)
Task affinity diagram
Interaction design
The challenge was to create an intuitive, flexible, and forgiving process for organizing results into a hierarchy. In addition to investigating how other MEL software applications had solved the problem, I sought inspiration from unconventional sources such as collaborative diagramming applications and an online card sorting tool. I rapidly generated variations of key interaction details.
Early explorations of how one might go about adding a result level to the right spot in a hierarchy — above, below, or in between other levels.
The results framework builder offers a menu of templates, each one using the terminology preferred by Mercy Corps or a major donor. These are controlled vocabularies that can be displayed in three languages. For programs using other languages or terminology, a custom template allows them to manually label each result level.
Several examples of results framework templates, including a customizable option. (Released product)
When I am responsible for visual design in addition to interaction design, I create high fidelity annotated mockups to reduce ambiguity for developers. This illustration depicts the transition between adding a new result level and linking your first indicator to that level.
Annotated mockup included in a Github use case
Every result level can have one or more performance indicators linked to it. The process needed to be flexible in order to accommodate the many program teams that had already entered indicators into the system, prior to the results framework being introduced. We made it possible for these programs to build a results framework and then assign pre-existing indicators to each result level.
Two indicators are linked to this result, one quantitative and one qualitative (released product)
Each performance indicator has a target number or percentage value, and program results are measured against those targets. In addition to a final “Life of Program” target, indicators can have periodic targets to track progress at regular intervals — annually, quarterly, monthly, etc. We implemented periodic targets in an earlier product release and made sure all options were accessible through the results framework builder.
This indicator has targets associated with a series of events (released product)
The ultimate purpose of the results framework builder was to provide the structure requires for the system to automatically generate standard program documentation that would otherwise be time consuming to create by hand. The logical framework (aka “logframe”) is one of those standard program documents required by major donors.
The logical framework can be viewed on screen or exported as an Excel document (released product)
The Indicator Performance Tracking Table (IPTT) is another type of standard document required by major donors. We built the IPTT prior to the results framework builder, and it originally displayed a simple list of indicators with no indication of which result level each one was intended to measure. Once program teams were able to define their results framework, the system could display indicators in the expected hierarchy.
The Indicator Performance Tracking Table (IPTT) can be viewed on screen or exported as an Excel document (released product)
In a follow-on release, we introduced the ability to export your results framework as an Excel diagram. This is another standard document shared with stakeholders, that is particularly fussy to lay out by hand. The design is based on a Mercy Corps template, but uses a neutral color scheme that can easily be customized for donor reports.
After releasing the results framework builder, we received feedback that users would occasionally link lower levels to the wrong upper level, and not realize it until after they had already invested a lot of time building their hierarchy and adding indicators to each level. I collaborated with UI designer / front end developer Paul Souders on a miniature “wayfinding” diagram that visually reinforces your location in the hierarchy, and also serves as a navigational element, jumping you to the details of the result level you want to review or edit. This enhancement was in development when I left the organization.
Results framework in diagram form, exportable to Excel (released product)
Early concept sketch on paper
Annotated mockups used to communicate interactions to developers.
Project details
Role: I was the founding User Experience Lead for internal enterprise scale applications at Mercy Corps. I played a key role in establishing a human-centered product design and development process. It was a startup-like environment, where I necessarily wore many hats, taking on UX research and serving as the team’s product owner.
Contributions
UX research and strategy
Conceptual design
Precedent study
Interaction and visual design
User stories and acceptance criteria
Acceptance and QA testing
Collaborators
Project manager: Emily Sinn
Developers: Cameron McFee, Ken Johnson, Sanjuro Jogdeo, Paul Souders
Subject matter experts / end user advocates / QA testers: The Performance and Quality team at Mercy Corps, especially Marie Bakke, Hanna Camp, and Carly Olenick.