Building an Impact Product during the COVID-19 Pandemic

By: Art Min, Erin Boehm

Overview

Working on a technology project where the team is spread across many geographies is challenging. Doing it during a global crisis adds additional uncertainty which exponentially increases the risks of project deliverables. Earlier this year, Erin and I had the opportunity to work with the World Health Organization (WHO) to define, build and deploy a MVP of a digital solution clearinghouse (DCH) focused on helping low and middle-income countries (LMICs) respond to the COVID-19 outbreak.

Welcome page for our MVP of the Digital Clearinghouse

Ministers of Health from LMICs were challenged in responding to the crisis as they didn’t have the resources or experts to help identify and qualify digital solutions such as contact tracing, predictive modeling and self-diagnosis tools. They were largely relying on social media and the news to inform them of what other countries were doing to respond to the crisis. This took precious time and further increased the risk of infections for their populations.

Our shared objective was to give Ministers of Health full information and data on the solutions being used by other governments to accelerate the procurement and deployment of solutions. We believed that this would have a material impact to help reduce the number of COVID-19 cases and save lives.

The DCH team was composed of people with medical, policy, logistics, technology and business backgrounds across 6 countries and timezones. We were asked to move quickly, as the entire project was meant to be delivered in “days not months.” This included interviewing key stakeholders, creating mockups, and building the minimum viable product (MVP). This document covers some key processes we used to coordinate this team and achieve project deliverables.

Our hope is that this document will help others working on remote teams organize their projects during the COVID-19 response. Please feel free to reach out if you have any questions.

Art Min spent 20 years leading teams to build Enterprise grade products at big companies and startups. For the last 3 years Art ran the Paul G Allen Family Foundation and led a team to build Enterprise grade products for intractable problems such as the Ebola crisis, poaching in Africa, coral reef conservation, near real time satellite reporting tools, and climate modeling.

Erin Boehm has designed, built, and implemented end-to-end business processes and business intelligence to support the Paul G. Allen Family Foundation ($2b of giving life to date) and Tech4Good projects at Vulcan Inc. Erin has 10 years of business analysis and project management experience working across the philanthropic, nonprofit, and government sectors.

Introduction

In mid-March of this year, I made the decision to try to help with the COVID-19 outbreak. I was fortunate to work on infectious disease programs at Vulcan and felt I could help an organization respond to the crisis. I reconnected with Steve Davis, ex CEO of Path, and he shared that he was leading the Digital Health Technology Group (DHTAG) at WHO. The DHTAG had a number of initiatives underway including a digital solution clearinghouse. They heard directly from Ministers of Health that they were using social media, the news, and input from their personal network to identify digital solutions to predict, understand, and treat infected people. Unfortunately, this meant that they did not have full information to determine which solutions to pursue and deploy. WHO had the idea to centralize some of this activity and help qualify solutions in accordance to emerging standards and best practices. These solutions would then be presented on a website that could be easily accessed by member states. Steve asked that I take on the Product Manager role to get a MVP launched quickly.

A small team of individuals from WHO and its partners were assigned to the project which really helped ensure that we were in alignment with WHO leadership and the community. I sat in on a few calls and it was clear that the project could benefit from some structure. So, we began building a plan…

Project Planning

The first thing we did was to define a framework for the project. The vision of the project was:

“To save lives by accelerating the identification and deployment of proven innovative digital solutions created by the global community of academics, scientists, technologists and entrepreneurs.”

From here, we defined a set of project principles and success factors:

1. Focus on nation level solutions with shareable data/evidence
2. Be additive to current efforts (there were a lot of clearinghouse like initiatives underway at the time)
3. Align to WHO priorities and most urgent needs
4. Scale with additional partners
5. Prioritize what’s working/worked in the past
6. Ship in days not weeks/months
7. Leverage existing WHO community to populate content
8. Be prepared to support broad set of users with different backgrounds

Key Processes

Most of us on the team weren’t familiar with the WHO project planning process and there wasn’t time to learn it. So, we came up with an Agile inspired process that included Daily Standups, Requirements/ Sprint Planning, and User Research. Fortunately, everyone on the team had access to the Internet which meant we could easily collaborate with hosted tools. Here’s a breakdown of the key tools and processes we used during the project:

Daily Standups: Google Docs and Zoom

Our Daily Standups were scheduled everyday from 8:00–8:30 am PST (including weekends!) The structure was as follows:

1. Key accomplishments completed yesterday
2. Key deliverables for the upcoming day
3. Blockers and Issues
4. Decisions — We documented all the decisions that we made during each standup. This proved to be really helpful in keeping things moving forward.

While I led most of these meetings, every team member was invited to contribute their thoughts and feedback to each topic. Given the diverse cultural backgrounds, I made an extra effort to be inclusive and would encourage people in private messaging in Zoom to speak up or ask their opinion. The dynamics of the meeting changed from me talking to very respectful discussions from each team member.

Requirements and Sprint Planning: Google Sheets

We simply didn’t have time to set up a Jira instance or deploy Atlassian tools as things were moving quickly. Also, not everyone was familiar with the terminology or processes. So, we decided to use Google Docs and Sheets as almost everyone had experience using them. We specifically used Google Sheets to manage all the requirements and bugs that came in. Note: each requirement was captured from Ministers of Health interviews and emails.

Screenshot of some of the requirements we defined for the MVP

User Research: Google Docs and Smartsheet

This process was the most critical in building the foundation for the project. We scheduled as many interviews as we could targeting LMICs in Africa, Middle East, and Southeast Asia. It was important to have a diverse group of participants as we wanted the clearinghouse to be a general purpose solution. Each interview was 45 mins. We started by asking each participant the following questions:

1. What is your role in dealing with your country’s COVID-19 outbreak?
2. Can you tell us about your day to day work in responding to the outbreak?
3. What’s the roughest part of your day?
4. What’s working really well?
5. What are some of your key workflows in getting access to digital resources to combat COVID-19
6. Where would you deploy resources and funding today?

As with any user research session, each one was different and more times than not we strayed into other topics. But, all sessions closed with getting feedback on a mockup that Erin pulled together:

Clearinghouse Mockups

This mockup was instrumental in getting detailed feedback that we could incorporate into the MVP. We walked each participant step by step through the mockups and got gut responses and feedback ranging from data privacy concerns to UI. We received unanimous feedback that such a thing would be useful for Ministers of Health.

Lessons Learned

After we deployed the MVP, we handed the project off to WHO to move to production. This gave us an opportunity to take a pause and collect feedback from the team members and some of our stakeholders on the project. So, Erin and I did a number of post mortem interviews to capture this feedback.

What worked well

1. Creating internal alignment: The team felt that it was very helpful to have a shared set of goals and a vision for the project. We regularly referred to the principles to inform us when making real time decisions.
2. Moving with urgency: Our bias for action helped us establish a team culture of learning fast and iterating on deliverables. This helped us unblock key issues as they arose.
3. User research: Given the broad experiences and capabilities of the team, it would have been easier to speculate and assert opinions instead of user requirements. Spending a bit of time up front to conduct the user research interviews really helped establish a North Star for the project.

What could be improved

1. Establishing measurable goals: We were challenged in defining a current state/ future state set of measurable benchmarks for the project. Things were moving fast and because we were building a new capability, there wasn’t a lot of data to define performance criteria for the project. But, in hindsight, we could have taken a crack at defining some quantitative goals around usability and solution deployments.
2. Getting official approvals from executive leadership: As we were working during a crisis, it was challenging to get time with internal executive leadership and partners. While emails were sent and reviews were done, we didn’t seek formal approval on project deliverables. This led to a few instances where we stalled on progress.
3. Setting expectations on the MVP: Despite using a proven tool and infrastructure, some external stakeholders were surprised by the minimal user interface and lack of integration into WHO’s infrastructure. While these requirements weren’t in scope of the MVP, it did lead to some concerns on scalability.

Closing

Working on the Digital Clearinghouse with WHO was a privilege and it was great to work with passionate people 100% focused on helping LMICs respond to the COVID-19 crisis. It’s our hope that this document will help others who are currently working on the crisis define a lightweight project management model across geographies and timezones.