Learning with cities: prototyping and testing with communities

From February 2022 to December 2022, the Centre for Public Impact (CPI) partnered with the Bloomberg Center for Public Innovation at Johns Hopkins University and the Bloomberg Harvard City Leadership Initiative - Innovation Track to deliver innovation training to cities globally.

Very rarely do designers and innovators arrive at their most dynamic ideas in their first attempt. Instead, they create, test, and iterate until they arrive at a final product that successfully meets their users’ needs. Similarly, in the government innovation space, it is quite unlikely that city leaders are able to comprehensively solve the very complex and sticky problems that persist in their communities in their first attempts. This is where prototyping and testing can be useful tools to ensure city leaders are extending their support and resources only to ideas that are proven to drive impact. 

• Prototypes are physical models built to represent an idea, program, or policy – this can be as simple as a sketch on a piece of paper or as complex as a highly detailed mockup or simulation of the real-world version. Prototypes help us gain feedback early, derisk ideas through intelligent failure, and co-design ideas in partnership with communities impacted most.   

• Testing refers to the process by which we gather real-time feedback based on our prototypes. This design strategy is effective because it allows us to gather feedback on our assumptions about what people may find most valuable and useful ahead of fully implementing ideas. Testing can happen at various stages of the design process, whether it occurs earlier with a low-fidelity prototype, such as a sketch or wireframe, or later on in the process with a high-fidelity prototype that is interactive and much more comprehensive. 

While there are a myriad of useful and effective best practices in prototyping and testing, we’ve observed several key lessons that we recommend cities consider adding to their toolbox as they prototype and test.

1. Invest in cross-sectoral collaboration

2. Maintain a growth mindset 

3. Partner with residents for input earlier rather than later

Invest in cross-sectoral collaboration

Cross-sectoral collaboration happens when “alliances of individuals and organizations from the nonprofit, government, philanthropic, and business sectors use their diverse perspectives and resources to jointly solve a societal problem and achieve a shared goal.” 

When cross-sectoral collaboration occurs, greater brain power, resources, and perspectives lead to more creative and powerful solutions 

While the 2020 COVID-19 pandemic influenced many city leaders to step outside of their normal ways of working and collaborate with non-traditional city partners, a prevailing pitfall in city government is the belief that they alone have to solve their cities’ most complex and difficult problems. We’ve observed that cross-sectoral collaboration has the power to substantially accelerate cities’ ability to innovate by closing existing knowledge and resource gaps that ultimately result in more successful outputs for residents. 

Santo Domingo, Dominican Republic

Santo Domingo’s core team is addressing the accumulation of solid waste in public spaces in their avenues, parks, and sidewalks in the Guandules sector of the city. The team is motivated to find solutions because, if unaddressed, waste can result in negative public health outcomes for their residents. 

Over the course of the prototyping and testing sprint, Santo Domingo’s core team found great success in collaborating with non-traditional community partners. For example, the team developed partnerships with a private waste collection organization and informal waste collectors. The team pursued a partnership with several informal waste collectors after learning how much more effective they were at collecting residents’ trash on the narrow roads that traditional collection trucks could not pass through. Additionally, the team plans to work with public schools to foster the collection of plastic waste by students in exchange for community projects to be implemented by City Hall. Looking beyond the testing sprint, the team will collaborate closely with non-traditional city partners to support piloting and, eventually, implementation. 

Maceió, Brazil

Maceió began their innovation journey with a broad vision of improving the quality and life outcomes of residents in the under-resourced neighborhood of Vergel. Through their community research, the team narrowed their scope to focus on increasing prosperity and health for children and their guardians, especially mothers. To address such a powerful challenge, they designed one equally powerful idea: to develop a network of parents and caretakers that coordinate childcare efforts in an environment where there aren’t resources to simply build new childcare centers. 

From their many storyboarding prototypes, the team learned that the key people to participate in the network may not necessarily be parents and guardians, but rather people who have the experience and desire to lead learning and fun activities for kids, such as teachers and professional caretakers. They also recognized that neither the city nor the residents can do this alone. Rather, this shared daycare network required the resources and expertise of various parties, including community organizations, city hall, schools, and families. Equipped with insightful feedback about the network’s features and a more comprehensive understanding of the multi-agency collaboration needed, the team will now turn their attention to developing a plan for how to mobilize the resources necessary to pilot the shared daycare network next year. 

Applying a growth mindset can result in greater learning

Prototyping and testing are useful tools because they help cities identify what works and what doesn’t in real-time ahead of spending limited resources. However, in order to make the most of prototyping and testing, it is imperative to have a growth mindset. 

Possessing a growth mindset means you believe “your talents can be developed through hard work, good strategies, and input from others.” By possessing a growth mindset, cities are most likely to remain nimble and responsive to resident feedback to ultimately deliver outcomes that are tailored to their residents’ needs. 

How we’ve seen cities apply this principle in action…

Charleston, South Carolina

Charleston is focusing on the longstanding issue of flooding and climate resilience. During the research phase, the team learned that while long-term resilience will ultimately need to address inequitable living conditions, the most urgent need is for residents to have equitable access to information and resources to prepare for regular flooding events. The team initially wanted to test exciting new initiatives focused on building long-term resilience but pivoted to address existing inequities in the City’s response to flooding, recognizing the need to build trust with those that have been historically underserved and feel they’ve been left behind.   

At each step of the testing process, the team prioritized listening to residents in order to both gain important feedback on the prototypes and to build trust with the community. For example, one of the team’s ideas was to build out a text notification application to warn residents about flooding and share about available resources in the area. To test this idea, the team mocked up 5 different 'modes of communication' (text, email, voice mail, app) and presented these to a group of residents to understand which mode best meets their needs. The team’s second idea focuses on developing an internal process to equitably provide flood preparedness materials like sandbags to residents in advance of a storm and was heavily informed by resident experiences in the aftermath of Hurricane Ian, which made landfall during the testing sprint. By possessing a growth mindset, the team was able to take the feedback they received and apply it to avoid investing in materials that wouldn’t have met the most significant and urgent needs of residents. The team is excited to continue using this new way of working in future initiatives.

Corpus Christi, Texas 

Corpus Christi is working to address barriers for unsheltered residents, specifically documentation barriers. For many unhoused residents, documentation and identification is required to receive services in the city. The team became interested in creating a low-barrier ID after hearing from unsheltered residents and service providers who shared that housing insecurity makes it significantly more difficult for individuals to keep track of sensitive documents such as birth certificates and social security cards that are often needed to access care. 

During their first round of testing at a local public library, the team received extensive feedback. A few of these suggestions included attaching cards to lanyards to make them easier to keep track of, adding medical and emergency contacts information, and DD214 information for unsheltered veterans. Instead of becoming discouraged by the quantity of feedback, the team focused on how they could improve and adapt, resulting in a stronger second prototype. As a result, the team will pilot this initiative in early 2023 with the support of the library and their community partners. 

Seek resident feedback early and often 

Governments often operate within a “closed-by-default environment”, in which departments minimize transparency to avoid criticism and blowback.” Typically, governments will opt to bring the community in at the end of the process, hoping to garner their support. However, many resident leaders have a desire to be a part of the full decision-making process rather than being isolated to a gut check once decisions have been made. 

Riverside, California

Riverside is hoping to attract and create green businesses in the city. During their testing and prototyping sprint, the team was intentional in getting feedback early and then pivoting their concepts to be as resident-focused as possible. In particular, they did a very early-stage prototype of their GREEN Riverside idea, creating a collective definition of what GREEN means to the city, prioritizing potential organization-wide environmental goals, and defining long-term sustainability goals. 

Before the testing sprint even began, they tested their potential seven-pronged definition and priority areas with environmental policy students and their advisor, learning from the already established local sustainability realm. From an early stage, the team realized that their definition was too all-encompassing and they needed to simplify and clarify exact goals and outputs. In their second round of prototyping, they tested a narrowed three-element green definition against the seven-pronged definition with 22 residents, including business owners/workers, and academic staff/students. While the team plans to conduct additional rounds of testing to further solidify this idea, capturing early feedback from those knowledgeable about the topic helped make the latter rounds of prototyping more fruitful.

Kalamazoo, Michigan

Kalamazoo is working to address sidewalk snow clearance in the city. The team began their testing and prototyping sprint with a focus on exploring two ideas: a 1:1 volunteer match program where neighbors could help one another clear sidewalk snow and opportunities for the city to take a more active role in clearing snow from sidewalks. In their first prototypes, the team learned that a volunteer-led program would not have sufficient personnel for large-scale snow clearing and other methods were needed to supplement the need. The team used both positive and constructive resident feedback and pivoted to “take a more incremental approach, realizing we don’t need to do it all right away. We can build over time and figure out the mechanism”.

With that spirit, the team divided their volunteer idea into a large-scale approach - changing ordinances to get the city to take a bigger role in clearing sidewalks - and a local approach - a volunteer program to help individuals who need help most. To further explore how the city could play a greater role in clearing sidewalks, particularly those routes that are most heavily used by our most vulnerable populations, the team will use an existing citywide survey to add questions about snow-clearing satisfaction and gauge interest in the city taking more responsibility. The evolution of these ideas throughout the testing sprint highlight the importance of having resident feedback early on and the value of refining ideas to both meet resident needs and fit community realities. The Kalamazoo team set out to find solutions to clear more sidewalks in the winter but ultimately came to understand that this issue was part of a larger problem of not living up to its values of supporting all modes of travel. The solutions identified through the Bloomberg Innovation Training will help Kalamazoo embrace its Connected City vision, in partnership with its residents, in all four seasons of the year.

Dublin, Ireland

The city of Dublin is working to address how our increased reliance on private cars impacts neighborhood liveability – increasing congestion, making streets less safe and reducing feelings of belonging within the city’s neighborhoods. The team’s ideas focused on increasing resident ownership of public spaces and reclaiming the public space of residential streets as more than just traffic vectors in their city.

The team’s first idea – a neighborhood commons – explored how new engagement methods could build trust and community ownership of public spaces. Through a process of co-design, resident groups and local schoolchildren shared their vision for the future of a local public green, the development of which will be explored in 2023.

Prototyping the team’s second idea of play streets eventually led to the city’s first successful Play Street pilot. The team started their testing process with a low-fidelity prototype to identify what streets would be best for the play street. They asked 17 residents to participate in a “Play Street Criteria Bingo Game” where residents could point out features of different streets that would be conducive or unconducive to a safe play street. Feedback from this test allowed the team to find potential street candidates. Once they secured both resident and city approval, the team increased the fidelity of the idea into a pilot play street where they blocked off the selected street for three hours and hosted games and activities for 40 children in the community.  With great feedback, the team is now exploring the best legal framework to allow parents to set up play streets in their neighborhoods, ensuring the sustainability of the play streets idea and giving ownership of these spaces to residents.

Portsmouth, Virginia 

Portsmouth is working to address growing youth violence amongst its middle school youth. Over the course of the testing and prototyping sprint, the team leveraged their already existing community relationships to gain insight from key community partners before testing with residents to inform their design choices. 

For the team’s “Create Your Future” Youth skills training programming, the team met first with leaders from a local mentoring organization, Big H.O.M.I.E.S (Heroes of Minority in Every Society), to discuss which program offerings have most resonated with youth. They were then able to bring a revised selection of program offerings to 6th graders, allowing for a higher fidelity first prototype. 

Similarly, while developing their initial wireframes for a community connection app, the “Pulse of Portsmouth,” the team first socialized the idea with their city’s IT team and with community youth non-profit partners who lead programming initiatives. This helped them gain insight into what features would be useful to include when testing with the community. The team found great success in collaborating with residents early on to help shape their initial prototypes. 

Birmingham, Alabama 

Birmingham’s core team has focused on improving and centralizing the business licensing and permitting process. While testing their Flip n’ Find idea – a physical "tool" that details the main phases of a business owner's licensing and permitting journey, – the team shared their initial prototype of a journey map with other city staff and external stakeholders knowledgeable about the process.

The first testing session with city staff resulted in the breaking down of silos across various departments, illuminated existing resources that city officials were unaware of, and identified the need for new services and information for accessing these services. Moreover, the team learned that many city staff members were business owners themselves and would benefit from additional resources. Their early feedback helped the team incorporate technical city knowledge and ultimate user feedback, enabling the team to run a higher fidelity test with more local business owners showcasing the Flip n’ Find tool as one of three prototypes, including a roadmap and a comprehensive checklist.

During the final testing session, small business owners voted on the prototype that would be most useful when navigating the complexities of the business license and permitting process. The layout of the  Flip n’ Find tool appealed to most, as it showcased complex information in a user-friendly format. The team plans on fine-tuning the Flip n’ Find model so that it can be incorporated as a tool used during the upcoming tax and license season.