Reducing bias in recruitment: what we’re learning

In September last year, the Centre for Public Impact co-created a Diversity, Equity and Inclusion (DEI) Policy. We did this because we recognise that “diversity enriches our commitment to a more just society, and enhances our ability to enact change…”

How does this policy influence how we recruit at CPI? As a first step, we began using the Applied platform - software designed to reduce biases in recruitment. Last year, we wrote about our experience of using the platform, some of the struggles we had, and the steps we took to rectify these issues.

In this blog, we offer another reflection, using our recent recruitment round in our Australia and New Zealand Aotearoa Office as a case study. We describe the process itself, as well as how it felt, and what we learnt, from the perspective of Thea Snow - the person recruiting - and David Murikumthara - the successful candidate.

Why do we want our recruitment journey to value diversity?

Thea: Diversity matters for so many reasons - far too many to set out in one blog post, let alone a short paragraph! But a key factor underpinning both the design and execution of this recruitment process was finding someone who would complement - but not mirror - my style and approach. I was looking for someone who would bring things to the role that I can’t - someone with different skills and, more importantly, shaped by a different life experience.

David: Diverse organisations build great things, but they also allow individuals to challenge themselves with exposure to different ways of thinking and working. I’ve enjoyed diverse teams in previous roles, but have still seen situations where different minorities have felt overlooked, or uncomfortable speaking up. I feel like you can tell a lot about an organisation through their recruitment process - and in the past, I’ve made decisions to accept or decline offers based on what I felt the recruitment process told me about an organisation’s culture.

Diverse organisations build great things, but they also allow individuals to challenge themselves with exposure to different ways of thinking and working.

Step One: Initial shortlisting of candidates

Thea: We had 79 people apply for this role, and wanted to shortlist ten. Using the Applied Platform, we did the initial filtering not by looking at people’s CVs, but rather by asking applicants to answer three questions (in 150 words):

1. What excites you about CPI’s work?

2. When are you at your best?

3. What can you bring to this role that you’d love us to know about?

We kept these questions simple because we know that even the framing of questions can dissuade people from applying. We also avoided asking questions about people’s past experiences - we felt that more general questions would open the role to fresh talent, including those from non-typical backgrounds.

Applicants’ responses were anonymised, and randomly allocated across a panel of four reviewers to review and rate out of five. You can read more about the sift process here.

We ended up with a diverse shortlist across gender, ethnicity and age, and were delighted by the quality of the answers, but were nevertheless left wondering:

1. Were the questions too broad, and word limit too tight, meaning we lost nuance and depth in answers?

2. Did the questions inadvertently create more rigidity than a cover letter would?

David: Cover letters sometimes feel formulaic and rigid - you’re often just ticking boxes, and I’ve certainly been guilty of only slightly tailoring cover letters when applying for jobs.

It was certainly refreshing to not be bound by the structure of a cover letter, but with such open-ended questions, I wondered if a more detailed job description would help ground candidates’ responses, acting as a guardrail so that people felt confident they were addressing CPI’s criteria. It was also hard not to refer to prior experience in my answers, and to stay under the word limit. 

Step Two: Written Exercise

Thea: People have different strengths: some people write well; some are excellent presenters; others prefer to draw, or map their ideas out conceptually. We were conscious of wanting to create a recruitment process which gave people with different skills a chance to shine. 

With this in mind, we designed an exercise, which we invited the top ten candidates to complete. The exercise began as follows: 

“You open your inbox on Monday morning, and you have three email requests - all requiring a response by Wednesday.” It then described the three requests, noting that, “because of everything you have on your plate, you only have time to respond to one.” 

Applicants were asked to: 

1. Explain which request they would respond to, and why; and

2. Develop a 3-page proposal/concept note. 

We asked them to spend no more than four hours on the exercise and encouraged them to be as creative as they liked.

Interestingly, only seven out of the ten candidates submitted exercises. Two formally withdrew, and one just didn’t submit.

From my perspective, the exercise was a fantastic way to see how candidates made decisions around prioritisation, and also allowed applicants’ creativity to shine. 

I reviewed the exercises together with CPI’s Executive Director, Adrian Brown. We were both really impressed by the quality and variety in responses we received.

David: I really enjoyed this exercise. It gave me a taste of the work I’d be doing with CPI, and confirmed my interest in CPI’s work. Whereas the short answer questions felt a little rigid due to the word count, I felt I could be more creative in this exercise.

Consulting taught and nurtured in me a very distinct, sometimes rigid style of thinking. The latitude offered to me in this task was refreshing. There were many ways to approach the task - I could play to my strengths and think more creatively.

I can imagine some candidates struggling to juggle this task with work, family and personal commitments. It was hard for me to dedicate a lot of time to the exercise - work was quite busy in my prior role. I found the time guidelines (3-4 hours for the task) to be instructive, and it helped me timebox myself. We also had around a week to finish the task (including the weekend), which I felt was reasonable.

Step Three: Interviews

Thea: Based on the exercises, we shortlisted four candidates to interview. It was only at that stage that we looked at CVs.

I wanted an interview process that felt relaxed - more like a conversation than an interrogation! For this reason, I decided to interview with just one other person - Carmella Grace De Guzman. I felt Carmella would be a great interview buddy for a few reasons - she’s a lot younger than me, has a different formal skillset, and also has personal experiences with racism, which I don’t.

Interviews ran really smoothly. Afterwards, Carmella and I disagreed about elements of certain candidates, which created useful constructive friction.

Nevertheless, it felt difficult to avoid unconscious biases in interviews - there are inevitably people you warm to more easily than others. I was left wondering, what does a really great and rigorous interview look like? How do you debias this part of the process? I feel like we’ve got more work to do here…

David: There are a number of great resources on interviewer bias, and taking bias out of interviews. It appeared to me that (despite Thea’s reservations!) CPI was doing a lot of things right - such as having a structured interview format to level the playing field, and gaps in between interview slots to prevent the contrast effect. 

I went through two rounds of interviews - one with Thea and Carmella, and another with Larry Kamener, CPI’s founder and chair. I was a little nervous, but a few different things helped me put my best foot forward in the interviews.

Even where I’ve been successful in job interviews, the interview process could feel transactional. In my interview with Carmella and Thea, Carmella started the interview by asking how I was - but positioned the question in such a way as to elicit an honest, considered response. It was useful beginning the proceedings with some introspection and vulnerability - this allowed me to delve into the very personal reasons why I wanted to join CPI later in the interview. Similarly, small talk with Larry about an aspect on my resume helped me feel more comfortable as we got into deeper conversation.

These starting points in both interviews were important to me, and together with ample time for questions, I felt like I really got a good sense of the values and culture behind CPI. 

What we learnt from the process

Thea: I think the three-part process worked really well, but I’m probably biased (!). So, what did candidates think? Turns out, most people enjoyed it. 54 candidates offered feedback on the process, and the mean satisfaction score was 8.7 out of 10. A lovely piece of feedback we received was, 

"I was so pleased with the humanity of your process!"

However, we did also get some constructive criticism.

One applicant noted that 150 was too few words to respond to sift questions. They explained,

“...it is hard to elaborate on experience, skills and thoughts in relation to the position… another 50-100 words would have made a big difference from my perspective.”

Another suggested that the framing of question two was too broad - 

“I felt that question 2, "When are you at your best?" was a little vague. I believe that if there was some kind of clarification I could have been more specific with my answers.”

We will certainly take this feedback on board for future processes. Right now, though, I’m just delighted that it resulted in the recruitment of David - I’m excited to see what we can do together!

I was left wondering, what does a really great and rigorous interview look like? How do you debias this part of the process? I feel like we’ve got more work to do here…

David: My interviews with Thea, Carmella and Larry helped me to understand what CPI’s vision was, and how its values (including diversity) would enable this vision.

Moving forward, I think it’s important to codify the above processes - ensuring everyone interacting with CPI recruitment has a positive experience, free from bias.

For some of you, the processes we have set out may feel over-engineered (and it certainly did seem like a lot of work for Thea and the recruitment team!). However, I had a really great experience because of this process: I felt comfortable sharing personal stories about my interest in government; I could really highlight my strengths, but equally felt comfortable highlighting development areas; and I felt like CPI was actively trying to build and foster diversity. 

Suffice to say, I’m very excited for the year ahead with Thea and CPI!

***

Refining our recruitment processes is an ongoing conversation at CPI, and we’d love to hear your feedback.