How to run an Interview like an Amazon Bar Raiser

The Interview is the front-door to your organization. How do you greet someone at the front door? How do you make that experience memorable? And how do you know if they should move into the house?

I was a pretty awful interviewer when I started out. If I interviewed you any time from 1997 to 2005-ish my sincere apologies. I had no clue what I was doing. It took many years and exposure to one of the best interview processes in the industry (Amazon) to really understand how to think about an interview in a structured and rational way.

I served as a Bar Raiser (BR) and a site lead for the BR program at Amazon for about 6 years. Membership in the BR community is tightly managed due to the outsized influence bar raisers have on the interview process and hiring overall. Only 2% of the engineering group ever makes it to become bar raisers and membership in that community is held in high esteem throughout AMZN.

I interviewed over a thousand people at AMZN at every level and hired many of them for my teams and even teams outside engineering e.g. legal and product management. Through interviewing I learned interpersonal skills, judgment, communication, driving decisions through data, dealing with ambiguity, and managing risk. I don’t think there’s a single activity outside of coding and systems design that’s had as big an impact on my career as a result.

Let me start with two extreme anti-patterns that I have seen over the years: 

Scenario #1: “The Best Friend”: In this scenario, an interview is conducted without any clear objective in mind. It’s a freestyle discussion. The interviewer goes in unprepared and wings it. They have no clear competency that they’re evaluating for and the thinking is “I’m a pretty good judge of character, I’ll just get to know this person, and that’ll be good enough”. The result? The interview generally runs smoothly and everyone’s happy. During the debrief the interviewer uses phrases like “We really hit it off. I feel like the candidate will fit in well here. We could’ve talked for hours”. They made a new best friend. Unfortunately, there’s no actual data gathered. The candidate is left thinking they nailed it but in reality, they never got to display their actual skills or experience level. The interviewer thinks “this person will fit right in” but the truth is they have no data and are rolling the dice. This was a like-ability test at best and at worst…waste of time. 

Scenario #2: “The Gotcha”: In this scenario, the interviewer sets out to test the limits of the candidate’s knowledge. The interviewer is well prepared with some interesting problems in the hopes of seeing if the candidate can answer them. "How many Jelly Beans are in the United States?" and so forth. The goal of this interview is to find the limits of the candidate’s knowledge by probing until those limits are exceeded. Subconsciously the interviewer is looking to impress the candidate with how smart they are. The interview is a series of gotchas looking to stump the candidate. Maybe a set of hints are given to see if they can figure it out in the moment. This results in a terrible experience for the candidate. Even if the interviewer thinks they meet the hiring bar, there’s a high likelihood the candidate with decline any offer made after having suffered through such a judgmental experience. Who wants to work at a company that’s trying to shame you from the outset? 

These are two clear extremes with many shades of grey in between. Where do you fall on the spectrum? 

“Interviewing is something managers do, I’m a developer, it’s not for me”. Not true. Managers and Individual Contributors alike should actively participate in the process. If you’ve never conducted an interview before maybe it’s time to ask why not? Like most things career-related you can be a bystander to this process or you can talk to your manager in your next 1x1. Asking how you can help build the team is a great conversation starter and shows your manager that you're thinking beyond your daily sprint tasks.

There’s a lot that goes into team building but it starts with recruiting people and bringing them into the organization. How the interview is constructed and run is one of the least consistent things I’ve seen across companies. That makes it a prime target for you to exert your influence. If you optimize the process towards something that works for you and your team you can have a big impact on your organization. 

Where do I start? Start by getting to know the process. Get introduced to your Recruiting partners. They need connection to the development teams they recruit for and you can be an ally. Ask how you can participate in the interview process either by conducting a phone screen, being a lunch buddy, interviewing candidates, or facilitating a debrief. The candidates you talk to may end up being your colleagues who you depend on why not try to get involved if you can? 

Goals of an interview

Let’s first agree on what the interview is for. This seems like such an obvious question but like all obvious things…everyone has a different view. Here’s mine: The interview is a data-gathering exercise. It’s a crude tool but it’s what we have.

The data you collect is intended to inform if a candidate can perform in a role. Is the person described on their resume capable of doing the job at hand? Have they shown they can take their knowledge and apply it in the real world? How much experience, skill, talent, curiosity, and energy are they bringing to the table? Have they been successful in a similar situation? Did they level up their team or drag it down?

Things you are NOT gathering data on: Are they the same age as the rest of the team? Do they like the same music you do? Do they hold the same political views? Do they live close to the office? Do they have children? Do you agree on the best restaurant in town? Etc. these are all inappropriate topics for an interview setting and if the conversation goes there you should switch gears and bring it back to the competencies you've been assigned.

How I structure an interview

Generally, interviews span multiple days so here's the structure I follow;

Days BEFORE the Interview

• A recruiter or hiring manager will send out a schedule to everyone participating in the interview panel along with the candidates’ resume and any key items of note. This could be an email but often this happens in a quick meeting called a "Pre-brief". At this point, you should be assigned 1 or 2 topics to explore during your interview. These are known as “Competencies” and your job is to gather specific examples of when the candidate demonstrated these competencies in their work history.

• Don’t deviate or ignore your competencies. By focusing your interview on the competencies you’ve been assigned you avoid the risk of asking the candidate the same questions as someone else in the interview loop. Make sure your questions are directed and deep enough to obtain the concrete data you’ll need to make a decision. Stick to your competencies!

• As part of my preparation, I’ll crack open a fresh text file and write down 2 or 3 questions I intend to ask based on the competencies I’ve been assigned. I save this file using the candidates’ full name so I can easily grab it later (⌘-space is your friend). If you've never interviewed for that competency before chances are your company will have an Interview Question bank you can draw from. Use it! If you don’t have an interview question bank that’s a great opportunity for you; why not make one and share it with your team?

10mins BEFORE the Interview

• I context switch. I’m about to represent the company. I don’t want to bring whatever is going on in my day into the interview. I try to make sure I’m projecting positive energy and that I’m selling the company right from the outset. If I’m having a stressful day or there’s an interesting problem I’m wrestling with I put it on the shelf.

• To get in the right headspace I refresh my memory on the candidate, open up their resume so I can refer to it (having already read it days earlier) and I crack open that text file I made earlier. I’m ready to take notes.

The Interview itself

Here’s a loose structure to get you started. I’ve found this works for me but you should tailor it to suit your particular style.

The Intro

• First 2 mins - Settle the candidate down. How are they doing? Who have they met with already? I ask this question even though I know the answer. It tells me if they’re taking notes, paying attention through the process, or if they’re completely disengaged.

• Next 5 minutes - I almost always say the following: “I want to start by providing you an overview of the opportunity and our company, but I don’t want to tell you things you already know; tell me what you’ve learned so far and I’ll try to fill in the blanks.”

  This prompt does many things all at once:

  • It settles everyone down. It’s a question without being a question. The candidate is already nervous and primed to be peppered with questions. This tells them they can relax for a few minutes. We’re just talking here. I’m going to be giving them information in a minute. No need to stress.

  • It gives me data: If they can recite what they’ve learned so far, that shows me they are paying attention, care about the interview, and are excited about the job. Maybe they’ve even done some research on their own. That’s great too. If they can’t provide any information, e.g. “I really don’t know all that much” this absence of information is data for me.

• Lastly, I provide some color on the position, the team, and the organization as appropriate. This is a nicely timed breather for them. The interview is up and running and we’re ready to switch to our Competencies.  

The Body of the Interview

This is where I spend most of my time. I’m focused on getting data that demonstrates how the candidate embodied the Competencies I’ve been assigned. You can employ multiple techniques here, but I recommend the S.T.A.R. method.

In a S.T.A.R. style interview, the candidate describes the “Situation” that existed and what “Task” they were assigned. They explain the “Action” they took and the Result it accomplished. Most candidates are so flustered or nervous that they don’t follow this exact structure but you can help them. In many cases, it’s necessary to probe and ask clarifying questions to be sure you’re capturing the right data. E.g. “what did YOU specifically do” or “what was YOUR role”. If they’re speaking in the abstract I just ask; “Can you give me a precise example?”

I take notes throughout the conversation but I pay attention to the candidate. I'm not buried in my laptop. I capture the bare minimum shorthand so I can remember what was discussed and document/expand on it later.

Selling the Opportunity: Good engineers don’t just want a job, they want to join a team that’s going to benefit their career long-term. The interview is a two-way street. It’s the primary data point for candidates evaluating the organization. I have turned down great jobs because I thought the interviewer asked me stupid questions or showed up unprepared. I want to work on a team that pushes me to raise my game, not where people can skate by with little effort. I assume candidates want that too and if they don’t well…I don’t want to work with them.

A smart candidate will be evaluating you and the organization throughout so you need to do everything I’ve described above whilst simultaneously providing a great experience to the candidate.

The Wrap Up

With about 5-8 minutes left I end with something like; “Thank you, this was great. I got what I needed, but we have a few minutes left. What questions can I answer for you?”

Can you guess what I’m doing by now? Yes, I’m extracting data…again. 

“judge a man by his questions rather than his answers.” - Voltaire.

If the candidate has no questions for me that’s a data point. If they have prepared questions ahead of time that’s a data point. If their questions are researched and insightful e.g. “I’ve noticed in your app that XYZ, have you considered ABC?” versus easily googled e.g. “where is the office located?” that’s a data point.

After the Interview

The session is over but I’m not done. I try to immediately clean up my notes whilst the conversation is fresh in my memory. With my raw notes cleaned up, I do a second pass and extract Positives and Deltas. I don’t like to use the term “Cons” or “Negatives” when referring to a person. I only include a Pro or Delta where I captured concrete data to support that claim. I make sure to avoid using statements that start with “I feel like…” or “I think this candidate would…”. These are red flags to me that I’m dealing in conjecture and hypotheticals. I try to stick strictly to data in my notes.

Finally, I cast my vote; am I "inclined” or “not inclined”? Did they meet expectations based on the information and the competencies I was looking for? At Amazon, we had a question “do they raise the bar”? Said differently it means “based on everyone who’s already an employee at this level, is the candidate better than 50% of the people in this competency?” As you can probably guess this was highly subjective and led to many heated debates in the debrief process. Since leaving Amazon I’ve tended not to think in these terms anymore.

The Debrief

The Debrief is when everyone that has met with the candidate, including the recruiting team and the hiring manager, gets together to discuss the interview and determine a hiring decision. The debrief might not happen for days after the interview so I make sure to complete my notes and vote as soon as possible.

During the debrief I just refer to my notes. I'm not a fan of "voiceovers" in general, I have something to say I put it down in the interview notes and I only do that if I have data to back it up.

In Closing

This process isn’t perfect and I’m always trying to improve it. The goal is to capture data and let it inform your decision making. To remove biases and reduce the entropy around the exercise. How does this compare to your process? Does it change how you're approaching it? Leave a comment with what you've seen work.

I hope this helps structure your thinking. I genuinely want people to get the job and I look for reasons to say "yes". Think about the process your company runs. Think about your role in it. Would you hire you? And if you got the offer…would you accept it?