Context is powerful. If you’ve lined up an interview with someone because they match your participant definition, you’ve already made the decision that on some level, you know that person.
Whether it’s for research, journalism or a job interview, that pretext is the lens and construct through which you’ve chosen to understand and empathise (or not) with someone. Interview bias is that lens, and everyone has it.
But even without such pretext, or even with strong ability to put bias aside when starting an interview; a first impression will do most of the work on its own.
In truth, by seeing or speaking to someone you are already fighting an uphill battle against bias within seconds. Bias that you as a researcher can’t afford. Bias that as a journalist might diminish the truth in your article. Or bias that as a manager could cause an imbalance in the workplace.
In this article, we’re going to talk about interviewer bias in both its technical and non-technical forms. We’ll also discuss tactics for professional interviewerrs to circumvent some of the damage that their own bias could be causing.
What Is Interviewer Bias, Actually
Let’s first look at the technical description of interviewer bias. The Oxford Reference database (as operated by the Oxford University Press Association) defines interviewer bias as follows:
[Interviewer Bias] is a distortion of response related to the person questioning informants in research. The interviewer’s expectations or opinions may interfere with their objectivity or interviewees may react differently to their personality or social background. Both mistrust and over-rapport can affect outcomes. Interviewer bias – Oxford Reference Database
From a practical point of view, the definition references the potential for two types of interference with the interview process:
- Any number of actions and behaviours made by the interviewer that cause the interviewee to react and respond in ways that conform with the interviewer’s bias. Examples of this could be:
- using certain language, phrases or leading questions
- outwardly expressing one’s own beliefs or answers to questions
- using certain vocal tenor, or inflections to imply a presumed answer
- Pretexting questions with implicative phrases, such as “I’m sure you know this already but…”
- using non-neutral body language that establishes a mood or projects onto the conversation
- dressing or provoking intrigue through appearance or presentation
- using certain language, phrases or leading questions
- A prejudiced perspective on data from an interview or interviews. Examples of this could be:
- dismissing a person or data from a particular interview based on your personal impression of the interviewee
- Gender or racial bias are significant factors here
- not recognising, or dismissing, patterns and insights from interview data that fails to align with your point of view
- highlighting or finding implied patterns and insights in data that align with your point of view even if not explicitly in the data
- not reviewing interview data at all and instead relying on your memory or impressions from an interview
- dismissing a person or data from a particular interview based on your personal impression of the interviewee
Now within these practical states of interference, and therefore distortion of outcomes, we have implied a long list of potential bias types. I’ll talk more about different bias types later in the article, but the overarching construct of implied social bias is perhaps best defined by the term “halo effect”.
To continue to quote the Oxford Reference database…
Halo Effect refers to a common bias, in the impression people form of others, by which attributes are often generalised. Implicitly nice people, for example, are assumed to have all nice attributes. This can lead to misleading judgements: for example, clever people may falsely be assumed to be knowledgeable about everything.
Halo effect – Oxford Reference Database
As if we didn’t already know, the halo effect and the studies around it are confirmation that regardless of who you are, first impressions are an ever present, and ever powerful influence in our lives.
It’s impossible to really say how much time a first impression takes; with study findings ranging from 5-30 minutes for a job interview, to 1/10 of a second for a social interaction.
The potentially dangerous part of these discoveries is that we have no control over it. Humans have, in the general sense, failed to build a social mechanism that hinders our instinctual biases. And because humans consciously reflect on the past, in the way other animals do not, it takes much longer for us to reprogram a first impression.
So if we literally cannot help ourselves from having bias in an interview, what can we do?
How To Identify & Recognise Interviewer Bias In Yourself
The first step in identifying interviewer bias in yourself is accepting that you have interviewer bias at all.
To be able to work with your interviewer bias, you’ll need to fast track through the 5 stages of grief: through denial and anger, past depression and bargaining, and right to the end: Acceptance.
If you can accept that you yourself are agist, racist, gender biased, homo/hetero-phobic and have any other worldy combination of socially biased perspectives; then you have a good platform to build from.
The sad part of this world is that we all have these biases, we all use stereotypes and generalisations to try and understand, judge or empathise with people when we interact with them. But if we don’t accept that we might have these, either through ignorance or claiming to stand above them, we are most prone to suggestions and actions that are in the best case a mild bias, or in the worst case racist or bigoted.
As an interviewer, and particularly a researcher, your role is to ask questions and listen to your interviewees without judgement; something that you have to accept is nigh on impossible before you can learn to work with it.
How To Identify Interviewer Bias In Others
For this article, when we refer to identifying bias in “others”, we’re really talking about your colleagues.
Whether it’s other people who do interviews, editors and clients, people that listen and take notes, or even people responsible for transcribing or coding data during analysis.
In most research work environments, identifying bias in others can often be very difficult. In fact, you’d often hope it’s difficult, as overt bias early on will have other implications.
For example, if a research project is designed around, influenced by, and makes no effort to challenge the overt bias of a client, the results of that project will often only confirm what that client assumed to know anyway. The client may then disregard the work as a waste of time/money, and refuse to acknowledge research as an important part of their process. These situations are common in industry research; a kind of self-fulfilling prophecy for clients who feel this work is unnecessary.
But the bias of people involved in the project may not even appear until the very end of a project, wherein the only real outcome is what one person has been saying all along. If this has happened and the data to support the claim is strong, then while the work may have missed things, it at the very least confirmed an assumption. However, if the data to support the initial claim is weak, but remains the only real insight from the work, it’s likely the project was off the rails from the beginning.
In point of fact, as researchers we are often trying to suppress our own bias – partially, fully, overtly or not. The respect most people have for the scientific process means that they attempt not to judge too much. And this type of bias, this subtle influence of our assumptions is extremely difficult to see in colleagues until it’s too late; in fact, if you ever even see it at all.
So what can you do about it?
Working With The Bias Of Colleagues, Friends And Clients
There are a number of ways to insulate qualitative work from the influence of your colleagues.
A tactic that some researchers use, and is now a standard process for the research ops team at Yleos, is a simple extension of what you will be doing anyway as a researcher: you interview people.
Whether it’s your colleague with a client; a team in a company, or you with an overseeing professor, the first interview you design for a project is an interview for you and your colleagues to identify, notate and hold accountable your own biases and assumptions about the project.
It doesn’t take much, or need much, but even if you’re writing questions for yourself and getting a roommate to ask you them; the best way you can counter your own bias is to use your interviewing skills to weed them out.
At Yleos any biases and patterns we find in ourselves at the start of a project are recorded, noted, and openly discussed with everyone involved. We often then go back to our notes as a team at each stage of the project ,from script development to analysis and reporting.
Sometimes it’s as simple (or perhaps as difficult…) as saying: “We know this is a bias in the team. Is this question informing an assumption, or leading towards confirmation instead?”
Working With Your Accepted Biases During An Actual Interview
If you’ve accepted that you have biases, there are three acts of self awareness you can use to improve and monitor your interviewer bias.
1. Maintain a state of active self review.
While ‘reflective’ self review is, well, reflective, and therefore after-the-fact; ‘active’ self review is a state of constant consciousness and self evaluation.
Although this is arguably the most effective tactic for improving your biases in life, this is very difficult to do at all, much less halfway through an interview…
As an interviewer you’re juggling a lot, everything from the schedule, time and duration, to the comfort of the interviewee as well as any colleagues in the room, your script, the notes you are taking, etc etc, etc.
As part of that dance researchers tend to be, on average, quite self aware. But can you do more? Can you do better? Can you consciously analyse and control your own haptics, your voice inflection, your facial reactions, even the wording choice of your questions?
This is not in an attempt to change who you are, but to identify traits, words and phrases in yourself that may (intentionally or not) trigger specific reactions in your interviewee. The ability to do this well is both a power for good as well as evil as it’s the root to manipulation as much as it can be a root too empathy.
Regardless of your own abilities, there are limits to everyone’s ability to do and improve this set of skills, so it’s good to have a vehicle for this to work within, thus our second act…
2. Quite literally ‘play a role’.
Playing a role means to create a character or persona for your interview that you adopt and embody for its duration.
It is, in short, to be an actor on a very small and intimate stage, wherein your goal is not to make the audience laugh or cry, but instead to allow them to be entirely and completely who they are and experience how they feel about things without your interruption or interference.
Now, this character could be designed in your current self image, or reflect what you feel to be your best properties. But we find it works best when you can use the first act (active self review), to identify your biases and then act the part of someone (you) who has no such biases.
Now many of you may think, “I do that anyway!” And if so, that’s great! I would hope so.
I want to emphasise it here because in our evaluation of investigative conversations between mentors and subjects (ie. interviewers and interviewees) we’ve seen that making the conscious decision to play a role often means avoiding the compulsion to represent your own truth (due to ego or habit). It also means you don’t feel like you’re lying because you are playing a role for someone else’s benefit, not yours.
The trick in this work is not to lie about who you are, but to not let who you are prevent someone else from representing their truth. So making the conscious decision to play that role allows you to put aside your instincts and feelings and be fully present for your interviewee.
3. Add an additional layer to your analysis process wherein you start by redefining your hypothesis.
Unlike points one and two, the third is a very quick framework for researchers to use pre-analysis or even during one-off interview debriefs.
The concept is simple. Before starting the analysis of interview data, particularly the tagging and coding of raw audio data, take 30 minutes to do the following with anyone involved in the analysis process.
- Review your original hypothesis for the study or interview(s)
- Write down 1-3 assumptions on what you’ll find/discover about each hypothesis
- Write down what you think you absolutely won’t find about each hypothesis
- Write down a list of things you know you learned from interviews that you didn’t previously know about the topic
- Write down the number of things you hope to find in the data that would be new insights for you
- Formulate additional drafted hypotheses of things you don’t expect to find but answers to which could, maybe, possibly, be hidden in the data
Now I’m going to call myself out here…
Many researchers would argue that the above process could potentially lead to a close-minded view of the data. Instead of removing bias, perhaps this process enforces it?
In short, this checklist is a way of consolidating the risk of bias to a short process at the start of the analysis process instead of opening the entire analysis process up to the risk of bias throughout.
I’m sure a number of researchers will have comments and arguments about this concept, and I hope you’ll all contribute to the comments and help evaluate, test and adjust this process!
Comment below with your thoughts on this!
The Crux
“Am I creating bias in my own research?” is a question we get all the time at Yleos.
In fact, in a study we did with 60 researchers, at least 10% of researchers brought this up as a concern they had in their work, independent of the questions asked (which were not on this topic at all).
In fact, I personally believe that if you haven’t asked yourself this question, you may be missing entire areas of discovery in your work!
However, this concern we have for our own bias is something we should learn to address constructively. Researchers who experience this doubt and struggle to justify the results of their research need to be sure that their results were not primarily driven by their own ideals.
And that’s ultimately why we wrote this article. The point is not to sell a fixed ideal of bias-resistance research, but rather to present options that may help some of you overcome the barriers to achieving confidently unbiased outcomes.
Regardless of whether or not you agree with the suggestions we’ve made, I hope you’ll contribute to the comments and help us find the best way forward.
And make sure to signup for our email short course on conducting great interviews below!
Thanks for this article goes out to the entire Yleos team, our beta community, and the teams at Growth Mechanics and Silicon Rhino for their knowledge contributions.
Want to learn more about how to conduct great qualitative interviews?
Whether you’re new to qual or looking for a new perspective on techniques and tactics, we’ve got the stuff!
“Built from a combination of unique content and interviews with user researchers from around the world; the Yleos Newsletter is a unique perspective on the world of qualitative research.”
Jerry C., Research Professional