Avoid biases in expert interviews. Pieter van Eck (ex-ATK)

Expert interviews are a great source of insights for due diligence and market assessment cases. They give you multiple individual perspectives on an industry, which you combine to superior market insights.

You must however be cautious not to introduce your personal biases in the process, and thereby adversely influencing your findings. During your case, there are four stages in which biases can creep in:

1. Scoping the project/market definition

2. Selecting experts

3. Structuring the interview

4. Summarizing the findings

By carefully planning each stage and using best practices, you can largely avoid biasing your findings.

Scoping the project/market definition

Sending out the request for experts is one of the first things that you do when starting a consulting project. You typically do not have a detailed view of the project at this stage, so you should allow for a wide industry definition. Share multiple examples of relevant companies and experts’ titles to help guide the expert networks (and make clear whether they should look for other similar companies or only the ones provided). If you do not correctly scope the project for the expert networks, you might end up with unequipped experts who will give you insufficient knowledge and prejudices instead of clear data points and well-founded expert advice.

Selecting experts

The project proceeds and you receive your first experts. Doing multiple expert interviews helps you triangulate the data points. Therefore, make sure to schedule calls with many experts relevant to each of your key questions. Your key questions obviously vary greatly between each case. They could relate to customers’ key buying factors, or sales persons’ thoughts on future sales channel strategies. You should also ensure to add a couple of unconventional interviewees: maybe former employees at an organization ensuring industry standards, regulatory bodies or even industry associations. This way, you are sure to get different perspectives and a holistic overview of the industry.

Structuring the interview

Be aware of the situation and tone of your conversation. You will typically be under time pressure, but you must not fall into the temptation of asking leading questions. You might otherwise push the interviewee to merely confirm your assumptions.

A good solution is to clearly state your objectives with the interview at the start, without presenting any of your data points or assumptions. Expert networks typically don’t charge if you cancel the call within 10 minutes – take that time to assess whether the expert really is knowledgeable! However, make sure that you stay humble and polite during the whole process. A good introduction for a qualified expert could be: “I want to understand the market for X, including its size, development trends as well as the characteristics and behavior of different suppliers and customers within the market. I want to hear your thoughts on this for the first half hour, and then we can spend the remaining time on some specific questions that I have prepared”. Allow the expert to speak freely, but feel free to nudge lightly in case you get clearly off topic.

So for the first part, go semi-structured with open questions. Be attentive to nuances or descriptions that contradict your assumptions and hypotheses and ask follow-ups accordingly. For the second part, run through all the shorter quantitative questions that are needed for your modelling.

After the first few interviews, you can revise your hypotheses, refine your quantitative part, and let it take a larger part of the interview hour.

Summarizing the findings

After you have completed your interviews you will frequently find that you have obtained some opposing views of some experts. In this case, it is crucial to separate the noise from the relevant data points. To form your own opinion, use some of the following procedures: First, check how many experts have stated the opposing view. Is it only one, or is it many? Second, check the profiles of the opposing experts. How many years of industry experience do they have? Who seems the more trustworthy and reliable expert to you? Third, if you are still undecided, call some of your top ranked experts from previous calls and ask them for their view on the issue at hand. This way you make sure that you will have an unbiased and differentiated view at the end of the case!