Using online research to build an evidence base

Business strategy must be evidence-based or it is just wishful thinking. (Tweet this)

As Jim Barksdale famously said: "If we have data, let’s look at data. If all we have are opinions, let’s go with mine."

But how exactly do you build an evidence base?

1. Get good at Google

You'd be amazed at how much quality data is available on the Internet. And Google is there to help you find (almost) all of it.

But using Google can be a bit like trying to drink from a fire hose. There is just too much information. And most of it is not very good. The bad and irrelevant information drowns out the useful information.

So you need to get good at using Google to sift through the dross to find those nuggets.

There are lots of good resources to help you use Google more effectively. It is definitely worth familiarising yourself with these. (Note: Not all search tips are equally useful for strategy. But I think that will be obvious enough.)

• 20 Google Search Tips to Use Google More Efficiently
• 23 Google Search Tips You'll Want to Learn - PCMag UK

Some of the more useful tips I have found are:

• Use quotes to specify precise words in a specific order. This can be helpful when searching for a specific quote, a proper name, or a specific report where you know the title.  
• Put a minus sign in front of words to exclude them. For example, if you're looking for smartphone technology other than the iPhone, you could search for "smartphone -iphone".
• Use the | bar to search for one word or another. For example, "Apple | Microsoft" will search for pages containing either or both terms.
• Put a tilde (~) sign in front of a word to look for synonyms. For example, "~classes" will search for classes or lessons or coaching, etc.
• Use "site:sitename.com" to search within a specific topic only.
• Use ".." to search between two numeric values. For example, "financial results 2015..2021".
• Use "location:placename" to search relative to a specific location.
• Use "filetype:" if you know what you are looking for is a specific filetype. For example, "filetype:pdf".
• Use the tabs. Click the "news" tab if you're looking for recent news stories. Click the "images" tab if you are looking for charts. This is particularly helpful as the better quality research sites will often use charts to represent their data and analysis.

Remember that most of the time you're trying to improve your search criteria to eliminate that which is not useful.

The other challenge can be to tap into the specific jargon which people use when writing about the industry you are interested in. If you've worked in that industry for a long time that's usually not a problem. But if you're an outsider or consultant, you should make a priority to master the language used.

It can be a bit hit and miss. Especially when you're starting with a new topic, line of enquiry or industry. So you have to keep trying it from different angles until you get it right.

2. Get good at scanning the results

No matter how good you get with Google, you're still going to need to process a lot of material to get the evidence you're looking for.

The first trick is to learn to recognise the reputable sources of information in the industry you're researching. Which are the quality edited journals? Which are the quality research companies? You can usually spot their URLs in the Google search results before you even click through to the page.

Then you look into the text itself. Which pages are spouting un-substantiated opinion? Which are written by lazy journalists trying to fill column inches or be the first to break a story without really understanding it? And which are providing high-quality, in-depth analysis backed by evidence and data?

"If we have data, let’s look at data. If all we have are opinions, let’s go with mine.”

- Jim Barksdale (tweet this)

Where there is data, is it of a high quality. Is it clearly described and defined? What is the sample size? Are the conclusions statistically significant?

What is the quality of the analysis? Is it clearly and logically reasoned? Does it make basic mistakes like confusing correlation and causality?

You have to kiss a lot of frogs to find a prince. When you're doing research you will have to scan and discard a lot of information. A lot of it is poor quality, repetitive, or not quite relevant to the topic you're researching. It takes time.

3. Get good at ChatGPT

The development of ChatGPT and other generative text models has been a huge boon for developing evidence bases. Think of ChatGPT as a combination of steps 1 and 2 above.

You can ask ChatGPT a question and it will provide an answer based on having scanned large parts of the internet for you. It uses a conversation interface. So if you need it to elaborate on or explain any parts of its answer, you can ask it to do that for you.

The secret to using ChatGPT is to phrase your questions (called prompts) well. The clearer you are about what you want to know, including the context in which you are asking and the level of detail you want, the more likely you are to get high-quality and useful answers. There are lots of guides on how to do this, but at the end of the day it boils down to practice and working out what works best for you.

There are alternatives to ChatGPT also, such as Google's Bard. You may want to ask both and compare the answers you get.

4. Focus on the contradictions and inconsistencies

Don't get lazy and only look for data which easily supports your views. This is known as confirmation bias. It is a well-researched phenomenon.

Look for as wide a range of evidence as possible.

Data sources which contradict each other or contradict the opinions of experienced people within your business are often the most interesting.

Assuming the data sources are credible, and the experience people are indeed knowledgeable, these contradictions often lead to the best insights.

Instead of assuming that one or the other must be "wrong", dig deeper. Look for some explanation under which both can be right.

Have you found an exception to a rule? Something which is true under one set of circumstances, but not under another? Have you uncovered an implicit assumption or bias in thinking? Have you found something that used to be true but no longer is? If so, what has changed?

Such seeming contradictions and inconsistencies provide the starting point for more considered analysis, insight, and sometimes strategic breakthroughs.

5. Make excellent notes

Once you've found the evidence you're looking for, it is important to keep good notes. You need to be able to recall what you've found easily. This could be because you've got a report to produce, or because you're in a meeting and someone is asking you to back up what you're saying.

You can use a general-purpose note-taking tool, like Evernote. But general-purpose tools don't know what your notes mean. Nor do they know how they should be organised. So often, you end up searching through your notes just like how you were searching for them through Google in the first place.

Alternatively, you can use a special-purpose tool like StratNavApp.com. StratNavApp.com is specifically designed for organising evidence for business strategy. It will organise your notes for you using familiar strategy analysis models. Evidence gathered in this way typically supports PESTEL analysis, Porter's 5 Forces analysis, and the Threats and Opportunities in a SWOT. StratNavApp.com will help you to connect the evidence you gathered directly to these analyses. It will also maintain the links back to the original sources of the information.

Not only will it make it easier for you to find them again, but it will help you develop your strategy analysis as you collect your evidence.

StratNavApp.com supports this with a convenient 'clipping' tool. Once you've found a web-page of interest, simply highlight the relevant text and click the button. StratNavApp.com will pull it through into the app, categorise it and link it into your existing analysis. Follow these instructions to install the clipping tool in your browser.

StratNavApp.com also integrates with Grammarly.com. Grammarly.com will help you to ensure that your notes are clearly and accurately written.

Conclusion

Evidence is the bedrock of good strategy. Get good at finding and interpreting it. (Tweet this)

What are your top tips for researching evidence? I'd love to hear your thoughts or questions in the comments.

See also:

• Everybody Lies: The evolution of market research | StrategicCoffee
• 6 top tips for strategy analysis
• More data usually beats better algorithms