Rethinking Big Data and Personalisation

I recently came across the claim that Data has become the basis for competitive advantage. Like the author of that article, that claim got me wondering if it was true (spoiler: yes, I think it is) and if so, what we should do about it.

To understand the import of this claim, we have to cast our minds way back into history.

In the agrarian age, the basis for competition was land. Feudal lords warred with each other to expand their territories. They profited by taxing the local populations. They forced them into military service to gain control of yet greater territories. In the industrial age, the basis for competition was resources. Nations and industrialists fought to control resources like gold, oil, coal and steel. They competed in terms of engineering ingenuity and patent protection.

But now in the information age, the basis for competition is information. Nations and businesses compete by gathering and processing information. Physical warfare and theft are being replaced by cyber-warfare, misinformation campaigns, hacking and identity theft. Of course, the agrarian, and industrial models continue in parallel in the background. But they are of relatively less importance as time goes by.

But what does this mean for businesses and competition?

Big Data and Machine Learning

The first implication is for big data and machine learning. In the early days of the information age, data described industrial concepts like stock units, accounts and transactions. These were relatively simple data constructs, and the volume of data was relatively low. The explosion of new devices of greater variety (think of the Internet of Things) means we're now collecting much greater volumes of data describing a much greater variety of real-world phenomena.

This so-called 'big data' is not only more voluminous but also less structured. This is where machine learning comes in. Big data is too vast and complex for humans to analyse and understand. So we're turning to computer algorithms to do it for us. Knowledge and understanding are being externalised. As humans, we see the results of machine learning, but we don't necessarily understand the thinking behind it.

The combination of big data and machine learning allows us to develop rich pictures of people and other real-world phenomena.

(See also: More data usually beats better algorithms)

Personalisation

Personalisation, in the sense of user experience design, is being presented as something new. But in some senses, it is anything but that. In the 'good old days' local craftspeople and traders serving local communities knew their customers, often personally. We lost that personal connection in business, when, in pursuit of the benefits of scale, we centralised customer engagement into call centres. Customers typically speak to a different agent each time they call. And the information available to the agent is very transactional. Things got even worse when, in pursuit of even greater benefits of scale, we moved everything online. Customer interactions were further simplified down to menu choices and button clicks.

Big data and machine learning offer an opportunity to re-introduce personalisation at scale. I am not suggesting we'll have computer systems that remember to ask me how my children are. But personalisation can be used to ensure the content, promotions and user experience I am exposed to is tailored to my specific requirements and circumstances.

A good way to explain personalisation is to offer examples of what it is not:

1. Every time I log on to my bank I presented with the same information about ringfencing, security and identity theft. It does not matter how frequently I see it, or what I do about it, the information is always the same.
2. My investment manager sends me a fortnightly newsletter full of insights which have nothing to do with my approach to investments. I consistently ignore them, but still, they keep coming.

As a result, I have become 'blind' to messages from my bank and investment manager - I simply don't bother to look at them. It is annoying for me, and a wasted opportunity for them.

Skills

Big data and machine learning mean that organisations need employees and leaders with different skillsets. In addition to engineers and accountants, organisations need data scientists and machine learning experts.

In these new fields, standards of competence are not yet clearly defined. Demand for skilled resources outstrips supply.

Impacts on Trust

The recent Facebook / Cambridge Analytica scandal has highlighted a number of problems in this new world:

1. Facebook allowed Cambridge Analytica to gain access to users' private data as a result of 'backdoors' in their algorithms. Facebook supposedly knew about these backdoors. But it had not prioritised fixing them.
2. Cambridge Analytica then tried to use that data for nefarious purposes. (I have heard quite a lot of doubt expressed as to whether they actually had any impact on the US presidential elections or the UK Brexit referendum.)
3. Facebook was not open and honest about the breach. It took an investigative journalist and a whistleblower to bring it out. Under the European GDPR, which comes into effect on 25 May 2018, Facebook would have been obliged to let every impacted user know, or face substantial fines. As it was, Facebook broke customers' trust, but not the law.
4. Once people started to become aware of the issue, they started looking at exactly what data Facebook had stored about them. They were surprised by just how voluminous and detailed it was. People have subsequently had similar revelations with Google. This is causing people to think more carefully about the value exchange in which they trade their personal data for free services.

My personal view is that people will continue to be happy to trade their data for services. But only if:

1. The data is being used in a way that they perceive to be of benefit to them. For many people, personalisation of advertising does not yet seem like a benefit, but that could change. And, of course, there are many other more direct benefits that could be delivered.
   AND
2. They trust the companies to look after their personal data and keep it safe.

The rise of the Personal Information Managers

A whole new category of services is arising in response to the challenges outlined above: the Personal Information Manager (PIM). A PIM acts as a single source of truth for an individual's data. It gives them control over which other organisations can access their personal data, for what purposes, and over what time periods. They offer organisations GDPR compliance. And they offer customers greater control.

Port.IM is an example of a PIM.

PIM's are a relatively new and not yet completely established class of application, which could play an increasingly important role as people grapple with how they want to manage their personal information.

Conclusions

Many of the biggest and fastest growing businesses of recent time - Facebook, Google and Amazon, for example - succeed largely on the basis of their ability to collect personal data and extract commercial advantages from it. More traditional organisations have, up until now, been able to continue to rely on the strength of their products and services to survive.

However, as more and more traditional competitors start to adopt these practices, those that don't risk being left behind.

Organisations should:

1. Take stock of the information they do and don't (but should) collect about customers and distributors.
2. Look for opportunities to experiment with big data, machine learning and personalisation to extract commercial value from data.
3. Ensure that the security of personal data is of paramount importance.
4. Work to ensure that customers understand exactly what data organisations collect about them and why it is in their interest to allow them to do so.
5. Explore relationships with PIM providers, rather than trying to solve all of the problems themselves.

photo credit: Christoph Scholz IT-Sicherheit Schloss mit binary Hintergrund digital via photopin (license)

Book Review: The Black Swan, by Nassim Taleb

The central premise of the Black Swan, by Nassim Taleb, is that rare outlying events have a greater influence on the world than do statistically predictable ones, but whilst our planning and analysis systems deal with the latter reasonably well, they are completely inadequate for the former.

Examples of rare outlying events that have had a significant event on our world include 9/11, the 2008/9 stock market crash, and, more recently, the UK #Brexit referendum.

Taleb does a reasonably good job of pointing out the flaws in what he terms the Gaussian model (most notably the normal distribution). He also does a good job in demonstrating our tendency to want to try to describe everything in these terms, regardless of whether or not it is appropriate, and therefore to tend to ignore everything that falls outside of this model.

However, he does err towards throwing the baby out with the bath water. The normal distribution (and indeed the other distributions he criticises) are useful in many applications. And that should be the real measure of any theory. All theories have limitations and domains beyond which they should be used, and yet remain useful if appropriately applied, and statistical distributions are not different in this regard.

He also fails to present any useful alternatives, and so does not really take us forward in any meaningful way. Thinking about it from my own perspective, scenario planning does provide one useful alternative which can help us to escape the confines and flaws of the so-called Gaussian model.

The Black Swan is a fascinating and eclectic read which may challenge many of our assumptions about the world and the way we analyse it. However, if you're looking for practical alternative solutions, you may be disappointed.

Data Analysis Lessons from "The implications of Brexit for monetary policy"

I recently attended Martin Weale's valedictory speech as a member of the Monetary Policy Committee (MPC), which was entitled: "The implications of Brexit for monetary policy", and was hosted by Resolution Foundation.

The session consisted of the speech itself, follow by panel comments from Sushil Wadhwani (economist and former MPC member), Chris Giles (Economics Editor at the Financial Times) and
Melanie Baker (UK economist at Morgan Stanley), followed by questions from the audience to the speaker and panel.

Martin Weale's speech was fascinating enough in its own right. However this is not a blog on economics or monetary policy, and so I will not even attempt to do it justice here. (If you're interested, you can read the speech itself here, and Resolution Foundation's write up of the event here.) Rather, I will pick up on two related points Sunil Wadhwani made in his remarks, and which I think have direct pertinence to business strategy.

1. Having the data is not enough

Martin showed two separate charts, one showing the weakening of the exchange rate, and the other showing the fall and recovery in the FTSE100 and the fall and not recovery in the FTSE250 immediately following the referendum. He suggested that the fall in the FTSE250 was more representative of sentiment regarding the UK because so much of the FTSE100 consisted of expected foreign earnings from multinationals with UK listings. He then suggested that the fall in the FTSE250 was not significant enough to allow one to draw conclusions (and did little more than confirm that "prices can go down as well as up").

Sunil countered that a better measure of confidence in the UK economy would be the FTSE250 in dollar terms. This had taken a pounding following the referendum, and painted a much more negative outlook than Martin had suggested. (This effectively combines the two charts.)

Whether you agree with Martin or Sunil, the exchange was a potent reminder that its not just what data you have, but also how you analyse it.

It reminded me of a project I worked on some years ago where the data we were seeing was showing a slight decline in the performance of a particular process. Because the decline appeared to be only slight it was not ringing any alarm bells (yet). However, I knew that the process (a) dealt with 6 discrete populations and (b) included a natural delay of some months. So I requested the underlying source data, and (1) split it into the populations, and (2) did a batch cohort analysis of each. This analysis revealed a much deeper - up to 50% for some populations - decline in process effectiveness. That definitely started the alarm bells ringing!

Yes, subtly improper analysis of data can render it very misleading!

2. You'll never have all of the data or analysis

Sunil further responded to Dr Weaver's conclusion that the data was still inconclusive by remarking that:

"You have to form judgments; because you are never as well informed as you would like to be; because the data is simply not there." (Tweet this!)

He went on to advise:

"Resist the temptation to wait for more data before acting. There will always be more data to wait for." (Tweet this!)

(That is my best recollection of the words that he used, but I cannot guarantee that it is verbatim.)

That is one of the key lessons I remember from the many case studies we did on my MBA programme. (I sometimes think that part of the objective of an MBA programme is to overload you with case studies and then put you on the spot in class in front of your peers to draw conclusions from what is inevitably inadequate data, as that is the closest they can get to what it feels like in real life within the classroom context!)

Its a lessons that has stood me in good stead ever since. In any strategic process, there is a time to collect more data, a time to conduct deeper analysis, and a time to accept that what you've got is good enough/as good as you're going to get and its time to make some decisions and move forward.

If you fail to learn that lesson, you invariable fall into 'paralysis by analysis': where data and analysis snowball and any chance of meaningful action falls by the wayside.

Conclusion

Data is a strategist's friend. It is the bedrock of analysis, reasoned decision making and feedback. But it is not without its pitfalls. Good use of data is as much an art as it is a science. And it is one every strategist does well to study carefully.

How to deal with chronic uncertainty (like Brexit) in business strategy

I've just read (yet another!) blog post advising business owners on what to do about Brexit. The conclusion: there is so much uncertainty about the outcome that business owners should just ignore it and carry on as if nothing had happened.

I have seldom heard such poor advice!

In the first instance, uncertainty is no excuse for burying your head in the sand. We live in uncertain times, and if it were, no-one would ever do anything. As a discipline, strategy has tried and tested ways of dealing with uncertainty.

Secondly, we now have significantly more information about the future than we had 3 weeks ago. To ignore that information would be myopic and foolish.

So, how does one deal with chronic uncertainty in a structured and proactive manner? Here is a 6-step approach:

1. Get the facts

After a referendum characterised by misinformation, it is important to remain appropriately informed. Key questions include:

1. What is the legal status of the referendum, and what, if anything could overturn it?
2. What is the actual process, steps to be taken, and timelines for leaving the EU? 
3. Who are the decision makers and power brokers, in both the UK in Europe, and what are they saying and doing?
4. What models exist for subsequent engagement with the EU and what do they entail?

Ignorance breeds fear, so get informed.

2. Identify possible outcomes

Following the referendum, there are a number of possible outcomes. At the highest level, these might include:

1. The UK does not leave the EU.
2. The UK leaves the EU under favourable terms (so-called Brexit-light).
3. The UK leaves the EU under unfavourable terms.
4. The UK leaves the EU, followed by other countries exiting and ultimately, the collapse of the EU itself.
5. The UK splits, with Scotland remaining a part of the EU and the rest of the UK exiting.

There are, of course many other combinations and permutations which might be worthy of consideration. Whilst it is probably impractical to consider them all, it is important to consider a wide range of possible outcomes.

3. Understand the circumstances and implications of each possible outcome

Within each possible outcome, it is important to develop an understanding of:

1. What are the future developments and circumstances which might make that outcome more or less likely to emerge, and
2. What are the implications of that outcome, in general, and for your business specifically.

It is important to develop as vivid a narrative for each possible outcome as is possible. That is, write a plausible story for each outcome a logical chain of actions, events and their consequences. The more vivid the narrative, the more instructive it will be in planning your response.

4. Implement an early warning system

Once you've identified the circumstances which might make it more likely for one outcome to emerge than another, you need to use that as a lens for monitoring developments on an ongoing basis. Make specific people responsible for monitoring specific issues and reporting them to the broader group on a regular basis. Review all of your plans every time there is a major development. Know in advance when you intend to act, and when you intend to sit tight and watch.

Include relevant factors into your competitor analysis (see 7 straight-forward steps to master competitor analysis) to keep one step ahead of the competition.

5. Prepare plans in advance for the most likely outcomes

Don't wait for your early warning system to tell you that something has happened. It's too late to start planning then. Prepare contingency plans for each of the possible outcomes. Add more detail to your plans as events develop and some outcomes become more likely, leaving the plans for the less likely outcomes. You don't need to execute your plans now, but you do want to know in advance who will do what when key outcomes do emerge.

The plans you develop for each of the likely outcomes may be different to the normal plans you'd implement for, say, the implementation of a large system. Plans should emphasise "if this then that" logic, review and decision points and accountabilities, and clear criteria for deciding when to push forward and when to hold back.

You may find that from your plans there emerge some actions which you'd take in the event of many or all outcomes, which expand the options available to you, and/or which are relatively inexpensive to complete. You may then decide to proceed with these "no regrets" actions immediately.

6. Deal with the uncertainty now

The preceding 5 steps deal with planning ahead for what might happen. But there are also things that you could be doing to better cope with the uncertainty right now.

In the case of Brexit, there are at a number of likely immediate considerations:

1. How are you suppliers, distributors and customers responding? For example, if business partners (especially foreign ones) are less inclined to enter into long-term contracts because of the uncertainty, how could that impact your business and your existing plans for growth or expansion and how will you respond? What could you do to help your partners overcome any such reticence.
2. A Brexit will inevitably place a huge demand on legal, regulatory, compliance and strategy resources. Do you need need to secure resource in advance, or risk losing out when there is a mad rush at the last minute (as some experienced as the Solvency II deadline approached)? What regulatory or competitive initiatives will be put on hold as regulators and competitors divert resources to deal with their own Brexit plans, and what will you do with the breathing space that might offer?
3. What are you doing to re-assure your staff, customers and partners that

Chronic uncertainty certainly complicates strategy, but it also offers many opportunities. It is important not to get stunned into inaction, like a deer caught in the headlights. Proactivity remains key to success.

For a confidential conversation about what Brexit might mean for your business, or how to deal with uncertainty in general, please contact me.

See also:

• Scenario Planning: A Practical Guide for Navigating Uncertainty
• Brexit: now what (a StrategicCoffee blog post from the day after the referendum)
• Scenario Planning

photo credit: Deer in the headlights via photopin (license)

Brexit: Now what?

Well, it happened. Despite most business leaders campaigning against it, Britain voted to leave the European Union.

Now what?

As regular readers will know, I usually advocate thinking strategically along three time horizons at the same time. Brexit is no exception.

The immediate priority

The good news is that nothing has changed - yet. The UK has voted to leave the EU but has not done so yet. As of today, it remains a member of the EU with all of the obligations and rights that membership entails.

But while nothing has changed yet, that does not mean that there is nothing to do.

The immediate priority must be to stabilise and re-assure. Just as Mark Carney, governor of the Bank of England, wasted no time in coming forward to re-assure markets that the BoE was doing everything it could to stabilise the currency (which dropped overnight to levels not seen since 1985), so too should business leaders be out in front of their staff and customers explaining what Brexit means for their business, what they are doing about it, and offering all re-assurance that they reasonably can.

Rapid and substantial change provides fertile soil for rumours and cultivates anxiety. It is the leaders role to provide a clear vision which re-assures and re-focuses, and strategy is the text with which they can do this.

In the medium term

In the medium term, Brexit is going to create a lot of work. I've heard it said there are some 80,000 pages of legislation which need to be unraveled and redrafted. Trade agreements will need to be renegotiated. All of this will inevitably knock-on impacts for firms supply chains, pricing, human resources, location and regulatory passporting practices and policies. One can see armies of lawyers, tax experts and procurement and policy experts, within government as well as within the private sector, working on this for years to come.

All of this effort comes at the expense of what those same people might otherwise be doing to help businesses develop and execute new products, services and other strategies. That is, there is a tremendous opportunity cost in Brexit.

Business without clear impact assessments, plans and resourcing already in place will need to move quickly to ensure all the capacity in the market has not already been absorbed by the time they are ready to act.

In the longer term

In the longer term, almost everyone's strategy needs to be revisited in light of the events of last night. We now all have a material strategic insight which none of had less than 24 hours ago. (Although this is about the longer-term implications of Brexit, that is not to say it can wait until later - this needs to be done right away in order to prepare for the longer-term future!)

Although we know the outcome of the referendum, there is still much uncertainty surrounding it.

• Who will succeed David Cameron as the leader of the Conservative Party and what will be the political consequences of that change of leadership and of the referendum itself? Will a special general election be called?
• When will Article 50 be invoked (marking the UK's official withdrawal from the EU and starting the 2 year time-frame within which the withdrawal must be completed)?
• How will rest of the EU and the world at large react: will they push the UK 'to the back of the queue as some have threatened, or welcome the UK into new trade agreements with open arms? Will other EU nations now seek to exit as well? Will the EU fight a rear-guard action to attempt to convince the UK to change its mind? Will Scotland go back to the polls for a second attempt at leaving the Union in order to remain within the EU?
• Will the UK remain a member of the European Economic Area (EEA) like Norway, or have to trade with Europe under the terms of the World Trade Organisation (WTO) like the USA or China.
• Will George Osborne issue an 'emergency budget' and what will this contain?
• What will the direct economic consequences be? Whilst the campaign was rife with difficult to reconcile scare-mongering predictions, we may yet experience a 'technical recession' - and if we do just how deep might it be?
• Which EU regulations will the UK adopt, what amendments might they make to those, and with what regulations might they replace those they don't adopt?

Retaining strategic focus in the face of such great uncertainty is extremely challenging, but the alternative - sitting back and waiting to see what happens - could prove disastrous. And it looks like it could take some years for the various inter-related issues to be fully resolved.

The logical place to go in the face of such uncertainty is Scenario Planning, a structured approach to developing and envisaging different possible futures, developing strategies which are more robust, and planning how to sense and respond to events as they then subsequently play out. 

The UK's EU referendum has been hailed as one of the most significant choices of our generation, and that is probably true. How we respond to it will define us, not just as a nation, but also as individual businesses. As a strategist, this is an exciting opportunity. Change inevitably creates winners and losers. Good strategy is what will decide which your business will be.

See also:

• Brexit could weaken the UK financial services sector

photo credit: 634747553 via photopin (license)