Beating the Kobayashi Maru – or the human vs machine experiment with Watson

“I don’t believe in no-win scenarios.” (James T. Kirk, Starship Captain)

When you are a strong believer in datadriven decision making, building strategies on real insights, and always sticking to facts rather than fiction – it’s a hard blow when one of the world’s leading artificial intelligence systems tells you, that you are not a nice person. It’s based on data – so it’s a fact.

Many industry leaders have evangelists who are excellent presenters and subject matter experts. It’s always a privilege when you get a chance to interview an evangelist. I met IBM’s Rashik Parmar, Watson evangelist, at IPExpo Nordic a few weeks ago.

WatsonDescription.png

Source: IBM

There is so much potential for big data analysis and the learnings and insights we gain, from combining the many available sources of accssible data to draw new conclusions and find answers. That’s basically what Watson does. And then makes the logical connections. Simply put.

IBM developed a small demo engine that would analyse your Twitter personality and generate those awesome charts we all love; and few of us know how to interpret. It was reassuring to see what a nice guy President Obama is on Twitter. And my friend, Rashik, had a similar profile – so all good.

Potus.png

Source: IBM’s demo app TweetMeWatson

Lucky for me, we couldn’t make it work for my profile until I got home. When I ran it, I found out I was

“Inconsiderate and a bit shrewd”

I will spare you the rest. Either I am very delusional about how I express myself, or there was something fishy going on here. But it’s based on data! It has to be true!

Before digging a hole in the garden to hide from the world – or the equivalent: deleting my Twitter account – I decided to think it through. What was Watson picking up on, what were the actual parameters used?

The Big Five (FFM) Personality Traits

Watson is grouping our personalities according to the Five Factor Model (FFM) Wikipedia explains:

The Big Five personality traits, also known as the five factor model (FFM), is a model based on common language descriptors of personality (lexical hypothesis). These descriptors are grouped together using a statistical technique called factor analysis (i.e. this model is not based on experiments).

This widely examined theory suggests five broad dimensions used by some psychologists to describe the human personality and psyche.[1][2] The five factors have been defined as openness to experienceconscientiousnessextraversionagreeableness, and neuroticism, often listed under the acronyms OCEAN or CANOE. Beneath each proposed global factor, a number of correlated and more specific primary factors are claimed. For example, extraversion is said to include such related qualities as gregariousness, assertiveness, excitement seeking, warmth, activity, and positive emotions.

220px-francis_galton_1850s It all sounds very reassuring, the term “Lexical Hypothesis” makes sense –  it was analysing words. This is a principle which was developed by British and German psychologists to identify a personality characteristic. It was used to determine risk of mental illness or criminal behaviour. Invented in 1884, by the way, by Sir Fancis Galton – a stern looking fellow.

But something as elusive and intangible as the human mind is so very hard to classify and illustrate in data points and charts. By creating a lexicon of words and adjectives that at the time were considered to be indicators for certain behaviours, they provided a tool to build profiles – and categorise people based by their choice of words.

Note that the method has also received a lot of criticism – many of them quite reassuring when you are on the receiving end of this exercise. Read more here. 

Phew – that means I can still be a nice person, just not when I tweet. Or speak.

It seemed safe to climb back out of the hole in the garden and meet the world face on. But knowing now what triggered my unpleasant profile, I decided to challenge Watson to a duel.

A duelling experiment

kobayashi-maru-02

@echrexperiment is the experimental Twitter profile where tweets were worded more carefully, where people and followers were thanked and nothing bad was happening in the world. No politics, no injustice, no gender inequality, no discrimination. And lots of cats.

exhrexperiment.png

 

After three weeks, I was a much nicer person. The traits that I seem to be exploiting negatively in my original profile are now contributing to a positive image.

Suddenly, uncompromising was a good thing.

Spock.png

“In academic vernacular, you cheated”

Like Captain James T. Kirk in Star Trek challenging Mr. Spock’s designed program, I cheated to win.

Most of my tweets were carefully drafted using positive semantics but remaining true to my usual topics of interest. I was not trying to be someone else, just focusing on being nice. Here’s a list of the parameters I introduced to make Watson love me more:

  • Following back – anyone who followed me, unless an obvious business account or egghead, was followed back as soon as I spotted them.
  • #FF – sometimes I used the FollowFriday hashtag to thank select people. It generated some nice interactions even between those mentioned, so I grouped them into categories – e.g. Danes, analysts, etc.
  • I thanked, and loved, and “awesome’d” and “great’ed” a lot.
  • Sharing – giving credit, not taking it. I always mentioned the source or the account where I had picked up a link.
  • Sharing the love – retweets were focused on positive news, positive sentiments and uplifting current events. I also checked the wording of the original tweet before RT’ing to avoid contamination of my positivity.
  • Getting personal – my personality and emotions were conveyed more by sharing private interests such as books, cats, travel and science fiction.
  • Language Disclaimer – all of the above choices were based on my non-native perception of the English language, and may have been different from Webster’s Dictionary which is the basic semantic interpreter used in lexical hypothesis.

What I didn’t do

Humour doesn’t travel well, so any jokes, irony, satire and cartoons were not part of echrexperiment. I may have gotten carried away occasionally, but consciously tried to avoid it.

Politics are a powerful emotional trigger, so I avoided RT’ing or engaging in conversations with political statements. That wasn’t the mission.

Automation is a powerful tool to increase the quantity of your social media posts, but with automation things like timing and engagement suffer. Sometimes, due to other news, automation may even lead to displaying insensitivity.

Automatic response is a convenient way to further promote your services and invite people to connect. But it just isn’t personal. Despite all these lovely people addressing me by name. I did not send messages to thank people for the follow, but I checked their profile and retweeted where I could to show my appreciation.

What Watson had to say about @echrexperiment

The app itself produces a lot of detail as you can see from above. Below I grouped the result into more familiar charts to share some highlights. To make sure I picked a really nice person as control, I chose President Obama’s Twitter @potus. But please remember – it’s probably mostly his staff tweeting. And they seem to have done an excellent job.

watson1

Obama – it appears – is very agreeable on Twitter, and my experimental lovely/nicey/catsy account matches this impression very nicely. We are both very open, although I am lagging on conscientousness, but hey –  I am not the President.

Digging deeper into selected parameters, revealed some interesting characteristics related to being a President or just trying to be a nice person.

We can all agree that values should be an important parameter if you are President of the United States. Strangely enough Obama wasn’t all that keen on change, and more inclined to be conservative. For self enhancement … we have identified the villain – the one parameter that makes my original Twitter account so repugnant. I leave the graph to stand on its own.

Meanwhile, President Obama scored a resounding Zero on self-enhancement – but he made it to the top already.

watson2

President Obama’s most distinguishing need is the need for structure. Love – it seems – he gets a plenty.

On the other hand, my original self seems to have enough structure in her life.

Watson3.png

But here’s the greatest insight from this entire exercise – other than confirming that it is possible to change who you are, or rather how you are perceived:

When it comes to curiosity, all you need to do is be a positive tweeter and include lots of cats.

 

IMG_0042.JPG

 

Timing is everything – and loyalty is earned

Looking for correlations between airline social customer service and growth.

The social media manager of an airline or airport has a challenging job dealing with complaints, bookings, questions about all and sundry … and emergencies.

 

 

It’s not a nine to five job – it’s a 24/7 task for a social media manager at airlines and airports. People have questions and need help anytime, anywhere. Including when stuck in elevators. When it comes to monitoring and responding, in some cases it’s a life-or-death situation. I hope, Amanda Carpenter survived the ordeal of waiting from Feb 14 until Sept 7 in an elevator at the airport before somebody on the social media team responded to her tweet.

Oh, and Ms. Carpenter is not just anyone – she is an accomplished CNN contributor. Not that this matters – regardless of who you are, if you need help you should get it while you are still breathing.

Just testing

Knowing her to be a journalist, I seriously hope this was only a test to check the response time of @Amtrak – but you never know.

Airlines and airports – as well as many travel & leisure providers – are industries where a Twitter conversation is an important channel for customer interactions. So, in 2013 I wanted to find out how and if these industries had embraced this opportunity on improving customer satisfaction and grow their business despite a challenging market situation.

With the help of Datasift, we analyzed the response time on Twitter of 33 different airlines worldwide over a period of 30 days. At the time – 2013 – there were more than 100,000 tweets from customers mentioning the airline either directly or via hashtag. If ran today, the numbers would have multiplied.

tweetsairlines

 

 

CEOs want social media presence to influence buying decisions

At least that’s the prevalent argument. Social media was about sharing and engaging with family and friends – who in turn influence our buying decisions. This was the major argument in favour of companies investing in these channels. And 3 years later it’s still up there as one of the reasons CEOs invest.

So, I asked my family, friends and wider network eight short questions about whether they had ever tried to contact an airline via social media, whether the airline responded, and whether you were satisfied with the response – leading to a positive customer experience. And repeated this survey in 2016 to see if things had changed.

Turns out, they hadn’t. The sample is in no way stastistically significant with 35 responses then, and 46 responses this year. But what is interesting, is that things hadn’t really changed that much. The airlines who were most responsive in 2013, were still the best and most appreciated in 2016. And those who sucked… well, they still sucked. Except two: American Airlines  and Lufthansa Group

 

guitar

It is often said that it is almost impossible to reverse a bad reputation, which United Airlines has felt ever since it lost the famous guitar.  But it is possible to build a great reputation as American Airlines has done, and jump 33 points on the J.D. Powers  customer satisfaction index for airline industries. Simply by focusing on social profiles and social interactions.

American Airlines started running regular workshops with their staff teaching all customer facing employees how to navigate in the social media space and to pick up trends and grumbles before they turn into storms. In 2013 they were rated below average – three years later they had climbed the ladder significantly.

Twitter – a marketing channel or a conversation?

This graph shows the response rate versus customer interactions for some of the airlines in the study. You will want to look for the white space – the gap between the organge response line and the yellow staples signifying number of customer mentions either directly via their Twitter profile or in hashtags. The whiter, the better.

detailtweetsairlines

Other than American Airlines, United and Delta, British Airlines stands out as being very non-responsive. In 2013, they “loved reading tweets” on their global Twitter account, but were only ”answering 09.00 – 17.00 GMT on weekdays.”

BA2013.png

For my analysis, I picked those airlines respondents had mentioned. Lufthansa was among those who were very present on Twitter showing promotional images of their aircrafts engine power, happy pilots and stewardesses, and pictures of the clouds in the sky. But for customer service, in 2013 Lufthansa referred people to download a detailed form on their website, and fax (!) or email it to their customer service centres.

Air travel remains for many people an uncomfortable, disappointing and grumble-worthy experience. Things have changed both for Lufthansa Group, American Airlines, United Airlines, and to some extent British Airways. But the grumbling persists.

ba2016

As with United Airlines and the lost guitar, the reputation of British Airways for non-supportive customer support remains a stigma. And the execution for both airlines appears still to be slightly lagging at least according to the 46 responses on my little survey.

Is there a correlation between focus on social customer service and growth?

Imagine if you could simply take these findings directly into the board room and demand more resources for your social customer service initiatives.

Unfortunately, it would require a lot more detail and a larger survey sample to draw any conclusions worthy of that, but the financial results before taxes and interest for some of the noteworthy airlines from the original study show some trends, especially if related to growth in passenger numbers.

KLM is famous for it’s pioneering efforts in engaging customers on Twitter with their many innovative ideas. But this analysis cannot illustrate the impact, if any, because they had since merged with Air France. But they are both on the top ten index of the world’s best airlines, so it can’t be all wrong. Similarly, Lufthansa Group now comprises Swiss who already then were performing well in the response rate plus acquired several more in the interim years.

We shouldn’t jump to conclusions. Even obvious correlations may be false friends.

In 2000 there were 327 deaths by people entangled in their bedsheets and per capita cheese consumption in the United States was 29.8lbs. This has grown to 717 deaths and 32.8lbs of cheese by 2009. A clear correlation over the years.

cheesandbedsheets

 Enjoy more of these obvious and very funny correlations by Tylver Vigen here.

What I am trying to illustrate with the cheese in the bed sheets is that we cannot draw any conclusive data from the social media engagement rate and the financial results or passenger growth in the airline industry. There are too many additional data points that influence or need to be filtered out. It requires more computing power than I have available. But it would be an obvious task for some of the now very hot artificial intelligences being launched by many IT vendors. What if I could ask IBM’s cognitive intelligence Watson  to do an analysis?

I did in fact ask Watson about something else – stay tuned for my next blog post on the Kobayashi Maru – or how I convinced Watson to change its impression of my Twitter personality. Take a look at @echrexperiment on Twitter to see how I did it.

So, is there a correlation between a company’s financial growth and turnover and how socially engaged they are in their customer service function?

The short answer is yes and no

Sorry, if this wasn’t helpful.

If you compare customer satisfaction index, the financial results before taxes and the passenger growth of these airlines in 2013 adding the filter of how they were rated in engagement on Twitter with their 2016 results and growth, all of them have grown. But not necessarily because of their satisfaction ratings or social media engagement, but due to other strategic measures such as mergers, geographic focus, improved fleet etc. The numbers provided in the below chart are based on the annual reports and official websites of each of the airlines comparing 2013 with 2015.

financeairlines

 

It can make all the difference in the world

On this chart one airline stands out with negative growth in passengers and the lowest financial growth 2013 – 2016 on results before taxes and interest. But it is also one of the most Social Airlines in terms of response rate.

Scandinavian Airlines Systems was facing bankruptcy in November 2012. Media reported hourly on the negotiations between trade unions and SAS leadership and executive board. They were trying to agree on terms that would make SAS more competitive and allow the airline to bring in more capital to avert the crisis.

Meanwhile, over the course of that week, travelers were deeply worried. But SAS had a social media strategy in place already. Following the infamous volcanic ash cloud closing down airspace in most of Europe in 2011, they had kicked off their social media channel with a focus on providing active assistance and service to their passengers.

During that dramatic week, one social media manager in particular – Cecilia Saberi – stood out with her calm and constructive responsiveness, her quiet charm with a twinkle in her eye. She worked day and night, slept on the sofa in the office for a few hours only to resume responding to concerned passengers, media and sensationalists. Her approach was sincere, open, genuine and fact-based. And she showed with every comment, every tweet, that people interact with people, not machines or corporations.

CeciliaTweet.png

(Bård tweeted a link to a newspaper site: “SAS in collapse. SAS very close to bankruptcy.” Cecilia responded within minutes: “Hello Bård, we are flying as usual, but of course all these speculations in media create unnecessary uncertainty. Have a nice day! //Cecilia”)

SAS did not lose their passengers during that week as far as is known. Because business continued as usual in a very unusual situation. As is often the case, despite sensationalist media reports creating issues without proper attention to facts.

I repeat – Timing is everything

As you can see from the below Skytrax 2015 airline ratings, customer satisfaction does not necessarily lead to better financial results or more bookings.

But during a crisis, loyalty and genuine openness and care – including responding while the response is still helpful and not leaving a journalist in an elevator for 6 months – can make the impossible possible and turn around a potentially disastrous situation into even better experiences.

Cecilia no longer works for SAS – but how she interacted has become the style of the social media team and is well appreciated by the customers. The seats may be shaky, there is no silver ware in business class, but Scandinavians remain loyal a little longer while SAS gets itself sorted.

customersatskytrax