The heart in Artificial Intelligence

Blog post originally published on State of Digital as part of a monthly column

My son Arthur has just been awarded a prize for story-telling at his primary school. So when I watched that short movie whose script was generated by artificial intelligence, based on thousands of sci-fi books and films, I could certainly see a lot of similarity between both outputs. For me this epitomizes the current state of AI… It is raw, forming, full of potential but still with a long way to go towards maturity.

Today we will be covering the heart in artificial intelligence. After all, if artificial intelligence is, by definition, artificial, how can it have a heart, how can it have emotions? On the other hand, if AI is the brain child of thinking, feeling people, how can it not have a heart? This is a critical question for us, marketers who want to trigger emotional reactions from consumers but who also rely always more on algorithms and automation. To answer it, we will first need to define artificial intelligence. We’ll explore the 7 outcomes we can expect from AI, and consider how we materialize these expectations today to enable everyone of us to fulfil our potential.

The building blocks of AI

To understand if a heart is beating inside artificial intelligence, we need to understand where AI comes from. Although AI has been all over the press lately, it is not news… It is rather a 30-year old corpus of work, aimed at creating intelligent machines, by combining three building blocks: machine learning, human learning and data science. And in many ways there is a strong analogy between AI and raising a child.

Just like children get their foundational learnings from their parents, teachers and by the school books they read, machine learning is based on known properties, and the machine learns from the data. Think if/then scenarios. If your son behaves well, then he will be treated by Santa. If your daughter sees a puddle, then she should not to stomp in it to keep her feet dry. This is also how machine learning works: if you liked that book, then you’ll probably like these ones too. If you bought a laptop, then you should consider this bag. These are just small, basic examples of a very complex field.

Kids learn fast that if they cry and shout they get your attention… Now, you will certainly want them to assimilate that such a behaviour is not a normal mode of expression. Human learning is how we make course corrections to the machine learning that is happening. Cortana, Microsoft’s digital personal assistant, has a team behind the scenes working on human learning so she can get smarter. This human learning gives the digital personal assistant more personality, and her responses to queries are more human because of this.

Data science is the third brick of artificial intelligence. Data science is the discovery of unknown properties, or connections, in data. In this case, the machine is presented with a massive amount of data and asked to find connections in it. This is how we might discover that watching a certain program in your youth increases your chance to marry a foreigner. We didn’t know there was a connection between these pieces of data until we went looking for that connection.

Human intervention

We have seen these three fields accelerating their capabilities recently due to the exponential development rate of our computing power. We are able to process, analyse and render an ever-growing amount of information, at an increasing pace. But where does that data that feeds machine learning, human learning and data science come from?
It comes from us! Artificial intelligence comes from us. In many ways, it is us.

Artificial intelligence is only as intelligent as the data it takes in. It is only as fair as the data it takes in. It is only as human as the data it takes in. It is only as socially acceptable as the data it takes in. I would like to share with you two examples of AI, which to a large extent illustrate how humans can influence how intelligent a bot can be.
Remember Tay, Microsoft first experiment as a Twitter bot? Tay learned from her inputs, which were hijacked by some people who wanted to influence her negatively. In this case, Tay incited high emotion from people who engaged with her or read about what happened with her, even if Tay, herself, did not express emotion and was merely a reflection of the hatred that fed her.

On the other end, Microsoft also created Xiaoice a couple years ago and it is a perfect example of where technology is going and why we think of conversations as a new platform for brands and commerce. Xiaoice is a chat-bot based on Bing search technology and big data. It draws on AI, social media, and machine learning so she can hold a proper conversation – the average exchange between Xiaoice and a user has 26 turns. She’s sensitive to emotions and remembers your previous chats. If you tell her about a breakup, she’ll check in with you. If you introduce her to a puppy through a photo, she’ll recognize the breed, ask you for its development. And to say this bot has been popular is an understatement. Three days after she was available, Xiaoice had been added to 1.5 million conversations on WeChat. Once added to Weibo, the Chinese micro-blogging service, it became one of the most popular celebrity accounts. And today, Xiaoice is used by over 40 million people.

Tay and Xiaoice are like two twins, split at birth and raised in two different environments, with different influences… Two very different individuals in the end.

Assessing our expectations

So, what can we reasonably expect from artificial intelligence?

As mentioned before the computing advancements have enabled a fast acceleration of three technologies which underpin the maturation of artificial intelligence: object recognition, natural language processing and speech. If the AI can see, speak and listen, it is not far from being able to exchange with human being transparently.

Actually, mid-October 2016, Microsoft researchers announced they had reached human parity with the word error rate (WER) for conversational speech recognition, meaning that their AI was as capable as a professional transcriber to write up an oral conversation. Language understanding and acquisition is not easy, and it is critical to the success of AI. If you travelled a bit, you will be familiar on the complexity implied by accents, dialects, pronunciation but also the fact that a same word may have several meanings based on the context. This progress was critical because without this piece of artificial intelligence, so many developments wouldn’t move forward. Think about how patient you would be with a digital personal assistant or a sales advisor that misunderstood most of what you said?

Natural language learning is a complex skill, as we know from watching our children learn to speak. But with our increased computing capabilities, not only are we able to recognize accurately the words but we are able to do this instantaneously. This unlocks new scenarios like Voice-to-text which allows deaf children to read the transcript of a discussion in real time or Skype Translator which not only has the natural language skills necessary for a conversation but can also translate into other languages.

Well, this outcome is one of many. Capitalizing on the progress of machine learning around object recognition, natural language processing and speech, we have seen our expectations towards AI graduate from the most basic to much more advanced outcomes.

The 7 outcomes of AI

According to Silicon-Valley analyst, Ray Wang, there are seven intertwined outcomes for artificial intelligence, based on what we are now able to program via machine learning.

1. Perception is an example of early machine learning, now totally engrained in our daily life. Drawing on existing data, the machine delivers information about what is happening now. The weather, traffic, sales volumes, stock prices – things that are measureable and reportable. This AI outcome brings us back to the core promise of search engines when based on a typed or voiced query, the machine learning understands the intent and provides the answer or links to the information. For humans, learning to express their perception, it’s pretty simple as well. A child can describe what is happening now with ease. We learn this almost immediately: it is dark; I am hot; or, based on these circumstances, I am joyful. To illustrate a more advanced Perception outcome, we can look at facial recognition and play with http://how-old.net which assesses your age based on your traits (and which we hate to be accurate).
2. Next, Notification. If I did not have my calendar delivering notifications, I would be a horrible colleague – late to meetings or just not showing up because I cannot hold my schedule in my mind. Here the intent is less explicitly verbalized, but it is still initiated by the user and the information remains factual without any analysis of the data. We learn notification early as well, perhaps starting with letting Mom know we’re hungry. Fact: I am hungry; Notification: I cry. It never stops – in school, we notify the teacher that we have the answer.
3. Suggestion is another area we have grown to be familiar with, and is now engrained in our daily life. You searched for these words, but “Did you mean?”… The machine learns from past behaviours and suggests alternative actions. We all love this machine learning with our Spotify account for instance. If I listen to a song and I like it, the AI suggests more songs for me to enjoy. And you can always retain that Human Learning capability to ensure that the AI never drifts from Justin Timberlake to Justin Bieber… Early suggestions were basic, but imagine what can influence them today: demographics, location, day, time, weather, behaviours, etc. The data sets are humongous but we are now capable to combine and process them in no time and identify new, maybe more obscure connections
4. Our children learn a nice drawing will trigger a smile from their parents, or that it’s time to wash their hands before a meal. Over time, we don’t even have to remind them; they just know it’s what’s next and it becomes Automation. A suggestion or a recommended action can grow into automation based on learning your preferences. If you follow avidly the progress of your favorite team, the AI will start to automatically inform you of their performance. If you always make a reservation for 7pm on Saturdays, your AI will start to spontaneuously fill in the date and time on your reservations. If you trigger the same report every Monday morning, the machine will start to pull the information for you and make it available in your Business Intelligence dashboard.
5. Predictions can be the hardest machine learning to train, because so many variables can affect this outcome. Think of a child who sees Daddy packing a suitcase; based on past behaviour, this toddler knows that this means Daddy is leaving for a few days, which is sad. But sometimes it also means that the child gets to travel with Daddy. What factors will alert the toddler about what outcome to expect? Microsoft has developed a program called Bing Predicts which combines and models all the data signals we can find, and comes up with incredibly accurate predictions. It initially explored popularity-based contests like American Idol, for which the web and social signals are very strong and highly correlate with popularity voting patterns. You search for information about that performer, his history, his latest video clip. At the same time, you comment the performance on Facebook or Twitter. By combining anonymized search patterns to social signals Bing Predicts could accurately project who would be eliminated each week during American Idol and who the eventual winner would be. More complex, we then turned to sporting events and even world political challenges. During the World Cup in Brazil, our team predicted accurately with 100% accuracy the winners of the final elimination round. During the last year Rugby World cup, we had 87% accuracy across the tournament. Surprised? In order to successfully predict a sporting event outcome, the number and type of signals we incorporated quadrupled from what we used to predict a basic popularity event like American Idol. This is because we recognize that popularity alone does not predict whether a team will win – Sorry for the fans. A fan base has however special insight into the abilities of their teams, and those fans are having constant discussions about their team. This is called the “Insider Knowledge.” We weighted their knowledge against player and team stats, tournament trends, game history, location and even weather conditions. This is how we were successful in our predictions.
6. If we manage to predict accurately the future, the next logical step after prediction is Prevention. Again Bing Predicts shines in this category: by analysing large samples of search queries, Microsoft scientists have been able to identify internet users who are suffering from pancreatic cancer even before they were diagnosed. The researchers focused on searches conducted on Bing that indicated someone had been diagnosed with pancreatic cancer. From there, they worked backward, looking for earlier queries that could have shown that the Bing user was experiencing symptoms before the diagnosis. Those early searches, they believe, can be warning flags.
7. Finally, Situational Awareness for AI comes close to mimicking human behaviour in decision making. We see situational awareness as a combination of many aspects of AI, from object recognition to conversational speech. Here’s an example:

These 7 outcomes are complex and require a lot of training and time to accomplish. They are also interconnected and not mutually exclusive. They actually build upon each other to offer the benefits of AI to us, users.

In conclusion, everything we’re seeing with AI is exciting and rich. We see the heart in AI every day, when we ask it to help uncover cancer, help two people connect when they don’t speak the same language. But where is the moral and ethical compass for artificial intelligence?

As alluded to through this article, AI is still at its infancy and it is our collective responsibility to set it on the right trajectory. At Microsoft we are committed to this, and partnered with the University of Cambridge and the Partnership on AI, two international authorities to help shape the future of that promising discipline. For some, AI is a modern Oedipus that will have to “kill the father”, take away our jobs, make ourselves redundant. But for someone like Satya Nadella, AI will actually enable people to fulfil their full potential as we have seen across the 7 outcomes of AI. So yes, for Microsoft, AI has a heart. It is the mankind’s heart.

The unbearable lightness of having

Ouch! I just walked yet on a lingering Lego bricks kindly left behind by one of my two boys as a token to their gratitude. Re-ouch! What did I trip over this time? Oh, just one of my wife's precious items from her impressive plastic bag collection (including this very special vintage edition by Tesco from 2007)...

Organised mess?

Many new parents will certainly sympathise, so yes I confess, I live in a flat that is cluttered... Books, toys, plastic and handbags, stilettos, pens and a few devices here and there (because the geek that I am does contribute to that mayhem, of course). No matter how creative you get with storage, they always seem to overflow. So the problem may not be the storage, but the content. Of course it is.

In fact, after having sold us alternatively the dreams that as the ultimate luxury was space or that some Swedish wizardry could help make more out of our jam-packed spaces, a more recent trend has emerged from the media. It is no longer about expanding or optimising micro-inches of cramped living space: it is now a matter of decluttering. If in the past, there was a relatively basic dichotomy between the have's and the have-not's, there is now amongst the upper-middle class a third category: the don't-want-to-have's. For them, it becomes a decision not to possess.

Inspired by Japanese Zen and Feng-Shui philosophies, this phenomenon is trending far and large in the press, as more and more books are released about how to tidy and clear out. You must admit this is in itself a bit schizophrenic... After all, avid fans may end up cluttering their house with books on decluttering!

Spring cleaning

Fad or trend? We are now in the very last days of Spring, and many of us have felt the almost therapeutic feeling of emptying cupboards and other hidden boxes from the junk we had been accumulating over the previous twelve months. Off with that candle holder in terracotta. To the bin the piles of Time Out magazines you have been promising your self to catch up on in order to be up to speed with what is hot... or, well, what was hot in June 2013 by the look of the cover of the edition you hold in your hand.

It feels good to reclaim some ground over the mess. It feels even better when you clear your conscious when you hand over your definitely too tight jeans to a charity on the high street. But it would be interesting to see how this trend evolves once the dust has settled. Nevertheless this phenomenon struck a cord with me (and not only because I have a profound admiration for Japan and obsessed by the necessity to bring order to chaos). It led me to another very contemporary divergence: possession versus materialism.

Is digitalisation cheating?

For years the concept of possession was necessarily associated to physical object. Wealth was measured by the ground you owned, the serfs ploughing your fields, the pile of gold you could put on the table... And then came the banks, and money got dematerialised. You had no more trinkets but access to money, an abstract concept. It was still your sweat and tears (or your servants), but it was no longer your very own treasure. There was no more attachment to the object itself, rather to its value.

Similarly, information which was once captured in pages, books and bookshelves was first digitised but still remained visible. It was on that floppy disk or in that server that was buzzing in the corner of the office. It was not looking like a good old book anymore, but it was still there. This changed with the rise of Cloud computing. With it, the virtualisation accelerates and objects further dematerialise. Like the golden nuggets an jewels which were replaced by bank statements, books, disks, CD, cassettes, external hard drives, servers... are disappearing from the local premises to see their quintessence hosted somewhere in the cloud.

Slowly the reticence of not being able to touch-to-own is fading. People are perceiving the value of virtualisation: easy and ubiquitous access; lower costs as you pay only for the storage you actually need; security of having your assets backed up in several locations... Of course there are hackers, like there were bank robbers, and there are still people who don't trust the cloud like many did not trust bankers and preferred to sleep with money under their matrass. But there are also genuine enthusiasts who are seeing in technology the opportunity to live the above-described trend to its fullest. 

Technologically-enhanced lives

I indeed recently met that technophile whose job was to educate businesses about the latest evolutions and what they entail in terms of opportunity. As a technologist, he had decided to explore how far he could go in adopting technologies which could help him get rid of the unnecessary. He got a chip inserted under the skin, a bit of code here and there, and off he went to dematerialise his home. Sensors capture his presence and switches on and off the wifi, the lights, the heating system, etc. automatically based on agreed gestures, rules and orders passed through his phone. The keys to his flat were rapidly gone too, as his unique identifier emitted by his chip could open the door lock through NFC. Whilst many of us switch between different screens, he opted to retain only one, acknowledging that smartphones nowadays are sufficiently powerful to be a TV, a PC, a watch and even a phone. Why having a fridge if you could get his daily food intake delivered fresh to his door, prepared to meet his dietary requirements? One by one, he went through his inventory and tried to get rid of what was not really needed. He wanted to go back to the basics... Connected basics. 

This leads to some interesting points of reflection: the digitalisation of the world implies the rise of a new paradigm where you can own without possessing. You still own information, tunes, photos... but they do not materially exist any more. This means that the renunciation to physical ownership does not necessarily jeopardise the codes of our Western societies. Pushed to the extreme, wealth could materialise in absence of physical possession whilst the poorest would be the ones anchored in a material world, unable to digitised... Internet behind a social walled garden, so to speak.

In that hypothetical, yet plausible world, Maslow's pyramid of needs may see "wifi access" being added to its lower, more basic needs. This is one of the scenarios that the Singularity University explores during their curriculum: "how to apply exponential technologies to address humanity’s grand challenges" with a democratised access to the internet as a prerequisite to avoid a new social rupture between the connected and the disconnected. This is also why companies like Google are exploring ways to give access to the internet in creative ways like the Loon project (and not to expand the reach of their advertising audience of course).

Tidying my thoughts

Personally, I am enthused by what new technologies can offer, and as a humanist, I believe in our ability to keep the potential demons at bay. Without going to the extreme of my technologist, I am slowly decluttering my flat, saving one foot nail at a time my physical integrity, my sanity, and hopefully a tiny bit of the planet by not consuming beyond what I really need. I am from the Generation X, that generation who has come to the world amidst the recession after years of prosperity. Because of that, I am more than ever convinced that we are therefore a transitional breed, and probably better suited than anyone to help facilitate and educate the change without being blinded by optimism or pessimism. We are an agent of change. For the better.