The heart in Artificial Intelligence

Blog post originally published on State of Digital as part of a monthly column

My son Arthur has just been awarded a prize for story-telling at his primary school. So when I watched that short movie whose script was generated by artificial intelligence, based on thousands of sci-fi books and films, I could certainly see a lot of similarity between both outputs. For me this epitomizes the current state of AI… It is raw, forming, full of potential but still with a long way to go towards maturity.

Today we will be covering the heart in artificial intelligence. After all, if artificial intelligence is, by definition, artificial, how can it have a heart, how can it have emotions? On the other hand, if AI is the brain child of thinking, feeling people, how can it not have a heart? This is a critical question for us, marketers who want to trigger emotional reactions from consumers but who also rely always more on algorithms and automation. To answer it, we will first need to define artificial intelligence. We’ll explore the 7 outcomes we can expect from AI, and consider how we materialize these expectations today to enable everyone of us to fulfil our potential.

The building blocks of AI

To understand if a heart is beating inside artificial intelligence, we need to understand where AI comes from. Although AI has been all over the press lately, it is not news… It is rather a 30-year old corpus of work, aimed at creating intelligent machines, by combining three building blocks: machine learning, human learning and data science. And in many ways there is a strong analogy between AI and raising a child.

Just like children get their foundational learnings from their parents, teachers and by the school books they read, machine learning is based on known properties, and the machine learns from the data. Think if/then scenarios. If your son behaves well, then he will be treated by Santa. If your daughter sees a puddle, then she should not to stomp in it to keep her feet dry. This is also how machine learning works: if you liked that book, then you’ll probably like these ones too. If you bought a laptop, then you should consider this bag. These are just small, basic examples of a very complex field.

Kids learn fast that if they cry and shout they get your attention… Now, you will certainly want them to assimilate that such a behaviour is not a normal mode of expression. Human learning is how we make course corrections to the machine learning that is happening. Cortana, Microsoft’s digital personal assistant, has a team behind the scenes working on human learning so she can get smarter. This human learning gives the digital personal assistant more personality, and her responses to queries are more human because of this.

Data science is the third brick of artificial intelligence. Data science is the discovery of unknown properties, or connections, in data. In this case, the machine is presented with a massive amount of data and asked to find connections in it. This is how we might discover that watching a certain program in your youth increases your chance to marry a foreigner. We didn’t know there was a connection between these pieces of data until we went looking for that connection.

Human intervention

We have seen these three fields accelerating their capabilities recently due to the exponential development rate of our computing power. We are able to process, analyse and render an ever-growing amount of information, at an increasing pace. But where does that data that feeds machine learning, human learning and data science come from?
It comes from us! Artificial intelligence comes from us. In many ways, it is us.

Artificial intelligence is only as intelligent as the data it takes in. It is only as fair as the data it takes in. It is only as human as the data it takes in. It is only as socially acceptable as the data it takes in. I would like to share with you two examples of AI, which to a large extent illustrate how humans can influence how intelligent a bot can be.
Remember Tay, Microsoft first experiment as a Twitter bot? Tay learned from her inputs, which were hijacked by some people who wanted to influence her negatively. In this case, Tay incited high emotion from people who engaged with her or read about what happened with her, even if Tay, herself, did not express emotion and was merely a reflection of the hatred that fed her.

On the other end, Microsoft also created Xiaoice a couple years ago and it is a perfect example of where technology is going and why we think of conversations as a new platform for brands and commerce. Xiaoice is a chat-bot based on Bing search technology and big data. It draws on AI, social media, and machine learning so she can hold a proper conversation – the average exchange between Xiaoice and a user has 26 turns. She’s sensitive to emotions and remembers your previous chats. If you tell her about a breakup, she’ll check in with you. If you introduce her to a puppy through a photo, she’ll recognize the breed, ask you for its development. And to say this bot has been popular is an understatement. Three days after she was available, Xiaoice had been added to 1.5 million conversations on WeChat. Once added to Weibo, the Chinese micro-blogging service, it became one of the most popular celebrity accounts. And today, Xiaoice is used by over 40 million people.

Tay and Xiaoice are like two twins, split at birth and raised in two different environments, with different influences… Two very different individuals in the end.

Assessing our expectations

So, what can we reasonably expect from artificial intelligence?

As mentioned before the computing advancements have enabled a fast acceleration of three technologies which underpin the maturation of artificial intelligence: object recognition, natural language processing and speech. If the AI can see, speak and listen, it is not far from being able to exchange with human being transparently.

Actually, mid-October 2016, Microsoft researchers announced they had reached human parity with the word error rate (WER) for conversational speech recognition, meaning that their AI was as capable as a professional transcriber to write up an oral conversation. Language understanding and acquisition is not easy, and it is critical to the success of AI. If you travelled a bit, you will be familiar on the complexity implied by accents, dialects, pronunciation but also the fact that a same word may have several meanings based on the context. This progress was critical because without this piece of artificial intelligence, so many developments wouldn’t move forward. Think about how patient you would be with a digital personal assistant or a sales advisor that misunderstood most of what you said?

Natural language learning is a complex skill, as we know from watching our children learn to speak. But with our increased computing capabilities, not only are we able to recognize accurately the words but we are able to do this instantaneously. This unlocks new scenarios like Voice-to-text which allows deaf children to read the transcript of a discussion in real time or Skype Translator which not only has the natural language skills necessary for a conversation but can also translate into other languages.

Well, this outcome is one of many. Capitalizing on the progress of machine learning around object recognition, natural language processing and speech, we have seen our expectations towards AI graduate from the most basic to much more advanced outcomes.

The 7 outcomes of AI

According to Silicon-Valley analyst, Ray Wang, there are seven intertwined outcomes for artificial intelligence, based on what we are now able to program via machine learning.

1. Perception is an example of early machine learning, now totally engrained in our daily life. Drawing on existing data, the machine delivers information about what is happening now. The weather, traffic, sales volumes, stock prices – things that are measureable and reportable. This AI outcome brings us back to the core promise of search engines when based on a typed or voiced query, the machine learning understands the intent and provides the answer or links to the information. For humans, learning to express their perception, it’s pretty simple as well. A child can describe what is happening now with ease. We learn this almost immediately: it is dark; I am hot; or, based on these circumstances, I am joyful. To illustrate a more advanced Perception outcome, we can look at facial recognition and play with http://how-old.net which assesses your age based on your traits (and which we hate to be accurate).
2. Next, Notification. If I did not have my calendar delivering notifications, I would be a horrible colleague – late to meetings or just not showing up because I cannot hold my schedule in my mind. Here the intent is less explicitly verbalized, but it is still initiated by the user and the information remains factual without any analysis of the data. We learn notification early as well, perhaps starting with letting Mom know we’re hungry. Fact: I am hungry; Notification: I cry. It never stops – in school, we notify the teacher that we have the answer.
3. Suggestion is another area we have grown to be familiar with, and is now engrained in our daily life. You searched for these words, but “Did you mean?”… The machine learns from past behaviours and suggests alternative actions. We all love this machine learning with our Spotify account for instance. If I listen to a song and I like it, the AI suggests more songs for me to enjoy. And you can always retain that Human Learning capability to ensure that the AI never drifts from Justin Timberlake to Justin Bieber… Early suggestions were basic, but imagine what can influence them today: demographics, location, day, time, weather, behaviours, etc. The data sets are humongous but we are now capable to combine and process them in no time and identify new, maybe more obscure connections
4. Our children learn a nice drawing will trigger a smile from their parents, or that it’s time to wash their hands before a meal. Over time, we don’t even have to remind them; they just know it’s what’s next and it becomes Automation. A suggestion or a recommended action can grow into automation based on learning your preferences. If you follow avidly the progress of your favorite team, the AI will start to automatically inform you of their performance. If you always make a reservation for 7pm on Saturdays, your AI will start to spontaneuously fill in the date and time on your reservations. If you trigger the same report every Monday morning, the machine will start to pull the information for you and make it available in your Business Intelligence dashboard.
5. Predictions can be the hardest machine learning to train, because so many variables can affect this outcome. Think of a child who sees Daddy packing a suitcase; based on past behaviour, this toddler knows that this means Daddy is leaving for a few days, which is sad. But sometimes it also means that the child gets to travel with Daddy. What factors will alert the toddler about what outcome to expect? Microsoft has developed a program called Bing Predicts which combines and models all the data signals we can find, and comes up with incredibly accurate predictions. It initially explored popularity-based contests like American Idol, for which the web and social signals are very strong and highly correlate with popularity voting patterns. You search for information about that performer, his history, his latest video clip. At the same time, you comment the performance on Facebook or Twitter. By combining anonymized search patterns to social signals Bing Predicts could accurately project who would be eliminated each week during American Idol and who the eventual winner would be. More complex, we then turned to sporting events and even world political challenges. During the World Cup in Brazil, our team predicted accurately with 100% accuracy the winners of the final elimination round. During the last year Rugby World cup, we had 87% accuracy across the tournament. Surprised? In order to successfully predict a sporting event outcome, the number and type of signals we incorporated quadrupled from what we used to predict a basic popularity event like American Idol. This is because we recognize that popularity alone does not predict whether a team will win – Sorry for the fans. A fan base has however special insight into the abilities of their teams, and those fans are having constant discussions about their team. This is called the “Insider Knowledge.” We weighted their knowledge against player and team stats, tournament trends, game history, location and even weather conditions. This is how we were successful in our predictions.
6. If we manage to predict accurately the future, the next logical step after prediction is Prevention. Again Bing Predicts shines in this category: by analysing large samples of search queries, Microsoft scientists have been able to identify internet users who are suffering from pancreatic cancer even before they were diagnosed. The researchers focused on searches conducted on Bing that indicated someone had been diagnosed with pancreatic cancer. From there, they worked backward, looking for earlier queries that could have shown that the Bing user was experiencing symptoms before the diagnosis. Those early searches, they believe, can be warning flags.
7. Finally, Situational Awareness for AI comes close to mimicking human behaviour in decision making. We see situational awareness as a combination of many aspects of AI, from object recognition to conversational speech. Here’s an example:

These 7 outcomes are complex and require a lot of training and time to accomplish. They are also interconnected and not mutually exclusive. They actually build upon each other to offer the benefits of AI to us, users.

In conclusion, everything we’re seeing with AI is exciting and rich. We see the heart in AI every day, when we ask it to help uncover cancer, help two people connect when they don’t speak the same language. But where is the moral and ethical compass for artificial intelligence?

As alluded to through this article, AI is still at its infancy and it is our collective responsibility to set it on the right trajectory. At Microsoft we are committed to this, and partnered with the University of Cambridge and the Partnership on AI, two international authorities to help shape the future of that promising discipline. For some, AI is a modern Oedipus that will have to “kill the father”, take away our jobs, make ourselves redundant. But for someone like Satya Nadella, AI will actually enable people to fulfil their full potential as we have seen across the 7 outcomes of AI. So yes, for Microsoft, AI has a heart. It is the mankind’s heart.

An Introduction to Conversational Commerce and Bots

Blog post originally published on State of Digital as part of a monthly column.

Conversations.

These days, there is a lot of chatter about talking – and it’s not that surprising considering the forays in AI, machine learning and natural language processing have made interactions with technology more conversational, more human. Bots are now capable to think, but also view, speak, and listen. So businesses are discussing how they can, could, will, and should be using bots to drive more personalised conversational interfaces with their customers.

“By 2020, customers will manage 85% of their relationship with the enterprise without interacting with a human.” – Gartner Predicts

I already prefer to use an ATM at the bank and I go to the self-checkout when doing my groceries. It doesn’t seem like that far fetched to see how conversational bots can become an extension of a self-service interface to your brand across the platform and channel your customer choose.

Bing has already a few chatbots integrated directly into the search results page, creating new, deeper engagement with the brand right here where the customer intent is verbalised. As I was planning my latest trip to Seattle, I’ve found it pretty useful to interact via the bot to quickly discover the parking availability for the restaurant I was heading to and to make sure it offered gluten-free dishes for the colleague who was joining me:

What are bots and chatbots?

There is no difference. In the beginning bots were just pieces of software designed to automate specific tasks. Today, bots have evolved thanks to the accelerations of our capabilities in machine learning and natural language process to comprehend and engage in conversations. They are acting as an interface that can be plugged into APIs or various data sources to deliver information on demand and help drive conversational commerce. In many ways, they are a new paradigm to how we will consume the web information. We used to go to web pages, then to interact with apps… We are about to exchange with this new interface through natural language which will, in turn, retrieve the information we are after. We will stop browsing or tapping: we will start asking.

What’s the difference between bots and digital assistants?

A digital assistant like Cortana, Siri, Alexa or the Google Assistant is more advanced than a chatbot. Some call them “meta agents”. In addition to being able to parse conversational language like a chatbot they are also incorporate additional layers of artificial intelligence to help merge utility, productivity, entertainment and the ability to accomplish actions together to become an intelligent agent. It is both contextually and historically aware, which means that it can provide relevant information, proactive recommendations, tailored to your preferences and situation. As such, Cortana is a predictive and proactive agent that can, based on interactions, learn to interact with you, other people and bots.

What’s the difference between bots and skills?

What do you want to teach the digital assistant to do? A skill. A skill teaches the digital assistant how to do something or to take an action, based on a voice command. I might ask, “Hey Cortana, ask HUE to dim the kitchen lights” or “Hey Alexa, book a UBER in ten minutes!”,  “Hey Cortana, let OPENTABLE book a table for two at John Howie Steakhouse at 7pm.”

Conversational Commerce – Connecting the dots

When it first launched, Domino’s Pizza Bot was super simple. Simple but effective. Customers typed the word Pizza and it would place for their connected account an “Easy Order.” The bot wasn’t really conversational in nature, but it started with an easy-to-use feature which allowed Domino’s to get their foot in the connected kitchen door in order to test and learn. Currently, Domino’s has both a true conversational commerce chatbot and Amazon Alexa skills with increased capabilities. Although retaining their ambition for a simple consumer benefit, ordering a pizza, they have upped the AI complexity to allow users to build their order from scratch, but also to track an order or reorder their most recent order.

Let’s take a high lever look at getting started with bots and what you need to keep top of mind as you get started.

Tips for getting started with bots:

1. Plan
   1. Set goals and expectation for what it can do.
   2. Focus on interactions that mean the most to your customers
2. Start Simple:
   1. Focus on building a feature that works amazingly and will delight your customers.
   2. Don’t reveal all the features at once. It can overwhelm your customers.
   3. Integrate the features into the flow of the conversation where they make sense.
3. Develop your bot:
   1. Choose a frame, like the Microsoft bot framework that can help you scale across channels.
   2. Don’t try to launch across every channel at first. Take a test-and-learn approach and roll out features and channels.
4. Monitor bots closely:
   1. Mistakes are inevitable – Learn from them and fix any identified errors/mistakes often and quickly.
      1. Examine it: What questions did people ask that the bot wasn’t able to answer?
      2. Teach it: What words does the bot not understand? Does it not get that veggie is another term for vegetarian?
      3. Humanize it: Does the tone of your brand shine through? Make the experiences more engaging?
   2. Ask your customer for feedback, especially early on. Give your customers the option at the end of the session if they’d like to participate to help make the bot better!
   3. Learn from customer interactions and feedback.
   4. Generate smarter, more personalized interactions capitalizing on the growing number of cognitive services available
5. Iterate and Adapt. Rinse & Repeat.

Everyone loves a good conversation. Bots and skills are opening up ways to communicate directly and more conversationally with your customers – providing them a more natural experience. It also can give your business a deeper look into the customer experience – including their emotions and sense of urgency during the interactions. Bots and skills can help you provide more personalized experiences that create a more meaningful connection with your customers.

So, what if we’d stop chatting, and started coding?

Podcast: speaking with Movidiam on AI, Chatbots and Augmented Reality

I was recently invited to contribute to Movidiam's podcast and share my thoughts on the future of search, chat bots, Artificial Intelligence, the rise of Augmented Reality and their impact on marketing. If you regularly read this blog, you will be familiar with some of the concepts and principles that I am touching base on, but nonetheless you may be inclined to stop reading and start listening to my French accent:

Here is a link to that podcast.

The evolution of search through a modern consumer journey

Blog post originally published on State of Digital as part of a monthly column.

The IAB UK has just released their digital advertising spend report for 2016 and announced that digital advertising grew at its fastest rate for nine years – up 17.3% to £10.3 billion. The last time annual growth was higher was in 2007 (+38%), when Apple released the iPhone leading to the accelerated adoption of smartphones, the emergence of the app paradigm, etc. The reason I call out that historical reference is because I genuinely believe we are at an equivalently critical inflexion point with the rise of AI: it grew out of search and is about to transform marketing as a whole.

As a matter of fact, the historical powerhouse of the digital landscape, Search, has continued to grow by +15% to £4.99 billion, which reflects two major trends:

• First advertisers have (finally) realised that search plays a role throughout the customer decision journey and not just at the final conversion step
• Second, search is, in itself, evolving drastically to offer new capabilities and surfaces of engagement for advertisers to capture new intents, new behaviours, new usages

Following a modern consumer journey and its query paths

As disruptive technologies reshape the digital marketing landscape, brands are investing time and effort to remain relevant and top-of-mind with consumers. For years, search marketers have obsessed over bottom of the funnel activity for its seemingly higher CTRs and conversion rates, in part fuelled by last click attribution models. But most marketers today agree that it is essential for a brand to appear across all stages of the funnel to drive brand affinity and recall.

Research at Bing Ads now allows us to quantify and better understand the customers behind the clicks and where they stand on the customer decision journey (CDJ). Consumers tend to go through five distinct stages, which vary depending on the type of purchase: initiation, research, comparison, transaction and experience. Consumers are subject to these same five CDJ stages, although each step will vary in length and importance depending on the cost of failure, frequency, cost and complexity of the task, and the shopper type.

As consumers go through a given decision journey, search plays a pivotal role and different types of queries will be used throughout:

• Category searches: Include broad search terms that are not product or brand specific, such as shoes, running shoes, hiking shoes. These usually appear during the initiation, research and comparison stages but can also take part during transaction
• Tangential searches include queries that are related to a given journey, but not in the exact same category. For instance, if I am searching for running shoes, a tangential search could be about preparing for a half marathon, running training program or running equipment. These are similar to category searches in that they usually get searched on before a transaction
• Competitor brand searches: Include your competitors’ brands or products. This is an opportunity for your brand to appear through conquest advertising. For Nike, these terms could include Under Armour, Adidas, etc. As consumers get further along their decision journey they begin to hone in on their purchase and more specific searches occur
• Brand searches: Include a specific brand or product name. Nike is the brand and product searches could include Nike Pegasus or Nike Running shoes. Brand terms are often the best performing keywords helping drive the most transactions

Search drives awareness

Traditionally search has placed a premium on brand keywords. The Bing Ads research team has found new insights on the role of non-brand search terms within today’s journeys. Brand Impact Study on usage data from a major US auto retailer determined that 72% of brand ad clicks (which includes category, competitor and tangential searches) had a non-brand keyword precede the brand click. In other words, retailers who do not run non-brand keywords throughout multiple stages of decision journeys are missing out on a majority of relevant searches and leaving gaps for their competitors.

Additionally, consumers who are exposed to a brand ad on a category or competitor query were 30% more likely to do a branded search. On average they had a 15% higher conversion rate compared to consumers who were not exposed to the brand ad. Having a branded ad appear in category and competitor brand queries improved brand affinity and recall, and increased the propensity for future brand searches.

Another Bing Ads research completed in Q3 2016 with a leader in the automotive insurance vertical measured brand awareness and perception based on exposure to ads within the search results pages. The key learning from this study was that searchers on Bing who saw a branded ad for non-branded search queries showed a statistically significant increase in brand awareness, perception and purchase intent. After being exposed to a branded ad, searchers indicated a 24% lift in unaided awareness, 28% lift in purchase intent and a 30% lift in viewing the retailer as a market leader. Brand awareness further improved when searchers clicked through on the ad and were taken to the brand’s landing page. In the study, consumers who were merely exposed to an ad without clicking on it, self-reported as more likely to take an action or next step

Although search marketers have long assumed some level of brand awareness in search, it is the amount of brand awareness shown here that proves impressive and indicates the importance of staying present throughout each stage of a journey

Search informs and educates

As consumers go through the initiation phase of the consumer journey and begin to do more research and consider purchasing, search can help with the decision-making process.

A Forrester Consumer Technographics study called out that 60% of consumers will use a search engine to find the product they want and 61% will read product reviews before making a purchase. And one of the most trustworthy sources in this research and comparison phase is none other than search engines. Furthermore, consumers consistently rely on search engines, more than any other source, as a reliable place to research about brands, products, or services that they are considering buying. According to Forrester, 49% of consumers reported that they rely on search to inform purchase decisions, and 19% of respondents identified search engines as the most influential source in driving their decisions.

New search experiences, new marketing expectations

The evolution of the consumer behaviours outlined above and the reaction from marketers by investing throughout the different stages of the consumer journeys are a sign of maturation for a discipline that is merely 15 years old. And yet, new search experiences powered by AI are coming to the fore, and are likely to further disrupt the model.

It has been well documented that search is no longer just a search box on a webpage with a list of links. It is literally breaking out of the box; it is changing in some dramatic and exciting ways. Once a function that used to be limited to a text box on a specific site has slowly been integrated into more and more of the technologies/devices, apps and sites we use every day, from phones to gaming consoles. As such, Search becomes more pervasive, but also more personal and predictive.

Conversations as a Platform is a great embodiment of how the Future of Search makes sense of these 3P (personal, predictive and pervasive). The rise of messaging apps where consumers now spend most of their time and technological advancements in NLP, AI and machine learning are creating rupture that is similar to what we have seen in decades past with the App paradigm. Conversations as a platform unlock a more human, personal way to discover, search for, access and interact with information. This new platform will enable us to interact with devices more intuitively, using natural language conversations, evolving us from mechanical keyboard and mouse to touch and beyond.

Search will never again be a constrained to writing in a search box. Instead, search will be a partner that can listen and communicate in dialogue with a consumer on any platform and any device. Thanks to new, more natural interfaces, voice search is becoming increasingly possible and accurate. We imagine a rich ecosystem of conversations, ones that include: people to people, people to your personal digital assistant, people to bots, and even personal digital assistants calling on bots on your behalf. That’s the world that you’re going to get to see in the years to come.

We have already covered the current rise of voice search as a result of mobile search adoption, but it is likely to further accelerate as search becomes more pervasive and turns your TV Set, you fridge or your car into a search box. ComScore predicts that by 2020, over 40% of the queries will be voiced rather than typed. It is worth noting that these experiences are often screenless which further disrupt the preconceptions that too many marketers have when they think they have nailed their search strategy.

How do you optimise your web presence for voice search? How do you market to chatbots? How will we see consumer intent evolve in these new experiences? We already seeing that 16% of searches every month had never searched for before. We also see that 25% of the clicks on the Bing Network are from searches not happening on Google… So, the versatility of the above-mentioned consumer journey is likely to continue to increase. And what an exhilarating evolution it is!

Podcast - glancing at Advertising Week on Chatbots, AI and Machine Learning

Back in March, as my team took part in Advertising Week, I was honoured to be invited by Jason Miller, from the Sophisticated Marketer podcast, to deep dive into the evolution of Artificial Intelligence, Chatbots and what it means for marketers and how they can utilize the technology now and prepare for the future:

Live from #AWEurope: Chatbots, AI and Machine Learning

The story of an American dream

A few months back, I attended a story-telling class during one of my visits to Seattle. The team at Assembly Intiman provided us with precious advice and guidance on how to bring anecdotes to life, to insufflate in paragraphs sparks of emotion that will take your readers or your audience on a journey. The inspired us with vivid examples from The Moth. They explained how by bringing the story to life through with personal examples and necessary details, whilst never losing sight of the greater purpose, you could turn a story line into a richer experience. One of the exercises consisted in randomly selecting a topic and elaborating a plot on the back of that situational teaser... I picked up: "Your first crush". Here is how it ended up, after a few minutes of reflection:

"Close your eyes, and imagine. In a few months, my wife, two kids and I will move to Seattle, WA. The winter will be over and a truck filled with our boxes will be parked in front of the house we have just bought. The last reminiscences of snow will be still melting in the shade of the large evergreens. I have taken a big job at Microsoft, and I am about to embark on a career path that should be rewarding in many ways... A dream. Some would even call it the American dream.

But let me share a French dream. Her name was Nathalie. I was a young teenager attending the local high school in a small town back home, in the Alps. I was then relatively good academically, certainly good athletically and socially... well, I had friends. I did not drink. I did not smoke. And in that picture of perfection, the only thing missing was The Girl.

The year before, though, I got to know Nathalie during drama classes. As others performed the year-end play, we were exchanging connivent smiles in the promiscuity of the improvised backstage. No words. Just candid glances and smiles. As the summer broke, we parted ways for the holidays, but as our respective birthdays came, letters made their way to our mailboxes. Innocuous pieces of paper, peppered here and there of clumsy attempts to infer some sort of feelings. As many mountaineers, I am probably better at dealing with the grandeur of my surroundings than with the depths of my inner self.

Nonetheless, as school resumed, I rushed to the school gate to see in which class I was, and I could not avoid searching for her name, secretly hoping we would be able to continue that flirt between maths and geography. And she was there, her name just a few rows above mine on the pupil list of class 7D. I remember smiling at that discovery... Until she arrived and went straight in, without a look for me.

Nathalie the perfect girl. She was smart, and beautiful. Her eyes were green with hazelnut shards scattered around her iris as if a glass marble had been crushed into her eye. It is funny that thirty years later I still remember that detail because for months I was unable to look at her. I stared at her, timidly, but was unable to make eye-contact. I spent hours listening to lectures, half-present, with my gaze wandering her golden locks and my respiration slowly getting in tune with her own. I have looked at her back for hours, for days, for months in fact but without the courage to ever face her. But then came the warm, inspiring month of April. Encouraged by the bourgeoning nature (and the encouraging whispers of friends), I dared to call her out as she strolled across the schoolyard:

- Hi?
- Hi.
- Do you want to go out with me?
- No!

She had replied without the hint of an hesitation. So, as a good French, I turned around, shrugged and went back to my friends as if nothing had happened. Disappointed, deeply hurt, but utterly dismissive of the whole situation (as you rightly do when out of touch with your feelings). I was a teenage boy after all. What would you expect?

But she came running at me. Not Nathalie, her best friend. She confessed that, although Nathalie fancied me, she did not want to hurt one of her own friends who also had feelings for me. Yes, that made sense. Somewhat. Deceived by my own success... That was good enough for my ego, even if disappointing for my libido.

That same best friend called me back a few years later. She had found my number in the phone book and although we had respectively moved to different parts of town, she thought it was a good idea to let me know that Nathalie was still very much into me, but that she could not dare to reach out, after the schoolyard anti-climax. So, in a non-act of emotional courage, I grabbed my phone immediatl... a couple months later, and called her to invite her out. We went to the cinema, spent the afternoon together and as I walked her home our hands touched. An almost imperceptible brush of skin. We looked at each other. Eyes in eyes. I tilted my head. She titled hers. Got closer, and as our lips touched each other... Nothing.

Just like that, years of fantasy vanished in a brief exchange of saliva. So as I reopen my eyes, and look at these boxes that we have just finished to pack, I cannot avoid but think that my American dream may well just be another fantasy. But, you know what? I don't really care, as it is still very much worth living these dreams."

Connected cows and the greener pastures of big data

Blog post originally published on State of Digital as part of a monthly column.


Remember the cringing router sound that accompanied your early internet connections? Being a 40-year old father of two, when I hear that screeeeeeeechhhhhhhhhh I can’t avoid feeling a smile pop on my face. That sound reminds me, with a touch of nostalgia, of the cry of a newborn. It makes me smile as fond memories have overshadowed the dirty nappies and sleepless nights.

To a certain extent, that sound was indeed the cry of a baby internet, when browsing was a commitment and a flaming logo the pinnacle of creativity. The worldwide web has since grown up. A lot.

Small steps towards big data

Looking back, that dial-up buzz was literally the first signal in our digital footprint. We signed on. And in that very moment big data spun to life.
What were we learning from those early signals? Not much. We knew how many people were online and roughly where they were. But as the web of documents developed, search engines arose and our ability to understand people increased. We started to know what you wanted. Think about it: you probably tell search engines things you wouldn’t tell your closest friends. We understood that your inputs into a search engine were, on a personal level, an expression of your desire, and on a global level, an expression of the world’s consciousness. The so-called Zeitgeist.

Although I have spent the last 10 years in the realm of online advertising, in today’s article I will focus less on search marketing, and more on the information infrastructure and machine learning that Bing is part of, looking at how this is influencing our future. What are we doing with our data footprint? Over the course of the last year I have asked many people across Europe how the idea of data collection made them feel. By large, the response was discomfort and hesitance. Until provided with more perspective.

The heights of data complexity

So let’s get back to our story. In order to understand the complexity and depth of the data infrastructure that we are part of, let’s contemplate what has changed since the emergence of the first search engines. With each of these four changes and associated amount of data surge that came with it, you need to visualize a growing mountain.
First is your search habit. From a few searches per day to multiple searches per hour, we are now searching constantly, and not just you but also the billions of people who got online in the recent decade.

Second is your search access. Most of us had access to a desktop computer 20 years ago. But just one. A grey, cold box, sealed to a desk. We certainly couldn’t put it in our pocket and take it with us to a party. It is not just computers, think about all the devices you own which are harnessing computing power: laptop, tablet, smartphones, TV, but also your car and now your fridge.

The third big change is your search expression. You have gone from using basic computer commands, with amp signs and inverted commas, to using more human a language. You’ve gone from asking “what” to asking “why…” and “how to…”. In fact we have seen the growth of queries starting by Why being three-fold the growth of What queries, which means we are no longer looking for information, we are looking for answers. You’ve layered sequential searches on top of these, in a complex web of intents.

Finally, the integration of search with other infrastructures has also changed. A search engine used to be an isolated service. Now it’s plugged into the social graph. This means that several points of contact are linked and with them a flurry of new signals, millions of them that only a few super-computers are able to capture, organize, model and render. Search engines are the database of intents, and social networks are the depository of sentiments. We have developed the ability to process, analyze and understand these two humongous, historical and real-time information sets together.

The search crystal ball

We can understand your sentiment for certain events or entities, estimate popularity trends, as well as predict outcomes of future events. Microsoft has developed a program called Bing Predicts which combines and models all the data signals we can find, and comes up with incredibly accurate predictions. We initially explored popularity-based contests like American Idol, for which the web and social signals are very strong and highly correlate with popularity voting patterns. Bing Predicts could accurately project who would be eliminated each week during American Idol and who the eventual winner would be. Just by using all of the signals that are out there.

Getting more complex, we turned to sporting events and even world political challenges. During the World Cup in Brazil, our team predicted accurately with 100% accuracy the winners of the final elimination round. During the last year Rugby World cup, we had 80% accuracy across the tournament. Surprised? In order to successfully predict a sporting event outcome, the number and type of signals we incorporated quadrupled from what we used to predict a basic popularity event like American Idol. This is because we recognize that popularity alone does not predict whether a team will win – Sorry for the fans. A fan base has however special insights into the abilities of their teams, and those fans are having constant discussions about their team. This is called the Insider Knowledge. We up-weighted their knowledge against player and team stats, tournament trends, game history, location and even weather conditions. This is how we were successful in our predictions.

We finally turned our attention to political events, and in particular the Scottish referendum two years ago. The process and results were presented at TEDxSuzhou.


We were and are predicting the future. Can you imagine a business need that this kind of prediction can answer? Of course you can! We’re experimenting right now with predicting the upcoming trends in fashion, in automobile, in technology – so we can help our advertisers make smarter business decisions.

So we saw how predictions can play a role in entertainment, sport or business, fine. Fine, until we find a way to make this kind of data infrastructure even more meaningful, at a society and mankind level. What can we do with this capability that goes beyond entertainment and the novelty factor? Can we use our big data to make a meaningful impact on society?

Up close

All of this is exciting on a global or country level. When we’re talking about millions of inputs, it’s no wonder you can make predictions and have an impact like this. It is just a massive sample size. What about bringing this big data infrastructure to a personal level? Is it possible for a machine to learn so much about you that it can accurately predict your next move? Or predict when you will need something, and provide it? That is the promise behind digital personal assistant like Cortana.
Cortana is not only on Windows Phone but also Android and iPhone. And since the release of Windows 10, she’s even on your desktop. As outlined in a previous article, you set up Cortana with some basic info about yourself, then use her to help you with things like scheduling and reminders and web searches. Before you know it, Cortana is spontaneously sending you an alert to inform you that you should leave the office now to be on time for your next appointment in Farringdon, because she found some congestion on your normal route. It doesn’t take Cortana long to learn so much about you that she can predict your next move and offer assistance.

A new layer of data in your coat

While our mobile phones aren’t exactly wearables, we sometimes behave as if they are, keeping them on our body no matter where we go. With wearables, two important things converge: big data infrastructure and your expectations.

When you hear “wearables”, you probably think of a smart watch or one of these fitness bands. But to go back to my introductory analogy, these are just the first baby steps towards the full potential of wearable and how that technology will be able to enhance our capabilities, as individuals or as professionals. Think about it: wearables can capture and communicate signals about your location, your manner of travel – whether you’re on foot or in a car – time of day, most recent queries, usual route home from work, the weather, your physiological state, etc.

So for instance, if your wearable identifies that your hydration is low, it could prompt a notification that factors in your location, whether you’re moving, what time of day it is and therefore whether the nearby branch of your favourite coffee shop is open. It could even cross-reference this with your earlier interest in gingerbread lattes, and the fact that it is raining, and direct you to the nearest open coffee shop with plenty of indoor seating and gingerbread lattes on the holiday menu. Your wearable might even send you an alert for a coupon the coffee shop is offering.

Greener pastures ahead

As the wearable technology grows, your expectations for your experience with technology in general will change. And that is for the better. After all what the point accumulating data points like hoarders unless you do something greater about it. And if I have learned something about the internet, is that it is a fertile ground for creative usage of untapped opportunities.

I am from the French Alps where I spent most of my summers walking the mountains with my grandmother. She used to herd cattle in these alpine pastures and she was telling me stories about how much each of her cows were almost like members of her family. They had names, and she could tell when something was wrong with any of them.
These days are gone. Nowadays a farm is no longer taking care of a small dozens of cows, but hundreds. The personal relationship of each animal is no longer an option. The story of the connected cows started with a farmer in Japan who was exhausted with the effort of figuring out the exact time his cows were fertile – because it is a very short window, only 12-18 hours every 21 days, and it happens usually between 10pm and 8am. Of course knowing this precise time of estrus would give farmers a chance to successfully inseminate the cows.

These are farms with hundreds of cows – you can image what a nightmare this would be to keep track. Could technology help? A farmer in Japan asked Fujitsu for help. Fujitsu consulted with some university researchers and they came up with this idea of putting wearables – pedometers – on the cows, and providing the data to Microsoft Azure, in the cloud, for analysis and alerts that go straight to the farmer’s smartphone.

It turns out that when a cow is in estrus, she paces. The number of steps she is taking increases tremendously, and this data alerts the farmer to the right moment for fertilization. The connected cow project has been 95% accurate – and that 5% where it misses the mark turns out to be when the cow actually skips the farm and goes missing.
Not only is this wearable incredibly accurate, it also helped the researches discover that there is an optimum window for fertilization if you’d like a female or if you’d like a male. With 70% probability, a farmer should fertilize in the first half of the estrus window if he needs more milk cows or if he needs more bulls. But it does not stop there… The Fujitsu researchers were able to also correlate pacing patterns with increased risks of genetic diseases and pathology.


It is amazing what data can tell you, if you know how to look at it. Sometimes creatively! This is the joy of data infrastructure. We can do wonderful things in the world when we collect, analyze and render the data that’s available to us. Microsoft is on the leading edge of this, with products like Power BI, Azure, our cloud platform but also Bing our search engine and its machine learning capabilities which can make sense of the millions other data points that come together to make big data smart, useful, creative and – yes – joyful. And you, what was the last time you found a creative inspiration in your data set?

The cycle of cliched cliches

The core essence of this blog has been, from its inception, the desire to speak to the cultural differences that would strike me in my daily encounters. It was also meant to bust, if not combat, cliches that so often minimise the enrichment inherent to cultural differences. Over time, the number of articles has reduced, because alterity became slowly normality, and it was getting harder to get surprised. That assimilation did not prevent me from being torn by a Franco-British Paradox, and from sometimes being reminded at my core where my roots were. 

Back to the roots.

Every now and then you may be tempted to go back to your origins and to embrace the stereotypes associated to your homeland. Only to toy with them with tongue-in-cheek references, that turns cliches on their head. I have had my eyes for years, 7 years in fact, on talented Scottish rider Danny MacAskill. I have always found that this athlete was capable of bringing a touch of intellectual poesy in his performance. He is in control of the image that his films convey, and there is always more to the jumps and rail slides.

In his latest release, A wee day out, Danny gives us again that perfect twist... He dives into the cliches of a dated country life, with its lot of high tea, scones, hay bales, green pastures, steam trains, stone walls, small crowds and deserted stations. You could almost expect Postman Pat to pop by with his cat and his red van. But the heights of Britishness in this setting is also creating a perfect stage for taking mountain biking to new heights. But rather than dwelling too much on it, how about taking a ride in that delightful Great Britain:




More from Danny MacAskill:

• Way Back Home, a tour of Scotland attractions
• Inspired bicycle, the origins
• Imaginate, an artist toying around
• Epecuen, when a devasted city ruin becomes a playground
• Cascadia, or how a colourful Spanish little village  

Lifting the lid of the search box: Excel tips for search marketers

Blog post originally published on State of Digital as part of a monthly column.

The motivational quote encourages us to “think outside the box”… But for once, I want you to think behind the box. Have you ever thought about the cogs, bots, artificial intelligence and other algorithms that are getting in motion every time someone hits the “search” button? The computing forces in motion are staggering and the amount of data that is crunched and produced is even more. Lines and lines of data, with in each of them a little piece of insight that could make all the difference in your marketing strategy. If only… If only you could overcome the spreadsheet blindness and see them.

Several of my posts in State of Digital are about visionary topics: the rise of AI, predictive modelling applied to individuals, digital transformation and the lessons that search engines could give to other businesses… This month, I was keen to be more tactical, and explore a little bit further the capabilities of the beloved Excel when it comes to making sense of large amount of data.

Picture that emotion

Advertising is all about creating an emotional bond between a brand and consumers. And since the infancy of our discipline, both marketers and agencies have been trying to visualize this connection. Focus groups, vox pop, surveys, research… They all had a go at it. But personally I never got fully satisfied by them. As a matter of fact, I don’t believe that people give you access to their core beliefs when prompted over the phone or in the street.

Then Social Media arose. And you must admit that they are simply great for that corollary use. After all, what are these platforms but the depository of our intimate sentiment?

So obviously you can have a quick peek at Facebook and look up for the fan pages for your brands. That is straightforward as people are overtly expressing their sentiment towards the brand through these pages. They revive dormant products and re-energize old brands. They also virtually stone others to death.

But what if you are interested in emotions more deeply engrained in consumer minds? And looking at visualizing them? Personally I used to visit photo sites like Flickr or Instagram and run a query on a given brand. Type Bailey’s and you will get hundreds of pictures showing up. The resulting mosaic is fantastically enriching. You will of course see people sipping their favorite liquor but also loads of pictures of dogs and cats named after that brand. What a better proof of brand advocacy than to name your beloved pet after a trademark? Or to tattoo the swoosh on your hip or a Harley Davidson eagle on your shoulder?

Think outside the (search) box

This digression meant to make a point: we can find consumer insights everywhere. You just need to be a bit creative to see them. I remember discovering Bing Ads Intelligence tool, and having one of these ah-ah moments.

This tool is by essence a brilliant search marketing tool that enables search marketers to make more informed choices when creating a campaign: based on historical and forecasted data from Bing search queries, it provides you for any keyword with traffic volumes, demographical and geographical information about the searchers, even indicative CPC for the different positions in the auction… And it is free!

If you have not downloaded it yet, I would strongly recommend you do. Even if you are not working in search marketing. In fact, I should say “especially if you are not working in search”.

This little freeware can indeed help a lot of marketers out there. As a matter of fact, in the current economic climate, when costs are cut to their bare minimum, can you afford to research what your audience’s actual demographic profile is and where they live? Are your pockets deep enough to run a regular research to audit your brand awareness against this audience?

Search engines are for finding

Digital expert and author John Battelle once qualified search engines as the database of intents. With over than 28m unique users in the UK alone, Bing offers you a statistically relevant sample of these intentions. So why not use the Bing Ads Intelligence to run your own piece of research? It provides you access to actual logs, so you can use them as proxy for your consumer intents. How many consumer have searched for your brand in the last month, and how many have for your competitors? That will provide you with a good indication of your brand awareness. Did they search on a PC or from a mobile device? Have queries increased after your latest local TV campaign? Was your regional billboard campaign efficient? Where are visitors searching from? London, Liverpool, outside the UK?

A lot of these questions can be answered and visualized by pressing a button in Excel. Three actually.

Three Excel features to make your data click

The first click should be on Bing Ads Intelligence. As said, you just have to enter a word (a brand for instance) and choose what you want to know: demographics, device usage… And since you are in Excel you can rapidly turn the data into a compelling visualization like this:





Now knowing more about who searched, my second click would be on another Excel free tool, PowerMap. If you are running Office 365, this is a native feature in your ribbon. If you are using an earlier version, you can download that add-on for free. Power Map is a 3D visualization add-on for Excel for mapping, exploring, and interacting with geographical and temporal data, enabling people to discover and share new insights. It allows Excel to automatically chart the search data to see where the customers who are searching for you are located, where your revenue is generated, where prospects are congregating, etc. I particularly like that if you have your data points for several periods, you can turn your map into a little video which will illustrate the evolution of your chosen KPI over time. The brilliant thing in this feature? Excel does all the work for you. Leveraging Bing technology, it recognizes the geo-information present in your spreadsheet and plots the relevant data points on a map accordingly.

Of course, neither tool will ever replace a full professional research or extensive monitoring application, but these are valuable indicators and visualizations for a superior desk research. Personally I find these free tools simply brilliant for those who are always looking for innovative ways to increase cost-efficiently their agility, identify new opportunities and niches.

Who, where and now when?

My third click would be to dig into the question of time and create a day parting heat map using the conditional formatting of Excel to rapidly identify when is my marketing sweet spot. How do you do that? Conditional formatting is a brilliant function that allows you to automatically change the format of a cell, based on values and parameters that you dictate. So let’s say that you have a search account report that you downloaded from Bing Ads with Hour of Day as the unit of time. It would originally look something like this (painful to watch you would reckon):


Highlight each column individually and use conditional formatting. We then see a picture of account performance by time of day, and a heat map emerge. I’ve left the Clicks, Impressions, Cost, and Converted Clicks columns untouched right now, as I prefer to display those with a different formatting.  The red vs. green distinction is a little too arbitrary for these numbers, so I go with a bar graph to show them instead.



Now this chart tells us a story. It portrays the ebb and flow of our daily traffic, and it clearly shows us both where we can pull back on our bids with day-parting modifiers – early morning from midnight to 4 AM – and where to be more aggressive. We have a real lost opportunity here starting at 4 PM to 8 PM to increase our bid modifier and gather even more traffic. Our Average Position and Average CPC metrics let us know exactly when our competitors are ramping up their spend, and the hours in which we should do the same… until about 10 PM, when our conversion rate drops again.

This report takes less than five minutes to pull, and no custom modifications were made to these rules aside from highlighting each column individually and selecting the right formatting.  Day-parting is the easy and obvious use for this kind of analysis, but it can also assist in ad reviews, geographic performance… really, in any case that leaves you staring at a spreadsheet for hours on end, mining for insights.

To go further

So you’ve manipulated and made your data more digestible. It has worked and you have identified some interesting data points but now you want to tie this in with the rest of your company’s data sources and tell a story. That is when tools like Power BI become handy. It indeed allows you to connect your data with a wide variety of data sources, from local on-premises databases to Excel worksheets to cloud services. Currently, over 59 different cloud services such as Facebook and Marketo have specific connectors, and you can connect to generic sources through XML, CSV, text, and ODBC. Data.gov for instance is a great base of public domain data set that you can mash with your own.

So what can this mean? How about comparing the demographic profile of your audience with the latest census? Identify the correlation between your customer engagement and the weather variations? There is so much data out there, it is really just about letting your creativity unleash the power of insights. Or when data becomes clear, and actionable.