November 18, 2019

Why does Google combine deep learning with a knowledge graph?

A lot of people in the knowledge graph and ontology communities see an ontology as something quite different from AI. When I (Robin here, Similar.ai co-founder) studied AI and Computer Science at university in the early 1990s, we studied both knowledge representation and connectionist models of the brain. It’s useful to understand why Google combines both in order to understand SEO better.

Knowledge representation is known nowadays as “good old fashioned AI”. It grew out of library sciences and the philosophical discipline of ontology, or the study of being, and dealt with how to classify experience.

Connectionism used networks of very simple information processors, to learn from examples, implicitly, rather than from an explicit instruction set, recipe or program. Connectionism grew into what we know as machine learning, and specifically neural networks and deep learning today.

2019 has seen an explosion in language models like BERT which help computers understand the meaning of words and sentences and match to the meaning of other words and sentences.https://giphy.com/embed/l0MYKdy90Zn2dDgpq

Understanding

Which paradigm (connectionism or good old fashioned AI) reigned ebbed and flowed through the decades. In the 1980s and 1990s expert systems were thought to be the way to go, but ultimately their inability to learn from data and their brittleness to nuance meant they failed to scale. Needing to explicitly teach every rule and irregularity is hard, especially when most people have little usable self-knowledge of where their competence comes from.

Deep learning has conquered domains of what we used to think of as perception and a lot more. Deep learning networks which can translate between Chinese and English and hundreds of other language pairs understand not just those languages, but understand something of language too. Deep learning models which can correctly identify more than 10,000 different types of thing from any image, understand not just those things, but something of the nature of vision itself.

Machine learning has been powered by algorithmic breakthroughs, huge compute power and a world wide web, an enormous trail of content, the footprints of all human digital activity.

But despite this level of nuance in grasping the meaning of language and images, deep learning so far has struggled with the everyday commonsense knowledge and shared vocabulary learnt mostly through play and being in the world itself. It does not do well at reasoning or understanding implicit knowledge.

Perhaps one day, when machines actively participate in the world instead of inhaling the exhaust of human activities, they too will glean understanding about everyday things. It’s hard to learn just by representing knowledge of the world without ever using that knowledge representation to intervene in the world.

Commonsense knowledge (Image by Jonny Lindner from Pixabay)

Intelligence is multi-faceted and many of the most interesting breakthroughs combine both neural networks and symbolic logic. For instance, when DeepMind’s AlphaGo beat the world’s best human player at Go, it used a combination of deep reinforcement learning and a “good old fashioned AI” approach. Similarly, Google has building its own knowledge graph of every day common knowledge. It also understands when sites publish their opinions and statements using semantically marked up HTML, such as the vocabularies from Schema.org.

But so much more of the web is unstructured natural language, images, documents and videos. To make sense of this — the world’s information — is Google’s mission. To do that, it can not only depend on semantic mark-up.

We can expect an explosion of semantically marked up data in the coming years, but it seems probable that the majority will remain in a form that currently only carbon-based bipedal sentient beings can comprehend, since they remain the target audience. Similarly, it would be strange to expect users to type their queries in Sparql (an ontology query language) or to speak that into Google Assistant, Siri, Alexa or Cortana.

Instead, search engines need to do a better job at interpreting queries, typed or spoken, to understand the intent behind them. Deep learning is an unmissable tool to do this. Google couples its knowledge graph with increasingly deploying more and more deep learning approaches to interpret the intent behind search queries and, to understand what web pages mean, and to match the two. Both approaches are necessary and both add value.

It may be that at some point in the future, a complete end-to-end approach to learning useful query understanding and web page interpretations is possible from scratch. Until then, most real world AI tasks will be improved by combining something like a knowledge graph and with machine learning’s deep perceptual abilities.

For SEOs, optimising for the needs of the user is going to work best. Rather than focus on the latest Google update — of which there are many every month — focus on what they are aimed at achieving.

It’s hard to keep up with all the different updates that Google makes and whether they use deep learning models like BERT, a knowledge graph, human curators, white listed sites or some completely different approach, you can be sure that they all have a common goal. They help Google work out how best to match pages to what users are looking for with the minimum amount of work.

If you create pages users are looking for, make it easy for Google to understand, match and create unique content for them, your site is going to get more impressions, clicks and revenue.