Skip to content

Archive for

Search = Recommendations

It is good to see that some artificial distinctions that have served to hamper progress in delivering truly intelligent computer interfaces are increasingly melting away. In particular, recommendations, when broadly defined, can beneficially serve as a unifying concept for a variety of computer capabilities. In The Learning Layer I defined a computer-generated recommendation as:

A recommendation is a suggestion generated by a system that is based at least in part on learning from usage behaviors.

In other words, a recommendation is an adaptive communication from the system to the user.

And I had this to say about search:

By the way, the results generated by modern Internet search engines are in practice almost always adaptive recommendations because they are influenced by behavioral information–at a minimum they use the behavioral information associated with people making links from one web page to another. This capability was the original technical breakthrough applied by Google that enabled their search engine to be so much more effective than that of their early competitors.

This feature of contemporary Internet-based search also provides a hint at the reason that the users of enterprise search whom I have talked with over the years have been so often underwhelmed by the performance of their internal searches compared with corresponding Internet versions. Historically, there has been little to no behavioral information embodied within the stored knowledge base of the enterprise, and so search inside the four walls of the business has been basically relegated to the sophisticated, but non-socially aware, pattern matching of text–similar to the way Internet search was before Google. Without social awareness, search can be a bit of a dud.

That’s why, as I mentioned in the previous blog, the computational engines behind the generation of search results and recommendations are inevitably converging, and we are therefore witnessing a voracious appetite of search engines for an ever larger and richer corpus of  behavioral information to work with—Google’s new “plus one” rating function being the most recent example.

On this same note, I just happened across this brief, recent write-up that struggles with categorizing a start-up, exemplifying the blurring of search and recommendations, and finishing with the point that, “. . . Google is also increasingly acting as a recommender system, rather than just a web search engine.” Indeed—now on to bringing this convergence to the enterprise . . .

The Recommendation Revolution, Continued . . .

That “hidden in plain sight” IT revolution of ubiquitous, adaptive recommendations continues to accelerate. Jive Software, a leader in enterprise social software, has just acquired Proximal Labs to bolster their machine learning, recommendation engine, and personalization capabilities. Given that Jive is reportedly preparing for an IPO, this move is a major endorsement of the key theme of The Learning Layer–that adaptive recommendations will be a standard part of enterprise computing. It also is another strong signal that, as I recently wrote, we are going to rapidly witness a standard, adaptive IT architecture emerge in which a layer of automated learning overlays and integrates the social layer, as well as the underlying process, content, and application layers. And I would be remiss if I didn’t mention our own initiative for bringing the learning layer to Microsoft SharePoint and other enterprise collaborative environments, Synxi, which includes a suite of auto-learning functions such as knowledge and expertise discovery, interest graph visualizations, and recommendation explanations.

Another recent and important signal of the recommendation revolution is Google’s recently announced “+1” function, an analog to Facebook’s “like” button. Google will use this tagging/rating function in generating responses to search requests. Google was the pioneer in using behavioral information to improve search results, and this is just the latest, and most dramatic, step in increasingly relying on behavioral cues and signals to deliver personalized search results that most hit the mark.

That is why, as I discussed in the book, responses to search requests should really be considered just a certain kind of adaptive recommendation, one in which more intentionality of the recommendation recipient can be inferred by virtue of the search term or phrase, but that is otherwise processed and delivered like any other type of recommendation. And it is why search processing and recommendation engines will inevitably generalize and converge.

Stay tuned . . .

Creative Combinatorics

The math of innovation is combinatorics. We create by building on what comes before, or more precisely, by decomposing, abstracting, recombining, and extending what comes before. The more combinations that we generate, the greater the creative potential. The complementary challenge is to effectively and presciently evaluate the combinations.

At the macro level, these combinatorics-based dynamics of innovation have often been exemplified by what is known as “clustering economics.” New York, for example, has had self-reinforcing advantages in financial markets; Silicon Valley has enjoyed unique, self-sustaining advantages in emerging technologies. But the advantages of the physical clustering of these and similar examples really boil down to advantages in combinatorics. The innumerable additional encounters, discussions, collaborations, and competitions due to physical proximity lead to massively greater idea combinations than would otherwise occur. And where there is superior infrastructure to evaluate and nurture (e.g., wide-spread, accessible venture capital) and implement (e.g., flexible, risk-taking resources) the ideas, the combinatoric advantages inevitably lead to extraordinary value creation.

Learning layers obey the same math. They create value directly through enhancing collective learning, but the ability to amplify the power of combinatorics is at the heart of their value proposition. The learning layer is an engine of serendipity driven not by physical proximities, but by virtual proximities within multi-dimensional “idea spaces.” Its aim is to bring to our attention useful affinities and clusters among ideas and people of which we might not otherwise be aware. A learning layer can be considered an example of what the cognitive scientist and author Steven Pinker calls discrete combinatorial systems, which, as I discuss in The Learning Layer, occupy a very special place in nature:

We know of, however, two discrete combinatorial systems in the natural universe: DNA and human language. Not coincidently, these systems have the unique capability of generating an endless stream of novelties through a combinatorics made infinite by the application of a kind of generative grammar–a grammar comprising a recursive set a rules for combining and extending the constituent elements.

The discrete combinatorial nature of the learning layer is yet another reason it occupies a special place in the realm of systems. In combination with us, and by continually learning from us, it can engender ever greater streams of valuable, recombinant innovations. Of course, we still need the proper environments to nurture and implement the best of these innovations, and that’s a challenge we need to really be focusing on because somewhere in that multi-dimensional innovation space a very important creative combination impatiently awaits . . .