Skip to content

Posts from the ‘Recommendations’ Category

Enterprise Personalized

The rise of machine learning-based personalized discovery features over the past few years is one of the biggest stories of IT. The statistics are truly staggering. For example, even as of several years ago, over 30% of Amazon’s sales were reportedly due to their personalized recommendations—the figure is no doubt even higher now. LinkedIn has reported that fully 50% of their users’ connections, group memberships, and job applications are driven by their personalized discovery features. And 75% of what people watch via Netflix is due to personalized recommendations. In addition, of course, targeted advertisements can be considered a form of personalized recommendations, as can personalized search, both of which have largely replaced their non-personalized precursors.

So in the consumer world automatic personalization has become an indispensable feature for users and a competitive imperative for providers. What about in the enterprise? Not so much—until now, that is. I wrote The Learning Layer to lay out the path toward making adaptive, personalized discovery a core feature of enterprise IT, and we at ManyWorlds are excited that our Synxi-brand technology is now making that vision a reality!

We are delivering adaptive discovery apps for the major social platforms that continuously learn from users’ experiences and apply this learning to provide users with real-time, personalized recommendations of knowledge and expertise, and which are sensitive to the context of their current activities. Even better, we also have connectors among these apps that extend a layer of learning across platforms. That means users can receive cross-contextualized and personalized recommendations of knowledge and expertise from one platform (e.g., SharePoint) based on what they are doing in another platform. And finally, we have booster products for enterprise search that enable search results to be personalized and/or additional personalized content to be recommended based on the context of a specific search result. That provides users, for the very first time, an enterprise search experience that tops the search experience internet search providers can deliver.

These learning layer technologies are collectively leading toward enterprises becoming truly personalized. And an enterprise personalized is an enterprise that is more productive, as well as being an enterprise that is more compelling to be a part of and to work with.

Just the Facts Ma’am?

Now that Siri has a bona fide competitor, Google Voice Search, a bit of a kerfuffle has emerged with regard to personalities or lack thereof of these assistants. While Siri strives to project some personality by being conversational and peppering her responses with a bit of whimsy, Google Voice Search is all about just giving us the facts. Each approach has advantages and its vocal adherents. And as the systems’ capabilities leap-frog one another with each new version, the latest incarnation of Google Voice Search seems to have gained some speed and effectiveness advantages versus the current incarnation of Siri. Of course, both of these incarnations promise to be fleeting given the pace of the respective development cycles.

Although Google labels their product “search,” the functionality has clearly already morphed more generally into a recommender—i.e., providing suggestions given a context of various of kinds. This trend is a reflection of a generalization noted in The Learning Layer—plain old search is really best considered just a recommendation in which the context is of a particular type, i.e., a search term provided by the user. The inevitable next step in general-purpose recommender technology is delivering “meta-recommendations”—that is, explanations as to why the recommendation was provided, particularly when an explanation is specifically asked for by the recommendation recipient. A capacity for a limited degree of explanatory capability has already been incorporated into the Apple and Google gals to some degree.

Then comes the really interesting advance—making the recommendations and even the explanations adaptive to the user.  That is, learning from her experiences with us to adapt her recommendations and explanations accordingly. Which is followed by one more short step in which aspects of her overall personality become adaptable to us and our particular circumstances as well. A little humor when called for, a bit of sympathy at other times; and all the while learning as to what works best and when, and tuning accordingly. I’ve got a feeling that at that point, which at the current pace of innovation, is not far away, always just providing the facts will be perceived to be somewhat stilted behavior—coming off like a cheesy movie version of AI of the 1960s.

So my guess is that there are times when, indeed, we are all Joe Friday’s, but more often than not we’ll welcome more than just the facts.

Learning from Netflix

Netflix recently published a blog that lays out some of their experiences with their recommender system.  The blog is notable in that Netflix was one of the pioneers of e-commerce recommendation engines, has one of the most famous recommendation engines, and packs a lot of details and good insights into the blog.

Here are a few takeaways:

75% of what people watch via Netflix is due to recommendations.  And given how impressively recommendations drive sales for businesses such as Amazon, it is not surprising that sophisticated recommender systems are becoming the norm in e-commerce.

“Everything is a Recommendation.”  Netflix uses this phrase to underscore the point that most of its interface now personalizes to the user. This approach is an inevitable direction for user interfaces most generally since it is clearly technically feasible and delivers business results.

Optimize for accuracy and serendipity.  People are complex and have diverse tastes, moods, etc.  A good recommender will try to help recommendation recipients keep from just re-paving their own cow paths.

Diversity of behavior types trump incremental algorithm advances.  Given already advanced algorithms, recommender improvement comes grudgingly if based on limited behavioral types (e.g., just course-grained ratings). Deriving inferences from a greater diversity of behaviors, as well as contextual cues, delivers greater advantages.

Explanations of recommendations are critically important.  Recommendations that are accompanied by explanations as to why the recommendation was delivered to the recipient are perceived to have greater levels of authority and credibility, and promote trust.

These are important takeaways for e-commerce, but they are just as applicable to enterprise adaptive discovery systems. In fact, as I discuss in The Learning Layer, because there are often more behavioral types, as well as topical structures, with which to work in the enterprise environment, adaptive systems in the enterprise have some inherent advantages versus those in purely consumer-facing environments.

I am often asked by executives about the value proposition of adaptive discovery or learning layer systems in the enterprise. A moment’s reflection on the ability to deliver the right knowledge and expertise to the right person at the right time generally suffices. But from a macro-economic standpoint, simply observing how recommendation systems have transformed e-commerce also should make the point.  On-line businesses without world-class personalized discovery capabilities simply cannot hope to compete going forward. The same lesson surely applies to systems within the enterprise.

Brain Pattern-based Recommender Systems–Coming Soon?

Now things are going to start to get real interesting! In The Learning Layer, I categorized the types of behaviors that could be used by recommender systems to infer people’s preferences and interests. The categories I described are:

  1.  Navigation and Access Behaviors (e.g., click streams, searches, transactions)
  2. Collaborative Behaviors (e.g., person-to-person communications, broadcasts, contributing comments and content)
  3. Reference Behaviors (e.g., saving, tagging, or organizing information)
  4. Direct Feedback Behaviors (e.g., ratings, comments)
  5. Self-Profiling and Subscription Behaviors (e.g., personal attributes and affiliations, subscriptions to topics and people)
  6. Physical Location and Environment (e.g., location relative to physical objects and people, lighting levels, local weather conditions)

Various subsets of these categories are already being used in a variety of systems to provide intelligent, personalized recommendations, with location-awareness being perhaps the most recent behavioral information to be leveraged, and the sensing of environmental conditions and the incorporation of that information in recommendation engines representing the very leading edge.

But I also described an intriguing, to say the least, seventh category–monitored attention and physiological response behaviors. This category includes the monitoring of extrinsic behaviors such as gaze, gestures, movements, as well as more intrinsic physiological “behaviors” such as heart rate, galvanic responses, and brain wave patterns and imaging. Exotic and futuristic stuff to be sure. But apparently it may not be as far off as one might think to actually be put into widespread practice, given this new advance in smart phone brain scanner systems.

Sure, it will take time for this type of technology to be cost effective, user-friendly, scale to the mass market, etc. But can there be any doubt that it will eventually play a role in providing information to our intelligent recommender systems?

Does Personalization Pave the Cow Paths?

Michael Hammer, the father of business reengineering, famously used the phrase “paving the cow paths” to describe the ritualizing of inefficient business practices. Now the pervasiveness of personalization of our systems is being accused of paving our cow paths by continuously reinforcing our narrow interests at the expense of exposing us to other points of view. This latest apocalyptic image being painted is one of a world where we are all increasingly locked into our parochial, polarized perspectives as the machine feeds us only what we want to hear. Big Brother turns out to be an algorithm.

I commented on this over at Greg Linden’s blog, but wanted to expand on those thoughts a bit here. Of course, I could first point out the irony that I only became aware of the Eli Pariser’s book about the perils of personalization, The Filter Bubble, through a personalized RSS feed, but I will move on to more substantive points—the main one being that it seems to me that a straw man, highly naïve personalization capability has been constructed to use as the primary foil of the criticism. Does such relatively crude personalization occur today and are some of the concerns, while overblown, valid? Yes. Are these relatively unsophisticated personalization functions likely to remain the state-of-the-art for long? Nope.

As I discuss in The Learning Layer, an advanced personalization capability includes the following features:

  1. A user-controlled tuning function that enables a user to explicitly adjust the “narrowness” of inference of the user’s interests in generating recommendations
  2. An “experimentation” capability within the recommendation algorithm to at least occasionally take the user outside her typical inferred areas of interest
  3. A recommendation explanation function that provides the user with the rationale for the recommendation, including a level of confidence the system has in making the explanation, and an indication when a recommendation is provided that is intentionally outside of the normal areas of interest

And by the way, there are actually two reasons to deliver the occasional experimental recommendation: first, yes, to subtly encourage the user to broaden her horizons, but less obviously, to also enable the recommendation algorithm to gain more information than it would otherwise have, enabling it to develop both a broader and a finer-grained perspective of the user’s “interest space.” This allows for increasingly sophisticated, nuanced, and beneficially serendipitous recommendations. As The Learning Layer puts it:

. . . the wise system will also sometimes take the user a bit off of her well-worn paths. Think of it as the system running little experiments. Only by “taking a jump” with some of these types of experimental recommendations every now and then can the system fine-tune its understanding of the user and get a feel for potential changes in tastes and preferences. The objective of every interaction of the socially aware system is to find the right balance of providing valuable learning to the user in the present, while also interacting so as to learn more about the user in order to become even more useful in the future. It takes a deft touch.

A deft touch indeed, but also completely doable and an inevitable feature of future personalization algorithms.

I’ve got to admit, my first reaction when I see yet another in a long line of hand wringing stories of how the Internet is making us stupider, into gadgets, amplifying our prejudices, etc., is to be dismissive. After all, amidst the overwhelmingly obvious advantages we gain from advances in technology (a boring “dog bites man” story), the opportunity is to sell a “man bites dog” negative counter-view. These stories invariable have two common themes: a naïve extrapolation from the current state of the technology and an assumption people are at best just passive entities, and at worst complete fools. History has shown these to ultimately be bad assumptions, and hence, the resulting stories cannot be taken too seriously.

On the other hand, looking beyond the “Chicken Little” part of these stories, there are some nuggets of insight that those of us building adaptive technologies can learn from. And a lesson from this latest one is that the type of more advanced auto-learning and recommendation capabilities featured in The Learning Layer is an imperative in avoiding a bad case of paving-the-cow-paths syndrome.

Search = Recommendations

It is good to see that some artificial distinctions that have served to hamper progress in delivering truly intelligent computer interfaces are increasingly melting away. In particular, recommendations, when broadly defined, can beneficially serve as a unifying concept for a variety of computer capabilities. In The Learning Layer I defined a computer-generated recommendation as:

A recommendation is a suggestion generated by a system that is based at least in part on learning from usage behaviors.

In other words, a recommendation is an adaptive communication from the system to the user.

And I had this to say about search:

By the way, the results generated by modern Internet search engines are in practice almost always adaptive recommendations because they are influenced by behavioral information–at a minimum they use the behavioral information associated with people making links from one web page to another. This capability was the original technical breakthrough applied by Google that enabled their search engine to be so much more effective than that of their early competitors.

This feature of contemporary Internet-based search also provides a hint at the reason that the users of enterprise search whom I have talked with over the years have been so often underwhelmed by the performance of their internal searches compared with corresponding Internet versions. Historically, there has been little to no behavioral information embodied within the stored knowledge base of the enterprise, and so search inside the four walls of the business has been basically relegated to the sophisticated, but non-socially aware, pattern matching of text–similar to the way Internet search was before Google. Without social awareness, search can be a bit of a dud.

That’s why, as I mentioned in the previous blog, the computational engines behind the generation of search results and recommendations are inevitably converging, and we are therefore witnessing a voracious appetite of search engines for an ever larger and richer corpus of  behavioral information to work with—Google’s new “plus one” rating function being the most recent example.

On this same note, I just happened across this brief, recent write-up that struggles with categorizing a start-up, exemplifying the blurring of search and recommendations, and finishing with the point that, “. . . Google is also increasingly acting as a recommender system, rather than just a web search engine.” Indeed—now on to bringing this convergence to the enterprise . . .

The Recommendation Revolution, Continued . . .

That “hidden in plain sight” IT revolution of ubiquitous, adaptive recommendations continues to accelerate. Jive Software, a leader in enterprise social software, has just acquired Proximal Labs to bolster their machine learning, recommendation engine, and personalization capabilities. Given that Jive is reportedly preparing for an IPO, this move is a major endorsement of the key theme of The Learning Layer–that adaptive recommendations will be a standard part of enterprise computing. It also is another strong signal that, as I recently wrote, we are going to rapidly witness a standard, adaptive IT architecture emerge in which a layer of automated learning overlays and integrates the social layer, as well as the underlying process, content, and application layers. And I would be remiss if I didn’t mention our own initiative for bringing the learning layer to Microsoft SharePoint and other enterprise collaborative environments, Synxi, which includes a suite of auto-learning functions such as knowledge and expertise discovery, interest graph visualizations, and recommendation explanations.

Another recent and important signal of the recommendation revolution is Google’s recently announced “+1” function, an analog to Facebook’s “like” button. Google will use this tagging/rating function in generating responses to search requests. Google was the pioneer in using behavioral information to improve search results, and this is just the latest, and most dramatic, step in increasingly relying on behavioral cues and signals to deliver personalized search results that most hit the mark.

That is why, as I discussed in the book, responses to search requests should really be considered just a certain kind of adaptive recommendation, one in which more intentionality of the recommendation recipient can be inferred by virtue of the search term or phrase, but that is otherwise processed and delivered like any other type of recommendation. And it is why search processing and recommendation engines will inevitably generalize and converge.

Stay tuned . . .

The Recommendation Revolution Accelerates

GigaOm recently published a blog by Mathew Ingram about’s application of Hunch’s recommendation function. Hunch uses your Facebook social graph as a basis for its recommendations. What was really eye-opening is that’s conversion rate of shoppers to buyers increased by up to 60% after implementing recommendations! That is huge, particularly given that was already a highly sophisticated e-commerce site. And it is consistent with the uplift that and other e-commerce sites that have implemented recommendations have experienced.

As I argue in The Learning Layer, as the positive results of recommendation engines applied in e-commerce keep rolling in, it is inevitable that recommendation capabilities will eventually become a ubiquitous e-commerce feature. And, of course, the same is true for applications within the enterprise—it’s just that the recommendations will be of people, content, and apps rather than that high-tech electric razor!