Open Knowledge: Hunting for the Next Layer


This is part of my ongoing Series on Technology, Disruption (and chess). In the first post of this Series, entitled: Hidden Knowledge, I talked about the phenomenon of software hitting its stride, disrupting and "laying every industry bare" - opening up knowledge and know-how that in the past was hidden and only known to "the few". If you see me talking a lot about chess throughout this Series, it's because its evolution through the centuries is a terrific allegory to illustrate the unexpected effects and opportunities that arise when technology/software transforms or disrupts industries and disciplines.


In the "closed world" you had to be lucky to fall-in with those that possessed 'the knowledge' either through happenstance and/or apprenticeships. But generally, if you were on the outside looking in- you had no recourse. As an example, most immigrant families in the United States had to labor for a generation or more for the opportunity to "break-into" certain fields and access the right networks (or establish their own). The almost century-long dominance of the Soviet chess machine I described in the previous post is another perfect illustration of this principle.

In this post I want to explore how we, as entrepreneurs, technologists and investors should operate in this new world of "Open Knowledge" as opposed to the "closed world" model. How should we think about launching new ventures in a world in which all photos we've ever taken or all music or movies we ever wanted to listen to or all recorded events in human history again are a click away on our cellphones or desktops? It's a world where soon, the greatest courses in any discipline taught by the leading minds at great universities will be available to anyone in the world with an internet connection. What kind of mindset do we need to equip ourselves to operate amidst such a landscape? It's my observation that a lot of us are still wired for the closed world.

The founders of Andreessen Horowitz call the phenomenon of disruption I'm describing as "software eating the world". It's clear that this is happening. As Ron Conway puts it- just think "Music:Apple, Radio:Pandora, Magazines/News:The Internet, Animated Film:Pixar, Direct Marketing:Google, DVD/Video:Netflix/Youtube, Tickets:Eventbrite, Mail/Fax:Email", and on and on.

But the assumption that many fledgling entrepreneurs make in the wake of this phenomenon- is to further assume that now that 'such and such' industry has been laid-bare and transformed by software so we should 'move on' and rush to participate in the laying bare of the next industry- as if software's disruption had had the final say. The assumption is that there are now no more secrets or so-called "hidden knowledge" and once certain startups have "eaten the space" and grown larger- it's a 'case-closed' situation devoid of new and mind-blowing opportunities. Peter Thiel has lectured eloquently about this mistaken assumption people make.

So whether you're an entrepreneur, tech investor or technologist- I think it's a good move to take some time to determine the following for yourself:

  • what will "the next layer" be within your industry or discipline, for it will surely come, and
  • what does the commoditization and availability of formerly hidden knowledge now allow me to try and do?
  • what secret do you know or can you uncover? don't underestimate the human component in this equation. 

This last bullet is important I think. Software is really about making processes more efficient, saving time, cutting waste out of various systems- it does not mean that the human element is eliminated. As an example- you can use a massive parallel processing chess-playing computer to check over your games- but it can't help you yourself calculate better when you're on your own! It's a tool- a resource to complement your own efforts and ingenuity. Why not use all the tools available to uncover the 'new hidden knowledge', the secrets Thiel talks about- and become a "layer hunter"?

In chess, for example, despite the proliferation of incredibly powerful chess-playing software upon which an entire new generation of grandmasters have been raised, why did the strongest young player in the world, Magnus Carlsen, feel the need to hire former World #1 Kasparov for a period of time as a coach some years ago? One major reason is that the fruits of 40+ years of world-class tournament competition, the enormity of a world champion's wisdom, experience and subtlety- is not yet 'available in silicon'. This is no different than a fledgling entrepreneur having a great CEO as mentor or a super-accomplished Board or Advisory Board around him/her.

Another big reason Carlsen hired Kasparov is that Kasparov was one of the great early adopters of chess software to aid his own preparation and training. He is also in possession of a database of 'secret' opening novelties and opening repertoires that he has assiduously cultivated and refined over decades with the help of strong computer analysis driven by his own insights. This database has achieved mythic status in chess circles and is believed to have been a massive competitive advantage for Kasparov during his playing days. It still looms large for those that seek him out as a coach.

Here we also see the phenomenon of the best in their fields always hunting for the slightest edge over their competition and becoming early adopters of the newest platforms and technologies. They are often the best layer hunters of all. We will see this pattern repeat itself again and again.

Germany, 1985  Working w/ Atari ST, 1987  Training World #1 Carlsen, 2009

Hunting for the Next Layer:

So how to go about hunting for the "next layer"?  I think one good technique is to look at what the people at the very top of their fields are experimenting with. They always seem to be on the bleeding edge, uncovering new approaches, new secrets. So if you're into education- what are the cutting-edge professors and entrepreneurs trying? If you're in VC, what are the best practitioners talking about amongst themselves? The same for sports, energy, healthcare, cloud, government- you name it. Find the best and get to know them. This was almost impossible a quarter century ago. Now there are no limits to who you can reach. Get in the same room with them. Period.

If you were in the room with Kasparov, (see the black and white photo above), in 1985, you would have been on the cutting edge of the development of the next layer of chess' thousand-year ascent.

If you were in conversations with Fred Wilson and/or Brad Feld when they started blogging about VC in 2004 you would have been privy to the first layer in the unbundling of VC.

If you were in the room with Sebastian Thrun when he first decided to open his Introduction to Artificial Intelligence to the world, you would have seen the coming of the next layer in .edu when 160,000 students in 190 countries enrolled.

What are the top people in your field talking about right now? Perhaps you are one of them yourself? Whatever the case- find out.

The other way to go is to do what Thiel suggests and go after very hard problems that have scared people off. But you have to do it without accepting other people's definition of the landscape. You need to do a lot of the core re-thinking. As Thiel says:

"The basic challenge is to find things that are hard but doable. You want to find a frontier. But don’t simply accept others’ definitions of the frontier. Existing priorities and ways of thinking need not be your own. Think things through and go find some secrets. There are many of them out there. Just remember that they are concealed not just by nature, but also by the people all around you."

In the next post we'll use this framework to assess the unbundling of the Venture Capital arena and apply these principles. Let's see what we come up with.

Click here for the next post in this Series