Hidden Knowledge: Chess & Tech

This post is part of my new Series on Technology, Disruption (and chess), as well as my Mentorship Series.

The Series is in large part about the phenomenon of technological disruption and its effects and consequences. Throughout I will use the ancient game of chess and its evolution/disruption vis-a-vis technology/software as an allegory. Chess represents an excellent prism through which to study the disruptive effects of technology/software when integrated and/or superimposed upon any discipline.

A few days ago Fred Wilson wrote this post about a short video piece featuring Peter Thiel and former World Chess Champion Gary Kasparov. It was the second blog post in the last few weeks in which Fred covered chess in some way- the last one being this one about Brooklyn Castle.

Fred’s posts inspired me to finally kick-off this Series as it’s a subject I’ve been thinking about for years.


The short version of the story is that I got into chess probably at age 8 but didn’t get very serious about it until I went to Stuyvesant High School in the early eighties- yes, in the “old building” – the same Greenwhich Village building on 15th Street off 1st Avenue that James Cagney attended when he was a kid. Even my buddy Mike Z, the guy who runs Stuyvesant’s computer science classes these days was around back then- as a student! Now his kids go to fancy “new” Stuyvesant off the west-side highway.

(Here’s the old building, James Cagney for good measure)

stuyvesant  cag2

Anyway- the Brooklyn Castle film that Fred recommended about those young chess players reminded me of my own chess team back in those days. With zero budget and no coach- our chess team managed to win 4 city, 3 state and 2 national high school championships while I was there. I just found one NY Times article about us here where they called us the “Kings of High School Chess”. It was exhilarating and propelled me into the world of tournament chess which was a fun part of my life until I settled down with a family and kids. I have a lot of stories to tell from those days, one of them can be found here.

Thinking back on those early years, it occurred to me that I really learned the most about chess not from books- but from the Russian guys who played on the team that I met in our school cafeteria when I was a lowly freshman fresh right off the boat from Brooklyn.

You see, back then all the action was in the cafeteria and it would really go on all day long. Parents these days would no doubt be outraged. It was literally like a casino. You could gamble relatively openly- and choose between one or two games of seven card stud or plenty of folks playing blitz chess, ping pong- whatever you were in the mood for. These were serious, intense games with good players and no messing around and it was worth participating just for the sheer brilliance of the insults and banter.  Every once in a while some ancient relic of an administrator from the James Cagney days would storm into the cafeteria and yell a little bit and try to break up the poker games- but people would just hold their hands below the table make small talk and suppress laughter until the old geezer would run out of breath and hobble out. The games would resume unabated. Periodically players got up and cashed-out to go to class- but when they got back they re-entered the games seamlessly. There was no-need for a pit-boss- we all were one community of players and without ever discussing it the whole group of us- whether you played poker or chess or other money games, were watching out for each other. To use the language of tech- you could say that we had established very sticky network effects- pre-Internet!

When I entered this carnival-like atmosphere for the first time in my life as a 15 year old kid- the environment shocked and intrigued me all at once. Of course I gravitated towards the tables where speed chess was being played. The language was exclusively Russian and though the players were in highschool they seemed like grown men to me. After weeks of just quietly watching the games, one of the best players- a Senior named Oleg- gruffly told me that my presence was annoying to him and that I may as well play a game so he could crush me. Indeed he rolled through my primitive game like a wrecking ball multiple times. Many a term of ridicule and mockery were levied in my direction in Russian- followed by unsuppressed laughter. I knew at whom they were directing it. To put it mildly I was not pleased and went right to the old Chess Shop on Thompson Street (see photo below) after school and picked up some opening books which I devoured at the expense of any school reading for the entire week.


The following week I arrived at the cafeteria only to have Oleg and others steamroll me. But the more I got my ass-kicked, the more I learned, the more I studied and the better I got. In time I was holding my own. Over the course of many months this closed group of Russian players took me under their wing. In time, I qualified to join the team and we all became good friends. While I was there, Stuyvesant’s chess team literally became a wrecking ball. Our closest competition not only locally, but at the Nationals, was actually only one other school: Madison High School from Brooklyn. Why? You guessed it- they were captained by a guy named John Litvinchuk- who was also “off-the-boat” from Russia and close to grandmaster strength.

When I think back on those pre-Internet days- it occurs to me that making progress in any field required *real* mentorship with people who had access to special knowledge. My chess mentors really had proprietary knowledge known only to “the few” and they knew it.  They also cared a lot and invested in me once I proved myself to them, spending countless hours training me to great effect. I remember them going so far as to lend me their old-school Russian chess books and showing me the important words I needed to know so I could follow the thread when I studied on my own. They didn’t do this for just anyone.

In those days only pockets of people in any given field or discipline had “the knowledge” and it wasn’t easily accessible. Some people never found it and were always on the outside looking in. It was like some kind of magic- invisible, cloistered- and you had to somehow hunt for it. In chess, it so happened that the Russians had that “specialized knowledge” because for decades the Soviet Union had prioritized excellence in chess. They formalized chess study in the schools, had sophisticated trainers, and invited the best young competitors throughout the USSR to elite training camps. The most well known was run by a legendary former World Champion, Mikhail Botvinnik and was named after him. This approach led to a near century of dominance. Players received real methodological training and had the benefit of a great deal of “proprietary work” that had been performed on opening theory and other aspects of the game. But make no mistake- this knowledge was “hidden” to the rest of the world. Perhaps only 100 people in the world possessed this arcane knowledge! It was literally like being in possession of the Holy Grail. The names of players that passed through Botvinnik’s training academy is literally a list of World Champions and elite Super-Grandmasters: Kasparov, Kramnik, Karpov, Shirov, Akopian, Ehlvest and many others, some of whom still are active today. (see photo below). It gives me chills to think about how powerful that knowledge was. It was an immense competitive advantage. You can still trace its influence to the students of Soviet Grandmasters who defected or emigrated after the collapse of the USSR. The students of these Botvinnik students themselves possessed huge sustaining advantages.


Of course the web and software opened up all of this arcane knowledge to a large extent. And in the same way that almost every industry has been laid bare, (including venture capital!)- the mysteries of chess have gradually been exposed by the web and by powerful chess-playing engines. Now you have both the phenomenon of younger and younger Grandmasters and a parallel phenomenon of immensely strong players emerging who have never had a serious coach/trainer but rather have learned everything from computers and practical experience. The youngest Grandmaster in the world today is not 20 years old, not 15 (as Bobby Fischer was in his day), but 12!! In fact, the strongest player in the world is not a Russian but actually a young Norwegian, Magnus Carlsen!  This is simply extraordinary and needs to be discussed in coming posts.

Chess is somewhat unique in that we have a universal rating system. So it goes beyond saying- “wow, that fellow Peter Thiel or Elon Musk is a genius”. Now we can say- wow- Magnus Carlsen is already among the strongest chessplayers in human history- as his rating just eclipsed that of both Fischer and now Kasparov at their peak.


So what can we derive from this? Well, quite simply- tech/the web/software is just making the best of us– well…. better. It’s clear. Something extraordinary is happening. What does this imply for the potential for human achievement in a diversity of fields? I think we are already seeing better entrepreneurs, better athletes, better venture capitalists, better engineers than there have ever been in human history. And yes, of course if Nicola Tesla had had our tech he too would have been even better (if that’s possible)… but it’s probably true.

We’ll also explore just what this “open knowledge” in chess and other industries/disciplines has led to in more detail. In many cases, the effects are not what we might think they are or logically should be. One of the main things I’ve observed is that despite this great ‘opening up’ of hidden knowledge, the mystery has only deepened in some respects and “secrets“, (as Thiel eloquently defines them), abound.

Click here for the next post in this Series