Wow - that earnestly gave me goosebumps. I'm a Googler myself and it's humbling seeing him casually describe, 10 years ago, a technology the industry was still in the early stages of developing, which has since taken the world by storm. What a rockstar.
Neural networks were already big 10 years ago, you have to go back 15 years to see before they started being popular.
From wikipedia:
> Between 2009 and 2012, ANNs began winning prizes in image recognition contests, approaching human level performance on various tasks, initially in pattern recognition and handwriting recognition.
That was when Neural networks became a big thing every tech person knew about, 2014 it was already in full swing and you had neural networks do stuff everywhere, like recognizing faces or classifying images.
NN were already a casual topic in my high school computer science class more than 20 years ago. I've always assumed they were already fairly common by that point. (~2000)
They were known in the field but had a reputation for being too slow. I remember a couple of early 2000s NIPS (now NeurIPS) people commenting about what a shame it was that NN were computationally infeasible, which was true in the era before GPUs took off.
Neural networks weren't the best models for character recognition, their breakout success was when they started being the best at recognize characters and other images which happened in the late 00's. OCR before then was really bad.
Might be hard to imagine today but back then OCR and image recognition was typically done with normal statistical regression models, and the neural networks they had then were worse than those.
Neural networks were mentioned -- not particularly often, but now and then -- in such non-rarefied publications as BYTE Magazine in the 1980s and 90s, AFAICR.
