Korva is a scroll of the internet’s best ideas.
OpenAI's first day: find a whiteboard
Showing up
The company had been announced in 2015, but its first working day came right after New Year's. Eleven or twelve people showed up at Greg Brockman's apartment around nine or nine-thirty on a weekday morning, with all the excitement of the first day of school.
Nothing to build
Then people looked around the room, and someone suggested they get a whiteboard. Brockman sent someone off to find one, and it arrived. They looked around again, wondering what they were supposed to do now, and the energy in the room collapsed. This was not a product startup, with something to build and customers to talk to. The goal was to make AGI, so someone said maybe they should write some papers, and then maybe they should think about some ideas.
Finding a rhythm
Nobody had any idea what they were doing, so they did what they knew how to do. Over some years of mostly chaotic stumbling, they found a rhythm for making and evaluating research bets, and a gradient they could progress along. An early paper on the unsupervised sentiment neuron turned into GPT-1 and then eventually the models after it.
How did OpenAI begin as an organization?
OpenAI was announced in December 2015 as a nonprofit research lab. Sam Altman and Elon Musk were its co-chairs, and the founders pledged $1 billion in total funding commitments. The announcement described a goal of building artificial general intelligence that benefits all of humanity.
How did a sentiment neuron lead to GPT-1?
In 2017, OpenAI researchers trained a model to predict the next character in a large collection of Amazon reviews. They found that a single unit inside the network had learned to track whether a review was positive or negative, though it was never told to. That result suggested that simply predicting what comes next could teach a model useful structure. In 2018, Alec Radford and colleagues published the first GPT paper, which pretrained a transformer on text and then fine-tuned it for specific tasks.
More from David Senra
