Korva is a scroll of the internet’s best ideas.
A rubber duck started talking back
The Duck Tradition
The tradition is simple. Around the third week, each student gets a physical duck and is encouraged to explain a confusing concept or a bug to it out loud. Putting the confusion into words tends to set off the light bulb.
Years ago, the course built a virtual duck: a chat window in CS50's programming interface where the duck quacked back once, twice, or three times at random. There is anecdotal evidence that this was enough to get students to work out what was going wrong, because they had already put their confusion into words.
The AI Duck
Then artificial intelligence arrived underneath it. The duck now runs on OpenAI's tools, and it is given a system prompt, a set of instructions in English. It is told to be a friendly and supportive teaching assistant for CS50, to also be a rubber duck, to answer only questions about CS50 and computer science, and to withhold full answers to problem sets because that would violate academic honesty. The student's question is pasted in below as the user prompt. That prompt is enough to bring back the quacking.
Since spring 2023, some 201,000 students and teachers have used the duck, averaging 35,000 prompts a day for a total of 9.4 million as of the day of the lecture.
Why does explaining a bug aloud help?
Rubber duck debugging was popularized by the 1999 book The Pragmatic Programmer by Andrew Hunt and David Thomas, which describes a programmer who carries a rubber duck and explains code to it line by line. Psychologists have a name for the effect: the self-explanation effect. In a 1989 study, Michelene Chi and colleagues found that students who explained worked examples to themselves learned more than students who just read them.
Why does a system prompt steer a chatbot?
Chat models are trained further on examples of following written instructions, with human ratings used to reward replies that follow them. OpenAI described this method in its 2022 InstructGPT paper. A system prompt sits ahead of the user's messages in the same stream of text, so the model reads it as the framing for everything that follows. The model treats it as strong guidance and never as a hard rule.
Why can't a prompt fully lock a chatbot down?
A language model receives its instructions and the user's words as one continuous run of text. It has no reliable way to separate the two. Programmer Simon Willison named the resulting weakness prompt injection in 2022, after users found they could talk models out of their original instructions. It remains an open problem in the field.
More from CS50
