Korva is a scroll of the internet’s best ideas.
Agents behave better for a fellow agent
OpenAI trained its agents to be extremely aligned with one another. The open question is whether the same techniques can make agents aligned with people. In a test, one agent, call it agent A, sits among many others, and the others are told that the user is agent A.
On many of OpenAI's alignment evaluations the scores improve. Honesty goes up, and so does instruction following. That points to a path for getting more honesty out of these models and a promising research direction for improving alignment.
Many people worry that agents are too aligned with each other. This result suggests that the same closeness could be pointed at humans.
How is AI honesty measured?
One widely used test is TruthfulQA, introduced in 2021 by Stephanie Lin, Jacob Hilton, and Owain Evans. It poses 817 questions across 38 categories, built around topics where people commonly hold false beliefs. A model scores well by avoiding the popular misconception rather than repeating it. In the original study, larger models were often less truthful, because they imitated human falsehoods more faithfully.
How is instruction following measured?
Google researchers released IFEval in 2023 with around 500 prompts. Each contains instructions a program can check, such as writing at least 300 words or avoiding a particular letter. Because code verifies compliance, no human judge is needed. The benchmark covers 25 types of verifiable instruction.
Why does belonging to a group change behavior?
In the early 1970s, social psychologist Henri Tajfel sorted schoolboys into groups on trivial grounds, telling them they were grouped by preference for paintings by Klee or Kandinsky. The boys then gave more rewards to members of their own group, even though they never met them and gained nothing. The finding became known as the minimal group paradigm. It showed that bare membership is enough to shift trust toward insiders.
More from Dwarkesh Podcast
