AI Learns to Do Office Work Inside Fake Office Software

If you want an AI assistant to learn Salesforce, you have to give it a fake Salesforce to practice in.

That kind of fake copy is the fastest growing type of data at Mercor, according to Oswald Nitski, chief product officer at Mercor, an AI data and evaluation company. He calls RL environments a hype term that every company defines differently. RL stands for reinforcement learning, in which a model learns by trying actions and getting scored, and here it means simulations of the apps an agent might have to use.

Building one starts with what Mercor calls a rich start state. Think of it as the world on a laptop, recreated in advance to represent all the data a worker would already have. That starting world can run to hundreds or thousands of files.

Then come tasks that train the agent to get useful work done with that software. Building these tasks is hard, Nitski says, because the agent has to act inside the simulated world, not just answer a question.

The shift, he says, is that the data agents are trained and tested on looks much more like the situations they will face on the job. To teach an agent to use a program like Salesforce, the training and test need a replica that behaves like Salesforce.

Setup is complicated, and this work is still at the frontier. Nitski notes that years ago preference ranking, in which people rank answers to teach a model what is better, was hard to set up, and supervised fine tuning, in which models are trained on clear examples of how to follow instructions, was hard to set up when instruction following models first arrived. Labs are now figuring out simulations, and eventually, he expects, enterprises will be able to build them too.