KorvaJoin the waitlist
Tech

Your Computer Activity Can Be Turned Into a Diary

Software layers app activity and messages into summaries of recent work, upcoming plans, and reminders.

AI image
The a16z Show · The Missing Layer for Personal AI | First Person Labs’ Ken Liu

Your computer produces an infinite stream of events about you, running around the clock.

That stream includes which apps are in use, how you move through websites, how typing happens, and how the machine connects to Wi-Fi. That stream is already available from the operating system, whether you use Apple, Microsoft, or Linux, said Ken Liu, co-founder of First Person Labs, which is building a personal context system called Second. He argues that people underestimate how much their computer already knows. As he put it: "It's just there, sitting there, waiting to be tapped, so to speak."

The diary itself is software being built on top of that raw material. The idea is to analyze the stream continuously and build understanding upward in layers.

For example, a stretch of typing in a particular app might mean you are sending a message. Switching between the browser and other apps might mean looking something up. You might be drafting a document.

The system learns faster if you connect other services, such as Slack, Signal, and WhatsApp. Those connections supply examples of who you talk to and what you discuss, so the software does not have to infer everything from app switches and typing patterns alone.

Once you grant permission, background agents keep running and turn the activity into diary-like summaries. In Liu's design, those summaries become live records that other software can ask about.

There is a record for what you did 10 minutes ago, what you plan to do next week, what your priorities appear to be, what you need to be reminded of, and a personal wiki of people and routines in your life. Another assistant or application could then consult that record instead of asking you to explain yourself from scratch.

Questions people ask

How is this different from my browser history?

Browser history is a flat list of pages you visited. This approach tries to infer the activity behind those visits, such as researching, writing, or messaging, and then tracks how activities change over time.

Does it have to remember everything I type?

It is not known what happens to raw typing data or how long it is kept. What Liu describes as the output is higher level activity, such as sending a message, doing research, or writing a Google doc.

Can other AI assistants actually use that diary?

Liu describes the summaries as portable context, with records that applications or your own agents can query and build on. The idea is that another assistant could consult that record instead of asking you to start from scratch.

More in Tech