KorvaJoin the waitlist
Tech

Phones and Cars See by Breaking Pictures Into Labeled Pixels

To recognize a face or a freeway lane, computers tag tiny pieces of an image and then calculate what they add up to.

AI image
AI Accelerator Institute · What are the top 7 branches of artificial intelligence?

Your phone does not really see your face, and a car does not really see the road. Each turns a picture into tiny labeled pieces, then does math to guess what those pieces add up to.

That ability comes from an AI skill called perception, essentially artificial senses that take in information to notice things and judge where they sit in space. It includes recognizing pictures, locating things inside them, dividing pictures into sections, and making sense of video.

A branch called computer vision gives computers that sight for digital photos and videos. By running learning models over pictures, computers can pick out people, faces, animals, and other subjects.

The more examples the model studies, the sharper it gets at learning about the context of visual data and telling pictures apart.

One tool for this job is a convolutional neural network, which works alongside the model to tag pixels and make predictions. It chops the picture into pixels, the tiny color squares that form a digital image, and gives those pixels tags or labels.

It then runs calculations called convolutions over those labeled pixels to make a prediction about what is pictured.

Self-driving systems with Tesla's navigational technology take in visual information to spot the street layout, lane lines, and things in the path and combine them into one map. Phone face unlock uses a similar check in miniature, measuring the face in front of the camera against a stored set of numbers describing the owner's face to decide whether to open.

The same machinery also powers object tracking, which follows an object after it is detected, and image classification, which predicts which group an image belongs to.

Questions people ask

Why does face unlock sometimes fail in bad light or with sunglasses?

Face unlock needs a clear view of the shape and details of your features. In dim light, heavy glare, or when sunglasses cover key areas, the camera captures less usable detail, so matching becomes harder. Some phones add infrared light to help in the dark.

How is this different from just memorizing my photo?

Your phone saves a mathematical template of your face when you set it up, a set of numbers describing key measurements rather than a photo to flip through. The vision model behind it was trained on many labeled faces to learn facial patterns in general. At unlock time, the phone maps your current face and checks it against your stored template.

Do self-driving cars use only cameras to see?

Many use a mix of cameras, radar, and laser-based lidar, plus detailed maps. Cameras are rich in color and texture for reading lanes, signs, and obstacles, while radar and lidar measure distance and shape more directly. Computer vision helps tie those signals into one map.

More in Tech