Research for machines that cannot wait.
We publish what we learn about making perception fast on small chips. Most of it comes from a camera that misbehaved on a real site.
Papers and notes
Latest from the lab.
Each one comes with the numbers behind it. Benchmarks can be reproduced with the open harness.
Paper
Every frame is a deadline: scheduling perception on one chip
How the Parallax Runtime fits detection, tracking and a policy check into 8 ms without a queue.
How the Parallax Runtime fits detection, tracking and a policy check into 8 ms without a queue.
Post
Training for the sensor, not the dataset
Why our models see the raw characteristics of real camera modules during training, and what it buys in the field.
Why our models see the raw characteristics of real camera modules during training, and what it buys in the field.
Paper
Quantization that survives low light
Int8 models lose most of their accuracy in the darkest tenth of the frame. A per-channel range method keeps it.
Int8 models lose most of their accuracy in the darkest tenth of the frame. A per-channel range method keeps it.
Post
Tracking without a second pass
Depth-Track reuses detector features to keep identities across frames in about a millisecond.
Depth-Track reuses detector features to keep identities across frames in about a millisecond.
Focus areas
Three problems we keep coming back to.
They are really one problem: the answer has to arrive before the next frame.
Scheduling
How a whole perception graph shares one frame budget on one chip, and what happens when a node runs long.
Compilation
Quantization, operator fusion and memory layout, compiled per chip so the working set stays on chip.
Perception
Models shaped by the sensor they will run behind: its noise, its optics and its shutter.
Open work
What we release.
Code and data we can share, we share.
- The benchmark harness behind every number on our benchmarks page.
- Sensor characterisation tools for measuring noise, response and shutter timing.
- Evaluation splits for low light and crowded scenes, with labels.
Work on perception that keeps up.
We are a small lab. If these problems sound like yours, we would like to hear from you.