Embedded cameras are now in almost everything: doorbells and baby monitors, phones and tablets, driver assistance systems, security systems and the inspection stations on a factory line. Each of these brings together sensor hardware, signal processing and the system around it, and the choices made early decide how hard the rest of the project will be.
From our own projects, the five most common applications are:
- IoT devices for home monitoring: smart doorbells, baby monitors and pet cameras that let people see and talk remotely.
- Phones and tablets: photos and video, but also face recognition, augmented reality and video calls.
- Automotive: driver assistance such as lane departure warning, collision avoidance and parking aids.
- Security and surveillance: monitoring and recording in homes, businesses, public spaces and industrial sites.
- Industrial automation: quality control, inspection, robot guidance and process monitoring.
Choosing a camera interface
Two standards dominate: MIPI CSI (the Mobile Industry Processor Interface camera serial interface) and UVC (USB Video Class). MIPI CSI was designed for mobile and embedded devices and handles data and power efficiently. UVC is the general purpose option that works across a wide range of hosts.
There are others as well. SPI and I2C cameras suit small, low bandwidth designs, LVDS suits high speed links over longer distances, and parallel cameras suit designs that need a wide data path.
The right choice depends on what the application needs technically, your target unit cost and volume, and your development budget. The simplest path is often a UVC camera, as long as your embedded platform has a USB PHY and driver support. On a Linux based product, a UVC camera works well for general applications. Being generic, it can limit resolution, frame rate and fine control of the camera. Camera sensors are also rarely UVC compatible out of the box, so extra components are needed to turn the sensor output into USB data, which adds cost.
A MIPI CSI camera removes those limits, but you have to write and maintain the driver, which is specialised embedded work. It also adds hardware complexity, which lengthens development and raises cost.
Engineering a MIPI CSI camera
MIPI CSI became widespread because it is the standard camera interface on phones and tablets, and that volume means camera modules are easy to find. It has become a regular part of our projects, but using it well takes specific expertise.
On the software side, someone has to write the operating system driver that configures the camera module (usually over I2C or SPI) and maps its MIPI CSI data onto the processor’s MIPI CSI PHY. Beyond that, the raw frames need to be captured, synchronised with the operating system’s video framework such as V4L2, and passed into a streaming pipeline such as GStreamer. How these pieces fit together has a large effect on what your application finally sees.
The hardware has to match. The MIPI CSI data lanes on the PCB need careful design: the layer stack, length matching, impedance control and guarding all need to be calculated. These details matter most when the target frame rate is high.
Our projects
We have designed complete camera solutions, hardware and drivers, including custom camera hardware and drivers for Linux based systems built with Yocto. Among them is a driver for the onsemi ARX3A0, a monochrome sensor that runs at up to 360 frames per second.
Fig 1 · Dual camera driver running on Linux on an evaluation kit
A UVC camera can look like the easier route, but converting MIPI CSI images to USB adds hardware that can upset your product’s cost structure, and it limits access to features specific to the camera chipset.
Alongside the cameras, we have designed high speed compute modules based on the NXP i.MX 8M Plus and paired them directly with camera modules we developed in house. One of these is a multi camera system that combines MIPI CSI and UVC cameras.
Fig 2 · Layout of our high speed compute module, simplified
Most of these systems run in infrastructure monitoring and healthcare, often with very little human intervention. You can read more in our case study on compute platforms for embedded cameras.