An RGB-D camera provides color images and depth measurements of a scene. RGB describes the color information; D describes the distance information. Together, they give software a basis for using an object’s visual appearance and its three-dimensional position in the same application.
The depth measurement can come from ToF, stereo vision or structured light. DOMI’s VGA ToF RGB-D cameras combine a color camera with ToF depth sensing. Understanding the measurement method and the relationship between the image streams helps you choose the data your project needs.
What do RGB and depth images contain?
An RGB image records color at image locations. A depth image stores a measured distance at each valid depth pixel, using a defined scale and coordinate convention. Software may display depth as grayscale or false color to make the distances easier to see; the display colors are a visualization of the measurement.
The two streams can have different resolutions, lenses and fields of view. They may also come from physically separate imagers. These details determine how a color pixel and a depth sample can be related.
How does an RGB-D camera obtain depth?
iToF depth with RGB imagery
In an iToF RGB-D camera, modulated illumination and the measured phase relationship of the returned signal provide the basis for depth estimation. The RGB imager supplies color and texture. Software can combine the two streams using the camera calibration and the capture timing required by the application.
Our dToF and iToF guide explains phase-based delay estimation. Color and geometry can then serve as inputs to recognition, mapping or obstacle-perception software, using the selected camera outputs and host pipeline.
| Depth-sensing method | How distance is obtained | Integration information to examine |
|---|---|---|
| Time-of-Flight | Estimates distance from returned light’s travel time or phase delay | Acquisition settings, working distance, target properties and ambient illumination |
| Stereo vision | Uses the disparity between views and the calibrated camera geometry | Baseline, camera calibration, correspondence quality and scene texture |
| Structured light | Uses a projected pattern and calibrated viewing geometry | Projector-camera geometry, pattern visibility and acquisition sequence |
RGB-D describes the combination of color and depth data. The actual camera architecture determines how that depth is measured and which output formats are available.
How do color and depth pixels correspond?
Relating the streams requires a geometric mapping. Intrinsic parameters describe an imager’s projection, including focal length and image center. Extrinsic parameters describe the position and orientation of one imager relative to another.
With a valid depth sample, software can calculate a three-dimensional point in the depth camera’s coordinate system. It can then transform the point into the color camera’s coordinates and project it onto the RGB image to find a corresponding color location.
Different viewpoints can create occlusions, and the two fields of view may only partly overlap. A point visible to one imager may be hidden from the other. Keep those regions in mind when selecting a camera position or using color to interpret depth data.
Spatial alignment and time synchronization address different needs. Alignment relates image coordinates. Timing describes when the images were captured. Moving objects make both relevant, so plan the required stream relationship around the application and the camera’s actual output configuration.
From depth measurements to a colored point cloud
A colored point cloud associates three-dimensional positions with RGB values. A typical processing path is:
- Read the depth samples using the camera’s documented distance scale and invalid-value convention.
- Use the depth calibration to calculate valid three-dimensional points.
- Transform those points into the color camera’s coordinate system.
- Project them into the RGB image and attach available color samples.
For geometric checks and assembly validation, see our RGB-D depth camera calibration guide. The software path should use the calibration and formats supplied for the chosen model.
Check alignment direction before passing RGB-D data to an algorithm
A depth image aligned into the RGB view uses the color image grid; a color image mapped into the depth view uses the depth grid. These are different processing outputs. Preserve the intrinsics for the output grid and the transformation direction, and identify pixels that have no valid shared observation. Resizing one image to match the other’s dimensions does not perform this geometric registration.
Open3D’s RGB-D odometry tutorial explicitly assumes synchronized color and depth registered in the same coordinate frame. That is a requirement of the illustrated processing pipeline, not a property automatically supplied by every RGB-D camera.
Before using such a pipeline, keep the calibration, distance scale, stream pair and timestamps with the capture. Check a stationary depth edge against color, then repeat with motion. If several depth points project into the same color pixel, the mapping needs an appropriate visibility rule; an occluded surface has no matching color observation to recover from that viewpoint.
Choosing a DOMI ToF RGB-D camera
DM-RGBD5002A combines VGA ToF depth sensing with RGB imaging through USB 2.0. It provides a starting point for projects that use color and spatial distance information together.
DM-RGBD5003A provides RGB images, depth and point-cloud output for host-based perception and measurement projects. Its available formats and tools should be selected around the data path the application will use.
Compare the models in our RGB-D camera range. Share your working distance, target scene, host platform and required outputs with us to discuss an evaluation configuration.
FAQs
What does RGB-D mean?
RGB-D means color information combined with depth information. The color image describes appearance, while valid depth samples describe distance. The camera's measurement method and calibration determine how these streams can be used together.
Which methods do RGB-D cameras use to obtain depth?
ToF, stereo vision and structured light are common depth-sensing routes. A ToF RGB-D camera uses an active depth sensor alongside color imaging. Check the specific camera architecture and available output modes when choosing a model.
How are RGB pixels matched to depth measurements?
Software uses camera calibration, the relative position of the imagers and the measured depth to map between the streams. A valid mapping also depends on the shared view of the scene. For moving targets, consider capture timing separately from spatial alignment.
What is needed to create a colored point cloud?
You need valid depth data, its distance scale and camera calibration, plus the geometric relationship to the color image. The camera or host software then projects depth-derived points into the RGB view and associates available colors with those points.
