Skip to main content

3D camera calibration modes

Aurora Imaging Library's Camera Calibration module (Mcal...) allows you to use true 3D camera calibration techniques to map pixel coordinates to real-world coordinates to determine the distance and orientation between the modeled camera and the image plane. This allows for results to be inquired with respect to any defined plane, which can be useful in both robotic and non-robotic applications. For more information on inquiring results with respect to any defined plane, see Inquiring pixel and world calibration points.

Tsai-based mode​

Tsai-based mode is a camera calibration mode based on Roger Y. Tsai's algorithm. This mode models your camera as a pinhole camera and can compensate for rotation, translation, scale, perspective, and radial lens distortions.

This mode allows for camera rotation about any axis without re-calibrating; you only need to specify the camera movement which has occurred. The mode requires a minimum of 6 points to perform a successful calibration and accuracy increases with the number of points. To allocate a Tsai-based camera calibration context, use McalAlloc with M_TSAI_BASED.

  • Calibrates with 1 grid image or list of calibration points. That is, you calibrate with one call to McalGrid or McalList.
  • The single grid image or list of calibration points must provide the best possible FOV coverage.
  • Requires that the optical axis of the camera be at least 30º away from the axis perpendicular to the camera calibration plane.

Intrinsic and extrinsic attributes​

Tsai-based mode calculates several intrinsic and extrinsic attributes of the pinhole camera that is used to model the real-world camera. Intrinsic attributes characterize properties of the camera itself (such as the focal length and the radial distortion), whereas extrinsic attributes characterize the camera's position and orientation.

One radial distortion coefficient is used to correct camera distortion (McalInquire with M_DISTORTION_RADIAL_1). For more information on the distortion model used for the Tsai-based calibration mode, see Radial distortion.

Tsai-based mode does not estimate the following two intrinsic attributes that are needed for accurate calibration; you can adjust their values set by default if not correct:

  • Aspect ratio (of the camera's charge-coupled device (CCD) elements). This refers to the ratio of the horizontal dimension of a single CCD element to its vertical dimension. Tsai-based mode sets this ratio to 1 by default. If the aspect ratio given by the camera manufacturer is not 1, specify its actual aspect ratio using McalControl with M_CCD_ASPECT_RATIO. [Image: cal_CCD.png]
  • Principal point. This refers to the point of intersection of the camera's optical axis with the image plane. You can specify the principal point using McalControl with M_PRINCIPAL_POINT_X and M_PRINCIPAL_POINT_Y, prior to calibrating. If McalGrid is used for calibration, the principal point will default to the central point of the image plane. If McalList is used for calibration, it is mandatory to set the principal point prior to calibrating. In this case, it is recommended to set the principal point to the central pixel of the captured image.

Calibrating with Tsai-based mode​

To successfully perform a full calibration using Tsai-based mode, you must ensure that the optical axis of your camera is at least 30º away from the axis perpendicular to the camera calibration plane. This angle provides the image perspective needed by Tsai-based mode to estimate some intrinsic and extrinsic attributes.

For camera setups that require a camera whose optical axis is perpendicular to the camera calibration plane, you must perform two calibrations: the first calibration must be performed with the camera inclined with respect to the camera calibration plane, using McalGrid or McalList with M_FULL_CALIBRATION; the second calibration must be performed after repositioning the camera, such that its optical axis is perpendicular to the camera calibration plane, using McalGrid or McalList with M_DISPLACE_CAMERA_COORD. Instead of calibrating a second time, you can, alternatively, move the camera coordinate system using McalSetCoordinateSystem if the movement is known with great precision. If you want to move the camera coordinate system by moving the tool coordinate system, you must first move the tool coordinate system as described in Special considerations concerning the tool and camera coordinate systems.

This mode requires that the image has perspective distortion. For this reason, you should not use this mode when using a camera with a telecentric lens; telecentric lenses negate perspective effects.

Note: Note that if you are using a robotics setup and need to transform positional results such that they are in the robot base coordinate system, you should calibrate using robotics mode. Alternatively, if you have already calibrated in Tsai-based mode, you can useMcalCalculateHandEye to calculate the transformation required.

Zhang-based mode​

Zhang-based mode is a camera calibration mode based on Z. Zhang's algorithm. This mode models your camera as a pinhole camera and can compensate for rotation, translation, scale, perspective, and radial lens distortions.

This mode allows for camera rotation about any axis without re-calibrating; you only need to specify the camera movement which has occurred. The mode requires a minimum of 8 points to perform a successful calibration and accuracy increases with the number of points. To allocate a Zhang-based camera calibration context, use McalAlloc with M_ZHANG_BASED.

  • Calibrates with a minimum of 3 grid images or lists of calibration points at different points of view. That is, you calibrate with multiple calls to McalGrid or McalList.
  • The accumulation of the different grid images or list of calibration points together provide the total FOV coverage. This means that the calibration data used to create your 3D camera model is not limited to a single view and the requirements for a large calibration grid are not as critical.
  • No restrictions on the angle of the optical axis of the camera relative to calibration plane for a single point of view (that is, it can be perfectly perpendicular). However, the accumulation of the different grid images or list of calibration points must include calibration data at different depths.

Intrinsic and extrinsic attributes​

Zhang-based mode calculates several intrinsic and extrinsic attributes of the pinhole camera that is used to model the real-world camera. Intrinsic attributes characterize properties of the camera itself (such as the focal length and the radial distortion), whereas extrinsic attributes characterize the camera's position and orientation.

Two radial distortion coefficients are used to correct camera distortion ( McalInquire withM_DISTORTION_RADIAL_1 and M_DISTORTION_RADIAL_2). For more information on the distortion model used for the Zhang-based calibration mode see Radial distortion.

Zhang-based mode does not estimate the following intrinsic attribute needed for accurate calibration; you can adjust its value set by default if not correct:

  • Aspect ratio (of the camera's charge-coupled device (CCD) elements). This refers to the ratio of the horizontal dimension of a single CCD element to its vertical dimension. Zhang-based mode sets this ratio to 1 by default. If the aspect ratio given by the camera manufacturer is not 1, specify its actual aspect ratio using McalControl with M_CCD_ASPECT_RATIO.

Zhang-based mode will estimate the following intrinsic attribute needed for accurate calibration:

Calibrating with Zhang-based mode​

To successfully perform a full calibration using the Zhang-based mode, make at least three calls to McalGrid or McalList with M_ACCUMULATE before performing a full calibration with M_FULL_CALIBRATION.

Calling McalGrid or McalList with M_ACCUMULATE more than three times can greatly improve the accuracy of the camera calibration.

Before each call, move the grid or camera such that the grid is not viewed on the same plane by the camera as a previous call to McalGrid or McalList with M_ACCUMULATE. It will automatically establish the relative location of each grid image (unlike M_3D_ROBOTICS mode). It is recommended to accumulate calibration data at various perceived depths, and at the edges of the working area of your application to accurately estimate the focal length and the radial distortion coefficients. The pose provided in the last call to M_ACCUMULATE will be used to automatically set the position of the absolute and camera coordinate systems.

This mode requires that the image has perspective distortion. For this reason, you should not use this mode when using a camera with a telecentric lens; telecentric lenses negate perspective effects.

Note: Note that if you are using a robotics setup and need to transform positional results such that they are in the robot base coordinate system, you should calibrate using robotics mode. Alternatively, if you have already calibrated in Zhang-based mode, you can useMcalCalculateHandEye to calculate the transformation required.

Zhang-based mode example​

The example 2dcamerato3dcameramapping.cpp demonstrates in part how to calibrate a camera using the Zhang-based mode and calls McalGrid with M_ACCUMULATE twelve times before performing a full calibration.

[Image: zhang_example.png]

To run/view this and other examples, use Aurora Imaging Example Launcher.

Robotics mode​

Robotics mode is a true 3D camera calibration technique that supports the robot base coordinate system. In addition to estimating the camera's intrinsic and extrinsic attributes, this mode also estimates the position and orientation of the robot base coordinate system with respect to the absolute coordinate system. This mode can use either the Tsai-based camera model or the Zhang-based camera model. When using the Tsai-based camera model in robotics mode, all calibration values are determined together. When using the Zhang-based camera model in robotics mode, the camera's intrinsic attributes are determined separately.

Like the other 3D camera calibration modes, robotics mode models the camera as an ideal pinhole camera. Depending on the camera model, the robotics mode estimates the same camera attributes estimated by Tsai-based mode or Zhang-based mode. It also allows for results to be inquired with respect to any defined plane. After 3D camera calibration, it is possible to move the relative coordinate system to another plane without having to calibrate again. For an example of how to move the relative coordinate system, see Moving the relative coordinate system to account for height.

The robotics mode is comprised of two distinct types of camera setups:

Moving camera robotics mode​

This calibration mode is used for camera setups where the camera is mounted on the last joint of a robot arm, while the calibration grid is stationary and in the camera's field of view. In this mode, a rigid link exists between the tool coordinate system and the camera coordinate system by default. You can disable the link between the coordinate systems by setting M_LINK_TOOL_AND_HEAD to M_DISABLE inMcalControl. The image below provides a visualization of what a typical M_MOVING_CAMERA setup looks like.

[Image: Eye_in_hand_calibration.png]

Stationary camera robotics mode​

The second robotics calibration mode is an M_STATIONARY_CAMERA type of M_3D_ROBOTICS, or eye-to-hand calibration. This calibration mode is used for a camera setup where the camera is stationary and the robot arm, with a calibration grid mounted on the last joint, is moving. By default, this calibration mode does not have a rigid link between the camera coordinate system and the tool coordinate system.McalControl with M_LINK_TOOL_AND_HEAD cannot be set to M_ENABLE because moving the tool coordinate system should not affect the camera coordinate system in this camera setup. The image below provides a visualization of what a typical M_STATIONARY_CAMERA setup looks like. Note that the placement of the stationary camera is not required to be overhead as depicted in the image.

[Image: Hand_on_eye_calibration.png]

Note: Note that in both McalGrid and McalList,M_DISPLACE_CAMERA_COORD is not supported for an M_STATIONARY_CAMERA type of M_3D_ROBOTICS camera calibration.

Steps to perform camera calibration in a 3D robotics mode​

The following steps provide a basic methodology for performing camera calibration in a 3D robotics mode:

  1. Allocate a camera calibration context in a robotics mode, use McalAlloc with M_3D_ROBOTICS or M_3D_ROBOTICS_ZHANG_BASED and set the ModeFlag parameter to specify your camera setup (M_MOVING_CAMERA or M_STATIONARY_CAMERA).
  2. Get the position and orientation of the tool holding the camera or grid, with respect to the robot base, using the robot software.
  3. Set the position and orientation for the tool coordinate system in the camera calibration context with respect to the robot base coordinate system using McalSetCoordinateSystem.
  4. Call McalGrid or McalList with M_ACCUMULATE to store calibration points in the camera calibration context.
  5. Physically move the camera to a new position over the working area. You must rotate the tool along at least two non-parallel axes. You must also move the grid or camera such that the grid is not viewed at the same plane by the camera as a previous call to McalGrid or McalList with M_ACCUMULATE. It is recommended to accumulate calibration data at various perceived depths, and at the edges of the working area of your application to accurately estimate the focal length and the radial distortion coefficients.
  6. Repeat steps 2 to 5 at least twice, to make a minimum total of three calls to McalGrid with M_ACCUMULATE. Calling this function with M_ACCUMULATE more than 3 times improves the accuracy of the camera calibration.
  7. When you are done accumulating data, you must call McalGrid or McalList with M_FULL_CALIBRATION and no image/calibration points to perform the full camera calibration. Once you perform a full camera calibration, you can no longer accumulate camera calibration poses.

Note: Note that, you cannot set M_CALIBRATION_PLANE to M_ABSOLUTE_COORDINATE_SYSTEM for an M_STATIONARY_CAMERA type of M_3D_ROBOTICS or M_3D_ROBOTICS_ZHANG_BASED camera calibration context.

Once your 3D-based camera calibration context is calibrated, you can move your camera to a new location and automatically reposition the camera coordinate system, assuming that you are not calibrating an M_STATIONARY_CAMERA type of M_3D_ROBOTICS or M_3D_ROBOTICS_ZHANG_BASED. To automatically reposition the coordinate system, you can use McalGrid or McalList with M_DISPLACE_CAMERA_COORD. This determines and displaces the camera coordinate system to its new location. Note that besides the camera coordinate system, this also displaces the tool coordinate system (if it is not an M_STATIONARY_CAMERA type of M_3D_ROBOTICS or M_3D_ROBOTICS_ZHANG_BASED); no other coordinate system is affected.

M_3D_ROBOTICS and M_3D_ROBOTICS_ZHANG_BASED calibration modes are more efficient than calibrating with M_TSAI_BASED or M_ZHANG_BASED mode and then using the results of McalCalculateHandEye to transform coordinates between the robot base and the absolute coordinate system.

Copyright © 2026 Zebra Technologies.