Main Content

estimateCameraIMUTransform

Estimate transformation from camera to IMU sensor using calibration data

Since R2024a

collapse all in page

    Syntax

    estimateCameraIMUTransform(imagePoints,patternPoints,imuMeasurements,cameraIntrinsics,imuParams)
    [tform,params] = estimateCameraIMUTransform(___)
    [___] = estimateCameraIMUTransform(___,options)

    Description

    tform = estimateCameraIMUTransform(imagePoints,patternPoints,imuMeasurements,cameraIntrinsics,imuParams) estimates the fixed SE(3) transformation from the camera to the IMU sensor frame using the distorted image point tracks of a calibration target board captured by the camera, the pattern points of the calibration target board in the world frame, the intrinsics of the camera, the IMU measurements corresponding to the calibration images, and the IMU noise model parameters. The estimateCameraIMUTransform function assumes that the camera and IMU are rigidly attached to each other. For an example, see Estimate Camera-to-IMU Transformation Using Extrinsic Calibration.

    By default, this function plots the progress of pose estimation and undistortion. To disable visualization, set the ShowProgress property of options to "none".

    [tform,params] = estimateCameraIMUTransform(___) additionally returns the estimated calibration parameters.

    [___] = estimateCameraIMUTransform(___,options) additionally specifies calibration options.

    Input Arguments

    collapse all

    Target point detections in the calibration image, specified as a P-by-2-by-N array or as a timetable object. P is the number of detected pattern point detections and each row represents the xy coordinate of a pattern point detection in the form [x y]. N is the number of calibration images.

    If imagePoints is specified as a timetable object, the timetable object contains the variable imagePoints as an N-by-P-by-2 array. The first page of the array represents the x points of the images and the second page of the array represents the y points of the images.

    You can use the detectCheckerboardPoints (Computer Vision Toolbox) and detectCircleGridPoints (Computer Vision Toolbox) functions to detect the target points in a calibration board.

    Note

    If you specify imagePoints as a P-by-2-by-N array, you must also specify either the ImageSampleRate or ImageTime properties in options. If both are specified, then estimateCameraIMUTransform uses ImageTime for accuracy.

    Calibration target board key pattern points in the world frame, specified as an N-by-2 matrix. M is the number of calibration images. Each row represents an xy coordinate.

    You can use the generateCheckerboardPoints (Computer Vision Toolbox) and generateCircleGridPoints (Computer Vision Toolbox) functions to get key pattern points.

    IMU accelerometer and gyroscope measurements, specified as a timetable object with the Accelerometer and Gyroscope variables. Each row represents measurements with the associated time step of the calibration data.

    Both Accelerometer and Gyroscope are specified in the timetable object as an N-by-3 numeric matrix. N is the total number of IMU readings in the calibration data.

    Accelerometer measurements are measured in meters per second squared. Gyroscope measurements are measured in radians per second.

    Camera intrinsics, specified as an cameraIntrinsics (Computer Vision Toolbox) object.

    Example: cameraIntrinsics([800 800],[320 240],[480 640]) creates a cameraIntrinsics object using the focal length, principal point, and image size of the camera.

    IMU noise model parameters, specified as a factorIMUParameters object.

    Example: factorIMUParameters(SampleRate=400,GyroscopeNoise=6.93e-5*eye(3),AccelerometerNoise=2.9e-6*eye(3)) creates a factorIMUParameters object using the sample rate, gyroscope noise, and accelerometer noise.

    Calibration options, specified as a cameraIMUCalibrationOptions object. If you do not specify options, then estimateCameraIMUTransform uses default calibration options.

    Example: cameraIMUCalibrationOptions(ShowProgress="camera-poses")

    Output Arguments

    collapse all

    SE(3) transformation from the camera frame to the IMU sensor frame, returned as an se3 object.

    If you set the InitialTransform property of options to a rigidtform3d (Image Processing Toolbox), then tform is a rigidtform3d object.

    Calibration optimization parameter estimates, returned as a cameraIMUParameters object.

    References

    [1] Qin, Tong, and Shaojie Shen. “Online Temporal Calibration for Monocular Visual-Inertial Systems,” 3662–69. Madrid, Spain: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2018. https://doi.org/10.1109/IROS.2018.8593603.

    [2] Furgale, Paul, Joern Rehder, and Siegwart Roland. “Unified Temporal and Spatial Calibration for Multi-Sensor Systems,” 1280–86. Tokyo, Japan: 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2013. https://doi.org/10.1109/IROS.2013.6696514.

    [3] Qin, Tong, Peiliang Li, and Shaojie Shen. “VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator.” IEEE Transactions on Robotics 34, no. 4 (August 2018): 1004–20. https://doi.org/10.1109/TRO.2018.2853729.

    Version History

    Introduced in R2024a

    See Also

    Objects

    Functions

    Topics