DUO Device API - Sensor Access & Control
Part of the DUO SDK, The DUO API provides low level access to device details, controls and configuration with the use of supported methods and parameters. This document outlines all functionality of the DUO API via the DUOLib library which provides a C interface paired with C++/C#/OpenCV/Qt Samples.
Here is a overview of the key methods and structures available in the API.
// Device Control
bool OpenDUO(DUOInstance *duo, uint32_t index = 0);
bool CloseDUO(DUOInstance duo);
bool StartDUO(DUOInstance duo, DUOFrameCallback frameCallback, void *pUserData, bool masterMode = true);
bool StopDUO(DUOInstance duo);
// Device Information
int EnumerateDUOResolutions(DUOResolutionInfo *resList,
int32_t resListSize,
int32_t width,
int32_t height,
int32_t binning,
float fps);
// Frame Callback
void (CALLBACK *DUOFrameCallback)(const PDUOFrame pFrameData, void *pUserData);
// Get DUO Parameters bool GetDUODeviceName(DUOInstance duo, char *val); bool GetDUOSerialNumber(DUOInstance duo, char *val); bool GetDUOFirmwareVersion(DUOInstance duo, char *val); bool GetDUOFirmwareBuild(DUOInstance duo, char *val); bool GetDUOResolutionInfo(DUOInstance duo, DUOResolutionInfo *resList); bool GetDUOExposure(DUOInstance duo, double *val); bool GetDUOExposureMS(DUOInstance duo, double *val); bool GetDUOAutoExposure(DUOInstance duo, bool *val); bool GetDUOGain(DUOInstance duo, double *val); bool GetDUOHFlip(DUOInstance duo, bool *val); bool GetDUOVFlip(DUOInstance duo, bool *val); bool GetDUOCameraSwap(DUOInstance duo, bool *val); bool GetDUOLedPWM(DUOInstance duo, (double *val); bool GetDUOFrameDimension(DUOInstance duo, uint32_t *width, uint32_t *height); bool GetDUOIMURange(DUOInstance duo, int *accel, int *gyro); // Get Calibration Parameters bool GetDUOCalibrationPresent(DUOInstance duo, bool *val); bool GetDUOFOV(DUOInstance duo, double *val); bool GetDUORectifiedFOV(DUOInstance duo, double *val); bool GetDUOUndistort(DUOInstance duo, bool *val); bool GetDUOIntrinsics(DUOInstance duo, DUO_INTR *val); bool GetDUOExtrinsics(DUOInstance duo, DUO_EXTR *val); bool GetDUOStereoParameters(DUOInstance duo, DUO_STEREO *val) // Set DUO Parameters bool SetDUOResolutionInfo(DUOInstance duo, DUOResolutionInfo resList); bool SetDUOExposure(DUOInstance duo, double val); bool SetDUOExposureMS(DUOInstance duo, double val); bool SetDUOAutoExposure(DUOInstance duo, bool val); bool SetDUOGain(DUOInstance duo, double val); bool SetDUOHFlip(DUOInstance duo, bool val); bool SetDUOVFlip(DUOInstance duo, bool val); bool SetDUOCameraSwap(DUOInstance duo, bool val); bool SetDUOLedPWM(DUOInstance duo, double val); bool SetDUOLedPWMSeq(DUOInstance duo, PDUOLEDSeq val, uint32_t size); bool SetDUOUndistort(DUOInstance duo, bool val); bool SetDUOIMURange(DUOInstance duo, int accel, int gyro); bool SetDUOIMURate(DUOInstance duo, double rate);
The DUO device instance is opaque pointer that represents a DUO handle. It is initialized by calling OpenDUO() function and it is used in all subsequent API calls.
void *DUOInstance
Retrieves the library version as a string.
char *GetDUOLibVersion()
Enumerates supported resolutions. Requires an allocated DUOResolutionInfo structure and resListSize parameter indicating number of allocated elements in resList array. Enumeration can be filtered/restricted by fixing one or more parameters such as width, height, binning or fps. The function returns number of resolutions found.
int EnumerateDUOResolutions(DUOResolutionInfo *resList,
int32_t resListSize,
int32_t width,
int32_t height,
int32_t binning,
float fps)
Called repeatedly by the DUOLib on successful fame capture. Returns PDUOFrame data with user data (pUserData) passed when calling StartDUO function.
void (CALLBACK *DUOFrameCallback)(const PDUOFrame pFrameData, void *pUserData)
Opens the DUO device and initialized the passed DUOInstance handle pointer. The index parameter specifies the zero based index of the camera to be opened (in case of using multiple DUO devices). The function returns true on success. This function must be called before using any API functions that require DUOInstance parameter. (All subsequent calls should use returned DUOInstance to access the device.)
bool OpenDUO(DUOInstance *duo, uint32_t index = 0)
Closes the DUO device. The function returns true on success.
bool CloseDUO(DUOInstance duo)
Starts capturing frames. Requires a frameCallback pointer to user defined DUOFrameCallback callback function. The frameCallback parameter can also be NULL. The pUserData is any user data that needs to be passed to the callback function. The function returns true on success.
bool StartDUO(DUOInstance duo, DUOFrameCallback frameCallback, void *pUserData, bool masterMode = true)
Stops capturing frames. The function returns true on success.
bool StopDUO(DUOInstance duo)
Used to get various parameter values from the DUO device.
Fills the user allocated char array pointer with human-readable DUO device name. The array size should be 260 bytes. The function returns true on success.
bool GetDUODeviceName(DUOInstance, char *val)
Fills the user allocated char array pointer with DUO serial number. This array size should be 260 bytes. The function returns true on success.
bool GetDUOSerialNumber(DUOInstance, char *val)
Fills the user allocated char array pointer with the DUO firmware version. Array size should be 260 bytes. The function returns true on success.
bool GetDUOFirmwareVersion(DUOInstance, char *val)
Fills the user allocated char array pointer with the DUO firmware build. The array size should be 260 bytes. The function returns true on success.
bool GetDUOFirmwareBuild(DUOInstance, char *val)
Fills the user supplied DUOResolutionInfo variable with the currently selected resolution info. The function returns true on success.
bool GetDUOResolutionInfo(DUOInstance, DUOResolutionInfo *resList)
Fills the user supplied double variable with current exposure value in percentage (range [0,100]). The function returns true on success.
The total frame time is divided between the exposure time and sensor readout time. When set to 100%, this means the sensor will open the shutter for the maximum allowed time before the readout, while maintaining the desired frame rate. Therefore the exposure percentage measure is relative and dependent on the set frame rate and resolution.
bool GetDUOExposure(DUOInstance, double *val)
Fills the user supplied double variable with current exposure value in milliseconds. The function returns true on success.
bool GetDUOExposureMS(DUOInstance, double *val)
Fills the user supplied boolean variable with current auto-exposure value (true, false). The function returns true on success.
bool GetDUOAutoExposure(DUOInstance, bool *val)
Fills the user supplied double variable with current gain value in percentage (range [0,100]). The function returns true on success.
bool GetDUOGain(DUOInstance, double *val)
Fills the user supplied boolean variable with current horizontal flip value. The function returns true on success.
bool GetDUOHFlip(DUOInstance, bool *val)
Fills the user supplied boolean variable with current vertical flip value. The function returns true on success.
bool GetDUOVFlip(DUOInstance, bool *val)
Fills the user supplied boolean variable with current camera swap value. The function returns true on success.
bool GetDUOCameraSwap(DUOInstance, bool *val)
Fills the user supplied double variable with current LED PWM (brightness) value in percentage (range [0,100]). The function returns true on success.
bool GetDUOLedPWM(DUOInstance, double *val)
Fills the user supplied variables with current width and height of the image. The function returns true on success.
bool GetDUOFrameDimension(DUOInstance, uint32_t *w, uint32_t *h)
Checks if DUO calibration data is present on the device. The DUO devices are factory calibrated. Fills the user supplied boolean variable with current calibration value (true, false). The function returns true on success.
bool GetDUOCalibrationPresent(DUOInstance, bool *val)
Fills the double array representing (leftHFOV, leftVFOV, rightHFOV, rightVFOV) for currently selected resolution. The function returns true on success.
bool GetDUOFOV(DUOInstance, double *val)
Fills the double array representing stereo rectified (leftHFOV, leftVFOV, rightHFOV, rightVFOV) for currently selected resolution. The function returns true on success.
bool GetDUORectifiedFOV(DUOInstance, double *val)
Fills the user supplied boolean variable with status of undistortion operation. The function returns true on success.
bool GetDUOUndistort(DUOInstance, bool *val)
Fills the user supplied DUO_INTR structure with the calibration intrinsics data. The function returns true on success.
bool GetDUOIntrinsics(DUOInstance, DUO_INTR *val)
Fills the user supplied DUO_EXTR structure with the calibration extrinsics data. The function returns true on success.
bool GetDUOExtrinsics(DUOInstance, DUO_EXTR *val)
Fills the user supplied DUO_STEREO structure with the calibration data. The function returns true on success.
bool GetDUOStereoParameters(DUOInstance, DUO_STEREO *val)
Fills the user supplied accel and gyro values with the current DUOAccelRange and DUOGyroRange range data. The function returns true on success.
bool GetDUOIMURange(DUOInstance, int *accel, int *gyro)
Used to set various parameter values for the DUO device.
Sets the desired resolution, binning and the frame rate. The user supplied DUOResolutionInfo parameter is obtained by calling EnumerateDUOResolutions function. The function returns true on success.
bool SetDUOResolutionInfo(DUOInstance, DUOResolutionInfo resList)
Sets exposure value to the user supplied double value in percentage (range [0,100]). The function returns true on success.
The total frame time is divided between the exposure time and sensor readout time. When set to 100%, this means the sensor will open the shutter for the maximum allowed time before the readout, while maintaining the desired frame rate. Therefore the exposure percentage measure is relative and dependent on the set frame rate and resolution.
bool SetDUOExposure(DUOInstance, double val)
Sets exposure value to the user supplied double value in milliseconds. The function returns true on success.
bool SetDUOExposureMS(DUOInstance, double val)
Sets auto-exposure value to the user supplied boolean value. The function returns true on success.
bool SetDUOAutoExposure(DUOInstance, bool val)
Sets gain value to the user supplied double value in percentage (range [0,100]). The function returns true on success.
bool SetDUOGain(DUOInstance, double val)
Sets horizontal flip value to the user supplied boolean value. The function returns true on success.
bool SetDUOHFlip(DUOInstance, bool val)
Sets vertical flip value to the user supplied boolean value. The function returns true on success.
bool SetDUOVFlip(DUOInstance, bool val)
Sets camera swap value to the user supplied boolean value. The function returns true on success.
bool SetDUOCameraSwap(DUOInstance, bool val)
Sets LED PWM value to the user supplied double value in percentage (range [0,100]). The function returns true on success.
bool SetDUOLedPWM(DUOInstance, double val)
Sets LED PWM sequence from the user supplied DUOLEDSeq array. The maximum size of the sequence is 64 steps. The user must supply the desired number of steps via size parameter. This function can only be called while the DUO is not capturing (i.e. before StartDUO call). The function returns true on success.
bool SetDUOLedPWMSeq(DUOInstance, PDUOLEDSeq val, uint32_t size)
Sets the undistortion operation from the user supplied boolean value. The function returns true on success.
bool SetDUOUndistort(DUOInstance, bool val)
Sets the IMU DUOAccelRange and DUOGyroRange range. The function returns true on success.
bool SetDUOIMURange(DUOInstance, int accel, int gyro)
Sets the IMU sampling rate. Valid values are within the following range [50Hz, 500Hz]. The function returns true on success.
bool SetDUOIMURate(DUOInstance, double rate)
Contains binning options used in resolution configuration. (1x1, 1x2, 1x4, 2x1, 2x2, 2x4)
DUO_BIN_ANY = -1, // Any binning mode available
DUO_BIN_NONE = 0, // No horizontal or vertical binning
DUO_BIN_HORIZONTAL2 = 1, // Horizontal binning by factor of 2
DUO_BIN_HORIZONTAL4 = 2, // Horizontal binning by factor of 4
DUO_BIN_VERTICAL2 = 4, // Vertical binning by factor of 2
DUO_BIN_VERTICAL4 = 8 // Vertical binning by factor of 4
Contains IMU accelerometer range options.
DUO_ACCEL_2G = 0, // DUO Accelerometer full scale range +/- 2g
DUO_ACCEL_4G, // DUO Accelerometer full scale range +/- 4g
DUO_ACCEL_8G, // DUO Accelerometer full scale range +/- 8g
DUO_ACCEL_16G // DUO Accelerometer full scale range +/- 16g
Contains IMU gyroscope range options.
DUO_GYRO_250 = 0, // DUO Gyroscope full scale range 250 deg/s
DUO_GYRO_500, // DUO Gyroscope full scale range 500 deg/s
DUO_GYRO_1000, // DUO Gyroscope full scale range 1000 deg/s
DUO_GYRO_2000 // DUO Gyroscope full scale range 2000 deg/s
DUOFrame structure holds the sensor data that is passed to user via DUOFrameCallback function. Starting with version v1.1.0.30 of the DUO SDK, in order to support research and applications related to 3D Mapping and Navigation and by the popular request from our customers the IMU data sampling rate is user configurable in the range [50Hz, 500Hz]. This means that there could be multiple IMU data samples per single DUO frame. Therefore, we have slightly modified the DUOFrame structure in order to facilitate this extended functionality. We have also added timestamp to individual DUOIMUSample.
Note: To avoid any time-stamp drift issues, both DUOFrame and DUOIMUSample timestamps are internally generated from the same clock source.
// DUO IMU data sample
typedef struct
{
uint32_t timeStamp; // DUO IMU time stamp in 100us increments
float tempData; // DUO temperature data in degrees Centigrade
float accelData[3]; // DUO accelerometer data (x,y,z) in g units
float gyroData[3]; // DUO gyroscope data (x,y,z) id degrees/s
}DUOIMUSample;
// The DUO IMU can run at faster rate than camera frame rate
// Therefore, there could be multiple IMU samples per single DUOFrame
#define DUO_MAX_IMU_SAMPLES 100
// DUO Frame
typedef struct
{
uint32_t width; // DUO frame width
uint32_t height; // DUO frame height
uint8_t ledSeqTag; // DUO frame LED tag
uint32_t timeStamp; // DUO frame time stamp in 100us increments
uint8_t* leftData; // DUO left frame image data (aligned to 64 byte boundary)
uint8_t* rightData; // DUO right frame image data (aligned to 64 byte boundary)
uint8_t IMUPresent; // True if IMU chip is present (DUO MLX)
uint32_t IMUSamples; // Number of IMU data samples in this frame
DUOIMUSample IMUData[DUO_MAX_IMU_SAMPLES]; // DUO IMU data samples
}DUOFrame, *PDUOFrame;
DUOResolutionInfo structure describes DUO frame size, sensor binning and target frame rate. This structure is populated by the EnumerateDUOResolutions function and should not be modified by hand.
typedef struct
{
int width; // Desired DUO frame width or -1 for any
int height; // Desired DUO frame height or -1 for any
int binning; // Horz/Vertical Binning od BIN_ANY for any
float fps; // Desired Framerate or -1 for any
float minFps; // Minimum Framerate for this resolution
float maxFps; // Maximum Framerate for this resolution
}DUOResolutionInfo, *PDUOResolutionInfo;
DUOLEDSeq structure describes the individual LED power values in percentage. The ledPwmValue[0] applies to the leftmost LED, ledPwmValue[1] applies to center LED and ledPwmValue[2] applies to right-most LED on the DUO.
typedef struct
{
uint8_t ledPwmValue[4]; // LED PWM values are in percentage [0,100]
};
The DUO_INTR structure describes the intrinsics calibration parameters of the DUO camera.
struct DUO_INTR
{
uint16_t width;
uint16_t height;
struct INTR
{
double k1, k2, k3; // Camera radial distortion coefficients
double k4, k5, k6; // Camera radial distortion coefficients
double p1, p2; // Camera tangential distortion coefficients
double fx, fy; // Camera focal lengths in pixel units
double cx, cy; // Camera principal point
};
INTR left;
INTR right;
};
The DUO_EXTR structure describes the extrinsics calibration parameters of the DUO camera.
struct DUO_EXTR
{
double rotation[9]; // Rotation matrix of the right camera with respect to the left one
double translation[3]; // Translation vector of the right camera with respect to the left one
};
The DUO_STEREO structure fully describes calibration parameters of the the DUO camera. These values are compatible with OpenCV.
struct DUO_STEREO
{
double M1[9], M2[9]; // 3x3 - Camera matrices (left, right)
double D1[8], D2[8]; // 1x8 - Camera distortion parameters (left, right)
double R[9]; // 3x3 - Rotation between left and right camera
double T[3]; // 3x1 - Translation vector between left and right camera
double R1[9], R2[9]; // 3x3 - Rectified rotation matrices (left, right)
double P1[12], P2[12]; // 3x4 - Rectified projection matrices (left, right)
double Q[16]; // 4x4 - Disparity to depth mapping matrix
};
You can easily integrate the DUO into application or system by using a portable C interface that allows for full access and control to the DUO device. Here is a quick summary of the steps for using the API:
The code snippets below demonstrate each step in more detail:
#include "../include/DUOLib.h" // Include DUO API header file #pragma comment(lib, "../lib/DUOLib.lib") // Link against DUOLib
DUOResolutionInfo ri;
// Select 320x240 resolution with 2x2 binning capturing at 30FPS
if(EnumerateDUOResolutions(&ri, 1, 320, 240, DUO_BIN_HORIZONTAL2+DUO_BIN_VERTICAL2, 30))
{
DUOInstance duo; // Creates instance of DUO
if(OpenDUO(&duo)) // Open DUO
{
// Set selected resolution
SetDUOResolutionInfo(duo, ri);
// Start capture and pass DUOCallback function that will be called on every frame captured
if(StartDUO(duo, DUOCallback, NULL))
{
...
}
}
}
void CALLBACK DUOCallback(const PDUOFrame pFrameData, void *pUserData)
{
...
}
StopDUO(duo); // Stop capture
CloseDUO(duo); // Close DUO
char tmp[260];
GetDUODeviceName(duo, tmp);
printf("DUO Device Name: '%s'\n", tmp);
GetDUODeviceName(duo, tmp);
printf("DUO Serial Number: %s\n", tmp);
GetDUOFirmwareVersion(duo, tmp);
printf("DUO Firmware Version: v%s\n", tmp);
GetDUOFirmwareBuild(duo, tmp);
printf("DUO Firmware Build: %s\n", tmp);
Review the samples provided with the SDK:
Relevant articles and links: