5.6. Hand-eye calibration

Hand-eye calibration is the process used to determine the transformation between a tracked sensor/marker and the camera coordinate system in order to position the camera in the 3D space of the tracking system. The following figure shows a representation of the elements involved in a hand-eye calibration.

Fig. 5.6 Hand-eye calibration

To find the geometrical relationship between the camera coordinate system and the tracking system, several images of a stationary feature (e.g., cross-hair, chessboard) from different points of view are acquired (or the other way round). Then, to solve the calibration problem we have to solve the following equation: AX=XB. Where A and B represent transformations to the marker/sensor and is given by the tracking system ([R1,T1] and [R2,T2] in the above image). X is the unknown transformation matrix we need to find ([R0,T0] in the above image). There are several closed-form solutions to solve the equation such as [Thompson2016], [Malti2013], [Tsai1989] , [Heller2016], and [Morgan2017] among others.

In the following videos Dr. Clarkson explains and demonstrates how to do a hand-eye calibration with a laparoscope and a chessboard:

5.6.1. Free-hand ultrasound calibration

Hand-eye calibration can also be used to perform a free-hand ultrasound calibration, where we want to find the origin of the US image in respect to the tracked ultrasound probe with a marker/sensor attached. The process is very similar. In this case we need a US-compatible phantom in which 2D points can be easily found on the 2D ultrasound images. There are several commercially available phantoms that can be used to perform this task, such as the CIRS Brachytherapy QA phantom. This phantom contains several wires that are perpendicular to each other, therefore the crossing point can be used as a feature. Alternatively, a water bath with a rubber support and a fixed steel pinhead can be used (see figure below).

Fig. 5.7 Image acquisition on a free hand ultrasound calibration

Then, in the same way we identified stationary features using a video image, here we find the coordinates of the pinhead on the ultrasound images, as shown below.

Fig. 5.8 Example of an ultrasound image showing the pinhead during the calibration process.

And the problem is solved as a least-squares minimisation. Examples: [Hu2016], [Mercier2005], [Lasso2014], among others. And a review: [Hsu2009].

5.6.2. Errors

The accuracy of this method depends on the tracking accuracy and the accuracy of the feature extraction from the images.