8.4. Visual Perception

One of the main problems in augmented reality, is that projecting computer graphics on top of video, or optics is unconvincing for the human visual system.

Most typically: If merged with video the artificial graphics appear in front of video, even if you are well calibrated such that the rendering is at the correct depth.

So, what research has there been, in the medical domain?

Relatively little: [Wang2017], mentioned [Sielhorst2006] and [KerstenOertel2015].

This video discusses more:

At the end of the video, it is mentioned that volume rendering could be used to give photorealistic effects. Historically this has not been possible in realtime.

However, since 2018, when NVidia released their real-time volume rendering capability, you can create awesome graphics in real time.

(see these pages, especially the videos at the bottom)

But even if you can do the rendering properly, what does the surgeon expect to see?

We need a collaboration between surgeons, technologists and HCI specialists, not just cool graphics! So this area is very much work in progress.