Augmented Reality enters the operating theatre

Augmented reality is a technology which enhances a normal real-world visual and sensorial experience through the real-time addition of two- or three-dimensional virtual content. It is already used in the most varied fields, from sport to the mechanical industry. It is beginning to make its appearance in the field of medicine with promising applications, especially in the surgical sector. To find out the potential in this field, we spoke with Marina Carbone who works at the Department of Information Engineering, a centre of excellence for computer-assisted surgery at the University of Pisa. In July 2013, Marina completed her doctorate at the Scuola Superiore Sant'Anna with a thesis on surgical navigation.

What role has augmented reality played in the development of surgery and how have they conditioned each other?

Augmented reality in the field of surgery was theorized over a decade ago. We started to try it in 2006 but it became a hot topic in 2016 when the first wearable commercial visors with an acceptable computational capability appeared.
In general, the scientific push began with the introduction of surgical navigators, highly advanced and widely-used systems that can now allow us to show a map of a patient on a screen next to the operating bed, obtained from preoperative imaged that are synchronised in real time with the surgical instrument.
From that moment onwards, also due to the trend towards increasingly less invasive and more personalised surgery, research has concentrated on providing doctors with the instruments to reach these goals. Lastly, as mentioned earlier, the availability of more and more advanced technology, including that devised for other sectors (for example, today’s commercial visors) pushed surgery itself and surgeons to imagine new application scenarios and carry research even further forward.

What are the advantages of using wearable augmented reality in surgery?

The objective is to place the navigator in front of the surgeon’s eyes. The advantages are obvious and many-fold, from the ergonomics of the surgery to the possibility for the surgeon to fully utilise the guidance made available by the navigation. Having navigation systems based on wearable visors would really mean the utopia of a transparent interface between the surgeon and the patient, capable of increasing the level of perception of reality seen by the surgeon, providing a whole series of information aimed at more accurate and personalised surgery.

Your team is working on a project called Vostars, in which augmented reality and surgery are the protagonists. Can you illustrate the main characteristics?

The University of Pisa is the lead partner of project VOSTARS, whose coordinator is Vincenzo Ferrari and for which I am the Scientific Manager. The project involves 12 partners from 4 European countries and aims to be the first to realise a visor that is fully designed around the surgeon’s needs, so as to genuinely state that augmented reality is being used to guide the surgeon’s gestures.
Nowadays, thanks to commercial visors, there is a lot of talk about augmented reality but none of the visors have been designed to comfortably work “at arm’s length”, or looking down at the patient, so, to say it in simple terms, they are uncomfortable and are unsuitable for accuracy under 5mm, so they cannot be used as a precise guide for the surgeon’s gestures. The VOSTARS visor would no longer be just an operating theatre accessory for practice prior to the operation, or to perform a few checks during the operation that are an end in themselves. The VOSTARS visor will be a fully fledged and wearable surgery navigator.

What other prospects are there for the use of augmented reality in the field of surgery?

There are many prospects. On the one hand, we have the frightening speed of the development of embedded technologies, providing more and more advanced computational capacity in ever smaller sizes, and with more miniaturised components. All this will soon lead to the development of even smaller, lighter and ergonomic visors, opening new scenarios for prolonger use.
On the other hand, we have the advances in network infrastructure. It will soon be possible to guide an operation from remote, like in the famous advert of a telephone company.
And finally, research and development in intraoperative imaging systems will increasingly allow for systems which can “update” the patient’s map in real time during the surgery, extending the possibility of using surgery navigators in a broader range of anatomical districts and in other surgery, so these navigators are not merely facilitating devices for some types of surgery but more as enabling devices in others.

Go back