Mixed reality 3D hologram technology used in surgery

Digital holograms are an emerging tool in operating rooms across the globe. Dr Travis Falconer, a specialist shoulder and elbow surgeon at Hollywood Private Hospital, WA, used a HoloLens mixed reality headset during surgery.

Mixed reality is the merger of real and virtual worlds with enhanced immersion and interaction.

Falconer was able to see a 3D hologram of the surgical plan over the patient’s anatomy in real time, which allowed him to use the planning software and refer to data analysis during surgery. By using hand gestures and voice commands, he was able to interact with the holographic objects.

The aim of using mixed reality techniques is to enhance precision and simplify complex surgeries. Being able to refer to the surgical plan during surgery provides another layer of checks and balances for surgeons.

Falconer said he was proud to be selected by medical technology company Stryker to trial the new medical technology in WA.

“I think this is just the start of what’s to come regarding pre-operative planning, the use of navigation, mixed reality and the introduction of robotic surgery into the upper limb,” Falconer said.

This type of technology allows the surgeons to gain access to computer-based solutions in real time and to access 3D holograms related to the patient imaging or the surgical technique during the procedure while remaining sterile. They can also remotely interact with colleagues located outside the theatre.¹

Director of Medical Services John Maxwell said the use of mixed reality was at the forefront of state-of-the-art technology in health care.

“It offers so much potential in the future, such as using artificial intelligence and case planning optimisation,” Maxwell said.

Reference

^{1 Gregory TM, Gregory J, Sledge J, Allard R, Mir O. Surgery guided by mixed reality: presentation of a proof of concept. Acta Orthop. 2018 Oct;89(5):480-483. doi: 10.1080/17453674.2018.1506974. PMID: 30350756; PMCID: PMC6202760.}