Augmented Reality in Surgery: How AR is revolutionising healthcare

Developments in augmented reality for the use in surgery are moving rapidly, with a report by GlobalData at the start of 2020 predicting the next 10 years will see the AR market in healthcare reach $76 billion (£58.1bn) and overtaking VR by 2030. Although still in its early stages, here are some key moments of augmented reality being introduced to the operating room so far: 

How Microsoft HoloLens is supporting operating theatres

Surgeons have been using the Microsoft HoloLens in operating theatres since 2018. The mixed reality headset is worn optically, allowing a computer generated image to be projected and displayed for the wearer to see simultaneously with the real world.

The HoloLens was praised by surgeons in the UK at St Mary’s Hospital and Imperial College London, who found the technology time-saving and beneficial at improving accuracy to provide the best outcome for the patient.

Using the Microsoft HoloLens surgeons were able to overlay 3D digital models of CT scans onto patients in the operating theatre to assist them in understanding the patient’s unique anatomy of blood vessels, tissue and bones prior to incision.

Dr Philip Pratt, a Research Fellow for Robotic Surgery in the Department of Surgery and Cancer at Imperial College London explains:

“This technology allows us to experience the data that we have collected from patients before their operation in the most realistic and natural way. You look at the leg and essentially see inside of it.”

While the Microsoft HoloLens has been praised for its accuracy in assisting surgeons, in May 2019, a study conducted by the Universy of Pisa using the headset brought this into question, by criticising the use of AR headsets for high-precision manual tasks.

During the study, twenty participants were asked to perform a series of “connect-the-dots” tasks under different conditions while wearing the Microsoft HoloLens and without.

The performance data showed the participants made greater errors using the headset than with the naked eye, prompting the study to conclude that AR headsets should not be used for high-precision manual tasks due to focal rivalry, which is the limited ability of the human eye to focus on two images, real and virtual, at the same time.

The world’s first AR guided surgery with VOSTARS

Fast forward to February 2020 and the same University of Pisa claimed that the world’s first augmented reality guided surgery had taken place at the Sant’Orsola hospital in Bologna, under lead surgeon Giovanni Badiali. 

The aim of the operation was to restore the patient’s biting functionality by resecting and repositioning their upper jaw, a procedure that would normally require external monitors to analyse the patient’s condition throughout.

Instead, Badiali wore an AR headset which allowed him to see data about the patient through the lens of the visor including heart rate, body temperature, blood pressure and breathing rate in addition to pre-operative images from CT, MRI and 3DUS scans.

The technology used was developed by the VOSTARS (Video and Optical See-Through Augmented Reality Surgical Systems) initiative, led by the University of Pisa in collaboration with 12 European partners. 

Commenting on the ground-breaking surgery, Badiali explains:

“We are on the edge of a medical revolution in terms of surgical navigation… We could be working seamlessly in the future. For surgeries, this means a great gain of time and a reduction of mental work to do the connections between the virtual and the real. All the information arrives in real time. Time is the greatest beneficiary of this technology.”

As this was a trialling procedure for the VOSTARS initiative, the future looks promising, with the headset expected to be widely available within the next 3 years. The next steps for the team are to develop a video feed into the visors, which would allow the surgeon to track the real time movements of camera-equipped instruments used in procedures.

Working collaboratively in theatre with Proximie

Augmented reality tools are also being used to help surgeons collaborate virtually during procedures. Proximie is a platform that combines AR and AI technology with live video stream, allowing surgeons to scrub-in on procedures remotely.

Founded by reconstructive plastic surgeon Nadine Hachach-Haram (watch her TED Talk here), the London based company is run by clinicians, with the technology first being used in a surgical procedure on a bomb blast victim in Gaza in 2016.

In September 2020, it was announced that 5000 procedures had taken place using the software, with an average of 700 procedures taking place each month in 35 countries across 5 continents. Dr Jim Porter, Director of Robotic Surgery for Providence St. Joseph Health and Swedish Medical Center in Seattle was recently able to guide a London based surgical team 4,700 miles away through a complex series of incisions on a patient with a rare form of cancer. He explains:

“I have used many remote-surgical systems and nothing compares. I particularly like the feature that offers remote surgeons a 360-degree view of every corner and every machine in the operating room. There’s also no lag or latency, meaning it’s a seamless experience when I share my expertise.”

A third of NHS hospitals are now using Proximie, with the aim of this increasing to 40% by the end of 2020. 


What’s happening in healthcare now

The summer of 2020 has seen a sharp increase in developments for the augmented reality surgery space, largely due to the advances in AR headset technology.

June 2020 saw the first augmented reality minimally invasive spine surgery take place at Rush University Medical Centre in Chicago. With lead surgeon Dr Frank Phillips using the Augmedics xvision Spine System to allow him to perform a lumbar fusion with spinal implants whilst having a 3D view of the patient’s spinal anatomy through their skin during the procedure. 

In August 2020 husband and wife team Jennifer and Jon Silva based at the Washington University in St Louis developed a cardiac hologram. Essentially, the hologram allows the surgeon to visualise a patient’s heart with real-time information in 3D, with trials indicating it could improve accuracy during minimally invasive arrhythmia (heart rhythm) surgeries. 

Build your AR marketing campaign with Reydar

At Reydar we are perfectly placed both technically and creatively to help you embrace AR within your company, making AR advertising accessible for companies of all sizes. To discuss how Reydar could bring your brand to life, get in touch today.