Augmented Reality in Surgery: How AR is revolutionising healthcare

Developments in augmented reality (AR) for the use in surgery are moving rapidly, with a report by GlobalData at the start of 2020 predicting the next 10 years will see the AR market in healthcare reach $76 billion (£58.1bn) and overtaking VR by 2030. Although still in its early stages, here are some key moments of augmented reality being introduced in surgery so far: 

What is augmented reality surgery?

Augmented reality (AR) in surgery refers to the use of computer-generated images, data, text, or other information overlaid onto a surgeon’s field of view (FOV) during a surgical procedure. The use of AR technology in this practice is to enhance the surgeon’s perception and understanding of relevant information, this could include the patient’s anatomy, medical history, or even live readings of the patient’s vital signs.

Ultimately, augmented reality in surgery aims to improve precision, safety and efficiency within the operating room.

What are the main benefits of using AR in surgery?

Augmented reality is becoming widely used in many industries, from gaming to retail each industry derives unique benefits from the technology. But few (if any) will derive the benefit to help save lives and improve healthcare systems like AR in surgery does. Here are some of the top benefits of AR surgery:

1. Improved surgical planning: By visualising the patient’s anatomy before the procedure, surgeons can identify potential risks and create more accurate surgical plans.
2. Enhanced precision and accuracy: AR allows surgeons to “see through” the patient’s skin and tissues, helping them navigate complex anatomical structures more precisely.
3. Reduced surgical complications: With better visualization and guidance, surgeons can minimize damage to healthy tissues, reducing the risk of complications and improving patient outcomes.
4. Shorter operation times: The improved efficiency and accuracy provided by AR can help reduce surgeries’ duration.
5. Faster recovery times: Minimally invasive procedures made possible by AR often result in quicker patient recovery times and reduced postoperative pain.

Although still in the early stages of use/development, AR surgery holds great promise for improving surgical processes and outcomes and revolutionising the medical industry. As this technology advances, it is expected that its applications in surgery and other areas of healthcare will also advance.

Now we have explored what AR surgery is, let’s take a look at how it is being used in the industry.

How Microsoft HoloLens is leading AR surgery

AR surgery sounds incredibly futuristic, yet it is a destination that the healthcare industry is guaranteed to reach in the coming years. AR being used in surgery has the potential to simplify processes, increase success rates, and most importantly, save time in operating theatres. Whilst AR surgery is in its infancy, it is an exciting road for the technology, and Microsoft Hololens is the driving force behind it all.

Surgeons have been using the Microsoft HoloLens in operating theatres since 2018. The mixed reality headset is worn optically, allowing a computer-generated image to be projected and displayed for the wearer to see simultaneously with the real world. The Microsoft Hololens has given a glimpse into the future of AR surgery, the technology still has a long way to go, but the device has highlighted how surgery can be simplified and streamlined by using AR.

The HoloLens was praised by surgeons in the UK at St Mary’s Hospital and Imperial College London, who found the technology time-saving and beneficial in improving accuracy to provide the best outcome for the patient.

Using the Microsoft HoloLens surgeons were able to overlay 3D AR digital models of CT scans onto patients in the operating theatre to assist them in understanding the patient’s unique anatomy of blood vessels, tissue and bones prior to incision.

Dr Philip Pratt, a Research Fellow for Robotic Surgery in the Department of Surgery and Cancer at Imperial College London explains:

“This technology allows us to experience the data that we have collected from patients before their operation in the most realistic and natural way. You look at the leg and essentially see inside of it.”

While the Microsoft HoloLens has been praised for its accuracy in using augmented reality to assist surgeons, in May 2019, a study conducted by the University of Pisa using the headset brought this into question, by criticising the use of AR headsets for high-precision manual tasks.

During the study, twenty participants were asked to perform a series of “connect-the-dots” tasks under different conditions while wearing the Microsoft HoloLens and without.

The performance data showed the participants made greater errors using the headset than with the naked eye, prompting the study to conclude that AR headsets should not be used for high-precision manual tasks due to focal rivalry, which is the limited ability of the human eye to focus on two images, real and virtual, at the same time. But it is important to remember that utilising AR in surgery is still a new practice, AR headsets technology, accuracy and sophistication, whilst advanced, still have a long way to go. As AR headsets become more advanced, their use in surgery is likely to become less of an obstacle, and more of an enhancer.

The world’s first AR-guided surgery with VOSTARS

Fast forward to February 2020 and the same University of Pisa claimed that the world’s first augmented reality guided surgery had taken place at the Sant’Orsola hospital in Bologna, under lead surgeon Giovanni Badiali. 

The aim of the operation was to restore the patient’s biting functionality by resecting and repositioning their upper jaw, a procedure that would normally require external monitors to analyse the patient’s condition throughout.

Instead, Badiali wore an AR headset during surgery which allowed him to see data about the patient through the lens of the visor including heart rate, body temperature, blood pressure and breathing rate in addition to pre-operative images from CT, MRI and 3DUS scans.

The technology used was developed by the VOSTARS (Video and Optical See-Through Augmented Reality Surgical Systems) initiative, led by the University of Pisa in collaboration with 12 European partners. 

Commenting on the ground-breaking AR surgery, Badiali explains:

“We are on the edge of a medical revolution in terms of surgical navigation… We could be working seamlessly in the future. For surgeries, this means a great gain of time and a reduction of mental work to do the connections between the virtual and the real. All the information arrives in real time. Time is the greatest beneficiary of this technology.”

As this was a trialling procedure for the VOSTARS initiative, the future looks promising, with the headset expected to be widely available within the next 3 years. The next steps for the team are to develop an AR video feed into the visors to be used in surgery, which would allow the surgeon to track the real-time movements of camera-equipped instruments used in procedures.

AR surgery with Proximie

Augmented reality tools are also being used to help surgeons collaborate virtually during procedures. Proximie is a platform that combines AR and AI technology with live video stream, allowing surgeons to scrub in on procedures remotely.

Founded by reconstructive plastic surgeon Nadine Hachach-Haram (watch her TED Talk here), the London-based company is run by clinicians, with the technology first being used in a surgical procedure on a bomb blast victim in Gaza in 2016.

In September 2020, it was announced that 5000 AR procedures had taken place using the software, with an average of 700 procedures taking place each month in 35 countries across 5 continents. Dr Jim Porter, Director of Robotic Surgery for Providence St. Joseph Health and Swedish Medical Center in Seattle was recently able to guide a London based surgical team 4,700 miles away through a complex series of incisions on a patient with a rare form of cancer. He explains:

“I have used many remote-surgical systems and nothing compares. I particularly like the feature that offers remote surgeons a 360-degree view of every corner and every machine in the operating room. There’s also no lag or latency, meaning it’s a seamless experience when I share my expertise.”

A third of NHS hospitals are now using Proximie, with the aim of this increasing to 40% by the end of 2020. 

What’s happening in healthcare now

The summer of 2020 has seen a sharp increase in developments for the augmented reality surgery space, largely due to the advances in AR headset technology.

June 2020 saw the first augmented reality minimally invasive spine surgery take place at Rush University Medical Centre in Chicago. With lead surgeon Dr Frank Phillips using the Augmedics xvision Spine System to allow him to perform a lumbar fusion with spinal implants whilst having a 3D view of the patient’s spinal anatomy through their skin during the procedure. 

In August 2020 husband and wife team Jennifer and Jon Silva based at the Washington University in St Louis developed a cardiac hologram. Essentially, the hologram allows the surgeon to visualise a patient’s heart with real-time information in 3D, with trials indicating it could improve accuracy during minimally invasive arrhythmia (heart rhythm) surgeries. 

Build your AR marketing campaign with Reydar

At Reydar we are perfectly placed both technically and creatively to help you embrace AR within your company, making AR advertising accessible for companies of all sizes. To discuss how Reydar could bring your brand to life, get in touch today.