Doctors test augmented reality during surgery

Read time 3min 40sec
A doctor using the Microsoft HoloLens while performing surgery.
A doctor using the Microsoft HoloLens while performing surgery.

Microsoft's mixed reality headset, the HoloLens, will soon adorn the heads of doctors as they perform delicate surgeries, such as heart operations, at Alder Hey Children's Hospital in the UK.

The term 'mixed reality' refers to the merging of the physical, virtual and augmented realities. Virtual reality (VR) went mainstream in early 2016 when consumer headsets by Oculus and Samsung were released, and augmented reality (AR) gained popularity when the game Pok'emon Go captivated millions of people around the world in July 2016.

Wearers of the HoloLens headset are able to see the physical world around them but have a virtual layer overlaid in front of their eyes. It also creates 3D holograms and lets wearers interact with them in the real world.

Alder Hey Children's Hospital, based in Liverpool, is one of the largest children's hospitals in the UK, caring for more than 270 000 kids each year. The hospital hopes introducing the HoloLens in operating theatres will improve surgical procedures.

"It's incredibly important that we embrace technology to get our best people out there and do extraordinary things," says Iain Hennessey, clinical director of innovation at Alder Hey. "At the end of the day, the health of children is more important than anything else."

Surgeons wearing the headsets will be able to see up-to-date information on a patient while operating.

"Imaging a patient's heart from the inside and from the outside is absolutely essential," says Rafael Guerrero, a cardiac surgeon at Alder Hey.

"I have to visualise that 3D view in my head in order to do this operation. You can display those images on a screen in the operating theatre sometimes, but it's not easily accessible; and I can't leave in the middle of an operation to go get more information about my patient. In many cases, the heart has already stopped in order for us to operate.

"Microsoft HoloLens and mixed reality will, in the future, enable me to have a patient's scans in front of me while I'm doing the operation. If I can use technology to obtain that information, to see those images in front of me, that helps me tremendously and improves the outcome for my patient."

Previously, before an operation, a team of clinicians, surgeons, doctors and nurses was assembled to review a patient's ultrasounds, CT scans and angiograms to determine the best course of treatment. The surgeon would have to use his hands to check up on this information if he needed it during surgery.

Guerrero says using the HoloLens to view medical images and notes in real-time means he can keep his hands free.

Along with the HoloLens, the hospital will introduce large-screen Surface Hubs in meeting rooms. These will allow doctors to collaborate and use one digital screen to share patient charts, tests and medical images that are traditionally written on paper and located in several places around the hospital.

The technology to work the HoloLens and Surface Hub in this situation is not operational yet. The hospital is working with Microsoft partner Black Marble to develop an app.

"HoloLens has powerful visualisation capabilities. Coupled with the Surface Hub, which is excellent for transforming collaborative experiences, we saw a range of opportunities for creating engaging user experiences," says Robert Hogg, chief executive of Black Marble.

"The common factor for both these devices is that they are delivered on the Universal Windows Platform, which enabled us to write the application once, and still take advantage of the best features of both devices."

Black Marble's app will utilise InkCanvas and InkToolbar so multiple people can write notes on the Surface Hub at once. These are then time-stamped and show the name of the author, and can be uploaded to HoloLens during a later operation.

Login with