New immersive technologies such as virtual reality (VR), augmented reality (AR) and mixed reality (MR) are currently being used in many different ways. From gaming, automation, education and therapy, these immersive technologies are helping train as well as simplify communication between people. (If you need a quick guide comparison guide on these technologies, check out VRFocus‘s guide here). For the first time, three surgeons from Mumbai and London became digital 3D avatars in an operating theatre at The Royal London Hospital and were then able speak to one another in real-time to discuss on how to operate on the patient with the aid of pre-uploaded patient scans.
Aetho’s Thrive software on the Microsoft Hololens is a MR application all about connecting people and information in immersive environments. Professor Shafi Ahmed at the NHS’s The Royal London Hospital was doing a colonoscopy operation on a patient wearing a Microsoft Hololens – as seen in the image below. Professor Ahmed explains that they chose to do this project to “think about the way we communicate from doctor-to-doctor or doctor-to-patient.”
Professor Ahmed was joined by Professor Shailesh Shrikhande, a Cancer Surgeon at Tata Memorial Hospital in Mumbai (the largest cancer hospital in India), as well as Mr Hitesh Patel, Consultant Colorectal Surgeon at BMI The London Independent Hospital. They were also joined by Ian Nott, Co-Founder and CTO of Aetho who was based in Atlanta, USA. All four participants wore a Microsoft Hololens, appearing as moving graphic avatars to one another, with each able to see and hear one another. They were able to look at pre-uploaded patient scans that appeared as three-dimensional holograms of the tumour. In the video below, you can see each specialist discuss and analyse the patient’s data through Professor Ahmed’s perspective, the footage captured from his Hololens.
VRFocus spoke to Professor Ahmed about the project in the video interview below. He explains that the team were connected into a virtual space where they could share the scans, images of the patients, interact with them and then discuss the case in more detail, similar to a multidisciplinary team meeting that surgeons normally do in healthcare practices. The experience was like having a very ‘lucid conversation’ about the patient. Apparently, after you get past the initial shock of feeling like Iron Man, the experience is no different to having a person sit next to you and conversing.
Professor Ahmed is very excited about being in the healthcare space right now and believes that they’re undergoing the fourth industrial revolution. “It’s a question about globalization, if you want help and support – well actually the whole world can support them. These are the type of technologies that will connect people, make the world much smaller and actually make healthcare more equitable”, he says. For the future of surgery, he’d like to teleport or ‘holoport’ himself into another part of the world, walk around the room, stand over the surgeon’s shoulder, see what they’re doing, give advice and then disappear. Although this might seem like this is far in the future, it’s the direction he sees it going and is something he is working on.
Aetho approached Professor Ahmed at Cannes Lions after seeing his talk about creating a digital avatar of himself using photogrammetry. Aetho were working on the concept of avatars, holograms and telepresence for their software Thrive. The two met and Professor Ahmed’s VR company Medical Realities then collaborated with Aetho and co-ordinated the project with the hospitals to do a world’s first MR conference call with 3D digital assets during a real-time surgery.
He explains that new technologies are severely needed because globally there is an increasing demand for healthcare, but not enough capacity to cope with it. Unfortunately, with little funding it’s difficult for public services like the NHS to justify new healthcare services. He hopes that by using new technologies such as these, that healthcare can be better, more efficient and eliminate the need to travel in order to do certain operations. He believes A.I. and robotic machines will take over routine jobs, and doctors as well as surgeons will have to re-design their roles in this future landscape.
Whatever the future holds, this is an exciting step for future healthcare operations. It could save a lot of money on expensive travel, save time on treating patients and free time for doctors and surgeons to treat more patients. If you want to find out more about the project, watch the video below. You can also find out more about how immersive technology is being implimented into the world of healthcare with VRFocus‘ The VR Doctor and Emotion Sensing series.