It is becoming increasingly common to see virtual reality (VR) technology used in healthcare. Dentists are being taught new procedures using VR and cardiologists are explaining complex medical conditions with use of 3D VR models. That isn’t the only application, however, as some doctors are using VR to help patients escape the hospital environment for a while.
St Joseph’s Children’s Hospital in Tampa, Florida has received an installation called the Infusionarium, an immersive virtual reality system that is specifically designed for young patients to help them relax after hours of chemotherapy and other treatments.
The system was designed by a former Disney Imagineer, Roger Holzburg who is the co-founder of Reimagine Well, a company which is attempting to create what is describes as immersive healing environments. Holzburg said: “Every month we go out and ask our patients, if you could heal anywhere in the universe, and there were absolutely no rules, what place would best promote your healing?”
Paediatric Haematology and Oncology specialist at the hospital, Dr. Mark Mogul is enthusiastic about the system. “While kids are in the hospital, to have this escape from the day-to-day routine of taking vital signs, getting blood drawn, chemotherapy – it’s such an amazing opportunity for them,” he said, “This Infusionarium… we’re excited for the kids, but the docs are just as excited for the other docs. It’s an incredible experience!”
The Infusionarium provides a number of experiences and programs that are designed for patients undergoing procedures and treatment such as chemotherapy, dialysis, radiation therapy and more.
There are also disease-specific ‘survival guides’ that provide instructions and resources, presented by scientists and medical experts to help educate and support patients and their families through the treatment process.
VRFocus will continue to bring you the latest on new VR technology for healthcare.
Use of virtual reality (VR) in healthcare is becoming increasingly common. The practice of dentistry seems to be one area that is using the technology to not only train new dentists, but also improve dental procedures.
Researchers at the University of Huddersfield are working on a new way of using VR to train surgeons. The VR surgery uses an Oculus Rift along with motion tracking sensors to give trainees a view of procedures in a 360-degree virtual operating theatre that removes the problems of blocked sightlines and blind spots that exist in most operating theatre environments when inhabited by assistants and nurses.
The project was begun by a PhD student named Yesh Pulijala, a dental surgeon who recognised the problems inherent in current methods of training. Pulijala’s supervisor, Professor Minhua
Ma has been involved in research into the use of computer games for education and training and co-authored a chapter in a book with Pulijala on the use of VR in surgery.
The project also involved Dr David Peebles, an acknowledged expert in cognitive processes. One of Dr Peebles own students, Matthew Pars is using the work done by Yesh Pulijala as a platform for further investigation into the subject. Dr Peebles said: “A lot of training of surgeons is about the nuts and bolts of doing the surgery,” said Dr Peebles. “But there is a whole aspect of their expertise that is called situation awareness, which is about developing a mental model of the whole task and the whole environment. It’s much broader than just doing the surgery and really vital for training novices. Now we are trying to encapsulate all the extra knowledge that experts have and Matt is exploring ways to incorporate it into the system that Yesh has designed.”
The VR surgery project underwent evaluation in India. A team consisting of Professor Ma, Dr Peebles, Yesh Pulijala and Matthew Pears. The team visited seven dental schools in India to demonstrate to technology. A video on the trip can be seen below.
VRFocus will continue to report on VR innovations in the healthcare industry.
Electronics company Samsung have partnered with The Hospital for Sick Children to create a space within the hospital containing Samsung electronics such as Gear VR headsets to allow patients and their families to relax away from the clinical environment.
The Hospital for Sick Children, also known by the abbreviation SickKids, is one of the largest research hospitals dedicated to children’s health in Canada. The staff at the hospital hope that the new Samsung Space facility will help children and families who are undergoing the often difficult and painful process of treatment.
“We are grateful to Samsung Canada for their generous gift which allowed us to transform the space physically as well as hire staff dedicated to developing programming and managing the Space,” says Karima Karmali, Director, Centre for Innovation & Excellence in Child and Family-Centred Care, SickKids. “We believe in the value of play in our commitment to children’s health and development, and know this inviting, innovative space will help our patients in their healing.”
The technology available in the Samsung Space includes several Gear VR headsets that allow patients to immerse themselves in environments such as the moon or Mars. There is also a huge 75” television, so families can watch videos and play games together, and a Synced Interactive Table, a augmented reality (AR) device that can bring a child’s favourite mobile phone app to life.
“We continue to be inspired by the courageous young patients at SickKids. As part of our continued, meaningful partnership with SickKids, we hope this new Samsung Space can make a difference by providing fun experiences that allow children to be children and help bring them closer to the people, places and moments they miss most,” said Mark Childs, Chief Brand Officer, Samsung Canada.
The Samsung Space is located on the 9th Floor of the hospital, and will be open six days a week to patients and their families.
Three new projects are in development at Lucile Packard Children’s Hospital Stanford that utilise virtual reality (VR) technology to improve the education, health and hospital experience of children who require medical care.
The three projects include the Stanford Virtual Heart, which allows doctors to explain often complex congenital heart defects to children and their parents by using a VR model of a heart, which patients or their parents can go inside of using VR headset to see the place where the defect is located and understand how it can affect health. Another example is Project Brave Heart, a VR program to help patients understand the procedures they will undergo as well as assist in learning relaxation techniques to help reduce distress and anxiety. Thirdly, there is the use of 3D VR modelling to help surgeons map the route during a complex procedure and practice the series of actions they will need to take by using images previously gathered from CT and MRI scans and converting them into a complete 3D model.
The three programs are currently in fairly early stages of development and implementation, but doctors hope that is the programs prove successful, they can be expanded into other hospital departments to help children with many types of conditions such as cancer, hearing issues and neurological diseases.
“Because we are situated in Silicon Valley, we are in an ideal position to be a vanguard in this space and to partner with the companies that are on the cutting edge of this technology,” says Stephen Roth, MD, MPH, chief of paediatric cardiology and director of the Children’s Heart Centre. “Thanks to our collaborators and supporters, we’ve leveraged this technology to make real progress in the medical field that can spread far beyond Stanford. It’s truly state-of-the-art, and we are very excited about it.”
VRFocus will continue to bring you news of developments for VR in healthcare.
We’re still a long way from from being able to provide timely treatment to everyone who needs it, but we could be on the brink of change thanks to VR
Few tech topics are hotter right now than virtual reality (VR). Though it’s been around for decades, VR has at last entered the world of consumer electronics via devices like the Oculus Rift and HTC Vive and, increasingly, headsets that can be used in conjunction with our mobile phones. But VR isn’t just a technological game-changer: it could transform the way we tackle mental health problems.
Not so long ago, talking about psychological problems was taboo. Now the scale of these disorders is no longer a secret. We know, for example, that one in four people will experience mental health issues at some point in their life. The ramifications from this ocean of distress aren’t merely personal; the socio-economic consequences are profound. Nearly half of all ill health in working age adults in the UK is psychological. Mental illness costs the UK economy £28 billion every year — and that’s excluding NHS costs.
When Virtual Reality (VR) content is experienced using a VR headset the outcome is a virtual world. In that world, the possibilities are limited only by the imagination. This approach has given education an opportunity to leapfrog existing teaching practice by providing educationally beneficial immersive experiences.
The first among these is that VR allows students to be absolutely captivated by the teaching. All teachers want their students to fully concentrate on their lesson. But teachers know that students do not, and sometimes cannot, concentrate that way. However, they will in their ‘spare time’ devote many hours of concentration to gaming. Working late into the night on their consoles reaching the next level is common, but how many of those millions would do the same to study.
Educational success is the key determinant for a good future. Unfortunately, many students are failed by the education system. These are not all disruptive, lazy or `stupid`. In many cases the current education system does not suit them and they get nothing from it except boredom and demotivation.
It has been demonstrated that anyone from the highest achiever to the ‘total academic failure’ can benefit from the way virtual worlds impart educational content. The playing of games for hours shows that people can easily concentrate for that length of time and that during that time they are learning – even though the lessons may seem meaningless outside the game itself. These lessons are absorbed without even knowing it is happening. How else can players rise through the levels and achieve their rewards.
Game on
Many gamers are keen users of VR headsets because they impart a deeper impression of the virtual world than a computer screen through immersive interaction. The cost of these devices is becoming reasonable as the hardware and its application areas evolve: Google’s Cardboard VR glasses in combination with an application for instance cost very little money while high-end devices with faster refresh rates, hand control, and motion capture used in gaming are becoming affordable being currently in the hundreds of pounds range.
Transferring the lessons of gaming to educational applications produces the immediate benefit that the students can fully engage in the virtual world and learn their lessons there. Students of any age can be fascinated by virtual worlds which means it can be deployed from pre-school through to tertiary education then continuously applied to in-work training.
The UK alone requires 70,000 new engineers every year to cover the current skills shortage. And a serious problem currently facing STEM (Science, Technology, Engineering and Mathematics) education is that students learn each subject in isolation. In a world of work where cross-disciplinary and cross-organisational team skills are required this is an obvious disadvantage to job seekers and employers.
Dassault Systèmes is working with several universities developing solutions to overcome this problem. This involves international teams of STEM students combining their different disciplines in real-time on the same projects. This is supported by using ‘intelligent’, cloud hosted 3D models as a shared and unifying resource to collaborate around. Working in cross-disciplinary teams means communicating, sharing and combining experiences and skills. Learning this way makes the students more employable because their educational experience and acquired set of skills closely matches their eventual work environment.
Virtual Worlds of STEM
The use of virtual worlds can easily supplement traditional chalk and talk lessons and replace 2D presentation that many students find uninspiring. Stepping into a virtual world shows students the what, how and why that make learning interesting and compelling. VR can be applied to most learning and is particularly appropriate for teaching STEM subjects or training people for engineering work. This is because of the highly-realistic 3D visualisation content including animated 3D simulations of very complex products, processes and systems.
Here the advantages are clear. Students can for example work on a virtual aero engine without the need to have a real one. The cost implications are considerable because not only is it way more cost-efficient to use a virtual engine but the risk of damage or harm is greatly reduced as well.
And a whole class of trainees can simultaneously work on the virtual model of an engine and see it in all the detail they need to fully understand it. Its operation can be simulated so that students can even be inside the engine as it is working. This is impossible in real-life and takes learning experience to a new level.
Currently Dassault Systèmes is partnering with thousands of educational establishments and courses at all levels including 40 British universities where design, architecture and engineering are benefitting from the universal language of 3D. Across the world up to 5 million students currently encounter our digital simulation capabilities based on the 3DEXPERIENCE platform as part of their education programme.
One project in a school for young engineers uses stereoscopic 3D visualisations of a car, a motorbike and a vintage vehicle to immerse students in the virtual world where they can understand and envisage theoretical concepts in way that makes it fun and engaging to learn. Based on Dassault Systèmes digitised 3D models students are encouraged to use their new skills to learn from and then apply what they have learned using these models again. So far, secondary education students have created virtual representations including a design for a fishing reel, a mixer, a land speed record beating vehicle, and a mechanical hand. These have been developed in the virtual world where anything is possible and learning is as exciting as it gets.
Home Work
Using virtual content and VR hardware allows students to work on their projects at a distance from their place of education – to effectively take the lesson away and work on that aero engine or any other project at home. This means students can develop their studies and overcome weaknesses in the same environment that they were using at school, college or work. Just like gaming they achieve instant feedback and rewards while moving up to the next level.
Training for work uses the same approach: to create virtual environments where people can experience dangerous and hazardous activities – without any risk to themselves or others. Setting up a nuclear power station, a deep mine or an oil refinery with its full detail in the virtual world means people can learn processes and practice safely as many times as they need to while gaining the confidence and experience required to operate in the real world.
This kind of training lets people experience virtually any event such a leak, a flood or a fire while learning how to deal with it. Developing digital content is educational itself since students work on industry leading design simulation application such as SOLIDWORKS. The development of virtual worlds brings together many different disciplines each of which learns about the others through collaboration. This in turn leads to innovation as departments, subjects and ideas merge.
Life Force
In medicine Dassault Systèmes showcased with the Living Heart Project how 3D modelling revolutionises personalised therapies: based on the patient data, a virtual heart is created that gives medical professionals the opportunity to simulate its specific activities as well as the operation of blood, nerves, stents and pacemakers. This is a major advance for doctors, surgeons and device manufacturers who can at last see the individual heart and its malfunctions – virtually and from within.
This and many other projects show how ´things´ work, how they interconnect and how it is possible to transition between the virtual and the real world that is benefitting both. Portable devices let the work be taken anywhere so people can remind and update themselves whenever they need to. The cost and time savings as well as the educational benefits are significant and carry on through the whole of life.
Even subjects beyond STEM can benefit greatly from virtual worlds. History can be re-created and brought back to life, physical and social geography modelled, creatures virtually re-animated. The introduction of AI (Artificial Intelligence) could be used to simulate a board room, a courtroom or any other encounter.
Companies from the automotive industry for instance are addressing their customers using their own virtual worlds: Small physical retail spaces can virtually demonstrate huge ranges of products such as vehicles and furniture with all options being available while potential buyers are able to individualise and experience their product as never before.
The low cost and speed of creating and deploying virtual worlds for education means that in India for example using VR has become part of the Digital India and Skills India programmes that include thousands of training partners and up to 55 million students who learn and train to outdistance the rest of the world towards the dream of making India the skills capital of the world.
There has been a revolution in the past few years with advanced software applications and hardware technologies becoming available to ´anyone´. The next years will see that more and more people are educated in a variety of ways using virtual worlds to see and experience the real world and study in new, more comprehensive ways. Its power to teach is well proven while its educational and training potentials are literally unlimited.
Berlin-based company Scopis has won clearance from the Food and Drug Administration (FDA) in the US for its augmented reality (AR) navigation system for ENT surgeries.
The technology provides visual guide paths superimposed over live imaging from the endoscope to guide surgeons towards difficult targets that may be hard to find with traditional methods.
The system relies on the Hybrid Navigation platform developed by Scopis that allows for real-time head tracking using optical and electromagnetic imaging, automatically aligning previously captured images with the real-time data coming in from the endoscope.
In the nextinstalment of feature series Dr. Raphael Olaiya, a NHS doctor and medical education academic, who works with the NHS on virtual reality (VR) immersive training programmes for doctors and nurses. Discusses the meeting point of VR and A.I. for healthcare.
We can see it, feel it, touch it, taste and even smell it, virtual reality (VR) is very tangible and its sophistication and fidelity are increasing at a rate where anyone who self-identifies as a technology fan must chase updates biweekly to prevent being left behind. An accepted consensus is that by 2030 VR/Augmented Reality (AR) with be the fundamental platform for mobile communication and be firmly embedded as a mainstay within the industries of energy, entertainment, education, training and healthcare.
Conversely A.I., artificial intelligence from the perspective of the average self-identified technology fan can only be perceived in depictions from what Hollywood directors spoon feed us. An optimistic but plausible feat all the same, however actualizing it seems much more of a lucid dream than reality for the masses. When Siri, Catana and Amazon Echo are publically self-hailing themselves as having artificially intelligent capability it throws the masses of technology hobbyists into confusion because the only thing they see here is glorified voice recognition search tools.
A.I.: Glorified Voice Recognition Search Tools
A.I. to the masses means firstly on demand adaptive learning and android-esque tendencies physically, emotionally or intellectually stemming from Hollywood’s portrayal of characters including J.A.R.V.I.S from Iron Man, Skynet, Bicentennial Man, Ex Machina, Star Wars amongst others. This example highlights a problem with the classification of the definition of A.I. which has lead to a degree of trivialization of commercial A.I. and reducing its appeal because the recurrent depictions are just simply not realistic technology right now.
Trivialization of Commercial A.I. and Reducing its Appeal
Delving deeper in A.I. technicalities, A.I. is certainly alive and kicking. IBM Watson at the forefront with Google Deepmind, Microsoft Oxford and currently changing the way businesses trade, the way doctors diagnose and how engineers construct. Deep learning algorithms that are able to re-synthesize formulae for future tasks based on the results of previous formulae. However, the reach of IBM Watson and its cousins seems somewhat disconnect to the masses. Why? This boils down to the sedimentary reasons of;
The current level of cutting edge A.I. in 2017 focuses on area specific ( i.e. IBM Watson X Sloan Kettering partnership focusing on more effective and efficient cancer diagnosis and treatment ) big data processing and delivers directed specific answers after arduous instructions and direction from the subject matter experts i.e. world-renowned oncologists. So the focus of technological advancement is not on A.I. for consumer tech rather for area specific specialist.
There is not a strong enough tie yet between the organisations/companies with mass consumer reach and the tech communities working hard on breakthrough expert level programming needed to adapt up to date artificial intelligence tech to a use that will serve the masses directly. This disconnection will soon bridge.
Tangible applications to channel current available A.I. tech into “perceived real A.I.” applications have not taken off in the open market yet. Eager A.I. fans wait for the anticipated killer/golden app. The app that will utilise A.I. technology in a way that bridges away from the island of gimmick and enters a realm of the value add whether it be saving time, money, or enhancing entertainment.
VR/AR represents a golden ticket opportunity to present to the mass consumer market a application of artificial intelligence that is very easy to relate to and get excited about for anybody. This highlights the importance of the user experience and perception of A.I., for example, an A.I. application field that billions of dollars are currently being invested to currently is an A.I. medical diagnosis tool for patients with no/delayed access to a doctor. It doesn’t take a controlled experiment for us to agree that the user experience would be more successful with a realistic CGI android style VR/AR artificially intelligent voice
A Golden Ticket Opportunity
Recognising an A.I. nurse/doctor as such more than a messenger style chat box despite having the exact same level of A.I. sophistication is such.
Whether or not we define or perceive A.I. as algorithmic formulae clever enough to generate more formulae and build upon past experience, in order to learn deeply coupled with a powerful searching function doesn’t take away the fact that how we interact with it is the most important feature of all! After all, A.I. is for the betterment of our lives, to increase joy and pleasure, ease suffering and further the realisation of our ability as humans to supersede ourselves.
VR and A.I. intertwining in a multidisciplinary approach to serve us more holistically is the goal. Healthcare a universal priority is a worthwhile focus point for this blending.
Application for A.I. and VR/AR for healthcare: (Non-exhaustive)
Hospital or healthcare facility management and work flow visualisation, planning and implementing.
On demand patient healthcare triage system to prioritise medical or surgical emergencies.
On demand patient home diagnosis. (limited without clinical judgement and examination)
Artificially intelligent VR/AR assistants for Medical doctors to reduce errors.
Medical education and simulation training adaptive to the own users learning style to optimise learning and retaining knowledge.
Demonstrating the neural pathway of the artificially intelligent system in use: There will be a point where the AI being used for healthcare reaches a point that we as humans cannot comprehend the process. VR/AR represents a method by which these processes can be explained to us more effectively than code.
So just as this article started to see it, feel it, touch it, taste and even smell to do this is to make A.I. a human experience.
You can contact Dr. Raphael Olaiya on this topic or his work at Medigage Ltd via raphael.olaiya@nhs.net.
Augmented Reality (AR) is becoming increasingly popular in the fields of both healthcare and design. Medical Technology firm Stryker have combined the two for designing hospital Operating Rooms.
It’s fairly essential for a doctor working on a life-saving operation to have the most efficient possible layout for their OR. The problem is that hospital Operating Rooms are often packed full of very expensive, heavy equipment. This makes the process of finding the best layout difficult and time-consuming.
Stryker appear to have found a solution. Utilising Microsoft’s HoloLens AR headset, designers and doctors can test out different designs and layouts and even pick which configuration of equipment they would prefer, before a single piece of equipment is moved.
This type of AR/VR functionality is becoming increasingly popular, as VRFocus have reported previously. You can see a video of the Stryker HoloLens solution in action below.
VRFocus will continue to report on AR and VR use in design and healthcare.
The Internet of Things and virtual reality have changed the way that healthcare works in many areas. The entire smart healthcare industry is currently estimated to hit a value of $1.5 Trillion (USD) but 2012, of which the ‘machine-to-machine’ segment alone is predicted to be worth $89 Billion (USD) by 2018.
Machine-to-machine healthcare involves devices such as sensors relaying information to hardware or software applications with no intervention by humans hands. A sensor on someone’s body can measure a value such as heart rate and relay that information through a data network to the relevant software where it is translated into useful information that can be addressed by healthcare professionals.
For example, if a patient equipped with a sensor experiences a sudden drop in heart rate, a signal can be sent to a nurse or doctor’s mobile phone informing them that the patient needs to be immediately tended to. The nurse or doctor can even check of the patient from a distance using virtual reality (VR) technology that will keep medical professionals apprised of the patient’s condition without needing to leave their office.
“Smart healthcare represents a more universal approach to healthcare that incorporates cutting-edge technologies to deal with healthcare data management and utilisation,” said TechVision Research Analyst Cecilia Van Cauwenberghe. “Communications are integrated into a single, consolidated infrastructure, thereby empowering communities and individuals with the necessary tools and knowledge to make more informed decisions.”
Though he also noted there were some challenges that needed to be overcome and added; “Smart healthcare environments will need to pursue opportunities both up-market, such as vertical and horizontal specialised solutions, and down-market, such as unbundled solutions, to integrate the full concept of Smart Cities with present local/regional capabilities.
For further updates on VR and technology in healthcare keep up with VRFocus
