AUGMENTED MEDICINE: LUNG DIAGNOSIS
(2019)
Education/Technology
HoloLens application, developed for my work at Leiden University's Centre for Innovation, In collaboration with the Leiden University Medical Centre.
The application enables a learning environment that realistically simulates the process of physical examination of patients presenting with shortness of breath. It hereby allows medical students and doctors to train themselves in diagnosing patients with lung diseases across a wide range of scenarios.
The HoloLens is used to place a hologram of an anatomic model of the human torso on top of the body of a real person. By using a real stethoscope that is being positionally tracked by the HoloLens, students can interact with the virtual torso. As they touch the hologram with the stethoscope, an audio sample of a lung sound that was recorded from a real patient is played back. The type of audio recording that is played back is determined by the position at which the stethoscope touches the hologram. By this, students learn to distinguish sounds of various lung diseases and learn how the sounds they hear relate to the position at which they place the stethoscope on the body. Since the application is networked, it can be used in a social learning setting where groups of students collaborate under the supervision of a teacher.
Next to developing the app, we created a gamified scenario for integrating the app into the medical curriculum. In this scenario, students meet an actor in the role of lung patient. The virtual torso is projected on the actor's body. By interviewing the actor and requesting additional information sources, such as radiology images and a patient file, students have to come up with the correct diagnosis and formulate a plan for treatment of the patient. By this, students can practice the full process of diagnosing lung diseases in a simulated environment that integrates many of the social, embodied and multimodal components of a real-world setting.
Credits
Concept:
Marlies Reinders
Arianne Pieterse
Beerend Hierck
Franka Luk
Jelger Kroese
Software development:
Jelger Kroese
App and UI design:
Jelger Kroese
3D model design:
Manon Zuurmond
Educational design:
Franka Luk
Arianne Pieterse
Beerend Hierck
Jelger Kroese
Project Management:
Franka Luk
Leontine van Melle