Seeing clearly: How augmented reality can help medical students understand complex anatomy

Jelger Kroese & LUMC
Author: Jelger Kroese & LUMC

Medical students at Leiden University Medical Centre (LUMC) can now gain advanced insight into the postoperative anatomy of kidney and pancreatic transplant patients using a new augmented reality (AR) app, AugMedicine – Transplant cases. The app, developed by LUMC in collaboration with Leiden University’s Centre for Innovation (CFI), focuses on enhancing students’ understanding of the spatial relations between 2D CT scans and 3D anatomical models. 

The limits of traditional teaching methods

Kidney transplantation differs from other types of surgery. Instead of removing diseased tissue or repairing damage, a new organ is implanted into the body. There are different ways to perform these complex operations,  for instance a combined kidney and pancreas transplantation is sometimes performed (e.g. in patients with type I diabetes).

The complexity of these surgical procedures and complications means it is challenging to teach students about them outside of an operation theatre. Traditional visualisations such as CT scans offer 2D images which are difficult for medical students or residents to interpret. In particular, it is difficult for students to understand the spatial relations between the cross-sections that CT scans offer and real life 3D anatomy: 

As a surgeon, I can see and feel the anatomy while performing an operation. Attempts to communicate this in the context of education often fail due to very enthusiastic misconceptions on the part of residents, trainee surgeons and other colleagues such as: “Can’t the bowel package push the kidney out of place?” or ”Where is that kidney artery connected to?”
Dorottya de Vries (Transplant surgeon at LUMC Transplant Center)

A problem with traditional medical teaching is that is has relied on using 2D images, such as CT scans, to describe the three-dimensional body. Teachers find it challenging to piece these images together so that students can understand the 3D structure inherent in anatomy.

Augmented reality as digital solution for learning about transplant cases

To overcome this obstacle, LUMC and CFI developed AugMedicine: Transplant cases, an AR app built for the Microsoft Hololens. Augmented reality is a digital technology that enriches the real world with 3D holograms, and holds transformative potential for learning environments. 

The app uses a head-mounted display (Microsoft Hololens) to visualise can project  3D holograms. What this means for the classroom is that you can still see and interact with fellow learners and the teacher, while viewing the 3D hologram. With this interactive augmented reality application it is possible to project anatomical 3D models of patients post-transplantation as holograms in the real-life environment.

Students viewing 2D images in spacial relation to 3D model

How does it work?

In a CT scan, the body is shown slice by slice. But it is not easy to translate all these separate images into the actual, three-dimensional situation.
Dorottya de Vries (Transplant surgeon at LUMC Transplant Center)

The app combines 2D CT scans with a 3D anatomical model of the transplanted organs. To achieve this, the app visualises CT scans on a virtual plane that intersects the 3D model at the same location as where the CT image was taken. This combination reveals the spatial relation between the 2D scans and the 3D human body, allowing students to see clearly what the images show. 

Users can manipulate the position at which the virtual plane intersects the 3D model by using hand gestures and scroll through the full set of CT images as if they were on a computer, but with the ability to observe a direct, 3D connection with the anatomy. This makes it easier for learners to understand the spatial relationship between 2D CT scans and 3D structures in the human body and provides students more insight into the complex anatomy of transplant cases and complications.

The full interface of the AugMedicine app

Maximising learning potential through augmented reality

During the development of the app, we decided to include an additional outcome to stimulate social learning through collaboration. For instance, multiple users can inspect, walk around, pinpoint and discuss a set of CT scans together.

This feature makes the application an ideal tool for education, enabling an active learning setting where groups of students can collaboratively learn about specific transplant cases under the supervision of a teacher.

A 2x gold award-winning app

AugMedicine – Transplant Cases was awarded not only one, but two HaloAwards by VR Days Europe (November 2019). The app was awarded GOLD in the categories for:

  • Best Applied (Healthcare/Enterprise)
  • Best University Project

The HaloAwards are presented by VR Days Europe to celebrate the most innovative and creative projects in VR and AR.

At the Centre for Innovation, we aim to lead in designing immersive and potent learning experiences by making creative use of the vast potential of mixed reality in education. We are delighted to have been recognised for our excellence in enhancing education with AR/VR technologies.

Get in touch with the author

Jelger Kroese
Educational Experience Designer & Mixed Reality Developer