Last updated July 15, 2020 at 12:11 pm
Using CT scans of a COVID-infected patient’s lungs, researchers have built a virtual 3D model, providing a unique insight into the virus.
An augmented reality view of a COVID-19 infected lung. Credit: La Trobe University
Why This Matters: New technologies will be critical for understanding COVID-19.
Believed to be a world first, researchers have created a virtual 3D model of COVID-infected lungs, which can be viewed through a HoloLens 2 mixed reality (MR) headset.
The visualisations mean that researchers can explore inside the patient’s lungs, allowing them to get closer to particular areas of interest.
The system has been developed over eight weeks by a team led by Professor Henry Duh at La Trobe University.
Medical imaging is critical in the delivery of healthcare
Duh says 3D modelling can show the transformation from healthy lungs to those infected by coronavirus – a sequence which is hard to pick up with 2D biomedical images – and provide a better understanding of how the virus invades the body.
Also: Picturing the future of medical imaging
“Medical imaging is indispensable to the delivery of healthcare and an essential component of clinical research. However, there is currently a lack of appropriate software to assimilate and derive maximum benefit from multimodal interactions with medical images in mixed reality environments.”
La Trobe University seeks to overcome that by building immersive medical tools using AR – the ability to superimpose 3D graphics over real-world views seen through a camera lens.
Leveraging Microsoft resources including HoloLens2, the University hopes the technology will one day be adopted in clinical diagnosis.
Credit: La Trobe University
Using AI to dig deeper
The team is now working toward using AI to automatically segment the lungs – or indeed any organs of interest – and render 3D scans from 2D medical images, to provide a deeper analysis of what it reveals.
Also: Using machine learning to predict medical outcomes
Tiffany Wright, education director for Microsoft Australia says: “It is an absolute privilege for Microsoft to be able to support the work of pioneering technologists like Professor Duh and his team at La Trobe.
“AI and the HoloLens2 are combined here to provide insights that would be hard to grasp in any other way – and the speed at which La Trobe was able to spin up this application is extremely impressive.”
According to Duh: “The ultimate goal of our project is to visualise the infected lungs and at the same time link to the outcomes of AI analysis.”
“Clinicians will be able to see the analytics and interact with the model to gain insights from the system.”
