Dive Brief:

• Siemens Healthineers has launched an Apple Vision Pro app to enable people to view interactive holograms of the human body, the medtech company said Wednesday.
• The app displays renderings of human bone and tissue that the user can manipulate using their hands. A video of the app shows the user rotating a model of the human skull and zooming in to view the brain.
• Siemens said the ability to render clinical cases on the mixed-reality headset without additional hardware has potential uses in patient communication, medical education and surgical planning.

Dive Insight:

The Apple Vision Pro app is built on the Cinematic Rendering technology that Siemens created to enable the photorealistic visualization of medical images. Siemens discussed the technology in a 2016 paper and found it enabled faster comprehension of surgical anatomy than conventional CT imaging in a 2018 trial. In 2019, the company made the technology available on Microsoft’s HoloLens2 headset.

Like the HoloLens2 app, the Apple Vision Pro version of Cinematic Rendering allows users to display virtual images of the human body in their real-world environment. Two of the images shared by Siemens show scans of the human brain superimposed over the living room where the user is standing.

The recently introduced Apple Vision Pro is a newer product than HoloLens2, which launched in 2019, and Siemens has taken advantage of the additional processing power. 

“We have further optimized our existing algorithm of Cinematic Reality to allow computationally intense methods to run on Apple Vision Pro’s M2 processor,” Sebastian Krueger, lead developer Cinematic Reality at Siemens, said in a statement. “The rendering technique is used to simulate the way light interacts with objects in a virtual environment, producing highly realistic lighting and reflections in the resulting images.” 

Siemens’ statement explains that the Apple Vision Pro software, which is available to download from the app store, is a prototype that is under development and not available commercially. The company sees a range of use cases for the app, from explaining clinical cases to patients to educating medical students. In the future, the app could be used to help surgeons plan procedures and communicate with specialists.