What is ARTEKMED?
ARTEKMED is a joint Bavarian research project focused on 3D augmented reality teleconsultation for medical emergencies and clinical consultations.
The system enables paramedics or junior doctors at the scene of an emergency to receive real-time guidance from a remote expert. The remote expert sees a 3D reconstruction of the local environment and can place annotations, drawings, and virtual objects that appear as holographic overlays in the field-of-view of the on-site person via a Microsoft HoloLens 2.
Key Features
- Telepresence system for accident sites and clinic-to-clinic consultation
- HoloLens 2 head-mounted display for on-site personnel
- 3D scene reconstruction from HMD cameras or ambient cameras in real-time
- Asymmetric teleconsultation: remote expert on a desktop sees the 3D reconstructed scene; local user sees expert’s annotations as holograms
- Studies on social acceptance, trust, and usability of digital doubles in medical settings
Technology
The system addresses several technical challenges:
- Real-time 3D reconstruction from a moving head-mounted display
- Accurate participant positioning and registration
- Efficient, low-latency, and secure multi-party video communication
- Handling unstable network conditions (e.g., in ambulances or remote sites)
Project Links
- Project website: www.artekmed.de
Videos
Publications
- Liebchen, Schrader-Reichling, Pankratz, Lazarovici, Kim, Tempfli, Eck & Prückner (2025)
- Liebchen, N., Schrader-Reichling, J., Pankratz, F., Lazarovici, M., Kim, S., Tempfli, J., Eck, U. & Prückner, S. (2025). Comparison of the ArtekMed mixed reality teleconsultation system with a standard video call system in critical care: user acceptance and feasibility analysis. Intensive Care Medicine Experimental, 13(1). 49.
- Yu, Roth, Strak, Pankratz, Reichling, Kraetsch, Weidert, Lazarovici, Navab & Eck (2023)
- Yu, K., Roth, D., Strak, R., Pankratz, F., Reichling, J., Kraetsch, C., Weidert, S., Lazarovici, M., Navab, N. & Eck, U. (2023). Mixed Reality 3D Teleconsultation for Emergency Decompressive Craniotomy: An Evaluation with Medical Residents.
- Song, Eck & Navab (2023)
- Song, T., Eck, U. & Navab, N. (2023). Leveraging Motion Tracking for Intuitive Interactions in a Tablet-Based 3D Scene Annotation System.
- Yu, Eck, Pankratz, Lazarovici, Wilhelm & Navab (2022)
- Yu, K., Eck, U., Pankratz, F., Lazarovici, M., Wilhelm, D. & Navab, N. (2022). Duplicated Reality for Co-located Augmented Reality Collaboration. IEEE Transactions on Visualization and Computer Graphics.
- Yu, Eck & Navab (2022)
- Yu, K., Eck, U. & Navab, N. (2022). Spatial Exploration with a WiM for Capturing 3D Dioramic Snapshots. 3DUI Contest Submission at IEEE Virtual Reality..
- Yu, Zacharis, Eck & Navab (2022)
- Yu, K., Zacharis, K., Eck, U. & Navab, N. (2022). Projective Bisector Mirror (PBM): Concept and Rationale. IEEE Transactions on Visualization and Computer Graphics.
- Strak, Yu, Pankratz, Lazarovici, Sandmeyer, Reichling, Weidert, Kraetsch, Roegele, Navab & others (2021)
- Strak, R., Yu, K., Pankratz, F., Lazarovici, M., Sandmeyer, B., Reichling, J., Weidert, S., Kraetsch, C., Roegele, B., Navab, N. & others (2021). Comparison Between Video-mediated and Asymmetric 3D Teleconsultation During a Preclinical Scenario.
- Yu, Winkler, Pankratz, Lazarovici, Wilhelm, Eck, Roth & Navab (2021)
- Yu, K., Winkler, A., Pankratz, F., Lazarovici, M., Wilhelm, D., Eck, U., Roth, D. & Navab, N. (2021). Magnoramas: Magnifying Dioramas for Precise Annotations in Asymmetric 3D Teleconsultation. Demo at IEEE Virtual Reality, Lisbon, Portugal..
- Roth, Yu, Pankratz, Gorbachev, Keller, Lazarovici, Wilhelm, Navab & Eck (2021)
- Roth, D., Yu, K., Pankratz, F., Gorbachev, G., Keller, A., Lazarovici, M., Wilhelm, D., Navab, N. & Eck, U. (2021). Real-time Mixed Reality Teleconsultation for Intensive Care Units in Pandemic Situations. 3DUI Contest Submission at IEEE Virtual Reality, Lisbon, Portugal..
- Yu, Winkler, Pankratz, Lazarovici, Wilhelm, Eck, Roth & Navab (2021)
- Yu, K., Winkler, A., Pankratz, F., Lazarovici, M., Wilhelm, D., Eck, U., Roth, D. & Navab, N. (2021). Magnoramas: Magnifying Dioramas for Precise Annotations in Asymmetric 3D Teleconsultation.
- Yu, Gorbachev, Eck, Pankratz, Navab & Roth (2021)
- Yu, K., Gorbachev, G., Eck, U., Pankratz, F., Navab, N. & Roth, D. (2021). Avatars for Teleconsultation: Effects of Avatar Embodiment Techniques on User Perception in 3D Asymmetric Telepresence. IEEE Transactions on Visualization and Computer Graphics, 27(11). 4129–4139.
- Roth, Yu, Pankratz, Gorbachev, Keller, Lazarovici, Wilhelm, Weidert, Navab & Eck (2021)
- Roth, D., Yu, K., Pankratz, F., Gorbachev, G., Keller, A., Lazarovici, M., Wilhelm, D., Weidert, S., Navab, N. & Eck, U. (2021). Real-time Mixed Reality Teleconsultation for Intensive Care Units in Pandemic Situations.