What is CAMC?

The Camera Augmented Mobile C-arm (CAMC) is a pioneering research project that began in 1998 with the work of Nassir Navab, Michael Mitschke, and Ali Bani-Hashemi at Siemens Corporate Research, Princeton, NJ. The core concept is to augment the X-ray image of a mobile C-arm with optical video in a way that both modalities are perfectly registered — enabling real-time overlay of live video on X-ray images.

The key innovation is a double-mirror system that places a half-silvered mirror in front of the X-ray detector such that a co-aligned camera captures the optical image from the exact same viewpoint as the X-ray source. This achieves optical and X-ray image alignment without any additional registration step.


Project History

1998 — Nassir Navab, Michael Mitschke, and Ali Bani-Hashemi at Siemens Corporate Research filed foundational patents (US 5,963,613; US 6,048,312; US 6,049,582; US 6,144,712; US 6,167,293) covering the mirror system for X-ray/video superimposition and a virtual detector concept.

1999 — First CAMC paper presented at IWAR, describing the prototype with a double-mirror system and a CCD camera. First 3D reconstruction using a low-cost mobile C-arm published.

2000 — Video-augmented X-ray guidance for needle placement demonstrated — the first direct optical augmentation of an X-ray image for clinical guidance.

2003 — Patent for real-time co-registered X-ray and optical imaging (US 6,628,977).

2006 — Visual servoing for automated C-arm repositioning demonstrated.

2007 — Multi-view opto-X-ray system published.

2009 — Parallax-free X-ray stitching using the CAMC.

2011 — Closed-form inverse kinematics for C-arm positioning.

2012 — Supervised classification for intra-operative AR visualization (Pauly et al.).

2015 — RGBDX: mirror-based RGBD X-ray system (Habert et al.).

2016–2018 — CBCT+RGBD calibration; pose-aware C-arm for 2D/3D registration; CT-to-X-ray via RGBD; SLAM-based inside-out tracking.

2020 — Pivot calibration for sensor-attached C-arms (Lee, Seibold, Fürnstahl, Farshad, Navab).


Videos


Publications

Fotouhi, Mehrfard, Song, Johnson, Osgood, Unberath, Armand & Navab (2020)
, , , , , , & (). Development and Pre-Clinical Analysis of Spatiotemporal-Aware Augmented Reality in Orthopedic Interventions. IEEE transactions on medical imaging, 40(2). 765–778. Retrieved from https://pubmed.ncbi.nlm.nih.gov/33166252/
Javad Fotouhi (2019)
(). Co-localized augmented human and X-ray observers in collaborative surgical ecosystem. IJCARS. Retrieved from https://link.springer.com/article/10.1007/s11548-019-02035-8
Fotouhi, Unberath, Song, Gu, Johnson, Osgood, Armand & Navab (2019)
, , , , , , & (). Interactive flying frustums (IFFs): spatially aware surgical data visualization. International journal of computer assisted radiology and surgery, 14(6). 913–922. Retrieved from https://link.springer.com/article/10.1007/s11548-019-01943-z
Fotouhi, Alexander, Unberath, Taylor, Lee, Fuerst, Johnson, Osgood, Taylor, Khanuja & others (2018)
, , , , , , , , , & (). An augmented reality system for total hip arthroplasty. Retrieved from https://www.spiedigitallibrary.org/conference-proceedings-of-spie/10576/105760J/Technical-note--an-augmented-reality-system-for-total-hip/10.1117/12.2322399.short?SSO=1
Fotouhi, Fuerst, Unberath, Reichenstein, Lee, Johnson, Osgood, Armand & Navab (2018)
, , , , , , , & (). Automatic intraoperative stitching of nonoverlapping cone-beam CT acquisitions. Medical physics, 45(6). 2463–2475. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/29569728
Unberath, Fotouhi, Hajek, Maier, Osgood, Taylor, Armand & Navab (2018)
, , , , , , & (). Augmented reality-based feedback for technician-in-the-loop C-arm repositioning. Healthcare technology letters, 5(5). 143–147. Retrieved from https://arxiv.org/abs/1806.08814
Befrui, Fischer, Fuerst, Lee, Fotouhi, Weidert, Johnson, Euler, Osgood, Navab & others (2018)
, , , , , , , , , & (). 3D augmented reality visualization for navigated osteosynthesis of pelvic fractures. Der Unfallchirurg, 121(4). 264–270. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/29500506
Hajek, Unberath, Fotouhi, Bier, Lee, Osgood, Maier, Armand & Navab (2018)
, , , , , , , & (). Closing the calibration loop: an inside-out-tracking paradigm for augmented reality in orthopedic surgery. Retrieved from https://arxiv.org/abs/1803.08610
Andress, Johnson, Unberath, Winkler, Yu, Fotouhi, Weidert, Osgood & Navab (2018)
, , , , , , , & (). On-the-fly augmented reality for orthopedic surgery using a multimodal fiducial. Journal of Medical Imaging, 5(2). 021209. Retrieved from https://arxiv.org/abs/1801.01560
Fotouhi, Alexander, Unberath, Taylor, Lee, Fuerst, Johnson, Osgood, Taylor, Khanuja & others (2018)
, , , , , , , , , & (). Plan in 2-D, execute in 3-D: an augmented reality solution for cup placement in total hip arthroplasty. Journal of Medical Imaging, 5(2). 021205. Retrieved from https://arxiv.org/abs/1801.01557
Tucker, Fotouhi, Unberath, Lee, Fuerst, Johnson, Armand, Osgood & Navab (2018)
, , , , , , , & (). Towards clinical translation of augmented orthopedic surgery: from pre-op CT to intra-op x-ray via RGBD sensing. Retrieved from http://spie.org/Publications/Proceedings/Paper/10.1117/12.2293675
Habert, Meng, Fallavollita & Navab (2017)
, , & (). Multi-layer Visualization for Medical Mixed Reality. arXiv preprint arXiv:1709.08962. Retrieved from https://arxiv.org/abs/1709.08962
Habert, Eck, Fallavollita, Parent, Navab & Cheriet (2017)
, , , , & (). Application of an RGBD augmented C-arm for minimally invasive scoliosis surgery assistance. Healthcare technology letters, 4(5). 179–183. Retrieved from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5683214/
Lee, Tateno, Fuerst, Tombari, Fotouhi, Osgood, Johnson & Navab (2017)
, , , , , , & (). [POSTER] Mixed Reality Support for Orthopaedic Surgery. Retrieved from https://ieeexplore.ieee.org/document/8088484/
Lee, Fuerst, Tateno, Johnson, Fotouhi, Osgood, Tombari & Navab (2017)
, , , , , , & (). Multi-modal imaging, model-based tracking, and mixed reality visualisation for orthopaedic surgery. Healthcare technology letters, 4(5). 168–173. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/29184659
Fotouhi, Fuerst, Johnson, Lee, Taylor, Osgood, Navab & Armand (2017)
, , , , , , & (). Pose-aware C-arm for automatic re-initialization of interventional 2D/3D image registration. International journal of computer assisted radiology and surgery, 12(7). 1221–1230. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/28527025
Fotouhi, Fuerst, Wein & Navab (2017)
, , & (). Can real-time RGBD enhance intraoperative cone-beam CT?. International journal of computer assisted radiology and surgery, 12(7). 1211–1219. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/28343303
Fallavollita, Wang, Weidert & Navab (2016)
, , & (). Augmented reality in orthopaedic interventions and education. In Computational Radiology for Orthopaedic Interventions. (pp. 251–269). Springer. Retrieved from https://link.springer.com/chapter/10.1007/978-3-319-23482-3_13
Wang, Habert, Ma, Huang, Fallavollita & Navab (2016)
, , , , & (). Precise 3D/2D calibration between a RGB-D sensor and a C-arm fluoroscope. International journal of computer assisted radiology and surgery, 11(8). 1385–1395. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/26811080
Fallavollita, Brand, Wang, Euler, Thaller, Navab & Weidert (2016)
, , , , , & (). An augmented reality C-arm for intraoperative assessment of the mechanical axis: a preclinical study. International journal of computer assisted radiology and surgery, 11(11). 2111–2117. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/27287762
Wang, Habert, zu Berge, Fallavollita & Navab (2016)
, , , & (). Inverse visualization concept for RGB-D augmented C-arms. Computers in biology and medicine, 77. 135–147. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/27544070
Fotouhi, Fuerst, Lee, Keicher, Fischer, Weidert, Euler, Navab & Osgood (2016)
, , , , , , , & (). Interventional 3D augmented reality for orthopedic and trauma surgery. Retrieved from https://www.researchgate.net/publication/319721644_Interventional_3D_Augmented_Reality_for_Orthopedic_and_Trauma_Surgery
Fischer, Fuerst, Lee, Fotouhi, Habert, Weidert, Euler, Osgood & Navab (2016)
, , , , , , , & (). Preclinical usability study of multiple augmented reality concepts for K-wire placement. International journal of computer assisted radiology and surgery, 11(6). 1007–1014. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/26995603
Lee, Fuerst, Fotouhi, Fischer, Osgood & Navab (2016)
, , , , & (). Calibration of RGBD camera and cone-beam CT for 3D intra-operative mixed reality visualization. International journal of computer assisted radiology and surgery, 11(6). 967–975. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/27059022
Diotte, Fallavollita, Wang, Weidert, Euler, Thaller & Navab (2015)
, , , , , & (). Multi-modal intra-operative navigation during distal locking of intramedullary nails. IEEE transactions on medical imaging, 34(2). 487–495. Retrieved from https://ieeexplore.ieee.org/document/6915708
Pauly, Diotte, Fallavollita, Weidert, Euler & Navab (2015)
, , , , & (). Machine learning-based augmented reality for improved surgical scene understanding. Computerized Medical Imaging and Graphics, 41. 55–60. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/24998759
Habert, Gardiazabal, Fallavollita & Navab (2015)
, , & (). Rgbdx: first design and experimental validation of a mirror-based RGBD X-ray imaging system. Retrieved from https://ieeexplore.ieee.org/document/7328053
Habert, Meng, Kehl, Wang, Tombari, Fallavollita & Navab (2015)
, , , , , & (). [POSTER] Augmenting Mobile C-arm Fluoroscopes via Stereo-RGBD Sensors for Multimodal Visualization. Retrieved from https://ieeexplore.ieee.org/document/7328064
Leucht, Habert, Wucherer, Weidert, Navab & Fallavollita (2015)
, , , , & (). [POSTER] Augmented Reality for Radiation Awareness. Retrieved from https://ieeexplore.ieee.org/document/7328061
Wang, Habert, Ma, Huang, Fallavollita & Navab (2015)
, , , , & (). [POSTER] RGB-D/C-arm Calibration and Application in Medical Augmented Reality. Retrieved from https://ieeexplore.ieee.org/document/7328071/
Londei, Esposito, Diotte, Weidert, Euler, Thaller, Navab & Fallavollita (2015)
, , , , , , & (). Intra-operative augmented reality in distal locking. International journal of computer assisted radiology and surgery, 10(9). 1395–1403. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/25814098
Fuerst, Fotouhi & Navab (2015)
, & (). Vision-based intraoperative cone-beam ct stitching for non-overlapping volumes. Retrieved from https://link.springer.com/chapter/10.1007/978-3-319-24553-9_48
Pauly, Diotte, Habert, Weidert, Euler, Fallavollita & Navab (2014)
, , , , , & (). Relevance-based visualization to improve surgeon perception. Retrieved from https://link.springer.com/chapter/10.1007/978-3-319-07521-1_19
Chen, Naik, Wang, Navab & Fallavollita (2014)
, , , & (). Video-guided calibration of an augmented reality mobile C-arm. International journal of computer assisted radiology and surgery, 9(6). 987–996. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/24664269
Fallavollita, Winkler, Habert, Wucherer, Stefan, Mansour, Ghotbi & Navab (2014)
, , , , , , & (). Desired-view controlled positioning of angiographic C-arms. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/25485436
Londei, Esposito, Diotte, Weidert, Euler, Thaller, Navab & Fallavollita (2014)
, , , , , , & (). The ‘augmented’ circles: a video-guided solution for the down-the-beam positioning of IM nail holes. Retrieved from https://link.springer.com/chapter/10.1007/978-3-319-07521-1_11
Chen, Wang, Fallavollita & Navab (2013)
, , & (). Precise X-ray and video overlay for augmented reality fluoroscopy. International journal of computer assisted radiology and surgery, 8(1). 29–38. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/22592259
Erat, Pauly, Weidert, Thaller, Euler, Mutschler, Navab & Fallavollita (2013)
, , , , , , & (). How a surgeon becomes superman by visualization of intelligently fused multi-modalities. Retrieved from https://www.spiedigitallibrary.org/conference-proceedings-of-spie/8671/86710L/How-a-surgeon-becomes-superman-by-visualization-of-intelligently-fused/10.1117/12.2006766.short
Wang, Fallavollita, Zou, Chen, Weidert & Navab (2012)
, , , , & (). Closed-form inverse kinematics for interventional C-arm X-ray imaging with six degrees of freedom: modeling and application. IEEE transactions on medical imaging, 31(5). 1086–1099. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/22293978
Chen, Wang, Fallavollita & Navab (2012)
, , & (). Error analysis of the X-ray projection geometry of camera-augmented mobile C-arm. Retrieved from https://www.researchgate.net/publication/258713841_Error_Analysis_of_the_X-ray_Projection_Geometry_of_Camera-Augmented_Mobile_C-arm
Diotte, Fallavollita, Wang, Weidert, Thaller, Euler & Navab (2012)
, , , , , & (). Radiation-free drill guidance in interlocking of intramedullary nails. Retrieved from https://link.springer.com/chapter/10.1007/978-3-642-33415-3_3
Wang, Fallavollita, Brand, Erat, Weidert, Thaller, Euler & Navab (2012)
, , , , , , & (). Intra-op measurement of the mechanical axis deviation: an evaluation study on 19 human cadaver legs. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/23286099
Pauly, Katouzian, Eslami, Fallavollita & Navab (2012)
, , , & (). Supervised classification for customized intraoperative augmented reality visualization. Retrieved from https://ieeexplore.ieee.org/document/6402589
Wang, Fallavollita, Wang, Kreiser & Navab (2012)
, , , & (). Augmented reality during angiography: integration of a virtual mirror for improved 2d/3d visualization. Retrieved from https://ieeexplore.ieee.org/document/6402565
Wang, Zou, Weidert, Landes, Euler, Burschka & Navab (2011)
, , , , , & (). Closed-form inverse kinematics for intra-operative mobile C-arm positioning with six degrees of freedom. Retrieved from https://www.spiedigitallibrary.org/conference-proceedings-of-spie/7964/79641A/Closed-form-inverse-kinematics-for-intra-operative-mobile-C-arm/10.1117/12.877835.short
Wang, Landes, Weidert, Blum, Von Der Heide, Euler & Navab (2010)
, , , , , & (). First animal cadaver study for interlocking of intramedullary nails under camera augmented mobile C-arm. Retrieved from https://link.springer.com/chapter/10.1007/978-3-642-13711-2_6
Wang, Traub, Weidert, Heining, Euler & Navab (2010)
, , , , & (). Parallax-free intra-operative X-ray image stitching. Medical Image Analysis, 14(5). 674–686. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/20561811
Wang, Zou, Weidert, Landes, Euler, Burschka & Navab (2010)
, , , , , & (). Modeling kinematics of mobile C-arm and operating table as an integrated six degrees of freedom imaging system. Retrieved from https://link.springer.com/chapter/10.1007/978-3-642-15699-1_51
Wang, Traub, Weidert, Heining, Euler & Navab (2009)
, , , , & (). Parallax-free long bone X-ray image stitching. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/20425985
Wang, Traub, Heining, Benhimane, Euler, Graumann & Navab (2008)
, , , , , & (). Long bone X-ray image stitching using Camera Augmented Mobile C-arm. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/18982651
Traub, Ahmadi, Padoy, Wang, Heining, Euler, Jannin & Navab (2008)
, , , , , , & (). Workflow based assessment of the camera augmented mobile c-arm system. AMIARCS-Augmented environments for Medical Imaging including Augmented Reality in Computer-aided Surgery. Retrieved from http://campar.in.tum.de/twiki/pub/AMIARCS08/ProceedingsOnline/36-44.pdf
Traub, Heibel, Dressel, Heining, Graumann & Navab (2007)
, , , , & (). A multi-view opto-Xray imaging system. Retrieved from https://link.springer.com/chapter/10.1007/978-3-540-75759-7_3
Heining, Wiesner, Euler & Navab (2006)
, , & (). Pedicle screw placement under video-augmented fluoroscopic control. First clinical application in a cadaver study. International Journal of Computer Assisted Radiology and Surgery, 1(Supplement 1). 189–190. Retrieved from http://campar.in.tum.de/Chair/PublicationDetail?pub=heining2006cars
Heining, Wiesner, Euler, Mutschler & Navab (2006)
, , , & (). Locking of intramedullary nails under video-augmented flouroscopic control: first clinical application in a cadaver study. Proceedings of The 6th Computer Assisted Orthopaedic Surgery (CAOS 2006). Retrieved from http://campar.in.tum.de/Chair/PublicationDetail?pub=heining2006caosCAMC
Mitschke & Navab (2003)
& (). Recovering the X-ray projection geometry for three-dimensional tomographic reconstruction with additional sensors: Attached camera versus external navigation system. Medical Image Analysis, 7(1). 65–78. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/12467722
Mitschke, Bani-Hashemi & Navab (2000)
, & (). Interventions under video-augmented X-ray guidance: Application to needle placement. Retrieved from https://link.springer.com/chapter/10.1007/978-3-540-40899-4_89
Mitschke & Navab (2000)
& (). Optimal configuration for dynamic calibration of projection geometry of X-ray C-arm systems. Retrieved from https://ieeexplore.ieee.org/document/852379