An Augmented Reality Platform for Medical Planning: Cases of Study in Mitral Valve and Cerebral Aneurysm Minimally Invasive Surgeries

Researching in new technologies for diagnosis, planning and medical treatment have allowed the development of computer tools that provide new ways of representing data obtained from routine imaging studies. Laboratory studies, such as computed tomography (CT) and magnetic resonance imaging (MRI) have become the standard for med- ical diagnosis, so today they are used to give a first diagnosis of the treatment to be followed depending on the patient’s condition. Also, surgeons use these images to obtain information about the best way to reach certain structures within the human body before the surgery. In this sense, augmented reality (AR) technologies provide a new form of data representation by combining the common analysis using images and the ability to superimpose virtual representations of any structure of the human body on a view of the real world on real-time. In this paper, we describe the development of a generic computer platform based on aug- mented reality technology for surgical planning. To validate the system, we focused in two cases of study: navigation through aorta artery for mitral valve repair surgery and navigation through vascular structures in the brain for the treatment of cerebral aneurysms.


The BRAVO project aims to create an advanced game based therapeutic environment able to help young people with ADHD (Attention Deficit Hyperactivity Disorder) in the acquisition of cognitive and behavioral skills on which ADHD impacts.

The BRAVO gaming environment aims to:

  • Improve the relationship of young patients with therapies;
  • Implement therapy processes customized;
  • Support therapists and families in the management of the rehabilitation program.

The Bravo virtual gaming system involves children in an environment where therapy is turned into a playground.

Users of BRAVO system will therefore be: patients, who will be able to access the games developed to benefit from personalized therapy; the therapists, who will be able to access the system to assign the therapies and control the evolution of the therapy administered.


The Project RESPIRO aims to build a network of expertise in the design and supply of technology products and services for the elderly and people with different forms of disability. The project is focused on the development of technologies and systems in the field of Ambient Assisted Living in order to support of prevention processes and self-management of health, diagnosis, treatment and follow-up with the aim to permit a comprehensive and personalized management of chronic disease in the home environment.

Augmented Reality in RFA of the Hepatic Tumours

Hepatic cancer is one of the most common solid cancers in the world. As surgery of hepatic cancer is seldom applicable, different solutions have been found to cure this disease. One of these is Liver Radiofrequency Ablation that consists in a needle insertion inside the liver parenchyma in order to reach the tumour and in an injection of a radiofrequency current to cause tumour cell necrosis for hyperthermia. The correct needle placement task is really difficult. The Augmented Reality can help the surgeon to place the needle as best as possible; the application can also help the surgeon during preoperative planning because it offers various visualization modality of 3D models of the patient’s organs obtained from the medical images.

Augmented Reality in Pediatric Laparoscopy

One trend in surgery is the transition from open procedures to minimally invasive interventions. The availability of a system for the pre-operative planning of a laparoscopic procedure can be of great help to the surgeon. This support is even more important in infantile laparoscopic surgery where is needed a good understanding of the exact conditions in which the patient’s organs are and the precise location of the operational site. The developed application allows the surgeon to gather information about the patient and her/his pathology, visualizing and interacting with the 3D models of the organs built from the patient’s medical images, measuring the dimensions of the organs and deciding the best insertion points of the trocars in the patient’s body. This choice can be visualized on the real patient using the Augmented Reality technology.

Human Anatomy Visualization and Navigation System for Image-Guided Surgery

Minimally invasive surgery offers advantages that make it the best choice for many diseases. Modern technologies give a great support to this kind of surgical procedures through medical image processing and visualization, 3D organ’s reconstruction and intra-operative surgical guidance.
An advanced visualization and navigation system has been developed which allows to visualize both the traditional patient information, as the CT image set, and a 3D model of the patient’s anatomy built from this.

Two different visualization modalities are available in real time and dynamically. According to the surgeon needs, it is possible to obtain the automatic reslicing of the orthogonal planes in order to have an accurate visualization of the 3D model and slices exactly next to the actual position of the surgical instrument tip. In addition, it is possible to activate the clipping modality that allows cutting the 3D model in correspondence of a chosen visualization plane. The system can be used as support for the diagnosis, for the surgical preoperative planning and also for an image-guided surgery.

Distance Measurement for an Accurate Laparoscopy Surgery

The idea of Minimally Invasive Surgery is to reduce the trauma for the patient by minimizing incisions and tissue retraction. Despite the improvement in outcomes, these techniques have their limitations and come at a cost to the surgeons. In particular, the lack of depth in perception and the difficulty in estimating the distance of the specific structures in laparoscopic surgery can impose limits on delicate dissection or suturing.

Augmented Reality technology has the potential to bring back the direct visualization advantage of open surgery to minimally invasive surgery and can increase the physician’s view with information gathered from patient medical images. In this work an Augmented Reality system has been developed where the position of the surgical instrument is detected by means of an optical tracker and this information is used to measure the distance between this instrument and the anatomical structures concerned. In addition, the system is able to warn the surgeon of possible contact with other organs located close to the organ of interest, using both visual and audio aids.