5 Wishes for Deep Learning in Medical Image Analysis

5 Wishes for Deep Learning in Medical Image Analysis


In my recent blog post on precision medicine, I mentioned how strong the need is for a better understanding of deep neural networks in order to be able to improve them, to explain their use to clinicians and eventually to authorise them from a regulatory point of view. Now, I would like to focus more on the field of deep learning and its application to medical image analysis, strongly influenced by my impressions from this year’s international conference on Medical Image Computing and Computer Assisted Interventions (MICCAI 2017), which was held from September 10-14 in Quebec, Canada.

Although deep learning has made an impact in a variety of other application areas, its influence on medical image computing – within the MICCAI community – was probably the highest so far. The impact of deep learning on medical image computing has arguably caused a significant increase of the number of papers on medical image analysis compared with those on computer-assisted interventions, as noted by Dr. Nassir Navab in his keynote. Deep learning, however, did not only create controversy regarding the distribution of papers at MICCAI, but also from a consideration that it could possibly render classical methods for mathematical modelling obsolete. These observations lead inevitably to the question: how will this field evolve?

As predictions are often very hard to make, I would rather formulate five ‘best’ wishes for the future…

  • be audacious with applications: we have to strive for applications for which machine learning is not straightforwardly applicable, particularly when obtaining data is difficult. We need more research on imaging and image analysis in complex environments such as the operating theatre, for example, the approach presented by Dr. Nicolas Padoy and his group on surgical workflow modelling solely based on endoscopic images.

  • be realistic about AI replacing clinicians: the lessons to be learnt from the race for the dream of 100% accuracy are certainly valuable, but this race is a dangerous one. Rather than raising such unrealistic goals, let us focus on understanding healthcare workflows better and provide intelligent solutions. These will provide clinicians with tools that allow them to make better decisions and focus on harder cases. This further reduces the risk for the next great disappointment regarding the capabilities of AI and eventually makes AI solutions more trustworthy. I thus advocate that we need to listen more carefully to clinicians such as Dr. Sunita Maheshwari.

  • be creative in employing and developing novel methods: 22 out of 255 accepted papers at MICCAI employ a U-net-type architecture, but only 5 papers rely on deep generative models. It is definitely great to see some network architectures establishing themselves as general tools for certain general applications. However, we need more research on generative methods, decision visualisation techniques, intelligent optimisation methods, semi- and weakly-supervised learning, and network architectures. As stated correctly by Dr. Nicolas Ayache, the MICCAI community is distinct from the computer vision community, by virtue of possessing its own imaging data, and therefore it should strive for its own specialised and theoretically founded approach. It was reassuring to see that one of the five young investigator awards went to Wufeng Xue for his creative end-to-end approach to cardiac left ventricle quantification.

  • be rigorous when evaluating machine learning methods: it was great to see a large variety of challenges organised in conjunction with MICCAI and many organisers did a terrific job regarding the careful preparation and evaluation of methods. The fact that industrial participants were also involved in these challenges shows that such efforts are more than an academic exercise. In order to move ahead, however, we all need to work even harder towards a standardisation of such challenges. This will not only ensure better comparability between results, but also greater generalisation capability in the evaluated methods.
  • be holistic: as suggested by Dr. Polina Golland after her keynote, we need to take the whole image acquisition process into account, and a series of promising steps in this direction was presented at MICCAI. However, we need to go even further and provide self-learning, workflow-aware, and robotic imaging solutions that are capable of imaging the right entity at the right time during an examination or a surgery, without any further assistance. For this reason, our laboratory is looking forward to attend the upcoming  International Conference on Intelligent Robots and Systems (IROS) taking place from September 24-28 in Vancouver, Canada.

To sum up, MICCAI 2017 was a great event uniting several generations of researchers, healthcare professionals and industrial experts. During many opportunities for personal exchange, I was extremely glad to hear that many academic and industrial participants share similar perspectives on the future of deep learning for medical image analysis, and medical imaging in general. After all, industry has an obligation to provide cheap, flexible and highly available cognitive solutions so that both researchers and clinicians are empowered to work on the future of digital healthcare.