Autopsy and Case Reports
Autopsy and Case Reports
Letter to the Editor

Virtual autopsy: Machine Learning and AI provide new opportunities for investigating minimal tumor burden and therapy resistance by cancer patients

Shane O’Sullivan; Andreas Holzinger; Dominic Wichmann; Paulo Hilario Nascimento Saldiva; Mohammed Imran Sajid; Kurt Zatloukal

Downloads: 3
Views: 862


One advantage yet to focus on in scientific literature is the beneficial use of virtual autopsy (virtopsy) for investigating minimal tumor burden. Our hypothesis is that virtopsy assists in the understanding of therapy resistance of cancer patients or cause of death in patients with minimal tumor burden.1 The well-established textbook scenario describes a patient dying from cancer by the tumor mass compressing surrounding tissue (e.g. brain tumors), or destroying surrounding tissue resulting in organ failure (e.g. multi-metastatic diseases), or destroying blood vessels causing lethal bleeding.

Furthermore, patients may die from tumor-induced cachexia, which is a consequence of the tumor’s interference with the body’s energy homeostasis. However there are some instances when it is difficult to explain a tumor-related death, particularly when a patient dies of head or neck cancer. For example, when major tumor mass is not detectable, there are no signs of cachexia, or evidence of an immediate consequence of chemotherapy or radiotherapy. When patients might die exhibiting a small metastasis in the lung or a small local tumor, it is concluded that they died of the cancer because there is no other apparent cause of death (e.g. cardiac disease). This is concluded despite none of the classical cancer-related causes of death being established. In view of this therefore is an unexplained mechanism of how a tumor can kill a patient.

The majority of current cancer therapies are aimed at killing tumor cells. This is done either directly (by chemical agents or radiation), or indirectly (by depriving the tumor from nutrients, or activating and redirecting the immune response against the cancer). This variety of therapeutic approaches is reflected in modern therapies such as PDL-1 blockers, VEGF-inhibitors, tumor-vaccines or Proteasome-inhibitors. However the observation that a fair number of patients with minimal detectable tumor mass die of cancer, given that treatment options described were considered, highlights gaps in knowledge that require filling as well as development of new potential therapeutic approaches.

As a first step we propose epidemiological studies be undertaken - these are required in order to obtain quantifiable data on how many and which type of cancer patients die of cancer with minimal tumor mass. As autopsies are rarely performed on patients whose cancer has been well-characterized during the course of the disease, systematic data on this poorly-characterized cause of cancer-related death do not exist. Furthermore, the assessment of total tumor mass in the body is difficult in disseminated diseases by traditional autopsy. In cases where minimal tumor burden has caused patients to die, we need to gain more knowledge (evidence-based practice). It is the combination of autopsy, pathology and virtopsy that truly defines or examines the entire body. When dealing with a localized disease, traditional autopsy is appropriate in order to cut out, weigh and measure parts that are affected. This however, cannot be performed when dealing with a disseminated disease with many small lesions in various organ systems. Virtopsy can be a very effective intervention to quantify tumor mass. Imaging would provide very important baseline data to compare different patients’ tumor mass and to exclude other non-cancer-related causes of death.1 Machine learning methods will also be of great assistance too, particularly for potential in-silico modelling.2,3 This would save time and effort; enabling what is not currently feasible in a wet-lab. Moreover, the principles of machine learning image analysis would enhance virtopsy.1

The enormous practical success of Machine Learning and Artificial Intelligence (AI) has led to more evidence-based decision-making in the medical domain.4 A very recent example with deep learning models demonstrated impressive results:5 the authors utilized a GoogleNet Inception v3 CNN architecture for the classification of skin lesions, using only pixels and disease labels as inputs. They pre-trained their network with 1.28 million images (1,000 object categories), and trained it on 129,450 clinical images, consisting of 2,032 different diseases. The performance was tested against 21 board-certified dermatologists on biopsy-proven clinical images with two critical binary classification use cases: keratinocyte carcinomas versus benign seborrheic keratoses; and malignant melanomas versus benign nevi.

The results demonstrated that such deep learning models can achieve a performance even beyond human experts. However, besides being resource-intensive and data-hungry, black-box machine learning and AI approaches have one enormous disadvantage in the medical domain – they are lacking transparency. Even if we understand the mathematical theory of machine learning model it is complicated, yet impossible to get insight into the internal working of such a model. This leads to a major question – can we trust such results?

Consequently, there is growing demand in interactive machine learning advances,6 which are not only well performing, but transparent, interpretable and trustworthy and include a human-in-the-loop.7 For the medical domain it is necessary to re-enact the machine decision-making process, to reproduce and to comprehend the learning and knowledge extraction process.8 For medical decision support it is of ultimate importance to understand the causality of learned representations.9,10 If human intelligence will be complemented by machine learning and at least in some cases even take precedence, humans must be able to understand and principally be able to interactively influence the machine decision process. This needs to make sense in order to close the gap between human thinking and machine “thinking”.11

Increasing legal and privacy characteristics are a massive motivation for this practice. The new European General Data Protection Regulation (GDPR and ISO/IEC 27001) entering into force on May, 25, 2018, will make black-box approaches difficult to use in any business, because they recognize that they are not able to explain why a decision has been made; this will make glass-box approaches essential12 and stimulate international research in interactive machine learning with the goal of making decisions interpretable, comprehensible and reproducible. In our example, this is not only useful for machine learning research, and for clinical decision making, but at the same time a big asset for the training of medical students.

We therefore call for a collaborative effort to generate quantifiable data on tumor burden of patients who died of cancer without evidence of classical causes of cancer-related deaths. These data could lay the foundation for discovering novel mechanisms of how a cancer may interfere with body function. There is increasing evidence from metabolomic studies that tumors may markedly interfere with metabolism as demonstrated by certain metabolic signatures that correlate with disease prognosis.13 Interestingly this interference was not related to signs of cachexia, and therefore is most likely different to presently-described mechanisms of cancer-induced cachexia.14 A broadening of the scientific basis on the relevance of metabolic interference of cancer could widen the field of cancer therapies, which target the cancer interference with the patient’s metabolism.15

In summary, this article presents and supports an add-on example of how virtopsy can propel medicine into the future, its impact, implications and application in investigating minimal tumor burden and therapy resistance by cancer patients. Great advances will be made by taking advantage of current progress in Machine Learning and Artificial Intelligence, however new approaches are needed that make use of a human-in-the-loop and above all in making transparent why and how a decision has been made.


O’Sullivan S, Holzinger A, Zatloukal K, Saldiva P, Sajid MI, Wichmann D. Machine learning enhanced virtual autopsy. Autops Case Rep. 2017;7(4):3-7. [].

Jeanquartier F, Jean-Quartier C, Cemernek D, Holzinger A. In silico modeling for tumor growth visualization. BMC Syst Biol. 2016;10(1):1-15. [PMID:27503052] [].

Jeanquartier F, Jean-Quartier C, Kotlyar M, et al. 2016. Machine learning for in silico modeling of tumor growth. In: Holzinger A, editor. Machine learning for health informatics, springer Lecture Notes in Artificial Intelligence LNAI 9605. Cham: Springer International Publishing. pp. 415-434.

Jordan MI, Mitchell TM. Machine learning: trends, perspectives, and prospects. Science. 2015;349(6245):255-60. [PMID:26185243] [].

Esteva A, Kuprel B, Novoa RA, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017;542(7639):115-8. [PMID:28117445] [].

Holzinger A. Interactive machine learning for health informatics: when do we need the human-in-the-loop? Brain Informatics. 2016;3(2):119-31. [PMID:27747607] [].

Girardi D, Küng J, Kleiser R, et al. Interactive knowledge discovery with the doctor-in-the-loop: a practical example of cerebral aneurysms research. Brain Informatics. 2016;3(3):133-43. [PMID:27747590] [].

Holzinger A, Plass M, Holzinger K, Crisan GC, Pintea C-M, Palade V. Towards interactive machine learning (iML): Applying ant colony algorithms to solve the traveling salesman problem with the human-in-the-loop approach. springer Lecture Notes in Computer Science LNCS 9817. Heidelberg: Springer; 2016. p. 81-95.

Pearl J. 2009. Causality: models, reasoning, and inference. 2nd ed. Cambridge: Cambridge University Press.

Gershman SJ, Horvitz EJ, Tenenbaum JB. Computational rationality: a converging paradigm for intelligence in brains, minds, and machines. Science. 2015;349(6245):273-8. [PMID:26185246] [].

Holzinger A. Trends in interactive knowledge discovery for personalized medicine: cognitive science meets machine learning. IEEE Intelligent Informatics Bulletin. 2014;15(1):6-14.

Holzinger A, Plass M, Holzinger K, Crisan GC, Pintea C-M, Palade V. A glass-box interactive machine learning approach for solving NP-hard problems with the human-in-the-loop. 2017. p. 1-26. [cited 2017 July 2017]. Available from:

Bertini I, Cacciatore S, Jensen BV, et al. Metabolomic NMR fingerprinting to identify and predict survival of patients with metastatic colorectal cancer. Cancer Res. 2012;72(1):356-64. [PMID:22080567] [].

Loumaye A, Thissen JP. Biomarkers of cancer cachexia. Clin Biochem. 2017;50(18):1281-88. [].

Argilés JM, López-Soriano FJ, Stemmler B, Busquets S. Novel targeted therapies for cancer cachexia. Biochem J. 2017;474(16):2663-78. [PMID:28751550] [].

Publication date:

5a86d5790e88259a35bd8e8d autopsy Articles
Links & Downloads

Autops Case Rep

Share this page
Page Sections