New data fusion method classifies skin lesions more accurately than previous algorithms


Dermatologists typically classify skin lesions based on several sources of data. Algorithms that merge information can support this classification. An international research team has now developed an algorithm that classifies skin lesions more accurately than previous algorithms using an improved data fusion process.

Many people around the world suffer from skin diseases. For the diagnosis, doctors often combine several sources of information. These include, for example, clinical images, microscopic images and metadata such as patient age and sex. Deep learning algorithms can support the classification of skin lesions by merging all information and evaluating it. Several such algorithms are already under development. However, to apply these learning algorithms in the clinic, they still need to be improved to achieve higher diagnostic accuracy.

New Data Fusion Method Improves Diagnostic Accuracy

A research team led by PD Dr. Tobias Lasser of the Munich Institute of Biomedical Engineering (MIBE) at the Technical University of Munich (TUM) has now developed a new learning algorithm – FusionM4Net – which displays an accuracy of higher mean diagnosis than previous algorithms. The code for FusionM4Net is freely available ( The new algorithm uses a so-called multi-step multi-modal data fusion process for multi-label classification of skin lesions.

  • Multimodal: The learning algorithm includes three different types of data: clinical images, microscopic images of the suspicious skin lesion, and patient metadata.
  • Multi-label: The researchers trained the multi-label skin classification algorithm, that is, it can differentiate between five different categories of skin lesions.
  • Multi-step: the new algorithm merges the available image data first, then the patient metadata. This two-step process allows the image data and metadata to be weighted in the algorithm’s decision-making process. This significantly distinguishes FusionM4Net from previous algorithms in this field, which merge all data at once.

To assess the diagnostic accuracy of an algorithm, it can be compared to the best existing classification for the dataset used, which is assigned the value 100%. The average diagnostic accuracy of FusionM4Net improved to 78.5% due to the multi-step fusion process, outperforming all other leading algorithms it was compared with.

Towards a clinical application

To aid reproducibility, a publicly available dataset was used to train the algorithm. However, in dermatology, data sets are not standardized everywhere. Depending on the clinic, different types of images and patient information may be available. Thus, for a real clinical deployment, the algorithm must be able to handle the type of data available in each specific clinic.

Together with the Department of Dermatology and Allergology at the University Hospital of LMU Munich, the research team is working intensively to make the algorithm operational for future clinical routine. To this end, the team is currently integrating many datasets that have been standardized for this clinic.

Future clinical use of routine algorithms with high diagnostic accuracy could help ensure that rare diseases are also detected by less experienced physicians and could mitigate decisions affected by stress or fatigue.

PD Dr. Tobias Lasser, Department of Computer Science and Munich School of BioEngineering, Technical University of Munich

Thus, learning algorithms could help improve the overall level of medical care.


Technical University of Munich (TUM)

Journal reference:

Tang, P. et al. (2022) FusionM4Net: A Multi-Step Multi-Modal Learning Algorithm for Multi-Label Classification of Skin Lesions. Analysis of medical images.


About Author

Comments are closed.