AI-Based Smartphone Apps to Assess Moles Are Missing Skin Cancer Diagnoses

Algorithm-based smartphone apps using artificial intelligence to test moles are not reliably able to detect melanoma and other skin cancers, according to a study published in The BMJ.

Researchers conducted a systematic review of the Cochrane Central Register of Controlled Trials, MEDLINE, Embase, CINAHL, CPCI, Zetoc, Science Citation Index, and online trial registers through April 10, 2019, to identify studies that evaluated algorithm-based smartphone apps to assess images of skin lesions suspicious for skin cancer.

The final analysis included nine studies that evaluated six different identifiable smartphone apps. Six studies verified results by using histology or follow-up (n=725 lesions), and three studies verified results by using expert recommendations (n=407 lesions). Studies were small and of poor methodological quality, with selective recruitment, high rates of unevaluable images, and differential verification.

Lesion selection and image acquisition were performed by clinicians rather than smartphone users. Two of the included apps, SkinScan and SkinVision, have been given Conformit Europenne (CE) marks, allowing them to be marketed across Europe. They are also available in Australia and New Zealand but have not been cleared by the U.S. Food and Drug Administration.

Poor sensitivity to detect skin cancer

SkinScan was evaluated in a single study (n=15; five melanomas) with 0% sensitivity and 100% specificity for the detection of melanoma. SkinVision was evaluated in two studies (n=252; 61 malignant or premalignant lesions) with a sensitivity of 80% (95% CI, 63-92) and a specificity of 78% (95% CI, 67-87) for the detection of malignant or premalignant lesions. Accuracy of the SkinVision app verified against expert recommendations was poor in three studies.

“Test performance is likely to be poorer than reported here when used in clinically relevant populations and by the intended users of the apps,” the researchers cautioned. “The current regulatory process for awarding the CE marking for algorithm-based apps does not provide adequate protection to the public.”