Autoderm

Summary

Your skin is your largest organ. It is also the most important when you make human connections. About 70% of the human population have been embarrassed or discriminated because of a skin disease. For example, teenagers will not upload a selfie on social media if they have a skin disease in the face. Skin conditions are widespread across all demographics, making them one of the leading reasons for medical consultations worldwide. In most cases, the first point of medical contact is in a primary care setting. Latest data from the British Journal of General Practice reveals that skin diseases are the most common reason for people to visit their general practitioners (GP) in England and Wales, and about 25% of all primary care physician consultations are made because of skin issues [1,2]. However, the traditional methods of diagnosis with visual speculations are dependent on experience, with a highly variable accuracy ranging between 48%–77% [3,4]. on the most common skin diseases.

With this background, it is only natural for people to venture beyond the norm looking for novel solutions to overcome these challenges. Artificial intelligence (AI) has been researched and tested for this purpose by many. Recent advancements in AI technology, particularly in image recognition and machine learning (ML) algorithms, bring promising solutions with its ability to enhance diagnostic accuracy, efficacy and expand healthcare access. Dr Escalé-Besa A. et al. looked at how GPs could use AI in a primary cafe setting and published and article called: "Potential of artificial intelligence in improving skin lesion diagnosis in primary care" [5].

Do we really need AI in healthcare?

Diagnosing skin conditions in a primary care setting is challenging for several reasons. The limited exposure for dermatology in medical school curricula, limited exposure to different skin conditions, and the limited resources make the correct diagnosis a constant struggle for a GP. This results in low accuracy for detecting tumours 22.4% and increasing number of referrals to dermatologists accounting for increased healthcare cost [6]. The use of AI in a primary care setting has been effective in overcoming these barriers, as it can analyse skin lesion images instantly providing decision support when the patient is present in the clinic. Unlike tele-dermatology, the patient will have to wait 24-48 hours for an answer. AI driven tools have been researched extensively in skin cancer detection in a clinical research setting, with greatly improved sensitivity (91.7%) and specificity (81.4%) [7].

Current study overview

In the year 2019 a study was set up to explore the use of AI in a urban primary care setting in an Catalonia, Spain, where there are no access to dermatologists on a daily basis, but either via online consultations or in person once a week. Escalé-Besa A. et al. researched an AI model by Autoderm to evaluate the use of diagnosing skin diseases in a live primary care setting. Other objectives of the study were to check the accuracy of the model, identify undetected skin conditions, assess patient participation rates, and gauge healthcare professionals' satisfaction with the AI. The AI model was capable of screening 44 different skin conditions. Study participants consisted of 100 consecutive patients who visited their GP with a skin concern. Each patient underwent a threefold assessment, first by AI, then by the GP and then by a dermatologist via tele-dermatology and if the diagnosis could not be verified by tele-dermatology, a skin biopsy was performed. The findings were compared to determine the accuracy of each method.

Method of assessment

It was a prospective multi-center study, where 6 primary care centers and 11 general practitioners in total were involved. All the study participants were first assessed by the GP who came to a diagnosis on their own, followed by running an anonymized picture of the lesion through the AI model to generate differential diagnosis, and finally a dermatologist’s diagnosis using tele- dermatology. Additionally, the GPs were questioned about the usefulness of the AI model in arriving at a diagnosis, generating differential diagnosis and the impact it has on the need of a tele-dermatology consult with a dermatologist. The study then compared the top diagnoses from each source to evaluate the model’s accuracy, sensitivity, and specificity. Top 3 GP diagnoses, Top 5 machine learning model differential diagnoses and dermatologist’s Top 3 assessments were compared to reach the conclusion.

Findings

The 100 cases assessed to determine the AI validity consisted of 36 different skin conditions, and 12 out of the 36 were not among the 44 conditions the AI model was trained to diagnose. Benign tumors like nevi, seborrheic keratosis and actinic keratosis were the most common diseases presented, followed by inflammatory conditions and infectious conditions respectively. The comparison of accuracy as observed in the study is summarized below.

Diagnostic Accuracy Comparison

Accuracy	AI	GP	Tele-dermatology
Top 1	39%	64%	72%
Top 3	61%	72%	90%
Top 5	72%	—	—

Table 1. Accuracy of the AI model when screened for skin diseases that the AI model had not been trained for. Total of 100 skin diseases.

Accuracy	AI	GP	Tele-dermatology
Top 1	48%	64%	72%
Top 3	75%	72%	90%
Top 5	89%	—	—

Table 2. Accuracy of the AI model when screened for skin diseases that the AI model was trained for. Total of 82 skin diseases.

The diagnostic accuracy of the AI model was on par with the GPs, but lower in regards to dermatologists considering only the conditions the AI was trained to identify. In this attempt, the accuracy increased up to 48% in top 1, 75% in top 3 and 89% in top 5. In this version, the top 5 AI accuracy was similar to top 3 dermatologist's accuracy as well.

The AI model had higher sensitivity than clinicians only in benign tumor detection. In diagnosing inflammatory conditions, the AI performed better than the GP's diagnosis in the top 3 category. Infectious diseases were also diagnosed at a higher accuracy rate than that of GP's in both top 3 and top 5 categories. Diagnosing malignant tumors was achieved with a higher sensitivity with GPs (92% in top 3) over AI model (67% in top 3).

When considering the diagnostic sensitivity of the malignant tumors, melanoma detection was done with 100% sensitivity in both GPs and AI model. Squamous cell carcinoma was detected with 100% sensitivity in top 3 GP diagnosis, and in top 5 AI diagnosis. The AI model did not achieve 100% sensitive diagnosis in Basal cell carcinoma, though the GP did, in this case it was a pigmented Basal cell carcinoma. However, even though it did not diagnose Basel Cell cancer, it diagnosed it as a malignant melanoma, so the treatment would have been similar - a biopsy. While 92% GPs considered the AI model to be a useful tool, 60% claimed it aided in final diagnosis by way of generating differential diagnosis. In 34% of cases a visit to a dermatologist could have been avoided.

Discussion

Overall, this was one of a kind clinical study on actual patients that had a skin concern and not databases of dermatology images. The GP used the app, compared it to analog diagnosing and history taking, which was submitted to a tele-dermatologist. The AI did not have access to any patient history only an image of the skin cancer. The AI model is “old” considering AI development, it was from 2019 and could only identify 44 different skin conditions. An AI model that was developed after the feedback from the study could identify 73 conditions, with similar accuracy as this study. The findings highlight the potential of AI as a decision support tool in diagnosing skin diseases, and the results can be a good example to future AI integration in primary care to improve diagnosis and treatment efficiency. Even though the AI model was on par with the GPs and less accurate to tele-dermatology diagnosis, the model was very important in generating differential diagnosis. This can be invaluable in a primary care setting to reduce time of consultation, reduce referral rates and assist the correct identification of the condition. Clinical input together with the AI expertise can ensure more favorable healthcare outcomes, reducing cost and increasing efficacy. Training the models with a wider variety of conditions, and involving them in more research can improve these models ensuring real-world validation.

Limitations

The sample size was relatively small. Since all the GPs involved volunteered to participate in the study, they are more or less interested in the field of dermatology with dermatology education, this fact may have increased the GP performance. Furthermore, the picture quality was labelled as bad in a minority of the cases presented, which may have affected the diagnostic accuracy.

Conclusion

Compatible with existing evidence, the study shows the potential using AI in a live clinical GP setting. Nearly 100% of the GPs thought the AI to be useful and used it to help them diagnose faster and more accurately. Further research should be done to see if other healthcare providers find can find a dermatology AI decision support tool can be useful.

References

1. Williams, C., & Little, P. (2017). The commonest new reason people present to general practitioners in England and Wales. British Journal of General Practice, 67(657), 307-313. https://doi.org/10.1111/j.1365-2133.2011.10464.x

2. Hussain, A., & Hussain, M. (2016). Dermatological consultations in primary care: a systematic review. Journal of Dermatological Treatment, 27(1), 1-10.

3. Federman, D. G. & Kirsner, R. S. The abilities of primary care physicians in dermatology.pdf. Am. J. Manag. Care 3(10), 1487–92 (1997). https://pubmed.ncbi.nlm.nih.gov/10178455/

4. Moreno, G., Tran, H., Chia, A. L. K., Lim, A. & Shumack, S. Prospective study to assess general practitioners’ dermatological diagnostic skills in a referral setting. Australas. J. Dermatol. 48(2), 77–82 (2007). https://doi:10.1111/j.1440-0960.2007.00340.x

5. Escalé-Besa A, Yélamos O, Vidal-Alaball J, et al. Exploring the potential of artificial intelligence in improving skin lesion diagnosis in primary care. Sci Rep. 2023;13(1):4293. Published 2023 Mar 15. doi:10.1038/s41598-023-31340-1 https://doi:10.1038/s41598-023-31340-1

6. E. Castillo-Arenas, V. Garrido, S. Serrano-Ortega, Skin Conditions in Primary Care: An Analysis of Referral Demand, Actas Dermo-Sifiliográficas (English Edition), Volume 105, Issue 3, 2014, Pages 271-275, ISSN 1578-2190 https://doi.org/10.1016/j.adengl.2013.10.005

7. Gonna N, Tran T, Bassett RL, et al. Sensitivity and specificity for skin cancer diagnosis in primary care providers: A systematic review. Springer Medicine. 2022;20:530-540. https://doi:10.1007/springermed567