Instance Segmentation for Melanoma Lesion Detection

Preface#

Project Overview#

This project designs an experimental device for collecting images of malignant skin tumors. It mainly consists of a camera, an LED soft light, and a dermatoscope. The device connects via USB to a computer or embedded devices such as a Raspberry Pi, and provides full-HD imaging with optical magnification up to 50×. After the dermatoscope captures images, the video stream is sent to a Qt application. On the edge device, the images are preprocessed and then transmitted via HTTP to an ECS server, where a deployed deep learning model performs detection. Meanwhile, data is stored in an RDS database, and images are archived and backed up to OBS object storage. Recognition results are fed back to the Qt UI to visualize the diagnosis. On the mobile app side, patients can also view their own diagnosis results. The “识瘤者” app uses the ArkUI framework, lightweight storage, and other components, improving development efficiency by 30%.

Project Background and Pain Points#

Skin is the largest organ by area in the human body. It serves as a barrier to maintain the stability of the internal environment and participates in metabolism. For an adult, the skin’s surface area is about 2 square meters and weighs around 16% of body weight. However, factors such as sunlight exposure, chemical carcinogens, and genetics can lead to malignant skin tumors. It is a low-grade malignant tumor originating from basal cells of the epidermis. Globally, there are on average 1.52 million new cases each year, and malignant skin tumors rank third among all skin tumors.

Our investigation shows that as years increase, the number of community hospitals also increases year by year. But on the right side, in community hospitals represented by Jingyang Central Hospital, we found that the proportion of staff with only high school or junior-college education has reached 63.8%. From this we can see that professional expertise is especially important for skin disease diagnosis. Because early-stage malignant skin tumors are difficult to distinguish, many cases are not discovered in time; most patients are already in late stages when the disease is found. But the five-year survival rate for late-stage malignant skin tumors is only 4.6%, while early-stage survival is 89%, so early detection, early treatment, and early follow-up are urgently needed.

After collecting information, we found that hospitals usually use dermatoscopes to observe skin lesions. Dermatoscopes are easy to use and reduce unnecessary biopsies. However, there are problems: images often have low contrast and are difficult to distinguish by the naked eye. Manual inspection is time-consuming, labor-intensive, and inefficient. There are not enough experts; in large top-tier hospitals, the doctor-to-patient ratio can be 1:70,000, and diagnosis heavily depends on expert experience.

System Architecture#

Our team is committed to solving these problems. The detection system uses a dermatoscope as the acquisition endpoint to capture lesion information. After data processing on the device side, it is transmitted to the server side for instance segmentation and other processing, and the results are returned to the user-side system.

So how do we implement it? First, the dermatoscope captures images, which are sent to the Qt application. The edge device preprocesses the images and then sends them to a Huawei Cloud ECS server for recognition. Data is stored in an RDS database, and images are archived and backed up to OBS object storage. Recognition results are fed back to the Qt UI. Our “识瘤者” app uses the ArkUI framework, lightweight storage, and more, improving development efficiency by over 30%.

Video can be streamed to the Qt UI in real time. After confirming the specific lesion region for recognition, the system returns lesion-region segmentation and category outputs.

Patients can also view their diagnosis records in the HarmonyOS app.

Doctors can manage patient data on the web side, and also answer questions on the web side to improve the professionalism level of doctors and hospitals.

Feature Demo#

Using the DAYU200 OpenHarmony developer board for edge-side deployment of an OpenHarmony-based interactive app, we implement visual guided operations for patient information collection to ensure quick onboarding.

1. Take photos on mobile

2. Image upload and recognition

3. View detection results

Product Value#

This project aims to break the professional-resource barrier in diagnosing subtle skin lesions in primary healthcare institutions through an edge-cloud collaborative AI-assisted diagnosis platform. Facing real-world pain points—difficulty of early-stage visual inspection for malignant skin tumors and extreme scarcity of expert resources in top-tier hospitals—the system combines advanced deep-learning instance segmentation with portable dermatoscope hardware, bringing high-precision AI screening capabilities down to the community level. This greatly improves the accuracy and efficiency of lesion recognition and helps patients seize the early-intervention window that can significantly improve survival rates. In addition, backed by Huawei Cloud’s strong computing infrastructure and the lightweight interactive experience of the OpenHarmony ecosystem, the “识瘤者” system not only provides patients with an accessible health-management and tracking channel, but also builds a digitally empowering platform for primary-care doctors that integrates case management with knowledge assessment—truly realizing the social value of technology accessibility and medical benevolence.