Pyramid Scene Parsing Network Research Articles

Effect of Landscape Elements on Public Psychology in Urban Park Waterfront Green Space: A Quantitative Study by Semantic Segmentation

Urban park waterfront green spaces provide positive mental health benefits to the public. In order to further explore the specific influence mechanism between landscape elements and public psychological response, 36 typical waterfront green areas in Xihu Park and Zuohai Park in Gulou District, Fuzhou City, Fujian Province, China, were selected for this study. We used semantic segmentation technology to quantitatively decompose the 36 scenes of landscape elements and obtained a public psychological response evaluation using virtual reality technology combined with questionnaire interviews. The main results showed that: (1) the Pyramid Scene Parsing Network (PSPNet) is a model suitable for quantitative decomposition of landscape elements of urban park waterfront green space; (2) the public’s overall evaluation of psychological responses to the 36 scenes was relatively high, with the psychological dimension scoring the highest; (3) different landscape elements showed significant differences in four dimensions. Among the elements, plant layer, pavement proportion, and commercial facilities all have an impact on the four dimensions; and (4) the contribution rate of the four element types to the public’s psychological response is shown as spatial element (37.9%) > facility element (35.1%) > natural element (25.0%) > construction element (2.0%). The obtained results reveal the influence of different landscape elements in urban park waterfront green spaces on public psychology and behavior. Meanwhile, it provides links and methods that can be involved in the planning and design of urban park waterfront green space, and also provides emerging technical support and objective data reference for subsequent research.

RIFATA: Remora improved invasive feedback artificial tree algorithm-enabled hybrid deep learning approach for root disease classification

Plant root disease classification is very crucial for sustainable agriculture and it is very complex to monitor the diseases manually. The detection of plant diseases uses image processing since it involves a significant amount of work and takes a long time to process. The field of deep learning (DL) is exciting and has shown potential in terms of accuracy. The existing approaches failed to visualize the spot diseases. Hence, this research develops an automated system for root disease categorization utilizing a Remora Improved Feedback Artificial Tree Algorithm (RIFATA)-based hybrid deep learning model. Initially, pre-processing is executed through a Gaussian filter and root area segmentation is done under the segmentation process employing Pyramid Scene Parsing Network (PSPNet), where PSPNet is optimally tuned using RIFATA. To expand the dimensionality of data, the augmentation process is carried out using methodologies like translation, rotation, cropping, shearing, random erasing, and color space shifting. By using hybrid deep learning approaches like Deep Q Network (DQN) and Deep Residual Neural Network (DRN), which effectively train the classifiers using the same RIFATA, root disease classification is achieved. For the Alfalfa root crowns dataset, the developed RIFATA-based hybrid deep learning system achieved higher accuracy, sensitivity, and specificity of 0.941, 0.960, and 0.921. Although the generated model performs better in terms of accuracy, it was unable to categorize the different types of root diseases and would have remained a potential future approach.

PCL–PTD Net: Parallel Cross-Learning-Based Pixel Transferred Deconvolutional Network for Building Extraction in Dense Building Areas With Shadow

Urban building segmentation from remote sensed imageries is challenging because there usually exists a variety of building features. Furthermore, very high spatial resolution imagery can provide many details of the urban building, such as styles, small gaps among buildings, building shadows, etc. Hence, satisfactory accuracy in detecting and extracting urban features from highly detailed images still remains. Deep learning semantic segmentation using baseline networks works well on building extraction; however, their ability in building extraction in shadows area, unclear building feature, and narrow gaps among buildings in dense building zone is still limited. In this article, we propose parallel cross-learning-based pixel transferred deconvolutional network (PCL–PTD net), and then is used to segment urban buildings from aerial photographs. The proposed method is evaluated and intercompared with traditional baseline networks. In PCL–PTD net, it is composed of parallel network, cross-learning functions, residual unit in encoder part, and PTD in decoder part. The performance is applied to three datasets (Inria aerial dataset, international society for photogrammetry and remote sensing Potsdam dataset, and UAV building dataset), to evaluate its accuracy and robustness. As a result, we found that PCL–PTD net can improve learning capacities of the supervised learning model in differentiating buildings in dense area and extracting buildings covered by shadows. As compared to the baseline networks, we found that proposed network shows superior performance compared to all eight networks (SegNet, U-net, pyramid scene parsing network, PixelDCL, DeeplabV3+, U-Net++, context feature enhancement networ, and improved ResU-Net). The experiments on three datasets also demonstrate the ability of proposed framework and indicating its performance.

An Effective WBC Segmentation and Classification Using MobilenetV3–ShufflenetV2 Based Deep Learning Framework

White Blood Cells are essential in keeping track of a person’s health. However, the pathologist’s experience will determine how the blood smear is evaluated. Furthermore, it is still challenging to classify WBCs accurately because they have various forms, sizes, and colors due to distinct cell subtypes and labeling methods. As a result, a powerful deep learning system for WBC categorization based on MobilenetV3-ShufflenetV2 is described in this research. Initially, the WBC images are segmented using an efficient Pyramid Scene Parsing Network (PSPNet). Following that, MobilenetV3 and an Artificial Gravitational Cuckoo Search (AGCS)-based technique are used to extract and select the global and local features from the segmented images. Finally, the WBC images are divided into five classes using a ShufflenetV2 model. The proposed approach is evaluated on blood cell count and detection (BCCD) and Raabin-Wbc datasets and achieves 99.19% and 99% accuracy, respectively. Moreover, the results are satisfactory when compared to existing algorithms.

Learned Gradient Compression for Distributed Deep Learning.

Training deep neural networks on large datasets containing high-dimensional data requires a large amount of computation. A solution to this problem is data-parallel distributed training, where a model is replicated into several computational nodes that have access to different chunks of the data. This approach, however, entails high communication rates and latency because of the computed gradients that need to be shared among nodes at every iteration. The problem becomes more pronounced in the case that there is wireless communication between the nodes (i.e., due to the limited network bandwidth). To address this problem, various compression methods have been proposed, including sparsification, quantization, and entropy encoding of the gradients. Existing methods leverage the intra-node information redundancy, that is, they compress gradients at each node independently. In contrast, we advocate that the gradients across the nodes are correlated and propose methods to leverage this inter-node redundancy to improve compression efficiency. Depending on the node communication protocol (parameter server or ring-allreduce), we propose two instances for the gradient compression that we coin Learned Gradient Compression (LGC). Our methods exploit an autoencoder (i.e., trained during the first stages of the distributed training) to capture the common information that exists in the gradients of the distributed nodes. To constrain the nodes' computational complexity, the autoencoder is realized with a lightweight neural network. We have tested our LGC methods on the image classification and semantic segmentation tasks using different convolutional neural networks (CNNs) [ResNet50, ResNet101, and pyramid scene parsing network (PSPNet)] and multiple datasets (ImageNet, Cifar10, and CamVid). The ResNet101 model trained for image classification on Cifar10 achieved significant compression rate reductions with the accuracy of 93.57%, which is lower than the baseline distributed training with uncompressed gradients only by 0.18%. The rate of the model is reduced by 8095× and 8× compared with the baseline and the state-of-the-art deep gradient compression (DGC) method, respectively.

A Transferable Learning Classification Model and Carbon Sequestration Estimation of Crops in Farmland Ecosystem

Frequent agricultural activities in farmland ecosystems bring challenges to crop information extraction from remote sensing (RS) imagery. The accurate spatiotemporal information of crops serves for regional decision support and ecological assessment, such as disaster monitoring and carbon sequestration. Most traditional machine learning algorithms are not appropriate for prediction classification due to the lack of historical ground samples and poor model transfer capabilities. Therefore, a transferable learning model including spatiotemporal capability was developed based on the UNet++ model by integrating feature fusion and upsampling of small samples for Sentinel-2A imagery. Classification experiments were conducted for 10 categories from 2019 to 2021 in Xinxiang City, Henan Province. The feature fusion and upsampling methods improved the performance of the UNet++ model, showing lower joint loss and higher mean intersection over union (mIoU) values. Compared with the UNet, DeepLab V3+, and the pyramid scene parsing network (PSPNet), the improved UNet++ model exhibits the best performance, with a joint loss of 0.432 and a mIoU of 0.871. Moreover, the overall accuracy and macro F1 values of prediction classification results based on the UNet++ model are higher than 83% and 58%, respectively. Based on the reclassification rules, about 3.48% of the farmland was damaged in 2021 due to continuous precipitation. The carbon sequestration of five crops (including corn, peanuts, soybean, rice, and other crops) is estimated, with a total carbon sequestration of 2460.56, 2549.16, and 1814.07 thousand tons in 2019, 2020, and 2021, respectively. The classification accuracy indicates that the improved model exhibits a better feature extraction and transferable learning capability in complex agricultural areas. This study provides a strategy for RS semantic segmentation and carbon sequestration estimation of crops based on a deep learning network.

Sichuan Pepper Recognition in Complex Environments: A Comparison Study of Traditional Segmentation versus Deep Learning Methods

At present, picking Sichuan pepper is mainly undertaken by people, which is inefficient and presents the possibility of workers getting hurt. It is necessary to develop an intelligent robot for picking Sichuan peppers in which the key technology is accurate segmentation by means of mechanical vision. In this study, we first took images of Sichuan peppers (Hanyuan variety) in an orchard under various conditions of light intensity, cluster numbers, and image occlusion by other elements such as leaves. Under these various image conditions, we compared the ability of different technologies to segment the images, examining both traditional image segmentation methods (RGB color space, HSV color space, k-means clustering algorithm) and deep learning algorithms (U-Net convolutional network, Pyramid Scene Parsing Network, DeeplabV3+ convolutional network). After the images had been segmented, we compared the effectiveness of each algorithm at identifying Sichuan peppers in the various types of image, using the Intersection Over Union(IOU) and Mean Pixel Accuracy(MPA) indexes to measure success. The results showed that the U-Net algorithm was the most effective in the case of single front-lit clusters light without occlusion, with an IOU of 87.23% and an MPA of 95.95%. In multiple front-lit clusters without occlusion, its IOU was 76.52% and its MPA was 94.33%. Based on these results, we propose applicable segmentation methods for an intelligent Sichuan pepper-picking robot which can identify the fruit in images from various growing environments. The research showed good accuracy for the recognition and segmentation of Sichuan peppers, which suggests that this method can provide technical support for the visual recognition of a pepper-picking robot in the field.

Artificial Intelligence‐Assisted Repair System for Structural and Electrical Restoration Using 3D Printing

The characteristic of material accumulation makes 3D printing competitive in remanufacturing and repairing. However, conventional repair methods require additional equipment and manual intervention to sequentially finish complicated processes such as global scanning and reverse modeling, which results in efficiency reduction and usage restriction. To address the existing shortcomings, an automatic repair system with artificial intelligence (AI) assistance is developed, which includes a semantic segmentation module, deep reinforcement learning (DRL) module, and composite printing device. The damaged part features are extracted by semantic segmentation from the captured real‐time images to establish DRL maps, where the print motion is simulated and transmitted to the printer. The results indicate that the applied bilateral segmentation network (BiSeNetV2) is 59.03% and 29.90% faster than pyramid scene parsing network (PSPNet) and DeepLabV3+ architecture with satisfying accuracy. The established DRL model based on actual printing achieves the optimization of agent learning speed and print quality. The automatic system improves the repair efficiency by 294% compared to the conventional methods, and enables both structural and electrical repair through high‐temperature polymer–metal printing. This intelligent system enables industrial robots to independently handle unexpected tasks in complex and changeable environments through interdisciplinary knowledge integration of advanced manufacturing and AI.

Multi scale feature extraction network with machine learning algorithms for water body extraction from remote sensing images

ABSTRACT Water Body Extraction (WBE) is a challenging task in remote sensing, owing to the complexity of recognizing surface body objects with rich texture, spatial, spectral, temporal, and radiometric features. The use of spectral indices has shown to be successful in separating surface water from its surroundings at the cost of knowledge of appropriate threshold values. In the absence of knowledge on threshold values, extracting the water from remote sensing data is challenging, which is addressed by several Machine Learning (ML) and Deep Learning (DL) algorithms. However, the effectiveness of both ML and DL classifications is witnessed from visual features to semantic categories at the cost of distinct recognition between the water body and non-water body features. In this paper, a novel Multi Scale Feature Extraction Network (MSFEN) for extracting the pixel-level features from medium resolution remote sensing images is proposed and used traditional ML classifiers to extract the surface water bodies using pixel-level features extracted by MSFEN. The proposed framework is trained and tested on Linear Imaging Self Scanning Sensor – III (LISS-III) multispectral satellite images over major water reservoirs in Tamilnadu, Karnataka, Madhya Pradesh, and Odisha. Experimental results indicate that the proposed model MSFEN+SVM provides accurate extraction results by outperforming the existing state-of-the-art models (Fully Convolutional Network (FCN), Unet, SegNet, Multi Scale Convolutional Neural Network (MSCNN), Deepwater Map, Pyramid Scene Parsing Network (PSPNet), Improved PSPNet, Multi-scale Water Extraction Convolutional Neural Network (MWEN) and Multi-Scale Lake Water Extraction Network (MSLWENet) in terms of performance metrics considered.

IoT Enabled Deep Learning Based Framework for Multiple Object Detection in Remote Sensing Images

Advanced collaborative and communication technologies play a significant role in intelligent services and applications, including artificial intelligence, Internet of Things (IoT), remote sensing, robotics, future generation wireless, and aerial access networks. These technologies improve connectivity, energy efficiency, and quality of services of various smart city applications, particularly in transportation, monitoring, healthcare, public services, and surveillance. A large amount of data can be obtained by IoT systems and then examined by deep learning methods for various applications, e.g., object detection or recognition. However, it is a challenging and complex task in smart remote monitoring applications (aerial and drone). Nevertheless, it has gained special consideration in recent years and has performed a pivotal role in different control and monitoring applications. This article presents an IoT-enabled smart surveillance solution for multiple object detection through segmentation. In particular, we aim to provide the concept of collaborative drones, deep learning, and IoT for improving surveillance applications in smart cities. We present an artificial intelligence-based system using the deep learning based segmentation model PSPNet (Pyramid Scene Parsing Network) for segmenting multiple objects. We used an aerial drone data set, implemented data augmentation techniques, and leveraged deep transfer learning to boost the system’s performance. We investigate and analyze the performance of the segmentation paradigm with different CNN (Convolution Neural Network) based architectures. The experimental results illustrate that data augmentation enhances the system’s performance by producing good accuracy results of multiple object segmentation. The accuracy of the developed system is 92% with VGG-16 (Visual Geometry Group), 93% with ResNet-50 (Residual Neural Network), and 95% with MobileNet.

A Method of Image Semantic Segmentation Based on PSPNet

Image semantic segmentation is a visual scene understanding task. The goal is to predict the category label of each pixel in the input image, so as to achieve object segmentation at the pixel level. Semantic segmentation is widely used in automatic driving, robotics, medical image analysis, video surveillance, and other fields. Therefore, improving the effect and accuracy of image semantic segmentation has important theoretical research significance and practical application value. This paper mainly introduces the pyramid scene parsing network PSPNet based on pyramid pooling and proposes a parameter optimization method based on PSPNet model using GPU distributed computing method. Finally, it is compared with other models in the field of semantic segmentation. The experimental results show that the accuracy of the improved PSPNet model in this paper has been significantly improved on Pascal VOC 2012 + 2017 data set.

Deep segmentation and classification of complex crops using multi-feature satellite imagery

Accurate remote sensing-based land use and crop maps provide important and timely information for decision support in large-scale agricultural monitoring. Most existing multi-crop products for complex agricultural areas based on traditional machine learning algorithms are not suitable for large agricultural management because of the poor model transfer capabilities. Therefore, a deep segmentation and classification model including spatiotemporal transfer across regions and years must be developed. In this study, a deep learning approach was developed based on the UNet++ architecture by integrating feature fusion and upsampling of small samples for large-scale land use and crop mapping. Classification experiments were conducted for ten categories at four sites using 10 m resolution Sentinel-2A images from 2019 to 2021 containing 4,194,304 pixels. The joint loss, including the label smoothing cross entropy and Dice coefficient, and the mean intersection over union (mIoU) were used to evaluate the model performance depending on different features and upsampling schemes. The joint loss and mIoU values of the training model and the prediction accuracies of the test sites indicate that the scheme including the upsampling and fusion of spectral bands, vegetation indices, and texture features yields the optimal model performance. For comparison, UNet, DeepLab V3+, Pyramid Scene Parsing Network (PSPNet), and random forest models were built. The improved UNet++ model exhibits the best performance, with an overall accuracy, kappa, and macro F1 above 91 %, 85 %, and 51 %, respectively. The deep segmentation and classification results without training samples demonstrate the spatiotemporal transfer capability of the UNet++ architecture during the key crop growth period. The patch parameter values also indicate that the improved model exhibits a better shape compactness and significantly reduces and suppresses patch fragmentation. Based on the analysis of the overall confusion matrix, the proposed method improves the classification accuracy for datasets with imbalanced land cover and crop types. This study provides a strategy for large-scale and complex land use and crop mapping based on the integration of feature fusion and deep learning.

A Diameter Measurement Method of Red Jujubes Trunk Based on Improved PSPNet

A trunk segmentation and a diameter measurement of red jujubes are important steps in harvesting red jujubes using vibration harvesting robots as the results directly affect the effectiveness of the harvesting. A trunk segmentation algorithm of red jujubes, based on improved Pyramid Scene Parsing Network (PSPNet), and a diameter measurement algorithm to realize the segmentation and diameter measurement of the trunk are proposed in this research. To this end, MobilenetV2 was selected as the backbone of PSPNet so that it could be adapted to embedded mobile applications. Meanwhile, the Convolutional Block Attention Module (CBAM) was embedded in the MobilenetV2 to enhance the feature extraction capability of the model. Furthermore, the Refinement Residual Blocks (RRBs) were introduced into the main branch and side branch of PSPNet to enhance the segmentation result. An algorithm to measure trunk diameter was proposed, which used the segmentation results to determine the trunk outline and the normal of the centerline. The Euclidean distance of the intersection point of the normal with the trunk profile was obtained and its average value was regarded as the final trunk diameter. Compared with the original PSPNet, the Intersection-over-Union (IoU) value, PA value and Fps of the improved model increased by 0.67%, 1.95% and 1.13, respectively, and the number of parameters was 5.00% of that of the original model. Compared with other segmentation networks, the improved model had fewer parameters and better segmentation results. Compared with the original network, the trunk diameter measurement algorithm proposed in this research reduced the average absolute error and the average relative error by 3.75 mm and 9.92%, respectively, and improved the average measurement accuracy by 9.92%. To sum up, the improved PSPNet jujube trunk segmentation algorithm and trunk diameter measurement algorithm can accurately segment and measure the diameter in the natural environment, which provides a theoretical basis and technical support for the clamping of jujube harvesting robots.

Rapid detection of incomplete coal and gangue based on improved PSPNet

Aiming at the rapid identification of coal and gangue under multi-scale, adhesion, and half-occlusion conditions, a semantic segmentation network of coal and gangue image (SSNet_CG) based on the pyramid scene parsing network(PSPNet) is proposed. Firstly, the backbone feature extraction network of PSPNet is optimized. For the one, the attention mechanism is embedded in the inverted residual block (IRB) to strengthen the detailed feature information of coal and gangue in image; for another, depthwise separable convolution (DSC) and atrous convolution (AC) are used to replace the typical convolution to reduce parameters. Subsequently, the number of feature levels in the original pyramid pooling module (PPM) are reduced to minimize parameters. Finally, two feature fusion channels are added to refine the coal and gangue segmentation boundary in the adhesive state. Compared with some classic recognition models, the results show that our method has the best effects, the MPA, mIoU and F1_scores are respectively 97.3, 95.4 and 0.98, and the single image test time is 0.027 s. This method can accurately identify multi-scale and partially blocked coals and gangues.

Object-of-Interest Perception in a Reconfigurable Rolling-Crawling Robot.

Cebrenus Rechenburgi, a member of the huntsman spider family have inspired researchers to adopt different locomotion modes in reconfigurable robotic development. Object-of-interest perception is crucial for such a robot to provide fundamental information on the traversed pathways and guide its locomotion mode transformation. Therefore, we present a object-of-interest perception in a reconfigurable rolling-crawling robot and identifying appropriate locomotion modes. We demonstrate it in Scorpio, our in-house developed robot with two locomotion modes: rolling and crawling. We train the locomotion mode recognition framework, named Pyramid Scene Parsing Network (PSPNet), with a self-collected dataset composed of two categories paths, unobstructed paths (e.g., floor) for rolling and obstructed paths (e.g., with person, railing, stairs, static objects and wall) for crawling, respectively. The efficiency of the proposed framework has been validated with evaluation metrics in offline and real-time field trial tests. The experiment results show that the trained model can achieve an mIOU score of 72.28 and 70.63 in offline and online testing, respectively for both environments. The proposed framework’s performance is compared with semantic framework (HRNet and Deeplabv3) where the proposed framework outperforms in terms of mIOU and speed. Furthermore, the experimental results has revealed that the robot’s maneuverability is stable, and the proposed framework can successfully determine the appropriate locomotion modes with enhanced accuracy during complex pathways.

Improved PSPNet-based water shoreline detection in complex inland river scenarios

The water shoreline is essential for unmanned surface vessels (USVs) to navigate autonomously. Many existing traditional water shoreline detections approaches not only fail to overcome the effects of water reflections, image inversions, and other factors but are also unsuitable for water shoreline detection in a variety of weather conditions and in complex inland river scenarios. Therefore, we propose a water shoreline detection approach based on an enhanced Pyramid Scene Parsing Network (PSPNet). We introduce a migration learning approach to the PSPNet feature backbone extraction network Resnet50 to improve training efficiency and add a Convolutional Block Attention Module (CBAM) attention mechanism module to improve the robustness of training. In addition, the pyramid pooling module adds the branch of the atrous convolution module. Finally, the waterfront segmentation map is processed by the Canny edge detection method, which detects the water shorelines. For the network's training and validation, we use the USVInland dataset, the world's first urban inland driverless dataset. The experimental results show that the segmentation accuracy MIou of this paper is 96.87% and Accuracy is 98.41, which are higher than some mainstream algorithms. It is capable of detecting water shorelines accurately in a variety of interior river situations.

Benchmarking Deep Learning Models for Tooth Structure Segmentation

A wide range of deep learning (DL) architectures with varying depths are available, with developers usually choosing one or a few of them for their specific task in a nonsystematic way. Benchmarking (i.e., the systematic comparison of state-of-the art architectures on a specific task) may provide guidance in the model development process and may allow developers to make better decisions. However, comprehensive benchmarking has not been performed in dentistry yet. We aimed to benchmark a range of architecture designs for 1 specific, exemplary case: tooth structure segmentation on dental bitewing radiographs. We built 72 models for tooth structure (enamel, dentin, pulp, fillings, crowns) segmentation by combining 6 different DL network architectures (U-Net, U-Net++, Feature Pyramid Networks, LinkNet, Pyramid Scene Parsing Network, Mask Attention Network) with 12 encoders from 3 different encoder families (ResNet, VGG, DenseNet) of varying depth (e.g., VGG13, VGG16, VGG19). On each model design, 3 initialization strategies (ImageNet, CheXpert, random initialization) were applied, resulting overall into 216 trained models, which were trained up to 200 epochs with the Adam optimizer (learning rate = 0.0001) and a batch size of 32. Our data set consisted of 1,625 human-annotated dental bitewing radiographs. We used a 5-fold cross-validation scheme and quantified model performances primarily by the F1-score. Initialization with ImageNet or CheXpert weights significantly outperformed random initialization (P < 0.05). Deeper and more complex models did not necessarily perform better than less complex alternatives. VGG-based models were more robust across model configurations, while more complex models (e.g., from the ResNet family) achieved peak performances. In conclusion, initializing models with pretrained weights may be recommended when training models for dental radiographic analysis. Less complex model architectures may be competitive alternatives if computational resources and training time are restricting factors. Models developed and found superior on nondental data sets may not show this behavior for dental domain-specific tasks.

A deep learning-based hybrid artificial intelligence model for the detection and severity assessment of vitiligo lesions

BackgroundWe aimed to establish and validate a deep learning-based hybrid artificial intelligence (AI) model for the objective morphometric and colorimetric assessment of vitiligo lesions.MethodsTwo main datasets containing curated images of vitiligo lesions from Chinese patients (Fitzpatrick skin types III or IV) were established, including one with 2,720 images for lesion localization study and the other with 1,262 images for lesion segmentation study. Besides, an additional test set containing 145 images of vitiligo lesions from other Fitzpatrick skin types (I, II, or V) was also generated. A 3-stage hybrid model was constructed. YOLO v3 (You Only Look Once, v3) architecture was trained and validated to classify and localize vitiligo lesions, with sensitivity and error rate as primary performance outcomes. Then a segmentation study comparing 3 deep convolutional neural networks (DCNNs), Pyramid Scene Parsing Network (PSPNet), UNet, and UNet++, was carried out based on the Jaccard index (JI). The architecture with the best performance was integrated into the model. Three add-on metrics, namely VAreaA, VAreaR, and VColor were finally developed to measure absolute, relative size changes and pigmentation, respectively. Agreement between the AI model and dermatologist evaluators were assessed.ResultsThe sensitivity of the YOLO v3 architecture to detect vitiligo lesions was 92.91% with an error rate of 14.98%. The UNet++ architecture outperformed the others in the segmentation study (JI, 0.79) and was integrated into the model. On the additional test set, however, the model achieved a lower detection sensitivity (72.41%) and a lower segmentation score (JI, 0.69). With respect to size changes, no difference was observed between the AI model, trained dermatologists (W=0.812, P<0.05), and Photoshop analysis (P=0.075, P=0.212 respectively), which all displayed good concordance.ConclusionsWe developed a novel, convenient, objective, and quantitative deep learning-based hybrid model which simultaneously evaluated both morphometric and colorimetric vitiligo lesions from patients with Fitzpatrick skin types III or IV, rendering it suitable for the assessment of severity of vitiligo lesions in Asians in both clinic and research scenarios. More work is also warranted for its use in other ethnic skin groups.

Semi-Supervised Adversarial Semantic Segmentation Network Using Transformer and Multiscale Convolution for High-Resolution Remote Sensing Imagery

Semantic segmentation is a crucial approach for remote sensing interpretation. High-precision semantic segmentation results are obtained at the cost of manually collecting massive pixelwise annotations. Remote sensing imagery contains complex and variable ground objects and obtaining abundant manual annotations is expensive and arduous. The semi-supervised learning (SSL) strategy can enhance the generalization capability of a model with a small number of labeled samples. In this study, a novel semi-supervised adversarial semantic segmentation network is developed for remote sensing information extraction. A multiscale input convolution module (MICM) is designed to extract sufficient local features, while a Transformer module (TM) is applied for long-range dependency modeling. These modules are integrated to construct a segmentation network with a double-branch encoder. Additionally, a double-branch discriminator network with different convolution kernel sizes is proposed. The segmentation network and discriminator network are jointly trained under the semi-supervised adversarial learning (SSAL) framework to improve its segmentation accuracy in cases with small amounts of labeled data. Taking building extraction as a case study, experiments on three datasets with different resolutions are conducted to validate the proposed network. Semi-supervised semantic segmentation models, in which DeepLabv2, the pyramid scene parsing network (PSPNet), UNet and TransUNet are taken as backbone networks, are utilized for performance comparisons. The results suggest that the approach effectively improves the accuracy of semantic segmentation. The F1 and mean intersection over union (mIoU) accuracy measures are improved by 0.82–11.83% and 0.74–7.5%, respectively, over those of other methods.

A Large-Area Flexible Tactile Sensor for Multi-Touch and Force Detection Using Electrical Impedance Tomography

For large area robot skin design, the distribution of rigid components and wires in traditional array sensors leads to the decrease of the flexibility and extensibility of sensors. The flexible sensors based on non-invasive electrical impedance tomography (EIT) can avoid these shortcomings. However, design of a large-area flexible tactile sensors using EIT technique is still challenging due to the tradeoff between manufacturability and spatial resolution. In this study, we proposed a novel non-array EIT-based tactile sensor which is made of a porous elastic polymer and ionic liquid. The sensor free from internal array electrodes is straightforward to manufacture and can cover a large area with low cost. To improve the spatial resolution and touch sensitivity, we adopted a deep learning scheme Pyramid Scene Parsing Network (PSPNet) to postprocess the originally reconstructed images to enhance the sensor tactile perception. With this data-driven method, we achieved a single-point position detection error of 7.5± 4.5 mm without using internal electrodes. To overcome the location dependency of EIT sensing problems, we proposed a method of sub-regional fitting to calibrate the distributed forces for the large-area flexible tactile sensor and obtained a quantitative relationship between the touch force and EIT measurement for the entire sensing area for continuous sensing. The real-time performance of the proposed prototype sensor system demonstrated that we can achieve more accuracy of multi-touch and distributed force detection on the non-array sensors with a temporal resolution of about 1.3 frames/sec for potential applications in human-robot interfaces.