Pixels In Region Research Articles

Complementary deep learning and chemometrics: A case of pear fruit centroid detection and spectral model application for fruit spectral image processing

A novel case of combining deep learning and chemometrics for spectral image processing is presented. The case involved the application of deep transfer learning for detecting and locating the fruit centroid to extract pixels for spectral model development and application. The selected fruit case involved a non-symmetrical fruit pear where the interesting area for spectral model application is not the centroid of the whole fruit unlike fruit such as apples but the centroid of the belly part of the pear fruit. Hence, the task of object detection is replaced with the task of symmetrical region (fruit belly) detection on the pear fruit such that the spectral model can be applied in the centroid pixels of the symmetrical region. For spectral modelling, the latent variables based regression technique called partial least-square (PLS) regression was used. For spectral modelling, PLS was preferred over deep learning as there was a low number of samples points to train a deep spectral model. The deep transfer learning allowed 100 % correct detection of the pear fruit belly part with the intersection over union score of 0.82. Furthermore, the RMSEP = 0.77 % was attained with the PLS modelling to predict dry matter. The presented approach can support the wide application of spectral imaging for fresh fruit analysis, particularly when imaging is performed simultaneously on multiple objects and the objects are non-symmetrical in shape.

AFM IMAGE SEGMENTATION BASED ON WAVE GROWTH OF LOCAL MAXIMUM REGIONS WITH THEIR SELECTION IN ORDER OF DECREASING VALUES

The problem of determining the number of objects in atomic force microscopy (AFM) images is considered. For the automatic (without operator participation) solution of this problem, segmentation is used, dividing images into areas containing objects of interest. Known segmentation algorithms based on the morphological watershed, defining the boundaries of areas by local minima of pixel brightness, having significant segmentation errors of AFM images and high computational complexity. Less computationally complex segmentation algorithms based on wave growth of regions require preliminary determination of the starting points of growth on AFM images under the control of an operator. Algorithms for growing regions without preliminary selection of the starting points of growth have the least computational complexity, but they segment the AFM image with a large error. To improve the accuracy of automatic determination of the number of objects in AFM images, a model and an algorithm for the wave growth of the regions of local maxima with their selection in decreasing order of values are proposed, which differ in the use of a brightness threshold varying from maximum to minimum to select growth pixels of regions or pixels attached to pixels of adjacent existing areas. The model provides parallel expansion of the boundaries of areas and automatic determination of the initial growth pixels during the segmentation process. The proposed model and algorithm make it possible to eliminate segmentation error characteristic of the marker watershed, growing areas and the Vincent – Sulli watershed, and thereby increase the accuracy of determining the number of objects in atomic force microscopy images.

Multitemporal Change Detection and Irregular Land Shape Area Measurement from Multispectral Sensor Images through BSO Algorithm

Pixel-based classification and area measurement play a vital role in satellite image processing. The accuracy in classification and area measurement is required for remote monitoring various regions such as water area, cultivation regions, reservoir water spread, and disease spread in cultivation area. Traditionally, Google Earth Pro-based area measurement has error in area measurement due to curvature nature and irregular land surface. Moreover, exact point identification on land surface on Earth pro is difficult due to frequent changes on the land surface. The problem in the land area measurement is mostly affected due to the man-made changes in that particular land area. In this paper, we solve land area measurement error problem through the automated single and multithresholding pixels of different iterations on the land surface by the BSO algorithm. The land cover region pixel intensity changes on the curvature region of land surface with respect to spatial and temporal variations which are identified through BSO optimization-based image segmentation for exact area measurement. For the experimentation of accurate measurement, land area images such as urban, semiurban, hill, and coastal region from LANDSAT and SENTINEL images for period 2016 to 2019 are taken for the land area measurement study. BSO enhances and segments the land regions such as road, building, water body, vegetation, bare land, hill, and coastal region of about 32% more than particle swarm optimization (PSO) algorithm. Furthermore, the urban land area measurement accuracy increases to about 97% than the irregular land surface area.

A fuzzy membership based comparison of the grey matter (GM) in cognitively normal (CN), mild cognitive impairment (MCI), and Alzheimer’s disease (AD) using brain images

Grey matter (GM) in human brain contains most of the important cells covering the regions involved in neurophysiological operations such as memory, emotions, decision making, etc. Alzheimer’s disease (AD) is a neurological disease that kills the brain cells in regions which are mostly involved in the neurophysiological operations. Mild Cognitive Impairment (MCI) is a stage between Cognitively Normal (CN) and AD, where a significant cognitive declination can be observed. The destruction of brain cells causes a reduction in the size of GM. Evaluation of changes in GM, may help in studying the overall brain transformations and accurate classification of different stages of AD. In this work, firstly skull of brain images is stripped for 5 different slices, then segmentation of GM is performed. Finally, the average number of pixels in grey region and the average atrophy in grey pixels per year is calculated and compared amongst CN, MCI, and AD patients of various ages and genders. It is observed that, for some subjects (in some particular ages) from different dementia stages, pattern of GM changes is almost identical. To solve this issue, we have used the concept of fuzzy membership functions to classify the dementia stages more accurately. It is observed from the comparison that average difference in the number of pixels between CN and MCI= 10.01%, CN and AD= 19.63%, MCI and AD= 10.72%. It can be also observed from the comparison that, the average atrophy in grey matter per year in CN= 1.92%, MCI= 3.13%, and AD= 4.33%.

Impact of illumination gradients on the raw, atmospherically and topographically corrected snow and vegetation areas of Jhelum basin, Western Himalayas

Mountainous terrains severely affect the sun-target-sensor geometry due to surface dispersion of solar radiation, resulting to variation in the observed radiance. The effect of topographic shadow on raw, atmospherically and topographically processed Normalized Difference Snow Index (NDSI) and Normalized Difference Vegetation Index (NDVI) values was investigated for Jhelum basin, Kashmir Himalaya. The results indicate that NDSI and NDVI derived from raw images show less area of snow and vegetation, due to underestimation of corresponding pixels in the shadow regions, in comparison to atmospherically and topographically processed datasets. Among the three datasets, raw images exhibited low values of minimum and maximum NDSI and NDVI for both shadow and sunlit slopes. All the three datasets show significantly higher values of minimum and maximum NDSI, NDVI and surface temperature in the sunlit slopes, relative to shadow slopes. This research demonstrates that topographic shadow has significant impact on the raw, DOS atmospheric corrected and C topographic processed Landsat-8 Operational Land Imager (OLI) datasets as well as on Thermal Infrared Sensor (TIRS) sensor.

The Diffuse X-Ray Background of the Insight-HXMT/LE Telescope in the Galactic Plane

Accurately estimating of diffuse X-ray background (DXB) is essential for the investigation of sources in the Galactic plane observed with Insight-HXMT/LE, which is a collimated telescope in the soft X-ray energy band with a relatively large field of view. In the high-Galactic-latitude region, DXB is dominated by the cosmic X-ray background, which is almost uniform, but DXB in the Galactic plane region is more complex due to the Galactic H i absorption and the contribution of the Galactic ridge X-ray emission. This study, as a part of background estimation of LE, focuses on estimating the contribution of DXB in the Galactic plane to Insight-HXMT/LE observations. We calculate DXB confined in a region of 0° < l < 360° and ∣b∣ < 10°, where l and b denote Galactic longitude and latitude, respectively, with the first 3 yr of Galactic-plane-scanning survey data of Insight-HXMT/LE. The Galactic plane is divided into 360 × 20 small pixels (1° × 1° per pixel), and a DXB spectrum is obtained for each pixel. An indirect method is developed for the pixels of the bright source regions, which brings a systematic error of ∼10%. The systematic error brought by the satellite attitude is ∼7% on average for all the pixels in the Galactic plane. The LE DXB spectrum obtained in this study is consistent with that reported by RXTE’s Proportional Counter Array.

Multi-perspective label based deep learning framework for cerebral vasculature segmentation in whole-brain fluorescence images.

The popularity of fluorescent labelling and mesoscopic optical imaging techniques enable the acquisition of whole mammalian brain vasculature images at capillary resolution. Segmentation of the cerebrovascular network is essential for analyzing the cerebrovascular structure and revealing the pathogenesis of brain diseases. Existing deep learning methods use a single type of annotated labels with the same pixel weight to train the neural network and segment vessels. Due to the variation in the shape, density and brightness of vessels in whole-brain fluorescence images, it is difficult for a neural network trained with a single type of label to segment all vessels accurately. To address this problem, we proposed a deep learning cerebral vasculature segmentation framework based on multi-perspective labels. First, the pixels in the central region of thick vessels and the skeleton region of vessels were extracted separately using morphological operations based on the binary annotated labels to generate two different labels. Then, we designed a three-stage 3D convolutional neural network containing three sub-networks, namely thick-vessel enhancement network, vessel skeleton enhancement network and multi-channel fusion segmentation network. The first two sub-networks were trained by the two labels generated in the previous step, respectively, and pre-segmented the vessels. The third sub-network was responsible for fusing the pre-segmented results to precisely segment the vessels. We validated our method on two mouse cerebral vascular datasets generated by different fluorescence imaging modalities. The results showed that our method outperforms the state-of-the-art methods, and the proposed method can be applied to segment the vasculature on large-scale volumes.

Computer-Aided Mural Digital Restoration under Generalized Regression Neural Network

Aiming at the digital protection of classical murals and according to the method of generalized regression neural network (GRNN), a digital restoration is proposed in the paper. Firstly, the existing defect photos are processed preliminarily, including the elimination of noise, the extraction of boundary pixels of the region to be repaired, and the establishment of several small block regions centered on these pixels. Then, similar known pixel regions are found as sample pixel blocks, which are used as input samples of GRNN. Finally, the GRNN is adopted to obtain the approximate estimation function, and the adaptive smoothing parameters are introduced to obtain the pixel information of the area to be repaired. Through model prediction, by acquiring the pixel information of the area to be repaired, the damaged area of the original image can be repaired. The method proposed is compared with the traditional repair methods, the results show that the method is close to the texture structure image restoration method in peak signal-to-noise ratio, and the restoration results are in line with the expectation.

Minimalistic fully convolution networks (MFCN): pixel-level classification for hyperspectral image with few labeled samples.

Most of the existing deep learning methods for hyperspectral image (HSI) classification use pixel-wise or patch-wise classification. In this paper, we propose an image-wise classification method, where the network input is the original hyperspectral cube rather than the spectral curve of each pixel (i.e., pixel-wise) or neighbor region of each pixel (i.e., patch-wise). Specifically, we propose a minimalistic fully convolution network (MFCN) and a semi-supervised loss function, which can perform pixel-level classification for HSI with few labeled samples. The comparison experiments demonstrated the progress of our methods, using three new benchmark HSI datasets (WHU-Hi-LongKou, WHU-Hi-HanChuan and WHU-Hi-HongHu) with wavelength range from 400 to 1000nm. In the comparison experiments, we randomly selected 25 labeled pixels from each class for training, equivalent to only 0.11%, 0.16%, and 0.14% of all labeled pixels for the three datasets, respectively. In addition, through ablation studies and theoretical analysis, we verified and analyzed the effectiveness and superiority of our design choices.

Attention‐guided black‐box adversarial attacks with large‐scale multiobjective evolutionary optimization

Fooling deep neural networks (DNNs) with black-box optimization has become a popular adversarial attack fashion, as the prior structural knowledge of DNNs is always unknown. Nevertheless, recent black-box adversarial attacks may struggle to balance their attack ability and visual quality of the generated adversarial examples (AEs) in tackling high-resolution images. In this paper, we propose an attention-guided black-box adversarial attack based on the large-scale multiobjective evolutionary optimization, termed LMOA. By considering the spatial semantic information of images, we first take advantage of the attention map to determine the perturbed pixels. Instead of attacking the entire image, reducing the perturbed pixels with the attention mechanism can help to avoid the notorious curse of dimensionality and thereby improve the performance of attacking. Second, a large-scale multiobjective evolutionary algorithm traverse the reduced pixels in the salient region. Benefiting from its characteristics, the generated AEs can fool target DNNs while being invisible by human vision. Extensive experimental results have verified the effectiveness of the proposed LMOA on the ImageNet data set. More importantly, it is more competitive to generate high-resolution AEs with the better visual quality than the existing black-box adversarial attacks.

Development of chipscale InGaN RGB displays using strain-relaxed nanosphere-defined nanopillars

Chip-scale red, green and blue (RGB) light emission on an InGaN/GaN multi-quantum well wafer adopting a top-down fabrication approach is demonstrated in this study, facilitated by shadow-masked nanosphere lithography for precise site-controlled nano-patterning. Exploiting the strain relaxation mechanism by fabricating arrays of nanosphere-defined nanopillars of two different dimensions utilizing a sequential shadow-masked nanosphere coating approach into the blue and green light-emitting pixel regions on a red-light emitting InGaN/GaN wafer, RGB light emission from a monolithic chip is demonstrated. The micro-sized RGB light-emitting pixels emit at 645 nm–680 nm, 510 nm–521 nm and 475 nm–498 nm respectively, achieving a maximum color gamut of 60% NTSC and 72% sRGB. Dimensional fluctuations of the nanopillars of 73% and 71% for the green and blue light-emitting pixels, respectively, are estimated from scanning electron microscope images of the fabricated device, corresponding to fluctuations in spectral blue-shifts of 5.4 nm and 21.2 nm as estimated by strain-coupled k · p Schrödinger calculations, consistent with observations from micro-photoluminescence (μ-PL) mapping which shows deviations of emission wavelengths for the RGB light-emitting pixels to be 8.9 nm, 14.9 nm and 23.7 nm, respectively. The RGB pixels are also configured in a matrix-addressable configuration to form an RGB microdisplay, demonstrating the feasibility of the approach towards chip-scale color displays.

Effect of shadow on atmospheric and topographic processed NDSI values in Chenab basin, western Himalayas

The present study was undertaken in the Chenab basin, western Himalayas to explore the effect of shadow on atmospherically and topographically corrected normalized difference snow index (NDSI) values. The research was conducted using the Landsat-8 Operational Land Imager (OLI) and Thermal Infrared Sensor (TIRS) datasets of two time periods i.e., February 2015 and February 2018. Our results suggested that total snow cover area (SCA) computed using NDSI from raw satellite images, under estimated SCA (viz. 10,938.66 and 7739.97 Km2); relative to atmospherically corrected (AC) (14,819.01 and 11,199.7 Km2) and topographically corrected (TC) (14,882.07 and 11,188.18 Km2) images during both the time periods (i.e., 2015 and 2018, respectively). This is mainly due to the non-identification of snow pixels in the shadow regions of raw imageries. Maximum SCA lies in the higher NDSI range in case of AC and TC images, unlike raw image where maximum SCA lies in lower NDSI range for both sunlit and shadow test sites. In the sunlit and shadow terrains, minimum NDSI values were found to be higher for TC images as compared to AC images, except few observations which exhibited similar values for both AC and TC images. In case of maximum NDSI values, most of the observations exhibited similar values for both AC and TC images, except few observations which depicted higher maximum NDSI value in AC images as compared to TC images in both the terrains. This observation ascertains the narrow range of NDSI values for TC images as compared to AC images, which could be attributed to reduced background reflection and low atmospheric scattering in the TC images. Comparison among shadow and sunlit snow terrains, reveals low values of minimum and maximum NDSI in the shadow site, which is ascertained by significant (p < 0.001) t-values of means difference between shadow and sunlit terrains. Moreover, snow surface temperature (SST) values computed for both the time periods, reveal low SST values for shadow sites (t-values −5.1 and − 10.57 for minimum SST, and − 9.1 and − 12.61 for maximum SST for the years 2015 and 2018, respectively) as compared to sunlit sites, and this observation validates that minimum amount of solar radiation reaches the shadow sites. Thus, it is concluded that AC and TC are helpful for delineation of snow pixels under shadow, but shadow has a significant impact on the spectral reflectance of snow, even in the AC and TC images. The influence of shadow should be taken into account for accurate estimation of physical properties and broadband albedo of the snow.

Association of left ventricular tissue heterogeneity and intramyocardial fat on computed tomography with ventricular arrhythmias in ischemic cardiomyopathy.

BackgroundGray zone, a measure of tissue heterogeneity on late gadolinium enhanced–cardiac magnetic resonance (LGE-CMR) imaging, has been shown to predict ventricular arrhythmias (VAs) in ischemic cardiomyopathy (ICM) patients. However, no studies have described whether left ventricular (LV) tissue heterogeneity and intramyocardial fat mass on contrast-enhanced computed tomography (CE-CT), which provides greater spatial resolution, is useful for assessing the risk of VAs in ICM patients with LV systolic dysfunction and no previous VAs.ObjectiveThe purpose of this proof-of-concept study was to determine the feasibility of measuring global LV tissue heterogeneity and intramyocardial fat mass by CE-CT for predicting the risk of VAs in ICM patients with LV systolic dysfunction and no previous history of VAs.MethodsPatients with left ventricular ejection fraction ≤35% and no previous VAs were enrolled in a prospective, observational registry and underwent LGE-CMR. From this cohort, patients with ICM who additionally received CE-CT were included in the present analysis. Gray zone on LGE-CMR was defined as myocardium with signal intensity (SI) > peak SI of healthy myocardium but <50% maximal SI. Tissue heterogeneity on CE-CT was defined as the standard deviation of the Hounsfield unit image gradients (HU/mm) within the myocardium. Intramyocardial fat on CE-CT was identified as regions of image pixels between –180 and –5 HU. The primary outcome was VAs, defined as appropriate implantable cardioverter-defibrillator shock or sudden arrhythmic death.ResultsThe study consisted of 47 ICM patients, 13 (27.7%) of whom experienced VA events during mean follow-up of 5.6 ± 3.4 years. Increasing tissue heterogeneity (per HU/mm) was significantly associated with VAs after multivariable adjustment, including for gray zone (odds ratio [OR] 1.22; P = .019). Consistently, patients with tissue heterogeneity values greater than or equal to the median (≥22.2 HU/mm) had >13-fold significantly increased risk of VA events, relative to patients with values lower than the median, after multivariable adjustment that included gray zone (OR 13.13; P = .028). The addition of tissue heterogeneity to gray zone improved prediction of VAs (area under receiver operating characteristic curve increased from 0.815 to 0.876). No association was found between intramyocardial fat mass on CE-CT and VAs (OR 1.00; P = .989).ConclusionIn ICM patients, CE-CT–derived LV tissue heterogeneity was independently associated with VAs and may represent a novel marker useful for risk stratification.

Cheating in (halftone-secret) visual cryptography: Analysis of blind authentication schemes

In visual cryptography (VC), cheating is an important security concern where dishonest participants will fool honest ones and make them accept a fake secret by providing fake shares. Share and blind authentications are two categories of cheating prevention, and the last one relies on the inherent robust of shares against cheating attacks. In the previous studies, cheating in VC only focuses on operating a ‘pixel block’ instead of a region of adjacent pixels. However, the well-known advantage of VC is to decode the secret image by using the human vision system (HVS), so it leads to a natural issue to reconsider cheating a region. In this paper, we formally address the binocular cheating attack (BCA) for a region to augment effectiveness of original cheating for a block. Finally, we demonstrate how to realize BCA by presenting non-trivial techniques against some blind authentication schemes, and further obtain implausible results. The BCA can also be applied to halftone secret.

Breast tumor segmentation in digital mammograms using spiculated regions

Mammogram image segmentation is the process of partitioning mammograms into meaningful and separate areas. However, during the segmentation process, masses are extracted and the spiculated regions of a mass, which contain significant characteristics of the mass margins, are omitted. The present research introduces a new method for segmentation of tumor mammograms that extracts the spiculated regions and the mass core. Generally, the pixels of a spiculated region are located along a line and the pixels of the mass core regions are similar. The proposed method extracts these regions using the differences between a pixel and its adjacent pixels. The proposed method uses three thresholds to delete redundant pixels from the spiculated regions and the mass core. These regions then are merged to form the segmented tumor. The results show that the respective mean of the Dice and Jaccard coefficients for the suggested segmentation method, respectively, are 0.9309 and 0.9024 for MIAS and 0.9557 and 0.9132 for DDSM. Quantitative analysis of the results confirms that the suggested segmentation method is comparable to other techniques and extracts the segmentation of tumor accurately.

Image declipping: Saturation correction in single images

High dynamic range (HDR) images present fine details in a scene and are visually more appealing than low dynamic range (LDR) images, since they contain a greater dynamic range of color gamut. HDR compatible displays are currently high-cost, therefore tone-mapping algorithms have widely been used to obtain high quality images for LDR screens with a lower cost. However, tone-mapped images may contain clipped pixel regions, which should be corrected to retrieve the lost information, to acquire visually pleasing LDR images. In a single image, the recovery of color and texture information in clipped regions is challenging, yet an attractive research field in image processing. Although there are several algorithms present in literature, developing a general framework for different types of image content is hard to achieve. This study proposes a single image declipping method based on linear embeddings, difference of pixels and block-search. Experimental results carried out on a tone-mapped HDR image dataset and LDR images demonstrate that the proposed algorithm is able to successfully recover saturated pixels in various types of images. Detailed statistical and visual comparisons show that this approach produces superior results on average for both tone-mapped and LDR images when compared to existing techniques.

Edge Guided Context Aggregation Network for Semantic Segmentation of Remote Sensing Imagery

Semantic segmentation of remote sensing imagery (RSI) has obtained great success with the development of deep convolutional neural networks (DCNNs). However, most of the existing algorithms focus on designing end-to-end DCNNs, but neglecting to consider the difficulty of segmentation in imbalance categories, especially for minority categories in RSI, which limits the performance of RSI semantic segmentation. In this paper, a novel edge guided context aggregation network (EGCAN) is proposed for the semantic segmentation of RSI. The Unet is employed as backbone. Meanwhile, an edge guided context aggregation branch and minority categories extraction branch are designed for a comprehensive enhancement of semantic modeling. Specifically, the edge guided context aggregation branch is proposed to promote entire semantic comprehension of RSI and further emphasize the representation of edge information, which consists of three modules: edge extraction module (EEM), dual expectation maximization attention module (DEMA), and edge guided module (EGM). EEM is created primarily for accurate edge tracking. According to that, DEMA aggregates global contextual features with different scales and the edge features along spatial and channel dimensions. Subsequently, EGM cascades the aggregated features into the decoder process to capture long-range dependencies and further emphasize the error-prone pixels in the edge region to acquire better semantic labels. Besides this, the exploited minority categories extraction branch is presented to acquire rich multi-scale contextual information through an elaborate hybrid spatial pyramid pooling module (HSPP) to distinguish categories taking a small percentage and background. On the Tianzhi Cup dataset, the proposed algorithm EGCAN achieved an overall accuracy of 84.1% and an average cross-merge ratio of 68.1%, with an accuracy improvement of 0.4% and 1.3% respectively compared to the classical Deeplabv3+ model. Extensive experimental results on the dataset released in ISPRS Vaihingen and Potsdam benchmarks also demonstrate the effectiveness of the proposed EGCAN over other state-of-the-art approaches.

A Tropical Cyclone Center Location Method Based on Satellite Image.

Accurately detecting and locating the center of the tropical cyclone is critical for the trajectory forecasting. This study proposed an automatic method for centers' location of the tropical cyclones based on the visible or the infrared satellite images. The morphological structure of the tropical cyclone is modeled using the circular pattern. The tropical cyclone center is located based on regional pixels instead of skeleton points. All pixels in a segmented cloud cluster vote for a 2-dimensional accumulator. The center of the cloud cluster is computed by the mean voting distances, which are calculated by fitting quadratic functions in every column of the two-dimensional (2D) accumulator. Then, a linear function is fitted according to the functional relationship between the mean voting distance and voting angle. The fitted coefficients of the linear function are the center coordinates of the tropical cyclone. The proposed method for centers location of the tropical cyclones is tested using visible and infrared satellite images. The results of center location are compared with the best track provided in JMA datasets.

Establishing and validating a spotted tongue recognition and extraction model based on multiscale convolutional neural network

ObjectiveIn tongue diagnosis, the location, color, and distribution of spots can be used to speculate on the viscera and severity of the heat evil. This work focuses on the image analysis method of artificial intelligence (AI) to study the spotted tongue recognition of traditional Chinese medicine (TCM). MethodsA model of spotted tongue recognition and extraction is designed, which is based on the principle of image deep learning and instance segmentation. This model includes multiscale feature map generation, region proposal searching, and target region recognition. Firstly, deep convolution network is used to build multiscale low- and high-abstraction feature maps after which, target candidate box generation algorithm and selection strategy are used to select high-quality target candidate regions. Finally, classification network is used for classifying target regions and calculating target region pixels. As a result, the region segmentation of spotted tongue is obtained. Under non-standard illumination conditions, various tongue images were taken by mobile phones, and experiments were conducted. ResultsThe spotted tongue recognition achieved an area under curve (AUC) of 92.40%, an accuracy of 84.30% with a sensitivity of 88.20%, a specificity of 94.19%, a recall of 88.20%, a regional pixel accuracy index pixel accuracy (PA) of 73.00%, a mean pixel accuracy (mPA) of 73.00%, an intersection over union (IoU) of 60.00%, and a mean intersection over union (mIoU) of 56.00%. ConclusionThe results of the study verify that the model is suitable for the application of the TCM tongue diagnosis system. Spotted tongue recognition via multiscale convolutional neural network (CNN) would help to improve spot classification and the accurate extraction of pixels of spot area as well as provide a practical method for intelligent tongue diagnosis of TCM.

Dairy cow lameness detection using a back curvature feature

• A new deep learning framework to detect dairy cow lameness was proposed. • This method was more stable in the antagonism of environmental robustness. • This method improves the detection accuracy of dairy cow lameness. Manual detection of lameness poses several problems, such as difficulty in finding sudden, severe or early lameness behavior. A dairy cow’s lameness is closely related to curvature of the cow’s back. Focusing on the curvature features of dairy cows’ backs, this study proposes a lameness detection method that combines machine vision technology with a deep learning algorithm. Firstly, the FLYOLOv3 algorithm was used to construct a Cow’s Back Position Extraction (CBPE) model to realize the extraction of the dairy cow’s back position coordinates. Simultaneously, a First to Last Frame Image Difference (FLFID) algorithm was used to construct a Cow’s Object Region Extraction (CORE) model to separate the dairy cow from the image background and obtain the pixel region of the dairy cow. Then, a Cow’s Back Curvature Extraction (CBCE) model was used to extract the dairy cow’s back curvature data from the acquired dairy cow’s back position and pixel region of the dairy cow. Finally, a Noise+ Bilateral Long Short-term Memory (BiLSTM) model was used to predict the curvature data and match the curvature features of the dairy cow’s lameness, so as to classify and detect dairy cow lameness. To verify the effectiveness of the algorithm, 567 videos were used to train the network model in a Long Short-term Memory (LSTM) model, a BiLSTM model, Noise+LSTM model, and the model proposed in this paper, respectively, and 243 videos were used for verification and testing. According to the fitting curvature data of the dairy cows’ back obtained by the algorithm used in this paper, it was found that the average classification accuracy of the model proposed in this research was 8.04%, 2.09%, and 5.78% higher than the average classification accuracy of the LSTM, BiLSTM, Noise+LSTM models, respectively. In the parallel experiment that classified the detection of dairy cow lameness, the average classification accuracy of the model proposed in this paper was 96.61%. The above results show that the lameness of dairy cows can be correctly detected through analysis of the curvature features of dairy cows' backs. The proposed method is a novel, deep learning-based method for dairy cow lameness early detection which may have significant economic impact on the dairy industry, and the proposed method provides an innovative means for detecting dairy cow lameness.