Image Block Research Articles

Image Classification Based on A Spatiotemporal Convolutional Neural Network

Image classification is usually viewed as a visual recognition task and has extensive applications. Traditional efforts for image classification typical apply convolutional neural network (CNN) to identify the category of images. However, due to the limited receptive fields, it is difficult for CNN-based methods to model the global relations in images. This drawback leads to low classification accuracy and difficulty in handling complex and diverse image data. To model the global relationships, some researchers have applied Transformers to image classification tasks. However, to satisfy with the serialization and parallelization requirements of Transformers, the images need to be divided into equally sized and non-overlapping image patches, which breaks the local information between adjacent image blocks, which losses the spatiotemporal information of the whole image. Also, because of the limited prior knowledge of Transformers, models often need to be pre-trained on large-scale datasets, resulting in high computational complexity. To simultaneously model the spatiotemporal information between adjacent image blocks and fully utilize the global information of the images, this work propose a novel Spatiotemporal Convolutional Networks enhanced Transformer (SCN-Transformer) model for the basic image classification task. The SCN-Transformers approach can extract both local, global and spatiotemporal information between adjacent image blocks at a lower computational cost. The model comprises three components. In specific, the stacked Transformer modules capture the local correlations in the image, the SCN module fuses the local and spatiotemporal information between adjacent image blocks, and leverages long-range dependencies between different image blocks to enhance the representative capabilities of model's features, allowing the model to learn semantics from different dimensions. The classification module is responsible for the final image classification. The final experiments on the ImageNet 1K dataset demonstrate that the present model can outperform the existing mainstream image classification methods, and also achieve a competitive accuracy score of 83.7%, which confirms the competitiveness of the approach on large-scale image datasets.

Security analysis of a reversible data hiding scheme in encrypted images by redundant space transfer

Redundant space transfer (RST) encryption can effectively enhance the security and embedding capacity of reversible data hiding in encrypted images. To demonstrate the security risks of RST encryption in the cloud environment, according to the known plaintext image and marked encrypted image(MEI) pairs, this paper proposes a known plaintext attack (KPA) based on hamming-weight of bit block. The proposed KPA method includes two stages. The first stage is to find the correct ciphertext image. For possible block sizes, all possible ciphertext images of MEI are reconstructed by deleting the embedded secret data. The Hamming weight image (HWI) of plaintext and all possible ciphertext images is calculated in m×8n domain (m, n is the image size). When the histogram distance between the plaintext HWI and a possible ciphertext HWI is the minimum, the correct ciphertext image is found. The second stage is to estimate the permutation keys. The bit block characteristic matrix (BBCM) is defined by using that the bit values of plaintext and ciphertext remain unchanged in the process of RST encryption. Based on the intersection of BBCM’s index set, the block permutation key is directly estimated. Then, the bit plane permutation key in each image block is further estimated. Experimental results show that for RST encryption, when the block size is not less than 3 × 3, the block permutation and bit plane permutation key can be estimated in less than 3 min by using one pair of plain–ciphertext. The estimation accuracy of block permutation key is more than 50%, and the estimation accuracy of bit plane permutation key is 10% for Qin’s RST algorithm, most of the content information of the ciphertext image is leaked.

Underwater calibration image enhancement based on image block decomposition and fusion

Underwater calibration image enhancement based on image block decomposition and fusion

A self-supervised fusion network for carotid plaque ultrasound image classification.

Carotid plaque classification from ultrasound images is crucial for predicting ischemic stroke risk. While deep learning has shown effectiveness, it heavily relies on substantial labeled datasets. Achieving high performance with limited labeled images is essential for clinical use. Self-supervised learning (SSL) offers a potential solution; however, the existing works mainly focus on constructing the SSL tasks, neglecting the use of multiple tasks for pretraining. To overcome these limitations, this study proposed a self-supervised fusion network (Fusion-SSL) for carotid plaque ultrasound image classification with limited labeled data. Fusion-SSL consists of two SSL tasks: classifying image block order (Ordering) and predicting image rotation angle (Rotating). A dual-branch residual neural network was developed to fuse feature presentations learned by the two tasks, which can extract richer visual boundary shape and contour information than a single task. In this experiment, 1270 carotid plaque ultrasound images were collected from 844 patients at Zhongnan Hospital (Wuhan, China). The results showed that Fusion-SSL outperforms single SSL methods across different percentages of labeled training data, ranging from 10 to 100%. Moreover, with only 40% labeled training data, Fusion-SSL achieved comparable results to a single SSL method (predicting image rotation angle) with 100% labeled data. These results indicate that Fusion-SSL could be beneficial for the classification of carotid plaques and the early warning of a stroke in clinical practice.

Enhancing Image Denoising Efficiency: Dynamic Transformations in BM3D Algorithm

This research aims to enhance the performance of image-denoising algorithms, particularly in the context of Block Matching 3D (BM3D) usage, focusing on improving image quality and retaining important information in noisy images. The novelty of this research lies in developing more effective and efficient image-denoising techniques by considering the characteristics of image blocks to improve denoising results. The method employed in this research involves the development of a new approach enabling the application of adaptive 2D and 3D transformations depending on the characteristics of the image block being processed. The research develops a new approach enabling the application of adaptive 2D and 3D transformations depending on the characteristics of the image block being processed. The results of this research indicate that the proposed adaptive approach in the BM3D image denoising algorithm can significantly improve denoising performance. Experimental results show that performing 2D transformations on blocks that do not have sufficiently similar blocks can yield better denoising results, especially at high noise levels.

A Feature-Oriented Reconstruction Method for Surface-Defect Detection on Aluminum Profiles

The number of defect samples on the surface of aluminum profiles is small, and the distribution of abnormal visual features is dispersed, such that the existing supervised detection methods cannot effectively detect undefined defects. At the same time, the normal texture of the aluminum profile surface presents non-uniform and non-periodic features, and this irregular distribution makes it difficult for classical reconstruction networks to accurately reconstruct the normal features, resulting in low performance of related unsupervised detection methods. Aiming at such problems, a feature-oriented reconstruction method of unsupervised surface-defect detection method for aluminum profiles is proposed. The aluminum profile image preprocessing stage uses techniques such as boundary extraction, background removal, and data normalization to process the original image and extract the image of the main part of the aluminum profile, which reduces the influence of irrelevant data features on the algorithm. The essential features learning stage precedes the feature-optimization module to eliminate the texture interference of the irregular distribution of the aluminum profile surface, and image blocks of the area images are reconstructed one by one to extract the features through the mask. The defect-detection stage compares the structural similarity of the feature images before and after the reconstruction, and comprehensively determines the detection results. The experimental results improve detection precision by 1.4% and the F1 value by 1.2% over the existing unsupervised methods, proving the effectiveness and superiority of the proposed method.

Applied Improved Canny Edge Detection for Diagnosis Medical Images of Human Brain Tumors

Medical image processing has become one of the crucial elements of the diagnostic process because of the increased usage of medical imaging recently, and clinicians' dependence on such computer-processed medical images in diagnosing patients. As the traditional Canny edge detection algorithm is sensitive to noise, it is easy to lose weak edge information when filtering out the noise, and its fixed parameters show poor adaptability. The suggested algorithm introduced the concept of image block intensity operator to replace image gradient. In addition, the computing speed of the suggested algorithm is relatively fast because it works block by block rather than pixel by pixel. Two adaptive threshold selection methods are presented, one based on the median accumulative histogram of image gradient magnitude and the other on the standard deviation for both types of image pixels (one with less edge information and the other with rich edge information). The proposed algorithm can be dividing into four stages: Input the medical digital image, convert the color medical image to gray-scale, applied improved canny edge detection, then calculate the MSE & PSNR Measures, in addition conduct a visual questionnaire by oncologists to find out which method that made the enhancement of the medical image clearer.

Privacy‐preserving in the smart healthcare system using steganography and chaotic functions based on DNA

AbstractThe smart healthcare system is one of the Internet of Things‐based applications that are increasingly used nowadays. One of the requirements of this system is privacy‐preserving, which should protect the disclosure of the patient record contents. In the present paper, a combination of chaotic functions and a new DNA‐based steganography method, and image blocking are suggested to protect patients' privacy‐preserving. First, the image is blocked, and the initial key for the chaotic function will be gained by using the centers of the blocks. Then the data and image are transformed into DNA sequences. Finally, one of the randomly chosen DNA strands of data is XOR with the center of one randomly selected block and will be placed in one of the block pixels. The failure to send the initial key separately to generate random numbers, as well as the random selection of secret data bits and image pixels for steganography has increased the security of the proposed method. The simulation results not only indicate the quality of the stego‐image but also show better performance of the proposed method than the existing methods.

Results of modeling different schemes of the spatial orientation and scanning series of base blocks of images to confront an unauthorized extraction of steganographic data

This work presents the results of modeling attempts at unauthorized extraction of steganocontent (halftone test images) under the condition of selective compromise of each of the two active processing parameters of the source array series of base blocks (BB) of content, i.e.: - the scheme scanning of BB series and the spatial processing of BB. The current program version ensures consistent realization of the main stages of content processing with the necessary settings parameters. As part of the modeling, it is suggested that the attacker has correctly identified one of the two current content processing parameters. Several modifications of the main schemes scanning of BB series and the spatial orientation of BB (rotation and horizontal mirroring) as an additional mechanism to counteract attempts of illegitimate content extraction are considered. The modeling was conducted on the examples of three types of images: - portrait, landscape, and mnemonic scheme. Manipulations with the spatial orientation parameter of BB strengthen the opportunities to counteract attempts at unauthorized data extraction. Characteristic quantitative and time histograms for different dimensions BB of content, changes in the peak of value signal-to-noise ratio for different types of schemes scanning BB series are presented, and samples of attacked test images are presented. The analysis and generalization of the main differences in the attack results using different parameters of the spatial processing of BB and ways of scanning series of BB of image-content are performed. Attention is drawn to the fact that the use of two active processing parameters of the source array of BB series is an effective and computationally «simple» means of counteracting attempts at unauthorized data extraction. The relationship between the stage of preprocessing the source content and the parameters of the formed arrays BB is emphasized. It is concluded that the introduction into the structure of the data extractor key, the elements of «The state of scanning» and «The spatial processing of BB», strengthens the overall capabilities to counteract attacks. The used processing parameters of the source array of BB series determine the structure of visual artifacts of attacked images but do not produce a simple solution to identify the attacked image at the level of classifying the type of source images. Prospective directions for further modeling of the main protection mechanisms within the proposed algorithm concept are indicated.

Classifier-guided multi-style tile image generation method

Image generative models for ceramic tile design lack style diversity and controllability of high-quality generated styles. It is difficult to find a series of ceramic tiles with the same texture but distinct styles, that makes it a challenging for users to select from a limited number of tiles with a single style. Although, Generative Adversarial Networks (GANs) can slightly increase the style diversity of tile images, the style controllability remains very weak. Additionally, concatenating generated tile image blocks to obtain a larger texture region can easily result in seams at the boundaries that decrease image quality. In this paper, we propose a style transfer method for ceramic tiles texture generation that combines a classifier-guided StyleGAN with AdaIN-GAN to overcome the above limitations. Firstly, we introduce a new conditional classifier-guided module into the StyleGAN. With the guidance of the input condition vector, the output image is made to have the tile style characteristics that match the vector. At the same time, the fusion of the condition vectors realizes the style gradient effect of the tile image to expand the style diversity. Secondly, we use the AdaIN-GAN to color the original texture in tile style. The style images generated by StyleGAN are then used as a dataset for model training to enhance the generalization ability of the model and achieve a style transfer effect with fixed texture features but significantly diverse styles. Finally, a linear weighted image stitching method is adopted, which uses an adaptive kernel linear weighted matrix to cover and splice arbitrary seams with image blocks, thereby successfully eliminating seams and enhancing image continuity. When this method is applied to high-resolution tile image generation, the method still maintains higher continuity and clearer image quality. Extensive experiments and human evaluation confirm the superior performance of the proposed method compared with other SOTA methods. The experimental results also verify that the new tile images generated by the proposed algorithm have diverse styles and meet the design requirements for tile style diversity.

An image crypt model based on enhanced two-dimensional sine-cosine chaotic map

Sensitive images are being transmitted over insecure media as network and information technology evolve at a rapid pace. As a result, people are becoming more concerned about information security, specifically digital image protection. For securing digital images, numerous image encryption algorithms have been proposed. From that, a chaotic map has been proved a perfect image protection tool as it contains efficient random behaviour. This work proposes a novel two-dimensional sine-cosine map for developing an image cryptosystem. In the developed framework, at first, two random chaotic series are generated by the developed sine-cosine map, then the given image is divided into two blocks, and each block is confused by utilizing different chaotic key series. In the diffusion process, the chaotic series is first changed into integer form, which is applicable for executing some arithmetic operation between chaotic values and image pixels. In the diffusion phase, each confused image block pixel is converted into binary form, and a bit shifting operation is executed depending on the integer form of chaotic series. In the final process, a bit-xor operation is performed between the partial diffused image and chaotic series to obtain the final encrypted image. The proposed model's security strength will be evaluated using statistical and differential attack analysis.

SSPNet: Scale and spatial priors guided generalizable and interpretable pedestrian attribute recognition

Global feature based Pedestrian Attribute Recognition (PAR) models are often poorly localized when using Grad-CAM for attribute response analysis, which has a significant impact on the interpretability, generalizability and performance. Previous researches have attempted to improve generalization and interpretation through meticulous model design, yet they often have neglected or underutilized effective prior information crucial for PAR. To this end, a novel Scale and Spatial Priors Guided Network (SSPNet) is proposed for PAR, which is mainly composed of the Adaptive Feature Scale Selection (AFSS) and Prior Location Extraction (PLE) modules. The AFSS module learns to provide reasonable scale prior information for different attribute groups, allowing the model to focus on different levels of feature maps with varying semantic granularity. The PLE module reveals potential attribute spatial prior information, which avoids unnecessary attention on irrelevant areas and lowers the risk of model over-fitting. More specifically, the scale prior in AFSS is adaptively learned from different layers of feature pyramid with maximum accuracy, while the spatial priors in PLE can be revealed from part feature with different granularity (such as image blocks, human pose keypoint and sparse sampling points). Besides, a novel IoU based attribute localization metric is proposed for Weakly-supervised Pedestrian Attribute Localization (WPAL) based on the improved Grad-CAM for attribute response mask. The experimental results on the intra-dataset and cross-dataset evaluations demonstrate the effectiveness of our proposed method in terms of mean accuracy (mA). Furthermore, it also achieves superior performance on the PCS dataset for attribute localization in terms of IoU. Code will be released at https://github.com/guotengg/SSPNet.

Rural disparities in emergency department utilization for migraine care.

To evaluate differences in emergency department (ED) utilization and quality of care for migraine in patients with rural and nonrural residences. Migraine is a significant problem in the United States with direct health-care utilization cost amounting to US $4.2 billion annually. A considerable portion of this cost is attributed to more than 4 million annual ED visits for migraine and headache. Previous research has documented health disparities among rural populations in other disease states, which can be influenced by factors such as socioeconomic status and health-care access. Given these associations, it was hypothesized that patients with rural residence in a national sample would have increased ED utilization for migraine compared to patients with nonrural residence. This was a cross-sectional epidemiologic study to evaluate rural disparities in ED utilization and quality of care for migraine in the United States in 2019. ED encounter data were collected from the Healthcare Cost and Utilization Project (HCUP) Nationwide Emergency Department Sample (NEDS) and Kentucky State Emergency Department Database (KY-SEDD). The primary outcome was crude and age-adjusted rates of ED encounters for migraine per 10,000 population. ED encounters were included if they had a primary International Classification of Diseases, Tenth Revision, Clinical Modification diagnosis code of G43.xx. ED encounters lacking sufficient data to classify into a geographic group were excluded. Secondary outcomes examined differences in quality of care, including mean ED charges and the proportion of encounters with medication administration, imaging, and nerve block service codes between groups. One hundred eighty-three thousand two hundred eleven national ED discharges were classified as rural patient encounters and 627,176 were classified as nonrural. The rural group had significantly higher crude and age-adjusted rates of ED utilization for migraine (crude: rural 39.8, 95% confidence interval [CI] 36.9-42.7; nonrural 22.2, 95% CI 21-23.5 and age-adjusted: rural 41.8, 95% CI 38.8-44.8; nonrural 23.4, 95% CI 22.1-24.7). Opioid utilization was higher in rural encounters (rural n = 26,764, 14.6%; nonrural n = 50,367, 8%; p < 0.001). A Kentucky sub-analysis found 5210 ED discharges were classified as Appalachian and 12,551 as non-Appalachian. The Appalachian group had significantly higher ED utilization rates for migraine compared to the non-Appalachian and national rural groups (crude: Appalachian 44.9, 95% CI 43.7-46.2; age-adjusted: Appalachian 47.4, 95% CI 46.1-48.8). The Kentucky Appalachian group also demonstrated significantly higher opioid analgesia use compared to the national rural group (Appalachian n = 1031, 19.8%; p < 0.001). This study suggests rural populations, particularly in Appalachia, may experience significantly higher ED utilization for migraine compared to nonrural patients. Moreover, rural populations were more likely to receive suboptimal migraine management with opioid analgesia. Multimodal health-care interventions should be developed to improve access to outpatient migraine care and further investigate potential risk factors in the rural population. With high ED utilization, the Appalachian population may benefit most from such an intervention.

Activity increases in empathy-related brain regions when children contribute to peers’ sadness and happiness

Responding empathically when causing peers’ emotions is critical to children’s interpersonal functioning, yet there are surprising gaps in the literature. Previous research has focused on empathy when witnessing others’ emotions instead of causing others’ emotions, on negative emotions instead of positive emotions, and on behavioral correlates instead of neural correlates. In this study, children (N = 38; Mage = 9.28 years; 50% female) completed a functional magnetic resonance imaging block design task in which they played a rigged game where they won and lost coins for themselves or peers and viewed their peers’ happiness and sadness. We used a region of interest approach to test whether activity in brain regions associated with positive and negative empathy in adults showed significantly greater activity in each condition (i.e., when children won and lost tokens for themselves and peers) compared with a fixation baseline. We predicted that experiencing self-conscious emotions, such as pride and guilt, would heighten the experience of empathy. Activity in the amygdala, which is associated with visceral arousal, and in the anterior insula and dorsal anterior cingulate cortex, which are associated with integrated arousal, increased significantly when winning and losing for oneself and peers and observing their resulting happy and sad facial expressions. Activity did not differ when playing for oneself versus peers, indicating that self-conscious emotions do not heighten empathy and instead support similar neural processes underlying firsthand and secondhand (empathic) emotions. These findings support that empathy during middle childhood involves the same brain regions as empathy during adulthood and that children experience firsthand and secondhand positive and negative emotions in similar ways.

Satellite–Terrestrial Collaborative Object Detection via Task-Inspired Framework

Recently, buoyed by advances in the space industry, low Earth orbit (LEO) satellites have become an important part of the Internet of Things (IoT). LEO satellites have entered the era of a big data link with IoT, how to deal with the data from the satellite IoT is a problem worthy of consideration. Conventional object detection method in optical remote sensing simply transmits the raw data to the ground. However, it ignores the properties of the images and the connection with the downstream task. To obtain efficient data transmission and accurate object detection, we propose a task-inspired satellite-terrestrial collaborative object detection framework called STCOD. It detects regions of interest (ROIs) and adopts a block-based adaptive sampling method to compress the background (BG) in optical remote sensing images by introducing satellite edge computing (SEC) on satellites. The STCOD framework also sets the transmission priority of image blocks according to their contributions to the task and uses fountain code to ensure the reliable transmission of important image blocks. We build a whole software simulation framework to validate our method, including the satellite module, the transmission module, and the terrestrial module. Extensive experimental results show that the STCOD framework can reduce the amount of downlink data decreased by 50.04% while losing the detection accuracy by 0.54%. In our simulated satellite-terrestrial link, the STCOD framework can reduce the number of satellite-to-terrestrial transmissions by half. When the packet loss rate is between 5% and 20%, the detection accuracy is lost only 0.05% to 0.5%.

Enhancing of coverless image steganography capacity based on image block features

Enhancing of coverless image steganography capacity based on image block features

Activation extending based on long-range dependencies for weakly supervised semantic segmentation.

Weakly supervised semantic segmentation (WSSS) principally obtains pseudo-labels based on the class activation maps (CAM) to handle expensive annotation resources. However, CAM easily involves false and local activation due to the the lack of annotation information. This paper suggests weakly supervised learning as semantic information mining to extend object mask. We proposes a novel architecture to mining semantic information by modeling through long-range dependencies from in-sample and inter-sample. Considering the confusion caused by the long-range dependencies, the images are divided into blocks and carried out self-attention operation on the premise of fewer classes to obtain long-range dependencies, to reduce false predictions. Moreover, we perform global to local weighted self-supervised contrastive learning among image blocks, and the local activation of CAM is transferred to different foreground area. Experiments verified that superior semantic details and more reliable pseudo-labels are captured through these suggested modules. Experiments on PASCAL VOC 2012 demonstrated the proposed model achieves 76.6% and 77.4% mIoU in val and test sets, which is superior to the comparison baselines.

Social concerns about anxious arousal explain the association between neural responses to anxious arousal pictures and social anxiety

Cognitive theories propose that social anxiety disorder involves heightened attention to anxious arousal symptoms due to worries that they may evoke rejection from others. Supporting this, studies have shown that social anxiety is related to greater attention to representations of anxious arousal and to anxiety sensitivity social concerns, which refers to sensitivity to feelings of anxious arousal during social situations. However, this has not yet been tested using neural indices of attention to images depicting anxious arousal. To examine these associations, the current study examined early and sustained attentional bias to anxious arousal images using the P2 and the late positive potential (LPP), respectively. Electroencephalogram data were collected while a non-clinical sample of undergraduate students (N = 106) viewed images of people exhibiting anxious arousal in addition to blocks of negative and neutral images from the IAPS. The neural response to anxious arousal images was isolated using residual scores (e.g., using linear regression to predict the P2 elicited by anxious arousal images from the P2 elicited by neutral images (P2neutral→AA) or negative images (P2negative→AA), then saving the unstandardized residuals). There was an indirect effect of the P2neutral→AA and P2negative→AA waveforms that was explained by anxiety sensitivity social concerns. Additionally, there was an indirect effect of both LPP waveforms on social anxiety symptoms during the early time window of the LPP (400–700 ms). At the later time window of the LPP (700–1000 ms), there was an indirect effect of the LPPneutral→AA residual waveform, but not the LPPnegative→AA, on social anxiety symptoms.

Cryptanalysis of iterative encryption and image sharing scheme based on the VQ attack

Reversible data hiding in encrypted images (RDHEI) is widely employed to protect privacy in images stored on cloud storage. Recently, Yu et al. [22] proposed an effective image encryption scheme called iterative encryption and image sharing (IEIS), which is highly compatible with RDHEI. In this study, we analyze the characteristics of IEIS and present a cryptanalysis scheme based on vector quantization (VQ) attack. To capture the pixel-changing pattern of an image block, the concept of pixel difference matrix (PDM) is introduced. VQ attack is subsequently utilized to estimate the ciphertext block. Furthermore, we employ a well-trained convolutional neural network (CNN) based discriminator to identify the proper block permutation sequence within the secret key space. Experimental results demonstrate the efficacy of our proposed cryptanalysis scheme. Based on our study, we conclude that the IEIS scheme raises security concerns.

Efficient Super-Resolution System With Block-Wise Hybridization and Quantized Winograd on FPGA

Super-resolution (SR) techniques aim to restore a high-resolution image from low-resolution images, which are often used to assist the enhancement of image/video quality under the rapid development of high-resolution and high-framerate media. Recently, neural network (NN)-based methods perform much better image reconstruction quality than classical approaches. However, the unacceptable computation complexity as well as the huge memory footprints of NNs limit the throughputs and scalability of these SR systems. In this work, we analyze several key issues in the design of NN-based SR systems first. Then, we propose a three-level systematic optimization methodology for SR systems to reduce computation overhead and keep image quality. At the algorithm level, we introduce image blocking to SR tasks and develop a block-wise SR algorithm based on the hybrid of NN and interpolation with a consistent image block evaluation metric. The configurable hybrid parameters help the SR algorithm to achieve a flexible trade-off between the computation overhead and image quality. At the operator level, we focus on the transpose convolution operators commonly used for upsampling in SR neural networks. We propose an efficient Winograd-based transposed convolution acceleration method. Through the efficient sub-convolutions conversion and the Winograd specialization, this methods enables unified Winograd transformations and simplified data access patterns. At the data level, we propose a novel quantization method for Winograd-aware SR neural networks to get better quantized accuracy. Comprehensive evaluations demonstrate the effectiveness of these optimizations. Our SR system reduces a large number of multiplications with great scalability and supports 4K@120fps and 8K@30fps outputs with acceptable image quality degradation.