V2 Dataset Research Articles

Lightweight MobileNet Model for Image Tempering Detection

In recent years, there has been a wide range of image manipulation identification challenges and an overview of image tampering detection and the relevance of applying deep learning models such as CNN and MobileNet for this purpose. The discussion then delves into the construction and setup of these models, which includes a block diagram as well as mathematical calculations for each layer. A literature study on Image tampering detection is also included in the discussion, comparing and contrasting various articles and their methodologies. The study then moves on to training and assessment datasets, such as the CASIA v2 dataset, and performance indicators like as accuracy and loss. Lastly, the performance characteristics of the MobileNet and CNN designs are compared. This work focuses on Image tampering detection using convolutional neural networks (CNNs) and the MobileNet architecture. We reviewed the MobileNet architecture's setup and block diagram, as well as its application to Image tampering detection. We also looked at significant literature on Image manipulation detection, such as major studies and their methodologies. Using the CASIA v2 dataset, we evaluated the performance of MobileNet and CNN architectures in terms of accuracy and loss. This paper offered an overview of the usage of deep learning and CNN architectures for image tampering detection and proved their accuracy in detecting manipulated images.

Responsiveness of the International Spinal Cord Injury Quality of Life Basic Data Set V2.0: An international longitudinal study

Context/Objective Examine the sensitivity of the International Spinal Cord Injury Quality of Life Basic Data Set V2.0 (QoL-BDS V2.0) to reflect changes in mobility and secondary health conditions (SHCs) between inpatient rehabilitation and one-year follow-up. Design International longitudinal study. Questionnaires were administered at baseline (Median 6 weeks, inter-quartile range 4–10 weeks post-onset) and after 12 months. Setting Spinal cord rehabilitation institutions in the US, Brazil, Australia and the Netherlands. Participants : Individuals with recent onset of spinal cord injury or disease (SCI/SCD) admitted to inpatient rehabilitation. Outcome measures The QoL-BDS V2.0, comprises four items on satisfaction with ‘life as a whole’, ‘physical health’, ‘psychological health’, and ‘social life’. Mobility level was measured with a single item and SHCs with the SCI Secondary Conditions Scale (SCI-SCS). Results Of the 160 participants, 61% had SCI, 48% had tetraplegia and 82% were wheelchair-users. Scores on ‘life as a whole’, ‘physical health’ and the total scale were significantly higher at follow-up compared to baseline in the total sample and the SCD subgroup, but not in the SCI subgroup. Increases in ‘physical health’, ‘psychological health’, ‘social life’ and the total score were significantly associated with improvements in the SCI-SCS or mobility scores. Participants with improved SCI-SCS and mobility at follow-up showed significantly more improvement in satisfaction with social life and the total score compared to participants without such favorable changes. Conclusion The results of this study provide partial evidence of responsiveness of the QoL-BDS V2.0 total score as a measure of QoL among individuals with SCI/SCD.

Multi-stage information diffusion for joint depth and surface normal estimation

Depth and surface normal estimations are important for 3D geometric perception, which has numerous applications including autonomous vehicles and robots. In this paper, we propose a lightweight Multi-stage Information Diffusion Network (MIDNet) for the simultaneous prediction of depth and surface normals from a single RGB image. To obtain semantic and detail-preserving features, we adopt a high-resolution network as our backbone to learn multi-scale features, which are then fused into shared features for the two tasks. To mutually boost each task, a Cross-Correlation Attention Module (CCAM) is proposed to adaptively integrate information for the prediction of the two tasks in multiple stages, including feature-level information interaction and task-level information interaction. Ablation studies show that the proposed multi-stage information diffusion strategy can boost the performance gain for the two tasks at different levels. Compared to current state-of-the-art methods on the NYU Depth V2, Stanford 2D-3D-Semantic and KITTI datasets, our method achieves superior performance for both monocular depth and surface normal estimation.

JSNet++: Dynamic Filters and Pointwise Correlation for 3D Point Cloud Instance and Semantic Segmentation

In this paper, we propose a novel joint instance and semantic segmentation approach, called JSNet++, to address the instance and semantic segmentation tasks of 3D point clouds simultaneously. We first introduce a basic joint segmentation framework (JSNet). It fuses features from different layers of the backbone network to obtain more discriminative features and makes the two tasks take advantage of each other with a joint instance and semantic segmentation (JISS) module. Specifically, the JISS transforms semantic features into instance embedding space, and then the transformed features are fused with instance features to facilitate instance segmentation. Meanwhile, the JISS module also makes semantic segmentation benefit from instance segmentation by aggregating instance features to semantic feature space. To further reduce the memory consumption of JSNet, we design a dynamic filters for convolution (DFConv) on point clouds. Specifically, we exploit the geometry and density information to generate the dynamic filters, which are used to perform depthwise convolution with the input features. Afterwards, we unify the spatial correlation and channel correlation into a module to fully explore the pointwise correlation in point clouds, and we develop an improved JISS module (JISS*) by using the pointwise correlation module to further improve the accuracy of segmentation. Finally, based on the JSNet, DFConv and JISS*, we propose a new joint segmentation network, termed JSNet++. Experimental results on the benchmarks S3DIS and ScanNet v2 datasets demonstrate the effectiveness of our approach, and our method achieves significant performance improvements over baseline on both instance and semantic segmentation.

Southern Europe and western Asian marine heatwaves (SEWA-MHWs): a dataset based on macroevents

Abstract. Marine heatwaves (MHWs) induce significant impacts on marine ecosystems. There is a growing need for knowledge about extreme climate events to better inform decision-makers on future climate-related risks. Here we present a unique observational dataset of MHW macroevents and their characteristics over the southern Europe and western Asian (SEWA) basins, named the SEWA-MHW dataset (https://doi.org//10.5281/zenodo.7153255; Bonino et al., 2022). The SEWA-MHW dataset is derived from the European Space Agency Sea Surface Temperature Climate Change Initiative (ESA SST CCI) v2 dataset, and it covers the 1981–2016 period. The methodological framework used to build the SEWA-MHW dataset is the novelty of this work. First, the MHWs detected in each grid point of the ESA CCI SST dataset are relative to a time-varying baseline climatology. Since intrinsic fluctuation and anthropogenic warming are redefining the mean climate, the baseline considers both the trend and the time-varying seasonal cycle. Second, using a connected component analysis, MHWs connected in space and time are aggregated in order to obtain macroevents. Basically, a macroevent-based dataset is obtained from a grid cell-based dataset without losing high-resolution (i.e., grid cell) information. The SEWA-MHW dataset can be used for many scientific applications. For example, we identified phases of the well-known MHW of summer 2003, and taking advantage of statistical clustering methods, we clustered the largest macroevents in SEWA basins based on shared metrics and characteristics.

Quantum-Involution inspire false positive reduction in pulmonary nodule detection

Lung cancer is a potentially fatal disease worldwide, and improving the accuracy of early lung cancer diagnosis plays a key role in improving the prognosis of patients. In recent years, as Deep Learning (DL) is increasingly applied in the field of Pulmonary Nodule Detection (PND), computer-aided systems for PND based on DL have made significant contributions. However, due to the stringent requirement for high sensitivity, many computer-aided PND systems inevitably have high false positive rate. Reducing the false positive rate continue to face challenge due to the variable morphology of pulmonary nodules. In this study, a novel MultiLayer Perceptron (MLP)-based False Positive Reduction network, Wave-Involution MLP (WINMLP), is proposed to reduce the false positive rate from a new perspective. We design a Progressive Multi-Scale Fusion block based on the novel Involution to fuse global features preferably. Moreover, inspired by Quantum theory, we design a CT-WaveMLP backbone, which transforms CT images into Wave functions and enhances feature extraction capability. We performed experiments on LUNA V2 dataset and the results show WINMLP achieves the average CPM of 0.861, which has a better performance compared with existing excellent methods.

THCANet: Two-layer hop cascaded asymptotic network for robot-driving road-scene semantic segmentation in RGB-D images

In several existing red–green–blue and depth (RGB-D) semantic segmentation algorithms, schemes are used to supplement contextual information through multilayer feature interactions. However, these approaches ignore the complementation of the contextual information and the introduction of noise interfering with the segmentation process. To minimize noise interference during this process, we introduce a two-layer hop cascaded asymptotic network (THCANet) for robot-driving road-scene semantic segmentation in RGB-D images. To exploit the depth map and supervision to strengthen semantic segmentation, we propose an attention cross-fusion module for the interactive combination of RGB-D features through multimodality weighting. Notably, the information of the two modalities reduces noise during fusion. After fusing features from the RGB-D modalities, we also use a novel multiscale context module to fuse features at multiple scales and employ a jump cascade architecture between the modules to recover lost context information and suppress irrelevant noise. Moreover, multiple supervision is performed at different segmentation stages to improve accuracy. The proposed THCANet system demonstrates the best performance on a robot-driving road dataset compared with similar methods, and its generalization ability is demonstrated using the NYU-Depth V2 dataset.

Monocular Depth Estimation of Noncooperative Spacecraft Based on Deep Learning

Estimating position and stereographic geometry information of noncooperative spacecraft through three-dimensional (3-D) reconstruction techniques is of great significance. Depth information acquisition is an important component of 3-D reconstruction. Monocular cameras are cheaper and more widely used than depth sensors. The relative positions of noncooperative spacecraft and our spacecraft can be calculated from depth maps of monocular depth estimation and the camera parameters, providing data support for subsequent tracking and capture missions. This paper proposes a monocular depth estimation network combining the convolutional neural network (CNN) and a vision transformer (VIT) to improve the prediction accuracy of few-shot samples. We extract detail features and global features from the CNN and VIT encoders, respectively, and then fuse deep features and shallow features by a skip-connected upsampling decoder. Compared with the representative depth estimation algorithms in recent years on the NYU-Depth V2 dataset, the proposed network structure combines the advantages of the CNN and VIT as well as estimates the global depth of the scene more accurately while maintaining details. To solve the lack of spacecraft data collection, a new dataset is made from 3-D simulation models. Experiments on the self-made dataset demonstrate the feasibility of this method in aerospace engineering.

Digital Phenotyping Models of Symptom Improvement in College Mental Health: Generalizability Across Two Cohorts.

Smartphones can be used to gain insight into mental health conditions through the collection of survey and sensor data. However, the external validity of this digital phenotyping data is still being explored, and there is a need to assess if predictive models derived from this data are generalizable. The first dataset (V1) of 632 college students was collected between December 2020 and May 2021. The second dataset (V2) was collected using the same app between November and December 2021 and included 66 students. Students in V1 could enroll in V2. The main difference between the V1 and V2 studies was that we focused on protocol methods in V2 to ensure digital phenotyping data had a lower degree of missing data than in the V1 dataset. We compared survey response counts and sensor data coverage across the two datasets. Additionally, we explored whether models trained to predict symptom survey improvement could generalize across datasets. Design changes in V2, such as a run-in period and data quality checks, resulted in significantly higher engagement and sensor data coverage. The best-performing model was able to predict a 50% change in mood with 28days of data, and models were able to generalize across datasets. The similarities between the features in V1 and V2 suggest that our features are valid across time. In addition, models must be able to generalize to new populations to be used in practice, so our experiments provide an encouraging result toward the potential of personalized digital mental health care.

Removal of Redundant Information via Discrete Representation for Monocular Depth Estimation

Monocular depth estimation aims at inferring three-dimensional (3D) cues from a single RGB image. Although existing methods have achieved a certain degree of success, the impact of redundant information has rarely been studied. We propose to improve estimation accuracy by implicitly eliminating redundant information. To this end, we creatively apply discrete representation to monocular depth estimation. By mapping continuous variables into the corresponding learning-based discrete latent space, a hierarchical multi-scale latent map is acquired as the decoder input. Removing redundant information can enhance prediction performance by making the depth estimator balance the local and global. Furthermore, to fully take advantage of the discrete representation, a lightweight fusion mechanism is introduced to aggregate information in multi-scale feature maps.Experiments on NYU Depth V2 dataset demonstrate that our network is competitive with the state of the arts.

FASFLNet: feature adaptive selection and fusion lightweight network for RGB-D indoor scene parsing.

RGB-D indoor scene parsing is a challenging task in computer vision. Conventional scene-parsing approaches based on manual feature extraction have proved inadequate in this area because indoor scenes are both unordered and complex. This study proposes a feature adaptive selection, and fusion lightweight network (FASFLNet) for RGB-D indoor scene parsing that is both efficient and accurate. The proposed FASFLNet utilizes a lightweight classification network (MobileNetV2), constituting the backbone of the feature extraction. This lightweight backbone model guarantees that FASFLNet is not only highly efficient but also provides good performance in terms of feature extraction. The additional information provided by depth images (specifically, spatial information such as the shape and scale of objects) is used in FASFLNet as supplemental information for feature-level adaptive fusion between the RGB and depth streams. Furthermore, during decoding, the features of different layers are fused from top-bottom and integrated at different layers for final pixel-level classification, resulting in an effect similar to that of pyramid supervision. Experimental results obtained on the NYU V2 and SUN RGB-D datasets indicate that the proposed FASFLNet outperforms existing state-of-the-art models and is both highly efficient and accurate.

SABV-Depth: A biologically inspired deep learning network for monocular depth estimation

Monocular depth estimation makes it possible for machines to perceive the real world. The prediction performance of the depth estimation network based on deep learning will be affected due to the depth of the deep network and the locality of convolution operations. The imitation of the biological visual system and its functional structure is becoming a research hotspot. In this paper, we study the interpretability relationship between the biological visual system and the monocular depth estimation network. By concretizing the attention mechanism in biological vision, we propose a monocular depth estimation network based on the self-attention mechanism, named SABV-Depth, which can improve prediction accuracy. Inspired by the biological visual interaction mechanism, we focus on the information transfer between each module of the network and improve the information retention ability, and enable the network to output a depth map with rich object information and detailed information. Further, a decoder module with an inner-connection is proposed to recover depth maps with sharp edge contours. Our method is experimentally validated on the KITTI dataset and NYU Depth V2 dataset. The results show that compared with other works, the proposed method improves prediction accuracy. Meanwhile, the depth map has more object information and detail information, and a better edge information processing effect.

Monocular Depth Estimation Using a Laplacian Image Pyramid with Local Planar Guidance Layers.

It is important to estimate the exact depth from 2D images, and many studies have been conducted for a long period of time to solve depth estimation problems. Recently, as research on estimating depth from monocular camera images based on deep learning is progressing, research for estimating accurate depths using various techniques is being conducted. However, depth estimation from 2D images has been a problem in predicting the boundary between objects. In this paper, we aim to predict sophisticated depths by emphasizing the precise boundaries between objects. We propose a depth estimation network with encoder-decoder structures using the Laplacian pyramid and local planar guidance method. In the process of upsampling the learned features using the encoder, the purpose of this step is to obtain a clearer depth map by guiding a more sophisticated boundary of an object using the Laplacian pyramid and local planar guidance techniques. We train and test our models with KITTI and NYU Depth V2 datasets. The proposed network constructs a DNN using only convolution and uses the ConvNext networks as a backbone. A trained model shows the performance of the absolute relative error (Abs_rel) 0.054 and root mean square error (RMSE) 2.252 based on the KITTI dataset and absolute relative error (Abs_rel) 0.102 and root mean square error 0.355 based on the NYU Depth V2 dataset. On the state-of-the-art monocular depth estimation, our network performance shows the fifth-best performance based on the KITTI Eigen split and the eighth-best performance based on the NYU Depth V2.

Integrated intracellular organization and its variations in human iPS cells

Understanding how a subset of expressed genes dictates cellular phenotype is a considerable challenge owing to the large numbers of molecules involved, their combinatorics and the plethora of cellular behaviours that they determine1,2. Here we reduced this complexity by focusing on cellular organization—a key readout and driver of cell behaviour3,4—at the level of major cellular structures that represent distinct organelles and functional machines, and generated the WTC-11 hiPSC Single-Cell Image Dataset v1, which contains more than 200,000 live cells in 3D, spanning 25 key cellular structures. The scale and quality of this dataset permitted the creation of a generalizable analysis framework to convert raw image data of cells and their structures into dimensionally reduced, quantitative measurements that can be interpreted by humans, and to facilitate data exploration. This framework embraces the vast cell-to-cell variability that is observed within a normal population, facilitates the integration of cell-by-cell structural data and allows quantitative analyses of distinct, separable aspects of organization within and across different cell populations. We found that the integrated intracellular organization of interphase cells was robust to the wide range of variation in cell shape in the population; that the average locations of some structures became polarized in cells at the edges of colonies while maintaining the ‘wiring’ of their interactions with other structures; and that, by contrast, changes in the location of structures during early mitotic reorganization were accompanied by changes in their wiring.

Modelling and forecasting the effects of increasing sea surface temperature on coral bleaching in the Indo-Pacific region

ABSTRACT The Coral Triangle (CT) and the South China Sea (SCS) are the world’s great tropical seas, located in the Indo-Pacific (IP) region. It is home to the richest marine ecosystem on Earth, with a total of 76% reef-building coral species as well as 37% coral reef fish species. Unfortunately, this sensitive area is now vulnerable to Sea Surface Temperature (SST) warming. This research explored the possible consequences of SST warming on the rich ecosystems of the IP region, specifically on bleaching of its coral reefs. Reefbase provided coral bleaching records together with the daily NOAA AVHRR Optimum Interpolation (OI) SST V2 dataset (OISSTv2) were used to explore the relationship between coral bleaching and SST in the IP region. Three different categories of monthly mean SST were tested as potential covariates: minimum SST, mean SST and maximum SST, obtained from the OISSTv2. The fitted logistic regression (LR) model revealed a significant and large correlation between coral bleaching and annual maximum monthly mean SST in the study area using the bleaching data from an online database and the time-series of AVHRR images. Predicted maps of coral bleaching based on the LR model were highly consistent with NOAA Coral Reef Watch (CRW) Degree heating Weeks (DHW) maps. However, some important discrepancies resulted from the more specific local fitting used in the LR model. The maximum SST was forecasted from 2020 to 2100 based on the Coupled Model Intercomparison Project Phase 5 (CMIP5) dataset under the Representative Concentration Pathways (RCP2.6) scenario. The fitted logistic regression model was employed to transform the forecasted maximum SST values into maps of the probability of coral bleaching from 2020 to 2100. The results provide considerable cause for concern, including the likelihood of widespread coral bleaching in many places in the IP region over the next 30 years.

Image Forgery Detection Using Neural Network In Python

The advancements of technology in every aspect of the current age are leading to the misuse of data. Researchers, therefore, face the challenging task of identifying these manipulated forms of data and distinguishing the real data from the manipulated. Splicing is one of the most common techniques used for digital image tampering; a selected area copied from the same or another image is pasted in an image. Image forgery detection is considered a reliable way to verify the authenticity of digital images. In this study, we proposed an approach based on the state-of-the-art deep learning architecture of ResNet50v2. The proposed model takes image batches as input and utilizes the weights of a YOLO convolutional neural network (CNN) by using the architecture of ResNet50v2. In this study, we used the CASIA_v1 and CASIA_v2 benchmark datasets, which contain two distinct categories, original and forgery, to detect image splicing. We used 80% of the data for the training and the remaining 20% for testing purposes. We also performed a comparative analysis between existing approaches and our proposed system. We evaluated the performance of our technique with the CASIA_v1 and CASIA_v2 datasets. Since the CASIA_v2 dataset is more comprehensive compared to the CASIA_v1 dataset, we obtained 99.3% accuracy for the fine-tuned model using transfer learning and 81% accuracy without transfer learning with the CASIA_v2 dataset. The results show the superiority of the proposed system

Multiple Instance Complementary Detection and Difficulty Evaluation for Weakly Supervised Object Detection in Remote Sensing Images

Weakly supervised object detection (WSOD) in remote sensing images (RSIs) has attracted lots of attention because it solely employs image-level labels to drive the model training. Most of the WSOD methods incline to mine salient object as positive instance, and the less salient objects are considered as negative instances, which will cause the problem of missing instances. In addition, the quantity of hard and easy instances is usually imbalanced, consequently, the cumulative loss of a large amount of easy instances dominates the training loss, which limits the upper bound of WSOD performance. To handle the first problem, a complementary detection network (CDN) is proposed, which consists of a complementary multiple instance detection network (CMIDN) and a complementary feature learning (CFL) module. The CDN can capture robust complementary information from two basic multiple instance detection networks (MIDNs) and mine more object instances. To handle the second problem, an instance difficulty evaluation metric named instance difficulty score (IDS) is proposed, which is employed as the weight of each instance in the training loss. Consequently, the hard instances will be assigned larger weights according to the IDS, which can improve the upper bound of WSOD performance. The ablation experiments demonstrate that our method significantly increases the baseline method by large margins, i.e. 23.6% (10.2%) mAP and 32.4% (13.1%) CorLoc gains on the NWPU VHR-10.v2 (DIOR) dataset. Our method obtains 58.1% (26.7%) mAP and 72.4% (47.9%) CorLoc on the NWPU VHR-10.v2 (DIOR) dataset, which achieves better performance compared with seven advanced WSOD methods.

A Novel Spatio-Temporal-Wise Network for Action Recognition

Action recognition is a challenging task that requires understanding the temporal relationships between frames. However, capturing and processing spatio-temporal and motion features is computationally expensive, making it difficult to apply to practical situations. We propose a novel approach called the Spatio-Temporal-Wise (STW) network to address this problem. The STW network inserts STW blocks, consisting of a Spatio-Temporal Fusion Module and a Temporal-Wise Module, into an existing 2D CNN. This approach requires very little additional computational overhead but brings huge performance improvements in recognizing human actions. The proposed method is evaluated on several public datasets, including Something-Something v1 & v2, Kinetics-400, UCF101, and HMDB51. STW achieved comparable or better performance on these datasets compared to state-of-the-art methods. Notably, the STW network improves recognition accuracy by 26.6% and 34.6% on the Something-Something v1 & v2 datasets, respectively, with less than 2% additional computational overhead. The results demonstrate that the STW network can significantly improve performance in action recognition tasks while requiring only a small additional computational overhead, which represents a promising direction for developing more efficient and effective approaches to handling temporal reasoning in action recognition, which may have important applications in the future.

A Novel Capability of Object Identification and Recognition Based on Integrated mWMM

In the last decade, there has been remarkable progress in the areas of object detection and recognition due to high-quality color images along with their depth maps provided by RGB-D cameras. They enable artificially intelligent machines to easily detect and recognize objects and make real-time decisions according to the given scenarios. Depth cues can improve the quality of object detection and recognition. The main purpose of this research study to find an optimized way of object detection and identification we propose techniques of object detection using two RGB-D datasets. The proposed methodology extracts image normally from depth maps and then performs clustering using the Modified Watson Mixture Model (mWMM). mWMM is challenging to handle when the quality of the image is not good. Hence, the proposed RGB-D-based system uses depth cues for segmentation with the help of mWMM. Then it extracts multiple features from the segmented images. The selected features are fed to the Artificial Neural Network (ANN) and Convolutional Neural Network (CNN) for detecting objects. We achieved 92.13% of mean accuracy over NYUv1 dataset and 90.00% of mean accuracy for the Redweb_v1 dataset. Finally, their results are compared and the proposed model with CNN outperforms other state-of-the-art methods. The proposed architecture can be used in autonomous cars, traffic monitoring, and sports scenes.

Fast Monocular Depth Estimation via Side Prediction Aggregation with Continuous Spatial Refinement

Recent works have validated the benefit of integrating spatial information into deep networks to improve pixel-level prediction tasks such as monocular depth estimation. However, how to efficiently and robustly integrate spatial cues retains as an open problem. In this paper, we introduce the Side Prediction Aggregation (termed SPA) method to enhance the embedding of scene structural information from low-level to high-level layers. To improve the estimation accuracy, the proposed method is further equipped with continuous Spatial Refinement Loss (termed SRL) at multiple resolutions with negligible extra computation. Besides, the proposed sequential network can further perform adversarial learning at multiple resolutions. Such an adversarial refinement strategy greatly improves the accuracy of estimated depth with a little extra computation. Without using any pre-trained models, our network achieves the the-state-of-art accuracy on KITTI, NYUD V2, and Cityscapes datasets, which has achieved real-time depth estimation online.