Irrelevant Images Research Articles

Joint multimodal entity-relation extraction based on temporal enhancement and similarity-gated attention

Joint Multimodal Entity and Relation Extraction (JMERE), which needs to combine complex image information to extract entity-relation quintuples from text sequences, posts higher requirements of the model’s multimodal feature fusion and selection capabilities. With the advancement of large pre-trained language models, existing studies focus on improving the feature alignments between textual and visual modalities. However, there remains a noticeable gap in capturing the temporal information present in textual sequences. In addition, these methods exhibit a certain deficiency in distinguishing irrelevant images when integrating image and text features, making them susceptible to interference from image information unrelated to the text. To address these challenges, we propose a temporally enhanced and similarity-gated attention network (TESGA) for joint multimodal entity relation extraction. Specifically, we first incorporate an LSTM-based Text Temporal Enhancement module to enhance the model’s ability to capture temporal information from the text. Next, we introduce a Text-Image Similarity-Gated Attention mechanism, which controls the degree of incorporating image information based on the consistency between image and text features. Subsequently, We design the entity and relation prediction module using a form-filling approach based on entity and relation types, and conduct prediction of entity-relation quintuples. Notably, apart from the JMERE task, our approach can also be applied to other tasks involving text-visual enhancement, such as Multimodal Named Entity Recognition (MNER) and Multimodal Relation Extraction (MRE). To demonstrate the effectiveness of our approach, our model is extensively experimented on three benchmark datasets and achieves state-of-the-art performance. Our code will be available upon paper acceptance.11https://github.com/vacuum-cup/TESGA.

KLSANet: Key local semantic alignment Network for few-shot image classification

Few-shot image classification involves recognizing new classes with a limited number of labeled samples. Current local descriptor-based methods, while leveraging consistent low-level features across visible and invisible classes, face challenges including redundant adjacent information, irrelevant partial representation, and limited interpretability. This paper proposes KLSANet, a few-shot image classification approach based on key local semantic alignment network, which aligns key local semantics for accurate classification. Furthermore, we introduce a key local screening module to mitigate the influence of semantically irrelevant image parts on classification. KLSANet demonstrates superior performance on three benchmark datasets (CUB, Stanford Dogs, Stanford Cars), outperforming state-of-the-art methods in 1-shot and 5-shot settings with average improvements of 3.95% and 2.56% respectively. Visualization experiments demonstrate the interpretability of KLSANet predictions. Code is available at: https://github.com/ZitZhengWang/KLSANet.

Age-Gating and Marketing Differences Between Storefront and Non-Storefront Cannabis Retailers

Objective: The study investigated whether California storefront and non-storefront cannabis retailers are adhering to online age-gating requirements and whether differences in website marketing practices exist. Methods: Websites of 134 storefront and 115 non-storefront licensed retailers were randomly selected. Bivariate associations were tested between retailer type and website marketing, age-gating methods, and presence of age-gating at various purchase stages. Results: Among the 200 (80.3%) websites with age-gating when entering, 182 (91%) employed an ineffective method where users click either “Yes” or “No” to confirm their age. Moreover, 49 (19.68%) websites lacked age-gating when entering. Amongst those requiring photo identification during checkout (n = 100, 40.16%), 97% allowed users to proceed after uploading an irrelevant image. Significantly more storefront retailers employed combined age-gating at entry, mandatory account registration, and age-gating during checkout than non-storefront retailers (X2 (1, N = 249) = 7.69, p < .01). Retailer websites frequently displayed “clean” labels (n = 200, 80.32%), followed by positive state claims (n = 198, 79.52%), physical health claims (n = 166, 66.67%), and mental health claims (n = 146, 58.63%). Significantly more storefront retailers displayed physical health claims (X2 (1, N = 249) = 7.52, p < .01) and health warnings than non-storefront retailers (X2 (1, N = 249) = 4.13, p = .04). Conclusions: Most cannabis retailers comply with age-gating requirements; however, methods employed are easily circumvented. Youths’ easy and unrestricted access to cannabis retailer websites may increase positive attitudes about cannabis and encourage use.

잠복결핵에 관한 온라인 건강 상담 현황

The study’s objective is to explore the health information needs of latent tuberculosis patients and their communities by analyzing data from the online health counselling platform, Naver Jisik-iN. Initially, 3,261 questions related to ‘latent tuberculosis’ were collected. Following the removal of duplicates and irrelevant image information, the final dataset for analysis comprised 2,198 questions. Text pre-processing, Latent Dirichlet Allocation (LDA) topic modelling, and Long Short-Term Memory (LSTM)-based text summarization model were used. Manual categorization was added to supplement the unsupervised learning process. Seven topics were identified using LDA, from which five specific topics (‘side effects’, ‘treatment’, ‘army’, ‘interaction’, and ‘infectiousness’) were derived. Subsequently, manual classification was conducted based on these five topics. Manual summary and LSTM-based text summarization results were consistent. Numerous individuals sought information about the potential for curing latent tuberculosis and the risk of tuberculosis development. Moreover, questions related to the interpretation of test results and interactions with other substances were widespread. Concerning side effects, issues predominantly revolved around drug discontinuation due to skin problems and elevated liver function tests. The findings reveal the prevalent concerns and inquiries of society regarding latent tuberculosis. The identified topics offer valuable insights into the key aspects of interest related to this condition.

Determination of quality classes for material extrusion additive manufacturing using image processing

Tools for implementing a systematic quality management are necessary for the use of material extrusion as an additive manufacturing process for products with high quality requirements. Well-defined quality classes are crucial for ensuring that the requirements for a product can be communicated transparently and that the existing properties can be evaluated. Furthermore, there is a lack of capable measurement equipment for the acquisition of process data during the production process. To address these challenges, the present paper introduces an image processing system that determines quality indicators for individual layers in terms of imperfect surface percentages and the number of imperfections. The central element of the hardware is an adaptive darkfield illumination, which leads to high-contrast images. In addition, five types of layer subareas are identified in a segmentation step. Unsupervised machine learning methods are then used to detect imperfections in each layer subarea. In the segmentation, the current layer can be distinguished from irrelevant image background regions with an F-measure of 0.981. For the layer-wise measurement of the quality indicators, relative measurement errors with standard deviations of 25 to 76.1% are found. After evaluating the capabilities of the image processing system, a proposal for limits of quality classes is derived by monitoring several material extrusion processes. For this purpose, three quality classes for each of the five layer subareas are deduced from the process scatter measured by the image processing system. The results are an important contribution to the industrialization of material extrusion in safety–critical areas such as medical technology or the aerospace industry.

Realizing Efficient On-Device Language-based Image Retrieval

Advances in deep learning have enabled accurate language-based search and retrieval, e.g., over user photos, in the cloud. Many users prefer to store their photos in the home due to privacy concerns. As such, a need arises for models that can perform cross-modal search on resource-limited devices. State-of-the-art cross-modal retrieval models achieve high accuracy through learning entangled representations that enable fine-grained similarity calculation between a language query and an image, but at the expense of having a prohibitively high retrieval latency. Alternatively, there is a new class of methods that exhibits good performance with low latency, but requires a lot more computational resources, and an order of magnitude more training data (i.e. large web-scraped datasets consisting of millions of image-caption pairs) making them infeasible to use in a commercial context. From a pragmatic perspective, none of the existing methods are suitable for developing commercial applications for low-latency cross-modal retrieval on low-resource devices. We propose CrispSearch, a cascaded approach that greatly reduces the retrieval latency with minimal loss in ranking accuracy for on-device language-based image retrieval. The idea behind our approach is to combine a light-weight and runtime-efficient coarse model with a fine re-ranking stage. Given a language query, the coarse model effectively filters out many of the irrelevant image candidates. After this filtering, only a handful of strong candidates will be selected and sent to a fine model for re-ranking. Extensive experimental results with two SOTA models for the fine re-ranking stage, on standard benchmark datasets show that CrispSearch results in a speedup of up to 38 times over the SOTA fine methods with negligible performance degradation. Moreover, our method does not require millions of training instances, making it a pragmatic solution to on-device search and retrieval.

Robust Training of Social Media Image Classification Models

Images shared on social media help crisis managers gain situational awareness and assess incurred damages, among other response tasks. As the volume and velocity of such content are typically high, real-time image classification has become an urgent need for faster disaster response. Recent advances in computer vision and deep neural networks have enabled the development of models for image classification for a number of tasks, including detecting crisis incidents, filtering irrelevant images, classifying images into specific humanitarian categories, and assessing the severity of the damage. To develop robust models, it is necessary to understand the capability of the publicly available pretrained models for these tasks, which remains to be underexplored in the crisis informatics literature. In this study, we address such limitations by investigating ten different network architectures for four different tasks using the largest publicly available datasets for these tasks. We also explore various data augmentation strategies, semisupervised techniques, and a multitask learning setup. In our extensive experiments, we achieve promising results.

Pericardial Effusion Detection on Post-Mortem Computed Tomography Images Using Convolutional Neural Networks.

Pericardial effusion can be a sign of significant underlying diease and, in some cases, may lead to death. Post-mortem computed tomography (PMCT) is a well-established tool to assist death investigation processes in the forensic setting. In practice, the scarcity of well-trained radiologists is a challenge in processing raw whole-body PMCT images for pericardial effusion detection. In this work, we propose a Pericardial Effusion Automatic Detection (PEAD) framework to automatically process raw whole-body PMCT images to filter out the irrelevant images with heart organ absent and focus on pericardial effusion detection. In PEAD, the standard convolutional neural network architectures of VGG and ResNet are carefully modified to fit the specific characteristics of PMCT images. The experimental results prove the effectiveness of the proposed framework and modified models. The modified VGG and ResNet models achieved superior detection accuracy than the standard architecture with reduced processing speed.

Deep-DM: Deep-driven deformable model for 3D image segmentation using limited data.

Objective - Medical image segmentation is essential for several clinical tasks, including diagnosis, surgical and treatment planning, and image-guided interventions. Deep Learning (DL) methods have become the state-of-the-art for several image segmentation scenarios. However, a large and well-annotated dataset is required to effectively train a DL model, which is usually difficult to obtain in clinical practice, especially for 3D images. Methods - In this paper, we proposed Deep-DM, a learning-guided deformable model framework for 3D medical imaging segmentation using limited training data. In the proposed method, an energy function is learned by a Convolutional Neural Network (CNN) and integrated into an explicit deformable model to drive the evolution of an initial surface towards the object to segment. Specifically, the learning-based energy function is iteratively retrieved from localized anatomical representations of the image containing the image information around the evolving surface at each iteration. By focusing on localized regions of interest, this representation excludes irrelevant image information, facilitating the learning process. Results and conclusion - The performance of the proposed method is demonstrated for the tasks of left ventricle and fetal head segmentation in ultrasound, left atrium segmentation in Magnetic Resonance, and bladder segmentation in Computed Tomography, using different numbers of training volumes in each study. The results obtained showed the feasibility of the proposed method to segment different anatomical structures in different imaging modalities. Moreover, the results also showed that the proposed approach is less dependent on the size of the training dataset in comparison with state-of-the-art DL-based segmentation methods, outperforming them for all tasks when a low number of samples is available. Significance - Overall, by offering a more robust and less data-intensive approach to accurately segmenting anatomical structures, the proposed method has the potential to enhance clinical tasks that require image segmentation strategies.

Statistical similarity matching and filtering for clinical image retrieval by machine learning approach

Clinical image retrieval plays a pivotal role in modern healthcare for diagnostics and research, but prior research has grappled with the challenge of achieving high accuracy due to limited filtering techniques. The proposed method includes statistical distance measurements for similarity comparison and a machine learning technique for image filtering. Throughout this framework, the search area for similarity matching is reduced by first filtering away irrelevant images using the probabilistic outcomes of the Support Vector Machine (SVM) classification as class predictions of search and database images. Resizing is done as part of the preprocessing. Then, using Principal Component Analysis (PCA), the preprocessed data’s textural features, visual characteristics, and low-level features are extracted. The study also suggested an adaptive similarity matching method centered on a linear integration of feature-level similarities on the individual-level level. The precision and ranking order details of the most appropriate images retrieved and predicted by SVMs are considered when calculating the feature weights. The system continually alters weights for every distinctive search to generate beneficial outcomes. The supervised and unsupervised learning strategies are studied to link low-level global image features in the generated PCA-based Eigen Space using their high-level semantic and visual classifications to reduce the semantic gap and enhance retrieval effectiveness. The ground-truth database used in experiments has 1594 unique medical images with 3 different databases. Our method significantly improves the precision and recall rates in image retrieval tasks by combining sophisticated feature extraction, data-driven algorithms, and deep learning models. Research obtained an impressive accuracy of 0.99, demonstrating the effectiveness of our approach. This novel methodology addresses the limitations of prior research and provides a robust and reliable solution for clinicians and researchers in the medical field seeking to access and analyze relevant clinical images.

Enhancing Autofocus in Non-Mydriatic Fundus Photography: A Fast and Robust Approach with Adaptive Window and Path-Optimized Search

Non-mydriatic fundus photography (NMFP) plays a vital role in diagnosing eye diseases, with its performance primarily dependent on the autofocus process. However, even minor maloperations or eye micro-movements can compromise fundus imaging quality, leading to autofocus inaccuracy and a heightened risk of misdiagnosis. To enhance the autofocus performance in NMFP, a fast and robust fundus autofocus method with adaptive window and path-optimized search is proposed. In this method, the adaptive focus window is used to suppress irrelevant image contents and correct the sharpness curve, and the path-optimized search is constructed to overcome the curve’s local extrema, in order to achieve rapid focus position convergence. This method was simulated and clinically studied with the self-developed autofocus system for NMFP. The results of 80 cases of human eye imaging show that, compared with similar autofocus methods, this method achieves a focus success rate of 90% with the least axial scanning, and can adapt to non-ideal imaging conditions such as pupil misalignment, eyelash occlusion, and nystagmus.

Virtually Possible: Enhancing Quality Control of 3D-Printed Medicines with Machine Vision Trained on Photorealistic Images.

Three-dimensional (3D) printing is an advanced pharmaceutical manufacturing technology, and concerted efforts are underway to establish its applicability to various industries. However, for any technology to achieve widespread adoption, robustness and reliability are critical factors. Machine vision (MV), a subset of artificial intelligence (AI), has emerged as a powerful tool to replace human inspection with unprecedented speed and accuracy. Previous studies have demonstrated the potential of MV in pharmaceutical processes. However, training models using real images proves to be both costly and time consuming. In this study, we present an alternative approach, where synthetic images were used to train models to classify the quality of dosage forms. We generated 200 photorealistic virtual images that replicated 3D-printed dosage forms, where seven machine learning techniques (MLTs) were used to perform image classification. By exploring various MV pipelines, including image resizing and transformation, we achieved remarkable classification accuracies of 80.8%, 74.3%, and 75.5% for capsules, tablets, and films, respectively, for classifying stereolithography (SLA)-printed dosage forms. Additionally, we subjected the MLTs to rigorous stress tests, evaluating their scalability to classify over 3000 images and their ability to handle irrelevant images, where accuracies of 66.5% (capsules), 72.0% (tablets), and 70.9% (films) were obtained. Moreover, model confidence was also measured, and Brier scores ranged from 0.20 to 0.40. Our results demonstrate promising proof of concept that virtual images exhibit great potential for image classification of SLA-printed dosage forms. By using photorealistic virtual images, which are faster and cheaper to generate, we pave the way for accelerated, reliable, and sustainable AI model development to enhance the quality control of 3D-printed medicines.

Sequence-wise multimodal biometric fingerprint and finger-vein recognition network (STMFPFV-Net)

The existing multimodal biometric fingerprint and vein deep learning features were found effective for biometric recognition. However, the current performance of the deep learning features is limited due to missing temporal image dependence and the extracted features can obfuscate some vital information because of irrelevant image features. This work proposes a sequence of filtered spatial and temporal multimodal fingerprint and finger vein network (FS-STMFPFV-Net). The overall proposed FS-STMFPFV-Net is achieved from two-channel independent learning to improve image variabilities. In the first channel, the image sequence is generated by aligning the fingerprint and finger vein images together inside the image generator. The sequences are built into the five layers of a deep convolution neural network fusion model to extract spatial sequence-wise features. The second channel is where the extracted sequence-wise features are remembered in the long short-term memory for interactions between temporal and sequence dimensions which finally generate their long-term temporal dependencies as complementary information. These sequences are fused together, and the discriminative sequence features are selected using feature selection. We have presented the ReliefFS feature selection which serve as a complementary basis for selecting compact features from the extracted CNN-based features. To evaluate the performance of the proposed FS-STMFPFV-Net, the NUPT-FPV, FVC-2002-DBs, and the CASIA dataset were used, which provides fingerprint, finger-vein, and palmprint image databases, for experimental validation. The proposed FS-STMFPFV-Net when evaluated using standard protocols offers more than 97% accuracy across the different databases and is ten times more computationally friendly than the existing algorithms.

Focalizing regions of biomarker relevance facilitates biomarker prediction on histopathological images

Image-based AI has thrived as a potentially revolutionary tool for predicting molecular biomarker statuses, which aids in categorizing patients for appropriate medical treatments. However, many methods using hematoxylin and eosin-stained (H&E) whole-slide images (WSIs) have been found to be inefficient because of the presence of numerous uninformative or irrelevant image patches. In this study, we introduced the region of biomarker relevance (ROB) concept to identify the morphological areas most closely associated with biomarkers for accurate status prediction. We actualized this concept within a framework called saliency ROB search (SRS) to enable efficient and effective predictions. By evaluating various lung adenocarcinoma (LUAD) biomarkers, we showcased the superior performance of SRS compared to current state-of-the-art AI approaches. These findings suggest that AI tools, built on the ROB concept, can achieve enhanced molecular biomarker prediction accuracy from pathological images.

Deep learning for few-shot white blood cell image classification and feature learning

ABSTRACT Differential counting of white blood cells (WBCs) in bone marrow using artificial intelligence (AI)-based models, such as convolutional neural network (CNN) and its various variants, can help physicians to efficiently diagnose many critical diseases such as leukaemia, AIDS and cancers. In this work, we implement a deep transfer learning to several CNN models to examine their effectiveness on automatically classifying WBCs into lymphocytes and non-lymphocytes groups. Our results show that transfer learning can enhance the training of the model and improve the model performance. We also discover that using image masking to remove irrelevant image pixels can further increase the accuracy of the model predictions. Moreover, we assess the impact of three data augmentation techniques to address the imbalance in the data set, which commonly occurs in many biological applications. Our results show that all the three examined data augmentation methods improve the classification results on both training and testing data sets. Altogether, we demonstrate that deep neural networks, when combined with transfer learning and imaging processing techniques, can serve as a powerful tool to conduct automatic differential counting of WBCs, and thus facilitate the diagnosis of the WBC-related disorders, monitor the disease progression and improve the effectiveness of therapeutics.

A Review of Seductive Details in Educational Activities Using Electronic Slide Presentations: Derailing the Learning Process.

This review article examines the use of seductive details in nursing education activities. Seductive details are irrelevant images that are placed into educational material. Although seductive details may be found in any type of educational material and at any academic level, this article focuses on the placement of seductive details in electronic slide presentations used in nursing education activities. Educators optimistically place seductive details in educational material to engage learners or provide a momentary break from the presentation. Although engaging the learner is important, educators must use methods that do not derail how the learner's brain remembers information. Added visuals should be relevant to the material needed to be understood. Using seductive details may adversely affect learning by overloading the working memory with extraneous cognitive information, affecting the ability of the learner to create a coherent mental model needed to store a long-term memory. Evidence-based information about seductive detail use is nonexistent in nursing education literature. Professional nurse educators must carefully consider the use of seductive details in electronic slide presentations. [J Contin Educ Nurs. 2023;54(6):261-267.].

Visual Description Augmented Integration Network for Multimodal Entity and Relation Extraction

Multimodal Named Entity Recognition (MNER) and multimodal Relationship Extraction (MRE) play an important role in processing multimodal data and understanding entity relationships across textual and visual domains. However, irrelevant image information may introduce noise that misleads the recognition of information. Additionally, visual and semantic features originate from different modalities, and modal disparity hinders semantic alignment. Therefore, this paper proposes the Visual Description Augmentation Integration Network (VDAIN), which introduces an image description generation technique that allows semantic features generated from image descriptions to be presented in the same modality as the semantic features of textual information. This not only reduces the modal gap but also captures more accurately the high-level semantic information and underlying visual structure in the images. To filter out the modal noise, we use VDAIN to adaptively fuse visual features, semantic features of image descriptions, and textual information, thus eliminating irrelevant modal noise. The F1 score of the proposed model in this paper reaches 75.8% and 87.78% for the MNER task and 82.54% for the MRE task on the three public data sets, respectively, which are significantly better than the baseline model. The experimental results demonstrate the effectiveness of the proposed method in solving the modal noise and modal gap problems.

Analysis: Flawed Datasets of Monkeypox Skin Images

The self-proclaimed first publicly available dataset of Monkeypox skin images consists of medically irrelevant images extracted from Google and photography repositories through a process denominated web-scrapping. Yet, this did not stop other researchers from employing it to build Machine Learning (ML) solutions aimed at computer-aided diagnosis of Monkeypox and other viral infections presenting skin lesions. Neither did it stop the reviewers or editors from publishing these subsequent works in peer-reviewed journals. Several of these works claimed extraordinary performance in the classification of Monkeypox, Chickenpox and Measles, employing ML and the aforementioned dataset. In this work, we analyse the initiator work that has catalysed the development of several ML solutions, and whose popularity is continuing to grow. Further, we provide a rebuttal experiment that showcases the risks of such methodologies, proving that the ML solutions do not necessarily obtain their performance from the features relevant to the diseases at issue.

Material and textural features-based retrieval using feature similarity mapping for synthetic images

Content Based image retrieval (CBIR) is one of the prominent application of image processing where image retrieval from the database is made automatic based on the content in it. The materialist details of the images contains high degree of information and can serve as important toll for analysis. Different variant of content based image retrieval work by locating an object in the image and then retrieving images with similar object in it. The method works using different similarity assessment techniques. In this research work, we have developed a new material feature and HMMD space CBIR technique which works by transforming the domain to HMMD space and then finding the material and Texture Features along with other relevant features. These features forms he feature vector and are then deployed to assess the similarity of the two images which then generates a relevant and irrelevant image search. The system has higher accuracy of assessment as verified by experimentation for precision, productivity and precision agriculture.

Federated Learning-Based Analysis of Human Sentiments and Physical Activities in Natural Disasters

In federated learning (FL), in addition to the training and speculating capacities of the global and local models, an appropriately annotated dataset is equally crucial. These datasets rely on annotation procedures that are error prone and laborious, which require personal inspection for training the overall dataset. In this study, we evaluate the effect of unlabeled data supplied by every participating node in active learning (AL) on the FL. We propose an AL-empowered FL paradigm that combines two application scenarios and assesses different AL techniques. We demonstrate the efficacy of AL by attaining equivalent performance in both centralized and FL with well-annotated data, utilizing limited data images with reduced human assistance during the annotation of the training sets. We establish that the proposed method is independent of the datasets and applications by assessing it using two distinct datasets and applications, human sentiments and human physical activities during natural disasters. We achieved viable results on both application domains that were relatively comparable to the optimal case, in which every data image was manually annotated and assessed (criterion 1). Consequently, a significant improvement of 5.5–6.7% was achieved using the active learning approaches on the training sets of the two datasets, which contained irrelevant images.