Image Search Research Articles

Protecting Drawings at Production and Regime Facilities Using Neural Network Technology and DLP Systems

The existing methods of object image processing are analyzed. Problems of using the considered methods within the framework of DLP systems are considered. A new object image processing method was presented that allows processing of complex images. A metric of accuracy and completeness of plagiarism image detection was used to evaluate the quality of the developed method. Testing was carried out using image ranking to analyze the ability of the model to search for semantics. Comparative testing was carried out with the method based on raster neural models. The advantages and disadvantages of the developed method were highlighted, as well as options for further developmentAs a result of this work, a method for processing object images in native format has been developed. The main advantage of the developed method is the high rate of accuracy (87 %) and completeness (100 %). This study can be useful for further research in the field of vector image analysis. Also, the developed method can be applied as a tool similarly to raster methods of image processing (image search, classification, search for objects in the image).

Investigating What Factors Influence Users’ Rating of Harmful Algorithmic Bias and Discrimination

There has been growing recognition of the crucial role users, especially those from marginalized groups, play in uncovering harmful algorithmic biases. However, it remains unclear how users’ identities and experiences might impact their rating of harmful biases. We present an online experiment (N=2,197) examining these factors: demographics, discrimination experiences, and social and technical knowledge. Participants were shown examples of image search results, including ones that previous literature has identified as biased against marginalized racial, gender, or sexual orientation groups. We found participants from marginalized gender or sexual orientation groups were more likely to rate the examples as more severely harmful. Belonging to marginalized races did not have a similar pattern. Additional factors affecting users’ ratings included discrimination experiences, and having friends or family belonging to marginalized demographics. A qualitative analysis offers insights into users' bias recognition, and why they see biases the way they do. We provide guidance for designing future methods to support effective user-driven auditing.

A framework for automatically generating composite keywords for geo-tagged street images

Due to the ubiquity of sensor-equipped cameras such as smartphones, images are associated with spatial metadata, including camera’s geographical location and viewing orientation, which can be used for automatically generating better semantic keywords about geo-tagged urban street images in addition to visual keywords extracted from image analysis. This study introduces a novel framework for auto-tagging images that integrates both spatial and visual properties to generate comprehensive and accurate tags. The framework operates through four phases: extraction, abstraction, composition, and assessment. Our research highlights the benefits of combining visual and spatial analyses, demonstrated through a case study using geo-tagged urban street images from Orlando, Pittsburgh, and Manhattan. Experimental results show that the proposed framework significantly enhances the accuracy of keyword-based searches compared to conventional methods. In particular, based on our experiments, image search using the tags generated by our proposed framework, referred to as descriptive tags, achieved an average precision improvement factor of 0.9 compared to conventional tags. Additionally, our proposed ranking algorithm, which extends the term frequency-inverse document frequency (TF-IDF) algorithm, resulted in improvement factors of 0.86 for mean average precision (MAP) and 0.57 for mean reciprocal rank (MRR). Moreover, our framework’s flexibility and robustness make it suitable for diverse applications, from smart cities to online shopping. The paper also includes a detailed evaluation and user study, confirming the precision and reliability of the generated tags.

Influence of Prior Imaging Review on Recommendations for Additional Diagnostic Testing: Retrospective Analysis of Imaging Reports in a Chiropractic Radiology Practice

ObjectiveThe purpose of this study was to assess the influence of prior imaging review on recommendations for additional diagnostic testing in an academic chiropractic radiology practice. A secondary aim was to explore the influence of prior imaging review on radiographic interpretation. MethodsWe retrospectively reviewed radiology reports generated from July 18, 2022, to July 18, 2023, from the Palmer College of Chiropractic main campus (Davenport, Iowa) clinic system. Imaging interpretation included an automated search for prior images in an internal picture archival and communication system (PACS). Images from regional health system databases were available and sought by radiologists when (1) unclear radiologic findings had potential clinical implications or (2) prior imaging could clarify potential problems detected in a clinical history. Data were abstracted to a secure adaptive electronic questionnaire and analyzed descriptively. ResultsWe reviewed 1712 radiographic and 165 musculoskeletal diagnostic ultrasound reports for 1552 unique individuals (811 [52.3%] females and 741 [47.7%] males) with a mean age of 42.1 years (range, 2-93 years). Prior imaging was described in 417 (22.2%) reports; 246 (58.9%) indicated images from internal PACS, 192 (46.0%) indicated images from external PACS, and 21 noted both internal and external PACS. Prior imaging findings were credited with answering a clinical question in 98 (23.5%), and a radiographic question in 228 (54.7%) of 417 reports. The process negated the need for follow-up diagnostic testing recommendations in 119 (28.5%) instances, leading to additional imaging recommendations in 19 (4.6%). ConclusionData obtained in this study suggest that comparing current and previous imaging may help reduce unnecessary additional imaging or follow-up diagnostic testing recommendations. Prior imaging review may also facilitate diagnostic confidence and interpretation clarity.

Media Literacy and Fact-Checking as Proactive and Reactive Responses to Misinformation in Kenya and Senegal

ABSTRACT This study examines how media professionals in Kenya and Senegal respond to misinformation using fact-checking and media literacy strategies and how they perceive these strategies for mitigating the spread of misinformation. We conducted 42 semi-structured in-depth interviews with people belonging to one of the following three groups: individuals employed in the news media such as journalists, reporters and editors; individuals working for either independent fact-checking organisations or doing fact-checking within the newsrooms of online and offline media; and individuals engaged in media policymaking. Findings indicate that media professionals in both Senegal and Kenya employ fact-checking strategies such as cross-checking information from primary, secondary and expert sources and verifying images and videos using reverse image search as reactive strategies. Consistent with inoculation theory, participants also promote media literacy as a proactive strategy to equip media producers and consumers with the knowledge and skills to critically engage with media content. Lastly, journalists, fact-checkers and policymakers lauded fact-checking and media literacy as effective strategies for mitigating the spread of misinformation, suggesting that both strategies should be used simultaneously to empower both media consumers and producers against misinformation.

A Portable Linux-based Firmware for NVMe Computational Storage Devices

Abstract: Over the years, interest in computational storage devices has been growing steadily. This is largely due to the rise of data-intensive applications, such as machine learning, online video distribution, astrophysics, and genomics. Moving compute operations closer to the data provides benefits in terms of scaling possibilities and energy efficiency. The development of computational storage devices has been limited by the need for specialized and complex hardware. In this work we propose a portable Linux-based firmware framework for the development of NVMe computational storage devices. Our firmware runs on a variety of hardware platforms ranging from expensive FPGA solutions to inexpensive off-the-shelf single board computers. The firmware leverages the vast Linux software ecosystem to facilitate the development and prototyping of novel computational storage devices. We benchmark our firmware on multiple hardware platforms and demonstrate its versatility through several computational examples including a content-aware disk image search engine based on natural language processing and AI-driven image recognition.

SMedIR: secure medical image retrieval framework with ConvNeXt-based indexing and searchable encryption in the cloud

The security and privacy of medical images are crucial due to their sensitive nature and the potential for severe consequences from unauthorized modifications, including data breaches and inaccurate diagnoses. This paper introduces a method for lossless medical image retrieval from encrypted images stored on third-party clouds. The proposed approach employs a symmetric integrity-centric image encryption scheme, leveraging multiple chaotic maps and cryptographic hash techniques, to ensure lossless image reconstruction. Medical images are first encrypted by the image owners and converted into hashcodes encapsulating essential features using a deep hashing technique with the ConvNeXt network as the backbone in parallel. To ensure index privacy, these hashcodes are encrypted in a searchable manner. The encrypted medical images, along with a secure index, are subsequently uploaded to cloud storage. Authorized medical image users can request similar medical images for diagnostic purposes by submitting a query image, from which a search trapdoor is generated and sent to the cloud. The retrieval process involves a secure similar image search over the encrypted indexes, followed by decryption along with integrity verification of the retrieved images. The proposed method has been rigorously tested on three standard medical datasets, demonstrating an improvement of 5-20% in retrieval accuracy compared to standard baselines. Formal security analysis and experimental results indicate that the proposed scheme offers enhanced security and retrieval accuracy, making it an effective solution for the encrypted storage and secure retrieval of medical image data.

Backward induction-based deep image search.

Conditional image retrieval (CIR), which involves retrieving images by a query image along with user-specified conditions, is essential in computer vision research for efficient image search and automated image analysis. The existing approaches, such as composed image retrieval (CoIR) methods, have been actively studied. However, these methods face challenges as they require either a triplet dataset or richly annotated image-text pairs, which are expensive to obtain. In this work, we demonstrate that CIR at the image-level concept can be achieved using an inverse mapping approach that explores the model's inductive knowledge. Our proposed CIR method, called Backward Search, updates the query embedding to conform to the condition. Specifically, the embedding of the query image is updated by predicting the probability of the label and minimizing the difference from the condition label. This enables CIR with image-level concepts while preserving the context of the query. In this paper, we introduce the Backward Search method that enables single and multi-conditional image retrieval. Moreover, we efficiently reduce the computation time by distilling the knowledge. We conduct experiments using the WikiArt, aPY, and CUB benchmark datasets. The proposed method achieves an average mAP@10 of 0.541 on the datasets, demonstrating a marked improvement compared to the CoIR methods in our comparative experiments. Furthermore, by employing knowledge distillation with the Backward Search model as the teacher, the student model achieves a significant reduction in computation time, up to 160 times faster with only a slight decrease in performance. The implementation of our method is available at the following URL: https://github.com/dhlee-work/BackwardSearch.

HAIGEN: Towards Human-AI Collaboration for Facilitating Creativity and Style Generation in Fashion Design

The process of fashion design usually involves sketching, refining, and coloring, with designers drawing inspiration from various images to fuel their creative endeavors. However, conventional image search methods often yield irrelevant results, impeding the design process. Moreover, creating and coloring sketches can be time-consuming and demanding, acting as a bottleneck in the design workflow. In this work, we introduce HAIGEN (Human-AI Collaboration for GENeration), an efficient fashion design system for Human-AI collaboration developed to aid designers. Specifically, HAIGEN consists of four modules. T2IM, located in the cloud, generates reference inspiration images directly from text prompts. With three other modules situated locally, the I2SM batch generates the image material library into a certain designer-style sketch material library. The SRM recommends similar sketches in the generated library to designers for further refinement, and the STM colors the refined sketch according to the styles of inspiration images. Through our system, any designer can perform local personalized fine-tuning and leverage the powerful generation capabilities of large models in the cloud, streamlining the entire design development process. Given that our approach integrates both cloud and local model deployment schemes, it effectively safeguards design privacy by avoiding the need to upload personalized data from local designers. We validated the effectiveness of each module through extensive qualitative and quantitative experiments. User surveys also confirmed that HAIGEN offers significant advantages in design efficiency, positioning it as a new generation of aid-tool for designers.

Cross-modality sub-image retrieval using contrastive multimodal image representations

In tissue characterization and cancer diagnostics, multimodal imaging has emerged as a powerful technique. Thanks to computational advances, large datasets can be exploited to discover patterns in pathologies and improve diagnosis. However, this requires efficient and scalable image retrieval methods. Cross-modality image retrieval is particularly challenging, since images of similar (or even the same) content captured by different modalities might share few common structures. We propose a new application-independent content-based image retrieval (CBIR) system for reverse (sub-)image search across modalities, which combines deep learning to generate representations (embedding the different modalities in a common space) with robust feature extraction and bag-of-words models for efficient and reliable retrieval. We illustrate its advantages through a replacement study, exploring a number of feature extractors and learned representations, as well as through comparison to recent (cross-modality) CBIR methods. For the task of (sub-)image retrieval on a (publicly available) dataset of brightfield and second harmonic generation microscopy images, the results show that our approach is superior to all tested alternatives. We discuss the shortcomings of the compared methods and observe the importance of equivariance and invariance properties of the learned representations and feature extractors in the CBIR pipeline. Code is available at: https://github.com/MIDA-group/CrossModal_ImgRetrieval.

Proyectar la imaginación. La influencia de las imágenes en red en la fase de prediseño en comunicación gráfica

Graphic communication projects begin with a strategic phase whose objective is to conceptualize a sensory universe that will be associated with the product or brand for which the project is developed. The objective is to emotionally connect the brand or product with the user. In this process, the "performativity" potential of images is used to define and communicate complex emotional qualities that are difficult to express through verbal language. Mostly, images are searched on the Internet and classified through hegemonic iconographic and iconological patterns. This article proposes to activate a critical look at the hybrid co-creation process that is generated between human and machine in the search for images on the net. Analyzing the agency of the non-human in this procedure will allow us to establish if the creative and methodological processes used contribute to establish and perpetuate a prevailing visual hegemony, or if they help to open fissures in its structure.

Prefabricated Envelope Green Remodeling Potential of Public Office Buildings in Korea

The public sector should reduce energy consumption and carbon emissions from the building stock, thereby serving as a role model for the private sector. In Korea, public buildings are leading the green remodeling business initiative as part of a carbon-neutral strategy. Building envelope retrofitting is essential for the green remodeling of existing buildings because it significantly affects the buildings’ aesthetic appearance, occupant comfort, and energy usage. From the perspectives of constructability and cost, prefabricated envelope systems offer various advantages and can contribute to the growth of the green remodeling business. To develop an effective prefabricated envelope system, a thorough analysis of the existing building stock must be conducted. Therefore, this study aims to investigate existing public office buildings in Korea to obtain a better understanding of the considerations necessary for developing prefabricated envelope systems. The survey utilized the image search and road-view functions of map services, following an appropriate sample design. Based on the survey results, the characteristics of the building types and envelopes, as well as considerations for developing prefabricated envelope systems are discussed. Furthermore, this study quantitatively analyzed the energy conservation potential through building energy simulations.

Multiple Eye Diseases Detection using Convolutional Neural Network

Many eye cases that will arise in the next few years will require early diagnosis for rapid intervention. Polyophthalmia diagnostic methods such as physical examination, examination and diagnosis will be limited to medical and professional methods. Therefore, automated processes are needed. There have been some studies on computer-aided diagnosis (CAD) of polyocular disease using tools such as experts, but these are limited to their knowledge base and therefore not accurate. Early diagnosis of polyophthalmia allows rapid intervention and treatment. This application uses convolutional neural networks for computer-aided diagnosis of various eye diseases and can be used by a common person outside the clinic. This algorithm was trained using transformation learning on a dataset of 100 poliocular disease images from Google image searches for “normal human poliocular disease” and “human poliocular disease.” It leverages the ImageNet model built using a convolutional neural network classifier and transforms its knowledge using transfer learning to train a new model. The new model can classify polyocular disease images into “normal” and “diseases such as bulging polyophthalmia, glaucoma, uveitis, strabismus, dry polyophthalmia, and color blindness.” The system is designed to use deep convolutional neural networks to take images as input and classify the images such as human polyocular disease and “human polyocular disease.” Multiple eye disease detection system using a neural network developed by VS Code. Here we are using python 3.8 as frontend and Mysql Server as backend.

Stylistic and architectural art trends of worldwide experience in designing orthodox churches and complexes

Currently, in the scientific and practical space of Ukraine, there is no substantiated and systematized concept of the sacred architecture of the Balkan countries and the countries of the near abroad. Many countries have experienced a number of important changes and transformations over the last fifty years, which undoubtedly influenced the development of sacred architecture. In this study, the author proposes to consider the modern development of Orthodox churches and complexes of the specified countries of world Orthodoxy. In order to solve the tasks, literary sources were analyzed, highlighting the peculiarities of the analysis of sacred architecture. The fundamental researchers of the analysis of modern architecture include: C. Jenks, O. E. Gutnov, V. L. Hight, A. V. Ikonnikov and others. Modern researchers in the field of design and analysis of sacred objects include the works of O. S. Sleptsova, Yu. I. Kryvoruchko, M. Yu. Kesler. Taking into account the diversity and multifacetedness of the studied sacred objects, this analysis helped establish the directions and criteria for the analysis of Orthodox churches. The article presents the results of the analysis of the practice of sacred design of temple buildings of the countries that are included in the list of Orthodox autocephalous churches of the World Orthodoxy, in particular the temple architecture of the countries: Poland, Serbia, Albania, Montenegro, Bosnia and Herzegovina, Romania, Bulgaria, Greece, Macedonia, Cyprus, Georgia, Armenia, Russia, Belarus. Most of the general finishing has close ties with historical prototypes and stylistic polyphony of development, primarily Byzantine architecture, South Balkan and other architectural schools of a certain region. Also, isolated cases of modern and innovative searches for the modern image of the national temple have been established. As illustrative material, the article is supplemented with images with an analysis of the influence of historical prototypes on modern samples, as well as a table indicating general trends.

Finding Medical Photographs of Patients Online: Randomized, Cross-Sectional Study.

Photographs from medical case reports published in academic journals have previously been found in online image search results. This means that patient photographs circulate beyond the original journal website and can be freely accessed online. While this raises ethical and legal concerns, no systematic study has documented how often this occurs. The aim of this cross-sectional study was to provide systematic evidence that patient photographs from case reports published in medical journals appear in Google Images search results. Research questions included the following: (1) what percentage of patient medical photographs published in case reports were found in Google Images search results? (2) what was the relationship between open access publication status and image availability? and (3) did the odds of finding patient photographs on third-party websites differ between searches conducted in 2020 and 2022? The main outcome measure assessed whether at least 1 photograph from each case report was found on Google Images when using a structured search. Secondary outcome variables included the image source and the availability of images on third-party websites over time. The characteristics of medical images were described using summary statistics. The association between the source of full-text availability and image availability on Google Images was tested using logistic regressions. Finally, we examined the trend of finding patient photographs using generalized estimating equations. From a random sample of 585 case reports indexed in PubMed, 186 contained patient photographs, for a total of 598 distinct images. For 142 (76.3%) out of 186 case reports, at least 1 photograph was found in Google Images search results. A total of 18.3% (110/598) of photographs included eye, face, or full body, including 10.9% (65/598) that could potentially identify the patient. The odds of finding an image from the case report online were higher if the full-text paper was available on ResearchGate (odds ratio [OR] 9.16, 95% CI 2.71-31.02), PubMed Central (OR 7.90, 95% CI 2.33-26.77), or Google Scholar (OR 6.07, 95% CI 2.77-13.29) than if the full-text was available solely through an open access journal (OR 5.33, 95% CI 2.31-12.28). However, all factors contributed to an increased risk of locating patient images online. Compared with the search in 2020, patient photographs were less likely to be found on third-party websites based on the 2022 search results (OR 0.61, 95% Cl 0.43-0.87). A high proportion of medical photographs from case reports was found on Google Images, raising ethical concerns with policy and practice implications. Journal publishers and corporations such as Google are best positioned to develop an effective remedy. Until then, it is crucial that patients are adequately informed about the potential risks and benefits of providing consent for clinicians to publish their images in medical journals.

Comparing point‐wise and pair‐wise relevance judgment with brain signals

AbstractHow to collect relevance judgment has long been an important problem in Information Retrieval (IR). A popular method is to collect relevance judgment in a point‐wise manner, in which assessors examine and give an absolute relevance score for each item independently of the others. As an alternative, pair‐wise relevance judgment, also named preference judgment, allows an assessor to compare two items side‐by‐side and express their preference for one over the other. Previous work has explored the differences between these two paradigms of relevance judgments from many different aspects. Most of these works are conducted through explicit/implicit feedback. However, few works investigate the underlying neurological mechanisms of the two paradigms. In this paper, we conduct a lab study to investigate and compare point‐wise and pair‐wise relevance judgment in image search scenarios. We study the neurological mechanisms of the two paradigms through an event‐related potential (ERP) analysis of the users' brain signals while viewing images during a search process. We have obtained several observations, such as search engine users tend to pay more attention to preferred items in the point‐wise paradigm but unpreferred items in the pair‐wise paradigm. Furthermore, we test the adoption of brain signals as implicit feedback for predicting pair‐wise relevance judgment, highlighting the feasibility of leveraging brain signals to understand users' relevance judgments.

Privacy-preserving geo-tagged image search in edge–cloud computing for IoT

The Internet of Things (IoT) generates a significant volume of geo-tagged images via surveillance sensors in edge–cloud computing environments. Image search is essential to facilitate information sharing, data analysis, and strategic decision-making. However, outsourced images are typically encrypted for privacy protection, posing a challenge in simultaneously searching for visual and geographical relevance on encrypted images. To address this, this paper proposes an edge intelligence empowered privacy-preserving top-k geo-tagged image search scheme for IoT in edge–cloud computing. The scheme presents a novel single-to-multi searchable encryption method for geo-tagged images that enables multiple users to perform secure nearest neighbor queries on a data source. Additionally, an extended anchor-based position determination method and an inner product-based distance calculation method are designed to enable geo-tagged image similarity calculation on ciphertext. Finally, a secure pruning method is introduced to improve query performance. Experiments are conducted to verify the performance of the scheme in terms of high efficiency and accuracy of the search.

Viral peer advice

Abstract By examining a collection of health-related memes, I propose ways to categorize the illocutionary force of memes functioning as the speech act of advice. 200 English-language memes were gathered in the period 2018–2023 through Google image searches. I found the affiliative and disaffiliative strategies that Placencia (2012) described within Yahoo! Respuestas advice exchanges; meme multimodality, however, offered additional versions of these strategies: characters generated affiliation or disaffiliation and more humor appeared in affiliative and disaffiliative moves. I highlight four characteristics of advising in memes: the speech act is performed indirectly; both characters and wordplay can create humor; peer advisors must signal what authorizes them to give advice; and advice deploys mythbusting. Thus, memes align interlocutors within vernacular online conversations.

SterAF: A Scene Text Recognizer with Appearance-Flow Rectification

As the demand for recognizing irregular text in natural scenes increases, people are increasingly realizing the value of such applications, such as license plate recognition systems, image search, handwriting recognition, and autonomous driving, which are profoundly changing our lives in the field of text recognition. Recent studies have shown that the recognition of curved text and perspective text has become an important challenge in the field of text recognition, and the correction of curved text is a key step to achieve accurate recognition. However, current methods use strained text image correction methods, resulting in poor recognition accuracy when recognizing curved text. Therefore, we propose an end-to-end framework called Scene Text Recognizer with Appearance-Flow rectification (SterAF), which includes a correction network and a recognition network. Specifically, the framework’s steps are as follows: first, the input text image is deformed through an appearance flow-based correction network to adaptively warp the text image, to prevent irregular and unnatural deformations of the text image. Second, a sequence-to-sequence recognition network predicts the sequence of characters in the corrected text image to accurately recognize the text in the image. Through subjective and objective experiments, our SterAF model has shown excellent performance in both qualitative and quantitative experiments.

An Analysis on Recent Approaches for Image Captioning

Image captioning is an interdisciplinary area that uses techniques from computer vision and natural language processing to provide a textual description of a picture. The Image captioning task is the process of understanding the scene present in the image by identifying objects and associated actions present to create a meaningful human-like caption which can be used for wide range of applications, including image retrieval, video indexing, assistive technology for the visually impaired, content-based image search, biomedicine, and autonomous cars. Formerly, Machine Learning was utilized for this purpose which will be extensive use of hand-crafted features such as Scale-Invariant Feature Transform (SIFT), Local Binary Patterns (LBP), the Histogram of Oriented Gradients (HOG), and combinations of these features. Extracting handmade characteristics from huge datasets is not straightforward or viable. Many deep learning-based techniques were later proposed. Deep Learning retrieval and template-based approaches were presented; however, both had drawbacks such as losing crucial objects. Recent breakthroughs in deep learning and natural language processing have resulted in considerable increases in image captioning system performance which involves adopting attention mechanisms, transformer-based architectures, multi modal connections, Object-Detection based encoder-decoder and many others. In this survey will explore some of the most recent techniques for image captioning, the datasets and evaluation measures that have been employed in deep learning-based automatic image captioning. The ultimate intention of this study is to act as a guide for researchers by emphasizing future directions for research work. Index Terms: image captioning, computer vision, deep learning, Textual description, natural language processing.