Open Image Research Articles

Performance Analysis of You Only Look Once Object Detection Models on Custom Datasets

Object detection, localization, and classification are now thought to be common study areas. The advancement of Deep Learning has aided researchers working in the field of object detection. By decreasing the model’s loss functi on, the accuracy of object detection may be boosted. Training and executing this model on a suitable GPUenabled machine can also cut down on total time. In this article, we employed the YOLO family models, namely YOLO v3, YOLO v4, and YOLO v5, and trained them using the transfer learning approach on custom datasets. These cutting- edge performers outperformed the competition on the Custom Dataset of Open Image Dataset V6 for various classes, achieving mean average precision mAP @0.5 97 %, 89 %, and 88 %, respectively. This is better than the initial trained models, which had mAP @0.5 values of 57.9 %, 43.5 %, and 33 % for YOLO v3, YOLO v4, and YOLO v5, respectively

Feasibility of Tc-99 m sestamibi uptake quantification with few-projection emission tomography.

Molecular breast imaging uses Tc-99 m sestamibi to obtain functional images of the breast. Determining the Tc-99 m sestamibi uptake in volumes of interest in the breast may be useful in assessing the response to neoadjuvant chemotherapy or for the purposes of breast cancer risk assessment. To determine, using Monte Carlo simulation, if emission tomography can be used to quantify the uptake of Tc-99 m sestamibi in molecular breast imaging and if so, to determine the accuracy as a function of the number of projections used in the reconstruction process. In this study, two voxelized breast models are implemented with different ratios of fibroglandular to fatty tissue and tumoral masses of varying dimensions. Monte Carlo simulation is used to calculate sets of projections, which assumes that each tumoral mass contains a given Tc-99 m activity. Projections are also calculated for a calibration phantom in order to correlate the known activity with the image pixel value. For each case, the total number of calculated projections is 36 and the reconstruction is carried out for 36, 18, 9, 7 and 5 projections, respectively, using an open source image reconstruction toolbox. Study data show that determination of Tc-99 m sestamibi uptake with and average error of 7% can be carried out with as little as 7 projections. Molecular breast emission tomography enables to accurately determine the Tc-99 m sestamibi tumoral mass uptake with the number of projections very close to the number of images currently acquired in clinical practice.

Skew Class-Balanced Re-Weighting for Unbiased Scene Graph Generation

An unbiased scene graph generation (SGG) algorithm referred to as Skew Class-Balanced Re-Weighting (SCR) is proposed for considering the unbiased predicate prediction caused by the long-tailed distribution. The prior works focus mainly on alleviating the deteriorating performances of the minority predicate predictions, showing drastic dropping recall scores, i.e., losing the majority predicate performances. It has not yet correctly analyzed the trade-off between majority and minority predicate performances in the limited SGG datasets. In this paper, to alleviate the issue, the Skew Class-Balanced Re-Weighting (SCR) loss function is considered for the unbiased SGG models. Leveraged by the skewness of biased predicate predictions, the SCR estimates the target predicate weight coefficient and then re-weights more to the biased predicates for better trading-off between the majority predicates and the minority ones. Extensive experiments conducted on the standard Visual Genome dataset and Open Image V4 and V6 show the performances and generality of the SCR with the traditional SGG models.

Validation of the open biological negative image set for a Portuguese population: Comparing Japanese and Portuguese samples and an exploration of low-order visual properties of the stimuli.

Recently, Shirai and Watanabe Royal Society Open Science, 9(1), 211128 (2022) developed OBNIS (Open Biological Negative Image Set), a comprehensive database containing images (primarily animals but also fruits, mushrooms, and vegetables) that visually elicit disgust, fear, or neither. OBNIS was initially validated for a Japanese population. In this article, we validated the color version of OBNIS for a Portuguese population. In study 1, the methodology of the original article was used. This allowed direct comparisons between the Portuguese and Japanese populations. Aside from a few emotional classification mismatches between disgust, fear, or neither-related images, we found that arousal and valence relate distinctively in both populations. In contrast to the Japanese sample, the Portuguese reported increased arousal for more positive valenced stimuli, suggesting that OBNIS images elicit positive emotions in the Portuguese population. These results showed important cross-cultural differences regarding OBNIS. In study 2, a methodological change was introduced: instead of the three classification options used originally (fear, disgust, or neither), six basic emotions were used (fear, disgust, sadness, surprise, anger, happiness), and a "neither" option, to confirm whether some of the originally "neither-related" images are associated with positive emotions (happiness). Additionally, the low-order visual properties of images (luminosity, contrast, chromatic complexity, and spatial frequency distribution) were explored due to their important role in emotion-related research. A fourth image group associated with happiness was found in the Portuguese sample. Moreover, image groups present differences regarding the low-order visual characteristics, which are correlated with arousal and valence ratings, highlighting the importance of controlling such characteristics in emotion-related research.

Robust Data-Aware Sensing Based on Half Quadratic Hashing for the Visual Internet of Things

In the Visual Internet of Things (VIoT) such as drones over flying ad-hoc networks (FANETs), terminal nodes need to preliminarily identify sensing data. Thus, data can be rapidly dispatched to dedicated edge nodes for further processing. Nonetheless, terminals may collect partially observed visual data, which cause biased labeling. In view of such, this study proposes half quadratic supervised discrete hashing (HQSDH) that can conquer the biased similarity estimation for partially observed images, especially when large continuous occlusion, missing values, or sample-specific outliers exist. The proposed method can automatically and adaptively adjust itself via HQ auxiliary variables to avoid oversensitivity to large errors caused by partially observed data. In this study, to solve HQ discrete optimization problems in HQSDH, HQ discrete cyclic coordinate descent (HQDCCD) is developed. Second, partially observed images may cause inconsistent binary codes and increase the Hamming distances. Thus, this study devises HQ balancing decorrelation constraints along with normalization regularization to accommodate the problems. Third, a systematic and generic solution that allows various HQ functions to jointly work in the same framework is modeled to provide versatility. Experiments on open image data sets were carried out for evaluation. The results showed that the proposed method provided more resilience against continuous occlusion, missing values, and sample-specific outliers than the baselines when dealing with partially observed images.

First person – Sara Rolfe

ABSTRACT First Person is a series of interviews with the first authors of a selection of papers published in Biology Open, helping researchers promote themselves alongside their papers. Sara Rolfe is first author on ‘ Deep learning enabled multi-organ segmentation of mouse embryos’, published in BiO. Sara is a Research Scientist in the lab of Murat Maga at Seattle Children's Research Institute, investigating quantifying morphology from 3D images, automated phenotyping, and open image science.

A Comparative Study of YOLOv5 and YOLOv7 Object Detection Algorithms

This paper presents a comparative analysis of the widely accepted YOLOv5 and the latest version of YOLO which is YOLOv7. Experiments were carried out by training a custom model with both YOLOv5 and YOLOv7 independently in order to consider which one of the two performs better in terms of precision, recall, mAP@0.5 and mAP@0.5:0.95. The dataset used in the experiment is a custom dataset for Remote Weapon Station which consists of 9,779 images containing 21,561 annotations of four classes gotten from Google Open Images Dataset, Roboflow Public Dataset and locally sourced dataset. The four classes are Persons, Handguns, Rifles and Knives. The experimental results of YOLOv7 were precision score of 52.8%, recall value of 56.4%, mAP@0.5 of 51.5% and mAP@0.5:0.95 of 31.5% while that of YOLOv5 were precision score of 62.6%, recall value of 53.4%, mAP@0.5 of 55.3% and mAP@0.5:0.95 of 34.2%. It was observed from the experiment conducted that YOLOv5 gave a better result than YOLOv7 in terms of precision, mAP@0.5 and mAP@0.5:0.95 overall while YOLOv7 has a higher recall value during testing than YOLOv5. YOLOv5 records 4.0% increase in accuracy compared to YOLOv7.

Intraoperative Assessment of Surgical Margins in Cancer Resection Surgery Via the Sub-Diffuse Optical Tomography "S-DOT"

In Head and Neck Squamous Cell Carcinomas (HNSCC), positive margins after surgical resection dominate the clinical outcome of patients. Between 15-30 precent of all HNSCC surgeries result in a positive margin (>5mm) that requires postoperative chemo-radiation, radiotherapy, and/or revision surgeries. To minimize local recurrence, margin assessment is typically performed through surgical pathology departments; however, this technique often takes more than 24 hours to process and only a small fraction of the total surgical margin is evaluated. A second, intraoperativeapproach used by surgeons to minimize local recurrence is frozen section analysis (FSA), which in time intensive and suffers heavily from sampling errors. To effect change, we have hypothesized a sub-diffuse optical tomography modality that can identify the differences in depth sensitivity of the fluorescence, Cetuximab-IRDye800CW conjugate (ICON), by the fluorescent photons collected from sub-diffused media as a function of the photon exit angle from the excited tissue. Based on promising preliminary simulations using Monte Carlo MATLAB, closed vs. open aperture fluorescence imaging in biological tissue demonstrated an enhanced true depth of fluorescence resolution up to 6mm, well beyond the insufficient 5mm mark. By identifying the deep margins from the fluorescent peaks within 5 minutes, we hope to improve complete oncological surgery by providing surgeons an exact region of interest on the specimen where the tumor is closest to the edge, so they may continue surgery by providing sugeons an exact region of interest on the specimen where the tumor is closest to the edge, so they may continue surgery on the spot to minimize the probability of cancer being left behind.

A Novel Geometric Parameter Self-Calibration Method for Portable CBCT Systems

In outdoor environments or environments with space restrictions, it is difficult to transport and use conventional computed tomography (CT) systems. Therefore, there is an urgent need for rapid reconstruction of portable cone-beam CT (CBCT) systems. However, owing to its portability and the characteristics of temporary construction environments, high precision spatial location is difficult to achieve with portable CBCT systems. To overcome these limitations, we propose an iterative self-calibration improvement method with a self-calculated initial value based on the projection relationship and image features. The CT value of an open field image was used as the weight value of the projection data in the subsequent experiments to reduce the nonlinear attenuation of the projection intensity. Subsequently, an initial value was obtained based on the invariance of the rotation axis. Finally, self-calibration was realized iteratively using the reconstructed image. This method overcomes the main problem of the rotation axis invariance calibration algorithm—high similarity between the adjacent positions of symmetrical homogeneous materials. The proposed method not only improves the precision of self-calibration based on the projection relationship, but also reduces the performance cost and solution time of the self-calibration algorithm based on the image features. Thus, it satisfies the precision requirements for self-calibration of portable CBCT systems.

Enhancement of Marine Lantern’s Visibility under High Haze Using AI Camera and Sensor-Based Control System

This thesis describes research to prevent maritime safety accidents by notifying navigational signs when sea fog and haze occur in the marine environment. Artificial intelligence, a camera sensor, an embedded board, and an LED marine lantern were used to conduct the research. A deep learning-based dehaze model was learned by collecting real marine environment and open haze image data sets. By applying this learned model to the original hazy images, we obtained clear dehaze images. Comparing those two images, the concentration level of sea fog was derived into the PSNR and SSIM values. The brightness of the marine lantern was controlled through serial communication with the derived PSNR and SSIM values in a realized sea fog environment. As a result, it was possible to autonomously control the brightness of the marine lantern according to the concentration of sea fog, unlike the current marine lanterns, which adjust their brightness manually. This novel-developed lantern can efficiently utilize power consumption while enhancing its visibility. This method can be used for other fog concentration estimation systems at the embedded board level, so that applicable for local weather expectations, UAM navigation, and autonomous driving for marine ships.

A novel approach for yoga pose estimation based on in-depth analysis of human body joint detection accuracy.

Virtual motion and pose from images and video can be estimated by detecting body joints and their interconnection. The human body has diverse and complicated poses in yoga, making its classification challenging. This study estimates yoga poses from the images using a neural network. Five different yoga poses, viz. downdog, tree, plank, warrior2, and goddess in the form of RGB images are used as the target inputs. The BlazePose model was used to localize the body joints of the yoga poses. It detected a maximum of 33 body joints, referred to as keypoints, covering almost all the body parts. Keypoints achieved from the model are considered as predicted joint locations. True keypoints, as the ground truth body joint for individual yoga poses, are identified manually using the open source image annotation tool named Makesense AI. A detailed analysis of the body joint detection accuracy is proposed in the form of percentage of corrected keypoints (PCK) and percentage of detected joints (PDJ) for individual body parts and individual body joints, respectively. An algorithm is designed to measure PCK and PDJ in which the distance between the predicted joint location and true joint location is calculated. The experiment evaluation suggests that the adopted model obtained 93.9% PCK for the goddess pose. The maximum PCK achieved for the goddess pose, i.e., 93.9%, PDJ evaluation was carried out in the staggering mode where maximum PDJ is obtained as 90% to 100% for almost all the body joints.

π-spaces and their open images

We study spaces that can be mapped onto the Baire space (i.e. the countable power of the countable discrete space) by a continuous quasi-open bijection. We give a characterization of such spaces in terms of Souslin schemes and call these spaces π-spaces. We show that every space that has a Lusin π-base is a π-space and that every second-countable π-space has a Lusin π-base. The main result of this paper is a characterization of continuous open images of π-space.

Examining the utility of a photorealistic virtual ear in otologic education

BackgroundOtolaryngology–head and neck surgical (OHNS) trainees’ operating exposure is supplemented by a combination of didactic teaching, textbook reading, and cadaveric dissections. Conventional teaching, however, may not adequately equip trainees with an understanding of complex visuospatial relationships of the middle ear. Both face and content validation were assessed of a novel three-dimensional (3D) photorealistic virtual ear simulation tool underwent face and content validation as an educational tool for OHNS trainees.MethodsA three-dimensional mesh reconstruction of open access imaging was generated using geometric modeling, which underwent global illumination, subsurface scattering, and texturing to create photorealistic virtual reality (VR) ear models were created from open access imaging and comiled into a educational platform. This was compiled into an educational VR platform which was explored to validate the face and content validity questionnaires in a prospective manner. OHNS post-graduate trainees were recruited from University of Toronto and University of Calgary OHNS programs. Participation was on a voluntary basis.ResultsTotal of 23 OHNS post-graduate trainees from the two universities were included in this study. The mean comfort level of otologic anatomy was rated 4.8 (± 2.2) out of 10. Senior residents possessed more otologic surgical experience (P < 0.001) and higher average comfort when compared to junior residents [6.7 (± 0.7) vs. 3.6 (± 1.9); P = 0.001]. Face and content validities were achieved in all respective domains with no significant difference between the two groups. Overall, respondents believed OtoVIS was a useful tool to learn otologic anatomy with a median score of 10.0 (8.3–10.0) and strongly agreed that OtoVIS should be added to OHNS training with a score of 10.0 (9.3–10.0).ConclusionsOtoVIS achieved both face and content validity as a photorealistic VR otologic simulator for teaching otologic anatomy in the postgraduate setting. As an immersive learning tool, it may supplement trainees’ understanding and residents endorsed its use.Graphical

Quaternion Relation Embedding for Scene Graph Generation

As an important visual understanding task, scene graph generation has been drawing widespread attention and could boost a broad range of downstream vision applications. Traditional scene graph generation methods based on different context refinements are trained with probabilistic chain rule, which treats objects and relationships as independent entities. Despite their surprisingly great progress, such a plain formulation unconsciously ignores the latent geometric structure of entities and relationships. To address this issue, we move beyond the traditional real-valued representations and use <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Quat</u> ernion <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">R</u> elation <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">E</u> mbedding (QuatRE) to generate scene graphs with more expressive hypercomplex representations. More specifically, we introduce the concept of quaternion representations, hyper-complex valued with three imaginary components for objects entities, then formulate the relation triplets with Hamilton product. Benefiting from explicitly modeling the latent inter-dependencies among all imaginary components and strong expressive capacity, our proposed QuatRE method could better capture the interactions between entities. More importantly, our novel QuatRE method can be treated as a plug-in and well generalized into other methods for performance improvement as it involves no additional layers. Finally, extensive comparisons of our proposed method against the state-of-the-art methods on two large-scale and widely-used datasets, i.e. Visual Genome and Open Images, demonstrated our superiority and generalization capability on various metrics for biased or unbiased inference.

Outlier Exposure for Open Set Crop Recognition From Multitemporal Image Sequences

When it comes to technology in agriculture, one of the most important aspects is farmland crop monitoring. However, in most cases, only the main crops are needed to be monitored by satellite images, due to their high territorial extension. Therefore, a semantic segmentation model for identifying plantations should correctly classify the majority classes and also automatically identify other unknown crops. Open Set Recognition aims to embrace both of these causes, so that the model can be more robust in the wild. This work adapts the framework of Outlier Exposure for open set image segmentation. Outlier Exposure was evaluated by adding it to three distinct methods for open set segmentation: Softmax Thresholding, OpenPCS and OpenPCS++. We conducted several experiments in order to enrich the discussion of the impact of Outlier Exposure on the semantic segmentation of crop imagery. Our methodology achieved a consistent increase for OpenPCS and OpenPCS++ methods, with an improvement of up to 7.5% in terms of area under the ROC curve if compared to previous work.

Three-dimensional template combined with MR-guided iodine-125 brachytherapy for recurrent brain metastases.

Treatment of recurrent brain metastases is extremely challenging. Here, we evaluated the feasibility and efficacy of an individualized three-dimensional template combined with MR-guided iodine-125 (125I) brachytherapy in the treatment of recurrent brain metastases. Twenty-eight patients with recurrent 38 brain metastases underwent 125I brachytherapy from December, 2017 to January, 2021. A pre-treatment brachytherapy plan and three-dimensional template were generated according to isovoxel T1-weighted MR images. 125I seeds were implanted under the guidance of three-dimensional template and 1.0-T open MR imaging. Dosimetry verification was performed based on CT/MR fusion images. Pre-operative and post-operative dosimetry parameters of D90, V100, and conformity index (CI) were compared. Overall response rate (ORR), disease control rate (DCR) at 6 months, and 1-year survival rate were calculated. Median overall survival (OS) measured from the date of 125I brachytherapy was estimated using Kaplan-Meier method. No significant differences were observed between pre-operative and post-operative D90, V100, and CI values (p > 0.05). The ORR and DCR at 6 months were 91.3% and 95.7%, respectively. The 1-year survival rate was 57.1%. The median OS was 14.1 months. Two cases of minor hemorrhage and 5 cases of symptomatic brain edema were observed during the study. All clinical symptoms were alleviated after corticosteroid treatment applied for 7 to 14 days. A three-dimensional template combined with MR-guided 125I brachytherapy in the treatment of recurrent brain metastases is feasible, safe, and effective. This novel 125I brachytherapy strategy is an attractive alternative in the treatment of brain metastases.

Estimating Compositions and Nutritional Values of Seed Mixes Based on Vision Transformers.

The cultivation of seed mixtures for local pastures is a traditional mixed cropping technique of cereals and legumes for producing, at a low production cost, a balanced animal feed in energy and protein in livestock systems. By considerably improving the autonomy and safety of agricultural systems, as well as reducing their impact on the environment, it is a type of crop that responds favorably to both the evolution of the European regulations on the use of phytosanitary products and the expectations of consumers who wish to increase their consumption of organic products. However, farmers find it difficult to adopt it because cereals and legumes do not ripen synchronously and the harvested seeds are heterogeneous, making it more difficult to assess their nutritional value. Many efforts therefore remain to be made to acquire and aggregate technical and economical references to evaluate to what extent the cultivation of seed mixtures could positively contribute to securing and reducing the costs of herd feeding. The work presented in this paper proposes new Artificial Intelligence techniques that could be transferred to an online or smartphone application to automatically estimate the nutritional value of harvested seed mixes to help farmers better manage the yield and thus engage them to promote and contribute to a better knowledge of this type of cultivation. For this purpose, an original open image dataset has been built containing 4,749 images of seed mixes, covering 11 seed varieties, with which 2 types of recent deep learning models have been trained. The results highlight the potential of this method and show that the best-performing model is a recent state-of-the-art vision transformer pre-trained with self-supervision (Bidirectional Encoder representation from Image Transformer). It allows an estimation of the nutritional value of seed mixtures with a coefficient of determination R2 score of 0.91, which demonstrates the interest of this type of approach, for its possible use on a large scale.

Guest Editorial Large-Scale Medical Image and Video Analytics for Clinical Decision Support

The papers in this special section focus on large-scale medical imaging and video analytics for clinical decision support systems. Biomedical images and videos are ubiquitous and overwhelming in volume, amounting to a database that can be measured in zettabytes.With increased access to open image and video datasets and the recent development of effective image and video analysis systems, there is a unique opportunity for the development of artificial intelligence (AI) systems that can be trained and tested on large-scale biomedical image and video databases. The special issue summarizes emerging methods associated with the development of computer-aided diagnostic systems. More specifically, the special issue discusses the development of methods for dealing with small or large or creating new datasets, biomedical image segmentation, and image classification. The development of new dataset methods allows us to develop methods for specific diseases, employ meta-learning for training on small datasets, or develop methods for reducing larger video datasets. Biomedical image segmentation is a primary focus of the special issue.

Common Thorax Diseases Recognition Using Zero-Shot Learning With Ontology in the Multi-Labeled ChestX-ray14 Data Set

Disease detection/recognition with limited data sets and labels in the medical image domain is a very costly and greatest challenge. Although open image data sets have increased recently, researches on this problem still need to be developed. Researches to diversify data sets are both costly and face the problem of subjectivity. Unseen classes can be trained with the Zero-Shot Learning (ZSL) in order to overcome this problem. In this paper, we aimed to strengthen ZSL by using ontology as an auxiliary information for class embeddings. In our approach, ZSL is supported by the image embeddings and class embeddings of the multi-labelled ChestX-ray14 data set, as well as the semantic data from DBpedia. In this paper, which we believe will be pioneering in the medical image domain, the Cosine, Hamming and Euclidean distances were taken into account in order to maximize the similarities.We trained ResNet50 neural network with different parameters on the multi-labelled ChestX-ray14 data set. 23.25% precision value in one-to-one matching and 29.59% precision value in at least one matching were obtained. We think that this paper will make a significant contribution to the medical image domain by detecting/recognizing unseen disease images.

Semantic Communication System Based on Semantic Slice Models Propagation

Traditional communication systems treat messages’ semantic aspects and meaning as irrelevant to communication, revealing its limitations in the era of artificial intelligence (AI), such as communication efficiency and intent-sharing among different entities. Through broadening the scope of the traditional communication system and the AI-based encoding techniques, in this manuscript, we present a novel semantic communication system, which involves the essential semantic information exploration, transmission and recovery for more efficient communications. Compared to other state-of-the-art semantic communication-related works, our proposed semantic communication system is characterized by the “flow of the intelligence” via the propagation of the model. Besides, the concept of semantic slice-models (SeSM) is proposed to enable flexible model-resembling under the different requirements of the model performance, channel situation and transmission goals. Specifically, a layer-based semantic communication system for images (LSCI) is built on the simulation platform to demonstrate the feasibility of the proposed system and a novel semantic metric called semantic service quality (SS) is proposed to evaluate the semantic communication systems. We evaluate the proposed system on Cityscapes and Open Images datasets, resulting in averaged 10% and 2% bit rate reduction over JPEG and JPEG2000, respectively. In comparison to LDPC, the proposed channel coding scheme can averagely save 2dB and 5dB in AWGN channel and Rayleigh fading channel, respectively.