Dish Image Research Articles

Tailoring the Nutritional Composition of Italian Foods to the US Nutrition5k Dataset for Food Image Recognition: Challenges and a Comparative Analysis

Background: Training of machine learning algorithms on dish images collected in other countries requires possible sources of systematic discrepancies, including country-specific food composition databases (FCDBs), to be tackled. The US Nutrition5k project provides for ~5000 dish images and related dish- and ingredient-level information on mass, energy, and macronutrients from the US FCDB. The aim of this study is to (1) identify challenges/solutions in linking the nutritional composition of Italian foods with food images from Nutrition5k and (2) assess potential differences in nutrient content estimated across the Italian and US FCDBs and their determinants. Methods: After food matching, expert data curation, and handling of missing values, dish-level ingredients from Nutrition5k were integrated with the Italian-FCDB-specific nutritional composition (86 components); dish-specific nutrient content was calculated by summing the corresponding ingredient-specific nutritional values. Measures of agreement/difference were calculated between Italian- and US-FCDB-specific content of energy and macronutrients. Potential determinants of identified differences were investigated with multiple robust regression models. Results: Dishes showed a median mass of 145 g and included three ingredients in median. Energy, proteins, fats, and carbohydrates showed moderate-to-strong agreement between Italian- and US-FCDB-specific content; carbohydrates showed the worst performance, with the Italian FCDB providing smaller median values (median raw difference between the Italian and US FCDBs: −2.10 g). Regression models on dishes suggested a role for mass, number of ingredients, and presence of recreated recipes, alone or jointly with differential use of raw/cooked ingredients across the two FCDBs. Conclusions: In the era of machine learning approaches for food image recognition, manual data curation in the alignment of FCDBs is worth the effort.

Retracted] MicroRNA‑320a suppresses tumour cell proliferation and invasion of renal cancer cells by targeting FoxM1.

Subsequently to the publication of the above paper, the authors drew to the attention of the Editorial Office that they made a couple of errors in terms of the data assembly in Figs. 2 and 4 in their paper; specifically, the Transwell assay data shown for the 'miR-320a+/FoxM1+' panel in Fig. 5D on p. 1923 also appeared as the 'ACTN/NC' data panel in Fig. 4E on the same page (Fig. 4E contained the erroneously duplicated panel). In addition, data featured in Fig. 2D of the above paper were strikingly similar to data that appeared in Fig. 6e of the following paper, published subsequently to this article, written by different authors (although a Dr Shiyue Zhao worked in the molecular biology laboratory of Harbin Medical University from 2017 to 2018, and the research collaboration was conducted with Dr Chenlong Li's research group): Li C, Zheng H, Hou W, Bao H, Xiong J, Che W, Gu Y, Sun H and Liang P: Long non-coding RNA linc00645 promotes. TGF-β-induced epithelial-mesenchymal transition by regulating miR-205-3p-ZEB1 axis in glioma. Cell Death Dis 10: 17, 2019. Finally, after having conducted an independent investigation of the data in this paper, the Editorial Office noted that one of the Petri dish images in Fig. 2C was also strikingly similar to data that appeared in Fig. 2H of the abovementioned article in the journal Cell Death & Disease. After having considered the authors' request for corrigendum, in view of the problems that were identified with the data, the Editor of Oncology Reports has decided that, owing to a lack of confidence in the presented data, the paper should instead be retracted from the journal. After having informed the authors of this decision, they accepted the decision to retract this paper. The Editor apologizes to the readership for any inconvenience caused. [Oncology Reports 40: 1917‑1926, 2018; DOI: 10.3892/or.2018.6597].

Identification of Urinary Tract Infection Causing Bacteria from Microbial Colony Images using Deep Learning

Abstract: Diagnosis of microbiological and bacterial infections involves extensive procedures ofsample culture and microscopic examination. The diagnosis process begins with the identification of symptoms followed by collection of test samples in the form of blood,scraps of skin lesions, mucus, sputum, urine etc. The entire process of bacteria routine culture and bacteriological examination may take as long as 10 to 11 days. Due to the long time required for the standard process of species identification, high costs and need of human expertise it is beneficial to use methods that do not rely on conventional methods. In this work we propose a comprehensive Deep Learning basedapproach to identify urinary Tract Infection (UTI) causing bacteria from microbial colony images of bacteria cultured on agar plates. The approach explains the use of VGG-19 and Inception-v3 deep learning architectures, in bacteria identification evaluated on The Annotated Germs for Automated Recognition (AGAR) dataset, an image dataset of microbial colonies cultured on agar plates. The findings affirmed the significant potential of employing deep learning techniques for the identification of microbial colonies and their classification using Petri dish images.

DeepTrayMeal: Automatic dietary assessment for Chinese tray meals based on deep learning

Tray meal is a popular way of eating in China, and tray-based automatic dietary assessment is important for public health. Relevant research is lacking because public tray meal datasets and suitable methods are unavailable. In this study, we established and published the first Chinese tray meal dataset, the ChinaLunchTray-99. We collected real-world 1185 tray meal images, covering 99 dish categories with corresponding manually annotated bounding box and category-level labels. We developed a new framework for automatic dietary assessment, which consists of dish image recognition, volume estimation and nutrition mapping. First, we demonstrated a tray meal detection model considering feature extraction, anchor scales, and loss function, resulting in a high mean Average Precision of 92.13%. Second, we proposed an automatic method to estimate volume via detection results and tray’s information. Finally, nutrients were mapped from the estimated volume. Our research can promote applications of automatic dietary assessment for Chinese tray meals.

Blowing minds with exploding dish names/images: The effect of implied explosion on consumer behavior in a restaurant context

Dish names and dish images can be widely found online, providing consumers with important information. Meanwhile, implied explosion (i.e., the perception of explosion induced by static stimuli) is increasingly utilized by real-world restaurants. The present research thus combines dish names, dish images, and implied explosion to examine the impact of implied explosion on various aspects of consumer behavior within a restaurant context. Three experiments demonstrated that exploding dish names and exploding dish images (i.e., dish names/dish images showing implied explosion) can create a more intense taste perception and a more favorable taste evaluation. Additionally, exploding dish images can enhance perceived dish liking and increase consumers’ willingness to pay. The present research suggests that exploding dish names/dish images are subtle but effective communication tools for the tourism industry, helping to deliver a more stimulating perception and experience to consumers and to generate higher margins. By exploring the effects of implied explosion, we also introduce the implied motion concept to the tourism management literature.

A two-stage deep counting for bacterial colonies from multi-sources

Automatic bacterial colony counting (BCC) from Petri dish images is full of challenges due to commonly exhibited overlapping, occlusion, and inconsistency problems. Modern deep learning methods require sufficient annotated data to identify colonies from a single source and yet usually perform badly on new sources, which makes it difficult to be extended. In this paper, a scalable two-stage deep learning framework is proposed to count bacterial colonies from multi-sources with both generic and specific features. In addition, a novel blending based strategy is proposed to augment the data and raise the robustness by increasing the sample complexity. We also suggest a refinement of the coarse results from existing BCC programmes could save a lot of time and effort for annotation. Four state-of-the-art methods are selected to compare with ours. Our method outperforms others in terms of mean average precision, averaged root-mean-square error and mean absolute percentage error. Hyper-parameters and other settings are also investigated in the experiments, which proved the scalability, robustness and accuracy of the proposed method.

Physically Consistent Preferential Bayesian Optimization for Food Arrangement

This letter considers the problem of estimating a preferred food arrangement for users from interactive pairwise comparisons using Computer Graphics (CG)-based dish images. As a foodservice industry requirement, we need to utilize domain rules for the geometry of the arrangement (e.g., the food layout of some Japanese dishes is reminiscent of mountains). However, those rules are qualitative and ambiguous; the estimated result might be physically inconsistent (e.g., each food physically interferes, and the arrangement becomes infeasible). To cope with this problem, we propose Physically Consistent Preferential Bayesian Optimization (PCPBO) as a method that obtains physically feasible and preferred arrangements that satisfy domain rules. PCPBO employs a bi-level optimization that combines a physical simulation-based optimization and a Preference-based Bayesian Optimization (PbBO). Our experimental results demonstrated the effectiveness of PCPBO on simulated and actual human users.

Accurate Real-Life Chinese Dish Recognition

Deep learning is a new research direction in the field of machine learning, which was introduced into machine learning to bring it closer to its original goal. Accurate dish recognition becomes increasingly important in the multimedia community since it can help cuisine recommendation, calorie management, service improvement, and other food computing tasks. Many novel approaches have been developed on web recipes and menu pictures, while few are concerned real-life dish image analysis. In this study, a deep learning-based prototype system is deployed in a Chinese canteen, and 28 dish types, 16,904 images, and 45,061 instances have been collected. Specifically, in the prototype system, three practical issues are explored, including the backbone network selection, the training strategy determination, and the minimum number of samples for model upgrading. Experimental results suggest that fine-tuned Faster-RCNN can serve as the backbone network of the prototype system since it outperforms the other four fine-tuned networks on dish recognition (accuracy, 98.10%; recall, 97.20%; MAP (mean average precession), 98.30%) and satisfies real-time requirement (0.15 second per image). Meanwhile, the transferred backbone network achieves superior results (MAP, 96.48%) over the same architecture trained from image scratches (MAP, 87.84%). On model upgrading, a good (MAP, 91.34%) to better (MAP, 96.48%) outcome is obtained when the training size is increased from 50 to 200 samples per dish type, and 150 and more instances should be annotated if a new dish type is added to the system’s recognition list. Conclusively, the real-life deployment and evaluation of the prototype system indicate that deep learning is full of potential to enhance customer experience through accurate daily dish recognition.

Detection of dish waste degree based on image processing and convolutional neural networks

AbstractMany restaurants have a certain amount of food waste. The monitoring of food waste will help restaurants to eliminate some dishes with outrageous waste and reduce waste from the source. In view of this, this research proposed a method to detect waste of dishes through image processing and deep learning technology. According to the remaining quantity of the dishes, the collected dish images were preliminarily divided into six levels, which were used as sample labels, and then the image of the uneaten dishes and the image of the dishes after eating were stacked as the input of the network. Trained in the InceptionV3, Xception, and ResNet18 network models, we find that compared with the single image data as the input, the effect of stacking the two images data as the input was better. The accuracy of sample label recognition increased by 6.97%, 5.81%, and 4.1% respectively. Further analysis discovered that the sample that predicted wrong on the test set data, its true label, and the predicted wrong label were adjacent. Therefore, with the help of the probability vector output by the trained network model, the definition method of the level of dish waste degree and its corresponding accuracy metric standard was further given. Finally, the recognition accuracy of the best network structure InceptionV3 on the test set data can reach 98.47%.

The Deep Features and Attention Mechanism-Based Method to Dish Healthcare Under Social IoT Systems: An Empirical Study With a Hand-Deep Local–Global Net

Many mobile apps of social Internet of Things (sIOT) systems can help us record and share daily events, such as health and sport events. In fact, healthy diet recognition is an important and challenging problem in dish health assessment. Via the collection and monitoring of data pertaining to our daily diet, we can work in collaborative ways to achieve dish image annotation based on sIOT systems to enhance deep features. To this end, this article proposes a deep feature and attention mechanism-based method for dish health assessment, which aims to apply a hand-deep local–global net (HDLGN) for dish image recognition. Then, food taste is used as health guidance for people who want to lose weight or follow doctors’ advice. First, the local attention mechanism is introduced to identify key areas of the dish image. Second, ingredient and handcrafted color features are extracted to learn deep features. Subsequently, we combine local and global attention mechanisms to return the dish taste as the recognition result. Finally, experiments show that our proposed method can effectively improve the accuracy of taste recognition.

Computer-aided scoring of erb-b2 receptor tyrosine kinase 2 (HER2) gene amplification status in breast cancer

BackgroundIdentification of HER2 protein overexpression and/or amplification of the HER2 gene are required to qualify breast cancer patients for HER2 targeted therapies. In situ hybridization (ISH) assays that identify HER2 gene amplification function as a stand-alone test for determination of HER2 status and rely on the manual quantification of the number of HER2 genes and copies of chromosome 17 to determine HER2 amplification. MethodsTo assist pathologists, we have developed the uPath HER2 Dual ISH Image Analysis for Breast (uPath HER2 DISH IA) algorithm, as an adjunctive aid in the determination of HER2 gene status in breast cancer specimens. The objective of this study was to compare uPath HER2 DISH image analysis vs manual read scoring of VENTANA HER2 DISH-stained breast carcinoma specimens with ground truth (GT) gene status as the reference. Three reader pathologists reviewed 220, formalin-fixed, paraffin-embedded (FFPE) breast cancer cases by both manual and uPath HER2 DISH IA methods. Scoring results from manual read (MR) and computer-assisted scores (image analysis, IA) were compared against the GT gene status generated by consensus of a panel of pathologists. The differences in agreement rates of HER2 gene status between manual, computer-assisted, and GT gene status were determined. ResultsThe positive percent agreement (PPA) and negative percent agreement (NPA) rates for image analysis (IA) vs GT were 97.2% (95% confidence interval [CI]: 95.0, 99.3) and 94.3% (95% CI: 90.8, 97.3) respectively. Comparison of agreement rates showed that the lower bounds of the 95% CIs for the difference of PPA and NPA for IA vs MR were –0.9% and –6.2%, respectively. Further, inter- and intra-reader agreement rates in the IA method were observed with point estimates of at least 96.7%. ConclusionsOverall, our data show that the uPath HER2 DISH IA is non-inferior to manual scoring and supports its use as an aid for pathologists in routine diagnosis of breast cancer.

A Dish Recognition Framework Using Transfer Learning

Dish understanding from digital media is an interesting problem, but it also contains a big challenge. The challenge comes from the complexity of ingredients in the dish. With the development of deep learning, several effective tools can solve the problem partially. In this work, the task of dish recognition is considered. A novel dish recognition method based on EfficientNet architecture and transfer learning is proposed. First, we modify the EfficientNet-B0 by adding several important layers. Second, we use transfer learning to utilize optimal parameters obtained from pretraining the model on ImageNet, and then retrain it on a new dataset of dish images, i.e., UEH-VDR dataset. The UEH-VDR dataset contains images about Vietnamese dishes collected from various sources. Experimental results show that the proposed method can achieve an accuracy of 92.33% for the task of recognizing a dish. It also works more effectively than other models based on popular convolutional neural networks such as VGG and ResNet. In addition, a mobile application is also developed based on the trained data to serve visitors who want to discover the Vietnamese culinary culture.

Mixed Dish Recognition With Contextual Relation and Domain Alignment

Mixed dish is a food category that contains different dishes mixed in one plate, and is popular in Eastern and Southeast Asia. Recognizing the individual dishes in a mixed dish image is important for health related applications, e.g. to calculate the nutrition values of the dish. However, most existing methods that focus on single dish classification are not applicable to the recognition of mixed dish images. The main challenge of mixed dish recognition comes from three aspects: a wide range of dish types, the complex dish combination with severe overlap between different dishes and the large visual variances of same dish type caused by different cooking/cutting methods applied in different canteens. In order to tackle these problems, we propose the contextual relation network that encodes the implicit and explicit contextual relations among multiple dishes from region-level features and label-level co-occurrence respectively. Besides, to address the visual variances of dish instances from different canteens, we introduce the domain adaption networks to align both local and global features, and eliminating domain gaps of dish features across different canteens. In addition, we collect a mixed dish image dataset containing 9254 mixed dish images from 6 canteens in Singapore. Extensive experiments on both our dataset and public one validate that our methods can achieve top performance for localizing and recognizing multiple dishes and solve the domain shift problem to a certain extent in mixed dish images.

Zero-Shot Ingredient Recognition by Multi-Relational Graph Convolutional Network

Recognizing ingredients for a given dish image is at the core of automatic dietary assessment, attracting increasing attention from both industry and academia. Nevertheless, the task is challenging due to the difficulty of collecting and labeling sufficient training data. On one hand, there are hundred thousands of food ingredients in the world, ranging from the common to rare. Collecting training samples for all of the ingredient categories is difficult. On the other hand, as the ingredient appearances exhibit huge visual variance during the food preparation, it requires to collect the training samples under different cooking and cutting methods for robust recognition. Since obtaining sufficient fully annotated training data is not easy, a more practical way of scaling up the recognition is to develop models that are capable of recognizing unseen ingredients. Therefore, in this paper, we target the problem of ingredient recognition with zero training samples. More specifically, we introduce multi-relational GCN (graph convolutional network) that integrates ingredient hierarchy, attribute as well as co-occurrence for zero-shot ingredient recognition. Extensive experiments on both Chinese and Japanese food datasets are performed to demonstrate the superior performance of multi-relational GCN and shed light on zero-shot ingredients recognition.

Promise and Ontological Ambiguity in the In vitro Meat Imagescape: From Laboratory Myotubes to the Cultured Burger

In vitro meat (IVM), also known as cultured meat, involves growing cells into muscle tissue to be eaten as food. The technology had its most high-profile moment in 2013 when a cultured burger was cooked and tasted in a press conference. Images of the burger featured in the international media and were circulated across the Internet. These images—literally marks on a two-dimensional surface—do important work in establishing what IVM is and what it can do. A combination of visual semiotics and narrative analysis shows that images of IVM afford readings of their story that are co-created by the viewer. Before the cultured burger, during 2011, images of IVM fell into four distinct categories: cell images, tissue images, flowcharts, and meat in a dish images. The narrative infrastructure of each image type affords different interpretations of what IVM can accomplish and what it is. The 2013 cultured burger images both draw upon and depart from these image types in an attempt to present IVM as a normal food stuff, and as ‘matter in place’ when placed on the plate. The analysis of individual images and the collection of images about a certain object or subject—known as the imagescape—is a productive approach to understanding the ontology and promise of IVM and is applicable to other areas of social life.

Reproducibility in the Assessment of HER2 DISH in Breast Cancer

OBJECTIVE: HER2 gene status is an important biological marker that determines both the prognosis and response to Trastuzumab in patients with breast cancers. Although FISH assay has long been used to determine the HER2 gene copy number, DISH is gaining popularity since it can be assessed with a conventional microscope. The DISH test relies on an individual inspection however, so inter-observer variation may be an issue. MATERIALS AND METHODS: In the current study, reproducibility in the assessment of HER2 DISH test was evaluated in 69 breast cancer samples with HER2 IHC score 2+ and 3+. Two independent investigators evaluated the digitallycaptured images of DISH, without the knowledge of HER2 IHC status. RESULTS: The HER2/CEP17 ratio obtained from both observers were highly correlated (Rho = 0.883, p < 0.001). Based on the ASCO 2013 guideline, agreement of both investigators for HER2 gene diagnosis was 92.75 % (ĸ = 0.848). The HER2/CEP17 ratio of the discordant cases ranged from 1.6 (non-amplifi ed HER2) to 2.6 (low level of HER2 gene amplifi cation). CONCLUSION: It is concluded that the DISH method produces good reproducibility between independent observers, and the result also implies that the HER2 gene copy number is fairly homogenous across an individual breast cancer sample.

EXTENDED HCN AND HCO+EMISSION IN THE STARBURST GALAXY M82

We mapped 3 mm continuum and line emission from the starburst galaxy M82 using the Combined Array for Research in Millimeter-wave Astronomy. We targeted the HCN, HCO$^{+}$, HNC, CS and HC$_{3}$N lines, but here we focus on the HCN and HCO$^{+}$ emission. The map covers a field of 1.2' with a ~5" resolution. The HCN and HCO$^{+}$ observations are combined with single dish images. The molecular gas in M82 had been previously found to be distributed in a molecular disk, coincident with the central starburst, and a galactic scale outflow which originates in the central starburst. With the new short spacings-corrected maps we derive some of the properties of the dense molecular gas in the base of the outflow. From the HCN and HCO$^{+}$ J=(1-0) line emission, and under the assumptions of the gas being optically thin and in local thermodynamic equilibrium, we place lower limits to the amount of dense molecular gas in the base of the outflow. The lower limits are $7\times10^{6}$ $M_{\odot}$ and $21\times10^{6}$ $M_{\odot}$, or $\gtrsim2\%$ of the total molecular mass in the outflow. The kinematics and spatial distribution of the dense gas outside the central starburst suggests that it is being expelled through chimneys. Assuming a constant outflow velocity, the derived outflow rate of dense molecular gas is $\geq0.3$ $M_{\odot}$ yr$^{-1}$, which would lower the starburst lifetime by $\geq5\%$. The energy required to expel this mass of dense gas is $(1-10)\times10^{52}$ erg.

Reproducibility in the assessment of HER2 DISH in breast cancer

HER2 gene status is an important biological marker that determines both the prognosis and response to Trastuzumab in patients with breast cancers. Although FISH assay has long been used to determine HER2 gene copy number, DISH is gaining popularity since it can be assessed with a conventional microscope. Because the DISH test relies on an individual inspection, inter-observer variation might be an issue. In the current study, reproducibility in the assessment of HER2 DISH test was evaluated in 69 breast cancer samples with HER2 IHC score 2+ and 3+. Two independent investigators evaluated the digitally-captured images of DISH (10–20 images per case), taken at x600 magnification, without the knowledge of HER2 IHC status. The HER2/CEP17 ratio obtained from both observers were highly correlated (R = 0.883, p

Detection and segmentation of virus plaque using HOG and SVM: toward automatic plaque assay.

Plaque assaying, measurement of the number, diameter, and area of plaques in a Petri dish image, is a standard procedure gauging the concentration of phage in biology. This paper presented a novel and effective method for implementing automatic plaque assaying. The method was mainly comprised of the following steps: In the training stage, after pre-processing the images for noise suppression, an initial training set was readied by sampling positive (with a plaque at the center) and negative (plaque-free) patches from the training images, and extracting the HOG features from each patch. The linear SVM classifier was trained in a self-learnt supervised learning strategy to avoid possible missing detection. Specifically, the training set which contained positive and negative patches sampled manually from training images was used to train the preliminary classifier which exhaustively searched the training images to predict the label for the unlabeled patches. The mislabeled patches were evaluated by experts and relabeled. And all the newly labeled patches and their corresponding HOG features were added to the initial training set to train the final classifier. In the testing stage, a sliding-window technique was first applied to the unseen image for obtaining HOG features, which were inputted into the classifier to predict whether the patch was positive. Second, a locally adaptive Otsu method was performed on the positive patches to segment the plaques. Finally, after removing the outliers, the parameters of the plaques were measured in the segmented plaques. The experimental results demonstrated that the accuracy of the proposed method was similar to the one measured manually by experts, but it took less than 30 seconds.

UNVEILING THE PHYSICAL PROPERTIES AND KINEMATICS OF MOLECULAR GAS IN THE ANTENNAE GALAXIES (NGC 4038/9) THROUGH HIGH-RESOLUTION CO (J= 3-2) OBSERVATIONS

We present a ~ 1" (100 pc) resolution 12CO (3-2) map of the nearby intermediate stage interacting galaxy pair NGC 4038/9 (the Antennae galaxies) obtained with the Submillimeter Array. We find that half the CO (3-2) emission originates in the overlap region where most of the tidally induced star formation had been previously found in shorter wavelength images, with the rest being centered on each of the nuclei. The gross distribution is consistent with lower resolution single dish images, but we show for the first time the detailed distribution of the warm and dense molecular gas across this galaxy pair at resolutions comparable to the size of a typical giant molecular complex. While we find that 58% (33/57) of the spatially resolved Giant Molecular Associations (GMAs; a few x 100 pc) are located in the overlap region, only \leqq 30% spatially coincides with the optically detected star clusters, suggesting that the bulk of the CO (3-2) emission traces the regions with very recent or near future star formation activity. The spatial distribution of the CO (3-2)/CO (1-0) integrated brightness temperature ratios mainly range between 0.3 and 0.8, which suggests that on average the CO (3-2) line in the Antennae is not completely thermalized and similar to the average values of nearby spirals. A higher ratio is seen in both nuclei and the southern complexes in the overlap region. Higher radiation field associated with intense star formation can account for the nucleus of NGC 4038 and the overlap region, but the nuclear region of NGC 4039 show relatively little star formation or AGN activities and cannot be easily explained. We show kinematical evidence that the high line ratio in NGC 4039 is possibly caused by gas inflow into the counter-rotating central disk.