Species Identification Task Research Articles

The Impact of Imperfect XAI on Human-AI Decision-Making

Explainability techniques are rapidly being developed to improve human-AI decision-making across various cooperative work settings. Consequently, previous research has evaluated how decision-makers collaborate with imperfect AI by investigating appropriate reliance and task performance with the aim of designing more human-centered computer-supported collaborative tools. Several human-centered explainable AI (XAI) techniques have been proposed in hopes of improving decision-makers' collaboration with AI; however, these techniques are grounded in findings from previous studies that primarily focus on the impact of incorrect AI advice. Few studies acknowledge the possibility of the explanations being incorrect even if the AI advice is correct. Thus, it is crucial to understand how imperfect XAI affects human-AI decision-making. In this work, we contribute a robust, mixed-methods user study with 136 participants to evaluate how incorrect explanations influence humans' decision-making behavior in a bird species identification task, taking into account their level of expertise and an explanation's level of assertiveness. Our findings reveal the influence of imperfect XAI and humans' level of expertise on their reliance on AI and human-AI team performance. We also discuss how explanations can deceive decision-makers during human-AI collaboration. Hence, we shed light on the impacts of imperfect XAI in the field of computer-supported cooperative work and provide guidelines for designers of human-AI collaboration systems.

Animal Species Recognition with Deep Convolutional Neural Networks from Ecological Camera Trap Images.

Accurate identification of animal species is necessary to understand biodiversity richness, monitor endangered species, and study the impact of climate change on species distribution within a specific region. Camera traps represent a passive monitoring technique that generates millions of ecological images. The vast numbers of images drive automated ecological analysis as essential, given that manual assessment of large datasets is laborious, time-consuming, and expensive. Deep learning networks have been advanced in the last few years to solve object and species identification tasks in the computer vision domain, providing state-of-the-art results. In our work, we trained and tested machine learning models to classify three animal groups (snakes, lizards, and toads) from camera trap images. We experimented with two pretrained models, VGG16 and ResNet50, and a self-trained convolutional neural network (CNN-1) with varying CNN layers and augmentation parameters. For multiclassification, CNN-1 achieved 72% accuracy, whereas VGG16 reached 87%, and ResNet50 attained 86% accuracy. These results demonstrate that the transfer learning approach outperforms the self-trained model performance. The models showed promising results in identifying species, especially those with challenging body sizes and vegetation.

Consensus Building in On-Line Citizen Science

A number of initiatives invite members of the public to perform online classification tasks such as identifying objects in images. These tasks are crucial to numerous large-scale Citizen Science projects in different disciplines, with volunteers using their knowledge and online support tools to, for example, identify species of wildlife or classify galaxies by their shapes. However, for complex classification tasks, such as this case study on identifying species of bumblebee, reaching an agreement between volunteers - or even between experts~-~may require consensus-building processes. Collaboration and teamwork approaches to problem solving and decision-making have been widely documented to improve both task performance and user learning in the real world. Most of these processes and projects are mediated online through feedback delivered in an asynchronous manner, and this article thus addresses a central research question: How do participants involved in species identification tasks respond to different forms of feedback provided in online collaboration, designed to support peer-learning and improve task performance? We tested four different approaches to feedback within a collaboration task, where participants reviewed their previously annotated data based on information curated from their peers on a long running online citizen science initiative. The selected interfaces have a strong foundation in social science and psychology literature and can be applied to citizen science practices as well as other online communities. Results showed that while all four approaches increased accuracy, there were differences based on the types of consensus that existed before collaboration. Such differences highlight the usefulness of different forms of feedback during collaboration for increasing data accuracy of identification and furthering users' expertise on identification tasks. We found that anonymised and goal-directed free text comments posted on social learning interfaces were most effective in improving data accuracy as well as creating opportunities for peer-learning, particularly where the species identification task was more difficult. This study has significant implications for extending the practice of citizen science across formal and informal learning environments and reaching out to a variety of users.

Automated Real-Time Identification of Medicinal Plants Species in Natural Environment Using Deep Learning Models-A Case Study from Borneo Region.

The identification of plant species is fundamental for the effective study and management of biodiversity. In a manual identification process, different characteristics of plants are measured as identification keys which are examined sequentially and adaptively to identify plant species. However, the manual process is laborious and time-consuming. Recently, technological development has called for more efficient methods to meet species’ identification requirements, such as developing digital-image-processing and pattern-recognition techniques. Despite several existing studies, there are still challenges in automating the identification of plant species accurately. This study proposed designing and developing an automated real-time plant species identification system of medicinal plants found across the Borneo region. The system is composed of a computer vision system that is used for training and testing a deep learning model, a knowledge base that acts as a dynamic database for storing plant images, together with auxiliary data, and a front-end mobile application as a user interface to the identification and feedback system. For the plant species identification task, an EfficientNet-B1-based deep learning model was adapted and trained/tested on a combined public and private plant species dataset. The proposed model achieved 87% and 84% Top-1 accuracies on a test set for the private and public datasets, respectively, which is more than a 10% accuracy improvement compared to the baseline model. During real-time system testing on the actual samples, using our mobile application, the accuracy slightly dropped to 78.5% (Top-1) and 82.6% (Top-5), which may be related to training data and testing conditions variability. A unique feature of the study is the provision of crowdsourcing feedback and geo-mapping of the species in the Borneo region, with the help of the mobile application. Nevertheless, the proposed system showed a promising direction toward real-time plant species identification system.

Species Detection and Segmentation of Multi-specimen Historical Herbaria

Historically, herbarium specimens have provided users with documented occurrences of plants in specific locations over time. Herbarium collections have therefore been the basis of systematic botany for centuries (Younis et al. 2020). According to the latest summary report based on the data from Index Herbariorum, there are around 3400 active herbaria in the world containing 397 million specimens that are spread across 182 countries (Thiers 2021). Exponential growth in high quality image capturing devices induced by the enormous amount of uncovered collections has further led to rising interest in large scale digitization initiatives across the world (Le Bras et al. 2017). As herbarium specimens are increasingly becoming digitised and accessible in online repositories, an important need has also emerged to develop automated tools to process and enrich these collections to facilitate better access to the preserved archives. This rising number of digitised herbarium sheets provides an opportunity to employ computer-based image processing techniques, such as deep learning, to automatically identify species and higher taxa (Carranza-Rojas and Joly 2018, Carranza-Rojas et al. 2017, Younis et al. 2020) or to extract other useful information from the herbaria sheets, such as detecting handwritten text, color bars, scales and barcodes. The species identification task works well for herbarium sheets that have only one species in a page. However, there are many herbarium books that have multiple species on the same page (as shown in Fig. 1) for which the complexity of the identification problem increases tremendously. It also involves a great deal of time and effort if they are to be enriched manually. In this work, we propose a pipeline that can automatically detect, identify, and enrich plant species in multi-specimen herbaria. The core idea of the pipeline is to detect unique plant species and handwritten text around the plant species and map the text to the correct plant species. As shown in Fig. 2, the proposed pipeline begins with the pre-processing of the images. The images are rotated and aligned such that the longest edge is maintained as its height. In the case of herbarium books, the pages are detected and morphological transformations are performed to reduce occlusions (Thirukokaranam Chandrasekar and Verstockt 2020). A YOLOv3 (You Only Look Once version 3) object detection model (Zhao and Li 2020) is trained from scratch to detect plants and text. The model was trained on a dataset of single species herbarium sheets with a mosaic augmentation technique to extend the plants model to detect multiple species. The first results of the training shows impressive results although it could be further improved with more labelled data. We also plan to train an object segmentation model and contrast its performance with the plant detection model for multi-specimen herbarium sheets. After detecting both the plants and the text, the text will be recognized with a state-of-the-art handwritten text recognition (HTR) model. The recognized text can then be matched with a database of specimens, to identify each detected specimen. Furthermore, additional textual metadata (e.g. date, locality, collector's name, institution) visible on the sheet will be recognized and used to enrich the collection.

Plant taxonomy-guided path-based tree classifier for large-scale plant species identification

A deep learning framework is proposed to recognize large-scale plant species by integrating attention-based deep feature extraction network and plant taxonomy-guided path-based tree classifier. First, a plant taxonomy is constructed for organizing large-scale fine-grained plant species hierarchically in a coarse-to-fine fashion. Second, a deep learning framework is proposed, where attention mechanism is used to remove useless feature components and a plant taxonomy-guided path-based two-layer tree classifier is used to replace the flat softmax classifier in traditional deep convolutional neural network structure. Furthermore, a specific path-based loss function and back-propagation method are proposed to optimize the weight parameters in both deep network and tree classifier. Experimental results on the Orchid2608 plant dataset can also prove that proposed deep attention network with path-based tree classifier can achieve improvements on large-scale plant species identification task.

Learning niche features to improve image-based species identification

Species identification is a critical task of ecological research. Having accurate and intelligent methods of species identification would improve our ability to study and conserve biodiversity with saving much time and effort. But image-based deep learning methods have rarely taken account of domain knowledge, and perform poorly on imbalanced training dataset and similar species. Here, we propose NicheNet which combines ecological niche model and image-based deep learning model together expanding from the joint model framework invented by previous research. We incorporate an optimization process for NicheNet, and examine its performance on identifying Chinese Galliformes comparing with image-only model and Geo-prior-based model. For assessing the ability of both NicheNet and image-only models on distinguishing similar species, we initiate a new criterion named Near Error Rate in this study which decomposes identification error rate on each species along its classification. We show that the joint models gain significantly improvement on identifying our study species compared with image-only model. Against on an image-only baseline 82.5%, we observe 7.73% improvement in top-1 accuracy for NicheNet, and a 6.21% increase for Geo-prior-based model. NicheNet gains a 15.3% increment of average F1-Score and gets a − 8.5% mean decrement in Near Species Error Rate while −4.4% in Near Genus Error Rate. Further cases analysis shows the background mechanism of NicheNet to improve image-only models. We demonstrate that NicheNet can learn the features from images and niche-prior, which is generated by niche model and finer than geo-prior by Geo-prior-based model, together to promote its ability on identifying species, especially on those with small training dataset and similar outlook. It is a flexible model framework, and we can introduce more biological features and domain models to strengthen its accuracy and robust in the future work. Our study shows that NicheNet can be widely adopted and will accurately accelerate automatic species identification task for biodiversity research and conservation.

Designing online species identification tools for biological recording: the impact on data quality and citizen science learning.

In recent years, the number and scale of environmental citizen science programmes that involve lay people in scientific research have increased rapidly. Many of these initiatives are concerned with the recording and identification of species, processes which are increasingly mediated through digital interfaces. Here, we address the growing need to understand the particular role of digital identification tools, both in generating scientific data and in supporting learning by lay people engaged in citizen science activities pertaining to biological recording communities. Starting from two well-known identification tools, namely identification keys and field guides, this study focuses on the decision-making and quality of learning processes underlying species identification tasks, by comparing three digital interfaces designed to identify bumblebee species. The three interfaces varied with respect to whether species were directly compared or filtered by matching on visual features; and whether the order of filters was directed by the interface or a user-driven open choice. A concurrent mixed-methods approach was adopted to compare how these different interfaces affected the ability of participants to make correct and quick species identifications, and to better understand how participants learned through using these interfaces. We found that the accuracy of identification and quality of learning were dependent upon the interface type, the difficulty of the specimen on the image being identified and the interaction between interface type and ‘image difficulty’. Specifically, interfaces based on filtering outperformed those based on direct visual comparison across all metrics, and an open choice of filters led to higher accuracy than the interface that directed the filtering. Our results have direct implications for the design of online identification technologies for biological recording, irrespective of whether the goal is to collect higher quality citizen science data, or to support user learning and engagement in these communities of practice.

Combining high-throughput imaging flow cytometry and deep learning for efficient species and life-cycle stage identification of phytoplankton

BackgroundPhytoplankton species identification and counting is a crucial step of water quality assessment. Especially drinking water reservoirs, bathing and ballast water need to be regularly monitored for harmful species. In times of multiple environmental threats like eutrophication, climate warming and introduction of invasive species more intensive monitoring would be helpful to develop adequate measures. However, traditional methods such as microscopic counting by experts or high throughput flow cytometry based on scattering and fluorescence signals are either too time-consuming or inaccurate for species identification tasks. The combination of high qualitative microscopy with high throughput and latest development in machine learning techniques can overcome this hurdle.ResultsIn this study, image based cytometry was used to collect ~ 47,000 images for brightfield and Chl a fluorescence at 60× magnification for nine common freshwater species of nano- and micro-phytoplankton. A deep neuronal network trained on these images was applied to identify the species and the corresponding life cycle stage during the batch cultivation. The results show the high potential of this approach, where species identity and their respective life cycle stage could be predicted with a high accuracy of 97%.ConclusionsThese findings could pave the way for reliable and fast phytoplankton species determination of indicator species as a crucial step in water quality assessment.

Employing fingerprinting of medicinal plants by means of LC-MS and machine learning for species identification task

A dataset of liquid chromatography-mass spectrometry measurements of medicinal plant extracts from 74 species was generated and used for training and validating plant species identification algorithms. Various strategies for data handling and feature space extraction were tested. Constrained Tucker decomposition, large-scale (more than 1500 variables) discrete Bayesian Networks and autoencoder based dimensionality reduction coupled with continuous Bayes classifier and logistic regression were optimized to achieve the best accuracy. Even with elimination of all retention time values accuracies of up to around 85% were achieved on validation set for plant species and plant organ identification. Benefits and drawbacks of used algortihms were discussed. Preliminary test showed that developed approaches exhibit tolerance to changes in data created by using different extraction methods and/or equipment. Dataset with more than 2200 chromatograms was published in an open repository.

Don't Know Much about Bumblebees?-A Study about Secondary School Students' Knowledge and Attitude Shows Educational Demand.

Many insects are threatened with extinction, which in the case of pollinating insects could lead to declining pollination services and reduced ecosystem biodiversity. This necessitates rethinking how we deal with nature in general. Schools are ideal places in which to instill a willingness to behave in an environmentally-friendly way. Whereas scientific studies and school textbooks stress the importance of honeybees as pollinators, the role of bumblebees is either underestimated or neglected. The aim of this study was to provide information concerning student knowledge and attitudes, which are important factors of an individual’s environmental awareness. A questionnaire with closed and open questions was developed, which also included drawing and species identification tasks. We surveyed 870 German secondary school students between 9 and 20 years of age. Our results indicate limited knowledge of bumblebees by students of all grades. Knowledge increased with higher grades but only with a small effect size. The attitude of students towards bumblebees was generally positive; however, this positivity declined with increasing grade of the participants. This correlation also had a small effect size. Our results are discussed, with a particular focus on future educational demand.

Towards automatic wild animal monitoring: Identification of animal species in camera-trap images using very deep convolutional neural networks

Non-intrusive monitoring of animals in the wild is possible using camera trapping networks. The cameras are triggered by sensors in order to disturb the animals as little as possible. This approach produces a high volume of data (in the order of thousands or millions of images) that demands laborious work to analyze both useless (incorrect detections, which are the most) and useful (images with presence of animals). In this work, we show that as soon as some obstacles are overcome, deep neural networks can cope with the problem of the automated species classification appropriately. As case of study, the most common 26 of 48 species from the Snapshot Serengeti (SSe) dataset were selected and the potential of the Very Deep Convolutional neural networks framework for the species identification task was analyzed. In the worst-case scenario (unbalanced training dataset containing empty images) the method reached 35.4% Top-1 and 60.4% Top-5 accuracy. For the best scenario (balanced dataset, images containing foreground animals only, and manually segmented) the accuracy reached a 88.9% Top-1 and 98.1% Top-5, respectively. To the best of our knowledge, this is the first published attempt on solving the automatic species recognition on the SSe dataset. In addition, a comparison with other approaches on a different dataset was carried out, showing that the architectures used in this work outperformed previous approaches. The limitations of the method, drawbacks, as well as new challenges in automatic camera-trap species classification are widely discussed.

SuperIDR: A Tool for Fish Identification and Information Retrieval

ABSTRACTStudents, fisheries professionals, and the general public may value computer‐facilitated assistance for fish identification and access to ecological and life history information. We developed SuperIDR, a software package supporting such applications, by utilizing the search and data retrieval capabilities of digital libraries, as well as key features of tablet PCs. We demonstrated SuperIDR utilizing a database with information on 207 freshwater fishes of Virginia. A user may annotate fish images and identify fishes by using a dichotomous key; searching for key words, similar images, subimages, or annotations on images; or combinations of these approaches. Students using the software demonstrated enhanced ability to correctly identify specimens. Their comments led to improvements, including the addition of new features. The PC‐based system for identifying freshwater fishes of Virginia may be downloaded and modified. SuperIDR is a prototype for PC‐based species identification applications—the system architecture and the open‐source software that we developed are applicable to other fish faunas and to a broader range of species identification tasks.

A Constructive Hybrid Structure Optimization Methodology for Radial Basis Probabilistic Neural Networks

In this paper, a novel heuristic structure optimization methodology for radial basis probabilistic neural networks (RBPNNs) is proposed. First, a minimum volume covering hyperspheres (MVCH) algorithm is proposed to select the initial hidden-layer centers of the RBPNN, and then the recursive orthogonal least square algorithm (ROLSA) combined with the particle swarm optimization (PSO) algorithm is adopted to further optimize the initial structure of the RBPNN. The proposed algorithms are evaluated through eight benchmark classification problems and two real-world application problems, a plant species identification task involving 50 plant species and a palmprint recognition task. Experimental results show that our proposed algorithm is feasible and efficient for the structure optimization of the RBPNN. The RBPNN achieves higher recognition rates and better classification efficiency than multilayer perceptron networks (MLPNs) and radial basis function neural networks (RBFNNs) in both tasks. Moreover, the experimental results illustrated that the generalization performance of the optimized RBPNN in the plant species identification task was markedly better than that of the optimized RBFNN.

Effectiveness of Reptile Species Identification - A Comparison of Dichotomous Key with an Identification Book

Species identification tasks are a prerequisite for an understanding of biodiversity. Here, we focused on different educational materials to foster the identification of six European reptile species. Our educational training unit was based on natural plastic models of six species and pupils either used an illustrated identification book or a dichotomous key. 71 secondary school pupils (6th graders) from four classes participated in our study and received a hands-on lesson about the identification of these species. 37 of them were aided by an identification book, and 34 received a dichotomous key. We found a significant increase in identification skills immediately after and with a delay of four weeks in both treatment groups. Further, we found no differences in achievement between both groups nor did we detect differences in the emotional variables (well-being, interest, boredom) or in the difficulty of the task. We propose that species identification should be taught in everyday school life using models and identification materials that allow a learner-centred environment.

Shape recognition based on neural networks trained by differential evolution algorithm

In this paper a new method for recognition of 2D occluded shapes based on neural networks using generalized differential evolution training algorithm is proposed. Firstly, a generalization strategy of differential evolution algorithm is introduced. And this global optimization algorithm is applied to train the multilayer perceptron neural networks. The proposed algorithms are evaluated through a plant species identification task involving 25 plant species. For this practical problem, a multiscale Fourier descriptors (MFDs) method is applied to the plant images to extract shape features. Finally, the experimental results show that our proposed GDE training method is feasible and efficient for large-scale shape recognition problem. Moreover, the experimental results illustrated that the GDE training algorithm combined with gradient-based training algorithms will achieve better convergence performance.

Cognitive achievements in identification skills

Species identification tasks are generally accepted as fundamental aspects of biodiversity education. Our educational training unit, therefore, focused on identification skills by introducing stuffed specimens in combination with identification books and preparation booklets.We limited the number of bird species to six. 492 secondary school pupils (163 from 8th grade [high stratification level] and 329 from 9th grade [medium stratification level]) participated in the study. Instruction-1 pupils received a hands-on lesson about the natural history of six bird species, while Instruction-2 pupils were taught in a conventional teacher-centred manner. Pupils from the hands-on classes achieved significantly better grades in two follow-up tests. The measure of well-being was significantly enhanced by the teaching technique, while interest, anxiety and boredom were not affected. Additionally, pupils appreciated the opportunity of self-regulation. In general, a shift from the unspecific to the specific was observed: for instance, from duck and gull to mallard and black-headed gull. A reduction in the number of species presented to pupils within a lesson seems to enhance learning and retention effects.

A novel full structure optimization algorithm for radial basis probabilistic neural networks

In this paper, a novel full structure optimization algorithm for radial basis probabilistic neural networks (RBPNN) is proposed. Firstly, a minimum volume covering hyperspheres (MVCH) algorithm is proposed to heuristically select the initial hidden layer centers of the RBPNN, and then the recursive orthogonal least square (ROLS) algorithm combined with the particle swarm optimization (PSO) algorithm is adopted to further optimize the initial structure of the RBPNN. Finally, the effectiveness and efficiency of our proposed algorithm are evaluated through a plant species identification task involving 50 plant species.

Complementary user‐centered methodologies for information seeking and use: System's design in the Biological Information Browsing Environment (BIBE)

AbstractComplementary, socially grounded, user‐centered methodologies are being used to design new information systems to support biodiversity informatics. Each of the methods—interviews, focus groups, field observations, immersion, and lab testing—has its own strengths and weaknesses. Methods vary in their ability to reveal the automatic processes of experts (that need to be learned by novices), data richness, and their ability to help interpret complex information needs and processes. When applied in concert, the methods provide a much clearer picture of the use of information while performing a real life information‐mediated task. This picture will be used to help inform the design of a new information system, Biological Information Browsing Environment (BIBE). The groups being studied are high school students, teachers, and volunteer adult groups performing biodiversity surveys. In this task the people must identify and record information about many species of flora and fauna. Most of the information tools they use for training and during the survey are designed to facilitate the difficult species identification task.