Sensors support machine learning

Abstract

Food Science and TechnologyVolume 33, Issue 4 p. 20-23 FeaturesFree Access Sensors support machine learning First published: 13 December 2019 https://doi.org/10.1002/fsat.3304_6.xAboutSectionsPDF ToolsExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Nik Watson of the University of Nottingham discusses whether online sensors and machine learning can deliver industry 4.0 to the food and drink manufacturing sector. Manufacturing is experiencing the 4th industrial revolution, which is the use of Industrial Digital Technologies (IDTs) to produce new and existing products. Industrial digital technologies include sensors, robotics, the industrial internet of things (IoT), additive manufacturing, artificial intelligence, virtual and augmented reality, digital twins and cloud computing. At the heart of Industry 4.0 is the enhanced collection and use of data. Industry 4.0 is predicted to have a positive impact of over £450bn to UK manufacturing over the next ten years1, with benefits such as increased productivity and reduced costs and environmental impacts. But what does this mean for the UK's largest manufacturing sector, food and drink? The food and drink sector is characterised by producing high volumes of low value products and faces constant challenges in terms of productivity, environmental sustainability, safety and labour availability. Food and drink is different to many other manufacturing sectors as it is dominated by small and medium enterprises which do not always have the necessary resources and expertise to adopt new technologies. Online sensors are one of the main types of IDT forecast to see greater use within the food and drink sector as their cost is generally less than other IDTs, and they can often be easily integrated with legacy processing equipment. Online sensors Online sensors are devices capable of performing inline measurements during manufacturing processes. There are a variety of different sensor technologies ranging from simple thermocouples to advanced methods, such as X-ray. Sensors collect data which can be used by manufacturers to make evidence-based decisions on their products and processes and are an essential component of autonomous processes. Sensors are usually characterised by their sensing modality (e.g. electromagnetic) and mode of operation (e.g. spectroscopic). To operate effectively in a food and drink manufacturing environment, sensors must be able to operate non-invasively in real-time. Although a vast range of different sensing technologies have been used in the food and drink manufacturing sector, the techniques that have seen the most widespread use are visible and near-infrared optical methods2. Recently, hyperspectral imaging has been experiencing greater use due to its high level of spatial and spectral resolution. Different sensors have been used within the sector for a range of applications including food quality inspection2, detection of contaminants3 and optimisation of processes such as cleaning4. Online sensors have the potential to deliver significant benefits to the food and drink sector but several challenges remain, which need to be addressed by the academic and industrial communities: The cost of the sensors is often too large to justify the benefits they deliver, especially for SME manufacturers. Sensor technologies can often require specific expertise to operate and interpret the data they collect. Food ingredients are inherently variable and food products are often complex multi-component systems requiring advanced signal and data analysis methods to interpret the sensor measurements. Food and drink manufacturing facilities are challenging environments in which to perform reliable measurements. Ultrasonic techniques utilise high frequency mechanical waves and are an attractive sensing technology due to their low cost and ability to monitor opaque materials non-destructively. Ultrasonic sensors have been used for a variety of applications within food and drink manufacturing, such as monitoring food texture and emulsion stability5. As with other sensor technologies, ultrasonic methods require the development of new data analysis methods to relate the ultrasonic measurements to the material or process being monitored in the industrial environment. Machine learning methods require large data sets, which makes them ideally suited to numerous applications in the food and drink sector, especially manufacturers producing large volumes of identical or similar products on a daily basis. Machine learning Machine learning is a type of artificial intelligence that utilises predictive algorithms for classification or regression problems. Supervised machine learning methods, such as Artificial Neural Networks (ANN), Support Vector Machines (SVM) and Random Forests (RF), use input data to train the algorithms6. Input data can come from a variety of sources including features extracted from sensor measurements and other parameters, such as the measurement environment (e.g. temperature, humidity and light levels). Advantages of machine learning techniques include the volume and variety of data they can process and their ability to improve as more or better data becomes available. In addition, they do not require the development of inversion models, accounting for complexities, such as material motion and variable temperature, and can process data quickly once the initial training stage has been completed. Machine learning methods require large data sets, which makes them ideally suited to numerous applications in the food and drink sector, especially manufacturers producing large volumes of identical or similar products on a daily basis. The combination of sensors and machine learning has seen some use within food and drink manufacturing most notably in computer vision applications for food quality inspection2. Although machine learning methods appear to be the perfect methodology for analysing data from online sensors, there are several challenges and limitations that adopters should always be aware of. Machine learning techniques are not an ‘out of the box’ autonomous technology. They require skilled users to train, validate and test the models. They also require expert users to determine when models need retraining, for example when a food product undergoes a formulation change. Machine learning methods are often termed ‘black box’ models. This means they can utilise sensor data to predict if a product is of acceptable quality or not. However they do not deliver information on why a product was deemed unacceptable, which is often necessary to enable changes in the production conditions. This issue is also true when selecting the most suitable machine learning method to utilise. Users often try a range of different methods and report their prediction accuracy without any understanding of why different algorithms perform better for specific applications. Supervised machine learning methods also require labelled data for training, which can often be more problematic or costly to collect than the sensor data. If we think of the quality assessment problem, it would not be difficult for a particular sensor to autonomously perform quality assessment measurements on 10,000 products (e.g. biscuits or chicken fillets). However each of the 10,000 products would need to be assigned a label relating to its quality, which is a resource intensive human task. For large problems, such as this, semi-supervised methods should be investigated. When using sensors and machine learning to make predictions in food and drink manufacturing environments, the prediction needs to be considered within the context of the specific application. For example, a prediction accuracy of 95% for determining the quality of biscuits may be acceptable and better than current industry standard methods (operator assessment). However is 95% prediction accuracy acceptable when trying to detect the presence of food allergens or foreign bodies in products? The final considerations users of machine learning need to be aware of are overfitting and generalisation. These issues relate to the predictions matching the training data too closely and the model's ability to make predictions on unseen data. Case studies Below are several case studies demonstrating the use of different sensors and machine learning methods within food and drink manufacturing. These aim to show the potential of the technologies and highlight some of the challenges discussed above. BREWNET The University of Nottingham is collaborating with the University of Leeds and Totally Brewed (SME craft brewer) to demonstrate how low cost ultrasonic sensors and machine learning can be used to monitor craft beer fermentation processes. Real-time sensor measurements would enable brewers to predict the optimal time to end the fermentation in addition to identifying any problems occurring during the process. The ABV % prediction from the ultrasonic sensor and machine learning models can be seen in Figure 1. Figure 1Open in figure viewerPowerPoint above left: How sensor measurements and machine learning can be used to predict the ABV % during beer fermentation. Above right: How the root mean square error reduces when more training data is available The results on the left show predictions from an ANN and a linear regression model. Both methods compare well to the actual ABV % value (recorded via typical sampling methods) but the ANN delivers more accurate predictions. Collecting a representative data set is challenging for some industrial applications. In craft brewing a particular beer may only be brewed once or twice a month. Although numerous ultrasonic measurements can be made from a single batch, data from numerous batches is desirable to create a data set more representative of the system. Figure 1 demonstrates this by showing how the error in the predicted values reduces as data collected from more batches is used to train the models. IoT enhanced factory cleaning The University of Nottingham is working on several projects investigating the use of sensors, data and robotics to transform cleaning and allergen detection within food factories. As part of this work researchers have been investigating the use of small and low cost Near-Infrared (NIR) sensors and supervised machine learning to identity different powdered foods containing known allergens. NIR spectra were recorded from over 50 different powdered foods and different machine learning algorithms were tested to determine their capabilities. The K-nearest neighbour (KNN) method was found to have the best classification accuracy with results over 98%. This may seem like a suitable method but detecting allergen-containing foods within production environments is a safety critical process and the responsible use of predictive algorithms needs to be considered in detail. Figure 2 displays principle components calculated from the recorded NIR spectra. Principle component analysis is a method for feature extraction from data and can be used to effectively visualise key data features. Figure 2Open in figure viewerPowerPoint left, Principle components calculated from NIR spectra of different powdered foods containing allergens Clean-in-place optimisation Cleaning of processing equipment is a critical operation within food and drink manufacturing but comes with a significant economic and environmental cost. The University of Nottingham has been working with Loughborough University and several industrial partners, funded through Innovate UK projects, to develop an intelligent multi-sensor technology to monitor the removal of surface fouling during cleaning of processing equipment4, 7. Trials have been performed at laboratory, pilot and full production scale to determine the most suitable sensor configurations and machine learning methods. In this project, several different machine learning methods were studied to determine their performance for predicting the presence of fouling using measurements detected by ultrasonic sensors. A test section with transparent sides was built so images could also be recorded during cleaning to help train and test the models. The results from the images (target data set) and predictions from the different machine learning models (K-nearest neighbour, support vector classifier, random forest, adaboost) can be seen in Figure 3. The results show that all machine learning methods give acceptable predictions (>99%) except the support vector classifier. Although the results have enabled the research team to determine the most suitable machine learning methods, this was only achieved via a trial and error approach, which is often the case when using machine learning. Figure 3Open in figure viewerPowerPoint Classification accuracy results for a range of machine learning methods. The algorithms were trained on data recorded using ultrasonic sensors whilst images were used as the target data Olive Oil Quality Assessment Olive oil has been produced for at least 8000 years and plays an important role in diet and health. Extra virgin olive oil is a high-value and vulnerable food product, which regularly tops the list of foods most at risk of fraud in the EU8. In the UK, government testing recently found that one third of olive oils sold in Britain are adulterated or breach quality standards9. Optical spectroscopy is embedded in the EU regulations for olive oil quality, with ultraviolet absorbance measurements indicating the level of oxidation in the sample10. Additionally, fluorescence excitation-emission matrices (Figure 4) show significant differences between extra virgin and refined olive oil. Figure 4Open in figure viewerPowerPoint Optical spectroscopy results from refined (left) and extra virgin (right) olive oil With funding from Innovate UK and EU H2020, Liquid Vision Innovation has developed an optical sensor that directly reports both the level of secondary oxidation and key fluorescence properties of olive oil samples. Their approach is direct (chemical-free), thus enabling on-site monitoring of key quality parameters of olive oil, creating opportunities for process optimisation at the mill and increased sampling frequency through the supply chain. Summary There is no doubt that food and drink manufacturing will experience an increased use of online sensors and machine learning methods in the future. However, for an effective uptake of this technology, there is a need for more affordable and easy to use sensors capable of measuring key quality parameters in ingredients and final products. Machine learning has great potential within the food and drink sector but there are many current limitations of the techniques which must always be considered for any new application. Nik Watson, Assistant Professor Faculty of Engineering, University of Nottingham, Nottingham NG7 2RD Acknowledgements The author would like to thank the numerous funders for their research (Innovate UK, EPSRC Network Plus: Connected Everything, EPSRC Network Plus: Internet of Food Things, EPSRC Horizon Digital Economy Research, and University of Nottingham Smart Products Beacon) and academic and industrial collaborators. Email nicholas.watson@nottingham.ac.uk REFERENCES 1 Made smarter review 2017, (https://www.gov.uk/government/publications/made-smarter-review) 2Sun, D. 2012. Computer Vision Technology in the Food and Beverage Industries. Woodhead Publishing. 3 Knowledge Transfer Network, 2014, Food Sensing Report. 4Escrig, E., Woolley, E., Rangappa, S., Simeone, A., Watson, N.J. 2014. Food Control 104: 358 5Awad, T.S., Moharram, H.A., Shaltout, O.E., Asker, D., Youssef, M.M. 2012. Applications of ultrasound in analysis, processing and quality control of food: a review. Food Research International 48: 410– 427 6Goodfellow, I., Bengio, Y., Courville, A. 2016. Deep Learning. MIT Press. 7Simeone, A., Deng, B., Watson, N., Woolley, E. 2018. Enhanced clean-In-place monitoring using ultraviolet induced fluorescence and neural networks. Sensors 18: 11 8 European Parliament, Committee on the Environment, Public Health and Food Safety, Draft Report on the food crisis, fraud in the food chain and the control thereof (2013/2091(INI)) http://www.europarl.europa.eu/sides/getDoc.do?pubRef=-//EP//NONSGML+COMPARL+PE-519.759+02+DOC+PDF+V0//EN&language=EN 9 The Times, One in three bottles of olive oil sold in Britain are fake or of poor quality, 27 August 2017 https://www.thetimes.co.uk/article/one-in-three-bottles-of-olive-oil-sold-in-britain-are-fake-or-of-poor-quality-3j9xkxlpn 10 European Commission, Commission Regulation (EEC) No 2568/91 of 11 July 1991 on the characteristics of olive oil and olive-residue oil and on the relevant methods of analysis, Official Journal of the European Union, L248, 1– 127, 1991. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A01991R2568-20151016 Volume33, Issue4December 2019Pages 20-23 FiguresReferencesRelatedInformation