Microsoft Cognitive Services Research Articles

College Chatbot

Creating a chatbot to be used by the students to get their queries responded easily from the college website. The College Chatbot has the capacity to make friendly conversations; respond the course and faculty details; give the link for the academic calendar; answer the frequently asked questions; calculate the fees based on the student's input; and give the timings, address, contacts, and events information of the departments like Union and Library. To build the chatbot, Microsoft Azure bot service as well as Microsoft cognitive services, namely, Text Analytics, LUIS, and QnA Maker are used. In order to address these problems, the College Enquiry Chatbot extends the implementation of the current chatbots by adding sentiment analysis and active learning

Assessing the association between urban features and human physiological stress response using wearable sensors in different urban contexts.

Public open space (POS) plays a significant role in fostering human health and wellbeing in cities. A major limitation of current research on POS and health is that there is little attention on the role of various urban features on people's mental health, in different urban context. This study employed wearable sensors (a wearable camera, Empatica 4 wristband and a GPS device) to measure human physiological responses to urban indicators, objectively. To do this, we selected six kinds of public open space (water area, transit area, green area, commercial area, motor traffic area and mixed office and residential area) and recruited 86 participants for an experimental study. Next, we detected urban features by using Microsoft Cognitive Services (MCS) and calculated a change score to assess human physiological stress responses based on galvanic skin response (GSR) and skin temperature from the wristband. Lastly, we applied random effect model and geographically weighted regression analysis to examine the relationship between urban indicators and human physiological stress responses. The findings show that urban flow (vehicles, bikes and people), waterbodies, greenery and places to sit are associated with the changes of human physiological stress response. The findings indicate that the type of urban context may confound the effect of green and blue urban features; i.e., the effect on physiological stress response can be positive or negative depending on the context. The paper highlights the relevance of considering urban context in research on associations between urban features and stress response.

Enhancing Optical Character Recognition on Images with Mixed Text Using Semantic Segmentation

Optical Character Recognition has made large strides in the field of recognizing printed and properly formatted text. However, the effort attributed to developing systems that are able to reliably apply OCR to both printed as well as handwritten text simultaneously, such as hand-filled forms, is lackadaisical. As Machine printed/typed text follows specific formats and fonts while handwritten texts are variable and non-uniform, it is very hard to classify and recognize using traditional OCR only. A pre-processing methodology employing semantic segmentation to identify, segment and crop boxes containing relevant text on a given image in order to improve the results of conventional online-available OCR engines is proposed here. In this paper, the authors have also provided a comparison of popular OCR engines like Microsoft Cognitive Services, Google Cloud Vision and AWS recognitions. We have proposed a pixel-wise classification technique to accurately identify the area of an image containing relevant text, to feed them to a conventional OCR engine in the hopes of improving the quality of the output. The proposed methodology also supports the digitization of mixed typed text documents with amended performance. The experimental study shows that the proposed pipeline architecture provides reliable and quality inputs through complex image preprocessing to Conventional OCR, which results in better accuracy and improved performance.

Xamã : Optical character recognition for multi-domain model management

The development of systems following model-driven engineering can include models from different domains. For example, to develop a mechatronic component one might need to combine expertise about mechanics, electronics, and software. Although these models belong to different domains, the changes in one model can affect other models causing inconsistencies in the entire system. Only few tools, however, support management of models from different domains. Indeed, these models are created using different modeling notations and it is not plausible to use a multitude of parsers geared toward each and every modeling notation. Therefore, to ensure maintenance of multi-domain systems, we need a uniform approach that would be independent from the peculiarities of the notation. Notation independence implies that such a uniform approach can only be based on elements commonly present in models of different domains, i.e., text, boxes, and lines. In this study, we investigate the suitability of optical character recognition (OCR) as a basis for such a uniformed approach. We select graphical models from various domains that typically combine textual and graphical elements. We start by analyzing the performance of Google Cloud Vision and Microsoft Cognitive Services, two off-the-shelf OCR services. Google Cloud Vision performed better than Microsoft Cognitive Services being able to detect text of 70% of model elements. Errors made by Google Cloud Vision are due to absence of support for text common in engineering formulas, e.g., Greek letters, equations, and subscripts. We identified the multi-line text error as one of the main issues of using OCR to recognize textual elements in models from different domains. This error happens when OCR misinterprets one textual element as two separate elements. To address the multi-line text error, we build Xamã on top of Google Cloud Vision. Xamã includes two approaches to identify whether the elements are positioned on a single line or multiple lines, and merge those identified as positioned on multiples lines. With and without shape detection, Xamã correctly identified 956 and 905 elements, respectively, out of 1171. Additionally, we compared the accuracy of Xamã and state-of-the-art tool img2UML, and we observe that Xamã outperformed img2UML in both precision and recall, being able to recognize 433 out of 614 textual elements as opposed to 171 by img2UML.

AI-Based Chatbot to Solve Modern-Day Enterprise Business Problems

Communicating with employees through forums and emails has become an increasingly popular way for many multinational companies to provide human resource services in real time. Today, employee chat service agents are often replaced by conversational software agents or chatbots. These systems are designed to communicate with human users through natural language, generally based on artificial intelligence (AI). Time and cost saving opportunities have led to the widespread deployment of AI-based chatbots. Chatbots are one of the most basic and popular examples of human-computer intelligent interaction (HCI). Designed to convincingly simulate the way humans behave as dialogue partners. In the proposed system, we propose a chat robot that can dynamically respond to employee human resource queries. The proposed HR system is based on the Microsoft Cognitive Services chatbot. This Microsoft Teams-based platform provides a broad foundation of intelligence and is trained based on various data sets provided by the organization's HR.

Facial attendance system technology using Microsoft Cognitive Services

Artificial intelligence is a necessity in the current times as it makes processes more economical and affordable in the long run. It also frees humans from performing banal tasks day in and day out. Once a process is automated the only check that is to be performed is whether it is turned on or not. Artificial intelligence is an automated process which is not prone to errors and even if an error is identified rectification is easy and can be applied system-wide without any delay. Taking attendance is a mundane but necessary task in many academic institutions. In this paper, an alternate and more efficient method of taking attendance is proposed that uses face recognition and IoT technology with the help of cognitive services. Facial attendance system technology is a combination of front end technology like angular, IoT device like Raspberry Pi and its peripherals, back end technology with Node.js framework, cloud technology like Firebase and Microsoft Cognitive Services. The main objective of using Microsoft Cognitive Services is for accuracy and fast development of the system. The process from face detection to face recognition is done in minimal span of time with the help of cognitive services.

Assessing personal exposure to urban greenery using wearable cameras and machine learning

Urban greenery is closely related to people's behaviour. With the advancement of science and technology in Artificial Intelligence, wearable sensors and cloud computing, the potential for studying the relationship between people and urban greenery through new data and technology is constantly being explored, such as assessing population exposure to urban greenery using multi-source big data. Taking one individual participant as a case study, this paper proposes and validates the effectiveness of using wearable camera (Narrative Clip 2) and machine learning (Applications Programming Interface of Microsoft Cognitive Service) to assess personal exposure to urban greenery. Microsoft API is used to identify urban greenery tags, including “flower”, “forest”, “garden”, “grass”, “green”, “plant”, “scene” and “tree”, in personal images taken by the wearable camera. Personal exposure to urban greenery is assessed by calculating the frequency of the urban greenery tags in all the images taken. Furthermore, the overall evaluation and regularity of personal exposure to urban greenery (including “static exposure” and “dynamic exposure”) are explored to identify the characteristics of individual's greenery lifelogging. This study makes a brave attempt that may contribute a new perspective in applying personal big data in studying individual behaviour.

Artificial Intelligence on Information Services

Artificial Intelligence as a Service (AIaaS) is the outsourcing of an external artificial intelligence (AI) company. AI as a service allows people and companies to try AI for a variety of purposes without much initial investment and little risk. The test could allow for the sampling of multiple public cloud platforms to explore different ways of machine learning. Different AI provider platforms offer a wide range of machine learning and AI styles. This diversity can best fit the organizational needs of AI because organizations need to evaluate features and price to see what works for you. Cloud AI service providers can provide the special hardware needed for other AI tasks, such as GPU-based processing of deep workloads. Buying the hardware and software needed to launch an AI cloud is expensive. Combined with labor and repair costs, as well as hardware changes for different operations, it makes AIaaS costs deter many organizations. AI cloud computing including Amazon Machine Learning, Microsoft Cognitive Services and Google Cloud Machine Learning can help potential organizations with their data. Having the opportunity to try out algorithms and the services of various providers can allow businesses to find out what works and allow measurement before committing. When something measuring is found on the scale, the resources of these major providers are available to back up your computer-assisted scale.

Usage of Artificial Vision Cloud Services as Building Blocks for Blind People Assistive Systems

This study has the objective of select the best service at image processing and recognition, running in the cloud, and best suited for usage in systems to aid and improve the daily lives of blind people. To accomplish this purpose, a set of candidate services was built, including Microsoft Cognitive Services and Google Cloud Vision. A test mobile app was developed to automatically take pictures, which are sent to the online cloud services for processing. The results and the functionalities were evaluated with the aim to measure their accuracy and relevance. The following variables were registered: relative accuracy, represented by the ratio of the number of accurate results vs. the number of results shown; confidence degree, representing the service accuracy (when provided by the service); and relevance, identifying situations that can be useful in the daily lives of the blind people. The results have shown that these two services, Microsoft Cognitive Services and Google Cloud Vision, provided good accuracy and significance, in supporting systems to help blind people in their daily tasks. It was chosen some functionalities in two APIs of services running in the cloud like face identification, image description, objects, and text recognition.

Robotic Automation Process – The operating system for the digital enterprise

Abstract Robotic Process Automation (RPA) is going into a “maturity market”. The main vendor providers surpassed USD 1 billion in evaluation and the research they are launching these days on the market will change again radically the business landscape. It can be seen already what is coming next to RPA: intelligent optical character recognition (IOCR), chat-bots, machine learning, big data analytics, cognitive platforms, anomaly detection, pattern analysis, voice recognition, data classification and many more. As a result the top vendors developed partnerships with the main leading artificial intelligence providers, such as: IBM Watson, Microsoft Artificial Intelligence, Microsoft Cognitive services, blockchain, Google etc. On the business part, the consulting companies who are implementing the RPA solution are moving from developing Proof-of-Concepts (POCs) and Pilots to helping clients with RAP global roll-outs and developing Centre of Excellences (CoE). As a result, the experiences gathered so far by the author on this kind of projects will be tackled also in this paper. In this article will we will present also some data related to automation for different business areas (eg. Accounts Payable, Accounts Receivable etc) and how an assessment can be done correctly in order to decide if a process can be automatized and, if yes, up to which extent (ie. percent). Moreover, through the case studies we will provide (1) how now the RPA is integrated with Artificial Intelligence and Cloud, (2) how can be scaled in order to face hypes, (3) how can interpret data and (4) what savings these technologies can bring to the organizations. All the aforementioned services made Robotics Process Automation a very powerful tool since a year ago when the author did the last research. A process that was mainly not recommended for automation or was partially automated can be now fully automated with more advantages, such as: money, non-FTE savings and fulfillment time.

Search Intention Analysis for Task- and User-Centered Visualization in Big Data Applications

A new approach for classifying users’ search intentions is described in this paper. The approach uses the parameters: word frequency, query length and entity matching for distinguishing the user's query into exploratory, targeted and analysis search. The approach focuses mainly on word frequency analysis, where different sources for word frequency data are considered such as the Wortschatz frequency service by the University of Leipzig and the Microsoft Ngram service (now part of the Microsoft Cognitive Services). The model is evaluated with the help of a survey tool and few machine learning techniques. The survey was conducted with more than one hundred users and on evaluating the model with the collected data, the results are satisfactory. In big data applications the search intention analysis can be used to identify the purpose of a performed search, to provide an optimal initially set of visualizations that respects the intended task of the user to work with the result data.