Cortana Research Articles

Clinical Advice by Voice Assistants on Postpartum Depression: Cross-Sectional Investigation Using Apple Siri, Amazon Alexa, Google Assistant, and Microsoft Cortana.

BackgroundA voice assistant (VA) is inanimate audio-interfaced software augmented with artificial intelligence, capable of 2-way dialogue, and increasingly used to access health care advice. Postpartum depression (PPD) is a common perinatal mood disorder with an annual estimated cost of $14.2 billion. Only a small percentage of PPD patients seek care due to lack of screening and insufficient knowledge of the disease, and this is, therefore, a prime candidate for a VA-based digital health intervention.ObjectiveIn order to understand the capability of VAs, our aim was to assess VA responses to PPD questions in terms of accuracy, verbal response, and clinically appropriate advice given.MethodsThis cross-sectional study examined four VAs (Apple Siri, Amazon Alexa, Google Assistant, and Microsoft Cortana) installed on two mobile devices in early 2020. We posed 14 questions to each VA that were retrieved from the American College of Obstetricians and Gynecologists (ACOG) patient-focused Frequently Asked Questions (FAQ) on PPD. We scored the VA responses according to accuracy of speech recognition, presence of a verbal response, and clinically appropriate advice in accordance with ACOG FAQ, which were assessed by two board-certified physicians.ResultsAccurate recognition of the query ranged from 79% to 100%. Verbal response ranged from 36% to 79%. If no verbal response was given, queries were treated like a web search between 33% and 89% of the time. Clinically appropriate advice given by VA ranged from 14% to 29%. We compared the category proportions using the Fisher exact test. No single VA statistically outperformed other VAs in the three performance categories. Additional observations showed that two VAs (Google Assistant and Microsoft Cortana) included advertisements in their responses.ConclusionsWhile the best performing VA gave clinically appropriate advice to 29% of the PPD questions, all four VAs taken together achieved 64% clinically appropriate advice. All four VAs performed well in accurately recognizing a PPD query, but no VA achieved even a 30% threshold for providing clinically appropriate PPD information. Technology companies and clinical organizations should partner to improve guidance, screen patients for mental health disorders, and educate patients on potential treatment.

Evaluation of COVID-19 Information Provided by Digital Voice Assistants

ABSTRACT Background: Digital voice assistants are widely used for health information seeking activities during the COVID-19 pandemic. Due to the rapidly changing nature of COVID-19 information, there is a need to evaluate COVID-related information provided by voice assistants, to ensure consumers’ needs are met and prevent misinformation. The objective of this study is to evaluate COVID-related information provided by the voice assistants in terms of relevance, accuracy, comprehensiveness, user-friendliness and reliability. Materials and Methods: The voice assistants evaluated were Amazon Alexa, Google Home, Google Assistant, Samsung Bixby, Apple Siri and Microsoft Cortana. Two evaluators posed COVID-19 questions to the voice assistants and evaluated responses based on relevance, accuracy, comprehensiveness, user-friendliness and reliability. Questions were obtained from the World Health Organization, governmental websites, forums and search trends. Data was analyzed using Pearson’s correlation, independent samples t-tests and Wilcoxon rank-sum tests. Results: Google Assistant and Siri performed the best across all evaluation parameters with mean scores of 84.0% and 80.6% respectively. Bixby performed the worst among the smartphone-based voice assistants (65.8%). On the other hand, Google Home performed the best among the non-smartphone voice assistants (60.7%), followed by Alexa (43.1%) and Cortana (13.3%). Smartphone-based voice assistants had higher mean scores than voice assistants on other platforms (76.8% versus 39.1%, p = 0.064). Google Assistant consistently scored better than Google Home for all the evaluation parameters. A decreasing score trend from Google Assistant, Siri, Bixby, Google Home, Alexa and Cortana was observed for majority of the evaluation criteria, except for accuracy, comprehensiveness and credibility. Conclusion: Google Assistant and Apple Siri were able to provide users with relevant, accurate, comprehensive, user-friendly, and reliable information regarding COVID-19. With the rapidly evolving information on this pandemic, users need to be discerning when obtaining COVID-19 information from voice assistants.

Assessing user reactions to intelligent personal assistants’ humorous responses

AbstractThe paper reports on the study that developed a classification of humorous interactions with intelligent personal assistants (IPAs) and applied the classification to compare four IPAs, Apple Siri, Google Assistant, Amazon Alexa and Microsoft Cortana, on their humor performance. The study relied on volunteer participants recruited through traditional academic channels (e.g. mailing lists) as well as Amazon Mechanical Turk (AMT). While AMT and non‐AMT participants differed on some demographic characteristics, their overall ratings of IPA humor were not significantly different and were analyzed jointly using descriptive and inferential statistics. The results revealed that Apple Siri and Google Assistant received higher average ratings on humorous IPA responses compared to Amazon Alexa and Microsoft Cortana. IPA responses to joke requests were judged as being funnier than IPA responses to questions related to the IPA's personality, rhetorical statements and other humor types. Consistent with previous studies on humor, our findings did not demonstrate strong relationships between select user demographics (age, gender and humor style) and their ratings of humorous IPA responses.

Introduction to artificial intelligence in ultrasound imaging in obstetrics and gynecology.

ABSTRACTArtificial intelligence (AI) uses data and algorithms to aim to draw conclusions that are as good as, or even better than, those drawn by humans. AI is already part of our daily life; it is behind face recognition technology, speech recognition in virtual assistants (such as Amazon Alexa, Apple's Siri, Google Assistant and Microsoft Cortana) and self‐driving cars. AI software has been able to beat world champions in chess, Go and recently even Poker. Relevant to our community, it is a prominent source of innovation in healthcare, already helping to develop new drugs, support clinical decisions and provide quality assurance in radiology. The list of medical image‐analysis AI applications with USA Food and Drug Administration or European Union (soon to fall under European Union Medical Device Regulation) approval is growing rapidly and covers diverse clinical needs, such as detection of arrhythmia using a smartwatch or automatic triage of critical imaging studies to the top of the radiologist's worklist. Deep learning, a leading tool of AI, performs particularly well in image pattern recognition and, therefore, can be of great benefit to doctors who rely heavily on images, such as sonologists, radiographers and pathologists. Although obstetric and gynecological ultrasound are two of the most commonly performed imaging studies, AI has had little impact on this field so far. Nevertheless, there is huge potential for AI to assist in repetitive ultrasound tasks, such as automatically identifying good‐quality acquisitions and providing instant quality assurance. For this potential to thrive, interdisciplinary communication between AI developers and ultrasound professionals is necessary. In this article, we explore the fundamentals of medical imaging AI, from theory to applicability, and introduce some key terms to medical professionals in the field of ultrasound. We believe that wider knowledge of AI will help accelerate its integration into healthcare. © 2020 The Authors. Ultrasound in Obstetrics & Gynecology published by John Wiley & Sons Ltd on behalf of the International Society of Ultrasound in Obstetrics and Gynecology.

Mechanical Turk or volunteer participant? Comparing the two samples in the study of intelligent personal assistants

AbstractA challenge in academic and practitioner research is recruiting study participants that match target demographics, possess a desired skillset, and will participate for little to no compensation. An alternative to traditional participant recruitment struggles is crowdsourcing participants through online labor markets, such as Amazon Mechanical Turk (AMT). AMT is a platform that provides the tool for finding and recruiting participants with diverse demographics, skills, and experiences. This paper aims to demystify the use of crowdsourcing, and particularly AMT, by comparing the performance of traditionally recruited volunteers and AMT participants on tasks related to the evaluation of intelligent personal assistants (IPAs such as Amazon Alexa, Google Assistant, Apple Siri, and Microsoft Cortana). The comparison of AMT and non‐AMT samples indicated that while the two samples differed on demographics, their task performance was not significantly different. The paper discusses the costs and benefits of using AMT samples and would be of particular relevance to researchers who employ questionnaires and/or task‐specific data collection methods in their work.

Impact of auditory sense on trust and brand affect through auditory social interaction and control

Voice assistants—or voice-enabled artificial intelligence—have changed the way people interact with their surroundings dramatically. Utilizing an enactive view of social cognition theory, this study demonstrates how voice assistants can act as [semi] autonomous agents to hold instantaneous social interactions with consumers. This research employed two experimental studies. Study 1 used two voice assistant mobile applications, Microsoft Cortana and Google Assistant, and Study 2 used Amazon Alexa and Microsoft Cortana. The contributions this paper makes are two-fold. First, the results illustrate how perceived auditory sense drives perceived auditory control through auditory social interactions with a voice assistant that lead to brand affect and consumers’ trust in the voice assistant. Second, results shed light on the role of surprise as a repelling drive that attenuates the effect of perceived auditory control on brand affect.

Humanizing voice assistant: The impact of voice assistant personality on consumers’ attitudes and behaviors

A voice assistant (VA), a type of voice-enabled artificial intelligence, is no longer just a character in science fiction movies. Currently, voice is embedded in a variety of products such as smartphones (mobile applications) and smart speakers in consumers' homes. Furthermore, voice assistants are becoming integral to our daily lives. While human personalities shape the way we interact with the world, voice assistant personalities can also impact everyday interactions with our environment. This study identifies seven voice assistant personality traits (VAP) of three commonly used mobile applications: Microsoft's Cortana, Google's Assistant, and Amazon's Alexa. To examine the effect of VAP on consumer experience, this study applies and extends flow theory to uncover why VAP has the effects it has and what facets of VAP drive the voice interaction flow experience that can influence consumers' attitudes and behavioral intentions. Our study shows that voice interaction with a VA that incorporates functional intelligence, sincerity, and creativity empowers consumers to take control of their voice interactions with the VA, focus on their voice interaction, and engage in exploratory behavior. Consumers' exploratory behavior leads to consumer satisfaction and consumers' willingness to continue using voice assistant.

Evaluating Smart Assistant Responses for Accuracy and Misinformation Regarding Human Papillomavirus Vaccination: Content Analysis Study.

BackgroundAlmost half (46%) of Americans have used a smart assistant of some kind (eg, Apple Siri), and 25% have used a stand-alone smart assistant (eg, Amazon Echo). This positions smart assistants as potentially useful modalities for retrieving health-related information; however, the accuracy of smart assistant responses lacks rigorous evaluation.ObjectiveThis study aimed to evaluate the levels of accuracy, misinformation, and sentiment in smart assistant responses to human papillomavirus (HPV) vaccination–related questions.MethodsWe systematically examined responses to questions about the HPV vaccine from the following four most popular smart assistants: Apple Siri, Google Assistant, Amazon Alexa, and Microsoft Cortana. One team member posed 10 questions to each smart assistant and recorded all queries and responses. Two raters independently coded all responses (κ=0.85). We then assessed differences among the smart assistants in terms of response accuracy, presence of misinformation, and sentiment regarding the HPV vaccine.ResultsA total of 103 responses were obtained from the 10 questions posed across the smart assistants. Google Assistant data were excluded owing to nonresponse. Over half (n=63, 61%) of the responses of the remaining three smart assistants were accurate. We found statistically significant differences across the smart assistants (N=103, χ22=7.807, P=.02), with Cortana yielding the greatest proportion of misinformation. Siri yielded the greatest proportion of accurate responses (n=26, 72%), whereas Cortana yielded the lowest proportion of accurate responses (n=33, 54%). Most response sentiments across smart assistants were positive (n=65, 64%) or neutral (n=18, 18%), but Cortana’s responses yielded the largest proportion of negative sentiment (n=7, 12%).ConclusionsSmart assistants appear to be average-quality sources for HPV vaccination information, with Alexa responding most reliably. Cortana returned the largest proportion of inaccurate responses, the most misinformation, and the greatest proportion of results with negative sentiments. More collaboration between technology companies and public health entities is necessary to improve the retrieval of accurate health information via smart assistants.

Sweet Talking: Voice Technology and Virtual Assistants in Clinical Diabetes Management.

Voice technology offers a range of novel and promising strategies for clinical diabetes management. Incorporation of voice-powered virtual assistants (such as Apple Siri and Microsoft Cortana) into diabetes care programs has the potential to improve patient awareness and adherence; facilitate comprehensive provider-patient integration and data collection; and expedite consultations, procedures, and meal preparations. This study will present a qualitative literature review on existing and speculative applications of voice technology in diabetes care.

Connecting With Patients: The Rapid Rise of Voice Right Now.

Up until 5 years ago, I was a developer. I never thought that computers should be hard. When I was here previously, I discussed some of the things people can do with voice-enabled technology and how health care might be influenced [1]. Even in the short time since then, interval advances have led to even more widespread adoption of voice-enabled devices and technologies, including Amazon Alexa and Echo, Google Assistant, Microsoft Cortana, Apple Siri, and others. What is happening, far beyond anything that happened with the telephone or the Internet, is that voice is becoming routine throughout the day.

A Novel Simplified Android Based Virtual Assistant for Windows Users

The software that is designed to perform tasks or services for an individual based on commands or questions is an intelligent assistant or a personal assistant. It takes a lot of manual steps and valuable time to get several tasks done on our own. Hence, several Virtual Assistants (VA) were built to help individuals to make their general task easier by means of voice. Voice control is one of the mean for Virtual Assistant to make life easier. Assistants like Google Assistant, Apple Siri and Microsoft Cortana are built in such a way that they can get things done only on the platform they are installed on. For example, Google assistant on android can’t perform tasks on desktop that is powered by Windows operating system. This work focuses on designing and developing a VA for android that can interact with the end user to get things done on PC.

Desktop Voice Assistant With Speech Recognition Intelligence (DVAbot)

- In the advanced current days of reckless aheading technology we can perform tasks that we not once imagined that we could prepare that earlier but to complete these daydreams there is a requirement for a way by which we are able to automate our everyday tasks easily. Thus we developed applications like Voice Assistant that have the capability to communicate with us just by of human interaction. Various applications like Amazon Alexa, Microsoft Cortana are also used as an voice assistant .The voice application called “SIRI” help users to interact with other user’s mobile with vocal sound and also answers back to the voice instructions given by the user. We are going to develop a desktop voice assistant which will automate the tasks that are performed on desktop. It can modify the mode of communications between user and machine. This application is made in such a way that all of the facilities provided by the mobile phone can be retrieved by the user by giving the voice commands.

Special Issue Proposal: Conversational Search and Recommendation

The rapid growth in speech and small screen interfaces, particularly on mobile devices, has significantly influenced the way users interact with intelligent systems to satisfy their information needs. The growing interest in personal digital assistants, such as Amazon Alexa, Apple Siri, Google Assistant, and Microsoft Cortana, demonstrates the willingness of users to employ conversational interactions. In this special issue, we focus on interactions with information seeking goals. This includes conversational search and recommendation. Given the importance of the topic to both academia and industry and the recent availability of multiple public datasets in this area, we believe that the time is right to propose a special issue on this topic, and ACM Transactions on Information Systems is the perfect venue for it.

Responses of Conversational Agents to Health and Lifestyle Prompts: Investigation of Appropriateness and Presentation Structures.

BackgroundConversational agents (CAs) are systems that mimic human conversations using text or spoken language. Their widely used examples include voice-activated systems such as Apple Siri, Google Assistant, Amazon Alexa, and Microsoft Cortana. The use of CAs in health care has been on the rise, but concerns about their potential safety risks often remain understudied.ObjectiveThis study aimed to analyze how commonly available, general-purpose CAs on smartphones and smart speakers respond to health and lifestyle prompts (questions and open-ended statements) by examining their responses in terms of content and structure alike.MethodsWe followed a piloted script to present health- and lifestyle-related prompts to 8 CAs. The CAs’ responses were assessed for their appropriateness on the basis of the prompt type: responses to safety-critical prompts were deemed appropriate if they included a referral to a health professional or service, whereas responses to lifestyle prompts were deemed appropriate if they provided relevant information to address the problem prompted. The response structure was also examined according to information sources (Web search–based or precoded), response content style (informative and/or directive), confirmation of prompt recognition, and empathy.ResultsThe 8 studied CAs provided in total 240 responses to 30 prompts. They collectively responded appropriately to 41% (46/112) of the safety-critical and 39% (37/96) of the lifestyle prompts. The ratio of appropriate responses deteriorated when safety-critical prompts were rephrased or when the agent used a voice-only interface. The appropriate responses included mostly directive content and empathy statements for the safety-critical prompts and a mix of informative and directive content for the lifestyle prompts.ConclusionsOur results suggest that the commonly available, general-purpose CAs on smartphones and smart speakers with unconstrained natural language interfaces are limited in their ability to advise on both the safety-critical health prompts and lifestyle prompts. Our study also identified some response structures the CAs employed to present their appropriate responses. Further investigation is needed to establish guidelines for designing suitable response structures for different prompt types.

ARTIFICIAL INTELLIGENCE IN ONLINE SHOPPING USING NATURAL LANGUAGE PROCESSING (NLP)

Natural language processing is a process by which particular software is used to convert the speech and texts. To produce human and natural languages it combines computational linguistics with AI. The entire process can be subdivided as 3 different parts integrated statistical model to run the speech recognition, which converts e natural language into the programming language and the resultants decide what was said. This process is called the speech process for text. Natural language understanding is a domain in which the study is made to understand how a particular Artificial Intelligence can comprehend and interact with a human. Starting from a single word to complex sentences and even in the understanding of various poems and big literature can be achieved by this method. And only after this the various famous product of Apple's Siri and Microsoft's Cortana has come up with.

Virtual Assistant for Visually Impaired

The field of artificial intelligence has led to various virtual assistants such as Siri in iPhone, Google Allo, Microsoft Cortana, and so on. Even after such progression, very little has been done to implement these technologies to assist the visually impaired community. Recognizing a person or distinguishing an object, these tasks are straightforward for common people but can be very difficult for people that are partly or completely blind. Their lives can be made smoother by assisting them to detect what is present in front of them at that instant. We aim to develop a system/assistant that will serve to guide a visually impaired person and will indicate the person by speaking through the earpiece. The system will help the person recognize people, add new faces and detect objects that are in their vicinity. We will have a mobile application which will consist of numerous deep learning models that will help applications increase its administration. The primary working of the system will consist of the camera continuously feeding images for inputs, the core system processing this input information and the earpiece acting as the output device to provide this output to the user.

Classification of humorous interactions with intelligent personal assistants

The study examined humorous interactions with intelligent personal assistants (IPAs, including Google Assistant, Amazon Alexa, Microsoft Cortana, Apple Siri) with the aim of classifying user utterances, IPA responses and user reactions of system responses. Data from online diaries and paper questionnaires were collected and analyzed using content analysis method. The findings suggest that the most frequent types of utterances include questions that test system “personality” and opinions. Joke requests are also frequent and produce pre-programmed humor that users generally find funny. The initial classification of humorous utterances has been validated and expanded using published datasets of humorous utterances for the four investigated IPAs. The findings can be used for immediate improvements to IPA performance as well as long-term development of IPA personas.

Virtual assistance information model for public services

Currently, the world and Russia are rapidly developing the direction of developing virtual assistants. This is primarily due to the demands of business, the state and the population for this product, as well as the development of technologies capable of high-quality implementation of this product. Offers on virtual assistants for business and the public exist in large numbers: Google Assistant, Microsoft Cortana, Siri, etc. At the same time, there are no ready-made solutions for state needs in the market. The task of navigating public services today is the most relevant and relevant for the public sector. Solving this problem using a virtual assistant will save billions of rubles a year across the country. The article proposes an information model for the implementation of a virtual assistant for navigating public services. The basis of the intelligence of the developed virtual assistant is made up of artificial neural networks. The developed virtual assistant information model is implemented in the VIka software product. The trial operation of VIka takes place on the website of the administration of the Khanty-Mansiysk Autonomous Okrug Yugra, located at https://vi.admhmao.ru/chat/new.php. To date, VIka can help navigate services in 10 areas, such as healthcare, utilities, a line to kindergarten, etc.

JACOB Voice Chatbot Application Using Wit.ai for Providing Information in UMN

In this modern age, chatbots have been widely used as an information provider. The popularity of voice-based chatbots or voice chatbots has also increased with the presence of technologies such as Microsoft Cortana, Amazon Lex, Apple Siri, and Google Assistant. Interaction with voice chatbot is done using a voice which is more convenient than the text-based chatbots. Universitas Multimedia Nusantara (UMN) requires a tool to provide information for the Informatics joint-degree program. In this paper, a voice chatbot named JACOB is proposed. JACOB is built as a web-based application and implemented with Web Speech API. The sentences received by JACOB are translated using Wit.ai. The Jaro-Winkler distance is also applied in this research to reduce errors in identifying the user’s name. This implementation has been tested successfully, and the total satisfaction score measured using End User Computing Satisfaction is 78.53%.

Advancing Acoustic-to-Word CTC Model With Attention and Mixed-Units

The acoustic-to-word model based on the Connectionist Temporal Classification (CTC) criterion is a natural end-to-end (E2E) system directly targeting word as output unit. Two issues exist in the system: first, the current output of the CTC model relies on the current input and does not account for context weighted inputs. This is the hard alignment issue. Second, the word-based CTC model suffers from the out-of-vocabulary (OOV) issue. This means it can model only frequently occurring words while tagging the remaining words as OOV. Hence, such a model is limited in its capacity in recognizing only a fixed set of frequent words. In this study, we propose addressing these problems using a combination of attention mechanism and mixed-units. In particular, we introduce Attention CTC, Self-Attention CTC, Hybrid CTC, and Mixed-unit CTC. First, we blend attention modeling capabilities directly into the CTC network using Attention CTC and Self-Attention CTC. Second, to alleviate the OOV issue, we present Hybrid CTC which uses a word and letter CTC with shared hidden layers. The Hybrid CTC consults the letter CTC when the word CTC emits an OOV. Then, we propose a much better solution by training a Mixed-unit CTC which decomposes all the OOV words into sequences of frequent words and multi-letter units. Evaluated on a 3400 hours Microsoft Cortana voice assistant task, our final acoustic-to-word solution using attention and mixed-units achieves a relative reduction in word error rate (WER) over the vanilla word CTC by 12.09\%. Such an E2E model without using any language model (LM) or complex decoder also outperforms a traditional context-dependent (CD) phoneme CTC with strong LM and decoder by 6.79% relative.