User's Surroundings Research Articles

NeighboAR: Efficient Object Retrieval using Proximity- and Gaze-based Object Grouping with an AR System

Humans only recognize a few items in a scene at once and memorize three to seven items in the short term. Such limitations can be mitigated using cognitive offloading (e.g., sticky notes, digital reminders). We studied whether a gaze-enabled Augmented Reality (AR) system could facilitate cognitive offloading and improve object retrieval performance. To this end, we developed NeighboAR, which detects objects in a user's surroundings and generates a graph that stores object proximity relationships and user's gaze dwell times for each object. In a controlled experiment, we asked N=17 participants to inspect randomly distributed objects and later recall the position of a given target object. Our results show that displaying the target together with the proximity object with the longest user gaze dwell time helps recalling the position of the target. Specifically, NeighboAR significantly reduces the retrieval time by 33%, number of errors by 71%, and perceived workload by 10%.

Innovation in Transparent Technology with a Deep Dive into Iot-Connected Smart Glass

The goal of the project "Innovation In Transparent Technology With A Deep Dive Into IoT-Connected Smart Glass" using YOLO Image Processing is to create a cutting-edge assistive technology solution that will increase the mobility and freedom of people who are blind or visually impaired. The goal of this project is to incorporate wearable smart glasses with real-time object recognition utilizing the YOLO (You Only Look Once) algorithm. The Arduino Nano microprocessor, buzzer, and sensors that are built into the smart glasses will allow for the real-time recognition of common objects and obstacles within the user's surroundings. The system will use an easy-to-use, accessible user interface to communicate detected objects and navigation hints to the user through buzzer and vibration.

Weather Forecast and it’s Visualization using Augmented Reality: Mobile App

Weather forecasting is the process by which meteorologists project the weather for the future. These forecasts are dependent on a wide range of meteorological factors, including temperature, wind, humidity, rainfall, and a sizable dataset. Devices like the DHT11 sensor, which measures temperature and humidity and provides readings for a particular location, are used to collect data. The real-time integration of digital information into a user's surroundings is known as augmented reality, or AR. Augmented reality adds more details and improves the visual experience of the surrounding environment. Its primary benefit is the way it blends digital and three-dimensional (3D) elements with the user's reality sense. Our project intends to combine augmented reality and a weather forecasting API in a smartphone application. Users will see 3D visuals thanks to this integration, providing a more interesting and practical approach to learn about the local weather conditions in their city. Keywords – Climate Prediction, Temperature, Humidity, Wind Speed, Rainfall, Weather Conditions, Weather Data, Real-time Forecast, Climate Monitoring, Atmospheric Conditions, Weather Updates, Weather Patterns, Weather App, Mobile Weather, Forecasting API, Augmented Reality, 3D Graphics, Local Weather, Regional Climate, Meteorological Data, DHT11 Sensor, Climatic Parameters.

Smart Shoes for Blind People

Abstract: Smart Shoes for blind people are a technological solution that aims to improve the mobility and independence of visually impaired individuals. The smart shoes are equipped with a variety of sensors and technologies that work together to provide real-time feedback about the user's surroundings, obstacles, and navigation directions. The main components of the smart shoes include an inertial measurement unit (IMU), GPS receiver, microcontroller, Bluetooth module, and a haptic feedback mechanism. The IMU consists of sensors such as accelerometers and gyroscopes that measure the movement and orientation of the shoe. The GPS receiver provides location data that is used to calculate the user's position and orientation. The microcontroller processes the sensor data and runs a navigation algorithm that calculates the safest and most efficient route for the user to take. The algorithm takes into account factors such as the user's destination, the terrain, obstacles, and other environmental factors. The Bluetooth module enables the user to connect to a smartphone or other device to receive audio or visual instructions and feedback. The haptic feedback mechanism, such as vibrating motors, provides tactile feedback to the user about the direction and proximity of obstacles. Smart shoes for blind people offer a unique and innovative solution to improve the mobility and independence of visually impaired individuals. They provide real-time feedback and navigation directions to help users navigate their environment safely and efficiently

Vision Application for Visually Impaired People

Abstract: The Vision Eye application is a novel solution that aims to assist visually impaired individuals in their daily lives. The application uses computer vision algorithms and artificial intelligence to detect the user's surroundings and provide them with real-time auditory feedback about the objects and people around them. The Vision application uses a smartphone camera to capture images, which are then analyzed by the app's algorithms to identify objects, people, and other environmental features. The app then converts this information into auditory feedback that is relayed to the user through headphones or earbuds. The application also includes voice recognition and GPS navigation to enhance the user's experience. The Vision application has the potential to significantly improve the quality of life of visually impaired individuals by providing them with greater independence and autonomy in their daily lives.

Ultrasonic Spectacles for Blind Person

The ultrasonic spatial code line for blind people using Node MCU, web camera, ultrasonic sensor, and output of speaker is an innovative system designed to aid visually impaired individuals in navigating their environment independently and safely. The system works by using a combination of hardware components and software algorithms to provide real-time spatial feedback to the user. The system consists of a Node MCU microcontroller, which acts as the central processing unit and controls the flow of data between the various components. A web camera is used to capture a live video feed of the user's surroundings, which is then processed using image recognition algorithms to identify and locate obstacles in the environment. An ultrasonic sensor is used to measure the distance between the user and any obstacles detected in the video feed. This information is then used to generate a tactile feedback map that is transmitted to the user via a speaker output. The tactile feedback map consists of a series of vibrations that correspond to the location and distance of the obstacles in the user's environment. The ultrasonic spatial code line is a critical need for blind individuals who struggle with navigating their environment independently. The system provides a practical solution that can help improve their quality of life by enabling them to move around more safely and confidently. Additionally, the system is relatively low-cost and easy to implement, making it accessible to a broader range of individuals. Key Word: Electronic Travel Aid – Visual Impairment- Machine learning – Navigation Aid - Mobility – Ultrasonic spectacles – Blind navigation;

Assistive mobile application for real-time 3D spatial audio soundscapes toward improving safe and independent navigation

Assistive technologies are used in a variety of contexts to improve the quality of life for individuals that may have one or more vision impairments. This paper describes a novel assistive technology platform that utilizes real-time 3D spatial audio to aid its users in safe and efficient navigation. This platform leverages modern 3D scanning technology on a mobile device to digitally construct a live 3D map of a user's surroundings as they move about their space. Within the digital 3D scan of the world, spatialized, virtual audio sources (i.e., speakers) provide the navigator with a real-time 3D stereo audio "soundscape." As the user moves about the world, the digital 3D map, and its resultant soundscape, are continuously updated and played back through headphones connected to the navigator's device. This paper details (1) the underlying technical components and how they were integrated to produce the mobile application that generates a dynamic soundscape on a consumer mobile device and (2) a methodology for analyzing the usage of the application. It is the aim of this application to assist individuals with vision impairments to navigate and understand spaces safely, efficiently, and independently.

Comparison of algorithms for the assessment of student attention in online learning systems

Various algorithms are used in online learning systems to compare students' attention in the classroom. Two algorithms are compared in this work. The first algorithm assesses student attentiveness in the classroom by examining their facial expressions. In order to determine if students are paying attention or not during the electronic learning process and to identify instances of academic dishonesty, this algorithm uses an interactive video-capture facial recognition technology. The Classroom Attentiveness Classification Tool (ClassACT), a system created to track student attention during several instructional stages within the learning environment, including lectures, group projects, evaluations, etc., is the second algorithm. ClassACT can tell the difference between attentive and inattentive behaviour by gathering data about the user, the user's surroundings, and the device itself using the many sensors integrated into the tablet. This essay contrasts these two algorithms in terms of methodology and outcomes.

VocabulARy: Learning Vocabulary in AR Supported by Keyword Visualisations.

Learning vocabulary in a primary or secondary language is enhanced when we encounter words in context. This context can be afforded by the place or activity we are engaged with. Existing learning environments include formal learning, mnemonics, flashcards, use of a dictionary or thesaurus, all leading to practice with new words in context. In this work, we propose an enhancement to the language learning process by providing the user with words and learning tools in context, with VocabulARy. VocabulARy visually annotates objects in AR, in the user's surroundings, with the corresponding English (first language) and Japanese (second language) words to enhance the language learning process. In addition to the written and audio description of each word, we also present the user with a keyword and its visualisation to enhance memory retention. We evaluate our prototype by comparing it to an alternate AR system that does not show an additional visualisation of the keyword, and, also, we compare it to two non-AR systems on a tablet, one with and one without visualising the keyword. Our results indicate that AR outperforms the tablet system regarding immediate recall, mental effort and task-completion time. Additionally, the visualisation approach scored significantly higher than showing only the written keyword with respect to immediate and delayed recall and learning efficiency, mental effort and task-completion time.

Smart Sound System Applied for the Extensive Care of People with Hearing Impairment

We, as normal people, have access to a potent communication tool, which is sound. Although we can continuously gather, analyse, and interpret sounds thanks to our sense of hearing, it can be challenging for people with hearing impairment to perceive their surroundings through sound. Also known as PWHI (People with Hearing Impairment). Auditory/phonic impairment is one of the most prevailing sensory deficits in humans at present. Fortunately, there is room to apply a solution to this issue, given the development of technology. Our project involves capturing ambient sounds from the user’s surroundings and notifying the user through a mobile application using IoT and Deep Learning. Its architecture offers sound recognition using a tool, such as a microphone, to capture sounds from the user's surroundings. These sounds are identified and categorized as ambient sounds, like a doorbell, baby cry, and dog barking; as well as emergency-related sounds, such as alarms, sirens, etc.

A PRODUCTIVE IBPRE MODEL FOR SECURE DATA SHARING IN BLOCKCHAIN TECHNOLOGY BASED IOT

Intelligent tailored services are necessary to fulfill the expectations of users. In order to provide a highly dependable service, additional information about the user's surroundings is required. In order to capture a vast quantity of data, the Internet of Things (IoT), which links multiple gadgets over the Internet, has been created. Although cloud computing is frequently utilized to effectively manage data, it has the potential to delay the transmission of data as well. In this work, we suggest a proxy re-encryption strategy to secure data sharing in cloud contexts. Data owners may outsource their encrypted data to the cloud using identity-based encryption, while proxy re-encryption construction will enable genuine user’s access to the data. An edge device serves as a proxy server for computationally expensive tasks since IoT devices have limited resources. Using information-centric networking capabilities, we may effectively serve cached material in the proxy, hence enhancing service quality and efficiently using network traffic. We've also built our system on block chain, a revolutionary new technology that makes it possible to centralize the sharing of data. It reduces the load on centralized systems and allows for more precise management of data access. The security study and assessment of our scheme reveal the potential of our technique in protecting data confidentiality, integrity, and security

Data-driven Context Detection Leveraging Passively Sensed Nearables for Recognizing Complex Activities of Daily Living

Wearable systems have unlocked new sensing paradigms in various applications such as human activity recognition, which can enhance effectiveness of mobile health applications. Current systems using wearables are not capable of understanding their surroundings, which limits their sensing capabilities. For instance, distinguishing certain activities such as attending a meeting or class, which have similar motion patterns but happen in different contexts, is challenging by merely using wearable motion sensors. This article focuses on understanding user's surroundings, i.e., environmental context, to enhance capability of wearables, with focus on detecting complex activities of daily living (ADL). We develop a methodology to automatically detect the context using passively observable information broadcasted by devices in users’ locale. This system does not require specific infrastructure or additional hardware. We develop a pattern extraction algorithm and probabilistic mapping between the context and activities to reduce the set of probable outcomes. The proposed system contains a general ADL classifier working with motion sensors, learns personalized context, and uses that to reduce the search space of activities to those that occur within a certain context. We collected real-world data of complex ADLs and by narrowing the search space with context, we improve average F1-score from 0.72 to 0.80.

Book review: KJ Varnum (ed.), Beyond Reality: Augmented, Virtual, and Mixed Reality in the Library

This is a book review of: Beyond reality: augmented, virtual, and mixed reality in the library, edited by K.J. Varnum and published in 2019 by the American Library Association. This slim volume is an interesting read. Arranged in nine chapters, it's a sampler of actual and potential applications of the three types of experience listed in the title. Augmented reality might be described as 'reality plus' – an overlay of electronically-generated content on the real world, such as is used in the 'heads-up displays' employed by fighter pilots and in some cars. Mixed reality involves some kind of electronically-delivered content, which is tied in with real-world locations, an example being the recent phenomenon of Pokemon-collecting, in which cartoon characters could be collected (often in proximity to real-world libraries) by use of a smart phone, which overlaid their images on the phone's video display of the user's surroundings. The effort required to communicate that concept in a single sentence is some indication of what a complex process is involved; the fact that it would have made little sense to an average reader of even ten years ago shows how far these concepts have penetrated popular culture. Virtual reality is an 'immersive' experience, often achieved through the use of head-mounted display (HMD) equipment, such as the Oculus Rift, which may permit the user to move around and interact with 'objects' in an environment that is completely generated by software.

LIDAR Assist Spatial Sensing for the Visually Impaired and Performance Analysis.

Echolocation enables people with impaired or no vision to comprehend the surrounding spatial information through the reflected sound. However, this technique often requires substantial training, and the accuracy of echolocation is subject to various conditions. Furthermore, the individuals who practice this sensing method must simultaneously generate the sound and process the received audio information. This paper proposes and evaluates a proof-of-concept light detection and ranging (LIDAR) assist spatial sensing (LASS) system, which intends to overcome these restrictions by obtaining the spatial information of the user's surroundings through a LIDAR sensor and translating the spatial information into the stereo sound of various pitches. The stereo sound of relative pitch represents the information regarding objects' angular orientation and horizontal distance, respectively, thus granting visually impaired users an enhanced spatial perception of his or her surrounding areas and potential obstacles. This paper is divided into two phases: Phase I is to engineer the hardware and software of the LASS system and Phase II focuses on the system efficacy study. The study, approved by the Penn State Institutional Review Board, included 18 student volunteers, who were recruited through the Penn State Department of Psychology Subject Pool. This paper demonstrates that the blindfolded individuals equipped with the LASS system are able to quantitatively identify the surrounding obstacles, differentiate their relative distance, and distinguish the angular location of multiple objects with minimal training.

Use of Phone Sensors to Enhance Distracted Pedestrians’ Safety

Studies have shown that using smartphones while walking—called distracted walking —significantly increases the risk of pedestrians colliding with dangerous objects. In this paper, we explore how to mitigate this problem by exploiting the phone's built-in sensors only and developing an application called BumpAlert . BumpAlert provides a generic solution without requiring any prior knowledge of the user's surroundings by estimating distances to nearby objects using the phone's speakers and microphones. This process is enhanced further by using the images acquired from the phone's rear camera, when necessary. We have evaluated BumpAlert under a variety of settings ranging from aisle to outdoor environments with walls, pillars, signboards, dustbins, and people, etc., that are common in our daily surroundings. Our evaluation has shown an average accuracy of BumpAlert to be higher than 95 percent with a less than 2 percent false-positive rate to detect frontal objects 2–4m away, which suffices for the user to react and avoid collision. Even though BumpAlert is unable to detect all dangerous situations, most participants of our user study feel safer when they walk with BumpAlert enabled. Integrating our current design of BumpAlert with other safety systems can provide a practical solution for protecting distracted pedestrians.

Getting fast, free, and anonymous answers to questions asked by people with visual impairments

Much of the information we encounter in daily life is visual. For people with visual impairments, this information is not inherently accessible. Technology can be used to provide access to this visual information, through automated tools like currency identifiers, object recognizers, and optical character recognition, which allow for immediate access to visual information in a users' environment. However, these tools are often are limited to a specific domain, and can fail in non-ideal conditions (e.g., an optical character recognition tool trying to identify handwritten text). By incorporating people into the information access pipeline, human-powered tools can allow for more complex information to be extracted from the user's surroundings. Human-powered access tools connect people who have disabilities to other people who can access information on their behalf. While people may not be as quick as automated tools, the person answering can make inferences based on prior knowledge and experiences, ask for clarification, and reason over the information provided. In this article, we explore the long-term public deployment and lessons learned from VizWiz Social, a human-powered access tool that connects people with visual impairments to sighted workers or friends and family members who can answer their visual questions. By analyzing the use of this service, and running controlled experiments to determine how users value different answer sources, we have been able to build upon the original VizWiz design to create social microvolunteering, a method for getting fast, free, and anonymous answers to visual questions.

Location-aware Mobile Services for a Smart City: Desing, Implementation and Deployment

A smart city is a high-performance urban context, where citizens are more aware of, and more integrated into the city life, thanks to an intelligent city information system. In this paper we design, implement and deploy a smart application that retrieves and conveys to the user relevant information on the user's surroundings. This case study application let us discuss the challenges involved in creating a location-aware mobile service based on live information coming from the city IT infrastructure. The service, that is currently being deployed in the Italian city of Cesena, has been designed with the goal of being a general model for future applications. In particular, we discuss location-aware and mobile development, cloud and cluster based geographical data storage, and spatial data computation. For each of these topics we provide implementation and deployment solutions based on currently available technology. In particular we propose an architecture based on a complex On-Line Transaction Processing (OLTP) infrastructure. Furthermore, this paper represents the first comprehensive, scientific study on the subject.

Titan: An Enabling Framework for Activity-Aware "Pervasive Apps" in Opportunistic Personal Area Networks

Upcoming ambient intelligence environments will boast ever larger number of sensor nodes readily available on body, in objects, and in the user's surroundings. We envision Apps, user-centric activity-aware pervasive computing applications. They use available sensors for activity recognition. They are downloadable from application repositories, much like current Apps for mobile phones. A key challenge is to provide Pervasive Apps in open-ended environments where resource availability cannot be predicted. We therefore introduce Titan, a service-oriented framework supporting design, development, deployment, and execution of activity-aware Pervasive Apps. With Titan, mobile devices inquire surrounding nodes about available services. Internet-based application repositories compose applications based on available services as a service graph. The mobile device maps the service graph to Titan Nodes. The execution of the service graph is distributed and can be remapped at run time upon changing resource availability. The framework is geared to streaming data processing and machine learning, which is key for activity recognition. We demonstrate Titan in a pervasive gaming application involving smart dice and a sensorized wristband. We comparatively present the implementation cost and performance and discuss how novel machine learning methodologies may enhance the flexibility of the mapping of service graphs to opportunistically available nodes.

Designing augmented reality interfaces

Most interactive computer graphics appear on a screen separate from the real world and the user's surroundings. However this does not always have to be the case. In augmented reality (AR) interfaces, three-dimensional virtual images appear superimposed over real objects. AR applications typically use head-mounted or handheld displays to make computer graphics appear in the user's environment.

User-centered location awareness

Mobile computing and communication devices now provide access information from nearly anywhere, and many of these devices know where they are. Much current research focuses on using location awareness provide information about a user's surroundings. We believe that geographic information can be personalized based on its relevance the user, with appropriate descriptions or granularity deliver valuable location-aware services. A location-aware memory aid, comMotion tracks a user's path via the Global Positioning System (GPS) and maintains to do lists for select locations consisting of text and voice recordings.