Desktop Environment Research Articles

IVA-former: invisible–visible query guided amodal mask measurement network for desktop object via hierarchical transformer

Abstract Instance segmentation of desktop objects is important for service robots. Most of the previous works for desktop environments are restricted to measuring the visible area of target objects. However, when a target object is placed behind another, the algorithm that only performs visible area segmentation is unable to provide accurate appearance information for the occluded objects. To solve this problem, we propose the invisible–visible query guided amodal mask measurement network based on a hierarchical transformer for desktop scenes, which can perceive the entire appearance of objects in the presence of occlusions. In this method, the RGB-D backbone is adopted to fuse the features from both RGB and depth images. Then, the pixel decoder is used to generate multi-scale feature maps. We then adopt a hierarchical transformer decoder to predict invisible, visible, and amodal masks simultaneously. To enhance the associations between the three prediction branches, we propose a query transform module to transfer object queries between adjacent branches. Since amodal masks are a combination of invisible and visible masks, we propose an invisible–visible mixture loss that takes masks from both invisible and visible branches to further supervise the network. Our method is trained on synthetic datasets for desktop objects and evaluated on both visible and amodal real-world datasets. Compared to other recent segmentation algorithms, our method achieves competitive performance.

Evaluating the role of cloud integration in mobile and desktop operating systems

The integration of cloud computing has fundamentally transformed the landscape of mobile and desktop operating systems, enabling new levels of functionality, efficiency, and user experience. This review examines the role of cloud integration in these operating systems and its implications for both users and developers. Cloud integration enhances mobile and desktop operating systems by offering scalable storage solutions, seamless data synchronization, and enhanced computing power. For mobile systems, cloud services provide users with the ability to access their data and applications from any device, promoting a consistent user experience across different platforms. This synchronization feature not only streamlines personal data management but also enables developers to create more dynamic and responsive applications that leverage cloud resources for realtime updates and performance improvements. In desktop environments, cloud integration facilitates the storage of large volumes of data and the execution of resourceintensive applications without burdening local hardware. Cloudbased virtual machines and application delivery platforms offer users the flexibility to run highperformance software on less powerful devices, thus broadening the accessibility and usability of advanced desktop applications. Furthermore, cloud integration supports collaborative tools and shared workspaces, enhancing productivity and fostering realtime collaboration among users. The evaluation of cloud integration also involves assessing its impact on security, data privacy, and system performance. Cloudbased solutions introduce new security considerations, such as the need for robust encryption and secure authentication protocols to protect sensitive data. Additionally, the reliance on internet connectivity for accessing cloud services can pose challenges for users in areas with limited or unreliable network access. In conclusion, cloud integration plays a crucial role in modernizing both mobile and desktop operating systems, driving improvements in data accessibility, application performance, and user experience. As cloud technologies continue to evolve, their influence on operating systems will likely expand, offering new opportunities for innovation and addressing emerging challenges in security and connectivity. This ongoing integration represents a significant shift towards more flexible, efficient, and interconnected computing environments.

Impact of Immersion in Virtual Reality (VR) and Desktop Environments on Spatial Ability

Impact of Immersion in Virtual Reality (VR) and Desktop Environments on Spatial Ability

Age and Gender Differences in the Pseudo-Haptic Effect on Computer Mouse Operation in a Desktop Environment.

Pseudo-haptics is a method that can provide a haptic sensation without requiring a physical haptic device. The effect of pseudo-haptics is known to depend on the individual, but it is unclear which factors cause individual differences. As the first study establishing a calibration method for these differences in future research, we examined the differences in the pseudo-haptic effect on mouse cursor operation in a desktop environment depending on the age and gender of the user. We conducted an online experiment and collected data from more than 400 participants. The participants performed a task of lifting a virtual object with a mouse pointer. We found that the effect of pseudo-haptics was greater in younger or male participants than in older or female participants. We also found that the effect of pseudo-haptics, which varied with age and gender, can be explained by habituation to the mouse in daily life and the accuracy of detecting the pointer position using vision or proprioception. Specifically, the pseudo-haptic effect was higher for those who used the mouse more frequently and had higher accuracy in identifying the pointer position using proprioception or vision. The results of the present study not only indicate the factors that cause age and gender differences but also provide hints for calibrating these differences.

Virtual Desktop Assistant for Linux

A desktop assistant on Linux is a software application designed to assist users in performing various tasks on their Linux- based desktop system. It typically integrates with the desktop environment and offers a user-friendly interface for performing common tasks such as launching applications, managing files and folders, and accessing system settings. The assistant can also be customized with plugins or extensions to add additional functionality. The main goal of a desktop assistant on Linux is to make the user experience as smooth and efficient as possible. By centralizing common tasks and providing an accessible interface, the assistant eliminates the need for users to navigate complex system settings or memorize multiple keyboard shortcuts. This, in turn, makes it easier for users to perform everyday tasks on their Linux-based desktop system, increasing productivity and overall satisfaction with their experience. Keywords— Deep Learning, Machine learning, NLP, Voice Assistant, Linux, Artificial Intelligence, Speech Recognition, LLMs.

How Learners’ Visuospatial Ability and Different Ways of Changing the Perspective Influence Learning About Movements in Desktop and Immersive Virtual Reality Environments

Virtual reality (VR) applications are developing rapidly, becoming more and more affordable, and offer various advantages for learning contexts. Dynamic visualizations are generally suitable for depicting continuous processes (e.g., different movement patterns), and particularly dynamic virtual 3D-objects can provide different perspectives on the movements. The present study investigated through a low immersive (desktop “VR”, Study 1) and a high immersive virtual environment (immersive VR; Study 2) the effectiveness of different interaction formats to view 3D-objects from different perspectives. Participants controlled either the orientation of the 3D-objects (Study 1, mouse interaction; Study 2, hand interaction via VR controllers) or their viewpoint in relation to the 3D-objects (Study 1, camera position; Study 2, position of participants’ own body). Additionally, the moderating influence of learners’ visuospatial ability was addressed. Dependent variables were pictorial recognition (easy, medium, difficult), factual knowledge, presence, and motion sickness. Results showed that higher-visuospatial-ability learners outperformed lower-visuospatial-ability learners. In Study 1, higher-visuospatial-ability learners showed higher recognition performance (difficult items) by controlling the camera position, whereas lower-visuospatial-ability learners suffered from this interaction format. In Study 2, higher-visuospatial-ability learners achieved better recognition performance (easy items) by controlling the 3D-models, whereas lower-visuospatial-ability learners tended to profit from moving around the 3D-objects (medium items). The immersive VR yielded more presence and higher motion sickness. This study clearly shows that different interaction formats to view 3D-objects from multiple perspectives in Desktop-VR are not transferable on a one-to-one basis into immersive VR. The results and implications for the design of virtual learning environments are discussed.

Visual Highlighting for Situated Brushing and Linking

AbstractBrushing and linking is widely used for visual analytics in desktop environments. However, using this approach to link many data items between situated (e.g., a virtual screen with data) and embedded views (e.g., highlighted objects in the physical environment) is largely unexplored. To this end, we study the effectiveness of visual highlighting techniques in helping users identify and link physical referents to brushed data marks in a situated scatterplot. In an exploratory virtual reality user study (N=20), we evaluated four highlighting techniques under different physical layouts and tasks. We discuss the effectiveness of these techniques, as well as implications for the design of brushing and linking operations in situated analytics.

Kiki – A Virtual AI Desktop Aider

The research“Kiki” is a voice-controlled virtual desktop assistant designed to enhance user productivity and streamline interaction with desktop environments. Leveraging natural language processing (NLP) techniques, Kiki enables users to perform various tasks through voice commands, ranging from opening applications to searching the web and playing music. The system employs state-of-the-art language models to understand user queries and generate contextually relevant responses, providing a seamless and personalized user experience. Key functionalities include task automation, information retrieval, and web search, allowing users to execute commands efficiently and effortlessly. By combining advanced NLP capabilities with intuitive voice interaction, Kiki represents a significant advancement in desktop assistant technology, promising to reshape the way individuals interact with their computers.

An Approach Towards to Real Time AI Desktop Voice Assistant

The advent of artificial intelligence (AI) has revolutionized human-computer interaction, making it more natural and intuitive. This research paper presents the development and implementation of an AI Desktop Voice Assistant designed to enhance productivity and accessibility for users. The voice assistant leverages advanced speech recognition, natural language processing (NLP), and machine learning techniques to understand and execute user commands. Key functionalities include voice-activated application control, web searches, personalized reminders, and real-time information retrieval. Our system integrates with widely-used APIs and services, providing a seamless user experience across various tasks. The paper discusses the architecture of the voice assistant, detailing the integration of components such as the speech recognition engine, NLP models, and the dialogue management system. We explore the challenges encountered during development, including accurate speech recognition in noisy environments and handling ambiguous user commands. Solutions implemented to address these challenges are also presented. Furthermore, we conduct a usability study to evaluate the effectiveness and user satisfaction of the voice assistant. The results indicate a high level of user engagement and satisfaction, demonstrating the practical benefits of incorporating AI-driven voice interfaces into desktop environments. Our findings contribute to the ongoing research in human-computer interaction, suggesting pathways for future enhancements in AI voice assistants..

Desktop as a Service using Local Computers

Abstract: Desktop-as-a-Service (DaaS) has gained prominence as a cloud-based solution for delivering desktop environments to users. However, traditional DaaS models often rely on remote data centers, posing concerns regarding data security, latency, and dependence on external infrastructure. In this paper, we propose a paradigm shift by introducing a novel approach to DaaS that leverages the untapped potential of local computing resources.Our system harnesses the idle computing power available within the user's local environment, including desktops, laptops, and servers, to create a distributed DaaS platform. By integrating virtualization technologies and dynamic resource allocation strategies, we enable the provisioning of desktop environments ondemand, effectively reducing the reliance on centralized data centers. Key components of our system include a centralized management platform for orchestrating resource allocation and workload distribution, virtualization techniques for isolating and managing desktop instances, and robust security measures to safeguard sensitive data. Through experimental validation, we demonstrate the feasibility and performance of our approach, showcasing improvements in data privacy, latency reduction, and resource utilization compared to traditional DaaS models. Our research presents a significant step towards decentralized and resilient computing ecosystems, offering users greater control over their desktop environments while maintaining the scalability and accessibility inherent to DaaS paradigms

Developing systems to encourage FAIR and secure research data

Welcome to first issue of IASSIST Quarterly for 2024; this is Volume 48 of the journal (IQ 48(1) 2024). The three papers in this issue represent different aspects of researchers’ use of data archives. Experts from three major data archives (ICPSR, IPUMS, and GESIS) share their experiences developing systems to make research resources in their archives more findable, accessible, and usable. Lafia, Million, and Hemphill in their article “Exploratory and directed search strategies at a social science data archive” investigate search strategies of users of a large social science data archive. In an effort to understand how users search for curated research data, the authors analyze data queries issued through the Inter-University Consortium for Political and Social Research (ICPSR) website’s search box. The study is meant to inform archives and repositories of their users’ experience and help develop systems that encourage dataset findability, accessibility and reuse. The authors identify two types of data searches, exploratory and directed searches. They find that while users that issued exploratory queries were able to navigate the ICPSR website successfully and refine their searches, they may benefit from more explicit support for query reformation. The authors suggest ways development of tools and training could improve users’ experience. In the article “Stewarding our resources: Building a sustainable IPUMS archival document access system” Daina Magnuson describes IPUMS experience building a web interface to support exploration and dissemination of archival materials associated with IPUMS International (IPUMS-I). Ancillary materials, including thousands of unique pieces of census and survey documentation were obtained by IPUMS-I during data acquisition efforts. This rich source material was curated and preserved by archival staff. Magnuson describes in details the development of a system that would make these resources findable, searchable and downloadable to internal users as well as IPUMS researchers. Strategies describe may inform other data archives considering developing similar tools. Deborah Wiltshire in her article “Developing canonical ‘Safe Researcher’ training materials for Trusted Research Environment” describes the development of training materials for researchers applying to access restricted data through Trusted Research Environment (TRE). The increased use of a virtual desktop environment, as opposed to physical safe rooms, offers researchers easier and more flexible access to restricted data; however, it offers fewer physical safeguards. Attending a mandatory ‘Safe Researcher’ training is one of the principles of the Five Safe Framework[i] developed by the UK’s Office for National Statistics to bridge this gap and assure the safe use of sensitive data. The process described in the article could be easily adapted by other TREs according to their needs and settings (e.g., in-person vs. online training). In the spirit of making information findable, accessible and reusable, we’re happy to share that the Open Journal System platform from the University of Alberta has enabled author linking to ORCID. This means that IQ authors are now able to connect their ORCID iDs to their IQ publications, so that their ORCID profiles will automatically update with those citations as their articles are published. For existing IQ account holders, log into your account at the upper right of the IQ page, and go to Profile > Public. Then click “Create or Connect your ORCID iD” (just below the Homepage URL field). For details, see this video walkthrough of profile authentication. New IQ users will automatically be prompted to link (or create) their ORCID iDs during IQ account creation. Authors are also encouraged to note their ORCID iD link with their affiliation and contact information in an endnote when submitting a manuscript for publication. We hope to see many of you at the 49th Annual IASSIST Conference in Halifax, Nova Scotia, Canada, May 28-31, 2024! Ofira Schwartz and Michele Hayslett, March 2024

3D Remote Assistance for NDT Inspections

In this work, we present a system architecture that is especially designed for remote ultrasound testing inspections (UT). The system itself is realized using a real-time session service and a web service based on a REST API (REST: Representational State Transfer; API: application programming interface). This web service is used to store data persistently, e.g., sample geometries, raw nondestructive testing (NDT) data and derived inspection results in a shared dataspace. In the current development state, the results consist of textures mapped onto the sample geometry. This approach allows us to display the UT results directly on the real sample using mixed reality technologies. We also implemented a feature to assist the inspector remotely by making use of the availability of this digital representation. Hence, it is necessary to share additional data like the current UT-signal, temporary position marks, user and device positions, etc. between the different participants. A real-time distribution of this highly dynamic data is required to create an effective assistance environment. Therefore, a separate session service is used. The inspection data generated in this temporary session can also be transferred to the afore mentioned dataspace to be saved persistently. The system features mixed reality visualization for the inspector and optionally a virtual reality or a 3D Desktop environment for one or more remote assistants.

3DNA: A Tool for Sculpting Brick-Based DNA Nanostructures

To assist in the speed and accuracy of designing brick-based DNA nanostructures, we introduce a lightweight software suite 3DNA that can be used to generate complex structures. Currently, implementation of this fabrication strategy involves working with generalized, typically commercial CAD software, ad-hoc sequence-generating scripts, and visualization software, which must often be integrated together with an experimental lab setup for handling the hundreds or thousands of constituent DNA sequences. 3DNA encapsulates the solutions to these challenges in one package by providing a customized, easy-to-use molecular canvas and back-end functionality to assist in both visualization and sequence design. The primary motivation behind this software is enabling broader use of the brick-based method for constructing rigid, 3D DNA-based nanostructures, first introduced in 2012. 3DNA is developed to provide a streamlined, real-time workflow for designing and implementing this type of 3D nanostructure by integrating different visualization and design modules. Due to its cross-platform nature, it can be used on the most popular desktop environments, i.e., Windows, Mac OS X, and various flavors of Linux. 3DNA utilizes toolbar-based navigation to create a user-friendly GUI and includes a customized feature to analyze the constituent DNA sequences. Finally, the oligonucleotide sequences themselves can either be created on the fly by a random sequence generator, or selected from a pre-existing set of sequences making up a larger molecular canvas.

Mapping landscape ecological patterns using numeric and categorical maps.

The reciprocal relationships between ecological process and landscape pattern are fundamental to landscape ecology. Landscape ecologists traditionally use raster maps portraying classified features such as land use or land cover categories, and metrics suggested by the patch-corridor-matrix conceptual model of pattern. Less attention has been given to the landscape gradient conceptual model and raster maps portraying numeric features such as greenness or percent vegetation cover. We introduce the open-source tool GraySpatCon to calculate and map a variety of landscape pattern metrics from both conceptual models using either categorical or numeric maps. The 51 metrics, drawn mostly from the landscape ecology and image processing literatures, are calculated from the frequencies of input pixel values and/or the pixel value adjacencies in an analysis region. GraySpatCon conducts either a moving window analysis which produces a continuous map of a pattern metric, or a global analysis which produces a single metric value. We describe an implementation in the GuidosToolbox desktop application which allows novice users to interactively explore GraySpatCon functionality. In the R desktop environment, we demonstrate several metrics using an example map of percent tree cover and illustrate a multi-scale moving window analysis to identify scale domains. Comparisons of computational efficiency indicate a substantial GraySpatCon advantage over related software in the R environment.

Desktop as a Service Using Local Computers

In today's fast-paced digital landscape, where businesses face growing demands to optimize productivity, enhance flexibility, and safeguard their data against ever-evolving threats, companies are constantly seeking innovative solutions. Desktop as a Service (DaaS) emerges as a transformative paradigm in this context, offering organizations virtualized desktop environments that address the core challenges faced by businesses. Whether it's the ability to seamlessly integrate with existing on- premise or cloud-based infrastructures, providing a secure environment through robust encryption and authentication protocols, or delivering an intuitive user experience that empowers employees to be more productive, DaaS ensures that businesses achieve optimal operational efficiency. This project aims to delve deeply into the comprehensive implementation and analysis of DaaS, examining its seamless integration into existing infrastructures, robust security protocols that help in mitigating cybersecurity risks, the user-centric design that ensures a smooth and delightful experience, and the overall positive impact it has on operational efficiency across various industries and use. Key Words: Cloud-Based Technologies, Infrastructure Optimization, Remote Access, Virtual Desktop Environments, Cost Optimization.

Blind and Visually Impaired Users’ Interactions with Digital Libraries: Help-Seeking Situations in Mobile and Desktop Environments

With the goal of developing information retrieval (IR) systems, including digital libraries (DLs), that support universal access, the authors conducted two studies to identify the help-seeking situations that blind and visually impaired (BVI) users encountered in DLs. First, the authors quantitatively compared BVI users’ help-seeking situations when interacting with the DL's mobile website (M.Web) and the mobile app (M.App). Using multiple data collection methods, the mobile study identified six situations BVI users faced more frequently when using M.Web than M.App. Second, the findings of the mobile study were qualitatively compared with the situations that occurred in the desktop environment, highlighting eight unique situations. It is more difficult for BVI users to interact with DLs in the mobile environment because of the reduced size of the mobile interface, complex DL structures, dynamic elements, diverse formats, and sight-centered design. Specific design recommendations were offered to enhance DLs in the mobile environment.

Geospatial Analysis in Web Browsers—Comparison Study on WebGIS Process-Based Applications

With the rapid development of internet technologies in recent years, the shift from the desktop to the web platform can be seen within geospatial analysis. While analytical tools, such as buffer or clip, are routinely used in desktop environments, WebGIS deals with geographic information, including geospatial analysis, within the online environment. The main aim of this paper is to perform a comparison and evaluation of vector-oriented online geoprocessing tools in a WebGIS environment, supported by the development of a custom solution for geospatial analysis. The application called GeOnline is developed and tested as a case study to demonstrate the availability of spatial analysis tools within the web browser. It implements the specialized geospatial library Turf.js, which allows using non-trivial geospatial analysis, such as intersect, clip or calculate centroids. It handles client-side processes. Both a functionality comparison and performance testing are carried out, while the paper primarily focuses on data-driven (data-based) analysis and not only on visual-driven (visual-based) analysis. The comparative study evaluates five geospatial tools (ArcGIS Online, GISCloud, CARTO, FOURSQUARE, GeOnline) and summarizes the solutions from different aspects, including the number of supported operations. Finally, performance tests on GeOnline separately and among alternative solutions are performed. While ArcGIS Online is considered the most comprehensive solution on the market, GeOnline performs well compared to alternative solutions.

Bilgi İşlem Merkezi için Ucuz Maliyetli Görüntüleme ve İkaz Sisteminin Geliştirilmesi

Continuity is crucial in systems that provide uninterruptible service. When errors or warnings occur, they should be reported to the responsible people as quickly as possible and fixed right away. Since manual control will cause errors, monitoring and control should be performed automatically. In order to fill this need, expensive hardware and software are used in the market. The purchased package includes a lot of extra features that are not required. Additionally, the majority of these systems use brand-specific sensors and other hardware components. In this study, the identical operation is carried out with significantly greater cost efficiency and brand independence by using microcontroller-based hardware. The designed system can be operated in desktop, web and mobile environments thanks to the computer program that is written. In the data processing center of a university hospital, the proposed cost-effective system has been used, and it has been discovered that it is capable of carrying out the same tasks as the more expensive systems.

Power-Based Side-Channel Attacks on Program Control Flow with Machine Learning Models

The control flow of a program represents valuable and sensitive information; in embedded systems, this information can take on even greater value as the resources, control flow, and execution of the system have more constraints and functional implications than modern desktop environments. Early works have demonstrated the possibility of recovering such control flow through power-based side-channel attacks in tightly constrained environments; however, they relied on meaningful differences in computational states or data dependency to distinguish between states in a state machine. This work applies more advanced machine learning techniques to state machines which perform identical operations in all branches of control flow. Complete control flow is recovered with 99% accuracy even in situations where 97% of work is outside of the control flow structures. This work demonstrates the efficacy of these approaches for recovering control flow information; continues developing available knowledge about power-based attacks on program control flow; and examines the applicability of multiple standard machine learning models to the problem of classification over power-based side-channel information.

Determination of online thin film buckling configuration by parametric optimization for flexible sensor application

A mini basket type mapping catheter consists of thin film flexible sensors and is applied in the medical field to measure the electrocardiography (ECG) signals in order to localize and quantize the physiological condition/status of heart. The flexible nature of the thin film changes the configuration with respect to the contact boundary conditions when it contacts a target surface. Therefore, to accurately localize the flexible sensor, the thin film flexible sensor’s configuration must be determined accurately in an on-line fashion. As a study of localizing the thin film flexible sensor, this study proposes an on-line thin film buckling configuration determination method using parametric optimization and interpolation technique. With the specific modulus of elasticity and dimensions of the thin film flexible sensor of the mapping catheter prototype, the buckling configuration with two point boundary condition under axial load can be calculated in desktop environment. The proposed calculation method is validated by mapping catheter sensor prototype test. The calculation/test results showed that the maximum overall length L, x_{a}, and y_{a} value error between the calculation and experiment are approximately 0.16 mm, − 0.12 mm. − 0.10 mm in 50 ms calculation time. The calculation result of the proposed method is also compared with that of the numerical simulation by FEM, which has approximately 0.44 mm y_{a} value error compared with that of the experiment.