Amazon Web Services Research Articles

Protection of confidential data in Amazon Elastic Block Store snapshots

The article examines security issues in an Amazon Web Services (AWS) environment with an emphasis on the risks associated with public Elastic Block Store (EBS) snapshots. Analyzes how user identification and authorization processes can contribute to security risks and leakage of confidential data. Methods for detecting and monitoring public snapshots are described in detail, as well as recommendations for changing permissions and ensuring the security of credentials. Emphasizes the importance of conducting investigations when public snapshots are discovered, and develops strategies to prevent such incidents, including staff training and enforcement of privacy policies. AWS's approach to security is highlighted, particularly through informing customers about the potential risks associated with public access to snapshots. The paper offers a comprehensive overview of security issues in an AWS environment, focusing on data management, and recommends specific measures to ensure the secure use of EBS snapshots. Potential vulnerabilities are illustrated with real-world examples, demonstrating the importance of properly managing access policies and the informed use of snapshots to prevent accidental data leakage. Particular attention is paid to the impact of open snapshots on various industries and the need for audits to identify potential threats.

SPSC: Stream Processing Framework Atop Serverless Computing for Industrial Big Data.

With the advance of smart manufacturing and information technologies, the volume of data to process is increasing accordingly. Current solutions for big data processing resort to distributed stream processing systems, such as Apache Flink and Spark. However, such frameworks face challenges of resource underutilization and high latency in big data application scenarios. In this article, we propose SPSC, a serverless-based stream computing framework where events are discretized into the atomic stream and stateless Lambda functions are taken as context-irrelevant operators, achieving task parallelism and inherent data parallelism in processing. Also, we implement a prototype of the framework on Amazon Web service (AWS) using AWS Lambda, AWS simple queue service, and AWS DynamoDB. The evaluation shows that compared with Alibaba's real-time computing Flink version, SPSC outperforms by 10.12% when the overhead is close.

Implementing web-based discrete choice experiments in transportation

Discrete choice experiments (DCE) are widely used in transport planning to elicit peoples' preferences for different alternatives and attributes. They rely on stated and/or revealed preference (SP/RP) data. Over the past years, web-based DCEs became popular in many disciplines. Yet, there is little literature on how to actually implement them. This paper provides a framework to implement DCEs in two ways, using the online survey platform Qualtrics in combination with R and Amazon Web Services. Both approaches, picture- and matrix table-based, are presented and explained step by step. A full working example is freely available on GitHub.

DETECTION, ANALYSIS AND PROTECTION OF CONFIDENTIAL DATA USING AMAZON MACIE MACHINE LEARNING TECHNOLOGY

Over the past decades, the field of data storage and processing has undergone significant changes and expansion, especially with the advent of cloud technologies and computing. Cloud services enable organizations to store and access large amounts of data through distributed systems. However, along with these new opportunities come new challenges, particularly in the area of protecting confidential data. Protecting sensitive data is an extremely important task for today's organizations, especially in the face of a growing number of digital threats and security breaches. In order to ensure reliable protection of valuable and sensitive information, developers and researchers are actively working on the development of new technologies and tools. One of the powerful tools used to identify, analyze and protect confidential data is the machine learning technology of the Amazon Macie service. Amazon Macie is an AWS cloud computing service that uses artificial intelligence and machine learning algorithms to automate data analysis and identify potential data security threats. The main purpose of this work is the detection, analysis and protection of confidential data using Amazon Macie machine learning technology. Amazon Macie is an innovative service developed by Amazon Web Services (AWS) that uses advanced machine learning algorithms for automated discovery and analysis of sensitive data. As part of the work, an analysis of the main machine learning algorithms, principles of data storage systems and methods of protecting confidential information was carried out. The working principles and capabilities of Amazon Macie, which uses advanced machine learning algorithms for automated data analysis and detection of potential threats to data security, were investigated.

Optimizing Cloud Resource Management Using PSO

This research explores how cloud resource management is changing in businesses, with a focus on Amazon Web Services (AWS) as the leader in cloud computing. It highlights how crucial excellent resource management is to attaining scalability, cost-effectiveness, and peak performance. The study explores on using Particle Swarm Optimization (PSO) as a cutting-edge optimization method in cloud computing settings. It talks about the difficulties brought on by fluctuating workloads and the requirement for clever resource allocation strategies. Additionally, the study assesses several optimization techniques using performance parameters including computing overhead, convergence time, and solution quality. These techniques include PSO, Genetic Algorithm (GA), and Firefly Algorithm (FA). In-depth simulations and case studies with organizations such as Siemens and Deloitte are used in the study to demonstrate how these algorithms work best in cloud environments to maximize resource usage, cut costs, and improve overall service quality. In the end, it emphasizes the continuous requirement for optimizing techniques to successfully handle the complexity of cloud computing ecosystems.

Gravitational wave alert generation infrastructure on your laptop

Gravitational wave alert generation infrastructure on your laptop

Integration of cloud technologies into virtual reality

Investigated the possibilities of integrating cloud technologies into virtual reality in order to further improve software tools and increase its efficiency and accessibility. Analysed architectural and infrastructure solutions for the use of cloud services, processing and storage of large data in virtual environments. Conducted a comprehensive scientific study of the possibilities of integrating cloud technologies into virtual reality in order to increase the efficiency and accessibility of this technological segment. Studied the challenges that arise when integrating cloud technologies, in particular, the problems of delays in data transmission and measures to ensure security and privacy in cloud environments. Considered the necessity of developing a cloud infrastructure in popular cloud services, in particular, Amazon Web Services (AWS), to optimise computing tasks and increase overall efficiency, as well as the possibility of scaling virtual reality technology. Article considers the possibility of creating a solution for interacting with cloud resources and using virtual reality applications. The services of a cloud provider are used to allocate computing resources, create a data warehouse, create and manage a virtual network, install and configure solutions for deploying and configuring server software components, and conduct an experiment. Evaluated the impact of cloud technology integration on the quality and availability of virtual reality, and provide prospects for the development of this association. In addition, considered the possibilities of creating optimised solutions for interacting with cloud resources and using virtual reality applications, and assessed the impact of cloud integration on the quality and availability of virtual reality. Developed a software solution and approach for building a cloud infrastructure, described in detail all the steps of creating and deploying a virtual server and data storage for its interaction with virtual reality systems. Prospects for the development of such an association open up wide horizons for innovative technologies in the information technology segment. The results of the study can become an important source of relevant strategic information for further improvement of cloud-virtual environments and development of innovative technologies in the information technology segment.

DESIGN AND DEVELOPMENT OF AI CLOUD-BASED VIDEO RECORDING SYSTEM FOR ATHLETE MOVEMENTS

This paper presents the design and development of an AI cloud-based video recording system for athlete move- ment analysis. The proposed system utilizes the Wemos D1 Mini microcontroller as the core hardware platform and a GoPro cam- era for high-quality video capture. By leveraging the capabilities of these components, the system enables real-time video re- cording of athlete movements, facilitating detailed performance analysis and feedback. Furthermore, the system seamlessly inte- grates with Amazon Web Services (AWS) IoT Core, enabling efficient data transmission and storage in the cloud. Through re- search and development, both the hardware and software components of the system were designed and implemented, ensuring robust performance and scalability. Experiments demonstrate the efficacy of the proposed solution in capturing high-fidelity video footage of athlete movements and securely transmitting it to the cloud for further analysis. This research lays the foundation for advanced athlete monitoring systems, offering valuable insights for coaches, trainers, and sports scientists to enhance training regimens and optimize performance.

AWS Cloud Computing Solutions: Optimizing Implementation for Businesses

This study delves into the optimization strategies of Amazon Web Services (AWS) cloud computing solutions tailored specifically for businesses. It provides an in-depth exploration of the transformative impact that cloud computing has had on modern business operations, emphasizing the pivotal role played by AWS in delivering scalable, flexible, and cost-effective resources. With projections indicating the global public cloud services market's growth to $332.3 billion in 2021, it becomes evident that cloud solutions are increasingly relied upon, with AWS commanding a significant 32% market share. This paper highlights the importance of optimizing AWS solutions, considering its extensive range of services designed to meet diverse business needs. However, transitioning to cloud environments brings challenges related to data security, compliance, integration, and migration complexities, as evidenced by pertinent studies and literature reviews. This research synthesizes insights gleaned from various studies, emphasizing the adaptability and cost-effectiveness of AWS while stressing the critical role of robust security measures in its implementation across industries and geographic regions. Understanding prevalent business requisites and challenges in cloud adoption, including scalability, cost efficiency, reliability, availability, and compliance, is essential for organizations seeking to harness cloud technology effectively. Furthermore, this paper provides an overview of AWS's services encompassing computing, storage, databases, machine learning, and AI, showcasing how these empower businesses to streamline operations, foster innovation, and scale dynamically. It elucidates successful instances where AWS was strategically implemented in Netflix, Airbnb, Jollibee Group, and Capital One, demonstrating its impact on scalability, innovation, and service enhancement. The paper outlines anticipated trends in AWS and cloud computing, focusing on user-friendly tools, advancements in machine learning, infrastructure improvements, increased automation, and deeper integration of AI and advanced analytics. Ultimately, this research emphasizes business's need to optimize AWS solutions, navigate complexities, foster innovation, and achieve operational excellence in an increasingly competitive market.

An IoT-Based Hydroponic Monitoring and Control System for Sustainable Food Production

The agricultural sector, historically the main source of food globally, faces unprecedented challenges due to decreasing soil fertility and unsustainable farming practices. This study presents a smart farming method, a tailored approach to cultivating food and arable crops, as a sustainable and innovative farming method. Hydroponics involves growing plants in nutrient water solutions rather than soil, establishing a controlled environment free from external conditions. We developed a sophisticated hydroponics system equipped with sensors monitoring temperature, humidity, pH, and light levels. Using IoT technologies, we enabled real-time data transmission from sensors to a dedicated web server - Amazon web services (AWS) and an Android mobile application powered by a Raspberry Pi3. This setup allows remote monitoring and control of the hydroponic system, reducing human intervention. The experiment validated the sensor network's efficiency in accurately transmitting data. The findings suggest that widespread adoption of hydroponic farming could substantially enhance global food security. Future advancements in this method could include real-time monitoring of plant growth rates by incorporating additional sensors, like carbon dioxide (CO2) sensors. This research is a significant step toward sustainable agriculture. It offers a technologically advanced solution capable of transforming food production methods and ensuring a more secure and sustainable food supply worldwide.

Leveraging AWS SNS/SQS for Real-Time Cache Synchronization in Distributed Systems

In distributed systems, maintaining consistency across multiple caches presents a significant challenge, often leading to data inconsistencies and degraded application performance. This paper proposes and evaluates a novel approach to cache synchronization using Amazon Web Services (AWS) Simple Notification Service (SNS) and Simple Queue Service (SQS). The proposed system leverages the pub-sub model of SNS combined with the reliable message queuing of SQS to create a scalable and robust cache synchronization mechanism. We present the architecture, implementation details, and performance analysis of this system, demonstrating significant reductions in database load and improvements in response times while ensuring eventual consistency across distributed caches. Comparative analysis with alternative solutions highlights the advantages of the SNS/SQS approach in terms of scalability, reliability, and ease of implementation. This research contributes to the field of distributed systems by offering a practical, cloud-native solution to the cache synchronization problem, applicable to a wide range of applications requiring consistent data across multiple instances or services.

Advancements in Cloud-Based Machine Learning: Navigating Deployment and Scalability

The widespread adoption of machine learning (ML) in various industries has brought to light significant challenges, particularly in deploying these complex models into production environments. The need for scalable, efficient, and robust solutions is paramount, and cloud computing emerges as a key player in this scenario. Cloud platforms offer the necessary infrastructure and tools to facilitate ML deployment, addressing issues like computational demand, data storage, and scalability. Within the cloud computing landscape, AWS SageMaker, a service provided by Amazon Web Services, has gained prominence. This paper undertakes a comprehensive review of the machine learning (ML) lifecycle within cloud-based platforms with a specific focus on AWS SageMaker. Additionally, this paper explores the critical aspect of scaling in ML deployment, analyzing both horizontal and vertical scaling methods within the context of cloud computing's dynamic resource management. This paper aims to deliver an in-depth analysis of the ML lifecycle in cloud platforms by elucidating the functionalities, benefits, and challenges of using AWS SageMaker in the broader spectrum of ML deployment and management.

Studi Independent Prediksi Harga Komoditi Petani Bawang Merah Menggunakan Amazon Forecast di PT. Techno International Mandira

The Certified Independent Study, under the Kampus Merdeka initiative and supported by the Ministry of Education, Culture, Research, and Technology (Kemendikbud Ristekdikti), fosters collaborative partnerships with key industry players. A prime example is PT. Techno International Mandira's contribution, which involves employing Amazon Forecast within Amazon Web Services for the predictive analysis of shallot farmers' commodity prices. This study centers on developing web-based predictive models to accurately forecast shallot market values. Utilizing Amazon Forecast's fully managed services, which integrate advanced machine learning algorithms, this project aims to provide vital insights into commodity pricing trends, thereby supporting shallot farmers in strategic decision-making. The methodology encompasses the application of Amazon Forecast for efficient and data-driven price prediction, demonstrating the practical utility of machine learning in agricultural economics.

Design and analyses of web scraping on burstable virtual machines

SummaryWeb scraping is a widely used technique for decision‐making, collecting, and structuring public data from the internet. As the volume of data continues to grow, the need for more efficient methods of data extraction becomes crucial. This article introduces a novel web scraping framework that utilizes Burstable virtual machines (VMs) on Amazon Web Services with the objective of reducing the monetary cost of execution while ensuring compliance with service level agreements (SLAs). To achieve this, the framework utilizes a combination of fixed and temporary Burstable VMs in a mixed cluster, which can be elastically scaled up to fulfill the SLA and scaled down to minimize monetary costs. Two strategies for handling VM allocation are proposed and evaluated: (i) a queue and SLA‐based strategy that employs queue size information and SLA criteria to determine the required number of VMs for the current scraping requests, and (ii) a credit‐based strategy that incorporates information about Burstable VM credits to effectively manage instance creation and termination. Experimental tests show that the proposed framework meets the defined SLA while achieving cost reductions of up to 74% compared to an approach that executes on fixed‐size clusters of Burstable instances.

Rescheduling serverless workloads across the cloud-to-edge continuum

Serverless computing was a breakthrough in Cloud computing due to its high elasticity capabilities and fine-grained pay-per-use model offered by the main public Cloud providers. Meanwhile, open-source serverless platforms supporting the FaaS (Function as a Service) model allow users to take advantage of many of their benefits while operating on the on-premises platforms of organizations. This opens the possibility to deploy and exploit them on the different layers of the cloud-to-edge continuum, either on IoT (Internet of Things) devices located at the Edge (i.e. next to data acquisition devices), in on-premises clusters closer to the data sources (i.e. Fog computing) or directly on the Cloud.This paper presents two strategies to mitigate the overload that disparate data ingestion rates may cause in low-powered devices at the Edge or Fog layers. To this end, it is proposed to delegate and reschedule serverless jobs between the different layers of the cloud-to-edge continuum using an open-source platform for event-driven file processing. To demonstrate the performance of these strategies, a use case for fire detection is proposed that includes processing in the Fog via minified Kubernetes clusters located near the Edge, in the private Cloud via on-premises elastic clusters and, finally, in the public Cloud by using the AWS (Amazon Web Services) Lambda FaaS service. The results indicate that these strategies can mitigate overloads in use cases involving processing across the cloud-to-edge continuum by coordinating several layers of computing resources.

Securing Construction Workers’ Data Security and Privacy with Blockchain Technology

The construction industry, characterized by its intricate network of stakeholders and diverse workforce, grapples with the challenge of managing information effectively. This study delves into this issue, recognizing the universal importance of safeguarding data, particularly amid rising concerns around unauthorized access and breaches. Aiming to harness the potential of blockchain technology to address these challenges, this study used hypothetical biographical and safety data of construction workers securely stored on a Hyperledger Fabric blockchain. Developed within the Amazon Web Services (AWS) cloud platform, this blockchain infrastructure emerged as a robust solution for enhancing data security and privacy. Anchored in the core principles of data security, the model emerges as a potent defender against the vulnerabilities of traditional data management systems. Beyond its immediate implications, this study exemplifies the marriage of blockchain technology and the construction sector, and its potential for reshaping workforce management, especially in high-risk projects and optimizing risk assessment, resource allocation, and safety measures to mitigate work-related injuries. Practical validation through transaction testing using Hyperledger Explorer validates the model’s feasibility and operational effectiveness, thus serving as a blueprint for the industry’s data management. Ultimately, this research not only showcases the promise of blockchain technology in addressing construction data security challenges but also underscores its practical applicability through comprehensive testing, thus heralding a new era of data management that harmonizes security and efficiency for stakeholders’ benefit.

Evaluation and Challenges of IoT Simulators for Intelligent Transportation System Applications

The Internet-of-Things (IoT) constructs a vast, intricate, and perpetually evolving ecosystem exerting profound societal implications. This labyrinthine nature often culminates in errors that directly impact human lives. A significant domain where this complexity materializes is Intelligent Transportation Systems (ITS). Present tools and methodologies inadequately accommodate the complex task of testing and validation, underscoring the urgency for comprehensive review and enhancement. This study aims to present a broad analysis of existing simulators utilized for ITS simulations. It delves into the role and effectiveness of such simulation tools, highlighting their limitations and proposing research directions. This paper scrutinizes both commercial and research-oriented IoT simulators for ITS, evaluating their features and simulation environment tools. We have detailed various ITS scenarios simulated within these frameworks, intending to gauge their readiness for real-world ITS applications and to elaborate on the challenges involved in ITS infrastructure implementation. The findings suggest that despite numerous simulators aiding the evolution of solutions for IoT challenges in recent years, their utility in actual ITS implementations remain uncertain. Consequently, we explore public cloud platforms offering IoT simulation capabilities, focusing particularly on the capabilities provided by the Amazon Web Services (AWS) IoT simulation for this study. Our research outlines the pressing challenges in this field, while proposing potential solutions and flagging opportunities for further research. This study paves the way towards improving the reliability and accuracy of IoT simulators in the context of ITS, which has immense potential to enhance the quality of human life.

Development of a Microservice-Based Storm Sewer Simulation System with IoT Devices for Early Warning in Urban Areas

This study proposes an integrated approach to developing a Microservice, Cloud Computing, and Software as a Service (SaaS)-based Real-Time Storm Sewer Simulation System (MBSS). The MBSS combined the Storm Water Management Model (SWMM) microservice running on the EC2 Amazon Web Services (AWS) cloud platform and an Internet of Things (IoT) monitoring device to prevent disasters in smart cities. The Python language and Docker container were used to develop the MBSS and Web API of the SWMM microservice. The IoT comprised a pressure water level meter, an Arduino, and a Raspberry Pi. After laboratory channel testing, the simulated and IoT-monitored water levels under different flow rates indicate that the simulated water level in MBSS was such as that monitored by the IoT. These findings suggest that MBSS is feasible and can be further used as a reference for smart urban early warning systems. The MBSS can be applied in on-site stormwater sewers during heavy rain, with the goal of issuing early warnings and reducing disaster damage. The use case can be the process by which the SWMM model parameters will be optimized based on the water level data from IoT monitoring devices in stormwater sewer systems. The predicted rainfall will then be used by the SWMM microservices of MBSS to simulate the water levels at all manholes. The status of the water levels will finally be applied to early warning.

Multi‐Ancestry Genetics for Alzheimer’s Disease: Alzheimer’s Disease Genetics Consortium

AbstractBackgroundAlzheimer’s disease (AD) affects individuals of all races and ethnicities, yet large‐scale genetic research to compare genetic makeups for risk or protection of AD between multi‐ancestry datasets is limited. The Alzheimer’s Disease Genetics Consortium (ADGC) assembled the largest multi‐ancestry dataset including 56,241 subjects from non‐Hispanic Whites (NHW), African Americans (AA), Hispanics (HI), and East Asians (EAS) to understand ancestral similarities and differences in genetic risk and protective variants for AD.MethodWe established a Steering Committee with 5 members, 4 main Workgroups including Primary Meta‐Analysis, Follow‐Up Analysis, Secondary Analysis, and Method Evaluation, and 3 affiliated Workgroups consisting of Phenotype Harmonization, Imputation Development, and genome‐wide association study (GWAS)/whole genome sequencing (WGS) Integration. A new computing infrastructure was developed for shared analytic space on Amazon Web Services (AWS), where all members from different institutes accessed to genetic and phenotypic data and conducted analysis under the master analysis plan with centrally developed pipelines and scripts. We conducted ancestry‐specific GWAS using single nucleotide polymorphisms (SNPs) for AD, cross‐ancestry meta‐analysis of 4 ancestry GWASs, fine‐mapping for allelic heterogeneity in known and novel loci for AD, and pathway analysis.ResultWe confirmed 12 known including APOE ɛ4 and ɛ2 variants and identified 2 novel loci for AD in cross‐ancestry meta‐analysis with P<5.4×10−8. African and Asian ancestry subjects displayed greater heterogeneity in effect sizes and significant levels with the previously known AD loci for AD. Ancestry‐specific fine‐mapping indicated presence of potential distinct causal variants across different ancestry groups from multiple known AD loci. Two APOE alleles had greatest effect sizes among East Asians compared to others (EAS>NHW>AA>HI), while we failed to observe significant association signals with AD in SHARPIN and TREM2 regions from African Americans. Of two novel loci, the association signal in the LRRC4C‐API5 region was contributed from all four ancestry groups; in contrast, the signal in the LHX5‐AS1‐RBM19 region was predominantly from Hispanics.ConclusionGenetic risk profiling in multi‐ancestry datasets provided unique opportunity to understand ancestry‐specific and cross‐ancestry genetic risk factors for AD. Further evaluation of these established and novel loci for AD by integrating GWAS and WGS data from ADSP is underway

Fuzzy Perspective of Online Games by Using Cryptography and Cooperative Game Theory

In this paper, we mathematically associate crypto-cloud computing with cooperative game theory in the presence of fuzzy uncertainty. For this purpose, we retrieve information from the database of Amazon Web Service. Then we construct a secure crypto-cloud game theoretical model and apply this model to online games under fuzzy uncertainty. Further, we suggest some fuzzy game theoretical solutions by proposing a novel twisted Edwards curve pairing-based scheme over finite fields having the property of indistinguishable under chosen ciphertext attacks. Finally, it is seen that costs of the numerical fuzzy solutions are reduced and their efficiency is increased compared with the numerical crisp ones.