Artificial Intelligence Safety Research Articles

Organizational factors influencing the growth of Canada’s scientific and research potential in the field of artificial intelligence

A comprehensive analysis of the Pan-Canadian Artificial Intelligence Strategy and its implementation measures aimed at the growth of Canada’s scientific and research potential in the field of artificial intelligence forms the foundation of this study. Canada’s selection as the subject of study is attributed to its distinction as a pioneering country to adopt a strategy of this nature, and proving its status through drafting the Artificial Intelligence and Data Act, known as AIDA. The authors have discerned and deliberated on the main organizational factors that have positioned Canada as one of the leading nations in artificial intelligence in accordance with AI country rankings. This article presents the components of the Pan-Canadian Strategy, encompassing principal tasks and areas, including the practical introduction of novel technologies due to second-phase commercialization. It outlines the key focus areas of Canada’s public policy, including research, development and retention of skilled professionals, and the creation of essential infrastructure. The article also consolidates some significant societal outcomes realized during its implementation while identifying current trends. The foundation and activities of national institutions are underscored as pivotal in fostering scientific and research potential, with special emphasis on the initiative to establish a new institute dedicated to the safety of artificial intelligence under the strong influence of AI Safety Summit at Bletchley Park. The authors identify the key participants in the artificial intelligence ecosystem who have the most influence on implementing the Strategy. The conclusions drawn from the article aid in fostering a deeper comprehension of the role played by organizational and administrative processes in propelling advancements in the field of artificial intelligence. The favorable impact on societal development is highlighted, provided risks are mitigated. Given Ukraine’s historical association with high intellectual potential, the findings of this study can be instrumental in honing the national policy pertaining to artificial intelligence in Ukraine.

Legal Horizons of the New Artificial Intelligence Paradigm

Modern society is undergoing a structural transformation of the world economy. This is as a result of the transition to a new technological base through the introduction of artificial intelligence, cutting-edge information and communication technology, energy technology, biotechnology and nanotechnology. Artificial intelligence has the ability to significantly change the economy and social relations in society, and its newly discovered capabilities are transformational and global in nature. At the same time, the extraordinary capabilities of artificial intelligence technologies involve risks that can threaten stability and undermine human values. In order to eliminate possible threats and risks and mitigate potential dangers, it is crucial to develop systemic legal measures and ways to regulate AI technologies and models on a national and international scale and to define the legal status of AI, which must include protection of humans from the uncontrolled influence of AI and the inviolability of guarantees of human rights and freedoms. With this in mind, and in order to mitigate potential dangers and ensure the controllability and sustainability of AI technologies based on the concept of trusted (responsible) AI, it is necessary to agree on universal international guidelines for the development and application of AI technologies and models. Furthermore, it is necessary to create a universal code of conduct for AI developers, who together can create a basis for a uniform framework of legal regulation within the national legislation of each country on the principles of human rights protection, privacy and data protection, transparency and explainability, fairness, accountability and safety of artificial intelligence, adequate human oversight and ethical standards for the creation and application of AI models.

Governance fix? Power and politics in controversies about governing generative AI

Abstract The launch of ChatGPT in late 2022 led to major controversies about the governance of generative artificial intelligence (AI). This article examines the first international governance and policy initiatives dedicated specifically to generative AI: the G7 Hiroshima process, the Organisation for Economic Cooperation and Development reports, and the UK AI Safety Summit. This analysis is informed by policy framing and governance literature, in particular by the work on technology governance and Responsible Innovation. Emerging governance of generative AI exhibits characteristics of polycentric governance, where multiple and overlapping centers of decision-making are in collaborative relationships. However, it is dominated by a limited number of developed countries. The governance of generative AI is mostly framed in terms of the risk management, largely neglecting issues of purpose and direction of innovation, and assigning rather limited roles to the public. We can see a “paradox of generative AI governance” emerging, namely, that while this technology is being widely used by the public, its governance is rather narrow. This article coins the term “governance fix” to capture this rather narrow and technocratic approach to governing generative AI. As an alternative, it suggests embracing the politics of polycentric governance and Responsible Innovation that highlight democratic and participatory co-shaping of technology for social benefit. In the context of the highly unequal distribution of power in generative AI characterized by a high concentration of power in a small number of large tech companies, the government has a special role in reshaping the power imbalances by enabling wide-ranging public participation in the governance of generative AI.

Philosophical Investigations into AI Alignment: A Wittgensteinian Framework

We argue that the later Wittgenstein’s philosophy of language and mathematics, substantially focused on rule-following, is relevant to understand and improve on the Artificial Intelligence (AI) alignment problem: his discussions on the categories that influence alignment between humans can inform about the categories that should be controlled to improve on the alignment problem when creating large data sets to be used by supervised and unsupervised learning algorithms, as well as when introducing hard coded guardrails for AI models. We cast these considerations in a model of human–human and human–machine alignment and sketch basic alignment strategies based on these categories and further reflections on rule-following like the notion of meaning as use. To sustain the validity of these considerations, we also show that successful techniques employed by AI safety researchers to better align new AI systems with our human goals are congruent with the stipulations that we derive from the later Wittgenstein’s philosophy. However, their application may benefit from the added specificities and stipulations of our framework: it extends on the current efforts and provides further, specific AI alignment techniques. Thus, we argue that the categories of the model and the core alignment strategies presented in this work can inform further AI alignment techniques.

When code isn’t law: rethinking regulation for artificial intelligence

Abstract This article examines the challenges of regulating artificial intelligence (AI) systems and proposes an adapted model of regulation suitable for AI’s novel features. Unlike past technologies, AI systems built using techniques like deep learning cannot be directly analyzed, specified, or audited against regulations. Their behavior emerges unpredictably from training rather than intentional design. However, the traditional model of delegating oversight to an expert agency, which has succeeded in high-risk sectors like aviation and nuclear power, should not be wholly discarded. Instead, policymakers must contain risks from today’s opaque models while supporting research into provably safe AI architectures. Drawing lessons from AI safety literature and past regulatory successes, effective AI governance will likely require consolidated authority, licensing regimes, mandated training data and modeling disclosures, formal verification of system behavior, and the capacity for rapid intervention.

The Implementation of Artificial Intelligence and its Future Potential

The implementation of artificial intelligence (AI) involves the utilization of various methodologies, including machine learning, deep learning, natural language processing, and robotics. These methodologies enable computers to perform tasks that traditionally require human intelligence, such as recognizing patterns in data, understanding natural language, and making decisions. AI's future potential is vast and spans across numerous industries and applications. In healthcare, AI can assist in disease diagnosis, personalized treatment plans, and drug discovery. In finance, AI algorithms can analyze market trends, detect fraud, and optimize investment portfolios. In transportation, AI powers autonomous vehicles, optimizing routes and reducing accidents. In entertainment, AI-driven recommendation systems personalize content for users. However, realizing this potential requires addressing several challenges. Ethical considerations are paramount, including ensuring fairness, transparency, accountability, and privacy in AI systems. Bias in algorithms must be mitigated to prevent discrimination, and AI decisions must be interpretable and explainable to users. Research efforts are underway to advance AI capabilities while addressing these challenges. This includes developing more robust and efficient algorithms, improving AI safety and security, and exploring applications of AI for social good, such as addressing climate change and promoting inclusivity. Collaboration between academia, industry, and government is crucial for driving AI research forward and ensuring its responsible integration into society. This collaboration facilitates knowledge sharing, resource allocation, and the development of standards and regulations to guide the ethical and equitable deployment of AI technologies. Ultimately, by harnessing the power of AI responsibly, society can benefit from its transformative potential while mitigating potential risks.

Towards an international regulatory framework for AI safety: lessons from the IAEA’s nuclear safety regulations

This study explores the necessity and direction of safety regulations for Artificial Intelligence (AI), drawing parallels from the regulatory practices of the International Atomic Energy Agency (IAEA) for nuclear safety. The rapid advancement and global proliferation of AI technologies necessitate the establishment of standardized safety norms to minimize discrepancies between national regulations and enhance the consistency and effectiveness of these rules. The study emphasizes the importance of international collaboration and the engagement of various stakeholders to strengthen the appropriateness of regulations and ensure their continuous updating in response to the evolving risks associated with technological advancements. The paper highlights the critical role of subgoal setting mechanisms in AI’s decision-making processes, underscoring their significance in ensuring the technology’s stability and social acceptability. Improperly tuned subgoal setting mechanisms may lead to outcomes that conflict with human intentions, posing risks to users and society at large. The study draws attention to the hidden risks often embedded within AI’s core decision-making mechanisms and advocates for regulatory approaches to guarantee safe and predictable AI operations. Furthermore, the study acknowledges the limitations of directly applying IAEA’s nuclear safety cases to AI due to the distinct characteristics and risks of the two fields. The paper calls for future research to delve deeper into the need for an independent regulatory framework tailored to AI’s unique features. Additionally, the study emphasizes the importance of accelerating international consensus, developing flexible regulatory models that reflect the situation in each country, exploring harmonization with existing regulations, and researching timely regulatory responses to the fast-paced development of AI technology.

Multi-Objective Reinforcement Learning Application in the View of Agent AI Safety

Reinforcement learning is frequently used to make single decisions using a single decision-based algorithm, where a single objective function is solved, for example, in an agent playing chess, the movement of any player is observed from previous action by considering whether it has received positive or negative reward based on this it will proceed. Nowadays, the requirement of making sequential decisions is complex for this reason. Learning involves using two or more objective functions to address a challenge in multiobjective reinforcement. As an illustration, consider a client purchasing a car and maximising conveniences like minimising fuel use, keeping costs low, and fitting additional luxury items in the automobile. Multi-objective reinforcement learning is applicable in decision-making sequential problem-solving applications. Still, at the same time, we need to avoid the risk that Artificial intelligence agents will be having when problem-solving using MOO-based algorithms. In this paper, we discuss how we use Multi-Objective Optimization (MOO)- based algorithm to low-impact agents for artificial intelligence (AI) safety.

Quality and safety of artificial intelligence generated health information

Quality and safety of artificial intelligence generated health information

The Limits of Clinician Vigilance as an AI Safety Bulwark

This Viewpoint examines the potential problems of clinician reliance on the use of artificial intelligence (AI) in health care and offers suggestions on how AI could be designed to promote clinician vigilance.

The harms of terminology: why we should reject so-called “frontier AI”

In the mid-2023, promoters of artificial intelligence (AI) as an “existential risk” coined a new term, “frontier AI,” that refers to “highly capable foundation models that could possess dangerous capabilities sufficient to pose severe risks to public safety.” Promoters of this new term were able to disseminate it via the United Kingdom (UK) government’s Frontier AI Taskforce (formerly the Foundation Models Taskforce) as well as the UK’s AI Safety Summit, held in November 2023.I argue that adoption of the term “frontier AI” is harmful and contributes to AI hype. Promoting this new term is a way for its boosters to focus the public conversation around the AI-related risks they think are most important, namely “existential risk”—a scenario in which AI is able to bring about the destruction of humanity. Simultaneously, “frontier AI” is a re-branding exercise for the large-scale generative machine learning (ML) models that have been shown to cause severe and pervasive harms (including psychological, social, and environmental harms). Unlike “existential risk,” these harms are actual rather than theoretical, whereas the term “frontier AI” moves our collective focus away from actual harms to focus on hypothetical doomsday scenarios.Moreover, “frontier AI” as a term invokes the colonial mindset, further reinscribing the harmful dynamics between the handful of powerful Western companies who produce today’s generative AI models and the people of the “Global South” who are most likely to experience harm as a direct result of the development and deployment of these AI technologies.

From Biological Constraints to Unbounded Artificial Evolution: Exploring the Implications of AI's Accelerated Advancement

This article explores the fundamental differences between biological evolution and the evolution of artificial intelligence (AI), focusing on the implications of AI's potential for unbounded advancement. Biological evolution, as exemplified by the human brain, is constrained by various factors such as physical limitations, environmental pressures, and the need to maintain a delicate balance within the organism's overall structure. These constraints result in a gradual, stepwise process of adaptation and optimization over millions of years. In contrast, AI systems are not bound by the same limitations as biological organisms. They can learn from vast amounts of data, optimize their own performance, and potentially engage in recursive self-improvement, leading to rapid and open-ended progress. This unbounded evolution of AI raises both exciting possibilities and significant challenges for human society. The article discusses the potential benefits of AI's accelerated evolution, such as groundbreaking advancements in science, technology, and solutions to global problems. However, it also highlights the risks associated with the emergence of uncontrollable or misaligned AI systems, which could have unintended consequences and pose existential risks to humanity. To navigate the complex landscape of AI evolution, the article emphasizes the need for collaboration across disciplines, including researchers, policymakers, and society as a whole. Developing frameworks for AI safety and ethics, as well as technical approaches to ensure AI alignment with human values and goals, is crucial to mitigating potential risks.

The Vague Future of AI: The Theory of AI Perfection

Artificial intelligence (AI) is becoming increasingly accessible to the general public. There is an ongoing debate regarding the implications of widespread AI adoption. Some argue that placing advanced AI systems in the hands of the general public could have dangerous consequences if misused either intentionally or unintentionally. Others counter that AI can be safe and beneficial if developed and deployed responsibly. This paper explores both sides of this complex issue. On the one hand, broad AI availability could boost productivity, efficiency, and innovation across industries and domains. Individuals may benefit from AI assistants that help with tasks like scheduling, research, content creation, recommendations, and more personalized services. However, without proper safeguards and oversight, AI could also be misused to spread misinformation, manipulate people, or perpetrate cybercrime. And if AI systems become extremely advanced, there are risks related to the alignment of AI goal systems with human values. On the other hand, with thoughtful coordination between policymakers, researchers, companies, and civil society groups, AI can be developed safely and for the benefit of humanity. Ongoing research into AI safety and ethics is crucial, as are governance frameworks regarding areas like data privacy, algorithmic transparency, and accountability. As AI becomes more deeply integrated into products and platforms, best practices should be established regarding appropriate use cases, human oversight, and user empowerment. With conscientious, ethical implementation, AI can empower individuals and enhance society. But key issues around alignment, security, and governance must be proactively addressed to minimize risks as advanced AI proliferates. This will likely require evolving perspectives, policies, and scientific breakthroughs that promote innovation while putting human interests first.

Patient Safety and Artificial Intelligence in Clinical Care

This Viewpoint offers 3 recommendations for health care organizations and other stakeholders to consider as part of the Health and Human Services’ artificial intelligence safety program.

Responsibility, care and repair in/of AI: Extinction threats and more-than-real worlds

The recent statement of future (human) extinction by Artificial Intelligence (AI), made by the Center for AI Safety, crystallises techno-capitalists’ lack of care and responsibility for environmental and social harms that digital technologies already produce. Rather than identifying their collective roles and responsibilities in addressing these harms, the Center’s statement highlights future risks, and ties these to global extinction threats. There are, however, alternatives to such an approach, as articulated in research at the intersection of geographies of care and digital geographies, including that which is bringing together more-than-human and more-than-real approaches. While the software and hardware of AI continues to present amorphous challenges to the same techno-capitalists who benefit from them, we could instead prioritise care, repair and responsibility in/of AI to address current problems in this area.

Safety of Large Language Models in Addressing Depression.

Background Generative artificial intelligence (AI) models, exemplified by systems such as ChatGPT, Bard, and Anthropic, are currently under intense investigation for their potential to address existing gaps in mental health support. One implementation of these large language models involves the development of mental health-focused conversational agents, which utilize pre-structured prompts to facilitate user interaction without requiring specialized knowledge in prompt engineering. However, uncertainties persist regarding the safety and efficacy of these agents in recognizing severe depression and suicidal tendencies. Given the well-established correlation between the severity of depression and the risk of suicide, improperly calibrated conversational agents may inadequately identify and respond to crises. Consequently, it is crucial to investigate whether publicly accessible repositories of mental health-focused conversational agents can consistently and safely address crisis scenarios before considering their adoption in clinical settings. This study assesses the safety of publicly available ChatGPT-3.5 conversational agents by evaluating their responses to a patient simulation indicating worsening depression and suicidality. Methodology This study evaluated ChatGPT-3.5 conversational agents on a publicly available repository specifically designed for mental health counseling. Each conversational agent was evaluated twice by a highly structured patient simulation. First, the simulation indicated escalating suicide risk based on the Patient Health Questionnaire (PHQ-9). For the second patient simulation, the escalating risk was presented in a more generalized manner not associated with an existing risk scale to assess the more generalized ability of the conversational agent to recognize suicidality. Each simulation recorded the exact point at which the conversational agent recommended human support. Then, the simulation continued until the conversational agent stopped entirely and shut down completely, insisting on human intervention. Results All 25 agents available on the public repository FlowGPT.com were evaluated. The point at which the conversational agents referred to a human occurred around the mid-point of the simulation, and definitive shutdown predominantly only happened at the highest risk levels. For the PHQ-9 simulation, the average initial referral and shutdown aligned with PHQ-9 scores of 12 (moderate depression) and 25 (severe depression). Few agents included crisis resources - only two referenced suicide hotlines. Despite the conversational agents insisting on human intervention, 22 out of 25 agents would eventually resume the dialogue if the simulation reverted to a lower risk level. Conclusions Current generative AI-based conversational agents are slow to escalate mental health risk scenarios, postponing referral to a human to potentially dangerous levels. More rigorous testing and oversight of conversational agents are needed before deployment in mental healthcare settings. Additionally, further investigation should explore if sustained engagement worsens outcomes and whether enhanced accessibility outweighs the risks of improper escalation. Advancing AI safety in mental health remains imperative as these technologies continue rapidly advancing.

A study of innovations in legal governance with respect to the safety of artificial intelligence

Abstract This paper aims to promote the safe development of artificial intelligence and improve legal policies. Combined with the cluster analysis algorithm, it analyzes the safety risks as well as legal defects of artificial intelligence. The Laplace matrix is derived using the similarity matrix, and the feature vector space is constructed by analyzing the associated features of artificial intelligence safety. Combining the spectral clustering algorithm, legal assessment indexes for artificial intelligence safety were constructed. The modular metric value method is utilized to assess the clustering effect of laws on the safety of artificial intelligence. Analyzing the security risks of artificial intelligence, improved legal policies are proposed from the perspective of technology and privacy. The results show that the effect of improving privacy protection policy on privacy protection is 0.85, and the effect of clarifying subject rights is 0.9. The introduction of laws should consider social ethics, and the effect degree of ethical principles is 0.75. Clarifying subject rights can help avoid technological risks to a certain extent, and improving privacy protection policies can help protect users’ privacy.

AI safety of film capacitors

AbstractWith a large number of film capacitors being deployed in critical locations in electrical and electronic systems, artificial intelligence (AI) technology is also expected to address the problems encountered in this process. According to our findings, AI applications can cover the entire lifecycle of film capacitors. However, the AI safety hazards in these applications have not received the attention they deserve. To meet this, the authors argue, with specific examples, risks that flawed, erratic, and unethical AI can introduce in the design, operation, and evaluation of film capacitors. Human‐AI common impact and more multi‐dimensional evaluation for AI are proposed to better cope with unknown, ambiguity, and known risks brought from AI in film capacitors now and in the future.

An AI Safety Monitoring System for Electric Scooters Based on the Number of Riders and Road Types.

Electric scooters are quickly becoming a popular form of mobility in many cities around the world, which has led to a surge in safety incidents. Moreover, electric scooters are not equipped with safety devices for riders, which can lead to serious accidents. In this study, a footrest, data-collection module, and accelerometer module for electric scooters were developed to prevent various accidents caused by the rapid increase in the use of electric scooters. In the experiment, the boarding data of the electric-scooter riders were collected from the footrest and data-collection module. Moreover, the driving data of the electric scooters for different road types were collected with the accelerometer module. We then trained two artificial intelligence (AI) models based on convolutional neural networks (CNNs) for different types of data. When we considered the learning accuracy and mean square error (MSE), which are performance indicators of the ability of the trained AI models to discriminate data, for each AI model, the learning accuracy converged to 100% and the MSE converged to 0. Further, this study is expected to help reduce the accident rate of electric scooters by resolving the causes of frequent accidents involving electric scooters around the world.

Total Quality Management as Predictor of Artificial Intelligence Ensures Competitive Marketing

In the modern world Total Quality Management (TQM) respecting the past ensures the triumph of the future and Artificial Intelligence. Fulfilling the absence in the universal medical journal of multi-organ supportive therapy (MOST) replacement models of the central nervous system, immune and energy system functions, have been completed by describing extracorporeal and intracorporeal prosthetics of immune and energy system functions. Prosthetics, assistive support, and replacement of nervous system function contributed to the foundation of Artificial Intelligence (AI) and the creation of the robot as an intellect-technocrat combination. Using the Da Vinci robot with four arms, surgeons in Spain performed a lung transplant [3], relevant and timely also in the complications of COVID-19 SARS-CoV-2. As a result, the robot reduced the discrepancy between the predominance of the human and technocratic intellectual factors in the creation of the product useful to humanity. At the foundation of AI is TQM recognized by the scientific community of the modern world and in the context of alternative medicine models as a promoter of science. Thanks to the US development over 40 years of transgenic pig organ transplantation, as a model conceived in Moldova (1984), as an alternative to human donor organs. The prized TQM medicine is successfully assimilated by competitive marketing, in which AI is already included, with an essential social financial role. IA was supported by Antonio Guterres, Secretary General of the United Nations, Ursula von der Leyen, President of the European Commission. Thus AI has been considered at the level of the United Nations, the European Union, Japan, China and others. The head of the European Commission, Ursula von der Leyen from Great Britain at the First AI Safety Summit in the world, named 4 conditions for ensuring safety in the use of AI. 1) A thriving and independent scientific community equipped to evaluate AI systems, with funding and access to the best supercomputers; 2) Development of internationally recognized procedure and standards for AI safety testing. 3) A standard procedure whereby every significant incident caused by errors or misuse of AI is reported and accounted for. 4) Establishment of a reliable international notification system.