Legal Challenges of Automated and Autonomous Systems

Abstract

With an increasing influence of computers and software, automation is affecting many areas of daily life. Autonomous systems have become a central notion, but many systems have reached only a lower level of automation and not yet full autonomy. Information technology and software have a strong impact and their industries are introducing their own business cultures. Even though autonomy will enable systems to act independently from direct human input and control in complex scenarios, the factors of responsibility, control, and attribution are of crucial importance for a legal framework. Legal responsibility has to serve as a safeguard of fundamental rights. Responsibility can be attributed by a special legal regime, and mandatory human override and fallback modes can assure human intervention and control. It is proposed to establish a precautionary regulatory regime for automated and autonomous systems to include general principles on responsibility, transparency, training, human override and fallback modes, design parameters for algorithms and artificial intelligence, and cyber security. States need to take a positivist approach, maintain their regulatory prerogative, and, in support of their exercise of legislative and executive functions, establish an expertise independent of industry in automation, autonomy, algorithms, and artificial intelligence.