Roger Clarke's Web-Site
© Xamax Consultancy Pty Ltd, 1995-2019
|Identity Matters||Other Topics||Waltzing Matilda||What's New|
Review Draft of 3 August 2019
Roger Clarke **
© Xamax Consultancy Pty Ltd, 2019
Available under an AEShareNet licence or a Creative Commons licence.
This document is at http://www.rogerclarke.com/EC/RAIC.html
Considerable public disquiet exists about various forms and aspects of Artificial Intelligence (AI). Some concerns are well-grounded, some are less so, and some are ill-informed. The risk exists that public opposition could result not only in justified constraints on AI but also in kneejerk regulatory measures that create unnecessary obstacles to desirable forms and applications of AI. A suite of proposals published by highly diverse organisations was used as a basis for a consolidated set of 50 Principles for Responsible AI. These offer guidance for practitioners in the various phases of the AI supply chain.
Artificial Intelligence (AI) excites a lot of people. And hyperbole about AI upsets a lot of other people. Engineers and computer scientists are busy sifting through the chaff to find the wheat, seeking to apply good ideas in order to achieve benefits without doing disproportionate harm. The hype and the hubbub are unhelpful distractions.
Of the concerns felt by the public, expressed on their behalf by advocacy organisations, and amplified by the media, many are vague, and some are unfounded. In an analysis published in Clarke (2019a), I concluded that the core of the problem is that "AI gives rise to errors of inference, of decision and of action, which arise from the more or less independent operation of artefacts, for which no rational explanations are available, and which may be incapable of investigation, correction and reparation". That needs to be unpacked a little further, to identify the key factors.
The first issue is that artefacts (i.e. human-made objects) are increasingly sensor-computer-actuator bundles. They are becoming capable not only of computing and communicating, but also of taking autonomous action that directly affects the real world. Two further factors are that applications of AI make assumptions about the suitability of the inferencing process and about the data on which it depends - and those assumptions may not be adequately tested and may be faulty, perhaps generally, or perhaps only in specific contexts of use.
The fourth issue is that inferencing processes used in AI are typically opaque - even to the technology's originators and the designers of AI-based artefacts. This means that systems behave a-rationally, in the sense that there is no human-understandable explanation of how inferences were drawn. In such circumstances, processes cannot satisfy the requirements of replicability, auditability, and correctability. Combined with the inadequacies of product liability law, this means that organisations can escape accountability, i.e. the buck stops in the middle of nowhere. Given that the world appears to be going through a power-shift towards artefacts, it's only natural that there is push-back from the public.
Some of these problems are outside the scope of engineers and computer scientists; but others fall squarely within their responsibility. What guidance is available to inventors, designers and implementors about the negative impacts that need to be prevented or mitigated, and the risks that need to be managed?
There's a very healthy marketplace for AI principles. Large IT companies and industry associations have felt the pressure of public concern about AI, and issued statements about how much care they're taking. Advocacy organisations have poked holes in those statements, and responded with much more substantial lists of what they think is needed. Recently, government agencies and regulators have clambered into the ring, most of them motivated by concern that innovation will be undermined if public acceptance of new technologies isn't assured, but some adopting a consumer rights perspective. Professional associations have pondered deeply (see, in particular, IEEE 2017), but continue to be very slow to deliver concise, actionable guidance for their members.
The purpose of this article is to present a set of 'Principles for Responsible AI' that reflects the last few years' accumulation of propositions. In order to assemble the list, I collected a suite of 30 such documents, from across a spectrum of organisations and countries. I then extracted the principles they put forward, identified commonalities, and built a super-set that endeavours to encompass them all. Supporting Documentation is provided, which identifies and provides access to the sources. This enables audits to be undertaken of the extraction process and the expressions chosen, and alternative interpretations to be made and alternative expressions formulated.
The resulting set of Principles might have been redundant if any of the 30 sources already covered the field satisfactorily. The coverage proved to be generally very thin, however, with almost all of the sources containing less than half of the complete set, and many of them being very limited in scope. The only authoritative document that was found to have substantial coverage was that of the European Commission (EC 2019), and even that only scores 74%. That document may, however, be more directly relevant than the 50 Principles presented here, at least to professionals whose focus is on goods and services specifically for EU countries, or for export to EU countries.
Reading through the set of 50 Principles at the end of this article provides a basis for professionals to contemplate the impacts that their work may have. If a copy is pinned up on the wall, the Principles can be discussed within teams, and referred to when differences of opinion arise. This section suggests a few further aspects whereby they can be of use to engineers and computer scientists.
Interpretation of principles is a task that distinguishes the role of the professional from that of the technician. In this case, AI technologies, the artefacts that AI is built into, the systems that incorporate the artefacts, and the uses that the systems are put to, exhibit a great deal of diversity. So no general recipe is possible.
A general Principle such as "ensure human control over autonomous behaviour of AI-based technology, artefacts and systems" (3.2) can't be expressed in a way that applies equally well in every situation, and certainly isn't suitable for imposition as a law, or even an industry Standard. Similarly, "Implement safeguards to avoid, prevent and mitigate negative impacts and implications" (4.4) has to be operationalised in each particular context, and designs will exhibit as much variety as the technologies, the artefacts and systems, and the circumstances in which they're applied.
There are many conflicts between these people-protective Principles and the interests of other stakeholders. The organisations that sponsor the development of AI technology or AI-based artefacts or systems can usually protect their own interests, but many other organisations may be involved, and there are impacts on the economy, society as a whole, and the environment.
Conflicts even arise within the 50 Principles. For example, it can be difficult to both"Ensure ... performance of intended functions" (7.1) and "Ensure people's physical health and safety ('nonmaleficence')" (4.1). To invoke a classic in the field, in the very first of Asimov's short stories in his long series on the Laws of Robotics, 'Robbie the robot' caused (minor) harm to the robot's owner, because she was knocked breathless by the act of getting her out of the path of a tractor.
The resolution of conflicts involves value judgements. Some may fall within the designer's sphere of responsibility; but the big calls depend on executive judgement and consultation with stakeholders and regulators.
The Principles need to be interpreted differently in different parts of the supply chain or industry network. A simple model of industry structure distinguishes research, invention, innovation, dissemination and application phases, with researchers producing AI technology, inventors using the ideas to produce AI-based artefacts, developers incorporating the artefacts within AI-based systems, purveyors of various kinds establishing an installed base of such systems, and user organisations applying them, resulting in impacts and implications.
The responsibilities of professionals playing roles higher up the supply-chain differ from the responsibilities of designers close to the applications end of the business. For example, "deliver transparency and auditability" (7.) calls for purveyors and users to ensure understandability by affected stakeholders of inferences, decisions and actions arising from AI-based systems. Higher up the chain, on the other hand, that Principle requires that inventors and innovators design transparency into AI technology, and ensure that developers and users of AI-based artefacts and systems can readily understand the nature of the underlying technology, and can thereby devise ways to fulfil their own transparency and auditability obligations.
A further challenge arises from the clumping together of rather different things under the banner of 'AI'. Many of the Principles are of greater significance in relation to some forms of AI than others; and many would therefore be more directly useful to professionals if they were re-phrased, re-framed or customised to particular forms of AI.
Robotics is migrating well beyond factory floors and automated warehouses, including to aircraft and watercraft. The large majority of the Principles are directly applicable to designers of autonomous vehicles and their support systems, both in controlled environments such as mines, quarries and dedicated bus routes, and where the vehicle is on the open road or in public airspace. On the other hand, the Principle "Design as an aid, for augmentation, collaboration and inter-operability" (2.1) is much less applicable to real-time control over a vehicle's attitude, position and course than it is to higher-order functions such as mission control and anticipation of changes in weather conditions.
A second area usefully regarded as being within the AI field is cyborgisation, the process of enhancing individual humans by technological means, resulting in hybrids of a human and one or more artefacts. Many forms of cyborgisation fall outside the field of AI, such as spectacles, implanted lenses, stents, inert hip-replacements and SCUBA gear. Some enhancements qualify, however, by combining sensors and actuators with computational intelligence. Prominent examples include heart pacemakers (since 1958), cochlear implants (since the 1960s, and commercially since 1978), and some replacement legs for above-knee amputees, in that the artificial knee contains software to sustain balance within the joint. In this context, "Respect each person's autonomy, freedom of choice and right to self-determination" (3.4) is writ particularly large, whereas "Ensure human review of inferences and decisions prior to action being taken" (3.5) requires careful interpretation, in order to balance the needs of urgent 'system real-time' actions against those of individual self-determination. Another Principle requiring care in its application is "Avoid services being conditional on the acceptance of AI-based artefacts and systems" (3.7), which may be in direct conflict with the express desire of an applicant for, say, a heart-pacemaker.
It might reasonably be expected that leading exemplars of well-articulated guidance relevant to cyborgisation would be provided by the medical implants field. On the other hand, studies conducted by investigative journalists and published in reputable media outlets have thrown considerable doubt on the effectiveness of these processes in protecting patients' interests and assuring quality and safety.
Two other forms of AI to which the 50 Principles are applicable are the relatively established area of rule-based expert systems, and the still-immature field variously referred to as AI/ML or neural networking. Unlike predecessor approaches such as algorithmic / procedural programming, the rule-based expert systems field embodies no conception of either a problem or a solution. A rule-base merely describes a problem-domain. Techniques in the machine-learning field differ even more from earlier approaches, in that they do not necessarily begin with active and careful modelling of a real-world problem-solution, problem or even problem-domain. Rather than comprising a set of entities and relationships that mirrors the key elements and processes of a real-world system, a neural network model may be simply a list of input variables and a list of output variables (and, in the case of 'deep' networks, intermediary variables). The weightings assigned to each connection reflect the the particular learning algorithm that was applied, and the characteristics of the training-set that was fed into it. The unpredictable implications of those decisions combine with questions about the semantics, accuracy and compatibility of the data to give rise to uncertainty about the technique's degree of affinity with the real world to which it is applied.
When these forms of AI are in play, a critical issue is how compliance can be achieved with the Principle "Ensure that people ... have access to humanly-understandable explanations of how [inferences, decisions and actions] came about" (6.3). Further challenges arise from the need to "Ensure that inferences are not drawn from data using invalid or unvalidated techniques" (7.6), "Test result validity ..." (7.7), and "Impose controls in order to ensure that the safeguards are in place and effective" (7.8). It is also unclear how neural networking techniques can enable performance of "audits of the justification, the proportionality, the transparency ..." (8.3). Given that these are all supported features of systems that AI/ML is seeking to supplant, it's problematical, not to mention morally dubious, for designers to argue that these Principles aren't applicable, merely because they're difficult to comply with.
This brief review has of course merely scratched the surface of the 50 Principles and their application to AI. It has, however, brought to light both the need for them to be not blindly but intelligently applied to the particular technology and in the particular context; and it has illustrated how public distrust of some AI technologies and AI-based artefacts and systems is grounded in reality rather than being imaginary.
A set of principles is no more than a checklist. A further key element of the framework needed for assuring responsible AI is a process within which the Principles can be used to guide design. The Principles imply this by saying "conduct impact assessment ..." (1.4). Impact assessment is an approach that originated in environmental contexts. It adopts perspectives different from those of the organisation that drives the initiative. It is accordingly commonly conducted by other organisations, outside the boundaries of the driving organisation.
Within the driving organisation, the well-established tools of risk assessment and risk management might fill this role. However, their conventional forms are inadequate for the purpose. This is because they adopt the perspective only of the organisation that is sponsoring the activity. The process includes the identification of stakeholders, but their interests are reflected only to the extent that harm to them may result in material harm to the sponsoring organisation.
Responsible application of AI is only possible if stakeholder analysis is undertaken in order not only to identify the categories of entities that are or may be affected by the particular project, but also to gain insight into those entities' needs and interests. Note too that the notion of a 'stakeholder' goes beyond the users of and participants in the system. People may be affected by a system, and therefore have a stake in it, even if they are what are sometimes called 'usees', outside the system. This arises, for example, in the operation of credit-reporting agencies, tenancy and insurance claim data-pools, and criminal intelligence systems. Users' dependants, their local communities, population segments and whole economic regions can be usees as well. In the context of autonomous vehicles, stakeholders include passengers, occupants of other vehicles, pedestrians, vehicle-owners, transport-service-providers, individuals in occupations whose existence is threatened by the new approach (e.g. taxi-, courier- and truck-drivers), their dependants, their employers, and their unions.
Risk assessment processes need to be conducted from the perspectives of each stakeholder group, to complement that undertaken from the organisation's perspective. The various, and inevitably in part conflicting, information needs to be then integrated, in order to deliver a multi-stakeholder risk management plan (Clarke 2019b).
The 50 Principles that this article presents were derived by consolidating those proposed by a diverse assortment of organisations. The set therefore has at least some superficial validity as a proxy for what 'the public as a whole' thinks AI needs to look like. The brief analysis possible within this short article has shown ways in which the Principles can be applied, and can be fitted within a risk assessment process. It has also surfaced a few of the challenges that AI researchers, inventors, developers, purveyors and users must address in order to satisfy the needs of stakeholders, and overcome the concerns of the general public, and of legislators, regulators, financiers and insurers.
Once technologies and process standards have matured, the 50 Principles for Responsible AI will be redundant, replaced by more specific formulations for each category of what is currently termed 'AI' technology. However, given that the first 60 years of the AI field have seen so little progress in establishing such guidance, the set presented here may be 'as good as it gets' for some time to come.
Also available is a 2-page PDF version, suitable for printing
AI offers prospects of considerable benefits and disbenefits. All entities involved in creating and applying AI have obligations to assess its short-term impacts and longer-term implications, to demonstrate the achievability of the postulated benefits, to be proactive in relation to disbenefits, and to involve stakeholders in the process.
1.1 Conceive and design only after ensuring adequate understanding of purposes and contexts
1.2 Justify objectives
1.3 Demonstrate the achievability of postulated benefits
1.4 Conduct impact assessment, including risk assessment from all stakeholders' perspectives
1.5 Publish sufficient information to stakeholders to enable them to conduct their own assessments
1.6 Conduct consultation with stakeholders and enable their participation in design
1.7 Reflect stakeholders' justified concerns in the design
1.8 Justify negative impacts on individuals ('proportionality')
1.9 Consider alternative, less harmful ways of achieving the same objectives
Considerable public disquiet exists in relation to the replacement of human decision-making by inhumane decision-making by AI-based artefacts and systems, and displacement of human workers by AI-based artefacts and systems.
2.1 Design as an aid, for augmentation, collaboration and inter-operability
2.2 Avoid design for replacement of people by independent artefacts or systems, except in circumstances in which those artefacts or systems are demonstrably more capable than people, and even then ensuring that the result is complementary to human capabilities
Considerable public disquiet exists in relation to the prospect of humans being subject to obscure AI-based processes, and ceding power to AI-based artefacts and systems.
3.1 Ensure human control over AI-based technology, artefacts and systems
3.2 In particular, ensure human control over autonomous behaviour of AI-based technology, artefacts and systems
3.3 Respect people's expectations in relation to personal data protections, including their awareness of data-usage, their consent, data minimisation, public visibility and design consultation and participation, and the relationship between data-usage and the data's original purpose
3.4 Respect each person's autonomy, freedom of choice and right to self-determination
3.5 Ensure human review of inferences and decisions prior to action being taken
3.6 Avoid deception of humans
3.7 Avoid services being conditional on the acceptance of AI-based artefacts and systems
All entities involved in creating and applying AI have obligations to provide safeguards for all human stakeholders, whether as users of AI-based artefacts and systems, or as usees affected by them, and to contribute to human stakeholders' wellbeing.
4.1 Ensure people's physical health and safety ('nonmaleficence')
4.2 Ensure people's psychological safety, by avoiding negative effects on their mental health, emotional state, inclusion in society, worth, and standing in comparison with other people
4.3 Contribute to people's wellbeing ('beneficence')
4.4 Implement safeguards to avoid, prevent and mitigate negative impacts and implications
4.5 Avoid violation of trust
4.6 Avoid the manipulation of vulnerable people, e.g. by taking advantage of individuals' tendencies to addictions such as gambling, and to letting pleasure overrule rationality
All entities involved in creating and applying AI have obligations to avoid, prevent and mitigate negative impacts on individuals, and to promote the interests of individuals.
5.1 Be just / fair / impartial, treat individuals equally, and avoid unfair discrimination and bias, not only where they are illegal, but also where they are materially inconsistent with public expectations
5.2 Ensure compliance with human rights laws
5.3 Avoid restrictions on, and promote, people's freedom of movement
5.4 Avoid interference with, and promote privacy, family, home or reputation
5.5 Avoid interference with, and promote, the rights of freedom of information, opinion and expression, of freedom of assembly, of freedom of association, of freedom to participate in public affairs, and of freedom to access public services
5.6 Where interference with human values or human rights is outweighed by other factors, ensure that the interference is no greater than is justified ('harm minimisation')
All entities have obligations in relation to due process and procedural fairness. These obligations can only be fulfilled if all entities involved in creating and applying AI ensure that humanly-understandable explanations are available to the people affected by AI-based inferences, decisions and actions.
6.1 Ensure that the fact that a process is AI-based is transparent to all stakeholders
6.2 Ensure that data provenance, and the means whereby inferences are drawn from it, decisions are made, and actions are taken, are logged and can be reconstructed
6.3 Ensure that people are aware of inferences, decisions and actions that affect them, and have access to humanly-understandable explanations of how they came about
All entities involved in creating and applying AI have obligations in relation to the quality of business processes, products and outcomes.
7.1 Ensure effective, efficient and adaptive performance of intended functions
7.2 Ensure data quality and data relevance
7.3 Justify the use of data, commensurate with each data-item's sensitivity
7.4 Ensure security safeguards against inappropriate data access, modification and deletion, commensurate with its sensitivity
7.5 Deal fairly with people ('faithfulness', 'fidelity')
7.6 Ensure that inferences are not drawn from data using invalid or unvalidated techniques
7.7 Test result validity, and address the problems that are detected
7.8 Impose controls in order to ensure that the safeguards are in place and effective
7.9 Conduct audits of safeguards and controls
All entities involved in creating and applying AI have obligations to ensure resistance to malfunctions (robustness) and recoverability when malfunctions occur (resilience), commensurate with the significance of the benefits, the data's sensitivity, and the potential for harm.
8.1 Deliver and sustain appropriate security safeguards against the risk of compromise of intended functions arising from both passive threats and active attacks, commensurate with the significance of the benefits and the potential to cause harm
8.2 Deliver and sustain appropriate security safeguards against the risk of inappropriate data access, modification and deletion, arising from both passive threats and active attacks, commensurate with the data's sensitivity
8.3 Conduct audits of the justification, the proportionality, the transparency, and the harm avoidance, prevention and mitigation measures and controls
8.4 Ensure resilience, in the sense of prompt and effective recovery from incidents
All entities involved in creating and applying AI have obligations in relation to due process and procedural fairness. The obligations include the entity ensuring that it is discoverable, and addressing problems as they arise.
9.1 Ensure that the responsible entity is apparent or can be readily discovered by any party
9.2 Ensure that effective remedies exist, in the form of complaints processes, appeals processes, and redress where harmful errors have occurred
Each entity's obligations in relation to due process and procedural fairness include the implementation of systematic problem-handling processes, and respect for and compliance with external problem-handling processes.
10.1 Ensure that complaints, appeals and redress processes operate effectively
10.2 Comply with external complaints, appeals and redress processes and outcomes, including, in particular, provision of timely, accurate and complete information relevant to cases
The two articles by the author provide access to c.70 references.
Clarke R. (2019a) 'Why the World Wants Controls over Artificial Intelligence' Computer Law & Security Review 35, 4 (2019) 423-433, PrePrint at http://www.rogerclarke.com/EC/AII.html
Clarke R. (2019b) 'Principles and Business Processes for Responsible AI' Computer Law & Security Review 35, 4 (2019) 410-422, PrePrint at http://www.rogerclarke.com/EC/AIP.html
EC (2019) 'Ethics Guidelines for Trustworthy AI' High-Level Expert Group on Artificial Intelligence, European Commission, April 2019, at https://ec.europa.eu/digital-single-market/en/news/ethics-guidelines-trustworthy-ai
IEEE (2017) 'Ethically Aligned Design: The IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems', Version 2. IEEE, December 2017. at http://standards.ieee.org/develop/indconn/ec/autonomous_systems.html
The first two documents identify, extract and cite the 30 source documents, and the third relates the 30 source documents to the consolidated set of 50 Principles:
Ethical Analysis and Information Technology (8 Source Documents)
Principles for AI: A SourceBook' (22 Source Documents)
The 50 Principles Cross-Referenced to the Source-Documents
Roger Clarke is Principal of Xamax Consultancy Pty Ltd, Canberra. He is also a Visiting Professor in Cyberspace Law & Policy at the University of N.S.W., and a Visiting Professor in the Research School of Computer Science at the Australian National University.
The content and infrastructure for these community service pages are provided by Roger Clarke through his consultancy company, Xamax.
From the site's beginnings in August 1994 until February 2009, the infrastructure was provided by the Australian National University. During that time, the site accumulated close to 30 million hits. It passed 60 million in early 2019.
Sponsored by the Gallery, Bunhybee Grasslands, the extended Clarke Family, Knights of the Spatchcock and their drummer
Xamax Consultancy Pty Ltd
ACN: 002 360 456
78 Sidaway St, Chapman ACT 2611 AUSTRALIA
Tel: +61 2 6288 6916
Created: 1 August 2019 - Last Amended: 3 August 2019 by Roger Clarke - Site Last Verified: 15 February 2009
This document is at www.rogerclarke.com/EC/RAIC.html