Roger Clarke's Web-Site
© Xamax Consultancy Pty Ltd, 1995-2024
Infrastructure
& Privacy
Matilda
Roger Clarke's Web-Site© Xamax Consultancy Pty Ltd, 1995-2024 |
|
|||||
| HOME | eBusiness |
Information Infrastructure |
Dataveillance & Privacy |
Identity Matters | Other Topics | |
| What's New |
Waltzing Matilda | Advanced Site-Search | ||||
Early-Version Working Paper of 24 November 2024
© Xamax Consultancy Pty Ltd, 2024
Available under an AEShareNet 
licence or a Creative
Commons 
licence.
This document is at http://rogerclarke.com/EC/RGAI.html
Generative Artificial Intelligence (GenAI) burst into public consciousness in 2022, and has sustained the aura of excitement for some time afterwards. Many expressions of concern have been voiced, but they have been swamped by the waves of enthusiastic promotion, initiated by providers and their investors, and willingly repeated by content intermediaries. The present work's purpose is to catalogue the characteristics of GenAI that have negative impacts and implications. That establishes a firm base for principles for its responsible application, and for a proposal for a more substantial reconception of Gen AI.
After a long gestation period, widespread availability of a new form of Artificial Intelligence (AI) became publicly available during 2022-23. The names applied to it vary, with Generative AI and GenAI commonly used, but also GAI (which risks confusion with a longstanding and quite different concept). The names of the most widely used exemplars of the two components (GPT and ChatGPT), are also used by many people in a generic sense. GenAI artefacts, viewed as a black box, receive textual input from a person, in the form of a question or request, and provide a synthetically-generated textual response to it.
The responses from ChatGPT evidence apparent authenticity, most commonly in the form of smoothly constructed sentences of moderate complexity, in communications styles requested or implied by the requestor. From the time of ChatGPT's release, the quality of its expression startled people. This encouraged users to impute considerable authority and even veracity to the content. The believability of its output in response to simple requests resulted in many users suspending disbelief, and assuming reliability in responses to further simple questions and to less trivial requests as well.
A wide variety of potential benefits of the technology have been asserted by its proponents. This has excited 'goldrush', 'bandwagon' behaviours. ChatGPT and some other GenAI artefacts have been experimented with and very rapidly adopted by many organisations and individuals and for a wide range of tasks. Their use is accompanied by a substantial lack of scepticism, and the use of their outputs commonly lacks quality assurance. The work reported here is motivated by the need to identify and address the risks arising from uncritical adoption of GenAI.
Concern has been expressed by a variety of people about the direct impacts of specific uses, and the indirect implications of various categories of application. Issues include bias, discrimination, disinformation and privacy (Baldassarre et al. 2023). A key factor that has attracted limited attention to date is the undermining of established mechanisms for righting wrongs and assigning legal and financial responsibility for harm caused. Many organisations and individuals are working on ways in which societies and economies can deal with the GenAI phenomenon. This paper is a contribution to that process. It draws on prior work in related areas, with the intention of crafting a set of principles for responsible application of Generative AI.
The next section presents a sufficiently rich model of the category of artefacts, based on descriptions of the technologies and resources underlying them, and descriptions of the contexts of use. This enables the identification of characteristics of GenAI relevant to their impacts and implications. Sources on responsible application of AI, 'big data' analytics, and AI/ML are reviewed, and their applicability to GenAI considered. This lays the foundations for the formulation of principles which, if respected, or codified and enforced, would address the negative impacts while enabling achievable benefits of the technology to be delivered. The final section proposes a more substantial reconception of GenAI, in order to achieve benefits with far less negative impact and with risk management integrated into the application process.
This section draws on academic, technical and populist sources in order to present accessible explanations of what GenAI does, and how it does it. The first sub-section provides an intentionally simplistic overview of the whole. The second sub-section then delves into somewhat greater depth in order to establish a platform for the subsequent analysis. The technologies underlying GenAI are being applied to data in a wide variety of forms, including text, code, audio/sound including the human voice, image, moving image (video), generated graphics and animations. The focus of this article, however, is on data whose source materials are text, and whose responses are also in textual form.
Many authors have endeavoured to explain GenAI in a manner that will convey adequately to whatever audience they have chosen to target. Many of these deliver highly impressionistic senses of the artefacts and what they do. Some are so heavily imbued with marketing-speak that such value as they have is masked. Here are three definitions that achieve a degree of understandability by a layperson while remaining reasonably consistent with the underlying technology:
"generative models encode a simplified representation of their training data and draw from it to create a new work that's similar, but not identical, to the original data" (IBM 2023)
"Generative AI models use neural networks to identify the patterns and structures within existing data to generate new and original content" (Nvidia 2024)
"Generative AI uses a computing process known as deep learning to analyze patterns in large sets of data and then replicates this to create new data that appears human-generated" (Crouse 2024)
A GenAI artefact can be regarded as a combination of two inter-related components, as depicted in Figure 1:
In Figure 1, the boundaries of the two components are depicted with curved rather than straight-line forms, because the internal architectures of existing GenAI artefacts vary, and continue to mature and change. For example, a response elicited from ChatGPT in October 2024 evidenced overlap between the functions of "generating coherent text" and "generating responses":
[ ChatGPT handles ] the conversation with you, interpreting your questions and generating responses. The underlying LLM does the heavy lifting of understanding language patterns, context, and generating coherent text based on the input it receives. [ ChatGPT's ] role is to facilitate interaction and provide responses, while the LLM provides the language processing capabilities that make those responses meaningful and relevant.
To generate the resource, the LLM processes considerable, and even vast, quantities of source-texts to produce a pre-stored, structured and highly-compressed representation of each of them. This requires a great deal of storage, a great deal of computing power, and hence a great deal of energy, plus cooling technology and materials for the heavily-worked and very dense silicon componentry. As a result, this function is currently performed in very large and specialised installations. With possible exceptions for highly specialised and small data collections, the background segment of LLMs appears likely to continue to operate remotely from users. At present, the foreground function generally also runs remotely from the user; but it requires less storage and is less computationally intensive, so it is feasible for it, or parts of it, to run locally to the user.
The previous sub-section provides sufficient information to enable the reader to follow the analysis and argument presented later. The present sub-section presents an explanation at a level a little deeper than that above, seeking to provide a bridge between superficial and technical descriptions. This is used in the following section to identify characteristics of GenAI that have significance for the analysis of the technology's impacts and implications. The functions of a GenAI artefact are outlined under six sub-headings, starting with the two functions of an LLM that establish the resource needed by the Chatbot, and then describing the sequence of functions performed when a user submits a request.
Language modelling is the underlying means whereby GenAI delivers responses, and effective modelling depends on the acquisition of source-texts. A model may be devised to address a particular problem-domain, such as corporate financial information (as the Bloomberg GenAI does). A specialist GenAI in, say, the diagnosis and treatment of melanomas, would of course need to be built from scientific and clinical information in that field; and one on environmental regulation would depend on access to statutory and case law, codes, and commentaries in that field. The emphasis to date, however, has been on general-purpose GenAI artefacts, which draw on a very wide catchment of documents.
The nature of the existing data collections on which GenAI depends is a key factor underpinning the quality or effectiveness of the result. The approach that is adopted with predecessor (AI/ML) techniques has commonly been to ingest and pre-process a more or less curated 'training-set'. This comprises a sufficiently large volume of data representing instances of a general class of a thing. The approach adopted with general-purpose GenAI artefacts, on the other hand, is to draw on the vast pools of data that have become available during the 'big data' era of the last few decades. This includes such collections as web-pages, research publications, media collections and social media (Kaubre 2024, Turing 2024). Responses may be fairly similar to one or some existing sources in the data collections, with differences arising variously from the use of multiple sources, the expression of the request, output-styling, and arbitrary changes made to avoid repetition in responses.
One important outcome of the indiscriminate rather than curated approach to acquiring source-texts is that no quality safeguards exist other than any relevant filters that may have been applied to each collection that is being appropriated. For example, an open-source refereed journal publishes only those articles submitted to it after they have been passed by one or more layers of reviewers and editors, and withdraws or qualifies any articles whose appropriateness is subsequently brought into material question. Publicly-accessible news outlets exercise varying degrees of self-control, from 'quality broadsheets' to 'tabloid' / 'yellow press' outlets. The content of social media on 'tech platforms' such as those operated by Meta, on the other hand, has been largely devoid of any meaningful quality control.
The acquisition process has inherited a feature from the data mining / data analytics field, which is most usefully described as 'data scrubbing'. The intention of this step is to identify and correct errors in a cost-effective manner, entirely or at least substantially by automated means. The point of reference of data scrubbing, rather than being a lack of correspondence with reality, is the existence of anomalies within the data-set. For example, Chu & Ilyas (2016) are concerned only with duplicate records, and with integrity constraints, or data quality rules that the database should conform to (such as missing values in mandatory data-items, and values in data-items that are not in the set of permitted values). Some methods discard or adjust outliers - which, in many contexts, deprives the collection of some of its most valuable content. Despite the technique's limited scope, and the substantial technical challenges involved, the circumspect term 'scrubbing' has been largely replaced by the terms 'data cleaning' and 'data cleansing', which mislead users into an expectation that the desired outcome of the 'scrubbing' has been achieved.
Moreover, the large majority of guidance in relation to data scrubbing relates to structured data. To the extent that text is embedded in row-and-column data, it is limited to constrained vocabularies, and the tests applied to it are entirely syntactic. Further, some of the 'cleansing' actions taken in relation to text-sources remove or substitute content on the basis of words or expressions judged on some grounds to be objectionable (OpenAI 2024). Such 'moderation' processes may (but may not) achieve the intentions of the designer, they deny access to information or warp the intention of content-utterer, and they may (but may not) be consistent with user needs.
A further consideration is the opaqueness to users of GenAI artefacts as to the text-sources that are in the collection as a whole, and the particular text-sources that have been drawn on in preparing any particular response. Particularly since GenAI became publicly available in 2022, an additional factor has arisen. Text-sources are commonly loaded into LLMs in an indiscriminate manner. As a result, text synthesised by GenAI artefacts is played back into the corpus, as source-material for the generation of future responses [ FIND DECENT REFS ]. This creates an echo-chamber, entrenching errors and bias inherent in the original collection, and creating an opportunity for devotees of causes to shift future responses in a particular direction, by 'gaming the system' through the submission of large numbers of customised requests. For example, an experimental study by Sharma et al. (2024) "found that participants engaged in more biased information querying with LLM-powered conversational search, and an opinionated LLM reinforcing their views exacerbated this bias".
Given these contextual factors, it would be unwise to assume that the data collection on which a GenAI artefact produces responses is fit for the user's particular purpose when they make a request or ask a question.
The term 'encoding' is used to refer to the production of structured representations of source-texts. An understanding of key aspects of the underlying LLM is essential, as a basis for appreciating some attributes of chatbot responses that are variously helpful to and threatening to the interests of stakeholders in the process.
There are many forms of Artificial Intelligence (AI), including some that have graduated from the AI pool and are now recognised in their own right. Examples of these include rule-based expert systems and pattern recognition particularly for audio, image and video. A form that has attracted a great deal of attention over the last decade is Machine Learning (ML or AI/ML). A widely-applied form of AI/ML utilises 'artificial neural networks' Hardesty 2017). It involves two steps:
In a business context, the instances might comprise data relating to a loan application, and the resulting classification might be a Yes/No decision about whether to lend or not, or a risk-rating applied to a Yes decision. A (somewhat sceptical) expression of the nature of the process is that AI/ML based on neural-networks involves feeding in lots of pictures of cats, to produce a new artefact that (in some sense) learns to distinguish cats from non-cats (Clark 2012).
Whereas AI/ML generates inferences about whether a particular phenomenon, as represented by data, belongs to some category of phenomena, GenAI applies very similar underlying techniques to a different purpose. Its aim is to create new data that has particular characteristics. The new data may be text that purports to answer a question, programming code that purports to perform a function, a static or animated diagram, or sound, an image or a series of images intended for rapid display (video).
A GenAI artefact intended to synthesise text also draws on another line of technical development called a language model (LM). An LM is a system "trained on string prediction tasks: that is, predicting the likelihood of a token (character, word or string) given either its preceding context or (in bidirectional and masked LMs) its surrounding context" (Bender et al. 2021, p.611). The capacity of contemporary computational and storage devices has enabled the firehose of 'big data' (Clarke 2016). That term refers to data, in the present context textual data, that has been appropriated, in some cases with dubious legality, from a wide array of sources, often with little consideration given to incompatibilities among the sources. The combination of technological capacity with vast quantities of raw data-resources has encouraged the recognition of patterns that enable the creation of fast ways of choosing an appropriate next word.
Based on such limited lingual archaeology as has been published (e.g. Foote 2023) and scans of Google Scholar, the notion Large Language Model (LLM) appears to have emerged during the early-mid-2010s. It appears to have been a form of re-badging of a developing field, much like the re-birth of 'data mining' as 'data analytics', or the overriding of the original, client-oriented 'World Wide Web' with organisation-empowering 'Web 2.0' technologies. LLM does, however, have the advantage of being rather more descriptive than many other business-fashion terms. From mid-2022, the particular LLM that has dominated public perceptions has been GPT, available as a series of versions from a not-for-profit turned for-profit, OpenAI. Others with a significant public profile in 2024 include LLaMA (by Meta), Gemini (by Google), Grok (by xAI), Claude, Mistral and Ernie.
Various approaches have been taken to the reduction of text into a form that can be readily manipulated. An approach associated with the explosion of GenAI from 2022 onwards has been the 'encoding' of source-text into a dense representation, such that similarities are closely associated with one another. Use of an Encoder alone - of which Bidirectional Encoder Representations from Transformers (BERT) was an important early exemplar - may be of value for functions such as the classification of text-sources and sentiment analysis (Talaat 2023). To support GenAI, however, the need is for representations that will efficiently support identification of suitable needles in a very large haystack, and 'decoding' of the 'dense representation' into a form from which convincing-looking text can be generated.
One mainstream architecture for the encoding segment of GenAI is transformers, a technique that was introduced by a team of researchers at Google (Vaswani et al. 2017). "In natural language processing, a transformer encodes each word in a corpus of text as a token [ a numerical representation of a chunk of data ] and then generates an attention map, which captures each token's relationships with all other tokens. This attention map helps the transformer understand [ internal, lingual ] context when it generates new text" (Zewe 2023).
One explanation of how the process delivers outcomes is that "In this huge corpus of text, words and sentences appear in sequences with certain dependencies. This recurrence helps the model understand how to cut text into statistical chunks that have some predictability. It learns the patterns of these blocks of text and uses this knowledge to propose what might come next" (Zewe 2023).
Another interpretation of the encoding techniques used in GenAI is that it is appropriate to "Think of ChatGPT as a blurry JPEG of all the text on the Web. It retains much of the information on the Web, in the same way, that a JPEG retains much of the information of a higher-resolution image, but, if you're looking for an exact sequence of bits, you won't find it; all you will ever get is an approximation. But, because the approximation is presented in the form of grammatical text, which ChatGPT excels at creating, it's usually acceptable" (Chiang 2023).
In AI/ML, many approaches involve what is referred to as 'supervised learning', whereby the designer imposes a moderate degree of structure on the nodes and connections within the network. Other approaches adopt 'unsupervised learning', whereby the software infers or invents nodes, based on measures of similarity among the instances fed to it as training data. This requires large numbers of instances to be processed, and results in structures that may be unknown and largely meaningless even to the 'designers' of the product. According to its proponents, however, this approach "can find previously unknown patterns in data using unlabelled datasets" (Nvidia 2023). It appears that, in the 'encoding' phase, LLM developers at least initially adopt unsupervised, automated learning approaches to cope with the vast volumes of data. It also appears that new models perform poorly until they have been subjected to refinements using 'supervised' approaches. These are inevitably somewhat human-intensive, experimental and iterative, and liable to have unforeseen consequences.
One further aspect of LLMs is the re-purposing of vast general-purpose models for more specific purposes: "A foundation model is an AI neural network - trained on mountains of raw data, generally with unsupervised learning - that can be adapted to accomplish a broad range of tasks" (Merritt 2023). See also Jones (2023). Bommasani et al. (2023) noted "the rise of models ... trained on broad data (generally using self-supervision at scale) that can be adapted to a wide range of downstream tasks. We call these models foundation models to underscore their critically central yet incomplete character". One result of this technically-lauded approach is that GenAI artefacts that are applied to reasonably specific purposes are generating responses from a vast resource, a large proportion of which is not semantically relevant to the specific purpose, but which has relevance in a structural-linguistic sense. These origins may influence the expression and even the apparent semantic content of the response.
The LLM's encoding phase results in a resource that can be accessed by a Chatbot in order to generate responses to users. The term 'chatbot' is widely used for any software designed to interact with humans in a moderately flexible, authentic and/or useful manner. Other terms in use include virtual assistant, conversational AI, and (more restrictively) web-interface. The core processes within early chatbots were developed using procedural (i.e. genuinely algorithmic) languages. Subsequently, rule-based approaches were adopted, and most recently LLM-based GenAI (Adamopoulou & Moussiades 2020).
The notion (although not yet the term) originated with the Turing Test about 1950. A gentle attempt to demonstrate the challenges and problems with the chatbot notion, Eliza (Weizenbaum 1966), instead demonstrated human users' inherent gullibility and willingness to suspend disbelief (Dodgson 2023). The origins of the term 'chatbot' are conventionally traced to the use of 'chatterbot' in a paper title by Michael Mauldin in 1994. The first mention of 'chatbot' located in the Google Scholar archive is as the name of a particular 'chatterbot' mentioned in de Angeli et al. (2001), which was built using the same technology as Wallace's Alice (Artificial Linguistic Internet Computer Entity) of 1995.
The primary chatbot that is specifically designed as a component of a GenAI artefact is OpenAI's ChatGPT, accessible at https://chatgpt.com/. This accepts and interprets natural-language text, and in more languages than just English. Microsoft offers Copilot as an alternative chatbot to ChatGPT, invoking the GPT-4 LLM. The service-names Claude, Ernie and Gemini each apply to the amalgam of the relevant provider's LLM and chatbot. Playing catch-up even moreso than other major tech platforms, Apple announced in mid-2024 its intention to integrate OpenAI's ChatGPT into iOS and macOS.
[ NEEDS MORE LIT-SEARCHING AND DETAIL ]
The process whereby Generative AI creates new data commences with interpretation of the task to be performed. The specialised chatbots used to enable users to interact with a GenAI artefact commonly use established techniques of natural language processing (NLP) to analyse the words and phrases in the request, possibly using inbuilt synonym tables. This generally involves identifying verbs and verb-forms that are related to the requestor's intent, for example distinguishing a question (seeking information) from a request to conduct a transaction; and recognising the nouns that define the subject-matter. The end-result is a set of tokens, compatible with those used within the LLM, which the chatbot passes to the relevant component of the LLM. In Figure 1, the encoder (discussed in sub-section (2) above) is depicted as part of the LLM only, whereas the decoder is depicted as the engine running at the intersection of the LLM and the Chatbot, at the heart of the extraction and decoding function.
The 'decoding' process involves decompression of the tightly structured data in the model. Because of the high compression-factor during encoding, the stored data is a highly 'lossy' interpretation of the source. The decompression process consequently delivers decoded data that is unlikely to result in text closely similar to any of the text that was encoded. However, LLM designers seek to compensate for this by applying supervised learning and reinforcement learning from human feedback (RLHF) in order to deliver a workable 'decoding' function.
[ NEEDS MORE LIT-SEARCHING AND DETAIL ]
The decoded new data is not in a form intelligible to users. The intent of the designer of a GenAI artefact is generally the production of synthetic text that is likely to appear authoritative, authentic and convincing to the person who asked the question. Prior work in the field of natural language generation (NLG) has resulted in tools for the production of understandable texts from non-lingual data. Integrating an LLM to an existing NLG tool may, however, involve conversion of the particular token scheme into an intermediate form.
The response that is generated for any particular request might exhibit similarities to one or more existing sources in the data collections, in terms of words and word-sequences. Given the scale of the source-texts represented in current LLMs, however, it appears more likely that the response will differ from the source-texts in at least style and potentially also content. The reasons for the differences include the approach adopted to filling in for the lossyness of the compression, filtering and merger of multiple sources, the expression of the request, and output-styling.
Inconsistencies in presentation and content among the sources may contribute to the generation of strangeness in some responses. An instance of this is, inconveniently, referred to as an 'artefact', not in the sense of a human-made object, but meaning a generated feature that arises from the handling of the data and that does not reflect an aspect of the real world. Examples of this that arise with image date include distortion, 'ghosting' and 'haloes'. GenAI artefacts may also be designed to generate arbitrary differences in responses, in order to establish and sustain an aura of intelligence and creativity and avoid giving the appearance of being a boring automaton.
[ NEEDS MORE LIT-SEARCHING AND DETAIL ]
After the response is provided, the ChatGPT page remains open for further interactions. It appears that further input will be assumed to be on the same topic as the previous input, at least if there is some commonality in the expressions used and/or only a relatively short time-lapse between one set of user input and the next. This retention of the thread-context is referred to as 'state-preservation', and enables some degree of cumulativeness of interactions, with the interactions intended to resemble a conversation between humans. To achieve a dialogue containing more than a few exchanges may require the user to perform the 'state management' task, for example by including a summary of the previous conversation with each follow-on communication (Bellow 2023).
Experiments with ChatGPT suggest that intended continuations may be misinterpreted. Some responses to follow-on input may appear bizarre, including apparently unreasoned or poorly-reasoned apologies, and new statements inconsistent with previous ones. This may reflect limits on the number of words, tokens or linguistic constructs the design can deal with in a single session. It is also possible that input that the user intends to commence a new request may be misinterpreted due to the previous interaction(s) being treated as part of the new context.
The descriptions in this section establish sufficient insight into GenAI technology to enable key characteristics to be identified that are relevant to its potential impacts and implications, and in particular those that may harm the interests of stakeholders.
AI has been described as brittle, opaque, greedy, shallow and tone-deaf, manipulative and hackable, biased, invasive and "faking it" (Willcocks et al. 2023). This section delves more deeply into GenAI, drawing on the descriptions of GenAI artefacts and their applications, to identify features that create likelihood of negative impacts and implications for stakeholders. In Table 1, 20 features are allocated into four groups as a reference-point for the subsequent analysis.
___________________
The source-texts that a GenAI artefact encodes into an LLM evidence issues that significantly affect the quality of responses and hence outcomes. The comments made here relate to current LLMs reflecting very-large-scale source-text collections. Some qualifications are likely to be appropriate where the designers intend a narrower scope of application than the universalist idea of 'a response to any question', and are selective in their incorporation of source-texts into the data collection and hence into the LLM.
Generally, there has been to date a strong leaning towards indiscriminate acquisition of source-texts, with little attention paid to the nature of the utterances. This reflects the bold assumptions of the 'big data' movement about 'more is better' and 'bad data gets overwhelmed by the maelstrom of good data'. In practice, the decoding that underlies response generation is blind to the question of data quality, and its selection criteria are such that bad data can have a disproportionate impact on the response and hence on the requestor's interpretation of the outcome. A further reason may be the difficulties involved in formulating operational criteria for the filtering of source-texts.
A related issue is inadequacies in the quality assurance of source-texts (Table 1 at 2.2). The IS field is well-acquainted with quality factors in the context of structured data. Appendix A provides an extract from Clarke (2018), which lists the key data quality factors (which are assessable both at the time of data-creation and subsequently) and information quality factors (which are assessable only at the time of use). Textual data presents different challenges, discussed in Clarke (2024). Appendix B reproduces a set of reliability criteria for textual data.
In practice, the text sources that are fed into LLMs vary enormously in the extent to which those quality factors are addressed. The terms 'disinformation' (for content that is misleading and is demonstrably intentionally so) and 'misinformation' (for other misleading content) have been prominent in discussions about both the source-texts in tech platforms supporting various forms of social media, and in the responses from GenAI artefacts. The analyses conducted by 'fact-checkers' suggest a high degree of both forms of malinformation exist in populist sources (Aimeur et al. 2023). Another issue, at 1.2 in Table 1, is that editorial standards are very low in social media, but have also plummeted within formal media as advertising revenues have been lost to tech platforms (ACCC 2023). In addition, low presentation quality gives rise to ambiguities, which are encoded into LLMs, and are then extracted and re-expressed in responses, resulting in misunderstandings of the intentions of the original source.
Source-texts bring with them intellectual baggage, or dominant perspectives, that reflect the period and the cultural context in which they were expressed, and the value-set of the utterer. Some of the more apparent areas in which preconceptions are inbuilt into a great deal of text are race / skin-colour / ethnicity (associated with colonialism), and gender (as re-evaluated in feminist literature). Table 1 at 1.4 notes that bias arising from embedded values comes in a great many other flavours, including age, class, caste, religion, wealth and educational attainment. In the words of Bender et al. (2021), "the tendency of training data ingested from the Internet to encode hegemonic worldviews [and] the tendency of LMs to amplify biases and other issues in the training data ..." (p.616) are key features of GenAI artefacts.
Some aspects of source-text quality have been addressed by GenAI developers. For example, some may perform de-duplication prior to encoding, and others say that they delete 'toxic' content, such as "removing documents which contain any word on the 'List of Dirty, Naughty, Obscene, or Otherwise Bad Words'" (Dodge et al. 2021, p.2). Two marketing terms arising from data mining and data analytics fields are used: 'data cleaning' and 'data cleansing'. The earlier, descriptive term 'data scrubbing' has the benefit of indicating that such processes are endeavours to, in some sense, clean or cleanse the source-text(s), but also that those endeavours may or may not achieve their purpose, and may or may not have unintended consequences for other quality attributes. The removal of source-texts represents historical revisionism (Table 1 at 1.5). It also constitutes yet another form of bias, and one that may be detrimental to the value that the action may be intended to support. For example, the removal of misogynistic, works from a corpus assists misogynists in arguing that claims about misogyny are exaggerated. [ NO REFS FOUND YET ]
A further serious issue has quickly arisen, labelled in Table 1 at 1.6 as 'Pollution by Synthetic Texts'. As responses have been produced by GenAI artefacts, this synthetic text has been fed back into collections of source-texts, whether directly by the LLM or Chatbot provider, or indirectly by requestors republishing it as though it were an original contribution. This creates the enormous risk of an echo-chamber in which variants of the same, partly information, partly misinformation and partly disinformation, accumulate and alter the balance in the collection. The result can be the creation an 'alternative, authoritative source of truth'.
A second cluster of issues at the heart of GenAI arises from the compression of source-texts into an 'encoded' form that reflects only the structural aspects of the original. The results of large numbers of encodings are then inter-related to form a cut-down, syntax-level only model of language. Table 1 at 2 reflects this by referring to limited understanding of language structure.
When compressed forms are 'decoded' as a basis for generating responses, they primarily reflect the structures of the selected sources, and only indirectly reflect their content. The process dissociates the text from the human context within which it originated: "Our human understanding of coherence derives from our ability to recognize interlocutors' beliefs ... and intentions ... within context ... That is, human language use takes place between individuals who share common ground and are mutually aware of that sharing (and its extent), who have communicative intents which they use language to convey, and who model each others' mental states as they communicate" (Bender et al. 2021, p.616). In contrast, "an LM is a system for haphazardly stitching together sequences of linguistic forms it has observed in its vast training data, according to probabilistic information about how they combine, but without any reference to meaning: a stochastic parrot" (p.617). Even a researcher strongly supportive of LLMs tenatively concludes that " ... BERT is not learning inference through semantic relations between premisses and conclusions. Instead it appears to be identifying certain lexical and structural patterns in the inference pairs" (Lappin 2024).
A cluster of the most serious concerns arise from the absence from the GenAI universe of any real-world referents for the 'tokens' that the process of encoding deals in (Table 1 at 3). The resulting language model, and the responses it generates, lack any connection with real-world phenomena. Human language emerged and evolved among individuals seeking to convey meaning to one another. Most of the primary concerns in the kinds of societies in which language originated were physical things, particularly safety, food and water, and actions to achieve, acquire or safeguard each of those things. As 'subjective' 'mind-stuff' became more prominent, referents still existed. Notions such as categories and counts represented abstractions beyond individual physical phenomena. Expressions about feelings, aesthetics and sentiments presume some commonality of experience between utterer and audience.
Semiotics is the study of the use of symbolic communication, including written and spoken languages. It encompasses three segments (Morris, 1938):
For example, among humans, the words 'rose', 'flower', 'smell', 'sweet' and 'bunch' have associations that are not merely grammatical, but also semantic (relating the word to real-world objects and contexts), and pragmatic (relating words to feelings). These elements of common contextual understanding are crucial to achieving understanding among humans. GenAI is limited to merely syntactical associations, dominated by sequence and proximity - which are low-order aspects of a language's grammar - without any semantic (text-to-thing) or pragmatic (text-to-person) aspect at all. There is therefore no sense in which either context is 'appreciated' or content is 'understood'. In Bender et al. (2021), concern is expressed about "the tendency of researchers and other people to mistake LM-driven performance gains for actual natural language understanding" (p.616).
Human language is generally used within some context, and frequently in a very rich context. Individuals interpret, or create, meaning in utterances, and the meanings may vary somewhat and even greatly between people; but the meanings are imputed within a context that is subject to at least some degree of sharing among those individuals. This enables many misunderstandings to be avoided, because of each individual's tendency to apply 'common sense' or 'normal understanding' (Weizenbaum 1967), involving great complexity and subtlety of contextual appreciation (Dreyfus 1992, pp.214-221). No such moderating influence exists within the GenAI realm.
A further shortfall in GenAI is that the response is generated without any appreciation of the nature of the requestor, or of any broader audience to which the response is intended to be communicated. A weak proxy may be used, for example, where the request includes an expression such as 'in the style of a tabloid newspaper / secondary school essay / scientific journal article'. Such 'styles' are, however, generated probabilistically, based on narrow, syntactical-level-only modelling of sources.
A human providing a textual response has the opportunity to also consider the nature and interests of the requestor, any intended audience, and third parties. A GenAI artefact has no such opportunity, and may produce text that is bland, or bias-reinforcing, or brutal, depending on the particular stakeholder's perspectives. It also has no capacity to consider the consequences that may flow from the response, whether for the requestor, any intended audience, or any particular category of third party. This recklessness may be transmitted further by the requestor's unthinking publication or application of the response.
Group 4 in Table 1 identifies potentially harmful attributes of the generative function itself. Some, and under some circumstances, a considerable amount of, the mis- and dis-information that is caught up in indiscriminately gathered, uncurated source-text collections finds its way into responses, and in many cases may affect inferences, decisions and actions, or be projected further afield. Because the components used to express responses are reasonably mature, and have been the subject of further investment by GenAI developers, responses have the appearance of authenticity, and hence are liable to lull requestors into a false sense of security.
A further feature of GenAI is its limited ability to express a rationale to go with what many requestors perceive as its pronouncements (4.2). This is a general problem with many currently-popular forms of machine learning. The search for 'explainable AI' (XAI) has delivered little progress, and 'explainability' is increasingly being abandoned for lesser notions of 'interpretability' (e.g. Linardatos et al. 2020). LLMs fail the criteria for transparency (Lipton 2018), and no means exists to generate 'post hoc' explanations of the operation of the 'black box' (Adamson 2022). Wigan et al. (2022) underline the significance of GenAI's inability to satisfy the explainability test.
The inadequacy is compounded by GenAI artefacts' general inability to provide quotations (4.3) - because it involves the abstraction of lingual structure and abandonment of the original text. The problem is further exacerbated in the all-too-common circumstances in which the artefact is in incapable of provision of citations of the source-texts on which it has drawn, or is even precluded by design from doing so, e.g. to avoid exposure of the fact that the provider has breached other parties' rights in relation to access to and/or use of the source-texts.
Another limiting factor is the set of criteria used in selecting modes of expression of the response. Rather than reflecting an understanding of the requestor's purpose and the nature of the intended audience (e.g. their educational background, interests and values), the primary driver is word-combination probabilities, and this may then be filtered through mechanisms whose purpose is to convey authenticity and avoid boring repetitions.
A final concern (Table 1 at 4.6) is the ease with which responses, particularly in the case of iterative questioning, result in inconsistencies, artefacts or hallucinations (Ji et al. 20123). These are referred to by ChatGPT's developer as "plausible-sounding but incorrect or nonsensical answers" (OpenAI 2022). Despite this, even responses of these kinds "are plausible enough that identifying them requires comparing them against the originals, which in this case means either the Web or our knowledge of the world ... [I]f a compression algorithm is designed to reconstruct text after ninety-nine percent of the original has been discarded, we should expect that significant portions of what it generates will be entirely fabricated" (Chiang 2023).
This section has outlined 20 aspects of GenAI that appear to be the underlying causes of difficulties. The next section considers the use of responses provided by GenAI artefacts, as a prelude to an assessment of impacts, implications and risks.
The previous sections have adopted a narrow conception of GenAI technology as artefact. This section considers the broader notion of GenAI technology-in-use. Important aspects of its use include the categories of users, requests, data-formats and responses, and the individual and organisational behaviours of users.
GenAI artefacts may be used in a wide variety of contexts. The user may be an individual human, for their personal purposes, gaining access to what might be compared with a digest of the multiple sources that might be delivered by using a search-engine. The user may be an individual as an agent for a small, informal group in a social setting. Alternatively, the individual and/or small group may be acting in a more structured environment, as employees, or in a formalised community group, and may apply the responses to more business-like purposes. The results may then be then integrated into the business processes of a corporation or an incorporated association. The user might, however, be an artefact, and the artefact may be capable of acting in the real world.
In Figure 2, a user is depicted as being within a context, composing a request, receiving a response, and applying it to one or more of a range of purposes ranging from mere addition to the user's body of knowledge, via changes in attitude, the drawing of an inference, the making of a decision, communication to others possibly with an influence on other parties' behaviour, to the performance of an action with impacts in the real world.
The categories of requests will vary, depending on the context and pattern of use. Some will be framed as questions, some as requests, while some will involve multiple, successive questions, requests and statements making up a dialogue between human(s) and an artefact. Depending on the context and pattern of use, the impact of the outcomes of an interaction with a GenAI artefact may be subject to adoption, moderation or rejection processes within and beyond an organisation. A first level of reflection and filtering may arise where the individual conducting or leading the interaction applies critical thought to the responses. On the other hand, the contrived fluidity and authoritativeness of expression of the responses may in many cases cause the individual to suspend their disbelief, and uncautiously embrace the responses as being reliable. Subsequent levels of reflection and filtering may arise through the dynamics of organisational processes; but enthusiasm and momentum may result in inferences, decisions and actions that are inappropriate or harmful.
The last century has seen a substantial increase in the social distance between organisations and the people who they deal with. This is typified by the disappearance of local bank managers and 'case managers' interacting directly with clients, and IT providers' abandonment of customer support and reliance on self-help among user communities. The current era of digitalisation involves the replacement of interpretation and management of the world through human perception and cognition, with processes that are almost entirely dependent on digital data (Brennen & Kreiss 2016). Digitalisation exacerbates the longstanding problems of social distance, in that organisations no longer relate to individuals, and instead make their judgements based on the digital personae that arise from the large pools of data associated with individual identities that the organisation has access to (Clarke 1994, 2019a). One of the effects of GenAI is to further amplify the social distance between organisations and people directly and indirectly affected by the organisations' actions. The inferences are drawn, and the decisions are made, in isolation from the affected parties. Further, the basis on which they drawn and made are obscured even from those participating in the process, let alone from affected individuals who are remote from the point of decision.
The desired attributes of the responses provided by Generative AI vary considerably, depending in particular on the form of the data to be produced (e.g. text, code, diagram, audio, image, video), and the nature of the request. A textual response needs to:
In order to deliver responses that are of satisfactory quality, each element of a GenAI artefact's process needs to fulfil a series of requirements, which the analysis in the preceding questions shows are generally not able to be delivered by GenAI technology. Where the request is expressed in written or spoken form in a natural language:
Interpretation of the response is a matter for the person(s) or artefact(s) that gain access to it. The overall design of a system that incorporates GenAI does, however, need to encompass means to ensure that users of responses are generally informed about their nature, and are provided with means to interrogate and/or collaborate with the AI artefact in order to have sufficient understanding of the strengths and weaknesses of the technique and its outputs, and how to ensure that they place only as much reliance on the quality of the Response as is justifiable. As Heersmink et al. (2024) expressed it, GenAI artefacts' "data and algorithmic opacity and their phenomenological and informational transparency can make it difficult for users to calibrate their trust correctly". The problem is compounded by naive republication by user organisations of providers' brochureware. See, for example, Table 3, extracted from a report by the California Government Operations Agency (Cal 2023).
Adapted from Table 2 Cal (2023, Table 2 , pp.11-12)
_______________
Interrogation and collaboration are very difficult functions to deliver with current tools. GenAI, like AI/ML, is merely empirical, based on large heaps of previous data. Inferences from them are a-rational, in the sense that there is no express algorithm, problem-solution or problem-definition underlying the output. This means that no humanly-understandable explanation of the reasoning underlying the output can be produced, and hence interactions by a human user with a Generative AI are limited in form, and stilted. Further, in the absence of explanations, human decision-makers cannot fulfil any obligations they may have to explain to interested parties (such as their managers, auditors, people affected by decisions, regulators and courts) why they made a decision or took an action. If it is infeasible to review decisions and actions, fairness and due process cannot be respected, and personal and organisational accountability is undermined.
GenAI's fallibility is acknowledged in text appearing at times in the footer of the ChatGPT input page at https://chatgpt.com/:
ChatGPT can make mistakes. Check important info.
The purpose of this section is to delve more deeply than OpenAI's bland statement about erroneous responses. For example, issues arise from responses that are not erroneous but are nonetheless misleading, and responses that appropriately reflect the content of the text sources relied upon, but give rise to negative consequences for some parties, particularly those who, or that, lack the power, resources or competencies to defend their interests.
Based on workshops with "thirty experts across industry, academia, civil society, and government", Solaiman et al. (2023, p.3) proposed seven categories of "social impact" (which extend into environmental and even political contexts): bias, stereotypes, and representational harms; cultural values and sensitive content; disparate performance; privacy and data protection; financial costs; environmental costs; and data and content moderation labor costs, and five categories of "broader societal context": trustworthiness and autonomy; inequality, marginalization, and violence; concentration of authority; labor and creativity; and ecosystem and environment. Those authors' depiction of categories is in Figure 3.
Reproduction of Figure 1 of Solaiman et al. (2023)
Bold and optimistic promotional statements about GenAI and its use have stimulated a flood of counter-statements of concern. Table 4 provides a checklist of concerns that have been expressed, drawing on such publications as Wach et al. (2023), Weidinger et al. (2023), pp.30-31, Zohny et al. (2023), Gupta et al. (2023), Schlagwein & Willcocks (2023), Hagendorff (2024). Distinctions are made between first-round, relatively direct Impacts, second-round, less direct Implications, and contingent outcomes or Risks.
Direct Impacts on Individuals
Direct Impacts on Society and the Polity
Indirect Implications
Risks
______________
In relation to the various forms of malinformation that can arise, the question has been raised as to whether providers of GenAI artefacts and services have, or should have an obligation to 'tell the truth' (Wachter et al. 2024). The responsibility might be operationalised as a duty of care to avoid the republication of misinformation and disinformation and manage hallucinations.
The previous section considered the use of GenAI artefacts, with an emphasis on the perspectives of its users, and the contexts in which they are acting. The impacts, implications and risks will to some extent affect users, but many other stakeholders are involved. A stakeholder in any particular use of a GenAI artefact is any party whose interests are, or may be, affected by that use. Users, as participants in the process, and the organisations for which they perform their actions, are stakeholders. The term 'usees' is descriptive of those parties that are affected, but are not participants (Berleur & Drumm 1991, Clarke 1992, Fischer-Hübner & Lindskog 2001, Baumer 2015).
Depending on the application area, many different parties may be stakeholders. The remainder of this section narrows the focus to individuals, such as loan applicants in credit evaluations, students in educational institutions, patients in health care contexts, and voters in political processes. This is partly because the majority of discussion to date has been about impacts on organisations, economies, societies and polities; and partly because individuals generally lack institutional or market power, and are likely to find it particularly difficult to defend their interests when confronted by decisions and actions involving GenAI.
The opaqueness of inferencing techniques used in GenAI have potentially serious consequences for some stakeholders, in particular those that rely on the protections referred to in various contexts as due process and procedural fairness. GenAI artefacts commonly exhibit a-rationality. That is because no description exists of how and why an outcome came about, and an ex post facto rationalisation of it may not be able to be constructed. In the fairly common circumstance in which a GenAI artefact changes over time, it is also likely to exhibit 'unreplicability', i.e. the process cannot be repeated. This undermines the scope for investigation, because a reconstruction of the sequence of events is infeasible. Inferencing and decision processes therefore become 'unauditable', in that an independent party such as an auditor, judge or coroner is precluded from identifying initial, intermediate and final states, and triggers for transitions between states. Further, because errors are undiscoverable, the GenAI's impact becomes 'uncorrectable'. The end result is that organisations and individuals become 'unaccountable', and escape responsibility for harmful decisions and actions (Clarke 2019b).
In (Clarke 2023), a set of impacts of AI generally was identified, which affect the economic and social interests of individuals. These, reproduced in Table 5 below, appear to also be impacts of GenAI artefacts in particular.
Reproduction of Table 2 of (Clarke 2023, p.29)
_____________________
The previous sections have described the nature and mechanics of GenAI artefacts, identified characteristics relevant to the technology's impacts, and considered the users and contexts of use, and their impacts and implications and the risks they give rise to. This lays the necessary foundations for the development of proposals to address downsides.
Long lists of impacts, implications and risks have been catalogued in the preceding sections. It is clear that regulatory measures are necessary, to provide protections, at least for individual users and usees, and perhaps much more broadly for communities, and societies, and economies. This section firstly outlines a generic regulatory framework, then considers several key factors in achieving responsible behaviour in relation to the application of GenAI, and finally draws on existing sources to propose a set of principles.
Various approaches to protecting interests have been proposed, emanating from various disciplinary and professional sources. For example, Weidinger et al. (2023) draws on safety engineering to propose a three-layered framework of capability evaluation, human interaction evaluation, and systemic impact evaluation. A copmrehensive framework to assist in the design and evaluation of regulatory schemes for disruptive IT was introduced in Clarke (2019d), in the context of AI generally. It has subsequently been further articulated, as shown in Figure 4, and applied to the platform-based business sector (Clarke 2020b), data protection (Clarke 2021), electronic markets (Clarke 2022a), and the application of AI to surveillance (Clarke 2022b).
It is common for the focus of discussions about regulation to fall mostly and even exclusively on layer (7) Formal Regulation. On the other hand, enforcement measures to achieve compliance with legislative provisions and mandated codes are only one of the mechanisms available. They tend to be inflexible and unadaptable, and take many years to come into operation. A broad segment exists in the middle layers (3)-(5) that are referred to as self-regulation or self-governance. These approaches (with features such as unenforceable codes, industry standards, customer charters and ombudspeople) are generally ineffectual unless a more formal overlay provides them with teeth. Frequently overlooked are natural regulatory mechanisms (Layer 1), such as competitive pressures and reputational damage ranging up to public opprobrium, and Infrastructural Regulation (Layer 2) can reinforce natural features, by building operational constraints into the sector. An example is encryption-by-default (such as SSL/TLS and HTTPS), which inbuilds data-security into message transmission.
Many innovations bring with them major challenges because pre-existing regulatory mechanisms are ineffective or not applicable. A long series of technological innovations that are partly-beneficial and partly-dangerous have been permitted to proliferate freely, with safeguards, controls and mitigation measures only emerging very slowly, delayed by the pleadings of parties interested in freedom of action and without regard to the harm caused to stakeholders. Contemporary society is reaping the industrial-era disbenefits of electricity generation from carboniferous materials, chemicals industries' products and waste, coolants, internal combustion engines, and nuclear plant failures and waste. It is now facing further, intensive threats from big-data analytics, very high energy consumption by highly computationally-intensive processes, digitalisation, AI generally, and AI/ML and GenAI in particular.
To deal with these threats, the precautionary principle needs to be applied. The weak form of the principle asserts that, if an action or policy is suspected of causing harm, and scientific consensus that it is not harmful is lacking, then the burden of proof falls on those taking the action (Wingspread 1998). The threats arising from irresponsible application of AI/ML appear to be sufficient to justify the strong form of mandation by formal law - which has to date seldom been applied other than in environmental contexts (TvH 2006): "When human activities may lead to morally unacceptable harm that is scientifically plausible but uncertain, actions shall be taken to avoid or diminish that potential harm".
The most effective regulatory schemes typically utilise elements from multiple layers of the heirarchy. Table 6, extracted from previous work, identifies criteria that can be applied when designing, or evaluating designs for, an effective regulatory regime.
Adapted from (Clarke 2014), Table 2
__________________
Nomatter what combination of regulatory measures is to be applied, and whether they are applied early in the life of an innovative technology or held back until damage has been done, it is essential that principles be developed that can achieve desired protections against negative impacts and implications, and deter, prevent, detect and mitigate risks.The following sub-section draws on existing sources to propose principles for the application of GenAI artefacts.
In prior work, a set of principles for responsible data analytics was presented (Clarke 2018), and a proposal articulated, which showed how those principles can be embedded in business processes (Clarke & Taylor 2019). The surge of interest in AI/ML during the decade 2010-2019 resulted in many organisations uttering 'principles for responsible AI', with varying degrees of superficiality and analytical depth, some of them generic, some specific to AI/ML. Several researchers drew on these many publications to abstract comprehensive sets, including Zeng et al. (2019). and Jobin et al. (2019). The set published in Clarke (2019c) comprises Themes, and Principles. Table 7 reproduces the 10 Themes, substituting 'GenAI for 'AI'. Appendix C reproduces the 50 Principles extracted from the literature, but with a second column containing suggestions for re-casting them in a form directly applicable to GenAI.
Adapted from
(Clarke
2019c), Table 4
The 50 Principles are reproduced in
Appendix
C, grouped under the 10 Themes
The following apply to each entity responsible for each of the five phases of AI: research, invention, innovation, dissemination and application.
GenAI offers prospects of considerable benefits and disbenefits. All entities involved in creating and applying GenAI have obligations to assess its short-term impacts and longer-term implications, to demonstrate the achievability of the postulated benefits, to be proactive in relation to disbenefits, and to involve stakeholders in the process.
Considerable public disquiet exists in relation to the replacement of human decision-making by inhumane decision-making by GenAI-based artefacts and systems, and displacement of human workers by GenAI-based artefacts and systems.
Considerable public disquiet exists in relation to the prospect of humans being subject to obscure GenAI-based processes, and ceding power to GenAI-based artefacts and systems.
All entities involved in creating and applying GenAI have obligations to provide safeguards for all human stakeholders, whether as users of GenAI-based artefacts and systems, or as usees affected by them, and to contribute to human stakeholders' wellbeing.
All entities involved in creating and applying GenAI have obligations to avoid, prevent and mitigate negative impacts on individuals, and to promote the interests of individuals.
All entities have obligations in relation to due process and procedural fairness. These obligations can only be fulfilled if all entities involved in creating and applying GenAI ensure that humanly-understandable explanations are available to the people affected by GenAI-based inferences, decisions and actions.
All entities involved in creating and applying GenAI have obligations in relation to the quality of business processes, products and outcomes.
All entities involved in creating and applying GenAI have obligations to ensure resistance to malfunctions (robustness) and recoverability when malfunctions occur (resilience), commensurate with the significance of the benefits, the data's sensitivity, and the potential for harm.
All entities involved in creating and applying GenAI have obligations in relation to due process and procedural fairness. The obligations include the entity ensuring that it is discoverable, and addressing problems as they arise.
Each entity's obligations in relation to due process and procedural fairness include the implementation of systematic problem-handling processes, and respect for and compliance with external problem-handling processes.
_______________
[ The applicability of the Themes in Table 7 and the Principles in Appendix C would benefit from re-assessment and adjustment to ensure that they appropriately address the specific characteristics of GenAI artefacts identified in the earlier sections of this paper. ]
[ WHAT APPROACH TO ADOPT? WHAT OUTCOMES? WHAT CONCLUSIONS? ]
The preceding section considered ways in which constructive reactions could be adopted to GenAI, accepting of its current form and nature, and seeking ways to exercise controls over its many weaknesses and the ease with which it is being, and will be, misused and abused. This section adopts a broader and proactive stance, proposing that the notions of AI in general, and GenAI in particular, are misconceived, that a quite different approach is needed to designing and applying such technologies, that technologies and their use need to be architected, and that appropriate features, safeguards and controls need to be embedded in both infrastructure and business processes.
The first sub-section considers the presumption that artefacts are to be designed as decision systems capable of autonomous action. The second sub-section summarises a previously-published proposal for reconception of the entire field of AI, in order to address problems that arise with it in all of its forms. The third sub-section argues that many of the issues that render GenAI dangerous to society derive from the indiscriminate monolithism of contemporary LLM, and that the transitional notion of a 'foundation model' needs to be urgently matured.
As summarised in Clarke (2019a), "The concept of 'automation' is concerned with the performance of a predetermined procedure, or response in predetermined ways to alternative stimuli. It is observable in humans, e.g. under hypnosis, and is designed-into many kinds of artefacts". Where machine learning techniques are applied, the procedure or response changes over time. The granting of autonomy to an artefact represents an act of delegation by humans to a device that is not directly subject to the responsibilities that are imposed on people and organisations. "The rather different notion of 'autonomy' means, in humans, the capacity for independent decision and action. Further, in some contexts, it also encompasses a claim to the right to exercise that capacity. It is associated with the notions of consciousness, sentience, self-awareness, free will and self-determination" (p.426).
Many discussions of automation in general, and robotics in particular, fail to reflect the notion of degrees or layers of autonomous behaviour. Table 8 distinguishes seven layers, categorised into two fundamentally different categories, 'decision systems' or 'action systems' that have direct impacts on the real world, and 'decision support systems' or 'action support systems' that may have significant influence on decisions and actions, but whose outputs are mediated by humans.
After Table 1 of Clarke (2019b, p.27)
|
Function of the Artefact |
Function of the Controller |
|
Decision or |
1. |
NIL |
Analyse, Decide, Act |
2. |
Analyse Options |
Decide among Options |
|
3. |
Advise re Options |
Decide among Options |
|
4. |
Recommend |
Approve / Reject Recommended Action |
|
Decision
or |
5. |
Notify
Impending |
Override / Veto |
6. |
Act and Inform |
Interrupt/Suspend/Cancel |
|
7. |
Act |
NIL |
Many researchers, innovators and marketers are motivated to exercise the strong form of technological determinism and argue that artefacts (Frankenstein's Monster, robots, 'cyborgs', 'AIs') are already, or will shortly be, more competent than humans, and the natural successor to homo sapiens, and that a technological 'singularity' is imminent (Vinge 1981, Kurzweil 2005). A more desirable and more likely path is recognition that both human intellect and artefactual capabilities have strengths and weaknesses, and that the appropriate approach is to blend strengths to manage weaknesses.
Sabherwal & Grover (2024) argue that "If the emergent instantiations of [GenAI] ... are associated with augmentation over automation, then the societal impacts of [GenAI] will be positive; otherwise, the societal impacts of [GenAI] will be negative" (p.18), and "The extent to which we can exercise regulatory control over the divergence between physical and digital reality will have profound implications for the directional impact of [GenAI] on society" (p.18). The following sub-sections propose that re-conception of both AI generally, and GenAI in particular, are critical to the achievement of outcomes that serve the interests of individuals and societies.
In Clarke (2023), it is argued that "the idea of 'artificial intelligence' was misdirected, and has resulted in a great deal of wasted effort during the last 70 years. We don't want artificial; we want real. We don't want more intelligence of a human kind; we want artefacts to contribute to our intellectual endeavours. Useful intelligence in an artefact, rather than being like human intelligence, needs to be not like it, and instead constructively different from it". A more useful notion is 'Complementary Artefact Intelligence' (CAC), whose attributes were proposed in Clarke (2019b, p.430) as:
Schneiderman (2020, pp.116-118) subsequently proposed a "shift from emulating humans to empowering people" and "extend abilities, empower users, enhance human performance" and "humans in the group; computers in the loop" (Schneiderman 2021, p.58, 2022).
The 'complementary artefact intelligence' conception provides a bridge across to the established notion of 'Augmented Intelligence': the integration of human intelligence and complementary artefact intelligence into a whole that is different from, and superior to, either working alone (Ashby 1956, Engelbart 1962, Zheng et al. 2017, Abbas and Michael 2022). The upper part of Figure 5 depicts that alternative approach to the dated idea of 'Artificial' Intelligence.
Reproduced from Clarke (2023, Figure 1, p.31)
The new conception also provides means of resolving the hitherto vague relationship between 'Artificial' Intelligence and robotics. Humans have means of performing actions in the real world. Artefacts have actuators. Actuators can be designed to operate independently of humans, but more can be gained by designing for synergy between the two, resulting in Augmented Actors. The combination of augmented actors with augmented intellect delivers capabilities of analysis, decision and action. This points to the slowly-developing field of 'cobotics' (Colgate et al. 1996, Peshkin et al. 1999).
This re-conception is as applicable to GenAI as to other forms of existing AI. The final sub-section presents a more specific proposal to extract value from GenAI while managing the negative impacts, implications and risks.
The current approach to GenAI artefacts is to present general-purpose services built by abstracting language models from vast quantities of text-sources gathered without regard for their provenance and characteristics. This 'convenience data' approach, because it has been indiscriminate, has vacuumed up sources of highly variable quality and mutual compatibility, and highly variable relevance to any particular request that a user makes. This problem has been exacerbated by feeding back into the source-texts the synthetic text generated by the technology, further polluting an already very dirty pool of material. Attempts to refine the pool, e.g. by data scrubbing and by the ex post facto deletion of instances of material that are judged on some limited grounds to infringe, for example, public tastes or public policy settings, have further confused the source-text collections, the linguistic models arising from them, and the responses generated from them.
An alternative approach can deliver greater reliability of responses, and enable misuse and abuse to be managed. The elements of this approach are outlined in Table 9.
________________
The proposals arising from the research reported in the final two sections of this paper are intended to redirect the conception, design and application of GenAI. The momentum that has arisen from the unbridled enthusiasm of promoters, combined with the inadequate scepticism and care of adopters, is such that it appears highly unlikely that sufficient organisations will heed the warnings and adapt their behaviours. It is therefore essential that the constructive use of the proposals in this paper be accompanied by public policy and legislative action to drive home the messages.
This paper has proposed Principles for responsible design and use of GenAI artefacts, and re-conception of them to fit within the context of decision and action support systems, artefact intelligence complementary to human intelligence, and augmented intelligence, and the maturation of robotics towards cobotics, to deliver augmented capability.
The proposals are grounded in a sufficiently rich description of the technology and use of contemporary GenAI artefacts, and analyses of those artefacts' characteristics, their impacts and implications, the risks they give rise to, and appropriate means of regulating the technology and its users.
The analyses and proposals require scrutiny, evaluation, reflinement and application firstly as guidance for the design and use of GenAI artefacts. Their second contribution is as a basis for devising regualtory regimes that will protect stakeholders against harms while encouraging and facilitating appropriate design, use and exploitation of the technology's promise.
Abbas R., Michael K., Michael M.G., Perakslis C. & Pitt, J. (2022) 'Machine learning, convergence digitalization, and the concentration of power: Enslavement by design using techno-biological behaviors' IEEE Transactions on Technology and Society 3,2 (2022) 76-88, at https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=9801492
ACCC (2021) 'Digital advertising services inquiry: Final report' Australian Competition and Consumer Commission, August 2021, at https://www.accc.gov.au/system/files/Digital%20advertising%20services%20inquiry%20-%20final%20report.pdf
Adamopoulou E. & Moussiades L. (2020) 'Chatbots: History, Technology, and Applications' Machine Learning with Applications 2 (Dec. 2020), at https://doi.org/10.1016/j.mlwa.2020.100006
Adamson G. (2022) 'Explainable artificial intelligence (XAI): A reason to believe?' Law in Context 37,3 (Apr. 2022), at https://search.informit.org/doi/pdf/10.3316/informit.553002286408436
Adensamer A, Gsenger R. & Klausner L.D. (2021) 'ÅgComputer says noÅh: Algorithmic decision support and organisational responsibility' Journal of Responsible Technology 7?8 (October 2021) 100014, at https://www.sciencedirect.com/science/article/pii/S266665962100007X
Aimeur E., Amri S. & Brassard G. (2023) 'Fake news, disinformation and misinformation in social media: a review' Social Network Analysis and Mining 13,30 (2023), at https://link.springer.com/content/pdf/10.1007/s13278-023-01028-5.pdf
Asaro P.M. (2019) 'AI Ethics in Predictive Policing: From Models of Threat to an Ethics of Care' IEEE Technology and Society Magazine 38,2 (June 2019) 40-53, at https://peterasaro.org/writing/Asaro_PredictivePolicing.pdf
Baldassarre M.T., Caivano D., B.F. Nieto, Gigante D. & Ragone A. (2023) 'The Social Impact of Generative AI: An Analysis on ChatGPT' Proc. GoodIT, September 2023, Lisbon, at https://dl.acm.org/doi/pdf/10.1145/3582515.3609555
Baumer E.P.S. (2015) 'Usees' Proc. 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI'15), April 2015
Bellow P. (2023) 'Maintaining conversation continuity with the ChatGPT API' OpenAI Community, December 2023, at https://community.openai.com/t/how-can-i-maintain-conversation-continuity-with-chat-gpt-api/574819/3
Bender E.M., Gebru T., McMillan-Major A. & Shmitchell S. (2021) 'On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?' Proc. FAccT'21, March 3-10, 2021, Virtual Event, Canada, pp.610-623, at https://dl.acm.org/doi/pdf/10.1145/3442188.3445922
Bennett Moses L. & Chan J. (2018) 'Algorithmic prediction in policing: assumptions, evaluation, and accountability' Policing and Society 28, 7 (2018) 806?822, at https://www.tandfonline.com/doi/full/10.1080/10439463.2016.1253695
Berleur J. & Drumm J. (Eds.) (1991) 'Information Technology Assessment' Proc. 4th IFIP-TC9 International Conference on Human Choice and Computers, Dublin, July 8-12, 1990, Elsevier Science Publishers (North-Holland), 1991
Bommasani Ret al. (2023) 'On the Opportunities and Risks of Foundation Models' arXiv, at https://arxiv.org/pdf/2108.07258
Borgesius F.J.Z. (2020) 'Strengthening legal protection against discrimination by algorithms and artificial intelligence' Intl J. of Human Rights 24, 10 (2020) 1572?1593, at https://www.tandfonline.com/doi/pdf/10.1080/13642987.2020.1743976
Brennen S. & Kreiss D. (2016) 'Digitalization and Digitization' International Encyclopedia of Communication Theory and Philosophy, October 2016, PrePrint at http://culturedigitally.org/2014/09/digitalization-and-digitization/
Cal (2023) 'Benefits and Risks of Generative Artificial Intelligence Report' State of California, November 2023, at https://www.govops.ca.gov/wp-content/uploads/sites/11/2023/11/GenAI-EO-1-Report_FINAL.pdf
Chiang T. (2023) 'ChatGPT Is a Blurry JPEG of the Web' The New Yorker, 9 February 2023, at https://www.newyorker.com/tech/annals-of-technology/chatgpt-is-a-blurry-jpeg-of-the-web#:~:text=Think%20of%20ChatGPT%20as%20a,ever%20get%20is%20an%20approximation
Chu X. & Ilyas I.F. (2016) 'Qualitative Data Cleaning' Proc. VLDB Endowment 9,13 (2016) at https://dl.acm.org/doi/pdf/10.14778/3007263.3007320
Clark L. (2012) 'Google's Artificial Brain Learns to Find Cat Videos' Wired Magazine, 26 June 2012, at https://www.wired.com/2012/06/google-x-neural-network/
Clarke R. (1992) 'Extra-Organisational Systems: A Challenge to the Software Engineering Paradigm' Proc. IFIP World Congress, Madrid, September 1992, PrePrint at https://www.rogerclarke.com/SOS/PaperExtraOrgSys.html
Clarke R. (1994) 'The Digital Persona and its Application to Data Surveillance' The Information Society 10,2 (June 1994) 77-92, PrePrint at http://www.rogerclarke.com/DV/DigPersona.html
Clarke R. (2014) 'The Regulation of Civilian Drones' Impacts on Public Safety' Computer Law & Security Review 30, 3 (June 2014) 263-285, PrePrint at http://www.rogerclarke.com/SOS/Drones-PS.html
Clarke R. (2016) 'Big Data, Big Risks' Information Systems Journal 26, 1 (January 2016) 77-90, PrePrint at http://www.rogerclarke.com/EC/BDBR.html
Clarke R. (2018) 'Guidelines for the Responsible Application of Data Analytics' Computer Law & Security Review 34,3 (May-Jun 2018) 467- 476, PrePrint at http://www.rogerclarke.com/EC/GDA.html
Clarke R. (2019a) 'Risks Inherent in the Digital Surveillance Economy: A Research Agenda' Journal of Information Technology 34,1 (Mar 2019) 59-80, PrePrint at http://www.rogerclarke.com/EC/DSE.html
Clarke R. (2019b) 'Why the World Wants Controls over Artificial Intelligence' Computer Law & Security Review 35, 4 (2019) 423-433, PrePrint at http://www.rogerclarke.com/EC/AII.html
Clarke R. (2019c) 'Principles and Business Processes for Responsible AI' Computer Law & Security Review 35,4 (2019) 410-422, PrePrint at http://www.rogerclarke.com/EC/AIP.html
Clarke R. (2019d) 'Regulatory Alternatives for AI' Computer Law & Security Review 35, 4 (2019) 398-409, PrePrint at http://www.rogerclarke.com/EC/AIR.html
Clarke R. (2020a) 'Responsible AI Technologies, Artefacts, Systems and Applications: 50 Principles' Xamax Consultancy Pty Ltd, 15 January 2020, at https://www.rogerclarke.com/EC/RAIC-50Ps-200115.pdf
Clarke R. (2020b) 'RegTech Opportunities in the Platform-Based Business Sector' Proc. Bled eConference, June 2020, pp. 79-106, PrePrint at http://rogerclarke.com/EC/RTFB.html
Clarke R. (2021) 'A Comprehensive Framework for Regulatory Regimes as a Basis for Effective Privacy Protection' Proc. 14th Computers, Privacy and Data Protection Conference (CPDP'21), Brussels, 27-29 January 2021, PrePrint at http://rogerclarke.com/DV/RMPP.html
Clarke R. (2022a) 'Research Opportunities in the Regulatory Aspects of Electronic Markets' Electronic Markets 32, 1 (Jan-Mar 2022) 179-200, PrePrint at http://rogerclarke.com/EC/RAEM.html
Clarke R. (2022b) 'Responsible Application of Artificial Intelligence to Surveillance: What Prospects?' Information Polity 27, 2 (Jun 2022) 175-191, PrePrint at http://rogerclarke.com/DV/AIP-S.html
Clarke R. (2023) 'The Re-Conception of AI: Beyond Artificial, and Beyond Intelligence' IEEE Transactions on Technology & Society 4,1 (March 2023) 24-33, PrePrint at http://rogerclarke.com/EC/AITS.html
Clarke R. (2024) 'The Authentication of Textual Assertions' Proc. 10th IS Foundations Workshop, ANU, 10-11 September 2024, PrePrint at http://rogerclarke.com/ID/ATS-ISF.html
Clarke R. & Taylor K. (2018) 'Towards Responsible Data Analytics: A Process Approach' Proc. Bled eConference, 17-20 June 2018, PrePrint at http://www.rogerclarke.com/EC/BDBP.html
Colgate J.E., Wannasuphoprasit W. & Peshkin M.A. (1996) 'Cobots: Robots for Collaboration With Human Operators' Paper No: IMECE1996-0367, pp. 433-439, 1996, at https://doi.org/10.1115/IMECE1996-0367
Crouse M. (2024) 'Generative AI Defined: How it Works, Benefits and Dangers' Tech Republic, 21 June 2024, at https://www.techrepublic.com/article/what-is-generative-ai/
De Angeli A., Johnson G.I. & Coventry L. (2001) 'The unfriendly user: exploring social reactions to chatterbots' Proc. Int'l Conf. on Affective Human Factors Design, Asean Academic Press, London, 2001, at https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=030e394decf7c07cc358d0f24fc3d5e46bc55219
Dodge J. et al. (2021) 'Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus' arXiv.org, 30 Sep 2021, at https://arxiv.org/pdf/2104.08758
Dodgson N. (2023) 'Artificial intelligence: ChatGPT and human gullibility' Policy Quarterly 19,3 (August 2023) 19-24, at https://ojs.victoria.ac.nz/pq/article/download/8308/7374/11943
Dreyfus H.L. (1992) 'What Computers Still Can't Do: A Critique of Artificial Reason' MIT Press, 1972, 1979, 1992
Engstrom D.F., Ho D.E., Sharkey C.M. & Cue?llar N.-F. (2020) 'Government by Algorithm: Artificial Intelligence in Federal Administrative Agencies' NYU School of Law, Public Law Research Paper No. 20-54, April 2020, at https://ssrn.com/abstract=3551505
Fischer-Hübner, S. & Lindskog, H. (2001). `Teaching Privacy-Enhancing Technologies,' Proceedings of the IFIP WG 11.8 2nd World Conference on Information Security Education, Perth, Australia, 2001
Jones E. (2023) 'What is a foundation model?' Lovelace Institute, 17 July 2023, at https://www.adalovelaceinstitute.org/resource/foundation-models-explainer/
Foote K.D. (2023) 'A Brief History of Large Language Models' Dataversity, 28 December 2023, at https://www.dataversity.net/a-brief-history-of-large-language-models/
Gupta M., Akiri C., Aryal K., Parker E. & Praharaj L. (2023) 'From ChatGPT to ThreatGPT: Impact of Generative AI in Cybersecurity and Privacy' IEEE Access 11 (2023) 80218-80254, at https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=10198233
Hagendorff T. (2024) 'Mapping the ethics of generative ai: A comprehensive scoping review' arXiv, 2024, at https://arxiv.org/pdf/2402.08323
Hardesty L. (2017) 'Explained: Neural networks' MIT News, 14 April 2017, at https://news.mit.edu/2017/explained-neural-networks-deep-learning-0414
Heersmink R., de Rooij B., V?zquez M.J.C. & Colombo M. (2024) 'A phenomenology and epistemology of large language models: transparency, trust, and trustworthiness' Ethics and Information Technology 26,41 (2024), at https://link.springer.com/article/10.1007/s10676-024-09777-3?fromPaywallRec=false
IBM (2023) 'What is generative AI?' IBM, 20 April 2023, at https://research.ibm.com/blog/what-is-generative-AI
Ji Z. et al. (2023) 'Survey of Hallucination in Natural Language Generation' ACM Comput. Surv. 55,12 (March 2023) 248:1-38, at https://dl.acm.org/doi/pdf/10.1145/3571730
Jobin A., Ienca M. & Vayena E. (2019) 'The global landscape of AI ethics guidelines' Nature Machine Intelligence 1 (September 2019) 389-399, at https://doi.org/10.1038/s42256-019-0088-2
Kaubre (2024) 'LLM Training Data: The 8 Main Public Data Sources' Oxylabs, 27 September 2024, at https://oxylabs.io/blog/llm-training-data
Kurzweil R. (2005) 'The singularity is near' Viking Books, 2005
Lappin S. (2024) 'Assessing the Strengths and Weaknesses of Large Language Models' J of Log Lang and Inf 33 (2024) 9?20, at https://doi.org/10.1007/s10849-023-09409-x
Linardatos P., Papastefanopoulos V. & Kotsiantis S. (2021) 'Explainable AI: A Review of Machine Learning Interpretability Methods' Entropy 23,18 (2021), at https://www.mdpi.com/1099-4300/23/1/18/pdf
Lipton Z.C. (2017) 'The Mythos of Model Interpretability' Queue 16,3 (May-June 2018) 31-57, at https://dl.acm.org/doi/pdf/10.1145/3236386.3241340
Merritt R. (2023) 'What Are Foundation Models?' Nvidia, 13 March 2023, at https://blogs.nvidia.com/blog/what-are-foundation-models/
Morris C. (1938) 'Foundations of the theory of signs' University of Chicago Press, 1938
Nvidia (2023) 'Large Language Models Explained' Nvidia, 31 May 2023, at https://www.nvidia.com/en-us/glossary/large-language-models/
Nvidia (2024) 'What is Generative AI?' Nvidia, 1 May 2024, at https://www.nvidia.com/en-gb/glossary/generative-ai/
OpenAI (2022) 'Introducing ChatGPT' OpenAI, 30 November 2022, at https://openai.com/index/chatgpt/, accessed 27 October 2024
OpenAI (2024) 'Moderation' OpenAI, undated, apparently of 2024, at https://platform.openai.com/docs/guides/moderation, accessed 27 October 2024
Peshkin M.A., Colgate J.E., Wannasuphoprasit W., Moore C.A., Gillespie R.B. & Akella P. (1999) Cobot Architecture' IEEE Trans. on Robotics and Automation 17,4 (August 2001) 377-390, at https://public.websites.umich.edu/~brentg/Web/Journal/PeshkinTROCobotArchitecture.pdf
Sabherwal R. & Grover V. (2024) 'The Societal Impacts of Generative Artificial Intelligence: A Balanced Perspective' Journal of the Association for Information Systems, 25,1 (2024) 13-22
Schlagwein D. & Willcocks L. (2023) ''ChatGPT et al.': The ethics of using (generative) artificial intelligence in research and science' Journal of Information Technology 38,3 (2023) 232?238
Sharma N., Liao Q.V. & Xiao Z. (2024) 'Generative Echo Chamber? Efects of LLM-Powered Search Systems on Diverse Information Seeking' Proc. CHI Åf24, May 2024, Honolulu, at https://dl.acm.org/doi/pdf/10.1145/3613904.3642459
Smith A. (2021) 'Face scanning and Åesocial scoringÅf AI can have Åecatastrophic effectsÅf on human rights, UN says' The Independent, 16 September 2021, at https://www.independent.co.uk/tech/artificial-intelligence-united-nations-face-scan-social-score-b1921311.html
Solaiman I. et al. (2023) 'Evaluating the Social Impact of Generative AI Systems in Systems and Society' arXiv, June 2023, at https://arxiv.org/pdf/2306.05949
Talaat A.S. (2023) 'Sentiment analysis classification system using hybrid BERT models' Journal of Big Data 10,110 (2023), at https://journalofbigdata.springeropen.com/articles/10.1186/s40537-023-00781-w
Turing (2024) 'Prominent Data Collection Methods and Tools for LLMs' Turing Institute, 2 May 2024, at https://www.turing.com/resources/data-collection-methods-and-tools-for-llms
TvH (2006) 'Telstra Corporation Limited v Hornsby Shire Council' NSWLEC 133 (24 March 2006), esp. paras. 113-183, at http://www.austlii.edu.au/au/cases/nsw/NSWLEC/2006/133.htm
Vaswani A., Jones L., Shazeer N., Parmar N., Uszkoreit J., Gomez A.N., Kaiser L. & Polosukhin I. (2017) 'Attention Is All You Need' Proc. 31st Conf. Neural Information Processing Systems (NIPS 2017), Long Beach, CA, at https://user.phil.hhu.de/~cwurm/wp-content/uploads/2020/01/7181-attention-is-all-you-need.pdf
Vinge V. (1981) 'True Names' Dell Binary Star, 1981
Wach K. et al. (2023) 'The dark side of generative artificial intelligence: A critical analysis of controversies and risks of ChatGPT' Entrepreneurial Business and Economics Review 11,2 (2023) 7-30, at https://doi.org/10.15678/EBER.2023.110201
Wachter S., Mittelstadt B. & Russell C. (2024) 'Do large language models have a legal duty to tell the truth?' R. Soc. Open Sci. 11:240197, at https://royalsocietypublishing.org/doi/pdf/10.1098/rsos.240197
Weidinger L. et al. (2023) 'Sociotechnical Safety Evaluation of Generative AI Systems' arXiv, 2023, at https://arxiv.org/pdf/2310.11986
Weizenbaum J. (1966) 'Eliza' Commun. ACM 9,1 (Jan 1966) 36-45, at https://dl.acm.org/doi/pdf/10.1145/365153.365168
Weizenbaum J. (1967) 'Contextual Understanding by Computers' Communications of the ACM l0,8 (August 1967) 474-480, at https://dl.acm.org/doi/pdf/10.1145/363534.363545
Wigan M., Adamson G., Ranim P., Dyson N. & Horton F. (2022) 'Chatbots and Explainable Artificial Intelligence' Proc. IEEE Int'l Symposium on Technology and Society (ISTAS), 2022, at https://ieeexplore.ieee.org/document/10227122
Willcocks L. et al. (2023) 'Maximizing Value with Automation and Transformation: A RealistÅfs Guide' Palgrave Macmillan, London, 2023
Wingspread (1998) 'Wingspread Conference on the Precautionary Principle' Wingspread Statement on the Precautionary Principle, January 1998, at http://sehn.org/wingspread-conference-on-the-precautionary-principle/
Zeng Y., Lu E. & Huangfu C. (2019) 'Linking Artificial Intelligence Principles' Proc. AAAI Workshop on Artificial Intelligence Safety (AAAI-Safe AI 2019), 27 January 2019, at https://arxiv.org/abs/1812.04814
Zewe A. (2023) 'Explained: Generative AI' MIT News 23 November 2023, at https://news.mit.edu/2023/explained-generative-ai-1109
Zohny H., McMillan J. & King M. (2023) 'Ethics of generative AI' Journal of Medical Ethics 49 (2023) 79?80, at https://jme.bmj.com/content/medethics/50/2/75.full.pdf
Reproduction of Table 1 of Clarke (2018)
Reproduction of Table 4 of Clarke (2024)
Criterion
|
Guidance
|
The Work
|
|
• The Author
|
Check their identity, affiliations, prior works
|
• The Venue(s) / Channel(s)
|
Check the nature, ownership, declared mission and reputation ofeach publication venue and channel
|
• The Style
|
Check for expression, rigour, polemic and satire
|
• The Values
|
Identify the values embedded in the content
|
• The Author's Purpose
|
Identify the declared and/or implied or inferred motivations forcreating the Work
|
The Author's Assertions
|
|
• Their Clarity
|
Isolate the key assertions, and examine for ambiguities andemotional language
|
• Their Internal Consistency
|
Identify other related passages within the Work, and check forinconsistencies among them
|
• The Sources
|
Identify declared, implied and inferred sources, including byWeb-searches on key expressions and passages, and evaluateseemingly key sources, using this Guidance
|
The Author's Argument
|
|
• Its Clarity
|
Extract and critique the flow of the author's argument
|
• Its Premises and Assumptions
|
Identify the starting-points of the argument
|
• The Logic
|
Test for fallacious forms of argument
|
Evidence and Counter-Evidence
|
|
• Other Sources
|
Independently of the Work identify other sources on key aspects ofthe Premises and Assumptions
|
• Their Content
|
Assess the nature and degree of in/consistency of the Work withthose Sources
|
Extended Version of Clarke (2020a)
Column 1 reproduces the 50 Principles for Responsible AI Technologies, Artefacts, Systems and Applications. These are intended to be applied by the entities responsible for all phases of AI research, invention, innovation, dissemination and application. They were derived by consolidating elements from 30 international sources on 'Ethical Analysis and IT' and 'Principles for AI'. Column 2 suggests wording relevant to the use of GenAI artefacts.
The 10 Themes and 50 Principles |
Relevance to the Use of GenAI Artefacts |
1. Assess Positive and Negative Impacts and Implications |
|
1.1 Conceive and design only after ensuring adequate understanding of purposes and contexts |
Think before asking a question or composing a request |
1.2 Justify objectives |
Ask yourself whether what your intended purpose is 'the right thing to do' |
1.3 Demonstrate the achievability of postulated benefits |
Recognise that, the bigger the negative consequences, the more confidence needs to exist in the expected positive outcomes |
1.4 Conduct impact assessment, including risk assessment from all stakeholders' perspectives |
Consider stakeholders and the interests they want to protect |
1.5 Publish sufficient information to stakeholders to enable them to conduct impact assessment |
Enable stakeholders to consider the impacts on their interests |
1.6 Conduct consultation with stakeholders and enable their participation in design |
Gain insights into stakeholders' views |
1.7 Reflect in the design the concerns of stakeholders |
Understand that all parties may benefit if the design balances all parties' interests, and mitigates unavoidable harm |
1.8 Justify negative impacts on individuals ('proportionality') |
Ensure that harm done is not disproportionate to the benefits achieved |
1.9 Consider alternative, less harmful ways of achieving the same objectives |
Remain awake to better opportunities to achieve much the same outcomes |
2. Complement Humans |
|
2.1 Design as an aid for people, to augment their capabilities, and support collaboration and inter-operability |
Think of a GenAI artefact as a tool for authors, analysts, decision-makers, and people who act in the real world. |
2.2 Avoid design for replacement of people by independent artefacts or systems, except in circumstances in which those artefacts or systems are demonstrably more capable than people, and even then ensuring that the result is complementary to human capabilities |
Recognise GenAI's limitations, and avoid thinking of it as an author, an analyst, a decision-maker, or an actor. Sign off on inferences, decisions and actions, to make clear that they are your responsibility, not that of a GenAI artefact |
3. Ensure Human Control |
|
3.1 Ensure human control over AI-based technology, artefacts and systems |
Think before using a response |
3.2 In particular, ensure control over autonomous behaviour of AI-based technology, artefacts and systems |
Avoid uncritical adoption of a response, and of use of it in reaching conclusions, making decisions and taking actions. Sign off on inferences, decisions and actions, to make clear that they are your responsibility, not that of a GenAI artefact |
3.3 Respect people's expectations in relation to personal data protections, including their awareness of data-usage, their consent, data minimisation, public visibility and design consultation and participation, and the relationship between data-usage and the data's original purpose |
When using GenAI, go beyond the basics of data protection law. People are threatened by the intrusion of unfeeling machines into organisations' decision-making about human beings. |
3.4 Respect each person's autonomy, freedom of choice and right to self-determination |
When using GenAI, recognise that people also see these higher-order values as being outside the frame of reference that machines can be made to use |
3.5 Ensure human review of inferences and decisions prior to action being taken |
When using GenAI, recognise that responsibility rests with organisations and their employees |
3.6 Avoid deception of humans |
When using GenAI, provide explanations, and not pretexts and cover stories. If you can't explain to yourself why you're doing something, don't do it. |
3.7 Avoid services being conditional on the acceptance of obscure AI-based artefacts and systems and opaque decisions |
When using GenAI, if you can't explain to others why you're doing something, don't do it. |
4. Ensure Human Safety and Wellbeing |
|
4.1 Ensure people's physical health and safety ('nonmaleficence') |
When using GenAI, intend to avoid doing bad things to people |
4.2 Ensure people's psychological safety, by avoiding negative effects on their mental health, emotional state, inclusion in society, worth, and standing in comparison with other people |
When using GenAI, intend to avoid doing bad things to people |
4.3 Ensure people's wellbeing ('beneficence') |
When using GenAI, intend to do good things for people |
4.4 Implement safeguards to avoid, prevent and mitigate negative impacts and implications |
When using GenAI, make sure intentions to avoid doing bad things are carried through |
4.5 Avoid violation of trust |
When using GenAI, make sure your behaviour is trustworthy |
4.6 Avoid the manipulation of vulnerable people, e.g. by taking advantage of individuals' tendencies to addictions such as gambling, and to letting pleasure overrule rationality |
When using GenAI, avoid exercising power over people not capable of protecting their own interests |
5. Ensure Consistency with Human Values and Human Rights |
|
5.1 Be just / fair / impartial, treat individuals equally, and avoid unfair discrimination and bias, not only where they are illegal, but also where they are materially inconsistent with public expectations |
Recognise that GenAI, particularly when using very large and indiscriminate sources of text, provides biassed responses and discriminates against people in unacceptable ways |
5.2 Ensure compliance with human rights laws |
When using GenAI, recognise that human rights responsibility rests with organisations and their employees |
5.3 Avoid restrictions on, and promote, people's freedom ofmovement |
When using GenAI, recognise that human rights responsibility rests with organisations and their employees |
5.4 Avoid interference with, and promote privacy, family, home orreputation |
When using GenAI, recognise that human rights responsibility rests with organisations and their employees |
5.5 Avoid interference with, and promote, the rights of freedom of information, opinion and expression, of freedom of assembly, of freedom of association, of freedom to participate in public affairs, and of freedom to access public services |
When using GenAI, recognise that human rights responsibility rests with organisations and their employees |
5.6 Where interference with human values or human rights is outweighed by other factors, ensure that the interference is no greater than is justified ('harm minimisation') |
When using GenAI, minimise the harm to people, and implement measures to mitigate the harm |
6. Deliver Transparency and Auditability |
|
6.1 Ensure that the fact that a process is AI-based is transparent to all stakeholders |
Be open and honest about the fact that GenAI is used |
6.2 Ensure that data provenance, and the means whereby inferences are drawn from it, decisions are made, and actions are taken, are logged and can be reconstructed |
Do not use GenAI that is opaque, because the business process and the rationale must be able to be explained to those affected, and to those exercising regulatory powers |
6.3 Ensure that people are aware of inferences, decisions and actions that affect them, and have access to humanly-understandable explanations of how they came about |
Be open and honest to affected people about the fact that GenAI is used, and be ready to explain the business process and the rationale to those affected |
7. Embed Quality Assurance |
|
7.1 Ensure effective, efficient and adaptive performance ofintended functions |
Do not use GenAI that is opaque, because responsibility rests with organisations and their employees, and the reliability of the business process and the rationale cannot be assured |
7.2 Ensure data quality and data relevance |
Do not use GenAI that is opaque, because responsibility rests with organisations and their employees, and the reliability of the data quality and data relevance cannot be assured |
7.3 Justify the use of data, commensurate with each data-item'ssensitivity |
When using GenAI, review the input provided to ensure that the process has used all relevant information, and only that information |
7.4 Ensure security safeguards against inappropriate data access, modification and deletion, commensurate with its sensitivity |
Do not use GenAI that would add the input provided into the source-texts resulting in breach of data protection requirements |
7.5 Deal fairly with people ('faithfulness', 'fidelity') |
Do not use GenAI responses without human review of the provisional inferences drawn, recommendations, decisions or actions |
7.6 Ensure that inferences are not drawn from data using invalid or unvalidated techniques |
When using GenAI, do not rely on responses without applying an understanding of the context, using your common sense, andapplying 'the pub test' |
7.7 Test result validity, and address the problems that are detected |
When using GenAI, do not rely on responses without checking their reasonableness in comparison with norms, cases, precedents or archetypes |
7.8 Impose controls in order to ensure that the safeguards are in place and effective |
When using GenAI, make sure intentions to avoid doing bad things are carried through |
7.9 Conduct audits of controls |
When using GenAI, check that controls are functioning as intended |
8. Exhibit Robustness and Resilience |
|
8.1 Deliver and sustain appropriate security safeguards against the risk of compromise of intended functions arising from both passive threats and active attacks, commensurate with the significance of the benefits and the potential to cause harm |
When using GenAI, make sure service threats and vulnerabilities are managed |
8.2 Deliver and sustain appropriate security safeguards against the risk of inappropriate data access, modification and deletion, arising from both passive threats and active attacks, commensurate with the data's sensitivity |
When using GenAI, make sure data threats and vulnerabilities are managed |
8.3 Conduct audits of the justification, the proportionality, the transparency, and the harm avoidance, prevention and mitigation measures and controls |
When using GenAI, check that controls are functioning as intended |
8.4 Ensure resilience, in the sense of prompt and effective recovery from incidents |
When using GenAI, check that fallback arrangements are in place |
9. Ensure Accountability for Legal and Moral Obligations |
|
9.1 Ensure that the responsible entity is apparent or can be readily discovered by any party |
When using GenAI, be open about where the responsibility lies |
9.2 Ensure that effective remedies exist, in the forms of complaints processes, appeals processes, and redress where harmful errors have occurred |
When using GenAI, operate well-documented processes to receive, address and manage incidents and cases |
10. Enforce, and Accept Enforcement of, Liabilities and Sanctions |
|
10.1 Ensure that complaints, appeals and redress processes operate effectively |
When using GenAI, check that controls are functioning as intended |
10.2 Comply with external complaints, appeals and redress processes and outcomes, including, in particular, provision of timely, accurate and complete information relevant to cases |
When using GenAI, cooperate constructively with regulatory agencies |
TEXT
Roger Clarke is Principal of Xamax Consultancy Pty Ltd, Canberra. He is also a Visiting Professorial Fellow associated with UNSW Law & Justice, and a Visiting Professor in the Research School of Computer Science at the Australian National University.
| Personalia |
Photographs Presentations Videos |
Access Statistics |
 |
The content and infrastructure for these community service pages are provided by Roger Clarke through his consultancy company, Xamax. From the site's beginnings in August 1994 until February 2009, the infrastructure was provided by the Australian National University. During that time, the site accumulated close to 30 million hits. It passed 75 million in late 2024. Sponsored by the Gallery, Bunhybee Grasslands, the extended Clarke Family, Knights of the Spatchcock and their drummer |
Xamax Consultancy Pty Ltd ACN: 002 360 456 78 Sidaway St, Chapman ACT 2611 AUSTRALIA Tel: +61 2 6288 6916 |
Created: 13 June 2024 - Last Amended: 24 November 2024 by Roger Clarke - Site Last Verified: 15 February 2009
This document is at www.rogerclarke.com/EC/RGAI.html
Mail to Webmaster - © Xamax Consultancy Pty Ltd, 1995-2024 - Privacy Policy