Roger Clarke's Web-Site © Xamax Consultancy Pty Ltd,  1995-2024
HOME	eBusiness	Information Infrastructure	Dataveillance & Privacy	Identity Matters	Other Topics
What's New		Waltzing Matilda	Advanced Site-Search

Recognise, Evaluate and Address
The Downsides of GenAI

Notes for a Panel Session on 'Generative AI in Healthcare and Beyond'
at ACIS'24, Uni. Canberra, 4 December 2024,
with Doug Vogel, Lemai Nguyen, and Nilmini Wickramasinghe

Version of 1 December 2024

Roger Clarke **

© Xamax Consultancy Pty Ltd, 2024

This document is at http://rogerclarke.com/EC/GAIH.html

The accompanying slide-set is at http://rogerclarke.com/EC/GAIH.pdf

Notes to Support the Slide-Set

1. My role on this Panel is to make sure we keep in view the high likelihood of negative impacts and implications arising from the use of AI artefacts.
2. Analysis has to begin with a sufficiently deep grasp of the technology underlying GenAI artefacts and the internal processes that generate responses.
3. There are various ways in which GenAI gets used. As I talk, I invite you to think about a few particular use-cases that you're familiar with
4. It's important to distinguish different modes of use. Here are three modes, each of which embodies particular opportunities and risks.
5. These modes can be applied to the corpus of healthcare knowledge in various ways, and to specific episodes in a specific patient's life.
6. Here are some suggestions of particular use-cases that you may have in mind in your own work, or indeed in your experiences as a healthcare client.
7. In a current working paper, I've assembled information about 20 impactful features of GenAI, grouped under four headings.
8. A first group of problems derive from the corpus on which the GenAI artefact draws. A couple of ways of addressing the problems are suggested.
9. It's vital that users of GenAI for instrumentalist purposes appreciate two key aspects: (1) The linguistics underlying text analysis and text generation is the superficial. (2) People readily suspend their disbelief in the content's reliability when they see statements that are expressed clearly and eloquently.
10. Not only are the underlying syntactics weak, but the text analysis and synthesis embody no semantics or pragmatics at all. Humans associate text with real-world phenomena; but GenAI has no knowledge whatsoever of people, processes, places or things.
11. Generally speaking, current forms of GenAI not only do not, but cannot, provide logical explanations of any reasoning underlying generated text. And, generally, they cannot or do not provide linkage back to the source-materials, and they thereby preclude humans searching out a rationale.
12. Where professional standards, and the law, demand the ability to reconstruct what happened, current GenAI (and AI/ML) fail the test.
13. You'll have your own particular use-cases in mind. Diagnosis is a pretty rich process, so I've been using that to support an examination of GenAI's 20 'Potentially Harmful Attributes'.
14. The jury's getting closer to delivering its verdict. For mission-critical tasks, there are a lot of problems with GenAI as it's currently designed and used. Which use-cases do have in mind that aren't mission-critical.

Author Affiliations

Roger Clarke is Principal of Xamax Consultancy Pty Ltd, Canberra. He is also a Visiting Professorial Fellow associated with UNSW Law & Justice, and a Visiting Professor in the Research School of Computer Science at the Australian National University.

The content and infrastructure for these community service pages are provided by Roger Clarke through his consultancy company, Xamax. From the site's beginnings in August 1994 until February 2009, the infrastructure was provided by the Australian National University. During that time, the site accumulated close to 30 million hits. It passed 75 million in late 2024. Sponsored by the Gallery, Bunhybee Grasslands, the extended Clarke Family, Knights of the Spatchcock and their drummer

Xamax Consultancy Pty Ltd ACN: 002 360 456 78 Sidaway St, Chapman ACT 2611 AUSTRALIA Tel: +61 2 6288 6916

Created: 1 December 2024 - Last Amended: 1 December 2024 by Roger Clarke - Site Last Verified: 15 February 2009
This document is at www.rogerclarke.com/EC/GAIH.html
Mail to Webmaster   -    © Xamax Consultancy Pty Ltd, 1995-2024   -    Privacy Policy