Roger Clarke’s Dataveillance and Information Privacy Home-Page
© Xamax Consultancy Pty Ltd, 1995-2015
|Identity Matters||Other Topics|
This segment of the site provides access to papers that I've published in the broad area of privacy and dataveillance, since the 1970s. Dataveillance is the systematic use of personal data systems in the investigation or monitoring of the actions or communications of one or more persons. I coined the term in 1986, as a contraction of 'data surveillance', and published an analysis of the concept in Clarke (1988).
The word has come into increasingly wide use, both in the academic community, and more generally, and made it into the 2009 edition of Webster's New Millennium™ Dictionary of English, and the Oxford in March 2012. The OED entry identifies two earlier uses, which I was not aware of at the time. (Neither was anyone else, and they have no citations, whereas my 1988 article has over 400; so I feel justified in claiming the credit).
A substantial set of resources has been available on this site since 1995, attracting c. 16 million hits to the end of 2014.
The resources in this segment of the site can be accessed in the following ways, most useful first:
This page is the Topic-Based Index of my papers on dataveillance and privacy themes, up-to-date at 31 Dec 2014
The What's New Page (because the indexes are never fully up-to-date) is here
The Chronological Index of my papers on dataveillance and
privacy themes is here, up-to-date at 31 Dec 2014
There is also a chronological index of Presentations here
There is a Search Facility, at the top-right-hand side of this and other index-pages
Alternative Indexes are available, as follows:
See also the following recent papers:
The first cluster of papers establishes the theory of dataveillance. In part, these papers recapitulate knowledge developed by other people, but they also contain a lot of original contributions.
My early work, from the mid-1980s, was specifically in the then new field of Dataveillance (a term that I coined). I later extended my work into surveillance more generally, and into specific, new forms that have become rampant and that lacked (and still lack) adequate controls.
Surveillance inevitably threatens the human value of privacy. Privacy is a complex notion that is still poorly understood, so it's been necessary to publish a number of papers on the concept. Of greatest importance is the need to get beyond the narrow notion of 'data privacy' – which is all that 'data protection laws' address. The additional dimensions that I use are:
See the following papers:
Surveillance is the systematic investigation or monitoring of the actions or communications of one or more persons. An overview is provided in 'The Fundamentals of 'Surveillance' (s.2 only, 2007). See also A Framework for Surveillance Analysis (2009).
Specific forms of Surveillance are addressed in the following sections.
As regards the concept generally, see also:
Physical Surveillance depends on a human watcher and/or listener, perhaps using aural and/or visual aids such as cameras to transmit images to a remote location or directional microphones. See:
I coined 'dataveillance' in the mid-1980s. The purpose was to draw attention to the substantial shift that was occurring from (expensive) physical and electronic surveillance of individuals to (cheap) surveillance of people's behaviour through the increasingly intensive data trails that their behaviour was generating. I've created a dozen or so neologisms, but dataveillance is far and away the most successful of them, making it into dictionaries by 2009. See also Wordspy, 2001.
For introductory and summary material, see these papers:
The base for my extensive work in this area was laid in a paper entitled 'Information Technology and Dataveillance' (1988, published in a major US journal). Physical and electronically enhanced monitoring of individuals and groups is expensive. The paper shows how information technology is enabling those old techniques to be replaced by highly automated, and therefore much cheaper, systematic observation of data about people. This new form of monitoring, whose descriptor I abbreviated to 'dataveillance', is potentially highly privacy-invasive. I later reviewed progress, in Dataveillance - 15 Years On (2003).
A second paper, 'Human Identification in Information Systems: Management Challenges and Public Policy Issues' (1994, published in a leading international journal), further develops a vital aspect of the argument: the nature of human identification as it is applied within information systems. Remarkably, there are very few works in any academic literature which address the question of such uses of human identity, and for this reason the paper had a long gestation period (from 1985 until 1994).
The paper 'The Digital Persona and Its Application to Data Surveillance' (1994, published in a leading international journal) introduced a new concept, the 'digital persona', as a tool in the analysis of behaviour on the 'net. It applies the tool, together with data surveillance theory, to predict the monitoring of the 'real-life' behaviour of individuals and groups through their net behaviour. The concept of 'nym' is related to (but not identical to) what I mean by 'digital persona'. Nyms are addressed in the following section. Progress with the concept was reviewed in Promise Unfulfilled: The Digital Persona Concept, Two Decades Later (2014).
The following two are less pretentious than the above papers, but they contain some important ideas:
See also the following papers, and the substantial segments on Identity and Authentication.
Visual Surveillance goes beyond Physical Surveillance in that it involves the capture and/or projection of images and video, perhaps with synchronised audio. See:
This theme was introduced in Person-Location and Person-Tracking Technologies, initially for presentation at the 1999 conference of Privacy and Data Protection Commissioners, and reported on in The Melbourne Age in August 1999 . A revised version was published in a leading international journal. An important analysis was provided as an Addendum: Relevant Characteristics of Person-Location and Person-Tracking Technologies.
Further forays into this area included 'Mobile Technologies' and 'Privacy on the Move: The Impacts of Mobile Technologies on Consumers and Citizens' (May 2003), followed by 'Wireless Transmission and Mobile Technologies' (October 2003)
The ideas were pursued further in 2008, with a pair of papers on 'You Are Where You've Been: Location Technologies' Deep Privacy Impact' – YAWYB, Sydney and YAWYB, Canberra.
A joint paper with Katina Michael was published in 2012: Location and Tracking of Mobile Devices: Überveillance Stalks the Streets, and presented at Southampton Uni in 2014 as Location and Tracking of Mobile Devices.
The term 'überveillance' was coined by Michael & Michael in 2006. It can be used to refer to:
See the separate segment of this site on 'Identity Matters', including Biometrics.
See the separate segment of this site on national id schemes in Australia. This section contains only those papers that have general relevance to the topic:
See the separate segment of this site on Authentication.
I coined the term 'PIT' in late 1998. An explanation is in Introducing PITs and PETs: Technologies Affecting Privacy (March 2001). See also my PITs and PETs Resources Site (February 2001). The remainder of this section provides references to papers on particular technologies that adversely affect privacy.
An early paper, 'Database Retrieval Technology and Subject Access Principles' (1984; with Graham Greenleaf, published in the Australian Computer Journal, but not available on the web), is concerned with data retrieval technology. It examines the scope for a particular form of database technology to render impractical an established privacy-protective mechanism, the so-called 'subject access principle'. This case study exemplifies the way in which developments in information technology undermine privacy protection laws. It provides a basis for understanding the impact of other developments, such as 'reverse access' to telephone directories, monitoring of energy usage, textual analysis, 'data mining', and the discovery of individual characteristics through the analysis of seemingly anonymous, statistical collections.
Field work, undertaken in both the United States and Australia between 1987 and 1992, resulted in a series of papers on Computer Matching:
The paper 'Trails in the Sand' (1996) highlights the wide range of transaction trails that people leave behind them.
Another technique, Profiling, was examined in 'Profiling: A Hidden Challenge to the Regulation of Data Surveillance' (1993). A later paper examines 'Customer Profiling and Privacy: Implications for the Finance Industry' (1997).
A comprehensive paper examines Direct Marketing (February 1998), including mail, fax, outbound tele(phone)-marketing, and Internet marketing. Related topics that have been addressed include spam, cookies and emailing lists.
Another area of importance has been public key infrastructure (PKI) to support digital signatures. A long series of papers published in that area is indexed here.
A separate segment of this site lists c. 50 papers from 1996 to 2001 on ePrivacy topics.
I've published several score more since then, including:
Search here using terms that describe the specific topic you're interested in (e.g. <"social networking">,< Google privacy>, <P2P>, <Web 2.0>, <mobile privacy>, <"deep packet inspection">, <ENUM>, <Gen-Y>, <"social networking">, <"social media">).
This concept dates to 1995. My key papers in this area are:
An early series of papers addresses security aspects:
updated in 2012:
A series of papers has considered technologies whose purpose is to directly support privacy. This work is primarily in:
I was active in the original W3C Working Group on P3P (Platform for Privacy Preferences). Unfortunately, the initiative fell so far short of its aspirations that it is not worthy of the name 'PET'. All of the following appeared in Privacy Law & Policy Reporter:
In the 1970s, government agencies and corporations resisted calls for privacy protections. During the intervening decades, many of them have come to recognise privacy as a factor that can harm their business, and that therefore needs to be addressed in a positive manner.
I have performed many consultancy assignments in this area during the last two decades. This section identifies published papers that are addressed specifically to business enterprises and government agencies, firstly in matters of general strategic significance, and secondly in the specific area of privacy impact assessments (PIAs).
The primary papers I've published on privacy as a strategic factor for corporations and government agencies are as follows:
Guidance on specific matters of a strategic nature is provided in:
I've performed a considerable number of consultancies in this area (see client list). Important among them have been the preparation of guidelines for the performance of PIAs (1998), the review of guidelines prepared by government agencies (1999, 2006), research reports on PIA laws, policies and practices in the Asia-Pacific (2007), lead-authorship in the drafting of the UK Information Commissioner's PIA Handbook (2007), and training materials for the Hong Kong Privacy Commissioner's staff (2010).
My published papers in the area are:
Clarke R. (1998) 'Privacy Impact Assessment Guidelines' Xamax Consultancy Pty Ltd, February 1998, at http://www.xamax.com.au/DV/PIA.html (long form), http://www.rogerclarke.com/DV/PIA.html (short form)
Clarke R. (2004) 'A History of Privacy Impact Assessments' Xamax Consultancy Pty Ltd, February 2004
Clarke R. (2004) 'The Search for Balance: The Past, Present and Future of Privacy Impact Assessments' Presentation, Queens University, Kingston ON, 9 June 2004
Clarke R. (2004) 'Identity Management; and PIAs' Presentation, Office of the Privacy Commissioner of Canada, 11 June 2004
Clarke R. (2005) Submission
re the Commonwealth Privacy Commissioner's PIA Guidelines Australian
Clarke R. (2005) Submission re the Victorian Privacy Commissioner's PIA Guidelines Australian Privacy Foundation, February 2005
Clarke R. (2008) 'Privacy Impact Assessment in Australian Contexts' Murdoch eLaw Journal 15, 1 (June 2008), Preprint at http://www.rogerclarke.com/DV/PIAAust.html
Clarke R. (2009) 'Privacy Impact Assessment: Its Origins and Development' Computer Law & Security Review 25, 2 (April 2009) 123-135, PrePrint at http://www.rogerclarke.com/DV/PIAHist-08.html
Warren A., Bayley R., Bennett C., Charlesworth A.J., Clarke R. & Oppenheim C. (2008) 'Privacy Impact Assessments: International experience as a basis for UK Guidance' Computer Law & Security Report 24, 3 (April-June 2008) 233-242
Warren A., Bayley R., Bennett C., Charlesworth A.J., Clarke R. & Oppenheim C. (2009) 'Privacy Impact Assessments: The UK Experience' Proc. 31st Int'l Conf. of Data Protection and Privacy Commissioners, Madrid, September 2009
Clarke R. (2010) 'PIAs in Australia - A Work-in-Progress Report' Review Draft of a chapter for Wright D. (ed.) (2011) 'Privacy Impact Assessments: Engaging stakeholders in protecting privacy' 30 September 2010, Xamax Consultancy Pty Ltd, PrePrint at http://www.rogerclarke.com/DV/PIAsAust-11.html
Clarke R. (2011) 'An Evaluation of Privacy Impact Assessment Guidance Documents' International Data Privacy Law 1, 2 (March 2011), PrePrint at http://www.rogerclarke.com/DV/PIAG-Eval.html
APF (2013) Meta-Principles for Privacy Protection', Australian Privacy Foundation, April 2013
APF (2013) 'Privacy Impact Assessments', Australian Privacy Foundation, April 2013
Clarke R. (2014) 'Approaches to Impact Assessment' Notes for a Panel Presentation at CPDP'14, Brussels, 22 January 2014
Consultancy Assignment, as lead-author:
ICO (2007a) 'Privacy Impact Assessments: International Study of their Application and Effects' UK Information Commissioner's Office, 2007 (lead-author), and sole author of several of the Appendices:
ICO (2007b) 'Privacy
Impact Assessment Handbook' UK Information Commissioner's Office, 2007
That is v.2 (Jun 2009). Here is an archive copy of v.1 (Nov 2007 – the version for which I was lead-author)
It is widely claimed that information technology is becoming pervasive, and is giving rise to an 'information economy' and an 'information society'. If that is the case, then its impacts will be substantial, and must be managed. This part of my collection of papers is concerned with public policy regarding dataveillance.
This section identifies clusters of papers on specific issues not elsewhere addressed in this index.
I identified what I then saw as the Five Most Vital Privacy Issues, (July 1997), for the Montreal Daily, Le Devoir, prior to the International Conference on Privacy, Montreal. A resource-page was maintained for a few years, 1996-2006, citing surveys of public attitudes to privacy issues.
A paper published in a major U.S. journal argued that Internet Privacy Concerns Confirm the Case for Intervention (1999), and that that last renegade among advanced western nations would find it necessary to establish a statutory framework for privacy protections in the private sector. This paper also specified the requirements of a genuinely 'co-regulatory' approach.
The serious inadequacies in the 'Fair Information Practices' model of privacy protection were stated in Beyond the OECD Guidelines: Privacy Protection for the 21st Century (Jan 2000), which identified the many additional steps urgently necessary if a collapse in consumer and citizen confidence were to be avoided.
Many of my papers pursue positive arguments about what specific policy measures should look like. The most important of these are:
A paper on the OECD's Data Protection Guidelines (1989) provides a template for evaluating laws, and proposals for laws, and has been applied in a number of countries.
Many of my papers examine the inadequacies of the 'official response' by Parliaments and Governments, which involves providing mere 'fair information practices' protections. The intellectual basis underlying thse arguments is in 'Beyond the OECD Guidelines: Privacy Protection for the 21st Century' (January 2000).
The following comments were made on the dreadful Privacy Amendment (Private Sector) Act 2000. It passed into law in December 2000, and became [in]effective on 21 December 2001:
A specific matter of concern is public key infrastructure (PKI), and the use of digital signatures for identity authentication. A long list of papers is indexed above. Two that are particularly oriented towards the policy aspects are 'eAuthentication: Where's the Public Interest?' (2003), and Identity Management; and PIAs (2004).
Another concern is 'ENUM' (March 2003).
Yet another is the mindless enthusiasm for biometrics that has arisen as a result of the dominance of 'national security' over both civil liberties and logic, aided and abetted by fraudulent representations by technology providers. A specific critique is at:
Another criticism was of the fatuous 'data breach notification' laws, a placebo that was still being discussed in Australia a full decade after such a law might have been of some value:
Papers between 1996 and 1999 were:
Older papers that are now of primarily historical interest are as follows:
Organisational self-regulation without a layer of statutory regulation underpinning it is not a form of privacy protection, but a mere pretence. This is addressed in a number of places, including:
A major paper provides guidance to corporations and government agencies concerning 'Privacy and Dataveillance; and Organisational Strategy' (1996, a keynote presentation at an Australian Conference).
Observations concerning a particular industry code are provided in 'Smart move by the smart card industry: The Smart Card Industry's Code of Conduct - Part I' (1996) and Part II (1997). Both were published in the Privacy Law & Policy Reporter.
A paper entitled 'Economic, Legal and Social Implications of Information Technology' (1988, published as an 'Issues and Opinions' piece in a leading US journal) provides a discussion of the ethics of academic endeavour in the information systems discipline. It argues that information technology's impacts are so great that detached observation is an inadequate stance for an information systems academic.
Information systems researchers must engage themselves in their subject-matter, and extend themselves beyond mere description and explanation, and even beyond the prediction of the outcomes of artefact design and interventions in organisations and society. Information systems researchers are irretrievably involved in the process of engineering organisations and society, and cannot meaningfully sustain the pretext that they are entirely uninterested in, and unaffected by, the processes around them. These issues are examined in 'Data Surveillance: Theory, Practice and Policy' (July 1997). That paper argues that policy issues in general, and information privacy in particular, are not only an appropriate area of focus for information systems researchers, but that they are also capable being approached in a sufficiently disciplined manner.
Here is the Australian Computer Society's ELSIC Committee, which I chaired 1984-95, and of which I continue to be an active member.
An outline is provided of 'Information Systems Audit & Information Privacy' (1997), and resources are provided for Privacy Impact Assessments (1998) and (2004).
Other papers on this topic include:
Self-regulation by research professional in the I.S. discipline is considered in considered in 'Information Privacy in a Globally Networked Society: Implications for Research', plus slide-set (December 2002). The contributions to the panel session were later published as 'Information Privacy in a Globally Networked Society: Implications for IS Research' (October 2003).
A further paper comments on the social responsibility of engineers, plus PowerPoint slides (2002).
Like other technologies, computing and telecommunications are capable of being applied to the benefit of humanity as a whole, or of particular interest groups within society. Use of information technology by the politically powerful as a means of exercising control over the thoughts and actions of members of the public, is a matter of especial concern to those living in democracies.
A paper entitled 'Information Technology: Weapon of Authoritarianism or Tool of Democracy?' (1994; presented at the World Congress of computing academics) identifies critical implications of information technology for democracies. It represented a response to a paper submitted to the conference by a senior government executive of a country that had previously been dominated by the U.S.S.R., and that has no tradition of democracy as it is known in 'western' countries.
This paper's importance is that it lifts the application of the theory of dataveillance from the individual and social levels to the political level, and is a first, tentative step toward the building of a bridge between the theory of dataveillance (developed, as it has been, largely from within the information systems discipline), towards broader theories arising in anthropology, sociology and political science.
Other papers that consider broader issues related to dataveillance include:
This section identifies papers that deal with matters specific to Australia.
Many years ago, I hoped to use this sub-section to maintain a list of current issues. But because privacy-invasive technologies and applications are so rampant, it's extremely difficult to keep up ...
I drafted a summary paper – a history of privacy in Australia (December 1998). And I started a running document on current developments, with this update.
But the most effective way to follow current issues is to subscribe to the privacy list.
Papers on National Identification schemes generally are in a separate section, above. This section addresses the specifics of the ongoing attempts by executives in Australian government agencies and Australian politicians to implement extremist social control mechanisms in this country.
During the period 1985-87, the Commonwealth Government developed a proposal to implement a central database of the Australian population, whose purpose, expressed in terms of the theory developed in this body of work, was the facilitation of dataveillance of all residents of this country. 'Just Another Piece of Plastic for Your Wallet: The Australia Card' (1987, published in an international technology policy journal) provides a carefully documented description and analysis of the proposal, a distillation of the issues, and a political history of the proposal's development and ultimate fate.
Another paper on the topic was 'National Identification Scheme - Costs and Benefits' (1986, published in an Australian journal).
Although the Australia Card proposal was withdrawn in the face of dramatically negative public opinion, the momentum that dataveillance applications of information technology had attained within the Commonwealth public sector was scarcely affected. 'The Resistible Rise of the National Personal Data System' (1992, published in an American journal of computers and law) documents a number of developments during the following three years. It is primarily a political history, expressed within the context set by the theory of dataveillance.
The untrustworthiness of a mere Prime Minister's assurances are underlined in 'The Tax File Number Scheme: A Case Study of Political Assurances and Function Creep' (1991, published in an Australian journal).
An attempt by law enforcement interests to create a highly privacy-intrusive scheme is documented in 'LEAN Times Ahead: The Proposed Law Enforcement Access Network' (1992).
The reasons for public concerns about government agencies are summarised in 'Why the Public Is Scared of the Public Sector' (1993, presented at an Australian conference).
The justification for a major computer matching scheme is shown to be incompetent or fraudulent, in 'Matches Played Under Rafferty's Rules: The Parallel Data Matching Program Is Not Only Privacy-Invasive But Economically Unjustifiable As Well' (1993, published in Privacy Law & Policy Reporter).
Echoes recur continually. In April 1999, the Commonwealth Auditor-General, Pat Barrett, sought to revive the process of extending the Tax File Number, this time into the Medicare arena (ANAO 1999). His proposal was then considered by the House of Representatives Standing Committee on Economics, Finance and Public Administration (EFPA). Too pressed for time to provide yet another time-wasting submission to yet another government committee, I submitted a brief letter (November 1999).
During the 1990s, bureaucrats achieved merger of all government benefits schemes, and correlation of identifiers, by means of Centrelink. Despite resistance going back 15 years, they eventually succeeded in combining the health insurance and pharmaceutical benefits schemes in to a single agency. Then in the early 2000s, the centralist's dream of a 'super-ministry' was achieved, euphemistically called Human Services, which enable the benefit schemes, health systems and child support agency to be drawn even closer together.
Meanwhile, a succession of attempts were made to upgrade drivers' licences into a much broader social control scheme. See Submission to Queensland Transport in relation to the Proposed Smartcard-Based Driver's Licence (2003).
Then in mid-2005, the Queensland Premier Beattie, in an endeavour to shift media attention away from serious problems in his State, suggested that an Australia Card was needed. The Prime Minister ran with it. A cluster of identity schemes was progressively drawn into one being developed by the 'Human Services' Minister dubbed the Access Card.
Advocacy groups and the media have caused a great deal of pertrubation in the statements made by politicians and bureaucrats about the scheme. The resolve with which it is being pursued by the Government appears set to bring down another Minister in due course.
Documentation about the schemes, the continual changes in Government pronouncements, and the privacy advocacy campaigns, is at:
Papers on the topic during this period include the following:
This series of attempts by the bureaucracy supported by extremist elements behind and within the Liberal Party, failed. (Even the reactionary Attorney-General Phillip Ruddock, didn't have the stomach for it). An opportunist Minister, Joe Hockey, offered to dress the proposal up differently, leading to ...
The story is told in this page on the Dept of Human Services' So-Called 'Access Card' (2006-07).
It collapsed. The APF had ensured that the media were well-informed. But, significantly, the proximate cause of the collapse was that rarity, a Senate Committee Chair (i.e. a Government member, not yet a Minister, but on the rise), whose Report comprehensively debunked it. (The Chair was Queensland Liberal Senator Brett Mason, whose PhD thesis and book had been highly uncomplimentary about privacy advocates).
Parallel State Government schemes included:
Presentations on the topic during this period include:
The Privacy Act, passed in December 1988, was carefully analysed in the following papers, which were distributed among a small community of interested researchers and government agencies:
A paper on 'Consumer Credit Reporting and Information Privacy Regulation', summarised the situation in the lead-up to the 1989 extensions to the Privacy Act (1989). The resultant legislation was described in 'Privacy Regulation of Consumer Credit Reporting' (June 1989, published in an Australian journal, not available on the web).
The disastrously bad amendment Bill of 2000 was analysed and severely criticised:
It came into force on 21 December 2001, exacerbating rather than solving problems. For subsequent analyses, see:
This section is provided primarily as a matter of record, most recent first.
During 1989-90, a series of papers was prepared under consultancy assignments for the Commonwealth Privacy Commissioner, on:
During the period 1985-1990, several Policy Papers were prepared for the Australian Computer Society, on:
The situation in health care is discussed in 'Current Health Care Information Privacy Issues' (1990, invited paper at an Australian conference).
A series of assessments of earlier proposals for privacy regulation was published:
During 1978-80, a series of papers were prepared for Michael Kirby(later a High Court judge), in relation to the design of the OECD Guidelines.
During 1976-77, a series of papers was prepared while I was a Research Officer the N.S.W. Privacy Committee. This included:
During 1972-75, several papers were prepared on behalf of the N.S.W. Branch of the Australian Computer Society. At that time, it appeared that the N.S.W. Government might push through privacy protection legislation that could have been harmful to the then immature computer industry. I led the professional association's lobbying against unreasonable regulation. I was too successful: N.S.W. passed no substantive law until 1998, and then came up with what was until December 2000 the world's worst privacy protection legislation ...
The content and infrastructure for these community service pages are provided by Roger Clarke through his consultancy company, Xamax.
From the site's beginnings in August 1994 until February 2009, the infrastructure was provided by the Australian National University. During that time, the site accumulated close to 30 million hits. It passed 40 million by the end of 2012.
Sponsored by Bunhybee Grasslands, the extended Clarke Family, Knights of the Spatchcock and their drummer
Xamax Consultancy Pty Ltd
ACN: 002 360 456
78 Sidaway St, Chapman ACT 2611 AUSTRALIA
Tel: +61 2 6288 6916
Created: 15 February 1995 - Last Amended: 1 January 2015 by Roger Clarke - Site Last Verified: 15 February 2009
This document is at www.rogerclarke.com/DV/index.html