New Millenium or New Dark Ages?

New Millenium or New Dark Ages?

Roger Clarke

Principal, Xamax Consultancy Pty Ltd, Canberra

Visiting Fellow, Department of Computer Science, Australian National University

Version of 29 November 1999

© Xamax Consultancy Pty Ltd, 1999

This paper was prepared for a keynote presentation at the Australasian Document Computing Symposium (ADCS'99), Coffs Harbour, 3 December 1999

This document is at

The slides used in the presentation are in a 180KB PowerPoint 4 file at


At the end of the twentieth century, documents of all kinds are available to an extent, with a degree of convenience, and for costs so low, that our forebears would be unable to believe our good fortune.

There's a common presumption that the Internet is 'bringing in the millenium', ensuring that we achieve and sustain openness, the end of inequities in the distribution of information, democracy, and human self-fulfilment.

Any such conclusion would be premature. The digital era has ambushed and beguiled us all. Its first-order impacts are being assimilated, but its second-order implications are not. Powerful institutions perceive their interests to be severely threatened by the last decade of technological change and by the shape of the 'information economy' and 'information society'. During the coming decade, we will see a fightback by those institutions, who will implement technological countermeasures, and demand and gain changes to the law.


1. Introduction

The business of electronic publishing, and the document computing technologies that underlie it, offer enormous benefits to society and the economy. Those benefits derive from enhancements to the freedom of individuals to access information. But those same technologies represent a dramatic threat to established players, and they are using their power to protect their interests.

This paper commences with a discussion of the nature of 'freedom of information' in the new context. It then assesses the scope for governments and major corporations to sustain their power, by stifling new business entrants, and denying the public access to information.

2. The Economics and Politics of FOI

The digital era evidences a distinctly different political economy from the now-defunct industrial age. In order to investigate those differences, a brief review is needed of the scene immediately prior to the revolution.

2.1 The Political Playing-Field

There are various justifications for access to information, including social, psychological, democratic, law and order and economic motivations. There is also a wide range of justifications for the denial of access to information. These include many narrow, sectional interests, but also some of broader concern, such as privacy, and the assurance of some degree of order in the processes of economic development, and of government.

The exercise of political choice is primarily dependent on inter-play between institutions that have political power, and that exercise it. Historical accident also plays a part (in particular, ambiguous wording in statutes and prior judgements, and new judicial inferences about their meaning).

FOI has to date been focussed mainly on government. The private sector has successfully avoided being subjected to the same rigours, partly through the exercise of economic power, and partly because of the economic rationalist philosophies that pervade political parties, which dictate that, even in advanced nations, economic concerns dominate social ones, and that, as a result, freedom for business enterprises is a higher ideal than freedoms for people.

During the two decades since the 1978 legislation affecting some of the Commonwealth public sector, there have been few accretions to the freedom side of FOI. In the States and Territories, small advances have arisen, usually in response to public scandals, but, in most jurisdictions, the granting of meaningful access to government data has been successfully avoided.

One area in which an increase in accessibility has occurred has been the progressive acceptance that environmental impact statements (EIS) are a necessary feature of major infrastructure proposals, and that public information, public consultation, and public involvement in design are elements of the EIS process. Regrettably, that development has not been extended to social impact statements for major initiatives,. In some countries, however, notably New Zealand and Canada, privacy impact assessments (PIAs) for major applications of information technology are entering the mainstream, and this is likely in Australia in the near future (Clarke 1997b).

There have, on the other hand, been plenty of extensions to existing abilities to deny access. Government agencies have a litany of excuses available, in the form of exemptions with wide applicability. Moreover, there are increasingly frequent refusals by agencies and Ministers to provide information to the Parliament and Parliamentary Committees, and Auditors-General are currently submitting to a trend towards gentler audits and audit reports. There has been some degree of tightening in relation to access to personal data, following passage of the Privacy Act 1988, although this primarily limits access by people, rather than access by governments agencies and corporations.

Individual people, lacking a firm power-base, have on the other hand been subject to ever-increasing demands that they provide personal information to government agencies. These demands have been coupled with economic disincentives against non-compliance, and, with occasional exceptions, the public has acceded to the power of governments.

The net effect is that personal data has been becoming increasingly open, information held by corporations remains largely protected, and information held by governments is largely protected, but subject to some limited access provisions.

2.2 The Political Economy of the Digital Era

Producers of all kinds of information have tended to use fairly elaborate 'production-lines' or 'value-chains', with successive individuals and organisations 'adding value' to a base product. Digital technologies have lowered the costs involved in those production processes, by rendering some steps unnecessary, by enabling people to perform other steps with less training or cheaper tools, and by providing cheap and quick transmission from one step to the next. The delays between the origination of data objects and their availability to users have been greatly reduced.

Digital technologies have quickly proven to be a double-edged sword, however. In addition to lowering the costs of production for authorised publishers, they also assisted the replication of data objects by parties who were not acting within the terms of a licence issued by the intellectual property owner. Unauthorised copying, adaptation and use are rife.

The context within which data objects exist has changed very quickly, and the courts are struggling to understand and to apply intellectual property law in the new context. This is considered in greater depth at Clarke & Dempsey (1999). If the controls that have hitherto kept the lid on widespread appropriation of copyright-objects were to cease to be effective, so the argument goes, the haemorrhage of revenue would remove the economic incentive to originate copyright-objects and to publish them.

Debates are currently raging about the shape of the new economics. See for example Lamberton (1971, 1996), Dyson (1995), Romer (with a populist description in Kelly 1996), Clarke (1994), Clarke (1999a), Clarke (1999c) and Shapiro & Varian (1999). These accept as given that the user-pays business model that has been common during the industrial age has been undermined by technology, and that alternative business models need to be applied.

The following aspects convey the extent of the changes involved:

3. The Dark Side of the Information Revolution

A reading of the preceding analysis might suggest that the changes that have been taking place are predominantly positive, in many ways, and especially for freedom of information. Certainly, that's the aspect of the digital era that most commentators focus upon.

This section identifies second-order effects that are too easily overlooked. They are so substantial that they threaten to undermine the conventional, pleasant expectations that the digital revolution will 'bring in the millennium'.

3.1 Quality Issues

Information quality is a term for the cluster of characteristics that 'good' information should have. Most important among them are accuracy, timeliness and completeness. Also significant are the provision of evidence of the sources of data which a work refers to, and identification of the author, date and location of publication, in order to enable auditability.

Within the publishing industries and the various information professions, sets of conventions have developed over the decades which provide some degree of assurance in relation to information quality. At the professional end of the data-production business, ranging from entertainment, via sport and news reporting, to statistics and reference information, longstanding business models have been undermined, and alternative business models have not been emerging rapidly enough.

A particularly poignant example of the rapidity with which the digital revolution has undermined a hitherto financially and culturally valuable business is the story of the latest (and, possibly, the last) decade of Encyclopaedia Britannica (EB).

In 1991, the company sold about 400,000 printed sets, and in 1997 about 10,000. (Tellingly, my source for this information is a quotation from the Managing Director of EB International, only available to subscribers to a for-fee service, E-Commerce Today). The collapse was triggered by the success of Microsoft Encarta and other CD-ROM versions of lower-quality but approximately equivalent collections sold in a convenient and inexpensive form. Since then, web-based information services have mushroomed. Despite its brand reputation, and the apparent quality and presumed value of the content the company owned, and even after scrambling to survive, revenue has halved, losses have accumulated, the company has changed hands several times, and survival remains uncertain (Rayport & Gerace 1997, Evans & Wurster 1997, Melcher 1997, Downes & Mui 1998, p.51, Shapiro & Varian 1999, pp. 19-21, 26).

A great deal of the information that has become available on the Internet is a result of amateurs playing author, editor, publisher and marketer, and in some cases doing them in a manner significantly less professional than was achieved by conventional publishers. Longstanding quality assurance conventions are only partly understood, and only partly respected, by the flood of new information-providers who have appeared on the Internet.

Other factors of concern are the discoverability of information, the identification of versions, and the archival of old information. A great many links are ephemeral, and with their demise disappear not only information, but also the ability to audit the claims of authors who refer to them.

Moreover, there remains a great deal of detail that is obscured or even falsified, resulting in what economists refer to as 'information asymmetries', i.e. inequality of information among the participants in decision-making processes, and hence imbalance of bargaining power, and inequities.

3.2 The Counter-Reformation

Content-providers who depended on their customers paying full value for access to it are seeing their business model undermined by the digital revolution: "there's likely to be a long, lean period for anyone trying to sell information. The problem is that there are so many reasonable and free options that are good enough" (Doyle B., quoted in Melcher 1997).

Rationally, corporations that are dependent for their survival on the exploitation of rights to information are taking action to defend their positions. The risk is that these defensive stances may be economically and culturally very harmful. This is highly reminiscent of the way in which the cyberpunk sci-fi novelists (e.g. Gibson 1984, Sterling 1988, Sterling 1989, Stephenson 1992) envision the 'hypercorps' retiring into enclaves, and dealing only with the identified and continuously monitored members of respectable, official society.

Two ways in which content-owners are battening down the hatches are through legal measures, and through the development and application of information technologies.

(1) Legal Protectionism

The following are key aspects of the argument advanced in Clarke & Dempsey (1999).

* Accidental Extension of Copyright-Owner Power

An important change in the effect of copyright law accompanied digitisation, and became even more apparent with the explosion of the Internet. The purchase of a book, or tuning to a broadcast radio or television channel, did not necessitate the acquisition of a copyright licence. The purchase or rental of digital media, on the other hand, generally does. Moreover, a workstation's mode of operation inherently involves the making of a succession of copies of the object, in memory and on the screen.

The move from atoms to bits has therefore resulted in an accidental extension to the legal rights that copyright owners enjoy: there has never previously been any right to preclude people from accessing data-objects, whether to read them, listen to them, look at them, or watch them. The new need for the consumer to have a licence has accidentally strengthened the hand of the copyright-owner.

* Enhanced Legal Protections for Information

One of the main weapons in the armoury of powerful organisations is the ability to arrange for favourable action in legislatures. For example, a draft Commonwealth Government Bill, the Copyright Amendment (Digital Agenda) Bill 1999, follows the lead of other governments in acceding to the demands of powerful corporations in relation to compilations such as databases, and to technologies which circumvent copyright protections.

A further, recent move has been the suppression of information about errors and security risks in software products: "Information derived from decompilation of a program ... cannot be used or communicated to others for any other purpose" (Ministerial Press Release relating to the Copyright Amendment (Computer Programs) Bill 1999, enacted 13 August 1999).

The major publishing interests have been able to significantly extend the scope of copyright, and hence protect their revenues, profits and empires. Moreover, they have arranged for activities that offend large copyright-ownership interests to be dealt with by the criminal courts (at public cost) rather than the civil courts (at their own cost).

There are also signs that both trade secrets law and the law of confidence may be in the process being developed in directions that provide yet greater protections for the interests of large corporations. This may extend even to the point of protecting ideas rather than expressions, and beyond the protection of data objects to, for example, preclude individuals from applying their accumulated, abstracted knowledge.

* Shift From Copyright to Contract

A further manoeuvre whereby copyright-objects can be protected against abuse is to rely less on copyright law and more on direct contractual arrangements between the publisher and the would-be reader. This would have the effect of undermining libraries, and restricting not only reproduction, but also access to the object.

Instances already exist, such as expensive hard-copy reports that are sold on the express condition that they cannot be lent to any other person, and databases that are remotely accessible by subscription.

* Threats to Statutory Licensing

Educational institutions have a statutory licence under the Copyright Act. They could establish digital collections of copyright-objects, irrespective of who they are owned by, and allow students and staff to make digital and printed copies from those collections. But the Act makes the licence subject to an equitable payment to the copyright-owner.

Photocopying was the subject of long negotiation between representatives of copyright owners and universities. During the last five years, the establishment of 'electronic reserves' of materials frequently accessed by students has been stymied by an ongoing battle between those same parties as to what 'equitable' means. This impasse represents a grave threat to the performance of the nation in the information era.

* Threats to Equitable Public Access

There are many different circumstances in which people access information. For example, some people do so as consumers, whereas the intention of other accesses is to use the data as a 'factor of production', in order to produce more information. Some accessors are disadvantaged, due to such factors as physical impairment, or the locality in which they live. Others enjoy privileges of various kinds.

The exercise of power that the copyright-owner gains from technological innovations and legislative amendments represents a very substantial negative effect on equitable public access to information. This is further discussed at (Clarke 1999a).

* Threats to Anonymous and Pseudonymous Access

Historically, a great deal of access to published works has been anonymous, in the form of purchase of books, access to books in libraries and viewing of films in theatres; or pseudonymous, e.g. the borrowing of books from libraries, or the hiring of a video. A relatively small proportion of access has been associated with an authenticated identity; and that has generally been where the material was being adapted or incorporated into another work.

This lack of identification is very important, because it sustains an environment in which information is generally accessible without fear of recrimination from authority-roles such as employers, competitors, teachers, parents and powerful vested interests. Any shift from copyright towards contract as the primary basis risks bringing with it a switch from predominantly anonymous to predominantly identified transactions; and with that would come a serious 'chilling effect' on information availability, and a direct threat to open society and democracy.

(2) Technological Protections

Powerful organisations have a variety of interests in preventing the exposure of data they control. They seek to protect information that would disclose such things as their corporate strategy and the directions of their technology investments; the nature of micro-monopolies from which they extract super-profits; and the details of their financial and taxation arrangements.

* Technologies That Protect Data-Objects

Many corporations also wish to exploit data under their control, and in order to do so seek out technologies that protect data objects. Some of these technologies are passive in nature, such as:

Others are active technologies, such as:

These are discussed in greater detail at (Clarke & Dempsey (1999) and in Clarke & Nees (1999).

The effect of such technologies is to preclude information access. In many cases, this may not change the current balances, but in some circumstances, it is likely to reduce the accessibility of information. For example, a publisher could combine:

* Technologies That Entrench Data-Object Protection

It's feasible that the Internet Protocol Suite could be adapted in a manner that favours the protection of data objects, and thereby undermines freedom of information. For example, it could facilitate existing techniques that assist corporation-owned servers to exercise control over personal workstations; and there has been a recent tendency towards the siphoning off of Internet bandwidth in order to support 'virtual private networks' (VPNs).

Alternatively, the open, public Internet could be circumvented by the emergence of a new architecture, developed and promulgated by an alliance of corporations and governments, and much more suited to their needs rather than that of consumers and citizens (and probably endowed with an insidious title such as Internet 3).

* Technologies That Protect Originators and Users

A variety of tools is available, many based on cryptographic methods, which enable people to protect their identity. This can be achieved either by denying it entirely, or by substituting it with a pseudonym and protecting the linkage between real and pseudo-identity through technical, organisational and legal measures (Clarke 1998, Clarke 1999b).

The focus during the last decade or so has been on the use of such tools by individuals, in order to send messages and distribute documents whose originator cannot be traced (e.g. using 'anonymous remailers'). There has been speculation, however, that the largest uses of these tools may be by governments, and by businesses dealing in particular kinds of data-objects, such as pornography.

Similar tools can also be used to access documents without leaving an identified trail as to who has viewed which documents (e.g. so-called 'anonymous web-surfing'). Of course, such tools are valuable not only to individuals, but also to governments and businesses conducting covert operations of various kinds.

4. Conclusions

The implications of these developments is that governments and corporations are in a position to enhance the restrictions on access to information. The golden era of information accessibility is under threat, because governments have successfully resisted FOI and now have additional weapons available to them, and major corporations are wielding their power to protect their own interests.

In the present information era, skirmishes around the edges of existing FOI laws are irrelevant. If freedom of information is to be sustained, let alone increased, then measures are needed now. Arguments against legal protectionism need to be advanced much more energetically, countervailing power needs to be mobilised against corporate and governmental interests, and information and networking technologies need to be carefully designed, to avoid protectionism becoming entrenched within the information infrastructure.

A serious battle is in train, with corporations manipulating governments, netheads wailing, the public non-aware, and FOI specialists largely non-comprehending. FOI activists can stay asleep and become even less relevant, or can inform themselves and become involved.


This presentation draws heavily on an invited paper presented at the Conference on 'Freedom of Information and the Right to Know', in Melbourne, on 19-20 August 1999. A revised version was published in First Monday, 4, 11 (November 1999).


Aucsmith D. (Ed. (1998) 'Information Hiding' Proc. Second Int'l Workshop, Lecture Notes in Computer Science Volume 1525, Springer-Verlag, 1998

Barlow J.P. (1994) 'The Economy of Ideas', Wired 2.03 (March 1994), at

Blackmore S. (1999) 'The Meme Machine', Oxford Uni. Press, 1999, from

Brin D. (1998) 'The Transparent Society' Addison-Wesley, 1998

Clarke R. (1992a) 'Fundamentals of 'Information Systems', at

Clarke R. (1992b) 'Knowledge', at

Clarke R. (1994) 'Electronic Support for Research Practice: The Inadequacy of Economic Analysis in a Time of Revolutionary Change', The Information Society 10,1 (March 1994), at

Clarke R. (1996) 'CyberCulture: Towards the Analysis That Internet Participants Need', March 1996, at

Clarke R. (1997a) 'Electronic Publishing: A Specialised Form of Electronic Commerce', Proc. 10th International Electronic Commerce Conf., Bled, Slovenia, June 1997 , at

Clarke R. (1997b) 'Privacy Impact Assessments', September 1997, at

Clarke R. (1998) 'Information Privacy On the Internet: Cyberspace Invades Personal Space', Telecommunication Journal of Australia 48, 2 (May/June 1998), at

Clarke R. (1999a) 'Key Issues in Electronic Commerce and Electronic Publishing', Proc. Conf. Information Online and On Disc 99, Sydney, 19 - 21 January 1999, at

Clarke R. (1999b) 'The Legal Context of Privacy-Enhancing and Privacy-Sympathetic Technologies', April 1999, at

Clarke R. (1999c) 'The Willingness of Net-Consumers to Pay: A Lack-of-Progress Report', Proc. 12th Electronic Commerce Conf., Bled, Slovenia, 8-9 June 1999, at

Clarke R. (1999d) 'Electronic Services Delivery: From Brochure-Ware to Entry Points', Proc. 12th Electronic Commerce Conf., Bled, Slovenia, 8-9 June 1999, at

Clarke R. (1999e) 'Information Wants To Be Free', August 1999, at

Clarke R. & Dempsey G. (1999) 'Electronic Trading in Copyright Objects and Its Implications for Universities', Proc. Australian EDUCAUSE'99 Conf, Sydney, 18-21 April 1999, at

Clarke R. & Nees S. (1999) 'Technological Protections for Digital Copyright Objects', 14 November 1999, at

Dawkins R. (1976) 'The Selfish Gene' Oxford Uni. Press, 1976, from

Dempsey G. (1998) 'Knowledge and Innovation in Intellectual Property: The Case of Computer Program Copyright' Unpublished PhD Thesis, Australian National University, March 1998, c/- Dr Gillian Dempsey

Downes L. & Mui C. (1998) 'Unleashing the Killer App, Harvard Business School Press, 1998, p. 51, at:

Dyson E. (1995) 'Intellectual Value' Wired 3.07 (July 1995), at

Evans P. & Wurster T. (1997) 'Strategy and the New Economics of Information', Harvard Business Review (Sept.-Oct. 1997) 71-82

Ghosh R.A. (1998) 'Cooking pot markets: an economic model for the trade in free goods and services on the Internet' First Monday 3 3 (March 1998), at

Gibson W. (1984) 'Neuromancer' Grafton/Collins, London, 1984

Kelly K. (1996) 'The Economics of Ideas', Wired 4.06 (June 1996), at

Lamberton D.M. (Ed.) (1971) 'The Economics of Information and Knowledge' Penguin 1971

Lamberton D.M. (Ed.) (1996) 'The Economics of Communication and Information' Edward Elgar, 1996

Melcher R.A. (1997) 'Dusting off the Britannica' Business Week, 20 Oct 97, at

Negroponte N. (1995) 'Being Digital' Hodder & Stoughton, 1995

Raymond E.E. (1998) 'The Cathedral and the Bazaar' First Monday 3 3 (March 1998), at

Rayport G. & Gerace, T. (1997) 'Encyclopaedia Britannica', Case 5-396-419, Harvard Business School, at

Rheingold H. (1992) 'A Slice of My Life in My Virtual Community', in Ludlow P. (Ed.) (1996), 'High Noon on the Electronic Frontier: Conceptual Issues in Cyberspace' MIT Press, 1996pp. 413-435

Rheingold H. (1994) 'The Virtual Community in a Computerized World' Secker & Warburg 1994, at

Roszak T. (1986) 'The Cult of Information' Pantheon 1986

Shapiro C. & Varian H.R. (1999) 'Information Rules: A Srtategic Guide to the Network Economy' Harvard Business School Press, 1999

Stallman R. (1992) 'Why Software Should Be Free', at

Stephenson M. (1992) 'Snowcrash', Bantam Spectra, 1992

Sterling B. (1988) 'Mirrorshades': The Cyberpunk Anthology', Preface, Ace, New York, 1988

Sterling B. (1991) 'Islands in the Net', Arbor House, 1989


Go to Roger's Home Page.

Go to the contents-page for this segment.

Send an email to Roger

Created: 1 November 1999

Last Amended: 29 November 1999

These community service pages are a joint offering of the Australian National University (which provides the infrastructure), and Roger Clarke (who provides the content).
The Australian National University
Visiting Fellow, Faculty of
Engineering and Information Technology,
Information Sciences Building Room 211
Xamax Consultancy Pty Ltd, ACN: 002 360 456
78 Sidaway St
Chapman ACT 2611 AUSTRALIA
Tel: +61 2 6288 1472, 6288 6916