Roger Clarke's Web-Site
© Xamax Consultancy Pty Ltd, 1995-2013
|Identity Matters||Other Topics||Waltzing Matilda||What's New|
Principal, Xamax Consultancy Pty Ltd, Canberra
Visiting Fellow, Department of Computer Science, Australian National University
Version of 17 August 1999, plus a small correction of 31 December 1999
© Xamax Consultancy Pty Ltd, 1999
This paper was prepared for presentation at a Conference on 'Freedom of Information and the Right to Know', Communications Law Centre and the International Commission of Jurists, Melbourne, 19-20 August 1999
It was subsequently republished in First Monday 4, 11 (November 1999), at http://firstmonday.org/issues/issue4_11/clarke/
This document is at http://www.rogerclarke.com/II/DarkAges.html
There's a common presumption that the Internet has brought with it the promise of openness, democracy, the end of inequities in the distribution of information, and human self-fulfilment. Any such conclusion would be premature.
The digital era has ambushed and beguiled us all. Its first-order impacts are being assimilated, but its second-order implications are not. Powerful institutions perceive their interests to be severely threatened by the last decade of technological change and by the shape of the emergent 'information economy'. Elements of their fightback are identified, particularly extensions to legal protectionism, and the active development and application of technologies that protect data from prying eyes.
Many of the features that have ensured a progressive balance between data protections and freedom of access to data have already been seriously eroded, and the new balance that emerges from the current period of turmoil may be far less friendly to public access and more like a New Dark Ages.
Networking technologies have enabled the emergence of social processes over distance ('cyberculture'), and communities that are dispersed or 'virtual' (Clarke 1996). The term 'cyberspace' was coined (fully 16 years ago) to give a name to the 'shared hallucination' that is Internet experience.
To place FOI in its contemporary setting, it's necessary to appreciate one of the defining aphorisms of cyberspace, which is that 'Information wants to be free'. The origins of this are traced in Clarke (1999e). It's an intensely ambiguous statement, transcendental to the point of offering a basis for a new religion, and hence it bears analysis and interpretation.
Firstly, an ambiguity of english language needs to be confronted. The word 'free' primarily implies 'without constraint', and only secondarily 'without payment'. Confusion is invited, because one of the implications of information being readily available is that it does tend to become gratis, that is to say that the manner in which production, storage, dissemination and access are paid for is through business models other than user-pays, such as sponsorship, advertising and subscription (Clarke 1999d).
Three clusters of interpretations are suggested:
The notion of 'freedom of information' has become ossified. Meanwhile, technology has been dancing around it, redefining the underlying concept of 'information', and the processes and the political economy of its production, its dissemination, and access to it.
This paper investigates the new meaning of 'freedom of information' in the information age, information society and information economy. It identifies how the precepts underlying lawyerly pontifications have been overtaken by the impacts of information technology, with the result that conventional discussions within the FOI establishment are picayune, and have degenerated to a focus on trivia at the level of how many angels can dance on a pinhead.
The preamble drew attention to the quantum shift that has occurred as a result of the emergence of the first substantial information infrastructure, the Internet. The concepts of 'data', 'information', 'knowledge' and 'wisdom' are outlined. This leads into a consideration of representations of data, such as documents, databases and directories. The paper then examines the forms in which data exists, and explains the recent shift from atoms to bits.
The first-order impacts of these changes are discussed, including the new behaviours that have quickly arisen, and the new theories of economics and political economy that have emerged in an attempt to explain them.
Less apparent are the second-order implications that are arising from the new information technologies. A consideration of these leads to the conclusion that the conventional expectation that the Internet is delivering greater freedom of access to information may be seriously misguided, and that instead it may result in loss of some of our existing freedoms.
This section establishes the foundation for the analysis that follows, by defining key terms, and summarising the characteristics of the digital era that set it far apart from its predecessors.
It's important to an assessment of 'freedom of information' in the present and future context to consider definitions of terms underlying the discussion. An analysis is presented in the Appendix. The key conclusions it reaches are that:
The 'cybernetic' era of the second half of the twentieth century advanced some sophisticated, but fundamentally mechanistic arguments that sought to reduce all of these ideas to cascading tiers of control loops. It failed: there is no continuum from data to wisdom; they need to be regarded as a set of related, but distinct, domains.
Using the above definitions, that which is stored is data, and information only exists where data has value to a person, in a context. Among the FOI community, it is uncommon for the distinction to be drawn between raw data and context-relevant information.
Data is abstract, but it is represented (or in the terms of copyright law, 'expressed') in concrete form, on physical media. Typical representational forms include scrolls, manuscripts, documents, files, records, databases and directories; and typical media include paper, magnetic tape and disks, optical storage, and integrated circuits (ICs, or 'chips'). This paper uses 'data object' as a generic term for all categories of item that embody representations of data.
The notion of 'freedom of information' is also abstract. In order to constrain the rights that they create, so-called FOI laws have been expressed in very narrow terms. What is needed is a legal right to access data, in any form on any medium, i.e. irrespective of the nature of the data object. The actual rights that exist in law apply not to all data objects, but only to a very limited sub-set of whatever the law at any point in time understands the term 'document' to mean.
At the turn of the century, any discussion of 'freedom of information' has to reflect the characteristics of information in the digital era. Uttering the mantra 'information wants to be free' is not enough; it's important to examine the technical features of information production and reticulation, and then to build from that an understanding of the political economy of the current era.
The form in which information was represented was once merely verbal. For several millennia, it has been expressed in tangible forms, such as scratches on rock, on vellum, on parchment and on paper. These scratches, no matter how sophisticated the script, were only able to be processed by humans reading and interpreting them.
Suddenly, during the last few decades, digital representations have become the mainstream. Ideas had previously been conceived as words, numbers, symbols, shapes, pictures and sounds, and they had required a person to interpret them. A succession of technologies have made it possible to express those ideas in patterns that can be stored on digital media, can be easily, cheaply and rapidly reproduced and disseminated, and can be manipulated by machines as well as people.
The information revolution that has occurred during the last couple of decades is underpinned by a cluster of related data-object technologies. They are:
In some cases (such as digital audio), the quality of digital formats quickly overtook that of their predecessor technologies. In a very short time, a great deal of copyrightable material has leapt from the physical to the electronic, or, as Negroponte (1995) preferred to put it, has migrated from atoms to bits.
During the last half-millennium, successive waves of information technologies have increased the accessibility of data, and changed its economics. These waves have included:
There can be little doubt that, in the economically better off countries, a larger proportion of the public are currently better educated about history, and more aware of current events, than would ever have been possible in the past.
The impact and implications of the recent digital revolution far exceed that of the printing press around 1500, and its effects are being felt far more rapidly. To invoke a simple example, within the space of a mere five years, students have come to expect that the answer to any question will be 'on the Internet', and are now surprised and even dismayed to find themselves directed to sources other than the World-Wide Web. And many of the impacts and implications cut much more deeply than that into the social and economic conventions of the late twentieth century.
The digital era evidences a distinctly different political economy from the now-defunct industrial age. In order to investigate those differences, a brief review is needed of the scene immediately prior to the revolution.
There are various justifications for access to information, including social, psychological, democratic, law and order and economic motivations. There is also a wide range of justifications for the denial of access to information. These include many narrow, sectional interests, but also some of broader concern, such as privacy, and the assurance of some degree of order in the processes of economic development, and of government.
The exercise of political choice is primarily dependent on inter-play between institutions that have political power, and that exercise it. Historical accident also plays a part (in particular, ambiguous wording in statutes and prior judgements, and new judicial inferences about their meaning).
FOI has to date been focussed mainly on government. The private sector has successfully avoided being subjected to the same rigours, partly through the exercise of economic power, and partly because of the economic rationalist philosophies that pervade political parties, which dictate that, even in advanced nations, economic concerns dominate social ones, and that, as a result, freedom for business enterprises is a higher ideal than freedoms for people.
During the two decades since the 1978 legislation affecting some of the Commonwealth public sector, there have been few accretions to the freedom side of FOI. In the States and Territories, small advances have arisen, usually in response to public scandals, but, in most jurisdictions, the granting of meaningful access to government data has been successfully avoided.
One area in which an increase in accessibility has occurred has been the progressive acceptance that environmental impact statements (EIS) are a necessary feature of major infrastructure proposals, and that public information, public consultation, and public involvement in design are elements of the EIS process. Regrettably, that development has not been extended to social impact statements for major initiatives,. In some countries, however, notably New Zealand and Canada, privacy impact assessments (PIAs) for major applications of information technology are entering the mainstream, and this is likely in Australia in the near future (Clarke 1997b).
There have, on the other hand, been plenty of extensions to existing abilities to deny access. Government agencies have a litany of excuses available, in the form of exemptions with wide applicability. Moreover, there are increasingly frequent refusals by agencies and Ministers to provide information to the Parliament and Parliamentary Committees, and Auditors-General are currently submitting to a trend towards gentler audits and audit reports. There has been some degree of tightening in relation to access to personal data, following passage of the Privacy Act 1988, although this primarily limits access by people, rather than access by governments agencies and corporations.
Individual people, lacking a firm power-base, have on the other hand been subject to ever-increasing demands that they provide personal information to government agencies. These demands have been coupled with economic disincentives against non-compliance, and, with occasional exceptions, the public has acceded to the power of governments.
The net effect is that personal data has been becoming increasingly open, information held by corporations remains largely protected, and information held by governments is largely protected, but subject to some limited access provisions.
Producers of all kinds of information have tended to use fairly elaborate 'production-lines' or 'value-chains', with successive individuals and organisations 'adding value' to a base product. Digital technologies have lowered the costs involved in those production processes, by rendering some steps unnecessary, by enabling people to perform other steps with less training or cheaper tools, and by providing cheap and quick transmission from one step to the next. The delays between the origination of data objects and their availability to users have been greatly reduced.
Digital technologies have quickly proven to be a double-edged sword, however. In addition to lowering the costs of production for authorised publishers, they also assisted the replication of data objects by parties who were not acting within the terms of a licence issued by the intellectual property owner. Unauthorised copying, adaptation and use are rife.
The context within which data objects exist has changed very quickly, and the courts are struggling to understand and to apply intellectual property law in the new context. This is considered in greater depth at Clarke & Dempsey (1999). If the controls that have hitherto kept the lid on widespread appropriation of copyright-objects were to cease to be effective, so the argument goes, the haemorrhage of revenue would remove the economic incentive to originate copyright-objects and to publish them.
Debates are currently raging about the shape of the new economics. See for example Lamberton (1971, 1996), Dyson (1995), Romer (with a populist description in Kelly 1996), Clarke (1994), Clarke (1999a), Clarke (1999c) and Shapiro & Varian (1999). These accept as given that the user-pays business model that has been common during the industrial age has been undermined by technology, and that alternative business models need to be applied.
The following aspects convey the extent of the changes involved:
A reading of the preceding analysis might suggest that the changes that have been taking place are predominantly positive, in many ways, and especially for freedom of information. Certainly, that's the aspect of the digital era that most commentators focus upon.
This section identifies second-order effects that are too easily overlooked. They are so substantial that they threaten to undermine the conventional, pleasant expectations that the digital revolution will 'bring in the millennium'.
Information quality is a term for the cluster of characteristics that 'good' information should have. Most important among them are accuracy, timeliness and completeness. Also significant are the provision of evidence of the sources of data which a work refers to, and identification of the author, date and location of publication, in order to enable auditability.
Within the publishing industries and the various information professions, sets of conventions have developed over the decades which provide some degree of assurance in relation to information quality. At the professional end of the data-production business, ranging from entertainment, via sport and news reporting, to statistics and reference information, longstanding business models have been undermined, and alternative business models have not been emerging rapidly enough.
A particularly poignant example of the rapidity with which the digital revolution has undermined a hitherto financially and culturally valuable business is the story of the latest (and, possibly, the last) decade of Encyclopaedia Britannica (EB).
In 1991, the company sold about 400,000 printed sets, and in 1997 about 10,000. (Tellingly, my source for this information is a quotation from the Managing Director of EB International, only available to subscribers to a for-fee service, E-Commerce Today). The collapse was triggered by the success of Microsoft Encarta and other CD-ROM versions of lower-quality but approximately equivalent collections sold in a convenient and inexpensive form. Since then, web-based information services have mushroomed. Despite its brand reputation, and the apparent quality and presumed value of the content the company owned, and even after scrambling to survive, revenue has halved, losses have accumulated, the company has changed hands several times, and survival remains uncertain (Rayport & Gerace 1997, Evans & Wurster 1997, Melcher 1997, Downes & Mui 1998, p.51, Shapiro & Varian 1999, pp. 19-21, 26).
A great deal of the information that has become available on the Internet is a result of amateurs playing author, editor, publisher and marketer, and in some cases doing them in a manner significantly less professional than was achieved by conventional publishers. Longstanding quality assurance conventions are only partly understood, and only partly respected, by the flood of new information-providers who have appeared on the Internet.
Other factors of concern are the discoverability of information, the identification of versions, and the archival of old information. A great many links are ephemeral, and with their demise disappear not only information, but also the ability to audit the claims of authors who refer to them.
Moreover, there remains a great deal of detail that is obscured or even falsified, resulting in what economists refer to as 'information asymmetries', i.e. inequality of information among the participants in decision-making processes, and hence imbalance of bargaining power, and inequities.
Content-providers who depended on their customers paying full value for access to it are seeing their business model undermined by the digital revolution: "there's likely to be a long, lean period for anyone trying to sell information. The problem is that there are so many reasonable and free options that are good enough" (Doyle B., quoted in Melcher 1997).
Rationally, corporations that are dependent for their survival on the exploitation of rights to information are taking action to defend their positions. The risk is that these defensive stances may be economically and culturally very harmful. This is highly reminiscent of the way in which the cyberpunk sci-fi novelists (e.g. Gibson 1984, Sterling 1988, Sterling 1989, Stephenson 1992) envision the 'hypercorps' retiring into enclaves, and dealing only with the identified and continuously monitored members of respectable, official society.
Two ways in which content-owners are battening down the hatches are through legal measures, and through the development and application of information technologies.
The following are key aspects of the argument advanced in Clarke & Dempsey (1999).
An important change in the effect of copyright law accompanied digitisation, and became even more apparent with the explosion of the Internet. The purchase of a book, or tuning to a broadcast radio or television channel, did not necessitate the acquisition of a copyright licence. The purchase or rental of digital media, on the other hand, generally does. Moreover, a workstation's mode of operation inherently involves the making of a succession of copies of the object, in memory and on the screen.
The move from atoms to bits has therefore resulted in an accidental extension to the legal rights that copyright owners enjoy: there has never previously been any right to preclude people from accessing data-objects, whether to read them, listen to them, look at them, or watch them. The new need for the consumer to have a licence has accidentally strengthened the hand of the copyright-owner.
One of the main weapons in the armoury of powerful organisations is the ability to arrange for favourable action in legislatures. For example, a draft Commonwealth Government Bill, the Copyright Amendment (Digital Agenda) Bill 1999, follows the lead of other governments in acceding to the demands of powerful corporations in relation to compilations such as databases, and to technologies which circumvent copyright protections.
A further, recent move has been the suppression of information about errors and security risks in software products: "Information derived from decompilation of a program ... cannot be used or communicated to others for any other purpose" (Ministerial Press Release relating to the Copyright Amendment (Computer Programs) Bill 1999, enacted 13 August 1999).
The major publishing interests have been able to significantly extend the scope of copyright, and hence protect their revenues, profits and empires. Moreover, they have arranged for activities that offend large copyright-ownership interests to be dealt with by the criminal courts (at public cost) rather than the civil courts (at their own cost).
There are also signs that both trade secrets law and the law of confidence may be in the process being developed in directions that provide yet greater protections for the interests of large corporations. This may extend even to the point of protecting ideas rather than expressions, and beyond the protection of data objects to, for example, preclude individuals from applying their accumulated, abstracted knowledge.
A further manoeuvre whereby copyright-objects can be protected against abuse is to rely less on copyright law and more on direct contractual arrangements between the publisher and the would-be reader. This would have the effect of undermining libraries, and restricting not only reproduction, but also access to the object.
Instances already exist, such as expensive hard-copy reports that are sold on the express condition that they cannot be lent to any other person, and databases that are remotely accessible by subscription.
Educational institutions have a statutory licence under the Copyright Act. They could establish digital collections of copyright-objects, irrespective of who they are owned by, and allow students and staff to make digital and printed copies from those collections. But the Act makes the licence subject to an equitable payment to the copyright-owner.
Photocopying was the subject of long negotiation between representatives of copyright owners and universities. During the last five years, the establishment of 'electronic reserves' of materials frequently accessed by students has been stymied by an ongoing battle between those same parties as to what 'equitable' means. This impasse represents a grave threat to the performance of the nation in the information era.
There are many different circumstances in which people access information. For example, some people do so as consumers, whereas the intention of other accesses is to use the data as a 'factor of production', in order to produce more information. Some accessors are disadvantaged, due to such factors as physical impairment, or the locality in which they live. Others enjoy privileges of various kinds.
The exercise of power that the copyright-owner gains from technological innovations and legislative amendments represents a very substantial negative effect on equitable public access to information. This is further discussed at (Clarke 1999a).
Historically, a great deal of access to published works has been anonymous, in the form of purchase of books, access to books in libraries and viewing of films in theatres; or pseudonymous, e.g. the borrowing of books from libraries, or the hiring of a video. A relatively small proportion of access has been associated with an authenticated identity; and that has generally been where the material was being adapted or incorporated into another work.
This lack of identification is very important, because it sustains an environment in which information is generally accessible without fear of recrimination from authority-roles such as employers, competitors, teachers, parents and powerful vested interests. Any shift from copyright towards contract as the primary basis risks bringing with it a switch from predominantly anonymous to predominantly identified transactions; and with that would come a serious 'chilling effect' on information availability, and a direct threat to open society and democracy.
Powerful organisations have a variety of interests in preventing the exposure of data they control. They seek to protect information that would disclose such things as their corporate strategy and the directions of their technology investments; the nature of micro-monopolies from which they extract super-profits; and the details of their financial and taxation arrangements.
Many corporations also wish to exploit data under their control, and in order to do so seek out technologies that protect data objects. Some of these technologies are passive in nature, such as:
Others are active technologies, such as:
These are described in greater detail at (Clarke & Dempsey (1999).
The effect of such technologies is to preclude information access. In many cases, this may not change the current balances, but in some circumstances, it is likely to reduce the accessibility of information. For example, a publisher could combine:
It's feasible that the Internet Protocol Suite could be adapted in a manner that favours the protection of data objects, and thereby undermines freedom of information. For example, it culd facilitate existing techniques that assist corporation-owned servers to exercise control over personal workstations; and there has been a recent tendency towards the siphoning off of Internet bandwidth in order to support 'virtual private networks' (VPNs).
Alternatively, the open, public Internet could be circumvented by the emergence of a new architecture, developed and promulgated by an alliance of corporations and governments, and much more suited to their needs rather than that of consumers and citizens (and probably endowed with an insidious title such as Internet 3).
A variety of tools is available, many based on cryptographic methods, which enable people to protect their identity. This can be achieved either by denying it entirely, or by substituting it with a pseudonym and protecting the linkage between real and pseudo-identity through technical, organisational and legal measures (Clarke 1998, Clarke 1999b).
The focus during the last decade or so has been on the use of such tools by individuals, in order to send messages and distribute documents whose originator cannot be traced (e.g. using 'anonymous remailers'). There has been speculation, however, that the largest uses of these tools may be by governments, and by businesses dealing in particular kinds of data-objects, such as pornography.
Similar tools can also be used to access documents without leaving an identified trail as to who has viewed which documents (e.g. so-called 'anonymous web-surfing'). Of course, such tools are valuable not only to individuals, but also to governments and businesses conducting covert operations of various kinds.
The implications of these developments is that governments and corporations are in a position to enhance the restrictions on access to information. The golden era of information accessibility is under threat, because governments have successfully resisted FOI and now have additional weapons available to them, and major corporations are wielding their power to protect their own interests.
In the present information era, skirmishes around the edges of existing FOI laws are irrelevant. If freedom of information is to be sustained, let alone increased, then measures are needed now. Arguments against legal protectionism need to be advanced much more energetically, countervailing power needs to be mobilised against corporate and governmental interests, and information and networking technologies need to be carefully designed, to avoid protectionism becoming entrenched within the information infrastructure.
A serious battle is in train, with corporations manipulating governments, netheads wailing, the public non-aware, and FOI specialists largely non-comprehending. FOI activists can stay asleep and become even less relevant, or can inform themselves and become involved.
Barlow J.P. (1994) 'The Economy of Ideas', Wired 2.03 (March 1994), at http://www.wired.com/wired/archive/2.03/economy.ideas_pr.html
Blackmore S. (1999) 'The Meme Machine', Oxford Uni. Press, 1999, from http://www.oup.co.uk/isbn/0-19-850365-2
Brin D. (1998) 'The Transparent Society' Addison-Wesley, 1998
Clarke R. (1992a) 'Fundamentals of 'Information Systems', at http://www.rogerclarke.com/SOS/ISFundas.html
Clarke R. (1992b) 'Knowledge', at http://www.rogerclarke.com/SOS/Know.html
Clarke R. (1994) 'Electronic Support for Research Practice: The Inadequacy of Economic Analysis in a Time of Revolutionary Change', The Information Society 10,1 (March 1994), at http://www.rogerclarke.com/II/ResPractice.html
Clarke R. (1996) 'CyberCulture: Towards the Analysis That Internet Participants Need', March 1996, at http://www.rogerclarke.com/II/CyberCulture.html
Clarke R. (1997a) 'Electronic Publishing: A Specialised Form of Electronic Commerce', Proc. 10th International Electronic Commerce Conf., Bled, Slovenia, June 1997 , at http://www.rogerclarke.com/EC/Bled97.html
Clarke R. (1997b) 'Privacy Impact Assessments', September 1997, at http://www.rogerclarke.com/DV/PIA.html
Clarke R. (1998) 'Information Privacy On the Internet: Cyberspace Invades Personal Space', Telecommunication Journal of Australia 48, 2 (May/June 1998), at http://www.rogerclarke.com/DV/IPrivacy.html
Clarke R. (1999a) 'Key Issues in Electronic Commerce and Electronic Publishing', Proc. Conf. Information Online and On Disc 99, Sydney, 19 - 21 January 1999, at http://www.rogerclarke.com/EC/Issues98.html
Clarke R. (1999b) 'The Legal Context of Privacy-Enhancing and Privacy-Sympathetic Technologies', April 1999, at http://www.rogerclarke.com/DV/Florham.html
Clarke R. (1999c) 'The Willingness of Net-Consumers to Pay: A Lack-of-Progress Report', Proc. 12th Electronic Commerce Conf., Bled, Slovenia, 8-9 June 1999, at http://www.rogerclarke.com/EC/WillPay.html
Clarke R. (1999d) 'Electronic Services Delivery: From Brochure-Ware to Entry Points', Proc. 12th Electronic Commerce Conf., Bled, Slovenia, 8-9 June 1999, at http://www.rogerclarke.com/EC/ESD.html
Clarke R. (1999e) 'Information Wants To Be Free', August 1999, at http://www.rogerclarke.com/II/IWtbF.html
Clarke R. & Dempsey G. (1999) 'Electronic Trading in Copyright Objects and Its Implications for Universities', Proc. Australian EDUCAUSE'99 Conf, Sydney, 18-21 April 1999, at http://www.rogerclarke.com/EC/ETCU.html
Dawkins R. (1976) 'The Selfish Gene' Oxford Uni. Press, 1976, from http://www.oup.co.uk/isbn/0-19-286092-5
Dempsey G. (1998) 'Knowledge and Innovation in Intellectual Property: The Case of Computer Program Copyright' Unpublished PhD Thesis, Australian National University, March 1998, c/- Dr Gillian Dempsey
Downes L. & Mui C. (1998) 'Unleashing the Killer App, Harvard Business School Press, 1998, p. 51, at: http://www.killer-apps.com/Contents/booktour/the_new_firm_rebuilding_with_bits.htm
Dyson E. (1995) 'Intellectual Value' Wired 3.07 (July 1995), at http://www.wired.com/wired/archive/3.07/dyson_pr.html
Evans P. & Wurster T. (1997) 'Strategy and the New Economics of Information', Harvard Business Review (Sept.-Oct. 1997) 71-82
Ghosh R.A. (1998) 'Cooking pot markets: an economic model for the trade in free goods and services on the Internet' First Monday 3 3 (March 1998), at http://www.firstmonday.dk/issues/issue3_3/ghosh/index.html
Gibson W. (1984) 'Neuromancer' Grafton/Collins, London, 1984
Kelly K. (1996) 'The Economics of Ideas', Wired 4.06 (June 1996), at http://www.wired.com/wired/archive/4.06/romer_pr.html
Lamberton D.M. (Ed.) (1971) 'The Economics of Information and Knowledge' Penguin 1971
Lamberton D.M. (Ed.) (1996) 'The Economics of Communication and Information' Edward Elgar, 1996
Melcher R.A. (1997) 'Dusting off the Britannica' Business Week, 20 Oct 97, at http://www.businessweek.com/1997/42/b3549124.htm
Negroponte N. (1995) 'Being Digital' Hodder & Stoughton, 1995
Raymond E.E. (1998) 'The Cathedral and the Bazaar' First Monday 3 3 (March 1998), at http://www.firstmonday.dk/issues/issue3_3/raymond/index.html
Rayport G. & Gerace, T. (1997) 'Encyclopaedia Britannica', Case 5-396-419, Harvard Business School, at http://www.hbsp.harvard.edu/hbsp/prod_detail.asp?396051
Rheingold H. (1992) 'A Slice of My Life in My Virtual Community', in Ludlow P. (Ed.) (1996), 'High Noon on the Electronic Frontier: Conceptual Issues in Cyberspace' MIT Press, 1996pp. 413-435
Rheingold H. (1994) 'The Virtual Community in a Computerized World' Secker & Warburg 1994, at http://www.rheingold.com/vc/book/
Roszak T. (1986) 'The Cult of Information' Pantheon 1986
Shapiro C. & Varian H.R. (1999) 'Information Rules: A Srtategic Guide to the Network Economy' Harvard Business School Press, 1999
Stallman R. (1992) 'Why Software Should Be Free', at http://www.gnu.org/philosophy/shouldbefree.html
Stephenson M. (1992) 'Snowcrash', Bantam Spectra, 1992
Sterling B. (1988) 'Mirrorshades': The Cyberpunk Anthology', Preface, Ace, New York, 1988
Sterling B. (1991) 'Islands in the Net', Arbor House, 1989
This Appendix is an abbreviated rendition of the analysis in Clarke (1992a), and Clarke (1992b).
The information systems discipline and profession regard 'data' as any symbol, sign or measure which is in a form that can be directly captured by a person or a machine. The most useful data represents or purports to represent facts and events in the real world. (Note that, while 'data' is technically a plural noun - the singular is 'datum', usage during the last fifty years has rendered it a generic term, qualified as, for example, 'data-item' for the singular and 'data-items' for the plural).
The vast majority of real-world facts never give rise to data. The background noise emanating from all points of the universe has been ignored for millions of years (until the last few decades, during which some astronomers have occasionally sampled a tiny amount of it). Some things about the trucks that carry goods in and out of a company's gates may be of great interest to someone (such as which trucks, when, what they carried in, and what they carried out). But it's rare to bother even measuring, let alone recording, the pressure in the tyres on the trucks, the number of chip-marks in the paintwork, the condition of the valves on the motor, or even the number of consecutive hours the driver has been at the wheel. There are myriad real-world facts that we let go by, and never capture as data.
Of the real-world data that we do capture, many kinds are very uninteresting. The contents of audio-tapes on which astronomers record the background noise emanating from various parts of the sky might on occasions contain a signal from a projectile launched from the earth, and just possibly might contain some pattern from which it would be possible to infer an inter-stellar event, or perhaps the existence of intelligent life somewhere in the universe. But usually the contents are extremely boring, and devoid of any value to anyone. Similarly, a great deal of the data captured by commerce, industry and government is either 'just for the record' or of interest for only a very short time, and then filed in case someone ever wants to look at it again.
What is it that makes data interesting or valuable? The most straightforward way in which data is useful is when it has relevance to a decision. When we make up in the morning, we don't usually think about what the weather is like outside until we are deciding what to do with the day (if it's a weekend) or what to wear (if it's a workday). Data about a delivery of a particular batch of baby-food to a particular supermarket is lost in the bowels of the company's database, never to come to light again, unless and until something exceptional happens, such as the bill not being paid, the customer complaining about short delivery, or an extortionist making a telephone call to claim that poison has been added to some of the bottles.
A narrow interpretation is that data is relevant and of value only if it makes a difference to a decision. A more general approach is to define it as relevant if it could make a difference. A yet broader conception recognises that value exists even in the absence of a decision, where the data is not what we would have expected, and therefore has 'surprisal' value ("Gosh! The government might survive the election yet!" Or "An injury incurred in training will keep the star fullback out of the Grand Final!"). Information is "a difference which makes a difference" (Barlow 1994, quoting Bateson [previous versions of this paper incorrectly had Barlow attributing the quotation to Shannon]).
The most useful definition is therefore that information is data that has value in a context. Until it is placed in an appropriate context, data is not information, and once it ceases to be in that context it ceases to be information. A critical element of that context is a person who is processing the data, and inter-relating it with other data and models. "Information is an activity -- a verb not a noun" (Barlow 1994).
It's important to relate information to two further concepts. Knowledge can be naively considered as a body of facts and principles accumulated by mankind over the course of time. This storehouse' notion is too mechanistic to be of much use, however. A more serviceable interpretation is as the matrix of impressions within which an individual situates newly acquired information, and which underlies selectivity, filtering and interpretation. Wisdom, meanwhile, is a very different notion from information and knowledge. It is judgement by an individual, using currently available knowledge and new information, based on criteria that are felt rather than expressed
The information systems discipline's definitions of data and information presented above avoid the most simplistic notions, but are still too mechanistic for some observers: "information, [even today], is no more than it has ever been: discrete little bundles of fact, sometimes useful, sometimes trivial, and never the substance of thought [and knowledge] ... The data processing model of thought ... coarsens subtle distinctions in the anatomy of mind ... Experience ... is more like a stew than a filing system ... Every piece of software has some repertory of basic assumptions, values, limitations embedded within it ... [For example], the vice of the spreadsheet is that its neat, mathematical facade, its rigorous logic, its profusion of numbers, may blind its user to the unexamined ideas and omissions that govern the calculations ... garbage in - gospel out. What we confront in the burgeoning surveillance machinery of our society is not a value-neutral technological process ... It is, rather, the social vision of the Utilitarian philosophers at last fully realized in the computer. It yields a world without shadows, secrets or mysteries, where everything has become a naked quantity" [Roszak 1986, pp.87,95,98,118,120,186-7].
The content and infrastructure for these community service pages are provided by Roger Clarke through his consultancy company, Xamax.
From the site's beginnings in August 1994 until February 2009, the infrastructure was provided by the Australian National University. During that time, the site accumulated close to 30 million hits. It passed 40 million by the end of 2012.
Sponsored by Bunhybee Grasslands, the extended Clarke Family, Knights of the Spatchcock and their drummer
Xamax Consultancy Pty Ltd
ACN: 002 360 456
78 Sidaway St, Chapman ACT 2611 AUSTRALIA
Tel: +61 2 6288 1472, 6288 6916
Created: 24 May 1999 - Last Amended: 17 August 1999 by Roger Clarke - Site Last Verified: 15 February 2009
This document is at www.rogerclarke.com/II/DarkAges.html