Roger Clarke's Web-Site


© Xamax Consultancy Pty Ltd,  1995-2016

Roger Clarke's 'File Discovery and Sharing Technos'

File-Discovery and File-Sharing Technologies (aka Peer-to-Peer or P2P):
MP3, Napster and Friends, and Their Impact on E-Publishing of Music, and Other Digital Objects

Roger Clarke

Principal, Xamax Consultancy Pty Ltd, Canberra

Visiting Fellow, Department of Computer Science, Australian National University

Revision of 30 July 2000; with many additional references up to 1 May 2001. Major Revision of 11 June 2001; with additional references to 22 December 2003

© Xamax Consultancy Pty Ltd, 2000-03

This document is at



I've written a series of papers that deal with aspect of e-publishing, and provide a workshop on the topic. This document is an annotated set of resources on a vitally important battle running from about 1998 to 2003.

Copyright was designed to provide an incentive for the production of new works, by assisting the producer to earn financial returns from them. It achieved this by creating a monopoly in the rights to reproduce, adapt and publish each work, and vesting those rights in the originator.

During the twentieth century, copyright laws enabled publishing organisations to generate substantial revenue streams from works of many different kinds. A proportion of the sale price to the consumer (but often a quite small proportion) accrued to the originator. The market power that publishers achieved was such that most originators were forced to sign away their copyright.

People who sought to break the monopoly (and hence to infringe the copyright owner's legal rights) needed to invest considerable effort and money to do so. A couple of breakouts occurred. Some were brought back under control, as occurred with Digital Audio Tape (DAT), which contains anti-copying features. A couple got away, such as consumer recording of films broadcast over television channels, which were recorded onto VCRs. This was almost impossible to prevent or prosecute, and was legalised. But, during most of the twentieth century, copyright arrangements worked, most of the time, at least for the publishing houses, and to some extent for the originators of the works. And consumers in economically advanced nations gained access to a vast array of material.

During the last decade or so, however, digital technologies have dramatically reduced the investment needed to make, adapt and re-publish copies of works. People are able to ignore copyright law.

This particular document provides background information and references on a series of software tools that have been designed in order to assist people to circumvent copyright, and share copyright works. Technological change is affecting all media. This document focuses on music, because that is an area in which the technological barriers protecting publishers' interests were particularly significant, and in which, from about 1998 onwards, the barriers are being torn down at a frenetic pace.

Background to MP3

The following is an edited version of a section from John Hedtke's 'MP3: Music on Your PC'.

MPEG (pronounced EM-peg) is an acronym for Motion Picture Experts Group and comprises a set of standards for compressing and storing digital audio and video. MP3 is short for "MPEG Audio Layer 3," and it identifies a way to store digital audio files. MP3 files give you near CD-quality sound in a file format that requires roughly 1MB for every minute of sound. (CDs and WAV files, by contrast, require about 11MB per minute).

A single song or track in MP3 format usually takes up between 3MB and 5MB, a reasonable download even at 28.8kbps. By the late 1990s, it was feasible to burn a CD with 2 hours of music on it, on a PC-connected consumer applicance costing a few hundred dollars, and to create a DVD disk containing more than 80 hours of music.

People became very interested in gaining access to MP3 files. A profusion of MP3 web sites, newsgroups, and ftp sites sprouted up across the Internet, in a form of 'swap-meet' that, because of its electronic nature, was dispersed around the globe, and never stopped.

Sound, as heard by animals, comprises pressure waves in the air whose effect on the eardrums is interpreted by the brain. These pressure waves can be converted into analogue form (i.e. another format whose characteristics resemble those of the sounds). Examples include phonograph records and audio cassette-tapes. With advances in computing, however, sound has been increasingly represented in digital form. This is done through sampling - the process of taking a snapshot of the sound many times per second. CDs store information in a digital audio format known as CD-DA, which is very similar to the standard WAV format and samples sound 44,000 times per second.

MP3 files are based on psychoacoustics - the study of how the human brain perceives sound. This science has determined that not all the sound we hear is perceived by the brain. To create an MP3 file, an MP3 encoder reads a WAV file and strips out the parts you won't miss hearing.

For example, most people can't hear sounds above 16KHz, so the encoder strips out any sounds above a preset threshold level. Loud sounds will mask quieter sounds at or near the same frequency, so the encoder removes these, too. By whittling away the parts you don't hear, the encoder creates a file that sounds almost the same but is dramatically smaller.

An MP3 file can also contain information about the file itself in a tag. T he tag can contain things like the artist's name, a graphic (usually the CD cover art), a URL for more information, another URL where you can buy the CD, the song's lyrics, the genre, and more.

See also:

Napster 1998-2002

Napster was a shortlived service, which enabled consumers who had MP3 files on their machines to have those files catalogued by a central service, and discovered and downloaded by other users. Hundreds of thousands of people made files available, and millions downloaded them.

Napster quite possibly exhibited the fastest growth of any Interneet-based service ever - which is quite a claim. It also underwent one of the fastest-ever implosions, when the courts supported the claims of the music industry juggernauts that it was a means whereby their copyright was being breached.

The company was unable to win the court-battles, and unable to adapt its technology such that only legal downloads were feasible. By late 2000 it was off the air, and by late 2002 the company was liquidated. The story is recounted in the sections following Background to Napster.

Alternatives to Napster

Several alternatives to Napster exist. From the viewpoint of people determined to beat the music publishing industry, Napster has a deficiency. The key problem is that it contains a 'single point of failure' or 'choke point', which can be attacked by an opponent such as RIAA. This is because the catalogue of files and their locations is stored centrally.

Some of the alternatives offer similar functionality without suffering from the same weakness. This is because not only the files are dispersed across many sites, but the catalogue is as well. To combat services with these characteristics, RIAA would need to gain injunctions against a critical mass of participants, and gain the support of law enforcement agencies to ensure that a critical mass of them were forced to comply.

Important examples of distributed resource discovery and sharing schemes include (in alphabetical order):

And here are a couple of resource sites:

The Next Round Begins

In April 2001, the Motion Picture Association of America (MPAA) declared war on organisations supporting the distribution of films over the Internet using Gnutella:

Suing a will-o'-the-wisp may be tricky. The Gnutella site explains it this way: "Gnutella is an open, decentralized, peer-to-peer search system that is mainly used to find files. Gnutella is neither a company nor a particular application. It is also not a Web site; in particular, it is not this one, which is merely a hub for Gnutella information. It is a name for a technology, like the terms 'e-mail' and 'web'".

During the third quarter of 2001, volumes grew rapidly on alternative P2P services, especially FastTrack, Audiogalaxy, iMesh and Gnutella. See:

In November 2002, a federal court judge in Chicago issued a temporary restraining order against Madster, formerly Aimster - 'Judge Orders Madster to Disable Servers', Excite News, 4 December 2002).

Technologies to Counter Copyright Infringement

The publishing industry is trying to develop technologies that will prevent copyright being infringed. This is challenging, but (with some qualifications) technically feasible. Outlines of the various passive and active protection technologies are provided in a paper I wrote jointly with a graduate student of mine, Stephen Nees.

Content industries are in the process of developing a variety if such technologies under the generic term 'Digital Rights Management', and the music industry association refers to the particular technology that they are trying to invent as the Strategic Digital Music Initiative (SDMI). These involve attempts to significantly extend the set of rights that are embodied in copyright law. They are also likely to be highly invasive of content consumers' privacy. I argued in a paper in early 2001 that there was a likelihood of a consumer backlash against the Digital Rights Management movement.

The Criminalisation of Copyright Infringement

Invention is one challenge, bu implementation of such technologies is a different challenge again. New devices would protect new media from being copied; but there would have to be transitional arrangements to cope with the wide array and vast volume of existing media and existing player technology.

The industry has in mind to enlist the assistance of legislatures to achieve its ends, and has a track record of successfully doing so. The legislative and enforcement measures that the publishing industry would need comprise:

One might ask whether it is appropriate for a national government to impose measures normally used for criminal matters in relation to what has always been a civil wrong. On the other hand, such measures have already been instituted in respect of some forms of copyright breach by at least the U.S. Digital Millenium Act and Australian copyright amendment legislation.

Why This Matters

The second half of the twentieth century brought unprecedented access to content of many different kinds. The tradition of access to information goes back much further, however, to at least the printing presses of Gutenberg and Caxton, the publication of the Bible in the vernacular (since the fifteenth century), and the emergence public libraries (since the nineteenth century).

The publishing industry appears to have been successful in arguing that information is a form of property that should be protected by the State as forcefully as real property and chattels, despite its inherent infinite replicability, and the criticality of free information flows to commerce, society and democracy.

The implications of content protection technologies, and of the criminalisation of content access, are no less than the loss of the freedom to access information. This is examined in depth in my paper on 'The Internet as Harbinger of a New Dark Ages'.

Further Reading

The following are works by variously myself, myself with friends, and a friend. They're much more prosaic and much less fun, but they contain important analysis supporting such apocalyptic visions:

A vast amount has been written on this topic. Here's a selection of the more coherent and considered articles:

Background to Napster

The following is an extract from Steve Allen's 'Share MP3's with Napster'.

Napster, created by a 19-year-old college student, Shawn Fanning, allows users to share MP3 files over the Net with other Napster users. They describe it as "a completely new way of thinking about music online" and an alternative to search engines that often yield broken links. You can search for music files among logged-in users or join one of the chat rooms and search there.

Napster's copyright policy page states "Napster respects copyright law and expects our users to do the same...You should be aware that some MP3 files may have been created or distributed without copyright owner authorization." They further state that the company will terminate the accounts of users who violate copyright laws.

Nonetheless, Napster is primarily used as a means for sharing commercial, illegally-copied music. Indeed, Napster has been sued by several bands and organizations, most notably the Recording Industry Association of America (RIAA). The suit was brought in December of 1999, and in mid-2001 is still not resolved. The RIAA claimed that Napster creates a black market for illegal copies of digital music, and is seeking $100,000 for each song title allegedly exchanged illegally.

The Napster client application is only 620Kb. The first time you run it, you will be asked if Napster should scan your hard drive for MP3 files. If you agree, then these can be made available for sharing with others, but you don't have to do this. You are also prompted for folder names that you will be sharing.

To find songs, just select the "search" button at the top of the Napster screen. Type in the name of the artist and/or song title, and you can see if it is available. If the song is found, a list of members who have it will be listed, along with information about their connection speed.

See also:

The Lawsuit by RIAA Against Napster

During 1999-2000, the Recording Industry Association of America (RIAA) became increasingly frantic about the impact that MP3 and Napster would have (and possibly was already having) on the revenue-streams of their member-corporations. A small number of established music-groups, notably Metallica, also attacked the immorality of people using music they hadn't paid for.

RIAA commenced an action against Napster Inc., seeking to at least force the company to prevent material owned by its members from being catalogued on the Napster database.

The following sections provide a timeline of information about the case. Here are some news services' archives:

Developments up to 2Q 2000

Here are some articles on the matter:

Developments in 3Q 2000

In July 2000, the recording industry gained an interim injunction that would have required Napster to close down its site until the case was completed. The interim injunction was reversed on appeal, leaving the service operational until the case is heard.

Here are some reports on the matter:

Developments in 4Q 2000

Napster usage volumes demonstrated the fastest growth of any Internet metric to date:

In October 2000, one of the Big 5 recording companies, Bertelsmann, did a deal with Napster:

And here are a couple of analyses:

Developments in 1Q 2001

A case brought against Napster found against the company. The judge indicated that an injunction to preclude or control Napster's service was imminent, but that further negotiations and hearings were needed in order to ensure appropriate terms for the injunction:

The company then announced that it would shortly be able to not only trace all downloads commencing in mid-2001, but also to retrospectively track the millions that had already been performed. Napster chief executive Mr Hank Barry was reported on 5 March as saying that "the file names that Napster would filter out of its system were those that had been reported to Napster by aggrieved copyright holders, including the band Metallica". The world was justifiably sceptical. Here are two reports:

Developments in 2Q 2001

The Judge kept trying hard to avoid actually closing the Napster service down, while imposing on Napster Inc. a requirement to respect copyright. Napster Inc. led the Judge and the RIAA a merry chase. They acknowledged that they could and would delete catalogue-entries for named files that copyright-owners identified as infringing their copyright; but they denied that it was Napster Inc.'s responsibility to find the catalogue-entries in the first place. See:

The key appears to be ambiguity in a Court of Appeals ruling which stated on the one hand that "[the burden is on the recording industry to] provide notice to Napster of copyrighted works and files containing such works available on the Napster system before Napster has the duty to disable access to the offending content"; but also stating, on the other hand, that "Napster, however, also bears the burden of policing the system within the limits of the system".

Developments in 3Q 2001

Slowly the noose tightened around Napster's throat, and by the beginning of July 2001 the service was all-but dead:

But by that time there had already been an upwelling of alternative technologies that did not offer the copyright-controlling behemoths the same kinds of choke-points. See the section below on Alternatives to Napster, and the continuation of the story below that.

Developments in 4Q 2001

And by the end of 2001, the digital era was starting to bite into music publisher turnover:

The End of Napster

Napster's cash reserves lasted until mid-2002. By then, with no alternative revenue flow being generated, manoeuvres began for Bertelsmann (one of the big-5 recording companies) to absorb it. The assumption was that Bertelsmann hoped to capitalise on the Napster name to attract paying subscribers:

In the end, Bertelsmann didn't pull off the takeover, and the company was liquidated in September 2002:

Developments during 2003

But the story won't die ...

Remarkably, the music industry then started to eat its own. Bertelsmann owns one of the large labels, BMG. It had broken ranks with the other majors, by injecting funds into Napster, and, when it closed its doors, acquiring its technology. In February 2003, a group of music publishers took Bertelsmann to court, claiming its support for Napster had damaged their businesses. See

Meanwhile, it was reported that 'Roxio [is] to relaunch Napster this year'. P., MacCentral, February 25, 2003 8:15 am ET

During mid-2003, the US record labels sent nastygrams to some hundreds of individuals, including a 12-year-old girl and a grandmother. They withdrew some of their accusations in the face of media pressure and signs that their evidence was flawed. They claimed over the following months to have won settlements from many other people. A report in December mentioned "at least 220 out-of-court settlements, usually for $US5000 or less".

The identities and contact-points of the people that they sent threatening letters to had been acquired from the records of ISPs. They had extracted those records from ISPs by means of pre-trial discovery, despite the fact that they had not at that stage initiated lawsuits against those individuals, nor against the ISPs. In most cases, they never did initiate a lawsuit.

In December 2003, a US appeals court reversed a lower court ruling, and refused the companies permission to extract copies of ISP's records by means of sub poena without first filing a lawsuit. To the appeals court, the industry's claimed legal basis, based on the 1998 DMCA, "borders upon the silly". To a layman it would appear to have been an abuse of process. See 'Judge: RIAA can't subpoena file-trader information', Grant Gross, IDG News Service, 19 December 2003.

Also in December 2003, the Dutch Supreme Court upheld the ruling of a lower court to the effect that KaZaA could not be held liable for the activity of its users. See 'Record labels rebuffed over online copyright', Reuters & Agence France-Presse, 22 December 2003.

xamaxsmall.gif missing
The content and infrastructure for these community service pages are provided by Roger Clarke through his consultancy company, Xamax.

From the site's beginnings in August 1994 until February 2009, the infrastructure was provided by the Australian National University. During that time, the site accumulated close to 30 million hits. It passed 50 million in early 2015.

Sponsored by Bunhybee Grasslands, the extended Clarke Family, Knights of the Spatchcock and their drummer
Xamax Consultancy Pty Ltd
ACN: 002 360 456
78 Sidaway St, Chapman ACT 2611 AUSTRALIA
Tel: +61 2 6288 6916

Created: 29 May 2000 - Last Amended: 30 July 2000; Additional References 2000: 30 Aug, 9 Sep, 3, 6 and 31 Oct, 1, 2, 5, 10 Nov, 5 Dec; 2001: 15, 19 Feb, 5, 9 Mar, 20 Apr, 1 May; Major revision of 11 June 2001; Additional references of 2001: 9 Jul, 12 Aug, 9 Sep; 2002: 5, 8 Jan, 16, 21 May, 2 October, 15 November, 5 December; 2003: 26 Feb, 22 Dec by Roger Clarke - Site Last Verified: 15 February 2009
This document is at
Mail to Webmaster   -    © Xamax Consultancy Pty Ltd, 1995-2013   -    Privacy Policy