Roger Clarke's Web-Site
© Xamax Consultancy Pty Ltd, 1995-2013
|Identity Matters||Other Topics||Waltzing Matilda||What's New|
Principal, Xamax Consultancy Pty Ltd, Canberra
Visiting Fellow, Department of Computer Science, Australian National University
Version of 14 February 1998, with extensions of 1 January 2004
© Xamax Consultancy Pty Ltd, 1997, 1998, 2004
Available under an AEShareNet licence
This document is at http://www.rogerclarke.com/II/InternetPS.html
A PowerPoint slide-set to accompany the document is at http://www.rogerclarke.com/II/InternetPS.ppt
This document provides a romanticised description of the process of the Internet, as though it were a special kind of postal system. Its purpose is as a 'first taste' of how the Internet functions.
Metaphors are dangerous. The longstanding idea of the Internet as an 'information superhighway' is quite facile. It has resulted in many, thoroughly misleading arguments-by-analogy.
The Internet is itself. It is different from every transport and communications technology that has preceded it. Indeed, it is so different that discussions about strategic and policy issues cannot afford to be based on vague impressions, anecdote and analogy; but must instead be presaged on an adequate understanding of Internet technology.
That said, people generally learn best by building upon what they already know. This document is therefore provided as a starting-point for newcomers. It is a fairy story, designed to make more sober descriptions more accessible, including those in the companion documents Internet Technology Primer and Origins and Nature of the Internet in Australia.
First the setting is described, and then the process whereby messages are transmitted over the Internet.
The setting for this fairy story is a land which has a moderately reliable, but highly bureaucratic postal service. It is a troubled land, beset by tempests, earthquakes and bushfires. Land, sea and air transport are continually interrupted, and highways are congested.
The postal service has applied all of the management buzzwords, and as a result is highly devolved, and outsources a great deal of the work to contractors and agents. It avoids excessive use of its market power, and hence each of its contractors and agents use somewhat different procedures. Our hero wants to send a big book through the postal service. But, according to the postal service's rules, the book is much too big to be sent. This story explains how, in these difficult circumstances, our hero contrives to get the book through to the intended recipient.
First, the book is broken into many segments, each of which is small enough to be acceptable to the postal service. The segmentation might be performed on the basis of chapters, sections, pages, paragraphs and illustrations, lines, or even letters.
Each segment is placed into a packet. The packets are numbered sequentially. Each packet has inscribed on it the packet identifier, the number of packets that make up the complete set, the address of the recipient, and the address of the sender.
The packets are submitted, one at a time, to a nearby agent of the postal service. If the agent has the ability to deliver a packet directly to the recipient, it does so. Otherwise, it must decide which agent to send it to, in order to move it closer to the intended recipient. To make this decision, it uses a combination of look-up tables, and currently available information about natural disasters and traffic congestion. Because the currently available information keeps changing, and there are many packets, successive packets that the agent handles may be sent in different directions.
Every time a postal services agent receives a packet from another agent, it makes a decision as to where to send it to next, in much the same way as the original agent did. Hence different packets may follow quite different paths from the sender to the recipient.
Some of the agents that are used along the way may only accept very small packets. If the packet is too big for the agent that it is to be sent to, the sending agent breaks it down into smaller packets, and inscribes on each of the resulting packets the details from the original packet. In order to ensure that the new packets can be distinguished from one another, it adds a sequential suffix to the identifier, and adjusts the total number of packets.
Each packet may pass through many agents, until it reaches an agent that is local to the intended recipient. The recipient receives packets, possibly all from the same nearby agent, but possibly from several agents. The packets may arrive in any order. Once they have all arrived, the recipient has to assemble them, in order to re-generate the complete book.
Sometimes, packets go missing. Because the delivery-time is unpredictable, the recipient has to make a decision as to how long to wait before deciding that a missing packet should be treated as having been lost. (It would be nice to have a registered-packet service, or at least an acknowledgement of receipt. But not all postal services agents can be relied upon to send back acknowledgements, and anyway all those extra packets would clog up even more a network that is already congested).
Once a recipient decides that packets have been lost, it sends a request to the sender, identifying which packets are missing, and requesting that they be sent again. The saga continues until all packets have arrived.
The postal service metaphor can be extended, in order to provide a basic explanation of several further features available on the Internet.
The description above assumes that the recipient waits until all packets have been received, before doing anything with the contents. This is consistent with the conventional view that software processes files, and that a file can only be used by one program at a time.
There is a variety of circumstances in which the file-oriented approach is inappropriate. A particular case is transmission of a song, a speech or a film. A file-oriented approach precludes the recipient from starting to hear or see the file until the transmission is complete.
An alternative approach is for the recipient to wait until it has received enough packets, and then release them to the appropriate program (such as an audio-player). The process is referred to as 'streaming', and the data is called 'streamed audio'. In addition to the data not being stored in a conventional file, streaming depends on the packets being received in the right sequence, and on the remaining packets being received quickly enough that the player can continue without a break.
The description provided earlier assumed that a single novel was to be transmitted from a single sender to a single recipient. But there may be many people who want to receive the novel (e.g. at midnight on the night of release of the latest 'Harry Potter' book).
Radio, television and other transmission media that use some part of the electromagnetic spectrum are used to 'broadcast' signals. In Internet terms, this is equivalent to the complete set of packets being transmitted from the sender to every recipient.
In the case of the Internet, this would be extremely wasteful of the available transmission capacity, because post office agents would receive mega-millions of packets whose payload is the same (only the address differs); and for some proportion of the recipients the message would be a complete waste of resources, because the individual concerned did not want to receive it.
An alternative approach is available, which uses the available transmission capacity much more efficiently. The principle is the same as is used for telephone numbers in the 13 series, and `print-on-demand' services.
Firstly, a number of dispersed service centres are established. People who wish to receive the novel can register with their local service centre. The novel is then transmitted to each service centre. Each service centre then re-transmits the novel to each person who has registered to receive it.
This reduces long-distance traffic (because only as many copies are sent over long distances as there are service centres). It also reduces short-distance traffic, because only as many copies are sent as are needed to satisfy the actual demand.
Events such as concerts and product launches can utilise both streaming of the feed (so that people can watch it 'as-live', seconds after it has occurred), and multi-casting (to prevent a popular event swamping the available transmission capacity).
A further aspect of importance is that none of the 'post offices', 'postal service agents' and 'service centres' need to be part of a monopoly. Provided that all participants are playing by the same rules for the transmission of packets, anyone can set up a service to perform any of the functions.
There are many aspects of the above descriptions that are reasonable depictions of how the Internet works, even though they contain technical deficiencies. Some other aspects are highly simplified, and even downright inaccurate, depictions. For example, when packets containing a physical piece of paper go missing, they are difficult to reproduce; whereas electronic content can be readily replicated and re-transmitted ...
Avoid pushing the post office metaphor too far; and never, ever use it as a basis for thinking about strategic or policy aspects of the Internet.
The story provided here was authored by myself. However it derives from discussions with my co-authors Gillian Dempsey, Rob O'Connor and Ooi Chuin Nee about a remark by Vint Cerf (quoted by Garfinkel & Spafford 1996, p. 453) to the effect that the IP protocol can be thought of "as sending a novel a page at a time, numbered and glued to the back of postcards. All the postcards from every user get thrown together and carried by the same trucks to their destinations, where they get sorted out. Sometimes, the postcards get delivered out of order. Sometimes, a postcard may not get delivered at all, but you can use the page numbers to request another copy. And, key for security, anyone in the postal service who handles the post cards can read the contents without the recipient or sender knowing about it".
I extended the story several years later. My thanks to the candidates in a unit that I offer within the University of Hong Kong MSc(ECom) degree. They pointed out to me that, for all its weaknesses, the metaphor could be used to explain several further aspects of the Internet's operation.
Garfinkel, S. & G. Spafford (1996) `Practical Unix and Internet Security', 2nd Edition, O'Reilly & Associates, 1996
The content and infrastructure for these community service pages are provided by Roger Clarke through his consultancy company, Xamax.
From the site's beginnings in August 1994 until February 2009, the infrastructure was provided by the Australian National University. During that time, the site accumulated close to 30 million hits. It passed 40 million by the end of 2012.
Sponsored by Bunhybee Grasslands, the extended Clarke Family, Knights of the Spatchcock and their drummer
Xamax Consultancy Pty Ltd
ACN: 002 360 456
78 Sidaway St, Chapman ACT 2611 AUSTRALIA
Tel: +61 2 6288 1472, 6288 6916
Created: 10 February 1998 - Last Amended: 1 January 2004; addition of FfE licence 5 March 2004 by Roger Clarke - Site Last Verified: 15 February 2009
This document is at www.rogerclarke.com/II/InternetPS.html