Roger Clarke's Web-Site

© Xamax Consultancy Pty Ltd,  1995-2024
Photo of Roger Clarke

Roger Clarke's 'Disciplined Lit. Reviews'

Disciplined Literature Reviews

Roger Clarke

Principal, Xamax Consultancy Pty Ltd, Canberra

Visiting Fellow, Department of Computer Science, Australian National University

Version of 15 July 2000

© Xamax Consultancy Pty Ltd, 2000

This document is at



I sent a Request for Information to the ISWorldNet List on 14 July 2000. This document summarises the information provided by that community. It is part of a cumulative archive of research information solicited via ISWorld, maintained by Ron Weber.

I was actually seeking information in relation to the disciplined examination of published research, or systematic literature review; but I referred to it using the term 'meta-analysis'. The problem is that that term is already used for a specific statistical technique.

This document presents the information that I've accumulated from responses relating to DLR.htmldisciplined literature reviews.

A separate document contains the information about the statistical technique called meta-analysis.


By the term 'disciplined literature review', I mean:

a disciplined examination of published research, in order to identify commonalities and differences in method and/or content, to classify papers within a pre-determined taxonomy, or to abtract a new taxonomy

Peter Seddon suggests that imposition of a degree of organisation on a moderately chaotic body of knowledge depends on "a combination of (1) grounded theory and (2) intuition based on a deep immersion in, and understanding of, the data".

The term 'systematic review' already exists in the medical research fraternity, and may be in the process of being applied within the IS discipline.

The term 'metatriangulation' has recently been coined to describe a related technique, and this may also be in the process of adoption within the IS discipline.


Cooper H.M. (1998) 'Synthesizing Research : A Guide for Literature Reviews', Sage, 3rd Ed., 1998. A 35-page outline of this book is available from Hope Koch

Fink A. (1998) 'Conducting Research Literature Reviews: From Paper to the Internet', Sage, 1998

Hart C. (1998) 'Doing a Literature Review: Releasing the Social Science Research Imagination', Sage, 1998


From me and Peter Seddon:

DeLone W. H. and McLean E. R. (1992) 'Information Systems Success: The Quest for the Dependent Variable', Information Systems Research 3, 1

Seddon, P.B. (1997) 'A Respecification and Extension of the DeLone and McLean Model of IS Success", Information Systems Research, 8, 3 (1997) 240-253

Seddon P.B., Staples S., Patnayakuni R. & Bowtell M. (1999) 'Dimensions of Information Systems Success', CAIS, 2, 20 (November 1999), at

From Varun Grover:

Kwon, T.H., and Zmud, R.W. (1987) 'Unifying the Fragmented Models of Information Systems Implementation', in Critical Issues in Information Systems Research, John Wiley & Sons Ltd, 1987

Prescott, M.B. (1995) 'Diffusion of Innovation Theory', Database for Advances in Information systems, 26, 2&3, 1995, 20-41

From Jos van Hillegersberg

van Hillegersberg, J., Kumar, K. and Welke, R.J. (1996) 'Directions in Object-Oriented Systems Development Research' In Dias Coelho, J., Jelassi, T., König, W., Krcmar, H., O'Callagham, R. and Sääksjärvi, M. (Eds.) Proc. of the 4th ECIS: European Conference on Information Systems, pp. 633-646. Lisbon

From Alan Dennis:

Fjermestad, Jerry and Hiltz, Starr Roxanne (1999) 'An Assessment of Group Support Systems Experimental Research: Methodology and Results' JMIS 15, 3 (Winter 1999) 7-150, Abstract (From the Editors' Introduction to the Special Issue: "The bulk of this issue is devoted to a single paper, "An Assessment of Group Support Systems Experimental Research: Methodology and Results," by Jerry Fjermestad and Starr Roxanne Hiltz. This landmark paper summarizes the methods and results of the whole corpus of experimental GSS studies published in the English language in refereed academic journals. ... Fjermestad and Hiltz offer a classification scheme for the dependent and independent variables in these experiments, examining which are initial causes, which are intervening variables, and which are outcome variables. Besides their exhaustive bibliography, they offer clear, readable tables summarizing each experiment, and detailing research methods, hypotheses, and results for all 200 experiments. The paper takes a first pass at making sense of the GSS literature").

From Mimi Hurt:

Elam J., Huber G. & Hurt M. (1986a) 'An Examination of the DSS Literature (1975-1985)' In Sol H. (Ed.) 'Decision Support Systems: A Decade in Perspective', Amsterdam: North Holland, 1986

Elam J., Huber G. & Hurt M. (1986b) 'An Examination of DSS Content in Major IS conference Proceedings (1980-1985)' in Proc. Int'l Conf. Infor. Syst., 1986

From Eleanor Wynn:

Orlikowski W.J. & Baroudi J.J. (1991) 'Studying Information Technology in Organizations: Research Approaches and Assumptions' Info. Sys. Research 2, 1 (March 1991) 1-28

Systematic Review

From Clare Atkins:

I am wondering if what you are referring to is what the medical and health community term 'systematic reviews'. This is the use of a formal methodology to gather all the literature relevant to a particular question, critically appraise that literature against pre-determined guidelines appropriate to the research method(s) that have been used and to synthesise the findings of the highest quality research that has been discovered during the process. Statistical techniques, referred to as meta-analysis are used as part of that synthesis if they are relevant.

The results of these systematic reviews and the methodology for creating them (which was developed by the University of York in the UK) as used by the medical and healthcare community, is maintained by the Cochrane Collaboration. Systematic reviews are the basis for the culture of evidence-based practice that has been gaining ground in these communities. Any on-line search for Cochrane or evidence based medicine will provide a wealth of results!

Dr Gail Louw and I presented a paper at the European Conference on Information Systems(ECIS2000) in Vienna in July 2000, which proposed investigating the application of the evidence-based concept to Information Systems. The use of systematic reviews would clearly be an essential ingredient in this. The ideas were met with some enthusiasm and as a result a group of us have formed to take the initiative further.

We are currently looking at adapting the Cochrane systematic review methodology for use with IS and looking for assistance to create critical appraisal guidelines for various kinds of qualitiative research methods and to trial the methodology with pilot reviews. We have not yet posted a notice to ISWorld although that is definitely on our list!

If you would like to know more about our initiative you might like to visit our website which can be accessed by navigating from the webpage of the Centre for IS Research and Development at Massey University.


The Cochrane Reviewers' Handbook 4.0 (Updated July 1999), at

Atkins C. & Louw G. (2000) 'Reclaiming Knowledge: A Case for Evidence-Based Information Systems' Proc. 8th Euro. Conf. Info. Syst., Vienna, July 2000, 39-45


From Carol S. Saunders:

This technique seeks to articulate the paradigms underlying extant theory and use the uncovered multiple paradigms to create even richer theoretical bases for understanding the phenomenon being studied. Lewis and Grimes (1999) suggest a three-phased model to explore variations in the assumptions of alternative paradigms, gain insights into the multiple paradigms and address emerging themes and the resulting theories. The three phases are:

Carol and colleagues have submitted a tutorial proposal for ICIS'2000, which is available on request to her.


From Carol S. Saunders and Traci A. Carte:

Lewis, M.W. & Grimes, A.J. (1999) 'Metatriangulation: Building Theory from Multiple Paradigms,' Academy of Management Review, 24(4), 672-690

xamaxsmall.gif missing
The content and infrastructure for these community service pages are provided by Roger Clarke through his consultancy company, Xamax.

From the site's beginnings in August 1994 until February 2009, the infrastructure was provided by the Australian National University. During that time, the site accumulated close to 30 million hits. It passed 65 million in early 2021.

Sponsored by the Gallery, Bunhybee Grasslands, the extended Clarke Family, Knights of the Spatchcock and their drummer
Xamax Consultancy Pty Ltd
ACN: 002 360 456
78 Sidaway St, Chapman ACT 2611 AUSTRALIA
Tel: +61 2 6288 6916

Created: 14 July 2000 - Last Amended: 16 July 2000 by Roger Clarke - Site Last Verified: 15 February 2009
This document is at
Mail to Webmaster   -    © Xamax Consultancy Pty Ltd, 1995-2022   -    Privacy Policy