Principal, Xamax Consultancy Pty Ltd, Canberra
Visiting Fellow, Department of Computer Science, Australian National University
Version of 6 June 2001
This paper was prepared as an invited plenary paper for the IFIP TC8 Working Conference on E-Commerce/E-Business, Salzburg, 22-23 June 2001
© Xamax Consultancy Pty Ltd, 2001
This document is at http://www.anu.edu.au/people/Roger.Clarke/EC/EBR0106.html
The slide-set that accompanies the presentation is at http://www.anu.edu.au/people/Roger.Clarke/EC/EBR0106.ppt
The characteristics of e-business are in direct conflict with the implicit assumptions underlying most academic research. Information systems research is challenging enough. The nature of the e-business domain presents even greater difficulties. Appropriate approaches are identified whereby relevant research into e-business can be undertaken. The delivery of real-world value while achieving sufficient rigour to satisfy the guardians of academic standards will, however, remain problematical.
Electronic commerce burst on the scene in the early 1990s (see, for example, Clarke 1993). Of the many disciplines whose techniques can be brought to bear on electronic commerce, Information Systems is an important one (Clarke 1990, 1992). Practitioners of the Information Systems discipline, in searching for appropriate ways to undertake research in the domain, have been confronted by many challenges. This paper's purpose is to identify and discuss those challenges, and propose appropriate ways of conducting research.
The paper commences by reviewing the nature of scientific research, and the assumptions that are implicit in it. It contrasts against this the rather different assumptions inherent in two primary alternative approaches, interpretivism and engineering research. The characteristics of e-business are delineated, and the audience for research in the e-business domain is defined. The preliminary results of an assessment of published e-business research are presented, and the substantial weaknesses in the body of work undertaken to date are identified. Suggestions are made as to how I.S. practitioners can go about addressing the challenges, and improving the quality of research into e-business.
The nature of scientific research has been the subject of many writings. For a discussion and a substantial collection of references, see Clarke (2000a). This section confines itself to a few key observations that enable development of the argument pursued in the rest of the paper.
Following Popper (1959, 1968), a theory is scientific if and only if it generates inferences that are refutable by reference to the real world. Following in particular Kuhn (1962), key features of the process of scientific research are as follows:
The following terms are used in this paper with the following meanings:
Inherent in scientific research are some meta-physical assumptions that are less frequently expressed, but that are nonetheless critical, in particular:
Some corollaries of those assumptions are that:
There are substantial advantages in the scientific approach. At least when applied to social phenomena, however, its inherent assumptions create difficulties. As the information systems discipline has matured, it has increasingly questioned those assumptions, and the techniques associated with scientistic research (e.g. Klein & Lyytinen 1985).
Interpretivism has provided an alternative basis for research, and has become well-accepted in the information systems discipline. It embodies alternative assumptions, which run counter to those of the scientific approach. In particular, these include the following:
A further alternative, usefully described as `engineering' research, focuses away from the social setting, and towards artefacts. These may be designed and constructed, or protoyped and trialled, in order to enable effective interventions to be undertaken in a particular domain; or existing artefacts may be experimented with, in order to identify the limits of their applicability, effectiveness or usefulness.
Since 1993, I have used a series of broad definitions (Clarke 1997a). By 'electronic commerce' and 'e-commerce', I mean the conduct of trading with the assistance of telecommunications and telecommunications-based tools. Correspondingly, 'electronic business' or 'e-business' is the conduct of business generally, with the assistance of telecommunications and telecommunications-based tools. e-Business therefore encompasses a wide range of activities such as e-commerce (Clarke 1993), e-publishing (Clarke 1997b) and electronic services delivery (Clarke 1999). Its scope extends across the business activities of all categories of organisations and individuals, whether undertaken for profit, or as a service to some community.
In recent years, there has been a trend within the U.S.A. to adopt a much narrower interpretation. This restricts it to business enterprises alone, and focuses on their internal processes. See, for example, Alter et al. (2001). Such narrowness seems to have been inspired by the belated adoption of a trade-marked semi-graphical representation of 'e-business' by a major information technology provider. It would be a serious disservice to the discipline if such a narrow and industry-driven interpretation were to become mainstream.
The terms e-commerce and e-business have been applied retrospectively to the use of electronic data interchange (EDI) and value-added networks (VANs) and various forms of on-line services, since the 1980s. The currency of the terms, however, dates from the explosion of public use of the Internet, and of key services enabled by the Internet, particularly email and the World Wide Web. The domain has become considerably segmented during its first decade, in particular through discovery that significantly different factors are at work in the various relationships among businesses, governments, consumers and citizens, giving rise to the B2B, B2C, C2C, B2G, G2B, G2C, G2G, C2G and C2B contexts.
Some key features of e-business distinguish it from most other domains that have been of interest to I.S. researchers. It is new and exciting, and with that comes a lack of circumspection, a very high level of 'marketing hype', and downright exaggeration. This has been a key factor in the casino mentality of the late 1990s, in which dot.com fervour turned investors into gamblers, and encouraged an unworldly belief in the economics of business having been re-written overnight.
The newness of the domain has meant that no-one was well-prepared to participate in it, much less conduct research into it. Awareness and then reflection necessarily precede understanding, general principles take time to be extracted, courses have to be devised, and most of the pioneers are by definition too busy doing it to explain it or train people in it. As a result, a great deal of the early development and application of new technologies has a high 'turkey-factor', and the opportunity for latecomers to apply '20-20 hindsight' is enormous.
Associated with this is the complete lack of stability in the phenomena within the domain. Technologies keep being re-defined. Perceptions of the purposes and potentials of the technologies change rapidly. Participant behaviour is dependent on perceptions at the time a technology or its application is first experienced, but is subject to learning, which in some cases is very brisk. The perceptions of the needs that are being addressed migrates. Technologies diverge, then converge, and products re-combine. Underlying all of this is a high degree of terminological confusion, with technologists and thoughtful observers struggling to apply existing words and phrases, and invent new ones, all the time having their contributions undermined by marketers whose understanding of the technologies is vastly exceeded by their enthusiasm, their imprecision, and their avarice.
A further crucial aspect of e-business is its reach and borderlessness. Beyond the hype, this means to research some significant additional confounding variables. One of these is the difficulty of defining populations, populations segments, and sampling frames. Another is the inability to rely any longer on geographical extent as a proxy for cultural commonality. Suddenly, key cultural factors have to be identified and controlled for, because, if due attention is not paid to the cultural setting(s), the results will be patent nonsense. This is an enormous challenge, given the utter inadequacy of theories of cultural dependency, and the lack of clarity even about the dimensions of what 'culture' means in the contexts of business services and electronic participation.
Despite the challenges, many I.S. academics have grasped the nettle, and attempted to conduct research into many different aspects of e-business. An important question that arises is: what are the audiences that are to be addressed by this research?
A great deal of research in the I.S. discipline, as in many others, is conducted primarily for the purpose of publication in research journals and presentation at research conferences, and hence as a means of progression within the academic profession. The audience is therefore other researchers, and in particular the gatekeepers, i.e. editors, program chairs, referees. and assessors of research grant applications (a group that is conventionally, but rather misleadingly, referred to as one's 'peers').
There are many further categories of people, however, who are interested in the outcomes of research into e-business. Among those desperate for insights are the developers of new technologies; the professionals, managers and individual workers who apply them; and process specialists who devise approaches whereby the technologies can be effectively applied within organisational contexts. Investors are equally eager for information that will enable them to select among the plethora of bright ideas that are in need of capital. A final category of real-world audience is policy analysts, particularly within government and applied social research organisations, but also in associations of corporations, consumers, citizens and workers, and in large corporations.
Those real-world audiences have interests very different from the research community. Their focus is on 'proof-of-concept' prototyping, the development of products and services, demonstration applications, understanding about users' reactions to them, and appreciation of how their features can be communicated to prospects. Particularly once they have developed a significant capital and/or psychic investment, they have a strong preference for the 'right' answers, or at least not the 'wrong' ones. They are uninterested in researchers whose choice of topic, hypotheses, method, analysis and timetable, are dictated primarily by the demands of journal editors.
This leads to a need to analyse several aspects of research that are often left to one side by academics. One of those aspects is the motivation for the research activity. The following alternatives need to be distinguished:
Another distinction that needs to be drawn might be referred to as the focus of the research. The following alternatives exist:
Activities can also be differentiated according to the nature of the research outcomes. The alternative forms they may be in are as follows:
The rigorous research that is attractive to journal editors is most commonly explanatory or perhaps predictive in nature. Normative outcomes must by their nature involve discussion of, and perhaps even adoption of, value-judgements. They are highly desirable for real-world audiences, but anathema to journal editors.
The previous section identified some very substantial challenges that need to be confronted in e-business research. Any researcher who values publishability in major academic journals well above the impact of their work on practitioners is well-advised to conduct their research in other, less dynamic domains.
Keen (1980) and others have drawn attention to the eternal tension between relevance and rigour. Associated with rigour are independence from sponsors' interests, through pure research motivation, explanatory outcomes, and quantitative data, with a preference for the ratio scales that enable the tools of statistical analysis to be brought to bear. Associated with relevance, on the other hand, are instrumentalist motivation, predictive and normative outcomes, and whatever data is collectable.
Despite the serious challenges, some e-business is being published, and not only in refereed proceedings such as those of the premier annual event in Bled (now in its 14th year), but also in journals.
A colleague and myself have been cataloguing e-business research publications in the primary information systems journals. Assessment is not yet complete of the small number of specialist EC journals, primarily the 18 issues of the International Journal of Electronic Commerce since mid-1996, Electronic Markets (in particular the 10 issues since February 1999) and the 6 volumes of Journal of Electronic Commerce Research since February 2000.
Cataloguing of 5 major research journals (MISQ, ISR, Information & Management, Decision Science and JMIS), and the primary 'emergent issues' journal (Commun. ACM) has been completed. The period assessed is that since the explosion of e-commerce in about 1993 until 2000. The following conclusions can be drawn:
The author has reviewed a great deal of research in the e-business domain, as a referee and conference program chair and member. This section explains why he considers that most e-business research is not of high quality, with little of it satisfying journal editors, little of it satisfying non-academic audiences, and almost none of it satisfying both.
The first observation that has to be made is that good research in any social domain that is multi-segmented, young and changing rapidly, is extremely difficult to devise and to implement. Established theories are in short supply; terminology is distorted; such source data as is available is tainted; models of participant behaviour are lacking; the researcher is frequently also a participant and even a protagonist; the populations being studied are poor survey-subjects; the staleness factor makes publication urgent; and the strong demand that exists for popularist reports stimulates premature release.
In addition, the management of research presents challenges that are particularly acute in the context of e-business. These include:
Some of the low quality in e-business research, however, is not excusable on the grounds discussed above. One recurrent poor research practice is the unquestioning application of convenience reference theories such as Hofstede's cultural factors theory and Rogers' innovation diffusion theory. Semi-structured interviews frequently masquerade as case studies, without the combination of depth and breadth that triangulation demands, and that is needed for any validity to be claimed. Meanwhile, consultancy work poses as [action] research.
Many problems are routinely evident in reports on surveys. These include:
Some research practices go beyond the question of quality to that of morality. As a contribution to a panel session at ICIS'2000, Clarke (2000b) identified a raft of ethical issues that arise in relation to the submission of papers arising from research, including matters relating to sponsorship, authorship, 'school of' manoeuvres, the depiction of the research method used, plagiarism, references and citations, depiction of the research's significance, consideration of the research's implications, economic factors, 'political correctness', and choice of submission venue(s).
This paper has portrayed a grim picture. It has identified many difficulties presented by e-business, and drawn attention to a litany of problems that arise even in conventional I.S. research, and which are yet more challenging to master in a new and dynamic context. This section sets out to re-build some confidence in the conduct of research in e-business, but on new terms.
The first observation is that there is progress. As evidenced in the section above on e-business research publications in journals, there is an e-business research literature, however limited; and hence the domain is no longer entirely `virgin territory' or `a green-fields site'.
In a relatively young discipline, focussing on a very new domain, non-empirical techniques have a considerable amount to offer. One is reviews of existing literatures, within the discipline, in cognate disciplines, and in less familiar reference disciplines. Conceptual research (often referred to as 'arm-chair reasoning', like this paper) can provide impetus to research, and assist empirical work to avoid weaknesses.
Other non-empirical techniques that can offer assistance in particular circumstances include the following:
Among the scientific techniques, laboratory experimentation, and where feasible field experimentation and quasi-experimental designs offer prospects.
Interpretivist techniques that are directly relevant to various problems that instrumentalist e-business research can address include:
A number of relevant techniques lie at the boundary of the scientific and interpretivist research approaches, and are used within both traditions. These include:
Finally, engineering techniques offer a great deal to some kinds of investigations, including:
The following suggestions are made about particularly suitable research techniques:
Finally, the observation is offered that, of all of the aspects of e-business research, consideration of implications is the least often addressed. Because of the enormous import of information technology, Clarke (1988) issued a plea for implications to be researched in conjunction with at least applications of technology, and preferably right at the source, in the engineering, experimentation and trial usage phases. The intervening 13 years have seen the power of I.T. explode, the convergence of computing and communications completed, the convergence with content make great advances, location and tracking of devices and people become mainstream, and early steps taken towards the integration of robotics within I.T.
The need for social impact assessment is now seriously overdue, if errors yet worse than those of nuclear scientists and engineers of the first half of the twentieth century are to be avoided. Areas in especial need of attention include:
e-Business research involves focussing a still-young discipline on a very young domain. The challenges of rapid change in technologies and behaviour, combined with inadequacies in research activities, are compounded by the confrontation between the academic demand for rigour and the real-world need for relevance.
It is easy to draw negative conclusions, such as that research will continue to be under-funded and inadequately performed, and that academics at the outset of their careers would be well-advised to choose less exciting domains in which to undergo their research training.
Some positive conclusions are possible, however. e-Business must be perceived as a research domain, and the mistake avoided of regarding it as a new discipline. It needs the insights and methods of established disciplines brought to bear on it. In many circumstances, it is essential that multiple disciplines be carefully integrated within a single research project. This is because existing theories are inadequate to provide a basis for describing, let alone understanding, far less predicting, the behaviour of individuals, organisations and mechanisms such as e-communities, marketspaces and e-enhanced societies.
The research methods devised to address particular research questions need to achieve both depth and breadth of understanding. In general, this requires either a compound technique such as case studies, or multiple complementary techniques, such as semi-structured interviews and surveys. Holistic approaches, and the integration of the insights of multiple disciplines, present special challenges, because they inevitably derive from distinct bodies of theory developed in order to describe rather different phenomena.
For real-world audiences to be attracted and informed, and for support to be gained from them, e-business research must be motivated by instrumentalism. Hence relevance needs to be regarded as the primary objective, and rigour as the constraint, rather than the other way around. This will create challenges for academics who need to achieve a modicum of publication in the highest-ranking and inevitably academically very demanding journals. It is therefore a responsibility of senior academics, who are less seriously impacted upon by the 'publish or perish' imperative, to devise, acquire funding for, and manage, e-business research programs.
Finally, the focus of research must not be limited to technology, applications of technology, and adoption of applications, but must also extend to their impact and their personal, legal, social and economic implications. Descriptive and explanatory outcomes must be complemented by predictive value, and must extend to normative outcomes, in order to inform the vital policy debates that surround e-business.
A substantial list of references is provided at Clarke (2000a).
Alter S., Ein-Dor P., Markus M.L., Scott J. & Vessey I. (2001) 'Does The Trend Toward E-Business Call For Changes In The Fundamental Concepts Of Information Systems? A Debate' Commun. AIS 5, 10 (April 2001), at http://cais.isworld.org/articles/5-10/
Galliers, R.D. (1992) 'Choosing Information Systems Research Approaches', in Galliers R.D. (ed., 1992) 'Information Systems Research: Issues, Methods and Practical Guidelines', Blackwell, 1992. pp. 144-162
Keen P. (1980) 'MIS Research: Reference Disciplines and a Cumulative Tradition' McLean E. (Ed.), Proc. 1st Int'l Conf. Info. Sys. 1980, 9-18
Klein H.K. & Lyytinen K. (1985) 'The Poverty of Scientism in Information Systems' in Mumford E., Hirschheim R., Fitzgerald G. & Wood-Harper T. (Eds.) (1985) 'Research Methods in Information Systems' North-Holland, 1985, pp. 131-161
Kuhn T. (1962) 'The Structure of Scientific Revolutions' Uni. of Chicago Press, 1st Edition 1962, 2nd Edition 1970
Popper K.R. (1959) 'The Logic of Scientific Discovery', Basic Books, 1959
Popper K.R. (1968) 'Conjecture and Refutations' Harper & Row, 1968
Zwass V. (1996) 'Electronic Commerce: Structures and Issues' Int'l J. Electronic Commerce 1,1 (Fall 1996) 3-23, at http://www.mhhe.com/business/mis/zwass/ecpaper.html
Clarke R. (1988) 'Appropriate Research Methods for Electronic Commerce' MIS Qtly 12,4 (December 1988) 517-9, at http://www.anu.edu.au/people/Roger.Clarke/DV/ELSIC.html
Clarke R. (1990) 'Information Systems: The Scope of the Domain', September 1992, at http://www.anu.edu.au/people/Roger.Clarke/SOS/ISDefn.html
Clarke R. (1992) 'Fundamentals of 'Information Systems', September 1992, at http://www.anu.edu.au/people/Roger.Clarke/SOS/ISFundas.html
Clarke R. (1993) 'EDI Is But One Element of Electronic Commerce' Proc. 6th International EDI Conference, Bled, Slovenia, June 1993, at http://www.anu.edu.au/people/Roger.Clarke/EC/Bled93.html
Clarke R. (1997a) 'Electronic Commerce Definitions', January 1997, at http://www.anu.edu.au/people/Roger.Clarke/EC/ECDefns.html
Clarke R. (1997b) Electronic Publishing: A Specialised Form of Electronic Commerce', Proc. 10th Int'l EC Conf., Bled, Slovenia (June 1997), at http://www.anu.edu.au/people/Roger.Clarke/EC/Bled97.html
Clarke R. (1999) 'Electronic Services Delivery: From Brochure-Ware to Entry Points', Proc. 12th Electronic Commerce Conf., Bled, Slovenia, June 1999, at http://www.anu.edu.au/people/Roger.Clarke/EC/ESD.html
Clarke R. (2000a) 'Appropriate Research Methods for Electronic Commerce', April 2000, at http://www.anu.edu.au/people/Roger.Clarke/EC/ResMeth.html
Clarke R. (2000b) `Ethical Issues in the Preparation and Submission of Research Papers in the I.S. Discipline' Panellist's Statement on 'IS Research Ethics', 6 December 2000, ICIS 2000, at http://www.anu.edu.au/people/Roger.Clarke/SOS/ResPubEth.html
Go to Roger's Home Page.
Go to the contents-page for this segment.
Send an email to Roger
Created: 28 May 2001
Last Amended: 6 June 2001
|These community service pages are a joint offering of the Australian National University (which provides the infrastructure), and Roger Clarke (who provides the content).|
| The Australian National University|
Visiting Fellow, Faculty of
Engineering and Information Technology,
Information Sciences Building Room 211
Pty Ltd, ACN: 002 360 456|
78 Sidaway St
Chapman ACT 2611 AUSTRALIA
Tel: +61 2 6288 1472, 6288 6916