The benefits of e-business performance measurement systems

The Benefits of e-business Performance Measurement Systems

This page intentionally left blank

The Benefits of e-business Performance Measurement Systems A report for CIMA – the Chartered Institute of Management Accountants Research Project No. CO: 1754848 David Barnes Royal Holloway, University of London and

Matthew Hinton Open University Business School

AMSTERDAM • BOSTON • HEIDELBERG • LONDON NEW YORK • OXFORD • PARIS • SAN DIEGO SAN FRANCISCO • SINGAPORE • SYDNEY • TOKYO CIMA Publishing is an imprint of Elsevier

CIMA Publishing is an imprint of Elsevier Linacre House, Jordan Hill, Oxford OX2 8DP, UK 30 Corporate Drive, Suite 400, Burlington, MA 01803, USA First edition 2008 Copyright © 2008, David Barnes and Matthew Hinton. Published by Elsevier Ltd. All rights reserved. The right of Jan Bebbington to be identified as the author of this work has been asserted in accordance with the Copyright, Designs and Patents Act 1988 No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means electronic, mechanical, photocopying, recording or otherwise without the prior written permission of the publisher Permissions may be sought directly from Elsevier’s Science & Technology Rights Department in Oxford, UK: phone (⫹44) (0) 1865 843830; fax (⫹44) (0) 1865 853333; email: [email protected]. Alternatively you can submit your request online by visiting the Elsevier web site at http://elsevier.com/locate/ permissions, and selecting Obtaining permission to use Elsevier material Notice No responsibility is assumed by the publisher for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or from any use or operation of any methods, products, instructions or ideas contained in the material herein British Library Cataloguing-in-Publication Data A catalogue record for this book is available from the British Library Library of Congress Cataloging-in-Publication Data A catalog record for this bookv is available from the Library of Congress ISBN: 978-1-85617-525-8 For information on all CIMA Publishing publications visit our web site at books.elsevier.com Typeset by Charon Tec Ltd (A Macmillan Company), Chennai, India www.charontec.com Printed and bound in Great Britain 07 08 09 10

10 9 8 7 6 5 4 3 2 1

Contents Researchers’ Contact Details

vi

Acknowledgements

vii

List of Abbreviations

viii

Executive Summary

ix

Chapter 1. Introduction

1

Chapter 2. Literature Review

7 15

Chapter 4. Individual Case Study Summaries

21

Chapter 5. Cross-Case Analysis

41

Chapter 6. Discussion

55

Chapter 7. Conclusions and Recommendations

63

References

69

Literature Generated from the Project (to Date)

75

Contents

Chapter 3. Research Methodology

v

Researchers’ Contact Details

Researchers’ Contact Details

David Barnes School of Management, Royal Holloway University of London, Egham, Surrey, TW20 0EX, UK, Tel: 01784 414348, Email: [email protected]

vi

Matthew Hinton Open University Business School, Walton Hall, Milton Keynes,, MK7 6AA, Tel: 01908 654737, Email: c.m.hinton@ open.ac.uk

Acknowledgements The authors would like to acknowledge the support of the Chartered Institute of Management Accountants (CIMA) and our two institutions, Royal Holloway, University of London and the Open University Business School (OUBS). Also our thanks to the various case study collaborators for their time and assistance. Finally, thanks are due to the anonymous reviewers of the initial draft of this report for their comments.

Acknowledgements vii

Prelims1-9781856175258.indd vii

4/11/2008 2:22:58 PM

1 Introduction

Ch001-9781856175258.indd 1

5/19/2008 5:09:47 PM

The Benefits of e-business Performance Measurement Systems

Electronic business (e-business) is the use of Internet-based information and communication technologies (ICTs) to conduct business (including sharing information, maintaining relationships and conducting transactions) within and between organisations (Poon and Swatman, 1999). [Although some authors have sought to distinguish between the terms e-business and e-commerce (e.g. Chaffey, 2002), e-business has now become the widely used term for any business use of Internet-based ICTs.] Over the last decade organisations of all kinds have rushed to join the online e-business community. By 2005 e-business was commonplace in British business practice. According to the European Commission, British use of the Internet in business was the highest in Europe with over 98% of British firms having some kind of online presence, with e-business accounting for 14% of business turnover (E-business watch, 2005). Internet use amongst the general population is amongst the highest in the world, with over 63% of the UK population having access (Internet World Stats, 2006). Whilst this has undoubtedly fuelled the growth of business-to-consumer (B2C) e-business, it is business-to-business (B2B) transactions that accountant for most of the financial value of e-business. In 2005, for every £100 of sales over the Internet, £25 was made by households, whilst £75 was by businesses (Office for National Statistics, 2005).

3

In order to participate in the new online business environment, businesses have had to make significant financial investments, not only in the necessary technologies but also in the processes and people necessary to operate them. Such investment might be justified on the basis of the increased efficiencies available from online operations or from improved effectiveness available from complementarities (i.e. from bundling of products and services), from lock-in (i.e. increasing switching costs) and from novelty (i.e. innovative products, new business models, etc.) (Amit and Zott, 2001). However, whether evaluating e-business investment proposals or monitoring the resulting online business operations requires the existence of an appropriate performance measurement system. Whilst many e-business researchers have argued that new kinds of performance measurement are needed for e-businesses (e.g. Tonchia, 2002), it is far from clear how, or even if, existing models of performance measurement need to be modified to make them suitable for the online environment. What little empirical research

Ch001-9781856175258.indd 3

5/19/2008 5:09:47 PM

The Benefits of e-business Performance Measurement Systems 4

has been conducted in this field has pointed to widespread dissatisfaction amongst e-businesses with their existing measures (Marr and Neely, 2001), the lack of any consensus amongst practitioners as to which measures are effective for measuring e-business performance and has questioned the accuracy of such measures that are in use (Hinton and Barnes, 2005a). There is an evident need to identify the features of an effective e-business performance measurement system. This report describes research that aimed to do this. The approach it followed was to study the performance measurement practices of organisations that seemed to have had some success in developing performance measurement systems suitable for the online environment. These were considered to be potentially exemplar organisations, which should be suitable for investigation as case studies. The research aimed to understand the exact nature of the performance measurement systems and practices in organisations with distinctive e-business performance metrics. From this it was hoped to identify a set of best practice recommendations. Additionally, the research also sought to determine: ◆ ◆

◆

◆

◆

The benefits that accrue from the use of these systems. The origins and process of development of e-business performance measurement systems. Any theoretical basis of e-business performance measurement systems. The links between the performance measurement systems for e-business and traditional business. The impact of specific e-business performance measurement systems on business performance.

The main body of the report is structured as follows. This Introduction is followed by a Literature Review that seeks to set the research in the context of the existing state of knowledge in the field of e-business performance measurement systems. It draws on work from the e-business, IT and performance measurement literature streams. The Research Methodology chapter outlines the approach to data collection and analysis taken in the research. It points up the advantages and limitations of the case study method adopted by the research. The empirical results are described in

Ch001-9781856175258.indd 4

5/19/2008 5:09:47 PM

The Benefits of e-business Performance Measurement Systems

the following two chapters, Individual Case Study Summaries and Cross-Case Analysis. The Discussion chapter draws the analysis of the case study findings together, discussing these in the light of current literature and the research methodology. The report closes by presenting its Conclusions and Recommendations.

5

Ch001-9781856175258.indd 5

5/19/2008 5:09:47 PM

2 Literature Review

Ch002-9781856175258.indd 7

5/19/2008 5:10:00 PM

The Benefits of e-business Performance Measurement Systems

As noted in the Introduction, the widespread adoption of e-business has necessitated significant financial investments by the affected organisations. Typically, the introduction of e-business has also resulted in changes in business processes and operating procedures. It might have been expected, therefore that there would have been a widespread concern about whether these investments have yielded the expected improvements in performance. More fundamentally, with the significant changes to business presaged by the advent of e-business, one might have expected some kind of debate about the adequacy of existing performance measurement systems for the online operating environment. As Straub et al. (2002: 117) argues “the unique characteristics underlying the Web may in some cases require new metrics, or at least the careful evaluation of existing ones to facilitate the development of innovative solutions to emerging problems”. One might expect to see a burgeoning literature discussing the need for distinctive performance metrics in e-businesses and reporting attempts to develop and implement these. It is therefore surprising to discover a relative dearth of academic literature in the field. Marr and Neely’s (2001) study of performance measurement practices in e-businesses remains a rare example of empirical research. Their study paints a picture of e-businesses measuring many different dimensions of performance. Yet, they report near universal dissatisfaction with existing measurement systems. This leads the authors to “question the appropriateness of existing performance measurement systems in today’s (digital) economy” (Marr and Neely, 2001: 214).

9

Recent years have seen something of a revolution in performance measurement “away from the traditional dominance of financial performance measures and towards the use of ‘portfolio’ approaches [.. and .. ] more holistic approaches that encompass both operational and strategic performance” (Holloway, 2001: 167). As Bourne et al. (2000) note “in the late 1980s and early 1990s, this dissatisfaction [with traditional backward looking and internally focussed accounting-based performance measurement systems] led to the development of “balanced” or “multi-dimensional” performance measurement frameworks. These new frameworks placed emphasis on non-financial, external and forward looking performance measures. They have included the performance measurement matrix (Keegan et al., 1989), the performance prism (Neely et al., 2002),

Ch002-9781856175258.indd 9

5/19/2008 5:10:00 PM

The Benefits of e-business Performance Measurement Systems 10

and the Business Excellence model (EFQM, 2005). However, all these frameworks have been overshadowed by the dominance of one particular model – “The Balanced Scorecard” (BSC) of Kaplan and Norton (1992). Marr and Schiuma’s (2003: 680) claim that the BSC is “the most influential and dominant concept in the field” is given weight by Neely’s (2005) review of recent performance measurement research. He notes its impact on practice, citing research showing that anything between 30 and 60 per cent of firms have adopted the BSC in some form, and on academic research, in which Kaplan and Norton’s writings on the BSC have dominated the citations in articles on performance measurement in the leading academic journals for the last decade. The BSC was first presented as a tool for organising performance measures into four key areas (the financial perspective, the customer perspective, the internal process perspective and the innovation and learning perspective). Since then, it has grown into a device for controlling the implementation of strategy (Fink et al., 2005) by linking the performance measures to organisational strategy and goals (Kaplan and Norton, 2000). It has become one of the preferred strategic performance management tools of many prominent public and private sector organisations (Radnor and Lovell, 2003). Given this dominance, it might be expected that performance measurement systems developed for use in leading e-businesses would also be based on the BSC. As a minimum it might be expected that an e-businesses performance measurement system would be based on a range of measures, other than the usual financial ones, and would seek to link e-business decisions and actions to organisational objectives and strategy. There are some criticisms of the rush to adopt the BSC and of the increased attention to performance measurement more generally. These centre on the costs associated with measuring what can often amount to literally hundreds of things at any one time. As Neely and Austin (2000) note, there might be something of a “measurement crisis is looming – measurement madness… the problem is that society is obsessed with measurement.” Neither, as Franco and Bourne (2003) argue, is there much evidence to support the assertion that the BSC or other performance measurement systems have much of an impact on organisational performance. Some researchers have offered evidence questioning the extent

Ch002-9781856175258.indd 10

5/19/2008 5:10:00 PM

There have been some attempts to adapt the BSC to measure and evaluate e-business. For example, Plant et al. (2003) follow the logic of the BSC to derive four additional perspectives (brand, service, market and technology) to the traditional customer perspective of the BSC. These new perspectives, they argue, are necessary to develop a comprehensive set of customer perspective metrics for e-business. It is, however, worth noting that their concern is primarily for business-to-customer (B2C) e-business. Bremser and Chung (2005) present a framework for developing performance measurement metrics in the e-business environment, which combines the BSC methodology with existing taxonomies of e-business models. However, this framework has yet to be tested empirically so it is not possible to determine whether it genuinely offers a way for firms to develop new metrics which the authors assert “are needed to implement e-business strategies and tactics”.

The Benefits of e-business Performance Measurement Systems

to which the BSC and its counterparts have been embraced in practice. For example, Johnston et al. (2002) found that successful large organisations tend to simplify their approach to performance measurement in order to concentrate on understanding the key drivers of business performance rather than worrying about the detailed application of performance measurement frameworks. In their review of the literature on performance measurement in the small- or medium-sized enterprise (SME) sector, Garengo et al. (2005) note that resource constraints tend to mean that small businesses tend to implement performance measurement systems in a fairly ad hoc fashion, limiting the range of measures used and rarely taking a holistic approach.

11

Reviewing the wider e-business literature, it is possible to identify a number of factors that it is claimed that an e-business performance measurement system should be concerned with the following: The performance of the website There has been a particular focus amongst some e-business researchers (usually from marketing disciplines) on developing measures to evaluate e-buisness websites. This evaluation is usually undertaken from the perspective of the user of the website – the customer. Building on the underpinning from the SERVQUAL methodology (Parasuraman et al., 1988), Zeithaml et al. (2000) propose

◆

Ch002-9781856175258.indd 11

5/19/2008 5:10:00 PM

The Benefits of e-business Performance Measurement Systems 12

a framework for understanding how customers evaluate websites. They identify eleven factors that customers use in their evaluations: access, ease of navigation, efficiency, flexibility, reliability, personalisation, security/privacy, responsiveness, assurance/ trust, site aesthetics and price knowledge. Barnes and Vidgen (2001) also build on this to develop their website evaluation questionnaire, WebQual. They conflate the criteria for website evaluation under three headings: usability, information quality and interaction quality. The performance of business processes Although measures to evaluate the design and performance of a website are important, setting up a website is perhaps the easiest part of e-business. Many companies have found that fulfilling the orders generated from their website can be problematic because their business processes are not sufficiently efficient to seamlessly pass on order information and complete the order without error and/or delay (Cox and Dale, 2001). Unless the goods and services required by customers are produced efficiently and delivered effectively, neither customer satisfaction nor profitability can be achieved (McGuffog, 1999). Accordingly, Siaz et al. (2002) argue that an e-business performance measurement system should focus on the management of business processes. It should enable the integration of intra-organisational processes to be measured, facilitate co-operation and promote information sharing via the Internet between the organisation and its supply chain partners, and measure the level of IT integration within the organisation. However, this is likely to be problematic as the degree of integration between existing processes and e-business processes varies widely in the various new e-business models that have emerged. Consequently, it is difficult to disaggregate and measure the contribution of each, a point often acknowledged with respect to wider information and communication technology (ICT) investments (Epstein and Rejc, 2005). As such, creating a single benchmark or set of all encompassing performance measures for e-business applications may not be feasible (Jutla et al., 1999). Nonetheless, process measures are one of the key metrics for e-business (Hinton et al., 2003) and particularly those concerned with efficiency. Wu et al. (2003) suggest that measures of efficiency in e-business should include the costs of production and transaction, general management activity,

◆

Ch002-9781856175258.indd 12

5/19/2008 5:10:00 PM

co-ordination with supply chain partners, marketing and customer acquisition. The Benefits of e-business Performance Measurement Systems

The performance of customers As Hinton et al. (2003) have also argued, e-business performance measures need to encompass end-state measures that enable the organisation to monitor its ability to satisfy its customers. Wu et al. (2003) suggest measures of sales performance (market share, sales volume, prices, numbers of new customers, numbers of customers retained), customer satisfaction (the extent to which customers are satisfied, encourage purchases from others, are loyal) and relationship development (the strength of the relationship with a customer, the likelihood of the relationship enduring) are required. It is argued that there is a strong correlation between customer retention and subsequent organisational growth and profitability. Reichheld and Schefter (2000) believe that profitability can be increased by as much as 85% through an increase in customer loyalty of as little as 5%, and they also contend that this ratio is even higher for Internet-based transactions. A further aspect of customer performance that has been subject to the application of performance metrics is their behaviour online as they interact with a website. Minocha et al. (2004) stress that the current research into the human–computer interface is primarily focussed on the usability of the core website. However, it is important to stress that a customer’s experience with e-business “extends beyond the interaction with the website. Other features such as security in credit card handling, delivery of products, post sales support via email, or with call centres, and so on influence customers’ perceptions of quality and value” (Minocha et al., 2004). It is clear that metrics which only measure website usability will not in themselves explain customer performance. Instead, customer performance needs to be understood in relation to the broader service delivery system (Voss, 2003).

◆

13

Linking e-business performance to business strategy Many writers have argued (e.g. Chang et al., 2003) that the adoption of e-business is an important strategic initiative for any organisation. Indeed, without a strategic imperative it may be difficult to justify the necessary investments. As such, there is a need for an e-business strategy that offers “an integrated co-ordinated and top-down approach towards e-business projects” (Rangone et al., 2002: 274). It

◆

Ch002-9781856175258.indd 13

5/19/2008 5:10:00 PM

The Benefits of e-business Performance Measurement Systems

is only through the strategic use of the technologies of e-business that any sort of competitive advantage can be secured (Porter, 2001). It therefore seems imperative that any performance measurement system in e-business is linked to the requirements of an organisation’s business strategy, and that performance measures are able to monitor e-business performance against strategic business objectives. In summary, the main points that emerge from the literature review are: ◆

14

◆

◆

◆

Ch002-9781856175258.indd 14

There is general recognition that the features of an effective performance measurement system include a portfolio of measures (including non-financial as well as financial measures) and a link to organisational strategy. The BSC is the best known and, reportedly, the most widely used performance measurement framework. However, there are concerns about the costs of installing and operating increasingly comprehensive performance measurement systems and doubts about their ensuring benefits and their impact on performance. Also some studies suggest that in practice some organisations, especially small businesses, limit the scope of their application of performance measurement frameworks. Despite the widespread adoption of e-business within British business (and the fact that the United Kingdom is amongst world’s leaders in e-business), there seems to have been surprisingly few reports of organisations adopting their performance measurement systems for the online environment. Neither does there seem to have been much interest from theoreticians in the development of e-business performance measurement systems. Although there is no clear guidance on what an effective e-business performance measurement system might look like, it might be expected to encompass a range of measures, including those concerned with the performance of the website, the performance of e-business processes and performance of customers. It might also be expected to seek to link e-business decisions and actions to organisational strategy and consequently be linked to the performance measurement system for all other organisational activities. It seems likely that organisations may be concerned about the difficulties and costs of adapting existing performance measurement systems for e-business, and what benefits may ensue.

5/19/2008 5:10:00 PM

3 Research Methodology

Ch003-9781856175258.indd 15

5/19/2008 5:10:12 PM

A case study method, based on qualitative data obtained from key informants within the organisation, was deemed the most suitable approach. Additional data came from company documents. The case study method is well suited to investigating contemporary practices within organisations (Yin, 1994). Qualitative case study research concentrates on the depth and quality of data gathered. This makes it potentially very strong in terms of both construct and internal validity. On the other hand, it can be criticised as lacking in external validity due to the limited number of sites that can be studied within the usual constraints of time and other limiting resources. However, it should be remembered that unlike quantitative research that can use statistical inference to generalise from a sample to a larger population, qualitative research relies on logical inference whereby case studies are generalisable to theoretical propositions and not populations (Yin, 1994).

The Benefits of e-business Performance Measurement Systems

The overall aim of the research was to identify the features and benefits of an effective e-business performance measurement system. The most appropriate research strategy was therefore thought to be one based on a study of organisations that had some success in developing performance measurement systems suitable for the online environment. These were arguably exemplar organisations that might be expected to display good practice in their approach to e-business performance measurement. It was thought that a close examination of such organisations should prove fruitful.

17

Determining the number of organisations to be investigated is always problematic in case study research. For this project, practical considerations were paramount as the work had to be completed within 12 months. In addition to such concerns, Eisenhardt (1989) argues that no further cases are necessary once “theoretical saturation” has been achieved. By this, she means the point at which incremental learning from any additional cases is minimal because the researchers are observing phenomena seen before. She argues that no more than 10 cases are necessary for most research projects. However, in order to ensure that organisations of many different kinds were included in the sample, it was decided to set a target number of 12 organisations. Gathering data of sufficient quality for this study required the identification of appropriate informants within suitable organisations.

Ch003-9781856175258.indd 17

5/19/2008 5:10:12 PM

The Benefits of e-business Performance Measurement Systems 18

Previous survey-based research undertaken by the authors had identified a number of individuals who claimed that their organisations were operating performance measurement systems suitable for their e-business operations (Hinton and Barnes, 2005b). In their responses to the postal questionnaire used in the survey, they had provided contact details to indicate their willingness to be contacted to participate in follow-up research. In the event, this approach resulted in nine managers participating in the study. As this number fell short of that desired, an additional group of organisations were targeted. This comprised the 2005 winners of the UK government’s annual national DTI E-Commerce Awards. There are a number of different categories for these awards, but all winners must demonstrate that they have achieved tangible gains by using the Internet or other Information and Communications Technologies. This group was thought to be worthy of investigation as they should comprise leading edge practitioners in e-business. They were asked if they would be prepared to discuss their approach to performance measurement within e-business. This resulted in an additional three companies agreeing to participate in the study. It should be noted that the case companies selected for study are not intended to be a sample in any statistical sense. The choice was purposive in that they were thought to be potential exemplars, in that they had some success in developing e-business performance measurement systems. However, the cases selected could also be considered to constitute a convenience sample to some extent, in that they were chosen from amongst organisations known to the researchers. To be included the case companies had to meet two criteria; namely, to be suitable e-businesses and to have managers willing to provide access to facilitate data collection. Although the research design does not necessitate the sample to contain any particular combination of organisations of any specific type or size, it is worth noting that in the event, the case organisations contained examples from both the public and private sector, manufacturers and service providers, large multinational quoted companies and small owner-managed businesses. The principal method of data collection in the research was via faceto-face interviews with a key informant in each organisation. The

Ch003-9781856175258.indd 18

5/19/2008 5:10:12 PM

The questions were designed to facilitate data collection on the following themes: ◆ ◆ ◆

◆ ◆

◆

◆

◆

The background to the company’s use of e-business. The company’s aims and objectives in its use of e-business. The company’s capital investment appraisal process for e-business. The performance metrics the company uses for e-business. Links with between performance metrics for e-business and traditional business. The origins and development of the company’s e-business performance measurement system. The benefits obtained from the company’s e-business performance measurement system. Any shortcomings of the company’s e-business performance measurement system.

The Benefits of e-business Performance Measurement Systems

interviews were semi-structured as this enables the researchers to explore interesting avenues for investigation as they emerge. Such interviews, particularly when conducted at the interviewees’ workplace enable the researcher to get close enough, not only to gather factual data, but also to gain an understanding of actions and meanings in their context (Bryman, 1988). A questioning framework was developed for use by the interviewers. Such frameworks are often used in this type of qualitative research, in order to focus and bound the discussions (Miles and Huberman, 1994).

19

Ten managers were interviewed in this way. With the permission of the subjects, all interviews were tape-recorded for subsequent transcription to facilitate data analysis. Interviews were typically of between one and two hours duration. All interviews were conducted in 2005. Although the use of a single respondent per case can reduce the reliability of the data gathered about each organisation (Bowman and Ambrosini, 1997), this risk had to be offset against the number of interviews that could be conducted and analysed in the time available for the research. Limiting the depth of study of case was felt to be a price worth paying in order to broaden the range of organisations studied. Amongst the organisations offering to participate there were a number of managers who were based outside the UK or in remote

Ch003-9781856175258.indd 19

5/19/2008 5:10:12 PM

The Benefits of e-business Performance Measurement Systems 20

areas. Because of the difficulties of travel and an inability to find mutually convenient times, face-to-face interviewing proved impossible. Accordingly, these individuals were invited to contribute to the research via email. They were provided with a questionnaire template, designed to prompt their responses to the key interview themes. Two managers made submissions in this way. Although use of a second data collection method can be challenged on the grounds of consistency, it was felt that some sacrifice of some methodological purity was worthwhile in order to increase the number of cases studied. In any event, as these two cases were chronologically the last to be studied, they were used largely to confirm the findings from the previous ten. By this stage the study was reaching a high level of theoretical saturation (Eisenhardt, 1989), as discussed above. The two data collection methods used resulted in written evidence that could then be analysed, either in the form of detailed transcriptions in the case of the face-to-face interviews or written submissions in the case of the email respondents. The approach to data analysis was to identify the main points and themes emerging from the datasets. A descriptive narrative was then produced for each case organisation under a set of broad set of topic headings. Finally a cross-case analysis was undertaken in order to deepen understanding and explanation, and as a means of drawing conclusions from all the data from the study (Miles and Huberman, 1994). This approach to data analysis is one which provides a structure within which to become intimately familiar with the data from each case which should help yield the depth of understanding necessary to offer meaningful interpretations (Eisenhardt, 1989).

Ch003-9781856175258.indd 20

5/19/2008 5:10:13 PM

4 Individual Case Study Summaries

Ch004-9781856175258.indd 21

5/19/2008 5:10:31 PM

Materialco* Materialco* is part of a large diversified multinational business. It produces speciality high-quality materials that are used in a number of different high-technology applications. Materialco’s use of e-business is evolving quite rapidly. Firstly, some of its most important customers have been adopting online bidding for contracts. Secondly, the company has been moving to the electronic exchange of documents for quality audit documentation. Although these changes are customer driven, Materialco has been proactive in developing its customer portal. Customers have access to full details of their order book and can track progress through the complete production process.

The Benefits of e-business Performance Measurement Systems

Each of the cases is briefly described below. It is important to note that the case descriptions refer to the situations prevailing in the organisations in 2005. In the dynamic environments of e-business, changes can and do happen remarkably quickly. Where requested by the organisation, a pseudonym (marked *) has been used and some of the case details disguised to ensure anonymity.

23

Capital investment appraisal follows a very rigid structured financial assessment process which addresses what return on investment can be achieved, how quickly it can pay back and what extra business it gives. The same philosophy is applied for all capital investments whatever their size. The investments in the enhancements to the company’s e-business provisions were justified on the basis of delivering a higher level of customer service without having to employ extra staff. Materialco does not have separate measures for e-business performance rather it is integral to its normal business operations. Changes to the quality management system in the last two years have made it easier to get a measure of customer satisfaction. There are now regular customer satisfaction questionnaires. This provides a number of simple measures: number of deliveries, number of concessions or rejections, number of paperwork queries, late deliveries and then an overall rating score. Customers can respond online.

Ch004-9781856175258.indd 23

5/19/2008 5:10:31 PM

The Benefits of e-business Performance Measurement Systems 24

Materialco also measures the aspects of e-business processes and production flow through its factory. The company provides internal customers with online, real-time data about the production process. For example, production engineers involved in different parts of the production process are able to look at each other’s workflows. This could eventually enable the customers to access detailed information about progress of their order within the entire production process. Performance benchmarks are both customer determined (e.g. delivery performance, quality performance) and internal (e.g. adherence to plan). Internally, production performance is measured online at every stage of production process against the weekly plan, in terms of time, volume and cost. Materialco’s use of e-business has not changed the performance measures it uses. Its measures for performance within the production process are fairly crude. It uses a series of measures, largely based on adherence to plan, which is based on financial targets. E-business has probably contributed to the company taking more notice of what its customers measure its performance against. Different customers have different measures of performance. Some count delivery times, not delivery to plan. Others count paperwork errors as being a discrepancy or a measure of performance. They have not developed any additional or different measures as a result of moving to e-business. Environmental performance measures have become more important recently, and e-business has made this more transparent. It is now much easier to monitor emissions and energy consumption. This helps in obtaining the necessary environmental approvals. E-business gives a much higher visibility to all aspects of business.

Telecom services* Telecom Services is part of the business services and solutions division of a large telecommunications company serving customers worldwide with a broad portfolio of ICT products. Telecom Services offers a range of consulting, design, implementation and in-life management services to support business customers in their choice and usage of its products. These services assist customers

Ch004-9781856175258.indd 24

5/19/2008 5:10:31 PM

The company’s approach is to develop specific products to satisfy each customer’s requirements. Product development is done by a team and the use of product teams is the way that most new products are launched and controlled. Teams are required to have five year financial plans. However, because technology develops very quickly in this industry, product lifecycles can be quite short. Therefore it is not sensible to plan for sales of specific products more than eighteen months to two years into the future. So, for budgetary purposes, sales income from new products is assumed to depreciate on a straight line basis over two years for DCF calculation purposes. The product development process is driven by electronic management tools (Livelink and ARIS) and managed through the use of online measurements. These measures broadly address cost, quality and time considerations. Progress is monitored at every stage via the Internet.

The Benefits of e-business Performance Measurement Systems

in quantifying their immediate requirement, exploring their infrastructure requirements and producing a business case to support these requirements.

25

Product quality is assessed via a series of “quality gates”. If a product fails to reach the right quality at any one gate, it will be aborted. The metrics used to assess product development are broadly equivalent to the traditional performance measurements but the way they are measured has changed significantly. Automation of this process has collapsed the timeframe for decision-making. Timing is important as it is essential to offer the product to the marketplace at the right point. In the past, performance was measured more on how many products were launched, irrespective of whether they hit the price point for the target. Now there is almost an expectation that if you get half way through and stop, you are actually saving the firm money by not spending further money. The main benefit of the company’s performance measurement system is the fact that anybody can see it. This transparency allows managers to see how their staff are performing online and intervene if necessary. Anybody on a team can see almost everything and so see how well they are doing. Project information is far easier to update and far more organised. The only real downside to the performance measurement systems used by the department

Ch004-9781856175258.indd 25

5/19/2008 5:10:31 PM

The Benefits of e-business Performance Measurement Systems 26

is the excessive amount of information provided and the general problems associated with online working as an inadequate discussion medium.

Coverpoint Coverpoint is based in the City of London serving the insurance industry by enabling insurance products to go online. It deals purely in business-to-business (B2B) insurance. The system it provides is essentially an Internet-based tool that enables insurance businesses and their channel partners (insurer, broker, agent) to quote, bind and manage insurance products online in real time. A key feature is that it incorporates RBS Trust Assured digital signature capability such that each step is signed electronically aiding security and contract certainty. The company makes investment decisions using a business case approach. When considering whether to put a product online, they look at a business case over time (targeting a pay back over two or three years). They evaluate the performance of these investments post implementation, by tracking to make sure that there is a return. If this is not the case, then efforts are made to enhance the income. Costs are fairly well-known when making an investment of this kind. Broadly, Coverpoint uses very traditional methods to manage performance. It sets sales targets and tries to hit them so the sales teams are sales focussed, accordingly. Many aspects of internal processes are driven by regulation. However, the driver for the company is keeping costs down. The adoption of e-business applications facilitates this. With this in mind, Coverpoint uses performance metrics that are matched to its various online activities. For example, online marketing activity is scrutinised for database efficiency, email traffic volumes and click through rates to their website. Conversion rates of firms signing up for their e-trading platform are also taken. However, measurements used to monitor performance afterwards are very traditional for the insurance industry. Customer quotes are monitored. If customers subsequently buy, Coverpoint then wants to assess whether they will buy again. The company also measures customer retention rates.

Ch004-9781856175258.indd 26

5/19/2008 5:10:31 PM

DLA Piper Rudnick Gray Cary DLA is a commercial law firm, offering advice on day-to-day issues, such as employment, commercial dealings, litigation, insurance, real estate, IT and intellectual property, as well as on more strategic activities such as mergers and acquisitions, banking deals and restructuring. Its clients include major household names as well as many small- and medium-sized enterprises. DLA’s strategy has been to differentiate itself within its market and to change the boundaries of its industry. The move to electronic processes has speeded up transactions and facilitated the instantaneous exchange of huge amounts of data and documentation. Furthermore, cost pressures are driving the commoditisation of any part of the business that can be de-skilled. By endeavouring to be at the forefront of technology DLA believe that this gives it a unique selling point by comparison to its competitors.

The Benefits of e-business Performance Measurement Systems

However, it does not take any direct measurements of customer satisfaction. Because of the B2B nature of this area of insurance the industry is very much relationship driven, so Coverpoint concentrates on maintaining these relationships. There is a range of measures of the technical performance of the website covering common measures such as CPU demand, how many people are using different applications, document transfer rates and so on.

27

DLA operate a number of online services. These cover a range of applications to support its own staff, generic help and support for clients, and portals offering industry-specific services (such as a property management portal). These systems offer different functionality from simple information sharing through to document sharing and collaborative working. DLA have found that it is difficult to determine the benefits derived from the performance measures currently in place. It measures ICT primarily in terms of deliverables such as system availability, transaction processing volumes, etc. Whilst it is possible to see how quickly transactions are undertaken, it is far more difficult to calculate how that adds value. DLA claims that it is able to link ICT performance metrics to company aims and objectives, and so it also attempts to measure the

Ch004-9781856175258.indd 27

5/19/2008 5:10:31 PM

The Benefits of e-business Performance Measurement Systems 28

strategic value of ICT. However, it is difficult to set traditional performance measures for ICT because it offers so many intangible benefits. This is very different to DLA’s approach to performance measurement in the rest of its business. It is normally very bottom line focussed and everything is costed out precisely. Every aspect of the business is examined in microscopic detail. It is possible to ascertain the cost of every single transaction as every minute of work time is accounted for (subject to everyone recording their time properly). ICT performance tends to be viewed slightly longer term. However, there has been no real attempt to modify these financial performance metrics in the light of the introduction of e-business. E-business tends to be seen as a service, something to have and to use as part of normal business activity. Accordingly, performance measures have become slightly more difficult and DLA now recognises the need to measure progress against business goals, as well as technical performance. It is easier to analyse other capital investments, because they are more easily understood as part of the business. Consequently, post-investment analysis in ICT is only done informally.

Softco* Softco is a software house with nearly thirty years of expertise in development and deployment of legacy software for enterprise platforms. Its software enables organisations to exploit their legacy assets by modernising their applications with contemporary architectures and web services. This enables them to reduce costs and increase flexibility, thereby extracting maximum value for money from their existing people, processes and technology. Softco’s customers span both the public and private sectors. The company has around 500 employees worldwide, with offices in the United Kingdom, United States and Japan. Softco has two main motivations for its use of e-business: customer relationship management and internal integration. A backcloth to this is the desire to reduce costs across all areas of the business. In looking to build stronger relationships with its customers, Softco uses the Internet to facilitate customer contact. It makes

Ch004-9781856175258.indd 28

5/19/2008 5:10:31 PM

The other motivation is to integrate the company by using the Internet to bring together the key departments of Sales, Marketing and Development. Internal examples of use of web-based technology include Sales, who have streamlined their administration by taking away paperwork, by installing an expenses management system and Human Resources which has introduced online systems that give employees more flexibility access to information, and cutting administration costs.

The Benefits of e-business Performance Measurement Systems

most of its sales to large business customers, selling either directly or through partners. Little is sold directly over the Internet. The main marketing use is in provision of information to boost brand and product awareness in the marketplace. The company also provides support and education through the web for existing customers, who can log on to a secure website. For example, customers can query their own reported instances and they can also licence any Softco products they have purchased. Softco also makes use of webcasts to interact with customers as well as sales and support staff in the field.

29

Softco does not undertake detailed capital investment appraisal. However, there are financial measurements for specific ICT investments using a conventional budgeting approach to manage expenditure. After installation, the impact of an investment is measured from information gleaned from the management information system. Softco does not have any specific performance measurements for e-business. At a corporate level the company is measured on EBITD (earnings before interest, taxation and depreciation). It also has a series of key performance indicators (KPIs). One of these is cost of sales, which has dropped considerably in the last three years. Turnover has gone up, whereas cost of sales has dropped markedly. This is unlikely to be all due to e-business, but there have certainly been considerable achievements in reducing the cost of sales, a lot of which seems to be related to infrastructure and systems that the company now has and the greater use of ICT. Most KPIs are financial. There are some tactical level measures. For example, the level of attendance at webcasts is measured. However, no explicit attempt is made to measure the use of e-business. The company’s metrics are focussed at the corporate level, with individual departments devising measures for their own purposes. Although the

Ch004-9781856175258.indd 29

5/19/2008 5:10:31 PM

The Benefits of e-business Performance Measurement Systems 30

company is making more use of web-based technologies, there is no specific attempt to measure how e-business is helping the company (and the various departments) to meet targets.

Organisation for economic co-operation and development The OECD is an inter-governmental organisation funded by its 30 member countries who share a commitment to democratic government and the market economy. It has relations with some 70 other countries, business and labour organisations and a wide range of non-government organisations. The OECD provides a forum in which governments can work together to address and identifying policy solutions for the economic, social and environmental challenges of globalisation. The OECD’s e-business activities are concerned with information sharing between the Secretariat (its core staff) and its member country governments (mainly ministries and delegations). For this purpose, the OECD introduced an Online Information System (OLISnet) a few years ago to better connect and share knowledge between its member countries. Through this service, more than 10,000 authorised officials can now access a wealth of information on policy and committee work, simply using an Internet browser over OECD’s private network. This includes access to more than 140,000 documents and publications, a range of latest published statistical products, as well as committee agendas, email alert services, committee discussion groups, selected information from its Intranet and website. OLISnet’s clients are committee delegates and other officials in national administrations and international organisations and agencies who, for business purposes, need to stay informed on committee work. Member country delegations approve access requests from their officials and non-government entities. Access requests from international organisations and non-member country officials/ non-government entities are approved by the corresponding directorate(s). There are no subscription or usage charges. Using the organisation’s private Internet network, clients have secure access to extensive meeting information, agendas, documents and

Ch004-9781856175258.indd 30

5/19/2008 5:10:31 PM

The OECD’s main approach to performance measurement in its use of OLISnet is via the Information Technology and Network Services Directorate’s periodical conduct of surveys to find out what its clients think of OLISnet and how they would like to see their interaction with the Secretariat improved. This approach links the organisation’s e-business performance measures to its aims and objectives for its use of e-business to a large extent, since the results of any surveys carried out on the OLISnet system are reported to the Executive Directorate and may be used to feed into any future plans for IT developments. This approach to performance measurement in e-business also fits with the OECD’s approach in its traditional business. In both cases, the OECD relies on the use of surveys with its member countries to assess its performance, judging both the quality and impact/potential impact of its results.

The Benefits of e-business Performance Measurement Systems

other material (according to their authorised access profile), as well as to publications and statistical products. They participate in discussion groups and other services which facilitate their interaction with the Secretariat on committee-related issues. As a secure channel for exchange of information on committee work-in-progress, OLISnet is the essential complement to the public dissemination of information about OECD activities on the Organisation’s website.

31

Port of London Authority The Port of London Authority (PLA) is a Public Trust, which works in the interests of all its stakeholders in the community. It is run on a commercial basis with any surplus invested back into the Port. The PLA’s main mission is to: ◆ ◆

◆

facilitate the safety of navigation on the tidal Thames, deliver value for money services to commercial customers and promote the potential of the Port of London, respect the environment of the tidal Thames and pursue principles of sustainable development.

The PLA does not itself operate cargo handling facilities but supports the operations of others. There are more than 70 independently owned wharves, terminals and port facilities along the

Ch004-9781856175258.indd 31

5/19/2008 5:10:31 PM

The Benefits of e-business Performance Measurement Systems

tidal River Thames. Vessels sailing to and from these port facilities – and all other craft on the tidal Thames – operate under the jurisdiction of the PLA, which is the statutory harbour, pilotage, navigation and conservancy authority for the tidal Thames. The PLA uses e-business technologies in a number of ways. Extensive use is made of a system called PACE that holds all the information about ships’ manifests. Customs agents and people with an interest in that information, principally freight forwarders who act as agents for the owners of the cargo, can access it and input Customs declarations electronically. The shipping lines provide the manifest information which is downloaded from their own computer system straight into the PACE system. Polaris (Port of London Authority River Information System) handles requests for pilots at particular points on the river. Ship agents enter information directly into Polaris via a webpage, saving a lot of re-keying work and allowing the more effective allocation of pilots.

32

The PLA also operates an extranet, providing information about the river, especially concerning environmental responsibilities, sites of special scientific interest and emergency procedures. The PLA’s main aim in its use of e-business and Internet technology is time-saving. It gets no public funding, but needs to raise funds by charges on shipping, which need to remain competitive, or customers could be driven away. For example, online vessel arrival and departure notification through PACE is attractive to agents because it saves them time and reduces errors in data transfer. The PLA does have a set of performance metrics that are based around its mission statement, but this performance measurement regime does not extend far into its e-business activities. There are a number of measures for PACE including the number of customers and the amount of time the system is available. The most important measure of performance of the PACE system would be the volume of cargo that moves through the port and is on the system. However, this is impossible to measure or verify. The PLA obtains regular feedback from its customers, although with its close knit customer community the absence of negative feedback is generally taken to be positive. In addition, it employs

Ch004-9781856175258.indd 32

5/19/2008 5:10:31 PM

Capital investment appraisal is driven through the annual budget process. Spending priorities are set by the Board, the Chief Harbour Master and the Executive Committee who examine bids and determine which projects to fund based on risk and customer demand. E-business proposals are considered alongside all other spending bids. Post-investment appraisal is used to identify the causes of any overspends.

10ticks.co.uk 10ticks.co.uk is an online business selling maths learning support materials (mostly workbooks) that comply with the UK’s National Curriculum for schools. 10ticks’ products can be bought by schools, for teachers to use in class or to set homework, or by parents, for use by children who may need additional support outside of school. The company’s products now cover all National Curriculum topics for Years 7–9, Intermediate and Higher GCSE syllabus. 10ticks is qualified for e-learning credits, which means that schools can pay for their software from a separate additional government funding scheme that is aimed at improving standards through approved software. Workbooks are purchased online and provided in electronic format, so that purchasers make their own hard copies.

The Benefits of e-business Performance Measurement Systems

a market research company to elicit independent customer feedback. The PLA have regular meetings with port users, shipping agents and major terminal operators. These provide forums for exchanges of views. The PLA’s ISO 9000 quality system also records complaints.

33

Although 10ticks is not really a technology company it could not be successful without technology. It could not have afforded to publish the workbooks as traditional books. So the technology has facilitated a new business model, which also enables 10ticks to provide ongoing support for its various customer groups. The 10ticks’s website has three distinct areas, each designed for the three customer groups: teachers, parents and children. For example, a child can get online help telling them how to complete each workbook and providing multiple choice questions to consolidate their learning. Children can track their progress over time. In the teacher area, once logged in there is a teacher-to-teacher chat

Ch004-9781856175258.indd 33

5/19/2008 5:10:31 PM

The Benefits of e-business Performance Measurement Systems 34

room. The aim is to create supportive communities in each area. In addition, 10ticks has developed an online facility that provides teachers with summary sheets of the marks that all of the pupils in their class have obtained on the 10ticks exercises. 10ticks does not engage in any formal capital investment appraisal. Its business model is to invest in the creation of intellectual property and then to minimise running costs. 10ticks has a lot of performance measures, particularly on the customer side of its operations. The website provides real-time data on web-traffic. Daily figures are collated for comparison purposes. These measure how many people have visited the different areas of the website, so that the company knows which are the most popular. There are also lots of statistics available from the server, e.g. the number of successful requests each day, where they are coming from, IT addresses, sales reports and so on. The company knows how many visitors it has; the challenge is to turn them into revenue. 10ticks does not keep any customer information on individual pupils or students, believing that there would be too much information to cope with. Financial measurement is done by looking at the monthly targets and comparing them year on year focussing on turnover. Turnover is used because 10ticks knows what the break even is for each month because most costs are fixed. The costs are very low and the company would be very profitable if it were not reinvesting. 10ticks keep a record of who has bought what. From the knowledge of what each school has bought, it is possible to work out how many children are using it from the demographics of the school. From the invoice records, the company knows whether customers have paid or not. The most important measure used to monitor performance is monthly turnover. This is monitored on a live spreadsheet. The company does not formally monitor cash flow or study the bank accounts, because this is cash rich business with no debts.

Torbay Hospital Torbay Hospital is a large general hospital serving the South Devon area. It is run by the South Devon Healthcare Trust, covering 300 square miles and a local population of 270,000 people. There are

Ch004-9781856175258.indd 34

5/19/2008 5:10:31 PM

The hospital is intending in the near future to order its materials through a system called Logistics Online. All available materials are listed in an online catalogue and ordered using an e-procurement system. In every ward, material stocks are checked weekly using bar code scanning. Materials are reordered manually at agreed predetermined levels for each ward. The system relies on NHS Logistics (the NHS’s supply arm) to physically deliver the goods. Logistics Online is based on web technology. Once an item is listed on the system, it can be ordered, receipted and authorised electronically by relevant ward staff. This enables the managers to go into the system and immediately see what has been ordered. Logistics Online provides better reporting and improved accountability. It should also prevent orders being lost. The capital investment required for the change to Logistics Online is minimal as the system uses much existing hardware and cabling. The system should enable the materials management team to take over all the materials ordering throughout the hospital, thereby alleviating the need for medical staff to do the ordering. This would allow nurses to spend more time looking after patients rather than having to order materials themselves. There is no intention to achieve cost saving in the ward by reducing the numbers of nurses or other staff, but they will be able to spend more time on clinical matters. Payback is seen in terms of increased patient through flow and improvements to patient care. However, this is difficult to measure.

The Benefits of e-business Performance Measurement Systems

six General Theatres at Torbay Hospital which undertake approximately 11,000 elective and urgent inpatient and day-case procedures each year.

35

NHS Logistics monitors various aspects of performance (e.g. ontime delivery performance). However, there is less measurement of material management performance within hospitals. Much of the NHS approach to performance measurement is based on government imposed clinical targets (such as patient waiting times). Consequently, it is difficult to discern a direct link between treatment times and materials management performance. Torbay hospital has a local measurement system for monitoring materials management performance. This flags any materials

Ch004-9781856175258.indd 35

5/19/2008 5:10:31 PM

The Benefits of e-business Performance Measurement Systems

shortages in the wards. The performance of the hospital’s materials management staff is monitored in terms of delivering materials to the wards. However, there is now a facility to request what materials are purchased and carried as stock by NHS Logistics. There are no specific performance measures for the e-business components of the materials procurement activity, but there are measures which try to capture performance of processes in which Logistics Online is embedded. In addition, the hospital is trying to capture longitudinal information about materials management performance via regular surveys of end-users (such as ward managers and other key staff). The intention is to assess the impact of the implementation of the Logistics Online system over time. The hospital itself has a target of making certain cost savings each year, based on the annual budget. The budget identifies cost-saving targets for all parts of the hospital, not just procurement.

36

Wealth Bank* Wealth Bank is a subsidiary global financial services group with around 70,000 employees worldwide. Wealth Bank itself is based in Germany, with approximately 1,000 employees. Wealth Bank serves the private banking business sector with financial services for wealthy customers. Wealth Bank services the market through a network of approximately 350 client advisors in Germany, who distribute its services. Its main focus is businessto-consumer (B2C) but it has recently started to make use of other financial institutions and financial intermediaries to use B2B channels as well. Although it remains a traditional bricks and mortar bank, Wealth Bank has developed two main e-business activities. Firstly, its clients can view their accounts and portfolios online via the bank’s website. However at the moment, clients cannot execute their own transactions on the website as this function is disabled technically. Transactions have to be executed by a Wealth Bank client advisor who has been fully trained on the system. Secondly, it has established a Wealth Bank product platform “bank-for-banks”. This enables financial intermediaries and other banks, typically smaller

Ch004-9781856175258.indd 36

5/19/2008 5:10:31 PM

or retail banks to get access to the full range of Wealth Bank’s products and services.

With respect to capital investments, the bank uses a net present value (NPV) approach to assess its e-business investment proposals. It also uses NPV in post-investment appraisal to compare actual against budget. Wealth Bank benchmarks against its competitors in the private wealth management and banking sector in measuring the performance of its business processes are used in e-business. Currently, it measures customer performance in e-business through the quantitative measurement of clicks on its website. It applies the same measures to its performance measurement in e-business as those used in traditional business. Its measures are underpinned by the Balanced Scorecard.

The Benefits of e-business Performance Measurement Systems

The company’s motivation for adopting e-business was the attraction of reducing the costs per transaction and increasing speed of response. It also aims to improve customer service by providing online access to real-time data for its clients. This should improve customer satisfaction, thereby increasing customer loyalty and retention. Also, the creation of the bank-for-banks platform can provide a channel for further growth.

37

LG Union* LG Union is one of Britain’s biggest trades unions with over a million members. The union provides a wide range of services to its members, including training and development for both the full time and volunteer workforces. The union’s objectives in adopting web-based technology are largely concerned with being able to provide a more proficient service to members and to be much more efficiently organised. The technology can reduce operating costs and enable its peripatetic staff to operate more efficiently. Enabling them to operate from home can eliminate many hours of driving, thereby increasing their ability to respond and resolve conflicts much quicker. If the union is seen as effective it enables more members to be recruited and improves staff motivation.

Ch004-9781856175258.indd 37

5/19/2008 5:10:32 PM

The Benefits of e-business Performance Measurement Systems 38

LG Union has both national and regional websites supporting the various elements of its geographic organisational structure. These support a range of information dissemination and remote communication facilities. This includes the decentralised membership databases, underpinned by ERP software. The union has also been experimenting with a number of other online initiatives, including the case management system, the booking of travel, print purchasing and access to the union’s journals. With respect to capital investment appraisal, the union does not review any investments in ICT in any way differently from other types of investments, adopting payback methods. However, investments in ICT do not command as high a level of attention that they probably deserve, as they tend not to be seen as strategically important. Accordingly, there is little attempt to evaluate changes to working practices that result from ICT investments. The use of performance measures within the organisation is complicated. The key measure of performance is membership growth. Other key measures are concerned with the effectiveness of both the processes and the outcomes of negotiations, and how effective the union is in protecting its members. Success can also be measured in some aspects of the union’s work in terms of its dealings with government, for example in whether certain laws have been changed and how much influence the union can have on the political agenda. The union has IIP, and uses this to review the performance of teams and individuals. There is a complex set of measures of performance, across a number of aspects on around 10 factors. However, there does not seem to be any linkage between these performance measures and the use of e-business. One of the major objectives of the union is concerned with the efficiency of the delivery of its services to members. The use of e-business is strongly related to that objective. The union conducts numerous surveys of staff, asking about its performance as an organisation. It also evaluates extensively what its members think, as part of IIP. The various surveys are used to quantify aspects of the union’s performance. These are used to indirectly make assessments of their use of e-business.

Ch004-9781856175258.indd 38

5/19/2008 5:10:32 PM

Trainingco*

Trainingco exploits e-business in two ways. Firstly, it uses the Internet within its own operations. It uses a software package to help with the management of its projects, particularly in the management of the virtual teams. Secondly, it incorporates e-learning technologies into its training products. A recent example was the creation of a virtual learning environment for a large government agency.

The Benefits of e-business Performance Measurement Systems

Trainingco designs and develops bespoke training solutions for corporate clients. It offers a “one stop shop”, being able to design training products and interventions in all media including paperbased text, multimedia CBT and online programs. It can also deliver face-to-face training courses and provide expert consultancy for a wide range of clients in both the private and public sectors. It is a small company, with only eight core employees. However, it has around 250 associates, who are generally selfemployed, from whom it can assemble virtual teams as required for each project. All Trainingco’s products are tailor-made for each client.

39

Trainingco does not have any formal method for appraising capital investments in new technology. It outsources its ICT support and maintains full backups because the data is so vital to its business. It makes investments in new machines when people complain about their functionality or speed. ICT is just viewed as being a vital part of the business. However, even in such a small company, there is a problem of lack of integration, with the same information being inputted and stored on three or four different systems. Trainingco does not appear to have any form of performance measurement for its internal operations. Because of its size many aspects of internal performance are evaluated informally. Each project is managed using the PRINCE2 process-based project management method. The extent to which it is used depends on clients. Some are more rigorous in its application than others. Trainingco sees PRINCE2 as offering a very good solid base on which to operate, but its selection of tools depends upon the circumstances of each project. Financial performance measures are the most important ones for driving the performance of the business.

Ch004-9781856175258.indd 39

5/19/2008 5:10:32 PM

The Benefits of e-business Performance Measurement Systems

For each project, costs are constantly reviewed and projections made. Monthly internal financial reviews are held for each project, to discuss progress and particularly how much can be invoiced for it. Time overruns on projects are very expensive. Trainingco always conducts a post-project review with the client to determine what went well, what went badly and how they could improve. It evaluates the extent to which learners have successfully transferred their learning into the workplace, what impact this has had on their key performance indicators and the return on investment. However, it is very difficult to separate out the e-business component of the overall project performance.

40

Ch004-9781856175258.indd 40

5/19/2008 5:10:32 PM

5 Cross-Case Analysis

Ch005-9781856175258.indd 41

5/19/2008 5:10:48 PM

Organisational types

It is possible to categorise the case organisations into three broad sectors. Within the multinational enterprise (MNE) private sector, there are four organisations, namely Materialco, Telecom Services, DLA and Wealth Bank. There are four from the private small- and medium-sized enterprise (SME) sector; Coverpoint and Trainingco are joined by the stock market listed medium-sized business

The Benefits of e-business Performance Measurement Systems

The case studies include examples from across a broad spectrum of organisational types. There is only one manufacturer (Materialco); all the other organisations are service organisations, in that they provide information-based products or services. There is a preponderance of organisations that mainly serve business-tobusiness markets (Materialco, Telecom Services, Coverpoint, DLA, Softco, Port of London Authority (PLA) and Trainingco). OECD serves its government members and other non-governmental organisation (NGOs). 10ticks.co.uk and Torbay Hospital serve both organisational customers and individual consumers. LG Union serves individual consumers, but only those individuals that are its members. Only Wealth Bank serves a consumer market, but this is restricted to wealthy individuals. Only 10ticks.co.uk and Coverpoint could be considered to be pure-play dotcom business; all the others operate in a mixed bricks and clicks environment. These details are summarised in Table 5.1.

43

Table 5.1: Case organisation details Case organisation

Organisation type

Business activities

Customers

Materialco Telecom Services Coverpoint DLA Softco OECD

Materials production ICT services Insurance Legal services Software Policy development

PLA 10 Ticks.co.uk Torbay Hospital

MNE MNE SME MNE Medium sized plc Intergovernmental NFP Public trust Micro-business NHS Trust

Wealth Bank LG Union Trainingco

MNE Trades union SME

Financial services Member services Training services

Businesses Businesses Businesses Businesses Businesses Governments & NGOs Businesses Individuals & schools Individuals & health providers Wealthy individuals Individual members Businesses

Ch005-9781856175258.indd 43

River management Educational products Health services

5/19/2008 5:10:48 PM

The Benefits of e-business Performance Measurement Systems

Table 5.2: Case organisation types

44

Organisational sector

Case organisation

MNE SME Public sector

Materialco, Telecom Services, DLA, Wealth Bank Coverpoint, Softco, 10 Ticks.co.uk, Trainingco OECD, PLA, Torbay Hospital, LG Union

Softco and the micro-business 10ticks.co.uk. The public sector is represented by an intergovernmental organisation (OECD), a public trust (PLA), a National Health Service (NHS) hospital trust (Torbay Hospital), a trades union (LG Union). This three-way split is shown in Table 5.2.

E-business capital expenditure appraisal One area of interest for the research is how organisations set about justifying the financial investments necessary for e-business and in particular what performance measures are used in the process. Some organisations use capital investment appraisal using formal financial methods (such as payback, discounted cashflow (DCF) or net present value (NPV)). These tend to be either large and/or financially sophisticated businesses: Materialco, Telecom Services and Wealth Bank from the MNE sector; PLA and LG Union from the public sector; and Coverpoint from the SMEs. They all treat investment proposals in e-business in the same manner as any other capital investment proposal. Interestingly, DLA, another large and sophisticated organisation treats e-business investments differently than other capital investment proposals. They reason that e-business has so many associated intangible benefits that are impossible to realistically quantify, that formal financial methods are inappropriate. So they take a more informal approach. There appeared, at best, to be only informal assessments in evidence in the other SMEs (Softco, 10ticks.co.uk and Trainingco) and at Torbay Hospital (public sector). Only Coverpoint and Wealth Bank (both of whom are fairly financially sophisticated) claimed to undertake any formal financial post-investment appraisal of their e-business investments. It was difficult to assess the extent to which the claims of the use of some kind of informal post-investment appraisal by other organisations were meaningful. What seems clear is that only

Ch005-9781856175258.indd 44

5/19/2008 5:10:48 PM

Table 5.3: E-business capital investment appraisal

Materialco Telecom Services DLA Wealth Bank

E-business investment appraisal MNEs Same formal financial process as all capex proposals Same formal financial process as all capex proposals (DCF) No formal financial justification (unlike other capex propoasals) Same formal financial process as all capex proposals (NPV)

Softco

SMEs Same formal financial process as all capex proposals (payback) No formal financial justification

10 Ticks.co.uk

No formal financial justification

Trainingco

No formal financial justification

Coverpoint

OECD PLA Torbay Hospital LG Union

Public sector no information Same formal financial process as all capex proposals No formal financial justification Same formal financial process as all capex proposals

Post-investment appraisal no information no information Informal (non-financial) assessment Financial assessment (NPV) actual vs. budget

Financial assessment of returns Informal (non-financial) assessment Informal (non-financial) assessment Informal (non-financial) assessment

The Benefits of e-business Performance Measurement Systems

Case organisation

45

no information Informal (non-financial) assessment Informal (non-financial) assessment Informal (non-financial) assessment

in the case of DLA was any attempt made to treat e-business any differently than other investment decisions. Table 5.3 summarises the approaches to e-business capital investment appraisal taken by the case organisations, grouped according to their sector.

E-business performance metrics It ought to be expected that the purpose of any organisational performance metrics is to track progress against a desired business outcome. So we should examine whether the case organisations have metrics in place appropriate to their e-business aims. This seems to be the case for four of the case organisations. Materialco wants to use e-business to improve customer service. They can

Ch005-9781856175258.indd 45

5/19/2008 5:10:48 PM

The Benefits of e-business Performance Measurement Systems 46

measure this directly through their metrics for customer satisfaction and indirectly by their metrics for delivery timeliness and quality. Telecom Services want to use e-business to improve their new product development (NPD) process and their ARIS system enables them to measure this in terms of cost, quality and time. OECD and LG Union all want to use e-business to improve their service to their customers (or members). In each case they have metrics that enable them to do so. It should be noted that these are all large organisations, with Materialco and Telecom Services from the MNE sector and OECD and LG Union from the public sector. The metrics in a further five case organisations enable them to assess their performance against only some of their e-business aims. DLA aims for faster transaction times and cost reductions from ebusiness. But they only measure transaction times and do not have measures in place to directly assess cost savings from e-business. Softco aims to improve customer service and reduce costs through e-business. They have metrics for costs but not for customer service. PLA aims to use e-business to improve customer service and speed transactions. Whilst customer satisfaction is measured, transaction times are not. Torbay Hospital wants to use e-business to improve materials management. Their metrics for order fulfilment times and stock levels are appropriate for this. However, they also aim for time savings from their staff, but have no metrics in this respect. Wealth Bank aims for cost savings, faster transactions and improved customer service from e-business. Whilst they have measures for the first two aims, they do not have any metrics for customer service other than transaction times. These organisations range across all three sectors, MNE (DLA and Wealth Bank), public sector (PLA and Torbay Hospital) and SME (Softco). The remaining SMEs appear not to have any alignment between their e-business aims and metrics. Trainingco aims to use e-business to improve internal communications and for product support. However, it has no metrics for either of these aims. Coverpoint and 10ticks.co.uk have no explicit aim for e-business. In both cases e-business is the basis of their business, relying on it as they do to facilitate their businesses. As such, it is difficult to see how they could develop a specific appropriate metric for this aim. Table 5.4 summarises these points. A review of the various performance metrics in the case organisations reveals something of a patchwork quilt of performance

Ch005-9781856175258.indd 46

5/19/2008 5:10:48 PM

Table 5.4: E-business aims and metrics E-business aims

Relevant e-metrics

Organisations with e-business aims and e-metrics aligned Materialco

Improve customer service

Customer satisfaction Delivery and quality

Telecom Services

Improved new product development process

Cost, quality, time (through ARIS)

OECD

Improve customer service (through improved information sharing)

Customer satisfaction

LG Union

Improve customer (member) service

Member satisfaction

Organisations with e-business aims and e-metrics partially aligned DLA

Faster transactions and cost reductions (to differentiate)

Transaction process times

Softco

Improve customer service and cost reductions through internal integration

Cost

PLA

Improve customer service Faster transactions

Customer satisfaction

Torbay Hospital

Faster order fulfilment, stock reductions, time saving for workers

Order fulfilment times Stock levels

Wealth Bank

Improve customer service, faster transactions and cost reductions

Cost per transaction Response time

The Benefits of e-business Performance Measurement Systems

Case organisation

47

Organisations with no alignment between e-business aims and e-metrics Trainingco

Improve internal communications Product support

–

Coverpoint

Facilitate business

–

10 Ticks.co.uk

Facilitate business

–

metrics in use. Two of the case organisations (Softco and OECD) measure only one aspect of e-business performance (costs and customer satisfaction, respectively). All the other organisations measure more than one aspect of performance. Only three of the case organisations measure performance on their websites. Perhaps not surprisingly for a web only businesses, both Coverpoint and 10ticks.co.uk measure web traffic. These two

Ch005-9781856175258.indd 47

5/19/2008 5:10:48 PM

The Benefits of e-business Performance Measurement Systems 48

companies, together with Wealth Bank also measure click through rates to analyse the behaviour of their online visitors. Coverpoint also measure the technical performance of their central processing unit (CPU) (e.g. available capacity) to ensure the ongoing availability and functionality of their website. DLA and PLA also make use of performance in relation to the availability of their respective computer systems. All the organisations except two (OECD and 10ticks.co.uk) measure some aspect of their internal business processes in relation to e-business. These typically relate to aspects of cost and/or quality and/or timeliness. Materialco measures production performance in terms of cost, quality and delivery against plan. It also measures the aspects of environmental performance in order to comply with regulations. As their processes are essentially project-based, both Telecom Services and Trainingco use project management performance management software (ARIS and PRINCE respectively) to provide cost, quality and timeliness performance metrics. Coverpoint, DLA and Wealth Bank are more concerned with transaction processing metrics. In Coverpoint’s case, they also need to record such data for regulatory purposes. Softco relies entirely on cost-based metrics. Torbay Hospital’s metrics are concerned with the materials management metrics of order fulfilment times and stock levels. PLA and LG Union’s process metrics are arguably concerned with quality. In PLA’s case this is manifested as measures of data accuracy, whilst LG Union monitors the outcomes of case handling and negotiations with employers. Eight of the case organisation’s measure one or more aspect of customer performance. Materialco, OECD, PLA, Torbay Hospital, LG Union and Trainingco all measure customer (or end-user or member) satisfaction explicitly. Materialco also have measures for delivery timeliness and quality. Coverpoint measures conversion rates (i.e. enquiries to sales) and customer retention rates. PLA also measure the total number of customers. 10ticks.co.uk’s customer metrics of sales value and sales volumes are more financially orientated. LG Union seems to stand alone in having a measure for staff satisfaction. None of the organisations has anything that amounts to a structured e-business performance measurement “system”, as would be the case

Ch005-9781856175258.indd 48

5/19/2008 5:10:48 PM

In the next stage of the analysis, we will examine the relationship between the e-business performance metrics identified above and the metrics used within the organisations’ traditional (i.e. non e-business) activities. In some cases, no new metrics have been introduced as a result of e-business. Rather existing metrics have been used to measure e-business. Materialco, Softco, Torbay Hospital, LG Union and Telecom Services fall into this category. However, in the latter’s case, existing measures have been automated. In two cases (OECD and Trainingco), existing metrics have been adapted in some way to allow their use within e-business. Only in four cases (Coverpoint, DLA, PLA and Wealth Bank) have completely new metrics have been introduced to accommodate e-business. It is worth noting that Wealth Bank claim to have incorporated their new ebusiness metrics within their existing performance measurement system, which is based on the Balanced Scorecard. As a completely online business start-up, all of 10ticks.co.uk’s performance metrics are concerned with e-business activities. There appears to be no sectoral pattern to the relationship between traditional and e-metrics.

The Benefits of e-business Performance Measurement Systems

with performance metrics derived from the Balanced Scorecard or similar underpinning framework. As such it would appear all the case organisations have something of an ad hoc approach to e-business performance metrics. Neither is there much evidence of “balance” between the different metrics in use. It is generally possible to discern some aspect of performance measurement that provides the predominant focus within the organisations. Considering this by sector may suggest a pattern. Financial performance is the main measurement focus for all the SMEs and two of the MNEs (Materialco and Telecom Services). The other two MNEs (DLA and Wealth Bank) focus on process metrics. Three of the public sectors organisations (OECD, PLA and LG Union) have a performance measurement focus on customer (or member) satisfaction. The fourth, Torbay Hospital focuses on the performance of processes. Table 5.5 summarises these points, again grouping the organisations according to their sectors.

49

An examination of the relationship link between e-business performance metrics and higher organisational level metrics also shows a mixed picture. Of those organisations that are using existing metrics for e-business, three have clear links between these measures and organisational level metrics (Materialco, Telecom Services and Softco). The latter use a system of key performance indicators (KPIs)

Ch005-9781856175258.indd 49

5/19/2008 5:10:48 PM

Ch005-9781856175258.indd 50

Mix of measures

Yes

Yes

Yes

Yes

Yes

Case organisation

Materialco

Telecom Services

DLA

Wealth Bank

Coverpoint

Web traffic Click through rates

Click through rates

–

–

–

Web metrics

SMEs Transaction process times (regulation driven)

Conversion rates Customer retention

–

–

–

Customer satisfaction Delivery and quality

MNEs Production cost, quality and time (vs. plan) Cost, quality, time (through ARIS) Transaction process times Transaction process vols. Cost per transaction Response time

Customer metrics

CPU technical performance

–

System availability

Environmental measures (regulation driven) –

Other metrics

Financial

Process

Process

Financial

Financial

Focus of e-metrics

The Benefits of e-business Performance Measurement Systems

Process metrics

Table 5.5: Case organisation e-business performance metrics

50

5/19/2008 5:10:48 PM

No Yes

Yes

No

Yes

Yes

Yes

Softco 10 Ticks.co.uk

Trainingco

OECD

PLA

Torbay Hospital

LG Union

–

–

–

–

– Web traffic Click through rates –

Order fulfilment times Stock level Successful case handling and negotiation (Quality)

Data accuracy (Quality)

– –

Customer satisfaction – (Post-project review)

– Sales values and volumes

– System availability

– Staff satisfaction

Customer satisfaction No. of customers Customer satisfaction End-user satisfaction Member satisfaction

Public sector

Cost, quality, time (through PRINCE2)

Cost –

Customer satisfaction

Process

Customer satisfaction Customer satisfaction

Financial Financial (sales revenue and cash) Financial

The Benefits of e-business Performance Measurement Systems

Ch005-9781856175258.indd 51

51

5/19/2008 5:10:48 PM

The Benefits of e-business Performance Measurement Systems 52

Table 5.6: Case organisation e-business performance metrics relationships Case organisation

E-metrics and traditional metrics

E-metrics and organisational level metrics

No new metrics Materialco Telecom Services Softco

Torbay Hospital LG Union

OECD Trainingco

Existing metrics No new e-metrics Existing metrics automated No new e-metrics Existing metrics No new e-metrics Existing metrics No new e-metrics Existing metrics No new e-metrics

Existing metrics linked to organisational level metrics Existing metrics linked to organisational level metrics Existing metrics linked to organisational level metrics (via KPIs) No link to organisational level metrics No link to organisational level metrics

Existing measures adapted Existing metrics adapted for E-metrics link to e-business organisational level metrics Existing metrics adapted for No link to organisational level e-business metrics Additional measures

Coverpoint DLA PLA Wealth Bank

10 Ticks.co.uk

Existing metrics plus new e-metrics Existing metrics plus new e-metrics Existing metrics plus new e-metrics Existing metrics plus new e-metrics (Based on BSC)

All metrics linked to organisational level metrics All metrics link to organisational level metrics All metrics link to organisational level metrics All metrics linked to organisational level metrics

All new measures All measures are e-business All metrics linked to (online business only) organisational level metrics

to achieve this. In the cases of the other two (Torbay Hospital and LG Union), there is no clear linkage. For the two organisations that have adapted existing metrics for e-business, OECD has links between these measures and organisational level metrics, whilst Trainingco does not. All four organisations that have developed new metrics for e-business (Coverpoint, DLA, PLA and Wealth Bank) have demonstrable links between these metrics and those at organisational level. Table 5.6 summarises these relationships.

Ch005-9781856175258.indd 52

5/19/2008 5:10:49 PM

In summary, the main findings that emerge from the cross-case analysis are: ◆

◆

◆

The case organisations tend to assess investment proposals for e-business in the same manner as any other capital investment proposal. Large and/or relatively financially sophisticated organisations tend to assess investment proposals for e-business in the same manner as any other capital investment proposal using formal financial methods (e.g. payback, DCF or NPV) to do so. The SMEs seem more likely to take a more informal approach to e-business capital investment appraisal. However, evidence from the case of one large and sophisticated organisation (DLA) demonstrates that it is possible to view e-business differently due to the difficulty of quantifying its intangible benefits. Only some of the case organisations had performance metrics in place appropriate to their expressed aims for their use of e-business. Some had some appropriate e-business metrics, whilst others had none. There tended to be more alignment between e-business aims and metrics in larger organisations, but less in small businesses. The case organisations appear to have adopted an ad hoc approach to the development of e-business performance metrics. E-business performance metrics included website and technology performance metrics, e-business process metrics and customer metrics.

Ch005-9781856175258.indd 53

The Benefits of e-business Performance Measurement Systems

The impression that emerges is one in which the organisations have incorporated additional performance metrics for e-business into their existing performance measurement systems, presumably because of a perceived need for these measures. Whether or not the new measures are linked into higher level organisational measures seems to depend on whether existing lower level organisational performance metrics already have such links. It seems clear that the advent of an e-business dimension to an organisation’s business activities has not prompted a reconsideration of the merits or otherwise of organisational performance measurement systems. A sectoral analysis shows that all the MNEs link their e-metrics to organisational level metrics. Whilst the organisations that do not make this link are from the public sector (Torbay Hospital and LG Union) and the SME sector (Trainingco).

53

5/19/2008 5:10:49 PM

The Benefits of e-business Performance Measurement Systems

◆

◆

◆

◆ 54

Ch005-9781856175258.indd 54

All but two of the organisations, measure more than one aspect of performance. However, no organisation had a comprehensive portfolio of e-business performance measurement systems. There tended to be a discernable focus to e-business performance measurement. SMEs seemed more likely to focus on finance measures, MNEs on finance or processes, and the public sector on customer measures. Some of the case organisations relied on existing metrics to monitor e-business performance. Some adapted existing metrics. Some introduced completely new e-business performance metrics. Whether or not e-business performance metrics are linked to higher level organisational measures seems to depend on whether links already exist within an organisation’s performance measurement system. MNEs seemed more likely to be able to link their e-metrics to organisational level metrics. There is no evidence to suggest that case organisations have undertaken a reconsideration of the merits or otherwise of existing organisational performance measurement systems in the light of the advent of e-business into their activities.

5/19/2008 5:10:49 PM

6 Discussion

Ch006-9781856175258.indd 55

5/19/2008 5:11:01 PM

Current literature contains a strong advocacy of the need for performance measurement systems to encompass a range of different measures, including non-financial as well as financial measures. Although there are some sceptical voices, there appears to be a consensus that good practice in performance measurement should be based on the use of a balanced portfolio of measures. It arguably follows that organisations developing performance measurement systems specifically for e-business would have looked to such approaches as the basis for these initiatives. This was born out by the case findings to some extent, in that all but two of the organisations studied were measuring more than one aspect of e-business performance. Examples of the metrics used included those associated with website performance, e-business processes and customers. These were all as might have been expected. However, the case organisations generally appear to have adopted an ad hoc approach to e-business performance measurement. No organisation had a true portfolio of e-business performance metrics, as would have been generated from use of the Balanced Scorecard (BSC) or similar performance measurement framework. There was never any sense of balance between the measures used, as advocated by the BSC and similar frameworks. Indeed any mention of the BSC was notably absent in any discussion about e-business performance measurement. In fact, the BSC was mentioned in only one organisation (Wealth Bank) as an influence. Furthermore, there was often a discernible focus for e-business performance measurement amongst the case organisations. For example, the small- or medium-sized enterprises (SMEs) tended to focus solely on finance measures and the public sector organisations on customer measures. These findings were in line with some of the descriptive literature that has reported that in practice some organisations, especially small businesses, often limit the scope of their application of performance measurement frameworks, because of concerns about installation and operation costs of more complex performance measurement systems.

The Benefits of e-business Performance Measurement Systems

This chapter will examine and discuss the main case study findings described in the light of the current literature and with regard to the research methodology.

57

The prescriptive literature also advocates that performance measures should be linked to organisational strategy. Thus it might also

Ch006-9781856175258.indd 57

5/19/2008 5:11:01 PM

The Benefits of e-business Performance Measurement Systems 58

be expected that e-business performance measurement systems would seek to link e-business performance both to the organisation’s strategy for e-business and to its broader organisational strategy, thereby linking e-business operations to all other organisational activities. The situation amongst the case organisations here was very patchy. Only some of them had performance metrics in place which seemed appropriate to their expressed aims for their use of e-business. There tended to be more alignment between e-business aims and metrics in larger organisations, but less in small businesses. Only some of the case organisations had e-business performance metrics that linked to higher-level organisational measures. The multinational enterprises (MNEs) seemed more likely to be able to link their e-metrics to organisational level metrics. Thus, there was only limited evidence of the influence of formalised performance measurement frameworks underpinning the development of e-business performance measurement especially in the smaller and less sophisticated organisations. Furthermore, there was little evidence from the case studies that pointed to any of the organisations having undertaken any kind of systematic review or audit of their existing performance measurement systems to determine what, if any, major changes are needed as a result of e-business. Rather, the organisations seem to have adopted an incremental approach to amending their existing performance measurement systems to accommodate the changes in their activities due to the advent of e-business. They seemed to have endeavoured to make as few changes as possible, utilising or, if necessary, adapting existing metrics to monitor e-business performance. There were very few instances when completely new metrics were introduced. Where new metrics were introduced they were simply added to the existing set of performance measures. Only in the case of 10ticks.co.uk were completely new measures introduced; and as a start-up business, this was clearly out of necessity. This reluctance to treat e-business performance measurement differently from traditional business and to embark on major overhauls of existing performance measurement systems was also demonstrated in capital investment appraisal. In only one case (DLA) was the normal capital investment appraisal processes set to one side for e-business investment.

Ch006-9781856175258.indd 58

5/19/2008 5:11:01 PM

One possible explanation is that organisations may be concerned about the difficulties and costs of adapting existing performance measurement systems for e-business, and are not convinced about the benefits that may ensue. Such a stance would certainly be inline with emerging concerns in the literature. There was evidence from some of the cases, particularly the larger organisations to support such an explanation. It would be surprising if managers in these organisations were unaware of performance measurement frameworks such as the BSC. So, it seems likely that use of these to further develop their performance measurement systems for e-business has been considered at least to some extent before rejection, even if temporarily. However, in some of the case organisations, especially the smaller ones, it was far from clear that a decision not to embark on major changes to their performance measurement systems to accommodate e-business had been taken in a very deliberate way. In these instances it seems more likely that little or no consideration was given to the issue. This may have been done out of ignorance or perhaps out of an intuitive recognition that the time and effort required could not be justified at present.

The Benefits of e-business Performance Measurement Systems

The prime conclusion from the investigation is that the organisations studied did not demonstrate the kind of performance measurement practices with regard to their e-business activities that might have been expected if they had followed the recommendations of the prescriptive performance measurement literature. We are therefore left to speculate why this is the case.

59

It may be that practitioners do not view e-business as requiring revolutionary changes in their business practices, including their performance measurement systems. This would accord with a view that e-business is now just business. It is possible that the view from within organisations is that the much heralded impact of e-business was just so much hype. May be that the changes required to accommodate e-business were not so great. May be businesses just took whatever changes were necessary in their strides. It may be that for the case study organisations, the impact of e-business is not yet that important to merit a complete overhaul of their performance measurement systems, and so incremental adaptation and change suffices for the moment. This certainly seems to be the case for all except 10ticks.co.uk, which as a dotcom start-up has a very simple and still evolving performance

Ch006-9781856175258.indd 59

5/19/2008 5:11:01 PM

The Benefits of e-business Performance Measurement Systems 60

measurement system. Changes due to e-business are still on-going and organisations face continuing uncertainty about the impact of changing technology. As such, the prevailing view with regard to e-business performance measurement is one of “wait and see” and “if it ain’t broke, don’t fix it”. May be more significant changes will be required in the future to provide the spur to more radical and comprehensive changes in e-business performance measurement systems. It is often the case that a crisis is required to trigger significant organisational change. Finally, the findings need to be assessed in the light of the methodology used in the research. As discussed in the Methodology chapter, the strength of a case study is that the much greater intimacy afforded by direct contact with key organisational members should increase the validity of the findings. The main caveat to this is that it relies on researchers gaining “full access to the knowledge and meanings of informants” (Easterby-Smith et al., 1991). This is dependant, firstly, on accessing an appropriately informed informant. Care was taken to do this. However, there is always a possibility that an inappropriate or ill-informed contact is involved. Nonetheless, direct contact provides a much greater opportunity to validate the knowledge of an informant than would be the case in more remote data gathering, such as via a randomly mailed questionnaire. Secondly, validity relies on gathering a sufficiency of data. Once again a direct contact, especially with the freedom of questioning offered by a semi-structured interview increases the likelihood of doing this. Thirdly, validity relies on the researcher correctly interpreting the data gleaned from the informant. Again, the case study is superior to other methods in a number of ways. Face-to-face interviews provide the opportunity to clarify responses and pursue interesting lines of enquiry as they arise. Neither questions nor answers are subject to the constraints of a questionnaire form. The acknowledged weakness of case study research is that it can only investigate a limited number of situations. This raises the issue of the generalisability of the findings; that is, the likelihood that “ideas and theories generated in one setting will also apply in another” (Easterby-Smith et al., 1991). In this type of research, this cannot be tested statistically, as one would test whether a sample is representative of a wider population in a survey. Nonetheless

Ch006-9781856175258.indd 60

5/19/2008 5:11:01 PM

Ch006-9781856175258.indd 61

The Benefits of e-business Performance Measurement Systems

the matter can be considered subjectively. The “population” from which our “sample” is drawn was intended to be organisations demonstrating some kind of “good practice” in e-business performance measurement. [“Good practice” could be adjudged against the practices of other organisations, or against some theoretical ideal. In the Literature chapter we have already noted the dearth of reports in either of these categories. Consequently, this research has examined the e-business performance measurement practices evidenced in its case organisations in the light of acknowledged good practice in performance measurement generically.] The cases were, to some extent, chosen by convenience and the willingness of respondents to provide access for data collection purposes. Nonetheless, the case organisations chosen should have provided examples of superior practice in e-business performance measurement when compared to organisations more generally. However, it is possible that the twelve cases investigated are not representative of good e-business performance measurement practice. In this research, like most case study research, the number of case organisations that can be investigated is limited because of constraints of time and other resources. Consequently, seeking to generalise from the findings from this research must be done with extreme caution.

61

5/19/2008 5:11:01 PM

7 Conclusions and Recommendations

Ch007-9781856175258.indd 63

5/19/2008 5:11:13 PM

The Benefits of e-business Performance Measurement Systems

The aim of this research was to identify the features and benefits of an effective e-business performance measurement system. This was done by studying the e-business performance measurement practices of twelve organisations thought to have had some success in developing performance measurement systems suitable for the online environment. Given that e-business has been heralded as being the cause of major disruptive change, we were expecting that these organisations would have given considerable attention to the challenge of adapting their performance measurement system to e-business. This proved not to be the case. We expected these organisations to have at least undertaken a systematic review of their existing performance measurement systems in the light of changes being wrought by e-business. Also, given the publicity associated with the Balanced Scorecard (BSC) and similar attention performance measurement frameworks in recent years, we expected any changes to performance measurement systems to be underpinned by such frameworks. This also proved not to be the case. Although some of the features of the e-business performance measurement systems were characteristic of the BSC, such changes that had been made were essentially ad hoc and incremental in scope. There was a marked reluctance to embark on major overhauls of existing performance measurement systems. This was surprising when considered in the light of the prescriptive literature. Although, as already noted, there are some siren voices that have caution that organisations, especially smaller ones may not always follow so-called good practice recommendations. Whilst this might be out of ignorance, it may also be based on an assessment that the potential benefits on offer from new and possibly more complex performance metrics might not be worth their associated costs.

65

Now it may be that the case organisations are not representative of their intended population; that is of good e-business performance measurement practice. There is no particular reason to fear this. All the case organisations are to some extent successful businesses, operating their e-business activities apparently without major problems. However, if we assume that the cases are representative of current good practice in e-business performance measurement, two major conclusions follow. Firstly, organisations do not currently feel the need to make major changes to their performance measurement system for e-business. Rather they are content to

Ch007-9781856175258.indd 65

5/19/2008 5:11:13 PM

The Benefits of e-business Performance Measurement Systems

make incremental changes. Secondly, it appears that best practice advice in performance measurement, as exemplified by the BSC, may be much less influential than that reported as typical (e.g. Neely, 2005), and less than that its advocates might like, certainly within the e-business environment. Several recommendations can be made as a result of these findings. This research offers an alternative to the received wisdom in the academic literature, namely that the BSC and similar frameworks invariably offer best practice. Indeed, the notion of “best practice” is contestable. Instead, the findings suggest an alternative “good practice” path that organisations can follow successfully. Consequently: Recommendation 1: Organisations should give conscious and deliberate consideration to whether they need to change their performance measurement systems radically to accommodate e-business. On this basis,

66

Recommendation 2: Organisations should foster incremental development of their performance measurement system to accommodate changes in e-business. Recommendation 3: Organisations should work within their existing performance measurement approach, either adapting existing metrics to monitor e-business performance, or (where they exist) simply adding new metrics to the existing set of performance measures. This offers organisations an appropriate approach within existing resource constraints, as well as addressing the growing concern over the costs associated with performance measurement. With this in mind, Recommendation 4: Management accountants should be encouraged to weigh up the relative costs and benefits of the performance measurement system. This research has begun to identify several gaps between the academic literature and current management practice (detailed above), which gives rise to: Recommendation 5: Academic researchers should consider more closely the impact of theory on the process of organisational

Ch007-9781856175258.indd 66

5/19/2008 5:11:13 PM

performance management, as well as the implications of empirical work on existing theories. The Benefits of e-business Performance Measurement Systems

The cases studies in this research offer an increased qualitative understanding of the issues surrounding performance measurement and e-business. However, there is a need to undertake further cases and survey work needs to be conducted in order to expand the range of industry sectors and the scope of performance measurement practice studied. In particular, quantitative methods need to be employed to test the emerging hypotheses suggested by this study, with regard to different organisational types and different performance metrics used in e-business. Also, the adoption, or otherwise, of best practice advice in performance measurement needs to be more critically assessed with respect to the dynamic environment of e-business.

67

Ch007-9781856175258.indd 67

5/19/2008 5:11:13 PM

References

BIB-9781856175258.indd 69

5/20/2008 11:51:25 AM

BIB-9781856175258.indd 71

References

Amit, R. and Zott, C. (2001) “Value creation in E-business”, Strategic Management Journal, Vol. 22, No. 6-7, pp. 493–520 Barnes, S. and Vidgen, R. (2001) “An evaluation of cyberbookshops: the WebQual method”, International Journal of Electronic Commerce, Vol. 6, No. 1, pp. 11–30 Bourne, M., Mills, J., Wilcox, M., Neely, A. and Platts, K. (2000) “Designing, implementing and updating performance measurement systems”, International Journal of Operations and Production Management, Vol. 20, No. 7, pp. 754–771 Bowman, C. and Ambrosini, V. (1997) “Using single respondents in strategy research”, British Journal of Management, Vol. 8, pp. 119–131 Bremser, W. and Chung, Q. (2005) “A framework for performance measurement in the e-business environment”, Electronic Commerce Research and Applications, Vol. 4, pp. 395–412 Bryman, A. (1988) Quantity and Quality in Social Research, Unwin Hyman, London Chaffey, D. (2002) E-business and e-commerce management, Pearson Educational, Harlow Chang, K., Jackson, J. and Grover, V. (2003) “E-commerce and corporate strategy: an executive perspective”, Information and Management, Vol. 40, pp. 663–675 Cox, J. and Dale, B.G. (2001) “Service quality and e-commerce: an exploratory analysis”, Managing Service Quality, Vol. 11, No. 2, pp. 121–131 Easterby-Smith, M., Thorpe, R. and Lowe, A. (1991) Management Research: An introduction, Sage, London E-business watch (2005) Overview of e-business developments – July 2005, European Commission Enterprise and Industry Directorate General, available at http://www.ebusiness-watch. org downloaded 31st January 2006 EFQM (2005) EFQM Excellence Model, available at (http://www. efqm.org), downloaded 15th March 2006 Eisenhardt, K.M. (1989) “Building theories from case study research”, Academy of Management Review, Vol. 14, No. 4, pp. 532–550 Epstein, M. and Rejc, A. (2005) “Measuring the payoffs of IT investments”, CMA Management, Vol. January, pp. 20–25 Fink, A., Marr, B., Siebe, A. and Kuhle, J. (2005) “The future scorecard: Combining external and internal scenarios to create

71

5/20/2008 11:51:26 AM

References 72

BIB-9781856175258.indd 72

strategic foresight”, Management Decision, Vol. 43, No. 3, pp. 360–381 Franco, M. and Bourne, M. (2003) Business performance measurement systems: a systematic review, Proceedings of the 10th EurOMA Conference, Lake Como, Italy. Garengo, P., Biazzo, S. and Bititci, U. (2005) “Performance measurement systems in SMEs: A review for a research agenda”, International Journal of Management Reviews, Vol. 7, No. 1, pp. 25–47 Holloway, J. (2001) “Investigating the impact of performance measurement”, International Journal of Business Performance Management, Vol. 3, No. 2, pp. 167–180 Hinton, M., Barnes, D. and Mieczkowska, S. (2003) “A Framework for Evaluating E-Business Investments in terms of Business Process Performance”, European Conference on Information Technology Evaluation, Madrid, 25–26 September Hinton, C.M. and Barnes, D.L. (2005a) “Towards a framework for evaluating the business process performance of e-business investments”, International Journal of Business Performance Management, Vol. 7, No. 1, pp. 87–99 Hinton C.M. and Barnes D.L. (2005b) Addressing the Failure of E-Business Performance Measurement Systems, 3rd Conference on Performance Measurement and Management Control, Nice, 22–23 September, 2005 Internet World Stats (2006) “Internet Usage Statistics 2005”, available at http://www.internetworldstats.com, downloaded 31st January 2006 Johnston, R., Brignall, S. and Fitzgerald, L. (2002) “‘Good enough’ performance measurement: A trade-off between activity and action”, Journal of the Operational Research Society, Vol. 53, No. 3, pp. 256–263 Jutla, D., Bodorik, P. and Wang, Y. (1999) “Developing Internet E-Commerce Benchmarks”, Information Systems, Vol. 24, No. 6, pp. 475–493 Kaplan, R.S. and Norton, D.P. (1992) “The balance scorecard-measures that drive performance”, Harvard Business Review, Vol. 70, No. 1, pp. 71–79 Kaplan, R.S. and Norton, D.P. (2000) “Having trouble with your strategy: Then map it”, Harvard Business Review, Vol. Sept–Oct, pp. 167–176

5/20/2008 11:51:26 AM

BIB-9781856175258.indd 73

References

Keegan, D.P., Eiler, R.G. and Jones, C.R. (1989) “Are your performance measures obsolete?”, Management Accounting, Vol. June, pp. 45–50 Marr, B. and Neely, A. (2001) “Organisational performance measurement in the emerging digital age”, International Journal of Business Performance Management, Vol. 3, No. 2, pp. 191–215 Marr, B. and Schumia, G. (2003) “Business performance measurement-past, present and future”, Management Decision, Vol. 41, No. 8, pp. 680–687 McGuffog, T. (1999) E-commerce and the value chain, Manufacturing Engineer, Vol. 78, No. 4, pp. 157–160 Miles, M. and Huberman, A.M. (1994) Qualitative Data Analysis, 2nd Edition, Sage, London Minocha, S., Dawson, L., Millard, N. and Roberts, D. (2004) “A Model of Customer’s Behaviour with (B2C) E-Commerce”, 18th British HCI Group Annual Conference – Design for Life, Leeds Metropolitan University, UK 6–10 September 2004 Neely, A. (2005) “The evolution of performance measurement research”, International Journal of Operations and Production Management, Vol. 25, No. 12, pp. 1264–1277 Neely, A., Adams, C. and Kennerley, M. (2002) The Performance Prism: The Scorecard for Measuring and Managing Business Success, Financial Times-Prentice Hall, Hemel Hempstead Neely, A. and Austin, R. (2000) Measuring operations performancepast present and future, Proceedings of the 2nd International Conference on Performance Measurement, Cambridge. Office for National Statistics (2005) Information and Communication Technology (ICT): Activity of UK Businesses, 2004, available at http://www.statistics.gov.uk/downloads/ theme_economy/ecommerce_report_2004.pdf, downloaded 19th January 2006. Parasuraman, A., Zeithaml, V.A. and Berry, L.L. (1988) “SERVQUAL: A multiple-item scale for measuring customer perceptions of service quality”, Journal of Retailing, Vol. Spring, pp. 12–40 Plant, R., Willcocks, L. and Olsen, N. (2003) “Measuring e-business performance: towards a revised balance scorecard approach”, Information systems and e-business management, Vol. 1, No. 3, pp. 265–282

73

5/20/2008 11:51:26 AM

References 74

BIB-9781856175258.indd 74

Poon, S. and Swatman, P. (1999) “An exploratory study of small business Internet commerce issues”, Information and Management, Vol. 35, pp. 9–18 Porter, M. (2001) “Strategy and the Internet”, Harvard Business Review, Vol. March, pp. 63–78 Radnor, Z.J. and Lovell, B. (2003) “Success factors for Implementation of the Balance Scorecard in a NHS Multiagency setting”, International Journal of Health Care and Quality Assurance, Vol. 16, No. 2, pp. 99–108 Rangone, A., Balocco, R., Bassani, P. and Andreoni, M.C. (2002) “A framework to support the formulation of Internet strategy in traditional enterprises”, International Journal of Business Performance Management, Vol. 4, No. 2, pp. 248–278 Reichheld, F. and Schefter, P. (2000) “E-Loyalty: your secret weapon on the web”, Harvard Business Review, Vol. 78, No. 4, pp. 105–113 Siaz, A., Bas, O., Escoto, P. and Franco, R. (2002) “Performance measurement for e-business enterprises”, International Journal of Business Performance Management, Vol. 4, No. 2, pp. 296–315 Straub, D.W., Hoffman, D.L., Weber, B.W. and Steinfield, C. (2002) “Measuring e-Commerce in Net-Enabled Organisations”, Information Systems Research, Vol. 13, No. 2, pp. 115–124 Tonchia, S. (2002) “Editorial”, International Journal of Business Performance Management, Vol. 4, No. 2/3/4, pp. 129–135 Voss, C.A. (2003) “Rethinking paradigms of service – service in a virtual environment”, International Journal of Operations and Production Management, Vol. 23, No. 1, pp. 88–104 Wu, F., Mahajan, V. and Balasubramanian, S. (2003) “An analysis of e-business adoption and its impact on business performance”, Journal of the Academy of Marketing Science, Vol. 31, No. 4, pp. 425–447 Yin, R.K. (1994) Case Study Research, 2nd Edition, Sage, London Zeithaml, V.A., Parasuraman, A. and Malhotra, A. (2000) “A conceptual framework for understanding e-service quality: implications for future research and managerial practice”, Marketing Science Institute Report, No. 00-115

5/20/2008 11:51:26 AM

Literature Generated from the Project (to Date)

BIB1-9781856175258.indd 75

5/19/2008 5:08:48 PM

Journal paper: Barnes, D.L. and Hinton, C.M. (2007) “Searching for e-business performance measurement systems”, Electronic Journal of Information Systems Evaluation, Volume 10, No. 1, pp. 1–8.

Literature Generated from the Project (to Date)

Conference papers: Barnes, D.L. and Hinton, C.M. (2006) Searching for e-business performance measurement systems, European Conference on Information Technology Evaluation, Genoa, Italy, 28–29 September. Hinton, C.M. and Barnes, D.L. (2006) The Benefits of an E-Business Performance Measurement System, Information Resources Management Association International Conference, Washington DC, 21–24 May.

77

BIB1-9781856175258.indd 77

5/19/2008 5:08:48 PM

Index

Index-9781856175258.indd 79

5/19/2008 5:09:02 PM

Academic researchers, 66–7 “Balanced” performance measurement frameworks, 9–10 Balanced Scorecard (BSC), 10–11, 57, 65 Bank-for-banks platform, 36–7 BSC, see Balanced Scorecard Business processes performance, 12–13 Business strategy and e-business performance, linking, 13–14 Business-to-business (B2B) insurance, 26

DLA Piper Rudnick Gray Cary, 27–8, 43, 46, 48 e-business performance metrics, 45–53 EBITD (earnings before interest, taxation and depreciation), 29 Environmental performance measures, 24 Exemplar organisations, 4, 17

Key performance indicators (KPIs), 29, 49 KPIs, see Key performance indicators LG Union, 37–8, 43, 46, 48 Literature review, 9, 14 Balanced Scorecard, 10–11 e-business performance and business strategy, linking, 13–14 performance of business process, 12–13 of customers, 13 of website, 11–12 Logistics Online, 35, 36

Index

Capital investment appraisal, 33, 44–5 Changes, due to e-business, 58, 59–60 Coverpoint, 26–7, 43, 44, 46, 47–8 Cross-case analysis, 20 capital expenditure appraisal, 44–5 determinations, 53–4 organisational types, 43–4 performance metrics, 45–53 Customer performance, 13 Customer satisfaction, 23, 37, 48, 49

ICT, see Information and communication technology Information and communication technology (ICT), 27–8, 29, 38, 39

81

Management accountants, 66 Materialco, 23–4, 43, 45–6, 48 MNEs, see Multinational enterprise “Multi-dimensional” performance measurement frameworks, 9–10 Multinational enterprise (MNE), 43, 49, 53, 58 Net present value (NPV), 37 NPV, see Net present value

Face-to-face interviews, 18–19, 20, 60 Financial investments, 3, 9 see also Capital investment appraisal Financial performance measures, 39–40, 49

OECD, see Organisation for economic co-operation and development Organisation for economic cooperation and development (OECD), 30–1, 43, 46 Organisational level metrics, 49, 52, 53, 58 Organisational strategy, 10, 57–8

Good practice recommendations, 65, 66–7

PACE system, 32 PLA, see Port of London Authority

Index-9781856175258.indd 81

5/19/2008 5:09:02 PM

Polaris (Port of London Authority River Information System), 32 Port of London Authority (PLA), 31–3, 46, 48 PRINCE2, 39 Product development process, 25 Product quality assessment, 25

Index

Qualitative case study research, 17 “Quality gates”, 25

82

Recommendations, for organisations, 66–7 Research methodology, 17 cross-case analysis, 20 data analysis, 19 data collection, 18–19 via email, 20

SMEs, see Small- or medium-sized enterprises Softco, 44, 46, 48 customer relationship management, 28–9 internal integration, 29 performance measurement, 29–30 Staff satisfaction, 48 Telecom services, 24–6, 43, 46, 48 10ticks.co.uk, 33–4, 43, 46, 47–8 Torbay Hospital, 34–6, 43, 44, 46, 48, 49 Trainingco, 39–40, 43, 46, 48 Wealth Bank, 36–7, 43, 44, 46, 48, 49 Website, performance of, 11–12 Widespread adoption, of e-business, 9

Small- or medium-sized enterprises (SMEs), 11, 43, 44, 46, 49, 57

Index-9781856175258.indd 82

5/19/2008 5:09:02 PM