Comparing Apples and Oranges? – Measurement Issues in the Academic World

Academic researchers are frequently confronted with the need to measure phenomena that cannot be directly observed – so-called latent constructs. In order to be able to do so, scales are created which consist of various items. In order to measure customer satisfaction, for example, rather than using one single question asking an individual about the perceived level of satisfaction, several questions are used that are subsequently compressed into a single value using statistical tools. The main idea behind this procedure is to cover different aspects of a construct.

Problems arise, however, when researchers modify their scales so that they better match their respective research goals. At first sight this procedure seems reasonable, but the modified scales are no longer comparable to those used by fellow researchers. In other words: the way in which one researcher measures customer satisfaction differs from the way in which another researcher conducts the measurement, in spite of the fact that the object of interest, satisfaction for example, has not changed.

One important aspect of this problem is called “scale purification”, which relates to the removal of items based on judgmental or statistical considerations. In their latest paper titled “A review of ‘scale purification’ in SCM research: statistical vs. judgmental criteria” published in the journal Supply Chain Management: An International Journal (Vol. 22, Iss. 4), Andreas Wieland (Copenhagen Business School), Christian F. Durach (TU Berlin), Joakim Kembro (Lund University), and Horst Treiblmaier (MU Vienna) look at the consequences of such practices in academic research. They describe under which circumstances scale purification is justified and what caveats exist.

In their paper they present a structured list of guidelines for purification decisions and closely investigate the importance of aspects such as reliability, validity, and parsimony of measurement scales. This research mainly targets academics, but it is equally important for students who want to get started with empirical research and practitioners (e.g. in marketing) who use surveys to better understand their customers.

Find out more about International Management at MODUL University Vienna!