Item Response Theory For Psychologists.pdf 5j6j18

Quality of Life Research 13: 715–716, 2004.

715

Book review

Item Response Theory for Psychologists By S.E. Embretson and S. Reise, Lawrence Erlbaum Associates, Mahwah, 2000, 376 pp, ISBN: 0-80582819-2 (pb), price £33.50/$42.50 (paperback) £68.50/$89.95 (hardback).

Relatively recently the importance of Item Response Theory (IRT) has been recognised in the field of quality of life research, and over the last few years there has been greatly increased use of it for in HRQL applications. It offers many advantages for questionnaire design, including powerful techniques for item selection and scale shortening. It enables instruments to be equated or cross-calibrated. It is a powerful tool for exploring such issues as cross-cultural differences. It offers a natural way of creating computer adaptive tests. However, many investigators and the majority of articles published in Quality of Life Research continue to use traditional psychometric methods almost exclusively. Both IRT and traditional methods are concerned with creating and using multi-item scales to evaluate ‘latent variables’ (hypothetical constructs) that cannot be measured directly. However, whereas traditional methods attempt to measure each person’s average response levels, IRT reformulates the problem in of estimating the probability that a person’s response to an item will lie in a particular category. To a statistician, this is equivalent to the contrast between linear regression and logistic regression. Irrespective of any scientific debate about the merits of the two approaches – and there is still some controversy over the roles of traditional methods vs. IRT and the extent to which one can mix them to obtain the best of both worlds – there are at least two factors contributing to the limited use of IRT. The first factor is the lack of IRT facilities in the mainstream statistical software packages. Specialist IRT software must be used, and many of the available programs are both unfriendly and poorly documented. It is also unsettling for new s to find that although there are a number of IRT software packages on the market, many of them only fit a few of the available models – selecting a package ties one in to a

limited range of models. Furthermore, despite the seeming similarity of the models, all too often the computer packages produce substantially different results from the same data set. The second factor is that there has been an absence of easily accessible texts that manage to explain the intricacies of IRT without recourse to complex equations and detailed mathematics. That is the niche that this book sets out to fill – ‘to explain the new measurement theory to a primarily psychological audience . . .(who) have had a first-year sequence in graduate statistics.’ Much of the theoretical development and many of the early applications of IRT were in the educational sector, and this is reflected in most books on the subject. However, since this book by Embretson and Reise focuses on the use of IRT in psychology, many of the examples come from quality of life applications. The book is divided into four main parts. First, a brief but fascinating historical introduction is provided. The second part opens with a chapter that contrasts IRT with traditional methods, and also explains why it has a number of potential advantages. Although many useful points are made, I found this a difficult chapter because at this stage IRT models have not been introduced and therefore remain an abstract concept – there is a sense of being sold something without knowing what it is one is buying. This same approach continues in the next chapter, in which the assumptions of IRT are discussed in detail. Only by page 50 is the one-parameter Rasch model introduced. The third part of the book introduces IRT models in detail, first dealing with binary models for items that have two outcomes. After this, the theory is extended to polytomous models for items with multiple response categories. These are of particular importance in HRQL, because many instruments have items that typically allow fouror five-response levels (such as, ‘Not at all’, ‘A

716 little’, ‘Very much’, ‘A lot’). A wide range of models is described, with examples. This is a useful reference for anyone seeking a simple exposition of the principal models used in IRT work – graded response, partial credit, rating scale and nominal response models. Unfortunately, as the authors acknowledge, it is difficult to use statistical goodness-of-fit methods to compare the various models and one is left without much advice as to how to proceed. ‘Ones philosophy may dictate which model to use … Also, it goes without saying that models will only be fit if appropriate and usable programs are readily available.’ I suspect many readers would welcome some explicit advice about how to select an appropriate model. How should they use their data to decide, or should they be considering the plausibility of the alternative theoretical models? How much does the choice of models matter, in of the end results when, for example, developing new instruments? Similarly, the advice concerning sample size requirements is scant and vague: ‘As shown by results here, some of the category threshold parameters were not well estimated in the graded response model with 350 examinees.’ This is all very well, but readers who seek to apply these methods are left without any real sense of how to proceed. I wonder how Embretson and Reise plan their studies? Many times I wished I could ask, ‘Yes, but what would you do in this situation? If you were to design a new instrument using IRT methods, would you collect data on 200 patients or 2000 patients, or how would you decide?’ In the absence of theoretical research, readers deserve to be given subjective guidance based upon the experience of experts and the current state of knowledge. Also in this part of the book are explanations of the methods for obtaining scores for individual persons, and calibrating the items. As is explained, IRT scoring is not straightforward and the three main strategies are described. For some of the models, such as partial credit and rating scale, it may in fact be adequate to use simple summated scales when scoring individual patients since the rank order is preserved. However, IRT scoring is shown to have many advantages including the ability to link (equate) scores obtained by using different instruments. An important chapter is the one on assessing the assumptions of the models,

particularly unidimensionality (‘It is tiresome to continuously recommend more research, but . . .’), and local independence (‘The best way of dealing with local dependence is to prevent its occurrence . . .’). This chapter also describes how to evaluate the fit of the models, including item fit and person fit. I would have welcomed a larger number of real-life examples here. Many practitioners of IRT have observed that even a few outliers in the form of poorly fitting subjects and items can disproportionately affect the fitting of the model, and thus advise that they should be excluded and the parameters re-estimated. The fourth and final part of the book focuses on applications of IRT, including differential item functioning, computer adaptive testing, test equating or linking scales, and using IRT for assessing construct validity. To this reviewer, chapters of this section were the most interesting of all and I would have been grateful for more abundant examples. This part also includes a review of commercially available computer software – which is always a necessary consideration for anyone seeking to apply methods like IRT; none of the larger statistical packages contain adequate routines for the necessary model-fitting, whereas the purpose-built programs employ better numerical algorithms, contain tailored diagnostics, and provide facilities for assessing model adequacy. Despite my reservations, this is a thorough and comprehensive introduction to IRT and is very useful reference book. It would be a helpful source to use when teaching students about IRT. The basis of much of the theory underlying IRT is covered. Overall, Embretson and Reise have largely succeeded in their objective of avoiding complex statistical equations, although inevitably for this subject a reasonable level of statistical competence remains necessary. The downside of avoiding equations is that at times the resultant text tends to be lengthy and clumsy. But I am still waiting for a book that provides a simple practical guide to the choice and application of IRT models, either in HRQL or related fields. Peter Fayers Institute of Applied Health Sciences University of Aberdeen Medical School UK