Read the summary and the most important questions on BDS: psychometrics

• 1 Week 1 classical test theory

• 1.1 CTT, DeVellis 2006

  This is a preview. There are 22 more flashcards available for chapter 1.1
  Show more cards here

• What is the reason classical test theory was created?

  • Many phenomena psychologists are interested in, are not directly observable.
    
  • For example, there are no overt, unambiguous, direct manifestations of variables such as patient satisfaction, loneliness, pain, or intention to follow medical recommendations.
    
  • as the next best thing, we seek proxy indicators that provide accurate information about unobservable variables.
  • CTT comprises a set of principles that allow us to determine how successful our proxy indicators are at estimating the unobservable variables of interest.
    
  • Thus, fundamentally, it concerns using observable information (such as scores on questionnaire items) to garner insights into variables (such as patient satisfaction) that cannot be directly observed.
    

• What is the formula classical test theory is based on?

  • X = t + e
    
  • where x is the observed score on an item or test (random)
  • t is the true score on an item or test (fixed)
  • e is the error. (random)
  • the error is the difference between true score and observed score
  • t = E(X)
  • the true score is the expectation of the observed score (fixed)

• What are the properties of item errors?

  • Errors in ctt are assumed to be random
    
  • random errors are equally probable in all directions and large errors are less likely than small ones
  • errors are item-specific: errors of different items are unrelated to eachother.
  • Because errors are random, their mean is zero. That is, when all sources of error are combined, they should cancel each other out and have little or no effect on the item mean
    

• How can you interpret the true score?

  • The true score is the expected observed score.
  • Lord and Novick describe this as: if we were able to repeatedly ask the same question and brain wash the individual in between each iteration, we would get varying answers. The true score is the mean of these answers.
  • since errors are random, sometimes the observed score is lower than the true score and sometimes it is higher. The observed scores vary around the true score.
  • after many iterations, these errors cancel eachother out and what is left is the true score.  

• What is item reliability?

  • Generally, to what extent does the instrument produce the same scores in the same circumstances
  • item reliability embodies the extend to which the observed scores are consistent over multiple unchanged conditions

• How can the standard error of measurement for an item be estimated from the reliability?

  • If we knew the item reliability, we could calculate the standard error of measurement.
  • sem = sd(x) * sqrt(1 - item reliability)
  • the standard deviation of observed scores times the square root of 1 - the reliability

• How is chronbach's alpha used in test construction?

  • A first draft of a test is created and you decide on an alpha level that you find acceptable.

  • Delete each item iteratively and recompute alpha (alpha-if-item-deleted)

  • Delete the item with the highest alpha-if-item-deleted

  • If alpha is now acceptable, stop
  • If alpha is still too low, go back to step 2

• How can chronbach's alpha be used to estimate the sem?

  • The chronbach's alpha can be implemented for the reliability in the formula for the standard error of measurement.
    
  • sd(X) * sqrt(1 - a) = sem
  • since chronbach's alpha is a lower bound estimate of the reliability, the standard error of measurement is an upper bound estimate of the true standard error of measurement.

• What are two other properties of items, next to their reliability?

  • Difficulty: a difficult item is one that fewer people answer correctly. More generally, one can interpret item difficulty as reflecting the strength of the attribute being assessed.
  • discrimination: the discrimination of an item describes how good the item can discriminate between individuals where the measured attribute is highly present, and individuals where the measured attribute is weakly present.

• How can you use the relation between predictive validity and reliability?

  • It can be used to disattenuate correlations:
    
  • Measurement error can be removed from a correlation coefficient, rxy, to estimate the correlation coefficient disattenuated of measurement error
  • disattenuation tells us whether the correlation between two sets of measures is low because of measurement error or because the two sets are really uncorrelated.
    
  • the formula is:
  • correlation between test x and test y / sqrt(reliability x * reliability y)
  • this gives the correlation between true scores