Psychometrics
77 flashcards · Psychology · by charlie42
What's operationalization?
A process of defining the measurement of a phenomenon which is not directly measurable, though its existence is inferred by other phenomena.
Two scale development theories
Classical test theory Item response theory
Original definition of validity
Property that a test measures what it purports to measure (Buckingham, 1921)
Two types of construct validity?
Translational validity Criterion validity
What is a criterion?
The criterion is an external measurement of the same thing. It is usually an established or widely-used test that is already considered valid.
Why translational validity called like that?
It assess the degree to which you accurately translated your construct into the operationalization
Another name for translational validity
Representational validity
Two sub-types of translational validity
Face validity Content validity
Translational validity usually measured using
A panel of experts
Empirical assessment of validity examines
how well a given measure relates to one or more external criterion
Empirical assessment of validity measures which validity
Criterion-related validity
Four sub-types of criterion-related validity
Convergent Discriminant Concurrent Predictive
Discriminant validity tests whether
concepts or measurements that are not supposed to be related are actually unrelated
Two statistical methods to demonstrate convergent and discriminant validity
Exploratory factor analysis Bivariate correlation matrix
Predictive validity is a degree to which
a measure successfully predicts a future outcome it's theoretically expected to predict (SAT -> GPA)
Classical test theory formula
Two kinds of measurement errors
Random Systematic
Most common measure of central tendency
Mean
What random error affects and doesn't affect in a distribution
Affects variability Doesn't affect central tendency
What systematic error affects and doesn't affect in a distribution
Shifts central tendency Doesn't affect variability
How the two kinds of errors affect reliability and validity of the measurement?
Random error reduces reliability Systematic error reduces validity
What's a z-score?
The number of standard deviations by which the value of a raw score is above or below the mean value of what is being observed or measured.
Reliability
get consistent results when measuring the same construct
Translational validity examines whether
the measure is a good reflection of its underlying construct
Convergent validity refers to the
degree to which two measures of constructs that theoretically should be related, are in fact related
Concurrent validity is demonstrated when
a test correlates well with a measure that has previously been validated
What's a construct?
A construct refers to a concept that can’t be directly observed, but can be measured by observing other indicators that are associated with it.
Another word for z-score
Standard score
What's content validity?
Content validity refers to the extent to which a measure represents all facets of a given construct. For example, a depression scale may lack content validity if it only assesses the affective dimension of depression but fails to take into account the behavioral dimension.
What's dimensionality of a test?
The number of latent variables that is measured by the test.
Different types of variance of each item in factor analysis
What's communality in factor analysis?
Definition of common variance that ranges between 0 and 1.
How do you interpret communality values?
Values closer to 1 suggest that extracted factors explain more of the variance of an individual item.
What's specific variance in factor analysis? Example.
Variance that is specific to a particular item Example: item 4 “All computers hate me” may have variance that is attributable to anxiety about computers in addition to anxiety about SPSS
In factor analysis, what's the values of the sum of all discovered factors' eigenvalues?
Total number of variables
Visualize scree plot
What are the axes of the scree plot?
x = component number y = eigenvalue
What does eigenvalue of a factor indicate?
Total variance accounted by the factor
If you had 20 variables, and the first extracted factor's eigenvalue was 8.543, that would indicate that the first factor extracted "captured" or "accounted for"...
about 42.7% of the variance in the data set.
If you had 20 variables, and the first extracted factor's eigenvalue was 8.543, that would indicate that the first factor extracted "captured" or "accounted for" about 42.7% of the variance in the data set. What accounts for the rest of the variance?
The remaining potential 19 factors would collectively explain the other 57.3% of the variance.
When confirmatory factor analysis is used?
When you want to test specific hypothesis about the structure or the number of dimensions underlying a set of variables
Why only the first two factors are retained? link
Because their eigenvalues are > 1
What does Difference indicate?
Difference between the current eigenvalue and the next
What does Proportion indicate?
Eigenvalue / Total number of variables = Proportion of variance explained
How much of the total variance is accounted for by the first two factors?
57% (Cumulative column)
Explain covariance formula (between two variables)
Sum of matches or mismatches between two variables: If both are above or both are below their means => positive value to add up => match If one is above its mean and another is below => negative value to add up => mismatch
Why correlation is commonly used instead of covariance?
Covariance value depends on the scale of the values. Correlation is scale-free (always between -1 and 1).
Multiple correlation coefficient is the ...
maximum correlation between a variable and linear combinations of a set of variables.
Why we don't keep factors with eigenvalues <1
When the eigenvalue is less than 1.0 the factor explains less information than a single item would have explained. Most researchers would not consider the information gained from such a factor to be sufficient to justify keeping that factor. link
What is this model called?
Correlated factors model
What is this model called?
Second-order factor model
What is the relation between the correlated factor model and the second-order factor model?
The second-order factor model says that the factors are correlated because they're all explained by one common factor
Property of relationship between general factor and group factors in the bi-factor model
They're orthogonal (uncorrelated)
What's a manifest variable?
A manifest variable is a variable or factor that can be directly measured or observed. It is the opposite of a latent variable, which is a factor that cannot be directly observed, and which needs a manifest variable assigned to it as an indicator to test whether it is present.
What's simple structure in factor analysis?
A pattern of loadings where each item loads strongly on only one of the factors, and much more weakly on the other factors.
What does scale usage heterogeneity refer to?
Personal and cultural tendencies in how people answer scales (all left, all right, all center).
How do deal with scale usage heterogeneity?
Center responses by subtracting the mean over all questions and dividing by standard deviation of response
How to calculate correlation between Flight Options and Easy Reservations?
One part is due to impact of g: 0.73*0.62=0.4526. The second part comes from F3*: 0.38*0.34=0.1292. =0.73*0.62 + 0.38*0.34
A model is identifiable if...
it is theoretically possible to learn the true values of this model's underlying parameters after obtaining an infinite number of observations from it. A model that fails to be identifiable is said to be non-identifiable or unidentifiable: two or more parametrizations are observationally equivalent. => no two different sets of parameters produce the same data
What's singularity in statistics?
In regression analysis, singularity is the extreme form of multicollinearity - when a perfect linear relationship exists between variables or, in other terms, when the correlation coefficient is equal to 1.0 or -1.0. Such absolute multicollinearity could arise when independent variable are linearly related in their definition. A simple example: two variables "height in centimeters" and "height in inches" are included in the regression model.
When are two measures congeneric?
If they measure the same trait except for the error of measurement. They don't need to be equivalent (on the same scale), just can need to be linearly dependent
When are two measures parallel or tau-equivalent?
Parallel tests measure the same construct in identical units of measurement and with the same precision (error variance). Their true scores are identical E.g. the same appliance measures the same thing two times
What's composite reliability?
It's a measure of internal consistency in scale items, like Cronbach's alpha (sometimes called construct reliability)
Difference between Cardinal and Ordinal numbers
Cardinal numbers indicate an amount—how many of something we have: one, two, three, four, five. Ordinal numbers indicate position in a series: first, second, third, fourth, fifth
Why is bifactor model called like that?
Because each item loads on two factors (here g-factor and Facet factors)
Visualize difference between: Correlated factor model Higher-order factor model Bi-factor model
What's the difference between simple and multiple regression?
Multiple regression includes two or more independent variables – sometimes called predictor variables.
How do you interpret if one coefficient in multiple regression is significant?
For every 1 unit increase in x1 (e.g. temperature), while holding all other independent variables constant, there is an average increase in y by beta1 that is unlikely to occur by chance.
Why sometimes when using multiple linear regression with two variables only one is significant, but when you do two separate simple regressions both are significant? Example.
The insignificant independent variable is actually dependent on another independent variable. (There is correlation between independent variables). Example: predicting ice cream sales based on temperature and amount of shorts worn: When we did multiple linear regression we looked at the relationship between shorts and sales while holding temperature constant and the relationship vanished. The true relationship between temperature and sales remained however.
What's standard error?
Standard deviation of a sample statistic (e.g. mean of a sample). It measures how accurately a sample of a certain size can measure a statistic of the whole population. The higher the sample size, the lower the variation in means will be, and the closer it will be to the actual mean of the population
Visualize mode, median, and mean of this distribution:
What's mode of a distribution?
The value that appears most often in a set of data values.
What's a discrete random variable? Example.
A discrete variable can assume only a finite number of real values within a given interval (as opposed to a continuous variable) An example of a discrete variable would be the score given by a judge to a gymnast in competition: the range is 0 to 10 and the score is always given to one decimal (e.g. a score of 8.5).
Formula for expected value and example with dice roll.
Visualize difference between probability and likelihood
How to find a point with the highest Likelihood in such a plot?
Find a point where the slope of the curve is 0
What's arg max of a function f(x)?
A value of x where f(x) has the largest value