Test Legitimacy: What it is, and why we give it a second thought..

Uploaded on:
Category: Travel / Places
Test Legitimacy: What it is, and why we give it a second thought. Legitimacy What is legitimacy? What is a build?: Meehl's nomological net Sorts of legitimacy Substance legitimacy Model related legitimacy Develop Legitimacy Incremental Legitimacy The multi-attribute multi-technique grid What is legitimacy?
Slide 1

Test Validity: What it is, and why we give it a second thought.

Slide 2

Validity What is legitimacy? What is a build?: Meehl’s nomological net Types of legitimacy Content legitimacy Criterion-related legitimacy Construct Validity Incremental Validity The multi-attribute multi-strategy network

Slide 3

What is legitimacy? The legitimacy of a test is the degree to which it gauges the develop that it is intended to measure As we might see, there are numerous routes for a test to come up short or succeed, and thusly: legitimacy is not a solitary measure

Slide 4

Paul Meehl: What is a build? Meehl’s meaning of a build has 6 principle components, as takes after: 1.) To say what a develop is intends to say what laws it is liable to. - This is a definition = you can decline to work with it or say why you think it is awful, however you can’t negate it -The total of all laws is known as a construct’s nomological system .

Slide 5

What does ‘nomological’ mean? I had constantly trusted it originated from: ad. L. nomin significance ‘name’ I wasn\'t right. Actually it originates from: ad. Gr. nom joining type of a word significance ‘law’ So ‘psychonomics’ is the laws\' investigation of the mind, and ‘nomological network’ alludes to a system of mental parts whose relations can be portrayed by laws or tenets

Slide 6

The nomological system comprises of: Adapted from: http://trochim.human.cornell.edu/kb/nomonet.htm i.) Representations of the ideas of premium (builds) CONSTRUCT OBS ii.) Their discernible appearances & iii.) The connections inside and in the middle of i.) and ii.)

Slide 7

Adapted from: http://trochim.human.cornell.edu/kb/nomonet.htm Theoretical recommendations CONSTRUCT Operationalized hypothetical develops OBS Correspondence rules Empirical perceptions

Slide 8

Paul Meehl: What is a develop? 2.) Laws may relate noticeable and hypothetical components -The relations must be ‘lawful’, however they may be either causal or factual ( what’s the connection in the middle of causal and measurable ? ) -What are the ‘theoretical elements’? Builds!

Slide 9

Paul Meehl: What is a develop? - What are the ‘theoretical elements’? Builds! - To escape from circularity and unadulterated theory about the properties of develops, we have to stay the nomological net solidly in some goal reality, thus: 3.) A build is just admissable if at any rate a laws\' percentage to which it is subject include observables If not, we could characterize a self-predictable system of thoughts that had no importance to this present reality (and numerous such systems have been characterized! For example, ) You ought to have the capacity to relate this thought of observables to our prior talk of data: what considers discernible is the thing that considers data (a noticeable contrast that has any kind of effect)

Slide 10

Paul Meehl: What is a develop? 4.) Elaboration of a construct’s nomological net = adapting more about that develop -We expound a build by drawing new relations, either between components as of now in the system , or between those components and new components outside of the system - This elaboration is decisively the work of psychometrics, and additionally the work of science as a rule

Slide 11

Paul Meehl: What is a build? 5.) Ockham’s razor + Einstein’s addendum - That is: make things as straightforward as could be allowed, however no less difficult 6.) Identity means ‘playing the same part in the same network’ - If it would appear that a duck, strolls like a duck, and quacks like a duck: then it is a duck!* -Or (in the soul of Gregory Bateson): If it has no effect, then it has no effect. * in any event pending further examination

Slide 12

How to gauge legitimacy Analyze the test\'s substance Relate test scores to particular criteria Examine the mental builds measured by the test

Slide 13

Construct legitimacy Construct legitimacy = t he degree to which a test measures the develop it cases to quantify Does a knowledge test measure insight? Does a neuroticism test measure neuroticism? What is idle antagonistic vibe since it is idle? As Meehl notes, develop legitimacy is exceptionally broad and regularly extremely hard to focus in an authoritative way If it would seem that an ability\'s measure or learning it should quantify, we say it has face legitimacy How would we be able to focus build legitimacy? (By what method will you know whether you get given a decent exam in this class?)

Slide 14

Construct legitimacy There are two sorts of build legitimacy: united legitimacy or discriminant legitimacy Convergent legitimacy (in some cases called exact legitimacy ) implies that the measure under thought concurs with different measures that are affirmed (or hypothetically expected to) to quantify the same things Divergent legitimacy implies that the measure under thought is unmistakable from different measures that are charged (or hypothetically expected to) to gauge distinctive things

Slide 15

Content legitimacy Content legitimacy = the degree to which the test evokes a scope of reactions over the scope of aptitudes, comprehension, or conduct the test measures; the degree to which it mirrors the particular planned area of substance In theoretical and/or complex spaces, it might be entirely hard to guarantee content legitimacy Could a test have develop legitimacy yet not content legitimacy?

Slide 16

Criterion-related legitimacy Criterion-related legitimacy relies on relating test scores to execution on some pertinent rule or set of criteria i.e. Approve tests against school marks, chief evaluations, or dollar estimation of beneficial work There are two sorts of model related legitimacy: simultaneous and prescient

Slide 17

Concurrent legitimacy Concurrent legitimacy = the legitimacy measure are accessible at the season of testing i.e. give the test to subjects who have been chosen for their monetary foundation or symptomatic gathering the legitimacy of the MMPI was resolved in this way

Slide 18

Predictive legitimacy Predictive legitimacy = the model are not accessible at the season of testing worried with how well test scores anticipate future execution For instance, IQ tests ought to connect with scholastic evaluations, grades, critical thinking aptitudes and so on. A decent r - esteem for most mental inquiries would be .60

Slide 19

What influences legitimacy? i.) Moderator variables : Those attributes that characterize gatherings, for example, sex, age, identity sort and so on - A test that is all around approved on one gathering may be less great with another -Validity is typically better with more heterogeneous gatherings, in light of the fact that the scope of practices and test scores is bigger And hence: ii.) Base rates : Tests are less successful when base rates are high or low (that is, at whatever point they are skewed from 50/50)

Slide 20

What influences legitimacy? iii.) Test length -For comparable reasons of the space\'s measure examined (think about the binomial rabbits or attempting to choose how one-sided a coin is), longer tests have a tendency to be more dependably identified with the foundation than shorter tests

Slide 21

Test length Informally, we can see that the same size changes, (for example, being 1 flip far from reasonable) have more effect to the extent of range under the bend when N is low Next class we consider how to consider this for different qualities in a more formal way

Slide 22

What influences legitimacy? iii.) Test length -For comparative reasons of the space\'s measure examined (think about the binomial rabbits or attempting to choose how one-sided a coin is), longer tests have a tendency to be more dependably identified with the foundation than shorter tests -Note that this relies on upon the inquiries being autonomous (= each inquiry expanding data) -When it is not, more tests are not more solid -eg. short types of WAIS -However, take note of that autonomy require just be incomplete (|r| < 1, yet not so much r = 0)

Slide 23

What influences legitimacy? iv.) The legitimacy\'s nature foundation -Criterion can be debased, particularly if the translation of test reactions is not very much determined, taking into account results to ‘feed back’ to standard -In such cases, there is perplexity between the approval criteria and the test outcomes = the circularity of self-satisfying prediction (a ‘dormitive principle’) - fundamentally we are then stuck at the hypothetical level of the nomological net, with no chance to get for observational study (= no data) to let us know we are incorrect

Slide 24

How to gauge build legitimacy i.) Get master judgments of the substance ii.) Analyze the inner consistency of the test (Tune in next class for how to do this, and why it is not entirely legitimacy, however it illuminates legitimacy) iii.) Study the connections between test scores and other non-test variables which are known/attempted to relate the same develop -eg. Meehl notice Binet’s vindication by educators iv.) Question your subjects about their reactions keeping in mind the end goal to evoke basic explanations behind their reactions. v.) Demonstrate expected changes after some time

Slide 25

How to quantify develop legitimacy vi.) Study the connections between test scores and other test scores which are known/attempted to identify with (or leave from) the build (Convergent versus discriminant legitimacy) - Multitrait-multimethod approach: Correlations of the same characteristic measured by the same and distinctive measures > relationships of an alternate attribute measured by the same and diverse measures [ We will take a gander at this in more detail in a minute.] What if relationships of measures of distinctive qualities utilizing the same technique > co

View more...