Significant Legitimacy.

Uploaded on:
Category: Music / Dance
Gaining from Kane, we see the legitimacy assessment as building a contention (e.g. ... advancing perspective/acknowledgment of scholarly educational programs (i.e., learning encounters; ...
Slide 1

Consequential Validity Mountain Plains Regional Resource Center Teleconference Series May 30, 2007 Elizabeth Towles-Reeves Jacqui Kearns National Alternate Assessment Center (NAAC)

Slide 2

Validity Should be Central We contend that the motivation behind the specialized documentation is to give information to bolster or discredit the legitimacy of the inductions from the substitute evaluations at both the understudy and program level. Gaining from Kane, we see the legitimacy assessment as building a contention (e.g., lawful, philosophical) to bolster or disprove the deductions with respect to the scores from the AA-AAS.

Slide 3

The Challenge of Documenting Technical Quality of Alternate Assessments heterogeneity of the gathering of understudies being surveyed and how they show information and abilities; regularly "adaptable" appraisal encounters; moderately little quantities of understudies/tests; developing perspective/acknowledgment of scholastic educational modules (i.e., learning encounters; the high level of contribution of the instructor/assessor in managing the appraisal (Gong & Marion, 2006); non-conventional appraisal approaches (e.g., portfolios, execution undertakings/occasions) for which there is a need to grow the conceptualization of specialized quality to assess these methodologies (Linn, Baker, & Dunbar, 1991).

Slide 4

Expanding Technical Quality Linn, et al. (1991) called attention to that we as of now (15 years back!) have the hypothetical apparatuses for extending legitimacy examinations, yet by and by legitimacy is typically seen too barely. Content structures are depicted, and details for the choice of things are accommodated institutionalized accomplishment tests. Connections with different tests and some of the time with instructor appraisals of accomplishment may likewise be introduced. Such data is applicable to judgments of legitimacy yet does not do equity to the idea (p. 16). We contend that AA-AAS specialized assessments are enduring the same destiny.

Slide 5

Shepard (1993) Shepard (1993) pushed a clear intends to organize legitimacy questions. Utilizing an assessment structure, she suggested that legitimacy studies be composed because of the inquiries: What does the testing rehearse case to do; What are the contentions for and against the planned points of the test; and What does the test do in the framework other than what it claims, for good or awful? (Shepard, 1993, p. 429). The inquiries are coordinated to worries about the develop, pertinence, understanding, and social results, individually. We trust that this methodology for organizing our inquiries is valuable.

Slide 6

INTERPRETATION OBSERVATION COGNITION The Assessment Triangle and Validity Evaluation VALIDITY EVALUATION Empirical Evidence Theory and Logic (contention) Consequential Features Reporting Alignment Item Analysis/DIF/Bias Measurement Error Scaling and Equating Standard Setting Assessment System Test Development Administration Scoring Student Population Academic Content Theory of Learning

Slide 7

Questions After exploring the materials on the site from the Inclusive Assessment Seminars, what questions do you have before we start taking a gander at the noteworthy legitimacy elements of AA-AAS frameworks?

Slide 8

What is Consequential Validity? Messick (1989) initially acquainted results with the legitimacy contention. Later, Shepard (1993, 1997) expanded the definition by belligerence one must examine both positive/negative and expected/unintended results of score-based surmisings to appropriately assess the legitimacy of the appraisal framework.

Slide 9

So What? There is overpowering backing for noting the "So What" question (Haertal, 1999; Kane, 2002; Kleinert et al., 2001; Lane & Stone 2002; Shepard, 1997), yet in the meantime contrasting partner sees must be incorporated to display a persuading legitimacy contention (Lane & Stone, 2002; Linn 1998; Ryan, 2002).

Slide 10

Intended Consequences Lane and Stone (2002) recommend that state evaluations are planned to effect: Student, educator, and director inspiration and exertion; Curriculum and instructional substance and techniques; Content and arrangement of classroom appraisals; Improved learning for all understudies; Professional advancement bolster; Use and nature of test readiness exercises; and Student, instructor, head, and open mindfulness and convictions about the evaluation, criteria for judging execution, and the utilization of evaluation results.

Slide 11

Unintended Consequences on occasion, nonetheless, Lane and Stone (2002) propose unintended results are conceivable, for example, Narrowing of educational modules and direction to concentrate just on the particular learning results surveyed; Use of test arrangement materials that are firmly connected to the evaluation without rolling out improvements to the educational programs and guideline; Use of unscrupulous test readiness materials; and Inappropriate utilization of test scores by managers.

Slide 12

Consequential Validity Evaluation Questions Before you consider examining any important legitimacy questions for your substitute appraisal judged against exchange accomplishment gauges (AA-AAS), you should decide: What is the motivation behind the AA-AAS? By what method will the scores of the AA-AAS be utilized ? What partners are imperative to helping you comprehend the outcomes of the AA-AAS: understudies, folks, instructors, heads, group individuals, specialists?

Slide 13

Consequential Validity Evaluation Questions Once you decide reason and utilize, you may then ask: What are the proposed and unintended results in light of the reason and utilization of the AA-AAS? Are the planned and unintended outcomes positive or negative?

Slide 14

Looking to our Past to Prepare for the Future Research on the important legitimacy of exchange appraisals from the point of view of: Students/Parents Research Questions: What advantages to understudies have accumulated from the investment in AA-AAS? What is the degree to which understudies have gotten to the general instruction educational modules? What is the effect of the AA-AAS on understudies\' IEP advancement? What is the relationship between understudy execution in AA-AAS and post-school life results? What understudy, educator, and instructional variables impact folks\' observations with respect to the AA-AAS?

Slide 15

Looking to our Past to Prepare for the Future Research on the weighty legitimacy of substitute appraisals from the point of view of: Teachers Research Questions: What advantages to educators have accumulated from the support of understudies in the AA-AAS? What is the degree to which interchange evaluations are a piece of day by day classroom schedule? What is the relationship between exchange evaluation scores and the measure of time spent chipping away at the appraisal? Whatever degree do instructor and instructional variables foresee interchange evaluation scores? Which understudy, educator, and instructional variables impact instructors\' recognitions with respect to the AA-AAS? What is the effect of the AA-AAS on educators\' day by day direction?

Slide 16

Looking to our Past to Prepare for the Future Research on the noteworthy legitimacy of interchange evaluations from the point of view of: School Research Questions: To what degree are understudies incorporated into the responsibility process? Is there any relationship between understudy execution in the AA-AAS and understudy execution in the general appraisal?

Slide 17

Prioritization There is no chance that a state can tackle exploring all these exploration learns on the double. How might you be able to go about prioritization of studies? Assemble partners Work through guided examination with respect to what studies might be imperative to direct in light of partner info Prioritize the main 2 considers for the short-term (next 2-3 years) and after that organize the main 2 concentrates on for the long haul (next 3-5 years)

Slide 18

Questions Any inquiries or illuminations?

Slide 19

Questions Have you accumulated a partner gathering to consider and organize significant legitimacy questions for the transient and long haul? Assuming this is the case, who was included? If not, what partners do you think will be imperative to have at the table? What weighty legitimacy contemplates have you performed in your state (i.e., taking a gander at outcomes identified with understudies, instructors, and schools)? What studies might you want to perform however are uncertain in the matter of how to assemble the information or behavior the study?

View more...