New resources on PhysPort.org - PDF Document

Presentation Transcript

  1. Using research-based assessment to improve teaching in your classroom and department: New resources on PhysPort.org PhysPort Supporting physics teaching with research-based resources (Formerly known as the PER User's Guide) Sarah B. McKagan Adrian Madsen Eleanor C. Sayre

  2. PhysPort Supporting physics teaching with research-based resources What is PhysPort? A web resource to support physics professors in using research-based teaching and assessment in their classes www.physport.org 2

  3. Motivation • Physics education researchers have created research results, teaching methods, curricula, and assessments that can dramatically improve physics education. • Most people who teach physics don’t know about these resources. • There is a need for a “one-stop shopping” place to find resources for research-based teaching. 3

  4. The PhysPort Team AAPT Sam McKagan (PI) Adrian Madsen (co-PI) Lyle Barbato (development lead) Matt Riggsbee (visual design) Brian Danielak (postdoc) KSU Ellie Sayre (PI) Bill Hsu (development lead) Eugene Vasserman (security lead) Cognition Technology Sandy Martinuk (user experience design lead) Alex Bell (user experience design assistant) 4

  5. PhysPort site content Now available: • Resources for research-based teaching methods • Resources for research-based assessment • Video workshops for LAs, TAs, & faculty Coming in Fall 2015: • Redesign and expansion of teaching methods • Assessment Data Explorer • Expert Recommendations Resources for research-based assessment Assessment Data Explorer 5

  6. Research and Development Process Faculty and Department Chair Interviews Chair Interviews Faculty and Department Personas of Users Site that meets real users’ needs Interviewed 24 physics faculty 
 and department chairs 
 about their teaching and assessment (to discover goals, motivations, needs, pain points etc.) 6

  7. Research and Development Process Faculty and Department Chair Interviews Personas of Users Site that meets real users’ needs Raphael the Motivated Novice Diane the Pragmatic Satisficer Marge the Proto- researcher Paula the Skeptic Tim the Seeker 7

  8. Research and Development Process Faculty and Department Chair Interviews Personas of Users Site that meets real users’ needs Personas combine characteristics of many different people to represent a coherent set of user needs Raphael Paula 8

  9. Research and Development Process Faculty and Department Chair Interviews Personas of Users Site that meets real users’ needs Raphael the Motivated Novice Diane the Pragmatic Satisficer Marge the Proto- researcher Paula the Skeptic Tim the Seeker 9

  10. Key Personas • New to research-based teaching • Cares about his students’ learning, eager to try new methods • Needs simple instructions and basic guidance Raphael the Motivated Novice • Some experience with research-based teaching • Wants to use evidence to demonstrate student learning. • Wants to know what works, how to use it, and what to do if she has trouble. Diane the Pragmatic Satisficer • Extensive experience with research-based teaching • Wants to go beyond the basics and address less well-defined aspects of learning, such as problem solving, reasoning skills, and attitudes Tim the Seeker 10

  11. Other personas (not used for site design) • Not convinced that research-based teaching is effective • Relies on intuition and experience to guide her teaching Isn’t going to use our site (she’ll learn from her colleagues who use the site) Paula the Skeptic • Extensive experience using and even creating research-based materials and strategies • Knows where to find most resources she needs Marge the Proto- researcher Doesn’t really need our site 11

  12. Research and Development Process Faculty and Department Chair Interviews Personas of Users Site that meets real users’ needs Examples from site: • Home page • Assessment resources • Assessment data explorer 12

  13. Start with biggest needs of users Teaching Methods I want to... • find a new teaching method • get implementation help • learn more about research-based teaching Assessment I want to... • interpret assessment results • assess the impact of reforms • assess advanced physics content or skills Troubleshooting I need help with... • covering enough material • supporting group work • arguments for skeptical colleagues 13

  14. Homepage Welcome to PhysPort (formerly known as the PER User's Guide), the go-to place for physics faculty to find resources based on physics education research (PER) to support your teaching. Learn more... 14

  15. Research and Development Process Faculty and Department Chair Interviews Personas of Users Site that meets real users’ needs Examples from site: • Home page • Assessment resources • Assessment data explorer 15

  16. How do we do assessment in physics? Physics classes: • Exams • Homework • Teaching evaluations • Assessment surveys Physics departments: • Drop-withdraw-fail rates • Student retention • Observations • Assessment surveys Focus on research-based assessment surveys 16

  17. What are Research-based Assessment Instruments? Force Concept Inventory (FCI) Force Motion Conceptual Evaluation (FMCE) and 50+ more These are: • Generally multiple-choice surveys • Carefully crafted questions • Conceptual topics across the physics curriculum • Additionally: beliefs, problem-solving skills, affect 17

  18. Find an Assessment Raphael the Motivated Novice • Which research-based assessment should I use? • Where do I get the assessment? Diane the Pragmatic Satisficer Tim the Seeker • How can I assess non-content skills? 18

  19. Find an Assessment

  20. Find an Assessment 20

  21. Find an Assessment 21

  22. Find an Assessment 22

  23. Learn about the Assessment Raphael the Motivated Novice • Which assessment should I use? • Where do I get the assessment? • How should I administer the assessment? Diane the Pragmatic Satisficer Tim the Seeker • How can I assess non-content skills? 23

  24. 24

  25. 25

  26. 26

  27. 27

  28. 28

  29. 29

  30. 30

  31. Visualize and Analyze Your Assessment Data • Your identity is protected • Your students’ identities are protected • We use one-way, cryptographically-secure transformations • We report on aggregate data 31

  32. Visualize and Analyze Your Results • How did I do on this assessment? • How do my assessment results compare to other students like mine? Raphael the Motivated Novice 32

  33. Visualize and Analyze Your Results 33

  34. Visualize and Analyze Your Results 34

  35. Your Results Over Time Diane the Pragmatic Satisficer • How do my results change over time? 35

  36. Your Results Over Time 36

  37. Question-by-Question Breakdown • How do my results break down on a question-by-question basis? • How do I use these results to make improvements in my class and department? Diane the Pragmatic Satisficer 37

  38. Question-by-Question Breakdown 38

  39. Question-by-Question Breakdown 39

  40. Question-by-Question Breakdown 40

  41. Compare Multiple Courses • How do the results in my department vary across different courses and instructors? • Is there a gender gap on these assessments in my class? Tim the Seeker 41

  42. Compare Multiple Courses 42

  43. Compare Multiple Courses 43

  44. Compare Multiple Courses 44

  45. Upload Assessment Results Raphael the Motivated Novice • When will I find the time to analyze my data? Diane the Pragmatic Satisficer 45

  46. Add Metadata 46

  47. Add Metadata 47

  48. Add Metadata 48

  49. 49

  50. Add Metadata 50