Mapping Learning: A Toolkit of Resources - PDF Document

0 Views
Presentation Transcript

  1. Mapping Learning: A Toolkit of Resources 1

  2. Mapping Learning: A Toolkit of Resources Institutions of higher education are complex spaces, with students learning in all corners of them, building upon the prior learning they bring with them. The complexity of our educational environments poses a challenge to understanding where students learn and how learning is reinforced and integrated across curricular, co-curricular, and work-based experiences. In its most recent survey of activity within the field, the National Institute for Learning Outcomes assessment (NILOA) has seen an increased awareness of the range of places that learning happens within institutions as well as the need to document and align learning throughout. While 77% report that their institutions are currently involved in curriculum mapping of some kind, only 50% indicate that all programs have learning outcomes and that those outcomes align throughout the institution (Jankowski, Timmer, Kinzie, & Kuh, 2018). Faculty are working to create a curriculum that intentionally builds in integrated learning opportunities over time for students to apply and practice as well as transfer their knowledge and skills through assignments, in and out of courses. Mapping has emerged as a key strategy for examining the alignment of the different elements of learning environments towards shared learning outcomes as well as to better understand where to assess and document learning. In addition, as assignments continue to take on prominence as a useful source of learning related to larger shared learning outcomes (Jankowski, Timmer, Kinzie, & Kuh, 2018), the need to map relationships between institution, co-curricular, general education, and program learning outcomes with courses and specific assignments takes on increasing importance. In this toolkit, we present a variety of information on the mapping process – what are the purposes and uses of maps, what can be mapped, and various approaches to engage with mapping learning. We assume the focus of mapping is on documenting learning, but the approaches addressed here would be applicable with a different focus or lens as well. We invite you to share with us additional examples, materials, resources, and modifications of the toolkit to add to this resource. If you have examples, please send them to niloa@education.illinois.edu. 2

  3. What is Mapping? Mapping is a tool for seeing relationships between different aspects of the institution based on learning outcomes. The most common form, curriculum mapping at a program-level, makes visible how courses in a curriculum align to the learning outcomes to which that curriculum strives. In its simplest version, the curriculum map is built on a two-dimension matrix, with the outcomes arrayed across the top (the x-axis) and courses listed down the left side (the y-axis). As depicted in figure 1, a mark is made in the box where a course addresses an outcome. Outcome 1 X Outcome 2 X X Outcome 3 Course 1 Course 2 Course 3 X X Figure 1: A basic curriculum map Such maps are a common hallmark of assessment systems and provide a means to examine if there are gaps in a program’s curriculum in relation to any learning outcomes. The process most commonly entails three different approaches led entirely by faculty. 1. An excel spreadsheet is electronically sent around to faculty and individual faculty members complete the sheet based on the courses they teach. Responses are then compiled and filed. 2. A program or department chair, in isolation, completes the map and submits it to an assessment management system. 3. Faculty come together to identify which courses align with which outcomes or where various learning outcomes are addressed. The limitation here, is that mapping under the first two approaches generates reports which can be pulled for review, but the maps are rarely used. Further, if two faculty members mapped the curriculum individually, there is no guarantee they would develop the same map. If students mapped where they thought the learning outcomes were addressed, there would be another map as well. How much does a learning outcome need to be covered to be counted on the map? Is it necessary for it to be assessed to appear? Do we even have a shared understanding of what the learning outcomes are to indicate the relationship between them and courses? Further, this approach focuses upon academic affairs at the expense of learning in other places. The same map could be utilized with co-curricular learning experiences by changing the title of course to learning experience (Figure 2). 3

  4. Co-curricular Learning Learning Experiences 1 Learning Experiences 2 Learning Experiences 3 Outcome 1 Outcome 2 Outcome 3 X X X X X Figure 2: Co-curricular learning map An approach focused on mapping that builds towards shared understanding of integrated design is one that brings groups together to discuss collectively where learning occurs, making explicit relationships that may not be wholly visible. It surfaces assumptions that may be directing energies in unproductive directions. Just as importantly, when completed as a collective enterprise, mapping becomes a means of generating consensus and collaborative ways to move forward (Jankowski & Marshall, 2017). This toolkit offers means of developing a more collective approach to mapping as well as to exploring the various elements of the learning environment that can be mapped beyond program-level learning. Why Curriculum Mapping? Curriculum mapping is inherently about alignment in educational environments around learning, and as Jankowski (2017) observes, alignment is “a mechanism by which to counteract incoherence and fragmentation of the college experience.” Mapping, therefore, is a strategy for visualizing the areas of where we think learning is happening that relates to specific learning outcomes. Teaching is an inherently collaborative activity, with faculty sharing students across classes, but conversations about how to leverage the collaborative nature of teaching rarely occur. Mapping opens up discussions about what outcomes mean, how they manifest in the curriculum, and how different courses foster shared learning outcomes. Before beginning any mapping experience, we need to be clear on what we are trying to map and why, who should be involved in the process, if we are mapping for purposes of reporting or improvement, and whether we are utilizing multiple lens to capture learning in a wider net. For mapping is undertaken to serve a variety of purposes including: • Provide an overview of the structure of the curriculum and the contribution of individual courses to the goals of the program; Explore alignment within a program, between general education and institutional goals, etc.; Identify where and how particular outcomes are expected, explicitly taught for, and assessed; Backward design the curriculum; Understand the nature and role of course pre-requisites; • • • • 4

  5. • • Identify program strengths - student learning outcomes that are thoroughly addressed Help departments identify gaps - learning outcomes that are addressed by only a few courses; Suggest whether students take courses in an optimal sequence; and/or • Note: An important part of any mapping exercise is to overlay course taking patterns of students. If the students are not moving through the way the curriculum is intended, we would not expect to see the progression in their learning. • Advising tools that provide students with an overview of the role of each course in the curriculum and why some courses should be taken in a particular order. Mapping also occurs at a variety of levels including: • • • • • • Within courses; Program; Between general education and the major; Co-curricular; Institution to mandates or standard bodies; and/or Learning Frameworks (such as Essential Learning Outcomes, Degree Qualifications Profile, Beta Credential Framework) These multiple possible conversations highlight the degree to which mapping functions as a lens. One would not use a microscope to look at the stars any more than one would look through a telescope to see an amoeba. Lenses enable viewers to see some things by screening out others. They focus attention on particular aspects. Learning, after all, does not happen in classrooms and labs alone. Students encounter co-curricular activities that build and reinforce learning, while others bring work experience or campus employment experiences to bear on their learning. Course taking patterns might be mapped to discern how students are moving through the curriculum, a valuable insight since, as Paul Gaston (2015) points out, they often move in and out or shift direction en route to degrees. Maps might also be used to identify relationships between what students learn and the competencies expected by potential employers—including campus employers of students. What lens is applied depends entirely on what question is being asked. Beginning to map, therefore, requires an intentional stance. Five questions can help to promote an intentional mapping effort (Jankowski & Marshall, 2017): 1. Purpose: What are we mapping and why? What pieces of the educational environment need to be aligned? 2. Scope: What parts of the learning environment are included or left out by this approach? 3. Participation: Who should be involved in the conversations? 5

  6. 4. Form: How many layers do our maps need to address educational complexity? 5. Limitations: What ways of seeing are we excluding in our maps? Mapping across a learning environment is productive, because, when done collectively and collaboratively, it begins to bridge the administrative divisions that separate an institution. Oftentimes, knowing how to collaborate between student affairs and academic affairs, for example, can pose a challenge. Mapping, however, engages both divisions in a process of shared discovery and meaning making that can yield organic, synergistic approaches to facilitating student learning. Using Maps Maps or the lessons learned from them need to be shared. For example, curriculum maps can help orient faculty, especially those new to a program, to the ways in which a curriculum is built to facilitate student learning iteratively. For students, as McMahon and O’Riordan (2006) observed, curriculum maps helped increase awareness of the alignment of the curriculum and facilitated better course-taking decisions. What is done with a map depends on what questions they were developed to answer and the context in which they were completed. A key point to bear in mind is that the uses of particular maps will most likely emerge as a result of what is learned. That being said, questions to consider in the useof maps include (Jankowski & Marshall, 2017): 1. Where are curriculum maps located and how can they be shared? 2. With whom might the curriculum maps be shared? 3. How and when will maps be updated for future use? Remember that mapping is as much about the process of seeing relationships as it is about completing a spreadsheet or any other kind of product. By mapping collectively and collaboratively, those involved, whether faculty or staff, are able to unpack assumptions about their own and others’ roles and contributions to the learning of students. 6

  7. Program-Level Curriculum Mapping At a program-level view, curriculum mapping entails exploring the relationships between the courses in a program and the program learning outcomes. In addition to documenting that the learning outcomes are addressed by the courses, figure 3 presents a scaffolded view of learning across a program. The use of (I) for introduced, (D) for developed, and (M) for mastered enables a faculty to focus attention on how learning is scaffolded over the course of the curriculum. Outcome 1 Outcome 2 Outcome 3 Course 1 Course 2 Course 3 Figure 3: Curriculum map showing scaffolding of learning I D M D I D M Program-level maps that bring faculty together to discuss learning help indicate how courses relate to each other, allow space for adjunct and part-time faculty to understand the role of different courses, and reveal if certain outcomes are addressed and reduce redundancy. Some questions to ask when undertaking curriculum mapping at a program-level include: • • • In the key courses, are all outcomes addressed, in a logical order? Do all the key courses address at least one outcome? Do multiple offerings of the same course address the same outcomes, at the same levels? Do some outcomes get more coverage than others? Are all outcomes first introduced and then reinforced? Are students expected to show high levels of learning too early? Do students get practice on all the outcomes before being assessed, e.g., in the capstone? Do all students, regardless of which electives they choose, experience a coherent progression and coverage of all outcomes? What do your electives, individually and collectively, contribute to the achievement of your student learning outcomes? • • • • • • Another layer of mapping at a program-level is exploring where learning is assessed or where artifacts are collected. The following page provides an image of a map based on when learning is assessed in relation to a learning outcome. Several key questions can help to guide mapping endeavors that seek to examine the alignment of curricula (Jankowski & Marshall, 2017): 1. How do courses increase expectations for learning in relation to particular outcomes? 2. How do assignments elicit demonstrations of particular learning outcomes? How are we assessing it and where? 3. How do pedagogies prepare students to make such demonstrations? 7

  8. 4. How do individual faculty/courses each contribute to the collective enterprise of helping students to demonstrate outcomes? In addition, to move from a program view to a wider lens, we recommend using the Questions of Learning developed by Norm Jones and Dan McInerney of Utah State University. See questions at the end of this section. Once maps are completed, they should be shared. For students, viewing a curriculum map at the start of a course and throughout the program help indicate how courses build on each other, showing how the various pieces fit together into a coherent whole. In addition, program- level maps should be shared with advisors to help reinforce the connection points and add in course recommendation decisions. Curriculum maps from a program can also be utilized to provide multiple on- and off-ramps for students as they move through and transfer. 8

  9. Questions of Learning Norman Jones & Daniel McInerney, Utah State University It’s both common and appropriate to think about teaching in terms of our individual interests, assumptions, and goals. The questions below suggest an additional possibility, helping faculty reflect on our work in the classroom from the perspectives of a course, disciplinary curriculum, general education program, and/or set of institutional learning goals. What roles do we play on each of these levels? These questions help highlight what we do, why we do it, and how we know we have achieved it, articulating what students are learning. Understanding my course 1. Who takes my course? 2. Why do they take my course? 3. What are my expectations for students entering my course? What do I assume they already know, understand, and can do? 4. What do I expect students finishing my course to know, understand and do? 5. How do I demonstrate that they know, understand and can do those things? 6. What courses are my students coming from and how does my course prepare them for their next course? Understanding my role in our major 1. Who takes my course in our major, and why do they take it? 2. What are the outcomes for our major? 3. Which of those outcomes are addressed in my course? 4. What can professors who will teach my students next assume they know, understand and can do because of my course? 5. What evidence can I use to demonstrate that students know, understand and can do those things? 6. How do I explain to students in our major the core knowledge and proficiencies their coursework develops? 7. What theories and practices do I use to ensure they have those proficiencies? 8. What pedagogies have I tried that I would not use again? Why? 9. What assignments allow students to demonstrate they have met the outcomes for the major and the course? 10.What outcomes from General Education are reinforced or built upon in my course?

  10. Understanding my role in our General Education program 1. Is my course a general education course, major course, or both? 2. Are there general education outcomes for my course that differ from the outcomes for our major courses? 3. Why is my course a general education course? 4. What do general education courses of this kind (i.e. humanities, life sciences, etc.) assure that students know, understand and are able to do in their domain? 5. How do I explain to non-majors the transferable skills my General Education course develops? 6. What proficiencies in our degree profile does this course strengthen? 7. How can I demonstrate my General Education course’s contributions to the overall degree proficiencies of my institution? 8. How does my General Education course prepare students for the next course they will take? At what level are those courses? (This is not about sequences; it is about general intellectual preparation without regard to a student’s major.) Understanding my role in our degree profile 1. What proficiencies are upper division students [post General Education or Associates degree] at my institution expected to demonstrate? 2. What proficiencies are graduating students at my institution expected to demonstrate? 3. To which of these proficiencies does my course contribute? 4. How can I demonstrate my course’s contributions to our degree profile proficiencies? 5. What proficiencies do my major courses develop best? 6. Which degree proficiencies are not developed in my course?

  11. Introductory Laboratory / Practicum Advanced Advanced Advanced Advanced Capstone Research Content Content Content Content Methods Course D Course A Course B Course C Course Course Course Content SLO 1: Disciplinary knowledge base (models and theories) Mastery / Assessed Introduced Reinforced Reinforced Reinforced Reinforced Mastery / Assessed Mastery / Assessed SLO 2: Disciplinary methods Introduced Reinforced Reinforced SLO 3: Disciplinary applications Introduced Reinforced Reinforced Reinforced Critical Thinking SLO 4: Analysis and use of evidence SLO 5: Evaluation, selection, and use of sources of information Communication SLO 6: Written communication skills SLO 7: Oral communication skills Integrity / Values SLO 8: Disciplinary ethical standards Mastery / Assessed Introduced Reinforced Reinforced Reinforced Mastery / Assessed Introduced Reinforced Reinforced Reinforced Mastery / Assessed Introduced Reinforced Reinforced Reinforced Mastery / Assessed Introduced Reinforced Reinforced Mastery / Assessed Mastery / Assessed Introduced Reinforced Reinforced SLO 9: Academic integrity Introduced Reinforced Reinforced Reinforced Reinforced Center for University Teaching, Learning, and Assessment http://uwf.edu/cutla/ Sample Curriculum Map (Level of Skill) Updated: 24 January 2017

  12. Introductory Laboratory / Practicum Advanced Advanced Advanced Advanced Capstone Research Content Content Content Content Methods Course D Course A Course B Course C Course Course Course Content SLO 1: Disciplinary knowledge base (models and theories) Capstone Portfolio Exam Questions Exam Questions Exam Questions Exam Questions Exam Questions Capstone Portfolio Capstone Portfolio SLO 2: Disciplinary methods Exam Questions Exam Questions Exam Questions SLO 3: Disciplinary applications Exam Questions Exam Questions Class Project Term Paper Critical Thinking SLO 4: Analysis and use of evidence SLO 5: Evaluation, selection, and use of sources of information Communication SLO 6: Written communication skills SLO 7: Oral communication skills Integrity / Values SLO 8: Disciplinary ethical standards Class Capstone Portfolio Term Paper Lab Paper Term Paper Presentation Annotated Bibliography Capstone Portfolio Term Paper Lab Paper Term Paper Reflection Essays Capstone Portfolio Lab Paper Term Paper Term Paper Class Class Class Poster Session Presentation Presentation Presentation Reflective Paper IRB/ACUC Proposal Class Assignments & Exams Reflective Paper Class Assignments & Exams Capstone Portfolio Class Exams & Term Paper Exams & Term Paper Exams & Term Paper Capstone Portfolio SLO 9: Academic integrity Class Exams Assignments & Exams Center for University Teaching, Learning, and Assessment http://uwf.edu/cutla/ Sample Curriculum Map (Assignments & Embedded Assessments) Updated: 24 January 2017

  13. General Education Mapping The next layer to add to a program-level map is considering the relationship between program- level learning outcomes and general education. This map would include exploration of the general education courses that support learning outcomes as well as co-curricular elements and how the integrated learning experience adds up to a degree. Signaling to learners the possible related careers in the map provides a fulsome lens of the entirety of a degree experience within what most would classify as a traditional, four-year institution (see Figure 4). Learning Outcomes General EducationMajor Activities and Experience That Provide Support Possible Careers Courses Learning Outcome 1 General education courses that support the learning outcomes Courses that address specific outcomes Co-curricular elements that support specific outcomes Possible career paths related to the map Learning Outcome 2 Figure 4. Degree-level relationship map Once the relationship between the various elements of the degree have been mapped, it is possible to crosswalk to various learning frameworks. The case study of McKendree University provides an example of such an approach. McKendree University McKendree University engaged with the DQP to refine their Diverse Perspectives outcome, as well as their innovative crosswalk of the DQP’s five areas of learning with McKendree’s seven student learning outcomes, the Association of American Colleges and Universities’ Liberal Education and America’s Promise (LEAP) Essential Learning Outcomes, and the National Collegiate Athletic Association’s (NCAA) Division II Life in the Balance key attributes. Download the full case study 13

  14. Non-Program Learning Mapping For learning experiences that do not equate to the traditional definitions of “programs” – learning can still be mapped whether in relation to employer frameworks, learning outcomes, standards, licensure requirements, and/or the Beta Credential Framework. Further, the assignment toolkit provides information on the ways to use assignments as a means to engage in conversations around learning demonstrations with different standardization bodies, employer communities, and others. The map on the following page provides an example. In addition, it is possible to map learning from a variety of places. While it is not recommended to tackle all of these areas at once, columns can be added one at a time to include additional layers into the conversation. Such a map may entail exploration of the following elements. • • • • • • • • Prior Learning Course Other required courses, recommended electives Activities, experiences that provide support Work-based learning experiences Certifications and Licensures Possible careers Learner Identified 14

  15. Learning Outcomes Prior Learning Courses Other Required Courses, Recommended Electives Activities and Experience That Provide Support Work-Based Learning Experiences Certifications and Licensures Possible Careers Learner Identified Learning Outcome 1 Prior learning that is accepted in relation to specific outcomes Courses that address specific outcomes Other courses that support and reinforce specific outcomes Co-curricular elements that support specific outcomes Employment and other experiences that reinforce specific outcomes Possible certifications connected to the outcomes Possible career paths related to the map Elements identified by learners as supporting learning outcomes Learning Outcome 2

  16. References Jankowski, N. A., Timmer, J. D., Kinzie, J., & Kuh, G. D. (2018). Assessment that matters: Trending toward practices that document authentic student learning. Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA). Jankowski, N. A., & Marshall, D. W. (2017). Degrees that matter: Moving higher education to a learning systems paradigm. Sterling, VA: Stylus Publishing. Gaston, P. L. (2015). General education transformed: How we can, why we must. Association of American Colleges and Universities. For more information, please contact: National Institute for Learning Outcomes Assessment (NILOA) University of Illinois at Urbana-Champaign 360 Education Building Champaign, IL 61820 learningoutcomesassessment.org niloa@education.illinois.edu Phone: 217.244.2155 17