Progress Monitoring in Reading: Why, What, and How A module

Progress Monitoring in Reading: Why, What, and How A module for pre-service and in-service professional development MN RTI Center Authors: Lisa Habedank Stewart, PhD & Adam Christ, graduate student Minnesota State University Moorhead www.scred.k12.mn.us click on RTI Center MN RtI Center 1

MN RTI Center Training Modules This module was developed with funding from the MN legislature It is part of a series of modules available from the MN RTI Center for use in preservice and inservice training: Module Title Authors 1. RTI Overview Kim Gibbons & Lisa Stewart 2. Measurement and RTI Overview Lisa Stewart 3. Curriculum Based Measurement and RTI Lisa Stewart 4. Universal Screening (Benchmarking): (Two parts) Lisa Stewart What, Why and How Using Screening Data 5. Progress Monitoring: (Two parts) Lisa Stewart & Adam Christ What, Why and How Using Progress Monitoring Data MN RtI Center 6. Evidence-Based Practices Ann Casey 7. Problem Solving in RTI Kerry Bollman 8. Differentiated Instruction Peggy Ballard 9. Tiered Service Delivery and Instruction Wendy Robinson 10. Leadership and RTI Jane Thompson & Ann Casey 11. Family involvement and RTI Amy Reschly 12. Five Areas of Reading Kerry Bollman 13. Schoolwide Organization Kim Gibbons 2

Overview This module is Part 1 of 2 Part 1: Why, What, How to Progress Monitor Why do it? What How do we mean by progress monitoring? do you do it? Part 2: Using Progress Monitoring Data MN RtI Center 3

Is this student making progress? Adam Gr 4 Repeated Reading 1:1, 20 min day 140 Repeated Reading 1:1, 10 min 2xday words correct per min (grade level passage) 130 120 110 100 90 80 wrc errors 70 60 50 40 30 20 10 0 MN RtI Center week 4

Assessment: One of the Key Components in RTI Curriculum and Instruction Assessment School Wide Organization & Problem Solving Systems (Teams, Process, etc) MN RtI Center 5 Adapted from Logan City School District, 2002

Assessment and Response to Intervention (RTI) A core feature of RTI is identifying a measurement system Screen large numbers of students Identify students in need of additional intervention Monitor students of concern more frequently 1 to 4x per month Typically weekly Diagnostic testing used for instructional planning to help target interventions as needed MN RtI Center 6

Screening Data and Progress Monitoring can be linked The goal is to have a cohesive system. If possible, use the same measures for both screening and progress monitoring (e.g, CBM). MN RtI Center Screen ALL students 3x per year (F, W, S) Strategic Support and Monitoring Students at Some Risk Intensive Support & Monitoring for Students at Extreme Risk 7

Why Monitor Progress? When teachers USE progress monitoring Students learn more! Teachers design better programs Teacher decision improves Students become more their performance Fleishman, 2005 MN RtI Center instructional making aware of Safer & 8

Why Monitor Progress? “In God we trust All others must have data.” Dr. Stan Deno MN RtI Center 9

Why Monitor Progress? We do NOT KNOW ahead of time whether an intervention will be successful for an individual student Do they ASSUME in the hospital that your heart is working just fine after your bypass surgery? After all the surgery works well for MOST patients . MN RtI Center 10

Why Progress Monitor Frequently? To change what you are doing with a student if it is not working (formative assessment) so you are effective and efficient with your time and instruction To help make decisions about instructional goals, materials, levels, and groups To aid in communication with parents To document progress for special education students as required for periodic and annual reviews MN RtI Center DRAFT May 27, 2009 11 Credit: based on slide by Dr. Kim Gibbons, SCRED

Change (and Keep Changing) if Instruction isn’t Working MN RtI Center 12 Credit: SCRED

You Can Also Use Monitoring Data to Celebrate Success! MN RtI Center 13

What is Progress Monitoring? Standardized measures Reliable Valid Tied to important educational outcomes Given frequently (e.g., weekly) Simple, brief, efficient, and cheap Sensitive to growth over short periods of time MN RtI Center 14

Measurement and RTI: Progress Monitoring Reliability coefficients of r .90 Well documented treatment validity! Test and scores are very sensitive to increases or decreases in student skills over time Evidence of what slope of progress (how much growth in a day, week or a month) is typical under what conditions can greatly increase your ability to make decisions VERY brief, easy to use, affordable, alternate forms, and results/reports are accessible immediately MN RtI Center

Standards for Scientifically Based Progress Monitoring National Center for Progress Monitoring www.studentprogress.org MN RtI Center Reliability Quality of a Good Test Validity Quality of a Good Test Sufficient Number of Alternate Forms and of Equal Difficulty Essential for Progress Monitoring Evidence of Sensitivity to Intervention Effects Critical for Progress Monitoring Benchmarks of Adequate Progress and Goal Setting Critical for Progress Monitoring Rates of Improvement are Specified Critical for Progress Monitoring Evidence of Impact on Teacher Decision-Making Critical for Formative Evaluation Evidence of Improved Instruction and Student DRAFT May 27, 2009 Achievement Gold Standard 16

Many tools that meet standards are members of the Curriculum-Based Measurement (CBM) “family.” www.studentprogress.org click on “Tools” MN RtI Center 17

Buyer Beware! Many tools may make claims about monitoring progress but Is it reliable and valid? Is it tied to important educational outcomes? Can it be given frequently (e.g., weekly)? Simple, brief, efficient, and cheap Sensitive to growth over short periods of time MN RtI Center 18

Short Term (Mastery) and Long Term Progress Monitoring Short Term Mastery Monitoring Test subskill mastery and individual lesson effectiveness Ex: Q&A, worksheets following directions unit tests, “hot” reads accuracy, skills “checks” CBE, cba MN RtI Center Long Term General Outcome Measures Test retention, generalization and progress toward overall general outcome (reading) Ex: CBM, DIBELS 19

Both Mastery Monitoring and Long Term Progress Monitoring are Important Sometimes mastering subskills doesn’t generalize to the general outcome or students don’t retain the information over time For example: Melissa is very good at decoding letters and reading individual words, but is not generalizing these skills to reading text with automaticity and comprehension. Adam was really good at using his comprehension strategies and using those when they were working on these skills in class (showed mastery), but when they moved on to another unit he quit using the strategies. MN RtI Center 20

How Often Do I Need to Monitor Progress? Informally we do this all the time! For small instructional adjustments the lesson, how much help to give, etc.) (repeat In a standardized way to make sure we are “on track” with this student?- depends on level of concern For students in reading who are behind already, monitor progress toward generalized outcome 1 to 4x per month, ideally weekly MN RtI Center 21

How Do You Collect Frequent Progress Monitoring Data? Which students? What measures? What materials? How often? Who collects the data? Where? When? MN RtI Center 22

Which students? Students of concern Below target Getting “extra” intervention or help Tier 2 or Tier 3 services MN RtI Center 23

What measures? Web-based sources for information on measures as well as access to materials, web-based data management, etc. www.aimsweb.com dibels.uoregon.edu www.edcheckup.com Disclaimer: More data management systems are being developed and marketed all the time to provide information, resources, and data management. This is just a sample of the types of web-based resources available. Be a critical consumer! MN RtI Center 24

What materials? When possible, students are monitored using grade level materials E.g., student reads a different grade level passage or “probe” each week If this is not possible due to frustration “test down” and use the highest grade level of materials possible Periodically “check” how the student is doing on grade level materials and move into grade level materials as soon as possible MN RtI Center 25

Example of Testing Down: Reading “Survey Level” Data for Gus, Gr 4 200 190 180 170 Note: 160 -slightly lower (4-8) in Gr 3 & 2, -and much lower (1-4) in Gr 1. 150 140 words correct per minute -Error rates high (5-15) in Gr 4, 130 120 110 100 90 80 70 60 50 40 30 20 10 MN RtI Center 0 Grade 1 Grade 2 Grade 3 Grade 4 Grade 5 26 Grade 6

What Material Should We Use To “instruct” Gus? To monitor Gus’ progress? MN RtI Center 27

How Often Do We Monitor Progress? Depends on the sensitivity of the measure and the level of concern we have about the student, but 1 to 4x per month typically For CBM Oral Reading Fluency Weekly with 1 passage (this is most common) Every 3 weeks with 3 passages MN RtI Center 28

Who does it? When? Where? Anyone trained in the procedures can collect progress monitoring data Classroom teachers, special education teachers, Title teachers, aides, related services staff, adult volunteers Be creative but careful When and Where? At a time and place that will provide valid information Use common sense MN RtI Center 29

Sharing the Data Just having progress monitoring data is not enough. You need to USE it. Scheduled graph review dates Grade level meetings Problem solving meetings MN RtI Center 30

Using Progress Monitoring Data: Is this intervention working? words correct per min (grade level passage) See MN RTI Center Module 2 for information on graphs, decision rules, etc Adam Gr 4 MN RtI Center 140 130 120 110 100 90 80 70 60 50 40 30 20 10 0 w rc errors 31

Remember: Garbage IN . Garbage OUT . Make sure your data have integrity or they won’t be good fer nuthin Training Integrity checks/refreshers Well chosen measures and materials MN RtI Center 32

Avoid Common Mistakes Don’t use the same passage/probe every week! Have an organized system in place Progress monitoring schedule for students Preprinted passages/probes in a binder An easy way to graph and look at the data Scheduled time to share/look at the data MN RtI Center 33

Remember: When teachers USE progress monitoring Students learn more! Teachers design better programs Teacher decision improves Students become more their performance Fleishman, 2005 MN RtI Center instructional making aware of Safer & 34

Web Resources Research Institute on Progress Monitoring http://progressmonitoring.net/ Includes A Study Group Content Module with 15 sections on CBM including activities http://progressmonitoring.org/pdf/cbmMOD1.pdf Progress Monitoring Leadership Team Content Module with 6 sections (e.g. measureable goals, decision making) including activities http://progressmonitoring.org/pdf/cbmMODldrshp.pdf Handouts, videos, and power point presentations Technical reports of CBM measures MN RtI Center 35

Web Resources, Cont’d www.studentprogress.org http://www.rti4success.org/ click on Progress monitoring on right side www.interventioncentral.org http://www.studentprogress.org/chart/chart.asp look for information on CBM, graphing, etc. www.aimsweb.com, www.edcheckup.com, dibels.uoregon.edu Look for information about progress monitoring as well as access to materials and graphing for progress monitoring MN RtI Center DRAFT May 27, 2009 36

Recommended Texts Riley-Tillman & Burns. (2009). Evaluating Educational Interventions. Guilford Press. MN RtI Center 37

Print Resources available with this module Safer & Fleishman. (2005). How student progress monitoring improves instruction, Educational Leadership, 62(5), 81-83. Fuchs. Progress monitoring within a multi-level prevention system. Retrieved June 5, 2009, from RTI Action Network Web site: http://www.rtinetwork.org/Essential/Assessment/Progress/ar/MultilevelPrev ention Fuchs & Fuchs What is scientifically-based research on progress monitoring? From the National Center on Student Progress Monitoring (studentprogress.org). Retrieved June 14, 2009, Jenkins, Hudson, & Hee Lee. Using CBM-Reading assessments to monitor progress. Retrieved June 5, 2009, from RTI Action Network Web site: http://www.rtinetwork.org/Essential/Assessment/Progress/ar/Using CBM/1 MN RtI Center 38

Activity for Teachers or Practicum Students Obtain progress monitoring probes and graphs Passages and graphing materials self-created or downloaded Sign up for an account with AIMSweb (instructor accounts and student accounts available) www.interventioncentral.org dibels.uoregon.edu www.aimsweb.org Practice administration and scoring Progress Monitor a “real” kid (ideally 2-4 kids of varying risk levels monitored for at least 7-10 weeks) Graph, analyze, and use data MN RtI Center 39

Quiz 1.) What are at least four reasons why teachers should monitor progress? 2) What is the difference between mastery monitoring and frequency progress monitoring toward a general outcome? 3) How often should you monitor progress for students receiving extra help ? MN RtI Center DRAFT May 27, 2009 40

Quiz (Cont’d) 4.) When possible, students are monitored using A.) above grade level materials. B.) grade level materials. C.) below grade level materials. 5.) Who can collect progress monitoring data? MN RtI Center 41

Quiz (Cont’d) 6.) To have integrity, your data collection must include what? A.) trained data collectors B.) integrity checks/refreshers C.) well chosen measures and materials D.) all of the above MN RtI Center 42

Quiz (Cont’d) 7.) True or False? The most important use of frequent progress monitoring is to aid in communication with parents. MN RtI Center

Note: The MN RTI Center does not endorse any particular product. Examples used are for instructional purposes only. Special Thanks: Thank you to Dr. Ann Casey, director of the MN RTI Center, for her leadership Thank you to Aimee Hochstein, Kristen Bouwman, and Nathan Rowe, Minnesota State University Moorhead graduate students, for editing, writing quizzes, and enhancing the quality of these training materials MN RtI Center