The Family Center on Technology and Disability
Assistive Technology Works
Newsletter for June 2009
June 2009 - Computer-Based Assessment and Instruction : Do They Make the Grade? |
|
Computer-Based Assessment and Instruction: In this Issue... This issue of FCTD News and Notes examines computer-based assessment and instruction from the perspective of individuals who are responsible for creating and administering several such programs for an expanding public school user base. Mel Lee and Michael Kasloff Speak Mr. Lee, Wireless Generation’s Director of Assessment, began his technology career as a contractor for the U.S. military. However, he says, “I believed I could make better use of my skills, so I joined the Peace Corps.” While with the Peace Corps, he spent two years in Ghana, teaching high school and participating in initiatives such as the USAID/Quality Improvement in Primary Schools program and providing training to district monitoring and evaluation teams. He also managed Peace Corps partnership projects, including one that resulted in the construction of the first computer facility in a Ghanaian school district. For his efforts, Mr. Lee was named first runner-up in the Foreign Education Volunteer of the Year in recognition of his activities in Ghana. Mr. Lee received B.S. and M.S. degrees from UCLA and an M.B.A from the Fuqua School of Business at Duke University. Mr. Kasloff, Wireless Generation’s Director of Curriculum and Instruction, spent 10 years in the New York City public schools as a middle school math teacher and, most recently, as an administrator. He was director of operations for NEST+m (New Explorations into Science, Technology and Math), the first K-12 public school in New York. Earlier, Mr. Lee was a member of the NEST+m design team tasked by New York City Schools Chancellor Harold Levy (2000-2002) with the creation of a K-12 model for the city’s sprawling school system. During his long stint with the New York public schools, Mr. Lee became aware of Wireless Generation’s efforts with hand-held assessment and instruction technology. “That was the first time there appeared to be a way I could use technology meaningfully in my school’s classrooms, because until then I had seen technology implemented so poorly there,” he recalls. Mr. Kasloff holds a Masters in Education from New York’s Bank Street College of Education and dual degrees in business management and computer science from the University of Pennsylvania. Supporting our interview with Mr. Lee and Mr. Kasloff are resources related to computer-based assessment and instruction. We also feature members of our Knowledge Network. The members spotlighted this month focus on computer-based classroom assessment and instruction as well as education technology in general. We invite you to contact these members for further information. Please share this newsletter with other organizations, families and professionals who may benefit from it. We invite you to visit us at http://www.fctd.info. We welcome feedback, new members and all who contribute to our growing knowledge base. Computer-Based Assessment and Instruction: An Interview with Wireless Generation’s
Computer-Based Assessment In grades K-6, formative assessment is observational and done one-on-one. Teachers assess at the beginning, middle, and end of year to collect benchmark scores that reveal how students are progressing toward learning key skills. Teachers also do what is known as “progress monitoring assessments” in between the benchmark administrations to gather additional information about each student’s strengths and needs. According to Mr. Lee, the Wireless Generation co-founders watched teachers give these assessments in elementary classrooms, juggling stopwatches, papers, pencils and other materials, then completing the copious paperwork involved in scoring, recording, and analyzing the data. The paper reports generally wound up in filing cabinets, snapshots of student learning no longer considered in making instructional decisions.
Working closely with university researchers and authors of common early reading and math formative assessments, Wireless Generation developed mCLASS software, which enables teachers to give the assessments more easily, efficiently, and accurately with a handheld device. While students use paper-based materials to do math and reading tasks, teachers record their responses and behaviors directly on the handheld. Because timing is automated, teachers can focus on students rather than administration protocols. Scoring is also automated and occurs immediately, giving teachers real-time information about students’ needs. Teachers “sync” their handheld devices to computers, and student data is uploaded to a secure Web site, where they, as well as principals and administrators, can view a range of different kinds of reports for analysis and instructional planning. Administrators have access to aggregate reports and can pull out data for virtually any student sub-population, determining how to best meet their particular needs. Teachers use mCLASS software to give a range of formative assessments including DIBELS (Dynamic Indicators of Basic Early Literacy Skills), developed by researchers at the University of Oregon, and the TPRI, developed by researchers at the University of Texas Center for Academic and Reading Skills, and at the Texas Institute for Measurement, Evaluation and Statistics at the University of Houston. Formative assessments for English language learners are also supported. Universal design principles vary, with the assessment authors providing guidance on whether the measures are appropriate for some or all students (ie, http://www.dibels.org/papers/Appropriateness_0207.pdf). “Teachers became big fans of the software, especially those who had been using paper and pencil for a long time, because it cuts the assessment time in half,” says Mr. Lee. “We can return that time to the classroom, which is great.” Lee continues, “Because formative assessment is meant to help teachers understand what instruction they should be providing to which students and when, the software had to do more than just save time. We’ve been fortunate to work closely with many schools and districts, and their feedback pointed the way to where technology could provide the greatest value in linking assessment and instruction.” What followed were new technology tools and features for teachers to move from data to action. “Teachers would say, ‘I have my data, but now what?’, so the Now What? tools are designed to help answer that question,” says Lee. “We have a Web report that lets teachers review students’ answers during a particular task; the accompanying Item-Level Advisor tool highlights and explains the significant errors and response patterns right on that report. Small Group Advisor is an instructional planning tool that the teacher uses to put her students into small groups according to what the data reveals about their shared needs, so that she can follow up with the right instruction for the right kids.” Similar support tools were introduced on the handheld device. The “Act” feature recommends instructional activities for individual students based upon their assessment results. mCLASS:Math guides the teacher through the process of diagnostically interviewing a student to probe his skills and strategies more deeply, and use the findings to make instructional decisions. “We recognize that every interaction a teacher has with software or other technology is an opportunity for professional development,” notes Mr. Kasloff. The mCLASS Web reports, for example, “avoid the familiar two-dimensional, flat approach to spitting out numbers and data for teachers. From the beginning, we have paired data with guidance, which helps the teacher to deepen her professional knowledge and skills in working with data.” Coping with the Bell Curve Many schools use technology-based assessment in their Response to Intervention (RTI) programs. “Teachers can do this kind of technology-based assessment more frequently. The progress of at-risk students can be monitored as often as every 1-2 weeks. These checks are incredibly important so that classroom time is maximized – if a particular intervention isn’t working for a struggling student, teachers know to change it.” Lee has spearheaded the development of a special version of mCLASS for RTI that allows general education teachers and special education staff to capture and share not just data, but also the specific interventions given, and for how long, the multi-dimensional tracking that is characteristic of formal RTI implementations. Computer-Based Instruction with a Teacher at the Center Burst:Reading is distinct from other paper-based and computer-based programs because it is embedded with new, patent-pending technology that matches curriculum to students’ needs based upon past and current assessment data. “This was a logical extension of our work in the formative assessment area,” says Mr. Kasloff.
The technology applies an algorithm that analyzes the data, groups students in the class who share needs, and creates a customized 10-day sequence of lessons at the right pace and intensity for each group. The teacher downloads the lessons, teaches for 10 days, and re-assesses her at-risk students. The whole process then repeats with the new assessment data informing the analysis, grouping, and selection of lessons. “It takes many hours outside of class time for a teacher to analyze data for each student, create groups for follow-up instruction, and then cull through instructional materials to find the right lessons for each day. Even the most accomplished, dedicated teachers can find it challenging,” says Mr. Kasloff. “This is another great example of how technology, when applied appropriately to classroom activities and with respect for the unique, human characteristics of teaching and learning, can help to make instruction more effective.” Flexible Instructional Content Universal design principles are kept in mind. The content of the Burst:Reading lessons is always multi-modal. For example, a sub-skill in literacy is the ability of students to distinguish sound and language. The skill can be taught using a physical manipulative employing rhymes and poems, movement of the body, in some cases even using a hand-puppet that simulates generation of language. The lesson includes these different modalities with the goal of meeting students where they are. Additional modalities can be added if necessary. Cultural relevance is also considered, and content can be instantly tweaked. “In terms of cultural relevance, our goal is to help teach students various ways to comprehend text,” Mr. Kasloff remarks. “This has become an increasingly sophisticated effort using a carefully crafted set of stories. To get to the approximately 10 stories needed for a specific instruction strand, we will produce approximately 40 stories around different themes and topics, and field test them broadly until we find the stories that have the most cultural relevance for students.” Computer-Based Programs in the Home For parents or guardians who want instructional content for use at home, Mr. Kasloff recommends FreeReading.net, a free open source reading intervention program also launched by Wireless Generation with an advisory board of leading reading researchers. Burst:Reading draws upon content from FreeReading.net to generate lessons. “FreeReading.net is a great resource to use at home,” says Mr. Kasloff. “It’s a research-based program that is constantly being improved and enhanced through the contributions of many thousands of teachers and researchers around the world. There is also a complete, sequenced curriculum on the site called Intervention A, also available for free.” The Last Word RESOURCES Articles
This article reports the findings of a pilot study investigating whether computer–based testing with text-to–speech (CBT-TTS) is an effective approach for providing individualized support to students with learning disabilities. Participants in the study were students grades 9-12 identified by resource room teachers. Students were tested on equivalent forms of a test using CBT-TTS technology and paper and pencil testing (PPT). The testing procedure, data collection and data analysis are described in detail in the article. Results indicated that students performed significantly higher when responding to items related to longer passages when using the CBT-TTS than PPT. Students also reported positive impressions of the CBT-TTS accommodations. Overall the study indicates that “providing computer based read-aloud support to high school students with learning disabilities can improve their performance in a multiple choice United State history and civics test.”
Middle Schools Adopt Gaming in Response-to-Intervention Strategies
Response to Intervention, Universal Design for Learning: Resources for Implementation There is still time to register for the The Institute will take place This year’s topics include:
The FCTD Institute is offered entirely online, so that you can participate when and where it is most convenient for you. The Institute web site will be available 24/7 for you to read and post questions and comments. Discussion moderators will check in several times a day to post their comments and to respond to participant questions. We look forward to your participation. For more information or to register, visit KNOWLEDGE NETWORK MEMBERS State Educational Technology Directors Association (SETDA) Center on Teaching and Learning (CTL) – University of Oregon CTL produces DIBELS (Dynamic Indicators of Basic Early Literacy Skills), a well-known nationally normed early reading assessment. DIBELS are a set of procedures and measures for assessing the acquisition of early literacy skills K-6. They are designed to be one-minute fluency measures used to regularly monitor development of early literacy and reading skills. DIBELS was developed to measure recognized and empirically validated skills related to outcomes. Each measure has been thoroughly researched and demonstrated to be reliable and valid indicators of early literacy development and predictive of later reading proficiency to aid in the early identification of students who are not progressing as expected. For further information, contact:
The National Repository of Online Courses (NROC) project supports the development and distribution of high-quality online courses to a worldwide audience. The goal of this project is to facilitate collaboration among a community of developers to create a library of online courses that are available to everyone. The goal of the MacArthur Series on Digital Media and Learning is to support discourse and help build an emerging field of study. The project publishes the Journal of Digital Media and Learning with MIT Press.
University of Texas Health Science Center’s Children’s Learning Institute (CLI): TPRI CLI is home to the Texas Primary Reading Inventory (TPRI), a valid and reliable assessment tool that provides a comprehensive picture of K-3 students’ reading proficiency. At each grade level the TPRI consists of a screening and an inventory. The screen permits the rapid assessment of individual students. Designations of risk status are yielded, which identify students who likely do not require additional assessment. The inventory is a detailed assessment of reading and reading-related skills that allows the teacher to gain more in-depth information that can be used to determine a child’s level of risk for reading problems. The screen and inventory, which help the teacher set learning objectives for each student, are individually administered and are designed to be given by a train ed teacher. For more information, contact: Funding provided by the US Department of Education under grant number H327F080003 Project Officer: Jo Ann McAnn |
As the digital age remakes almost all aspects of society, including K-12 public school education, it is no wonder that computer-based assessments and instruction are beginning to make inroads into the classroom, where paper-based approaches have held sway for generations. Some computer-based assessment and instruction innovations, mainly focusing on reading and math, have already taken root and have been well-received by educators, as well as parents. Other approaches and tools are emerging from the development stage. Although the U.S. public school classroom, which has generally been reluctant to embrace technology-based appraisal and teaching tools, has evidenced enthusiasm for these technology-based innovations, questions remain about the value of the tools to both general and special education students.
Mel Lee and Michael Kasloff share a passion for finding ways to help teachers and students to be successful. It was that passion that drew them to Wireless Generation. This Brooklyn, NY-based company was founded nine years ago by two Rhodes Scholars, Larry Berger and Greg Gunn, who believed that technology could help educators to improve teaching and learning. Mr. Lee and Mr. Kasloff were particularly attracted by the founders’ philosophy of immersion in the workings of schools and current education research in order to develop technology solutions that support teachers in providing effective instruction.
“Larry and Greg started Wireless Generation after having watched schools purchase a lot of education technology products that sat in boxes, gathering dust because they were hard for teachers to use, or didn’t address actual challenges,” says Mr. Kasloff. “They believe that technology does have a place in schools, but that providers have an obligation to develop products and services that reflect a deep and accurate understanding of how teaching and learning actually happen.”
“Technology has clearly brought great efficiency to other professional fields,” says Mr. Lee. “Technology-based formative assessments could make the entire process much less burdensome for teachers, and much more valuable.”
In a class in which the bulk of the students are in the bell curve of grade-appropriate math, a much smaller percentage may need remediation because they are at least a year behind their grade in math, and a similar-sized group may be above grade level. Are these computer-based programs effective for all three groups? 
A student seated in front of a computer clicking and listening to an unseen instructor via headphones is a familiar image. When Wireless Generation developed its Burst:Reading technology-powered intervention for grades k-3, it decided that the teacher had to remain at the center of the action. “Having the teacher deliver the instruction, and engage and inspire students is particularly important when you are intervening in the early grades,” says Mr. Kasloff. “We recognized that technology could help every teacher to be adept at differentiating instruction for their students, with all of their varied needs, but as in our approach to assessment, technology is a support, not a replacement.”
“One of the great benefits of computer-based instructional materials is that they are easily updated or modified to meet students’ needs. Traditional printed textbooks are updated only every 5-7 years,” says Mr. Kasloff. “Computer-based programs are far more flexible and conducive to tailoring instruction to the needs of the individual learner than having a teacher take an entire class of 25 kids through a textbook in lockstep, regardless of where each child may be in his development.”
Mr. Lee explains that Wireless Generation’s assessment offerings, with one exception, are not designed for in-home use. But to involve parents in their child’s education, many school districts using mCLASS assessment software also subscribe to a feature called Home Connect. After each benchmark assessment, the teacher sends a letter home to each child’s parent or guardian, explaining the assessment results in clear, easy-to-understand language and suggesting learning activities to do at home. Many teachers also use the Home Connect letters as part of parent-teacher conferences. The letters can be written in multiple languages.
“Traditionally, K-12 education has been slower to embrace technology than other industries and areas of society. Nevertheless, we have found that if the technology is designed well for the tasks at hand, educators at all levels of technology engagement, from infrequent users to experts find it easy to use and beneficial,” says Mr. Lee.
Founded in 2001, the organization is the principal association representing the state directors for educational technology. SETDA membership includes ed tech directors and staff from state education departments of 50 states, the District of Columbia and American Samoa. For more information, contact:
Part of the University of Oregon’s College of Education, the center is a network of inclusive learning communities, undergrad and graduate students. Graduates become teachers, administrators, clinicians, social service professionals and educational research scientists. CTL research faculty and grad students collaborate with school districts and agency partners. In addition to academic programs, the center includes an alliance of nationally prominent organizations, institutes and affiliated research and outreach units.
The Monterey Institute for Technology and Education is an educational non-profit organization committed to improving access to education. MITE (pronounced “mighty”) manages a range of projects from establishing systems for the development and distribution of open educational content to efficacy studies and other educational research. Two of MITE’s projects are the National Repository of Online Courses and the MacArthur Series on Digital Media and Learning.
EdTech Associates offers AT consulting services to help parents, individuals and schools select and use available technologies designed to support children with learning disabilities via AT evaluations and training services. AT evaluation includes an initial consultation with parents, a review of records, an on-site consultation and observations, an evaluation session with the student, a follow-up appointment to review findings and recommendations, a written report completed within four weeks of the evaluation session and personalized training on application software and AT tools for students and/or parents. For additional information, contact:
The institute combines data and studies from psychology, neuro-development, education and child development to provide solutions derived from, and supported by, documented research. CLI services include clinical assessment, diagnosis and treatment of learning disorders and research on techniques to enhance a child’s home and learning environment. The institute staff includes experts in child development, education, medicine, neuro-development and research analysis.