The Family Center on Technology and Disability

The paradigm has remained unchanged since the Gutenberg bible was produced on a printing press, since the Model-T replaced custom-built one-of-a kind automobiles: less expensive ubiquitous technology eventually subsumes more expensive specialized technology thereby benefiting many instead of a few. According to Richard Ladner, Ph.D., a professor of computer science at the University of Washington, that age-old paradigm will hold true for consumer technology applications employed by children with disabilities.

The emergence of consumer technology with AT-type applications as a factor in the education of students with disabilities is already becoming evident as these apps enhance digital age accessibility for children and students with special needs. Increasing in popularity and use as mobile learning tools are cellphones, smartphones, hand-held MP3 players and personal digital assistants. Because so many students, including those with disabilities, are adept with these products’ standard technologies, educators have begun incorporating them into classroom settings. Some mobile technologies duplicate apps available on desktops and laptops, while others make use of the features of handheld devices.

The functionalities of PDAs and smartphones now extend far beyond calendar and address management, their original purposes.  Computers and hand-held devices can run thousands of programs, including educational tools for children with disabilities. For example, iPhone apps include a pocket communication device for speech-impaired users. Students can use mobile and smartphones and PDAs to set a reminder to take medication, to store lesson notes and assignments and to record lessons. They can use a mobile dictionary to check spelling or load photos and maps of unfamiliar locations. Teachers can use them to transmit messages to a parent.

According to Dr. Ladner, using Google Maps on a cellphone or smartphone transforms the device from a phone into a simple GPS tool that can aid students in navigating neighborhoods and build confidence in navigational skills, enhancing a child’s independence.  In fact, some high school students on the autism spectrum have reportedly employed iPods to load social video scripts, social cues and other content to help them cope with unfamiliar social situations.

In the meantime, the evolution of consumer technology continues to transform the utility of existing technology: smartphones are taking on the characteristics of PDAs by providing Internet access, video and a continually morphing smorgasbord of other apps; PDAs are becoming more akin to phones by offering dialing and Bluetooth capabilities. 

Although mobile devices remain prohibited at many schools nationwide, these examples of ubiquitous consumer electronics offer the potential to aid users by employing third-party apps, visual and audio reminders, predictive word software and voice recognition. The hope is that as teachers and administrators become more aware of the benefits of these devices and applications for students with special needs, their classroom use will be permitted and encouraged. 

“Much of the consumer technology that surrounds us can be adapted for classroom and non-classroom educational use for children with disabilities in the K-12 range and beyond,” Dr. Ladner insists. “As less expensive consumer technology takes on universal use,” he adds, “parents and teachers will eventually come to see that the academic horizons of many children with disabilities need not be limited to the K-12 timeframe.”  In fact, Dr. Ladner, who has worked closely with deaf and blind students for many years, sees the advent – and acceleration – of consumer apps use by students with disabilities ultimately resulting in a dramatic increase in the number of deaf and blind students achieving Ph.D. status in the years ahead.  Rather than simply being consumers he envisions that students with disabilities will become computer professionals who will be creators of apps that make life easier for themselves and others like them.

Richard Ladner, Ph.D., Speaks
He’s the Boeing Professor in Computer Science and Engineering at the University of Washington who earned his bachelors degree in 1965 yet remains fascinated by consumer technology’s fast-evolving potential for those with special needs. He devoted years of his career to research in theoretical computer science before returning to his roots: both his parents were deaf. He’s turned his attention to accessibility technology research, especially for the deaf, deaf/blind, blind and hard-of-hearing. He leads the AccessComputing Alliance which has the goal of increasing the number and success of students with disabilities in computing fields (http://www.washington.edu/accesscomputing/ ).

According to Dr. Lander, his career emphasis pivoted on two salient moments.

“I’ve always had an interest in the accessibility of technology because I saw the impact of TTY on my parents’ lives. I also had many deaf and deaf/blind friends here in Seattle. Technology innovations in the 90s, like instant messaging and the development of TTY relay services, had a real impact. In the mid-80s, for example, I procured some computers for a TTY relay service. I donated them after obtaining them from IBM. They were old PCs. They were used for the local TTY relay service before it became ubiquitous nationwide.”

In 2002, he recalls, the arrival of a blind student at the University of Washington had a significant and pivotal impact on his life and work. “Sangyun Hahn (http://www.cs.washington.edu/homes/syhahn/), from Korea, was and is totally blind. Because of my interest I was assigned to be his temporary advisor. I learned that he was experiencing difficulty in some classes because he was unable to access materials, especially textbook graphics, in a timely manner.

Corel/DRAW (http://www.corel.com/servlet/Satellite/us/en/Product/1208530085783?trkid=NASEMGglGR#tabview=tab7) was used to bring in a figure that the user wanted to make acceptable, trace it out, modify or simplify it, type in the Braille, print it on a Tiger embosser (http://www.viewplus.com/products/braille-printers/elite-braille-printers/), put it on black and white and have it on swell paper.”

That method, Dr. Ladner remembers, “was rather primitive. I devised some alternative methods to speed that process. This investigation on Sangyun’s behalf got me started in the development of my current Tactile Graphics Project (http://tactilegraphics.cs.washington.edu/), which features Tactile Graphics Assistant software and is aimed at improving blind students’ access to graphical images.

“Sangyun was very helpful during the initial two years of this discovery process, which included participatory design. That started the ball rolling. Then I got some really good grad students, including Jeff Bigham.”  Dr. Bigham (http://www.cs.rochester.edu/~jbigham/), now an assistant professor of computer science at the University of Rochester, aided Dr. Ladner on several groundbreaking research projects dealing with the adaptation of ubiquitous consumer technology for deaf and blind students.

Dr. Ladner’s second pivotal moment, he says, involved his longtime UW research colleague, Eve Riskin (http://www.ee.washington.edu/faculty/riskin/).  “Eve asked, ‘Why don’t we do video compression so that we can turn a smartphone into a video phone for deaf people?’ She said to me, ‘You’re one of the few people who knows anything about video compression because we’ve been working on it for several years.’ She was right, but I’d never thought about this application and its possible benefit to deaf individuals and deaf society. Eve persuaded me to become involved in the Mobile ASL Project (http://mobileasl.cs.washington.edu/), which was designed to help make wireless cellphone communication a reality for deaf Americans. That project remains operational. We’ve implemented it, it works and we’re demonstrating it for companies.”

Today, under the banner of developing educational technology for deaf and blind students anywhere, Dr. Ladner is focused on four initiatives currently underway, three of which require no “special” hardware or software.

For blind students:
WebAnywhere is a screen reader for the web that provides free access to the web anywhere via a webpage.(http://webanywhere.cs.washington.edu/)

Tactile Graphics Project not only improves blind students’ access to graphical images but also accelerates the translation of math and science figures.

For deaf students:
ClassInFocus (http://dhhcybercommunity.cs.washington.edu/projects/classinfocus/index.html), is an online multimedia classroom web-based conferencing platform for deaf and hard of hearing students to access remote interpreters and captioners and improve their visual access to live, in-person classes.

American Sign Language ASL-STEM Forum (http://aslstem.cs.washington.edu/), is a collaboration of educators, interpreters, captioners, students, science professionals and others aimed at helping to build ASL’s technical vocabulary.

Supporting our interview with Richard Ladner are resources related to the adaptation of consumer technology for students with disabilities. We also feature members of our Knowledge Network. We invite you to contact these mem¬bers for further information. Please share this newsletter with other organizations, families and professionals who may benefit from it. We invite you to visit us at http://www.fctd.info. We welcome feedback, new members and all who contribute to our growing knowledge base.  

 

Consumer Apps as AT:  They’re Not Just for College Students

An Interview with Richard Ladner, Ph.D., Boeing Professor in Computer Science and Engineering,
University of Washington
 

Until recently, much of Richard Ladner’s and his students’ work regarding the use of select AT-type consumer technology applications, he says, was perceived to be beneficial mainly for college students. Four of his team’s recent projects, however, belie that perception. All, he says, can benefit K-12 students too.

Designed for blind students, WebAnywhere (http://webanywhere.cs.washington.edu/) is a screen reader that provides free web access. The Tactile Graphics Project (http://tactilegraphics.cs.washington.edu/) is designed to improve blind students’ access to graphical images by accelerating the translation of math and science figures in textbooks into a tactile form.

For deaf students, Dr. Ladner’s teams created ClassInFocus (http://dhhcybercommunity.cs.washington.edu/projects/classinfocus/index.html), an online multimedia classroom conferencing platform for deaf and hard of hearing students to access remote interpreters and captioners and improve their visual access to live, in-person classes. They also created the ASL-STEM Forum  (http://aslstem.cs.washington.edu/), a collaboration of educators, interpreters, captioners, students and others aimed at building the  technical vocabulary in science, technology, engineering, and math (STEM) fields for American Sign Language (ASL).

WebAnywhere: Providing Access to Computers and the Web
WebAnywhere was originated, he explains, “as a system that enabled a blind person to go into a library or an Internet café and plug headphones into the computer, bring up WebAnywhere and travel the web. There’s no special software installed. WebAnywhere is a screen reader as a web service. The user installs nothing beyond what is normally on a computer. He or she just runs a webpage.”

Created for “anyone of any age who can use a computer and wants web pages,” WebAnywhere, Dr. Ladner insists, is useful for students K-college “as well as for older individuals who are going blind.  The large print is helpful.  Jaws screen readers (http://www.freedomscientific.com/products/fs/jaws-product-page.asp) don’t highlight like WebAnywhere, nor does Jaws have our large print. Unlike Jaws, however, WebAnywhere only does the web.”

 The principal creator of  WebAnywhere was Jeff Bigham, who was a UW graduate student when the project was conceived in 2007 and is now a 20-something assistant professor at the University of Rochester. “Jeff is an astounding young man,” Dr. Ladner notes. “He won the TR-35 Award (http://www.technologyreview.com/tr35/) presented by the MIT Technology Review to under-35 innovators. He invented WebAnywhere.”

The project also was a recipient of a Mellon Award for Technology Collaboration. “We’re still spending the grant we were awarded for  the open-source project,” Dr. Ladner says.

Although WebAnywhere was created with blind students in mind, he comments, “our screen reader is appropriate for students with print disabilities, including dyslexia, because we highlight whatever is being read by the student.”

For students with low vision, he adds, “there’s a large-print version at the top of the screen that scrolls.  We’ve added a Universal Design element to make it useful for students who are not blind.”

WebAnywhere provides free web service and uses a web proxy  – which acts as an intermediary – to transform web pages to be navigated and read out loud. The use of a web proxy may result in the user being blocked from some sites. The proxy, he explains, intercepts web requests, inserts speech and highlighting and delivers new pages to the user.  While the proxy means that WebAnywhere sometimes cannot be used in a classroom setting, “it’s excellent for use in a library or at home.”  

Considered by Dr. Ladner to have been among the major breakthrough access technologies of the past quarter-century, screen readers, he says, “have made computers accessible to blind people – if they can afford it.” While good screen readers, he notes, cost around $1,000, “the computer itself – a PC – can be bought for as little as $300.”

Because WebAnywhere is both effective and free, he says, it has attained a level of popularity and use that was unanticipated. “We have a server here at UW that I’m upgrading because user volume is so high.”  

The WebAnywhere System: How It Works
WebAnywhere “is a webpage,” Dr. Ladner explains. “User requests aimed at the web go through the proxy. The proxy analyzes the page and determines what ought to be spoken and generates the appropriate speech to the browser. The speech is then played on an embedded player in the browser, a Flash Player, for example. The files are MP3 files. The page that’s delivered is transformed by inserting appropriate Javascript. The page that’s brought in is a sub-page. The WebAnywhere information is at the top of the page. The original page is at the bottom of the screen.”

Jeff Bigham, he says, “did some deft work in arranging for real-time speech generation, which is slower now due to the current high user volume we’re experiencing.”

The WebAnywhere page has been designed for quick and easy navigation, Dr. Ladner says. All navigation is performed via keyboard. Large print at the top of the page highlights in real time the content being read further down the page so that users can more easily read along with WebAnywhere. “This also assists developers in tracking the order in which a user reads a page.”

Navigation by keyboard, he emphasizes, involves few keystrokes. “Suppose the user is filling out a form. She can go to the next form entry by hitting control-I to  select the form entry. If she is moving from heading to heading she hits control-H.  A user wanting to quickly review the results of a Google search can have the first part read to her and can then move to the next part, or by using the tab key she can move from URL to URL.”

An open source project with developers worldwide, WebAnywhere is used in more than 40 countries and is available in 27 languages.

Tactile Graphics Project: Opening a Traditional K-12 Bottleneck
The Tactile Graphics Project accelerates the translation of figures and diagrams from STEM (Science, Technology, Engineering, Math) textbooks for blind students.  The project focuses on the translation into embossed diagrams with Braille text of figures in high school and college textbooks that emphasize charts and graphs and thus are less illustration-rich than K-8 textbooks but easier and faster to translate. “Translation from figures to a tactile form has historically been a very slow process even under the best of circumstances,” Dr. Ladner notes. “The result has been a chronic shortage of properly translated textbooks at all levels. However, the Tactile Graphics Project has successfully broken that bottleneck for high school and college students who are blind,” he says.

K-8 textbooks are another story, however. “The richness of K-8 illustrations will continue to defy efforts to accelerate the production process.”  Therefore, he predicts, the traditional way of producing tactile graphics will not be phased out. “This tradition will not go away overnight and those who are skilled at it will not lose their jobs.”  However, he adds, “there will be a new job class. The members of that class will be able to produce lots of tactile graphics fast for high school and college work.”

The three problem areas the Tactile Graphics Project was designed to address, he explains, are text, math, and graphics.

The problem of text, he says, was solved by others prior to the project’s start-up.  “We could scan a book, find the text in it, do accurate optical character recognition (OCR) and produce an accurate Braille translation using Braille translation software like Duxbury or Braille2000. Consequently, we didn’t need to work on text at all.” Math and graphics, however, were another matter.

Math Translations: “The InftyReader Is Invaluable”
The InftyReader (http://www.inftyreader.org/) he explains, has been invaluable in providing math translations. The InftyReader is OCR software that recognizes scientific documents, including mathematical formulae, and outputs the recognition results into various file formats. This software, Dr. Ladner says, can translate the math to LaTex (http://en.wikipedia.org/wiki/LaTeX), a mark-up language for math. “The mark-up language can then be translated into beautiful-looking printed math” or can be converted by Duxbury (http://www.duxburysystems.com/) Braille translation software to Nemeth Code (http://www.washington.edu/doit/Stem/articles?42), a type of Braille used for math and science notations.

The InftyReader, he concludes, “is the most important innovation of its kind in the past 5-10 years. I highly recommend it for math Braille translations.”

Solving the graphics problem in translations to Braille, he says, still involves working with text.  “Almost all graphics have text embedded in them. Some of the text is at an angle or even vertical.  At UW we developed a process enabling us to find the text, remove the text, remember where it should go and put that text in an image. The image of the text can then be converted to Braille by the process we already know.”

Tactile Graphics Assistant and the Workflow Process: “Our Way Is Much Less Expensive”
The linchpin of the tactile graphics image translation process for graphics is the Tactile Graphics Assistant (TGA). The TGA, Dr. Ladner explains, is a software program created by his UW team to separate text from an image so that the text can later be replaced by Braille and reinserted onto the image. “In order to streamline the text selection process, the TGA employs machine learning to recognize text so that large groups -- possibly hundreds -- of images can be translated at a time,” he explains.   

“Using Photoshop, we take the original image, pre-process it, clean it up and make it more usable. Our TGA extracts three parts: a location file which remember where all the text resides, the pure graphic which has no text, and the image of the text.”

The location file is an XML file,” he continues. “The text as an image is processed first into text by OCR, then translated into Braille. This uses what we already know how to do. We then merge these files with the pure graphic into a new image. This new image will be much larger, the size of a Braille sheet of paper. Most of these steps require editing because OCR is somewhat inaccurate and the Braille may not be in the exactly right position and thus has to be moved.”  Adobe Illustrator (http://tryit.adobe.com/us/cs5/illustrator/tw1/?sdid=FNRPC&), he adds, can perform that task. “We’ve had much product support from Adobe.  They’ve been very interested in our work for the past five years.”

His team has translated five books using this process. “The first book, Computer Architecture: A Quantitative Approach, 3rd Edition, consisted of 230 figures and took a long time to translate, 25 minutes per figure, because, frankly, we didn’t yet know what we were doing. We were developing a workflow process as we went along.”

The next book, Advanced Mathematical Concepts, Precalculus with Applications, consisting of nearly 1,100 figures, required just 6.3 minutes per figure. “That’s human time, but we were, in some cases, actually translating about 300 figures at a time,” Dr. Ladner notes.

An Introduction to Astrophysics took somewhat longer – 10.2 minutes per figure for 467 figures -- because the figures were more complicated.  

All these embossable figures for these books, he emphasizes, “are on the web and can be downloaded free.”

Only three team members translated the books. “They weren’t yet skilled in the process or the books would have been produced even faster.”

When refined, the workflow process, he explains, included the following steps:

The most time-consuming aspect of the workflow, he says, “was the movement of the Braille once the content was in an illustrator file.  Almost every figure needed to be touched.”

In addition to Photoshop and Adobe Illustrator, the workflow process included Omnipage (http://www.nuance.com/imaging/products/omnipage.asp), OCR software and Duxbury (http://www.duxburysystems.com/) Braille translation software.  All are commercial products, Dr. Ladner emphasizes, “and they are not free.” The total software price tag, he says, is less than $2,000. “The cost is not much, though, for a production house,” he asserts, “especially when commercial production of a Braille textbook can cost tens of thousands of dollars. There are companies that provide that service. They can charge $10-$20,000 per college textbook. Our way may prove to be much less expensive.”

TGA: Advantages and Disadvantages
The advantages of TGA, Dr. Ladner says, “are much faster production, batch processing instead of one figure at a time and avoidance of tedious work.”  

A disadvantage, he says, is that the product “might be of lower quality than custom translation; quality would improve if figures were produced individually but that level of customization might take too long -- the moment when the material is needed may have passed. There’s a trade-off between timeliness and quality, he points out. Our work emphasizes timeliness.”

Another significant disadvantage, he notes, is that the necessary technology must be mastered. “Those who produce tactile graphics today one figure at a time will have to learn a lot of new technology.” The result, he claims, is that a change in work force might be necessary. “That will have to happen. We no longer perform many of the tasks we once performed.”  For example, he points out, “individuals were once paid to help place phone calls. We would dial an operator to make a call. Placing a call is now automated. The operator job no longer exists.”

Seven half-day TGA training sessions have been held, two at the National Braille Association (http://www.nationalbraille.org/), three at Accessing Higher Ground (http://www.colorado.edu/ATconference/about2009.html) and two at UW.  In some cases, Dr. Ladner says, “some trainees just weren’t proficient enough on Photoshop and Illustrator even with training. At UW some trained for multiple days. If what we’ve begun ever becomes a reality on a bigger scale significant training will be a must.”

A New Direction
Although Braille is critical for reading and writing literacy for blind children, access to information via speech is often desirable. Dr. Ladner and his students have developed a method to print proprietary codes on embossed figures so that the text can be spoken using a digital pen. An alternative is to use a public domain code, called  a QR (Quick Response) code (http://en.wikipedia.org/wiki/QR_Code) developed in Japan to allow its contents to be decoded at high speed.   The code can then be spoken by a cell phone application using its camera.  

“There’s a lot of science behind the TGA so not just anyone can write that code, which is sophisticated and employs machine learning and computational geometry. The average programmer could not do it. We are in the process of licensing the source code to a company that will hopefully use it their own produce to accelerate the production of tactile graphics.”

So far, he notes, he has addressed neither his team’s digital pen nor its cellphone alternative in peer-reviewed journals. “Instead of publishing we’ve focused on our work, although we’ll attempt to publish a paper on the digital pen and cell phone work.”

Dr. Ladner is not concerned about the lack of published articles on his Tactile Graphics effort. “After all,” he chortles, “I’ve been around way too long to worry about ‘publish or perish.’ My goal was to remove a bottleneck that has long stifled blind students’ academic progress and we have made some giant steps toward that goal.”

Consumer Apps-Supported Ed Tech for Deaf Students
Dr. Ladner and his team also utilized web  technology to address two curriculum accessibility issues important to deaf and hard-of-hearing students: better curriculum access via web-based multi-media conferencing (ClassInFocus) and the expansion and documentation of ASL-STEM vocabulary (ASL-STEM Forum).

He describes ClassInFocus as an online multimedia classroom platform for deaf and hard of hearing students to access remote interpreters and captioners and improve their visual access to live, in-person classes.

According to Dr. Ladner, the typical mainstream classroom with a deaf student might have  two interpreters, who rotate periodically. In his model classroom, a deaf/blind student might receive tactile interpreting. “If the student has some residual hearing the instructor wears an FM transponder, the student a receiver.”  The student hears the instructor directly through his hearing aids. The classroom could also include a standing interpreter as well as captioning on a small screen. “Some students prefer captioning; and many need notetakers because notetaking requires their visual attention,” he adds.

In a ClassInFocus classroom, he explains, the deaf student has a laptop with the instructor, the PowerPoint screen, the interpreter and captioning all in front of the room and close together on the same screen.” The objective, he adds, “is to reduce visual dispersion, which is a problem for many deaf students. We want to bring all sources of information to a single screen.”

“It’s Not Your Typical One-Size-Fits-All Conferencing System”
Visual dispersion, he explains, can be reduced by consolidating video, text and shared materials. Layout flexibility should also be enhanced.  In the ClassInFocus environment the student chooses how she prefers to receive instruction. “She can opt for a public display, an interpreter, a picture of shared material, chat information on the side and storage for notes. All this would be set up by the student.

“This is not your typical conferencing system where one size fits all,” Dr. Ladner points out. “Ours is more customizable. The solution is to consolidate video, text, shared material together on a single screen. Hopefully, students in the classroom can continue to pick up information for the class as well.”

Finding the best interpreter is sometimes a problem, he concedes. “Say this is an advanced chemistry class and the interpreter is a local person and has never taken a chemistry course. However, maybe there is an interpreter in another city with knowledge of the subject who can interpret remotely.”

The addition of a remote interpreter, he explains, makes the ClassInFocus classroom a multi-media teleconference consisting of interpreters at one end and maybe a real-time captioner at another remote location. “The participants can also see the public display and the instructor and whatever else they need to see because they will have their own display customized to their needs. To achieve this result there is a need for display mechanisms that are not one-size-fits-all because these individuals have differing roles in this teleconference. This is what makes the ClassInFocus classroom unique,” Dr. Ladner declares

Preserving Missed Content
Preservation of missed content is a crucial issue for deaf and hard-of-hearing students, but is an issue that was left unaddressed by ClassInFocus, Dr. Ladner admits. “These students miss so much in class; they only have one channel – the visual channel -- but they’d like to go back over the material. So recording a teleconference would be a great benefit. The technology is there but eats up a lot of disk space..”

Will Dr. Ladner’s team offer a solution for missed content preservation in the very near future? Dr. Ladner isn’t sure. Anna Cavender, the UW Ph.D. candidate who created ClassInFocus has just  graduated, he notes. “With Anna gone I’d like to address the issue in collaboration with the National Technical Institute for the Deaf (NTID) or Gallaudet. They have a significant vested interest in it. With these sources available I’m hoping to record the material in an efficient way.  Maybe the captionists would like to be able to have their part be editable or the student have the ability to highlight, for example. Maybe the slides could be written on. Preserving the content is most important to the student but the ability to edit is important too.  Gallaudet has been interested in this for a long time and has made progress in ways to achieve this result.”

Now that mass storage has become relatively inexpensive, “and we know how to do compression and know how to economize, this seems to be a problem awaiting a solution. Having all the information about the class in one place makes it possible” he declares.


ClassInFocus: How It Works; Technology Out of the Box
ClassInFocus was developed using Adobe’s Collaborative Framework  application programming interface (API) (http://en.wikipedia.org/wiki/Application_programming_interface), which provides for shared resources and individualized viewing. “One of our participants implemented three camera feeds, one on the PowerPoint, one on the instructor and one on himself. If the PowerPoint is loaded ahead of time it will show up as a separate PowerPoint window. There’s a chat window available on the page. Content can be moved and enlarged or made smaller,” Dr. Ladner explains. “It’s very customizable because of the various roles.”

According to Dr. Ladner, the Adobe Collaborative Framework, which is now called Adobe LiveCycle Collaboration Service (http://www.adobe.com/devnet/flashplatform/services/collaboration/).  “is a beautiful framework for building applications.” Because the framework is an API, he explains, “developers can use many available functions, but there is much work to be done to make it an application. “

The step from API to application requires the assistance of a professional software developer, he insists. “This is not something a teacher, for example, can do unaided, unless the teacher is very skilled.” The API framework uses the Flex programming language “and provides users with very cool functions that allow them to shift video and do peer-to-peer video, which helps reduce the time delay. It also makes available all the infrastructure features Adobe offers with its framework.”

Back in 2008 his team conducted a focus group discussion via Adobe Connect (http://tryit.adobe.com/us/connect/a/?sdid=DJZGI). “Adobe Connect is a service that roughly parallels ClassInFocus although it lacks some features unique to ClassInFocus.” Users, he says, can subscribe to Adobe Connect and set up their own server. Adobe Connect, he remarks, “can be employed out of the box for users without programming,” but it wouldn’t have all of the individualized features ClassInFocus offers.”

However, Adobe Connect as a visual dispersal alternative received somewhat mixed reviews from focus group participants, Dr. Ladner says. “One student said, ‘It’s a little distracting in combination with the video. When people are signing it’s easy to overlook what is going on and then we suddenly realize we’re behind the class.’” Such feedback in 2008, he states, was the motivation for creating ClassInFocus.

For deaf individuals, he emphasizes, all information passes through their eyes. Therefore, most deaf people are able to only focus on one item at a time. “Even if all relevant information is on a single screen, a deaf person can focus on one item on that screen at a time.”

Notification is an alternative Anna Cavender has studied, Dr. Ladner says. “If a student is in a classroom and the slide changes and the student is watching the interpreter, the student might want notification that the slide has changed. One of Anna’s major innovations was to try to build in notifications and evaluate them.”

Ms. Cavender, he explains, conducted a study that included six students and one faculty member. “The study setting was a pre-recorded lecture. Whenever a slide changed she had a choice of whether to wiggle the window, blink it or insert a colored border. I liked the wiggle. It was eye-catching. She measured the layout choices, content questions and preferences using eyetracking to ascertain what the students actually looked at.”

Study results revealed that students who liked notifications felt they helped during a lecture, were more likely to look at them and were more likely to benefit from them. Students who did not like notifications didn’t mind them, were less likely to look at them or were unaffected by them. “No strong negatives,” Dr. Ladner remarks.

Eyetracking revealed the following: one student, despite large captions and an interpreter in the windows on his screen looked mostly at the captions, almost never at the interpreter. “This student was happy to see the captions.” Another student moved the captions and interpreter windows to the opposite side of the screen. “You could see through the interpreter, actually, to the slides. This student watched the interpreter quite a bit.”

The study, he adds, “enabled us to pinpoint the different accessibility-related needs of these students.” Having individualized views seemed to be important to the students in the study, he notes.  He cautions, however, that he does not attach statistical significance to the study because the sample size is too small.

He also cautions that such studies, large or small, may be unnecessary. “These studies are not easy to conduct. We need the cooperation of interpreters, captionists, teachers and students. In any case, there is some doubt as to whether such studies tell us much that’s usable in helping us in our prime task:  building truly innovative technology. I have a good idea what will probably work and not work from smaller studies, which may make a big study extraneous.”

ClassInFocus: the Work Continues
Dr. Ladner has set follow-up efforts in motion to support the initial progress of ClassInFocus. He says he’s seeking better control of video by adding cut, zoom and transparency capabilities. “The video was handled with Flash, which doesn’t do the best job with sign language; incorporating techniques from the MobileASL project into the video would be an improvement (http://mobileasl.cs.washington..edu).That kind of incorporation was impossible using the current API. I hope Adobe will make video incorporation available because it makes sense.”

He says his team is making the captioning pod more compatible with the software that captionists already use.  Team member Katie Sullivan, who is deaf herself, has been a driving force in the development of the captioning pod, he says. “She came to us from Olin College as a research intern.  Katie’s designed much of the captioning pod. She’s a good hacker and she was motivated.” “I always try to have students with disabilities on my teams,” he notes.

Work is also underway on the development of a speech recognition pod that, if it works, could replace a captionist. “Imagine a professor wearing a Bluetooth headset that picks up the audio very well and would then produce real-time speech recognition and captions. It’s not there yet. We’re trying to make it work in a noisy classroom situation. Speech recognition works best in a quiet room with a very good microphone. When speech is run through a system like the Adobe framework the consequent compression may distort it.”  

He is also hoping to experiment with ClassInFocus in a classroom situation with multiple students. “We’ve used ClassInFocus in non-classroom settings because it worked. However, we held a teleconference with a colleague at Adobe at a moment’s notice – and it worked! We hope to make progress on recording for future playback.”
 
ASL-STEM Forum: Enabling ASL for STEM to Grow
The ASL-STEM Forum, Dr. Ladner says, “is one of my favorite projects,” which is not a surprising pronouncement for a son of deaf parents who has been tinkering with new technology for 45 years. “The truth is that even as deaf students move up in science and engineering, their sign language is lagging behind, which is not shocking considering the traditional dearth of deaf students in those disciplines.”     

He cites Ron Painter, a deaf grad student in chemistry at Stanford, as an example. “Ron worked with his interpreters to invent his own signs for some of the chemistry terms he was using. He’s a very flexible guy but he wanted to make sure he got the information from his classes. He did this for himself. The signing he created is undocumented. He was just one deaf guy taking graduate chemistry at one university and there’s no one else who’s deaf who’s taking chemistry except at another university and that person, wherever he or she is, is also making up signing on the fly with his or her interpreters.”

According to Dr. Ladner, “There had to be a way to build a forum where people can share signing information.”  The forum he envisioned would not include only grad students but all students K-12 and beyond. After all, he remarks, “there are different signs all over the country for concepts as universal as ‘algorithm.’”  

The ASL-STEM Forum (http://aslstem.cs.washington.edu/), he explains, was designed to add universal ASL terminology to the ASL scientific vocabulary and to connect deaf students who are dispersed nationwide. His solution: a community-driven, video-enabled social network – ASL-STEM Forum -- to which sign language researchers, deaf students, sign language interpreters and ASL users in STEM fields could contribute vocabulary information. The project has recently added a Facebook group which can be accessed at http://www.facebook.com/group.php?gid=136709286355047&ref=ts.

Existing ASL-STEM dictionaries are few, their sign vocabularies skimpy. For example, he says, the Educational Resource Center on Deafness TSD (http://www.tsdvideo.org) contains 500 signs and DeafSTEM at Shodor (http://shordor.org/deafstemterms) features 625 signs. The largest appears to be Science/Mathematics Sign Lexicon at the National Technical Institute for the Deaf (http://www.rit.edu/~w-msse/pages/lexicon/wordlist.php?letter=a) with 1,320 signs. A forum, however, he emphasizes, differs in purpose and form from a dictionary. A forum, he explains, “matches language development. A dictionary is a once-in-time snapshot. The earlier dictionary URLs will eventually fade because they won’t be as relevant, timely or fluid as a forum – and they’re not very large.”

Words usually enter community dictionaries, like the Urban Dictionary (http://www.urbandictionary.com/), Wikipedia (http://www.commonsensemedia.org/wikipedia-tips?gclid=COO6gpLMvqICFQG3sgodwVlF5Q) and Wiktionary (http://www.wiktionary.org/) before entering standard sources like the Oxford English Dictionary, he asserts.  
 
On the forum’s homepage, he says, most sign contributions fall under the heading of ‘Computer Science.’   “Anyone can join, just register, get an account and contribute.”  

More input will be garnered from Gallaudet professor Caroline Solomon who will cover the university’s biology, chemistry and physics courses. He anticipates that further leadership will be supplied by NTID and Rochester Institute of Technology, which have collaborated with Dr. Ladner on other projects involving consumer technology applications. To attract more forum contributions, he says, incentives will be in the form of competitions and the ranking of contributors by number and quality of signs contributed.  

MobileAccessibiity: A Bridge to the Future of AT
Dr. Ladner’s team is involved in a non-education project that emphasizes consumer technology applications as aids to individuals with disabilities. Called MobileAccessibility (http://mobileaccessibility.cs.washington.edu/), the project is a collaboration between UW and the University of Rochester aimed at providing useful mobile functionality to blind, low-vision and deaf-blind users. “We’re leveraging the sensors that modern cellphones already offer in order to keep our devices inexpensive and utilize remote web services to process requests.”  The team, he adds, is developing applications on mobile phones using Google’s Android platform, and on the iPhone. “We’re designing, implementing and evaluating our prototypes with blind, low-vision and deaf-blind participants via focus groups, interviews, lab studies and field studies.”

Importantly, he adds, “we use fully-automated and human-powered web services to balance the cost and capability of our services.”

The idea, he explains, “is to use the sensors on cellphones to help people with disabilities to solve accessibility problems.” For example, he adds, “GPS can assist people in determining their location. Google Maps (http://maps.google.com/) can be used by individuals of all ages to learn what is nearby. All that’s needed is the appropriate accessible software.”

A Camera for the Blind?
One of his MobileAccessibility student team members, Chandrika Jayant, is exploring ways for blind individuals to use a camera to solve problems.

“Say a blind person wants to take a portrait photo of his/her family, for example, and no one else is available to take the picture. What if he had software that enabled him to point the camera at a subject, whose voice acts as a homing device? The camera then may announce, ‘There is only one person in the picture.’ The photographer says to his subjects, ‘Why doesn’t everyone move a little closer.’  Now all family members are in the picture but they are in the right-hand corner. The photographer takes a voice reading and moves the camera a little more. Now they’re centered – and the blind photographer takes the photo.”

All the pieces to build that application exist, he declares.  “Software is available that can detect where faces are. This and much more is doable. It’s simply a programming issue.   

“I have an oven in my house that has a digital display. All I want to know is the oven’s current temperature. Can’t I just take a picture of it and it will tell me? This is a problem involving optical character recognition and image processing.”

Dr. Ladner cites the usefulness of GPS when combined with a microphone. “Say a deaf person is in a situation where announcements are being broadcast over a loudspeaker. The deaf person turns up her phone and it picks up the speech emanating from the loudspeaker and sends it to speech recognition – or it sends it to a human-powered service that will tell the deaf person what is being said.”  

When he refers to multi-purpose sensors that can be used to solve accessibility problems, he is not referring to special-purpose custom products. “It’s an iPhone. It’s a Google Evo phone. These are things that already exist that can be programmed for specific purposes.”

Ubiquitous devices like these, he emphasizes, properly programmed to perform accessibility-related tasks, offer a glimpse of a supportive and exciting new dimension in the future of assistive technology, a future, he hopes, the AT community will embrace.  Simple accessibility focused applications for modern multi-sensored smart phones have the promise of being an inexpensive alternative to high priced special purpose hardware with limited uses. Projects in this direction include Ideal Group’s Android Accessibility Project (http://accessibility-android.info/) and Project Possibility.

 

 

RESOURCES
 

ARTICLES

iPod and IPhone Touch Apps for (Special) Education
By Eric Sailers
ScribD (April 2010)
In an update of a list prepared in 2008 by the author, Samuel Sennott and David Niemeijer, Mr. Sailers lists applications that can be used with iPod and iPhone Touch systems and purchased through the iTunes Store. Each application is titled and described and displays the icon that will appear on the touch screen after purchase and downloading. The list is organized in the following sections:

Applications include flashcards, expressions, Spanish-to-English, sign language, auditory feedback, behavior charts, dictionaries and word prediction, calculator and tip/tax helper, music creation, song activities, art and photo and PACMan, Also offered are apps that make phones accessible via magnification or large print contacts, sound amplification and voice control, and audio dialing.
http://www.scribd.com/mobile/documents/24470331/download?commit=Download+Now&secret_password

 

The Fraser Project: iPods Help Asperger’s Kids Navigate Life
By Maura Lerner
Minneapolis Star-Tribune (July 25, 2009)
Teenagers on the autism spectrum often must depend on others for social and daily living cues. The Fraser Project at Minneapolis’s Fraser Child and Family Center hopes to decrease that dependence via iPod and iTunes applications. The project’s goal is to teach teenagers with ASD to use their iPods to keep lists, view videos and employ other aids that help them interact appropriately with others and manage daily activities in a more independent way.
http://www.startribune.com/lifestyle/health/51702302.html?page=1&c=y


There’s an App for That: North American Technology Assists Paris Metro Passengers with Special Needs
Newswise (June, 2010)
Developed by Ryerson University’s (Canada) Digital Media Zone (DMZ) and Appear Networks of Sweden, the Mobile Transit Companion uses context-aware self-adaptive computing to deliver live, customized data to passengers en route. The app is currently utilized by Paris Metro passengers, who can now download the free software and set up their user profile. Information is then automatically sent to their mobile phones according to their profile and contextual information. For example, they can receive notifications such as the location of station elevators and Metro transfer points; facilities and nearby businesses; updated schedules for connecting trains; and availability of services at each station. For blind and low-vision passengers, the Mobile Transit Companion employs a speech-enabled, gesture-driven user interface. For other user groups, including the elderly, deaf or hard of hearing individuals, or passengers with reduced mobility, the application offers an intuitive and easy-to-use visual interface. The Mobile Transit Companion’s positioning function is integrated with a web-based monitoring service so that public transit staff can easily locate and assist passengers with special needs. The only existing social tracking application for public transit passengers, this monitoring service can be shared with friends and family. Using the interface, the transit authority can message selected passengers, a helpful feature for deaf or hard of hearing passengers who can’t hear announcements made over public address loudspeakers.  Messages are communicated to each passenger according to the information on his or her profile. For example, the message will be interpreted from text to speech if a passenger has indicated that he or she has low vision.
http://www.newswise.com/articles/there-s-an-app-for-that-north-american-technology-assists-paris-metro-passengers-with-special-needs


Google’s Phone Apps for the Blind and Everyone Else
By Michael Helft
New York Times (April 2, 2009)
The popularity of touch phones has inspired the inexpensive creation of software applications, according to New York Times blogger Michael Helft. An application created for a specific audience, he writes, may also prove useful for unanticipated users. Fitting into this category are several apps created on Google’s Android software designed to make the touch-screen T-Mobile G1 accessible for the visually impaired. These apps, however, are also useful for individuals whose vision is unimpaired, such as drivers who do not want to take their eyes from the road, for runners, or for individuals whose vision outdoors is temporarily impaired by the sun’s glare. The text to speech feature makes this application useful for anyone, he writes, who chooses, or is unable, to look at the screen while dialing a phone or inputting information. This short article includes links to additional information, including videos demonstrating the applications.
http://bits.blogs.nytimes.com/2009/04/02/googles-phone-apps-for-the-blind-and-everyone-else/


State of the eNation: Social Networking Websites Accessibility Report
AbilityNet (2008)
AbilityNet tests websites and applications and then publishes The State of the eNation, a detailed description of its test findings. This edition of State of the eNation focuses on the social networking applications MySpace, Facebook, YouTube, Yahoo, and Bebo. Although all of the social networking applications have features that are accessible, reports the web team that conducted the study, sections of their respective websites have accessibility weaknesses making the sites, in some cases, impossible for individuals with vision impairments to access. This article rates each application with respect to its compliance with the provisions of Section 508 of the Electronic and Information Technology Act.
http://www.abilitynet.net/docs/enation/2008SocialNetworkingSites.doc


WEBSITES

EyeJot
A clientless, offline video chat system similar in some aspects to Skype, EyeJot is a video messaging system that functions like email but provides visual information to ensure accurate communication. Using a webcam, users send video messages through the computer. Similar to instant messaging, this system allows for rapid communication but with the addition of a visual of the sender as he or she communicates the message. Examples of the system are provided on the website. Links to providers of webcams are also provided. There is no software to purchase or download. Consumers create their account at no cost and install a webcam on their computer. This is another communication option for people who use visual language, such as ASL, or for users who need to see facial expressions in order to accurately interpret a communicated message. http://www.eyejot.com/about.html


It All Starts with Communication: Providing Accessible Cellphones for People with Special Needs
ETO Engineering
The ETO Engineering website provides information on accessible off-the-shelf cellphones and related services. The site describes cellphone features and products for blind/low vision individuals and for those with hearing, speech, mobility and cognitive impairments. Although the focus of this website is accessible cell phones – the company manufactures cellphones for residents of the Carolinas – it provides information on other accessible products, services and resources, including switches, mounts, and computer access solutions. The site also includes information about teaching math and science to children with special needs in North Carolina and adapting toys for children who live near the company’s facilities. A resource section provides links to information about accessible cell phone services. http://www.etoengineering.com/


SOFTWARE

Equation Editor
Microsoft
Equation Editor is a Microsoft Office math component providing users with the ability to enter complex math equations into a Word document. This component is useful for high school or college students in advanced math/calculus classes. It is also helpful in workplace situations in which math functions are necessary. However, the tool provides only minimal symbols and thus cannot adequately support engineering or other high level technical tasks.
http://www.microsoft.com/education/insertequation.mspx


iTracker
EyeTwig
iTracker is an open source software program that tracks eye movement in real time through a web camera and translates that eye movement into smooth mouse movements. No calibration or equipment in addition to the webcam is required. iTracker’s open source status enables developers to integrate it into their software design applications. Thanks to iTracker, users are able to control the mouse with discrete head movements and to “dwell for clicking” with a built-in timer. iTracker responds to voice activation and offers global shortcut keys.
http://www.eyetwig.com/


BLOGS

IPhone Footprint: iPhone Sign Language App for Visual Communication
iPhone Footprint blogs discuss applications available for the iPhone. In this blog, the writer investigates a sign language app consisting of 800 animated sign images that can be purchased for $7.99 and used with the iPhone. These animated images are described as an ASL phrase book rather than a dictionary of terms. Signs can be placed in alphabetic order. Also available is a free “Lite” version. A link to the Sign Smith ASL Lite site is included. Both web pages show an avatar performing a specific sign, a useful feature for users wishing to learn basic signs.  It is a potentially useful resource for both those who are deaf and those who want to communicate with a deaf person.
http://www.iphonefootprint.com/2008/10/iphone-sign-language-app-for-visual-communication/

 

 

KNOWLEDGE NETWORK MEMBERS

 

Coalition of Organizations for Accessible Technology (COAT)
Formed in 2007 to help federal legislation keep pace with the impact of accelerating technology development on individuals with disabilities, COAT consists of more than 230 national, regional, state and local organizations.  COAT aims to ensure that legislative and regulatory safeguards help provide access to evolving high-speed broadband, wireless and other web-based technologies. To that end, the organization addresses the following issues:

For more information, see http://www.coataccess.org/node/51

 

Accessibility Partners, LLC
Accessibility Partners helps organizations improve accessibility aspects of “next generation” technology. Staffed by technologists, subject matter experts and consultants, the company provides advisory expertise in the following areas:

For more information, contact:
Accessibility Partners
Phone: (301) 717-7717   
http://www.accessibilitypartners.com/about.html


Audio Description Associates
This company provides access to the visual elements of television programs, films, museum exhibitions, theater shows and events. Such visual elements include action, costumes, settings, gestures, facial expressions and other visually engaging images. While audio description makes the images in children’s books more accessible to children with low vision or who are blind, the words a describer uses can aid children in the development of more sophisticated language skills.  For further information, contact:
Audio Description Associates, LLP
6502 Westmoreland Avenue
Takoma Park, MD 20912
Phone: (301) 920-0218 (24 hours)
Fax: (208) 445-0079
Email: jsnyder@audiodescribe.com
http://www.audiodescribe.com/


Center for Advanced Communications Policy (CACP)
Housed at Georgia Tech, CACP is an impartial research and policy development organization focused on advanced communications. CACP monitors and assesses legislative and regulatory issues and helps create a clearer understanding of the evolving technology landscape. Areas of expertise pertaining to individuals with disabilities include:

For additional information, contact:
Center for Advanced Communications Policy
Georgia Technological Institute
Ivan Allen College
500 10th Street NW
Atlanta, GA 30332
Phone: 404.385.4614 Fax: 404.385.0269
Contact: Helena Mitchell, Executive Director
http://www.cacp.gatech.edu/
.

CSD Interpreting Online (CSDIO)
CSDIO enables deaf and hearing people in the same room to communicate through an interpreter at a distant location, replacing traditional community-based interpreting services when they are unavailable or impractical. All CSDIO interpreters are certified and available all day, 365 days a year. The company’s services are ADA-compliant.  CSDIO is compatible with most available video equipment.
CSD Interpreting Online
102 North Krohn Place
Sioux Falls, SD 57103
Phone: (605) 367-5760 (Voice); (605) 367-5761 (TTY); (800) 642-6410 (Toll free, Voice); (866) 273-3323 (Toll free, TTY)
Videophone: (605) 496-0738
Email: inquiry@c-s-d.org
http://www.csdio.com/


Inclusive Technologies
This company provides consulting services on accessible consumer information and communication technologies to clients that include public agencies, researchers, purchasers, consumer advocates and policy-makers.  Inclusive Technologies specializes in consumer technology that can be adapted to meet the needs of individuals with disabilities mainly by emphasizing aspects of Universal Design.  For more information, see:
http://inclusive.com/about/

 

 

Project Officer: Jo Ann McCann
Project Director:  Jacqueline Hess
Newslettter Editor:  Thomas H. Allen
Design and Distribution:  Ana-Maria Gutierrez