The FingerReader is a wearable device that assists in reading printed text. It is a tool both for visually impaired people that require help with accessing printed text, as well as an aid for language translation. Wearers scan a text line with their finger and receive an audio feedback of the words and a haptic feedback of the layout: start and end of line, new line, and other cues. The FingerReader algorithm knows to detect and give feedback when the user veers away from the baseline of the text, and helps them maintain a straight scanning motion within the line.

The device relies heavily on the camera feed and its associated algorithms. When the user puts their finger on the page at the start of a new line, these algorithms make a bunch of guesses about where the text's baseline sits, based on density -- text ascenders and descenders are less dense than the median.This then allows the FingerReader to constrain the guesses it needs to make for each new frame of video as the user moves their finger. It can then much more easily track each individual word, cropping it out of the image and sending it to software that parses the characters and translates them into audio in real-time.

For the prototype, the FingerReader was tethered to a laptop, upon which these calculations were made, but the team is now developing a version of the open-source software that will run on Android phones.The team also tested two methods of guiding the user's finger: haptic motors on the top and bottom of the finger that vibrated when the user's finger started to move away from the line, alerting them to move their finger back up or down the page; and a musical tone that sounds when the finger strays. Users testing the system evinced no preference for one or the other, so the team elected to go with the audio sensor, since the haptic motors add weight and bulk to the device.