Tomat Navigator
Screen reader software is an essential tool for individuals with visual impairments to access and navigate webpages. However, it can sometimes fall short in conveying the visual layout and structure of a webpage. As a result, users may miss crucial information or have to rely heavily on their memory to complete navigation tasks.
Objective:
To research and develop a complementary technology to the screen reader, which offers a continuous, interactive, and non-visual representation of a webpage's structure through audio-tactile feedback. This technology will leverage artificial intelligence to provide users with contextual assistance in navigating webpages efficiently.
Method:
In my research, I conducted a comprehensive review of academic literature to gather design recommendations for assistive technologies aimed at improving internet navigation for individuals with visual impairments. Based on these findings, I designed and prototyped various software and hardware solutions to complement existing screen reader navigation strategies.
To fund and recruit participants for my study, I partnered with academic, government, and research organizations. In my role as project lead, I conducted codesign sessions, focus groups, and usability evaluations with visually impaired participants and assistive technology experts. This collaborative approach enabled me to obtain valuable feedback and insights from the end-users and experts, which helped refine and optimize the technology for better usability and accessibility.
Tools:
Raspberry Pico
Python
JavaScript
FreeCAD
3D printing
AI
Results:
The Touch Matrix Assistive Technology Navigator (TOMAT Navigator) is an affordable, open-source assistive technology that leverages AI and tactile technologies to facilitate natural and intuitive internet navigation for individuals with visual impairments.
A Google Chrome extension is used to extract relevant HTML elements from a webpage and enable filtering, favoriting, navigation, and API calls to artificial intelligence functions (such as NLP and image description) in response to user input gestures. This is facilitated by a hardware interface, which uses a matrix of vibrators controlled by a Raspberry Pi microcontroller to represent the webpage elements and transmit user gestures to the Chrome extension.
Currently, we are in the process of clearing the legal paperwork to release the project's results as an open-source under the CERN-OHL-P license. Additionally, we are working on obtaining OSHWA certification for the technology and the publication of three research papers in peer-reviewed journals. These publications will showcase the positive impact of the TOMAT Navigator on improving the accessibility and usability of the internet for individuals with visual impairments.
