An intuitive accessible web automation user interface

In recent years, the Web has become an ever more sophisticated and irreplaceable tool in our daily lives. While the visual Web has advanced at a rapid pace, assistive technology has not been able to keep up, increasingly putting visually impaired users at a disadvantage. Web automation has the potential to bridge the accessibility divide between the ways blind and sighted people access the Web; specifically, it can enable blind people to accomplish web browsing tasks that were previously slow, hard, or even impossible to achieve. In this paper, we propose and evaluate an intuitive and accessible web automation interface. We validate the design in a Wizard-of-Oz user study with visually-impaired subjects and show that the proposed approach has the potential to significantly increase accessibility and usability of web pages, reduce interaction time, and increase user satisfaction. Our findings demonstrate the feasibility of and emphasize the pressing need for truly accessible web automation technologies.