Bespoke Visual Assistance: What and How do Blind and Low-Vision People Create with Agentic Programming?

Ellie Seehorn, Gene S-H Kim, Aziz Zeidieh, Ather Jammoa, Kun Lee, Aditi Shah, Jaylin Herskovitz, Anhong Guo, Venkatesh Potluri

AI-powered assistive technologies have long supported blind and low vision (BLV) people in everyday tasks, but they are general-purpose and often fall short of meeting complex, individualized, in-situ accessibility needs. Though agentic programming tools, like GitHub Copilot, have the potential to bridge this gap by lowering the technical barriers to building personal AT using natural language, the practical applicability of this creation paradigm has been unknown. We address this knowledge gap through a two-phase longitudinal co-design study with five tech-savvy BLV users using ProgramAT, an agentic programming tool that supports the creation, iteration, and testing of camera-based AT. Overall, co-designers created over 37 tools, with some addressing needs unmet by any existing commercial AT such as identifying Uber rides or interpreting hand gestures. Qualitative feedback from our co-designers and analysis of development logs surface BLV strategies for tool creation, along with key challenges including model capability limits, specification conflicts, and barriers to successful creation. We discuss recommendations to provide appropriate conversational scaffolding, community tool sharing capabilities, and support for specialized models and personal datasets for future agentic programming environments to empower BLV users to create bespoke visual assistance for themselves.