People with visual impairments often struggle to create content that relies heavily on visual elements, particularly when conveying spatial and structural information. Existing accessible drawing tools, which construct images line by line, are suitable for simple tasks like math but not for more expressive artwork. On the other hand, emerging generative AI-based text-to-image tools can produce expressive illustrations from descriptions in natural language, but they lack precise control over image composition and properties. To address this gap, our work integrates generative AI with a constructive approach that provides users with enhanced control and editing capabilities. Our system, AltCanvas, features a tile-based interface enabling users to construct visual scenes incrementally, with each tile representing an object within the scene. Users can add, edit, move, and arrange objects while receiving speech and audio feedback. Once completed, the scene can be rendered as a color illustration or as a vector for tactile graphic generation. Involving 14 blind or low-vision users in design and evaluation, we found that participants effectively used the AltCanvas workflow to create illustrations.
Our system, AltCanvas, features a tile-based interface enabling users to construct visual scenes incrementally, with each tile representing an object within the scene. Users can add, edit, move, and arrange objects while receiving speech and audio feedback.
Users can hear descriptions about the current state of the canvas through keyboard commands, as well as chat with the system about the image.
Size and location edit operations are supported through keyboard actions and sonification, providing a real-time editing experience. Additionally, tile-based interactions and radar scan functions help users understand spatial and relative orientation.
At the end of the editing interaction, users can render the image with various background styles for different users and create tactile graphics.
@misc{lee2024altcanvastilebasedimageeditor,
title={AltCanvas: A Tile-Based Image Editor with Generative AI for Blind or Visually Impaired People},
author={Seonghee Lee and Maho Kohga and Steve Landau and Sile O'Modhrain and Hari Subramonyam},
year={2024},
eprint={2408.10240},
archivePrefix={arXiv},
primaryClass={cs.HC},
url={https://arxiv.org/abs/2408.10240},
}