Importance

Data science has become essential in modern society, with growing career opportunities and widespread adoption in educational curricula. However, blind and low-vision (BLV) students are significantly underserved in this field, often lacking the tools necessary for meaningful engagement with data. In partnership with Perkins Access Consulting at Perkins School for the Blind, and working closely with accessibility consultant Sina Bahram from Prime Access Consulting and AI consultant Vikram Kumaran, we are addressing the critical need for accessible data science tools in K-12 education.

Leveraging a cutting-edge large language model (LLM) from generative AI technologies, we will create a multimodal data exploration environment. By enabling BLV students to interact with data through voice commands, sonification, and AI-generated text descriptions, we aim to transform the educational experience and broaden participation in STEM.

We are developing and researching an AI-powered agent called DAVAI (Data Analysis through Voice and Artificial Intelligence), which will be embedded as a plugin in the Common Online Data Analysis Platform (CODAP). DAVAI will provide the interface between the user, the generative AI model, and CODAP. It will interpret BLV users’ verbal and typed commands to perform data transformations, generate data representations, facilitate non-sequential navigation and exploration of data representations, and provide text and sonified descriptions of data representations.

BLV students are the primary audience for DAVAI, but we envision that the plugin will be useful for all learners, including students with physical disabilities that affect interaction through mouse and keyboard and learners with cognitive disabilities who prefer using voice to control software and to hear descriptions of graphs. DAVAI will also facilitate data exploration by sighted users, making it easier to focus on exploration rather than the steps needed to produce data representations.

Research

Two main research questions will guide project work:

• In what ways can generative AI/LLM-based technologies or approaches be extended, designed, and leveraged to provide interfaces for BLV users’ interaction with and exploration of data?
• What effect does the availability of interactive and generative technologies have on BLV users’ ability to engage with and make meaning of datasets?