ASPIRE listens to a talk as it happens, describes each new slide with AI, and speaks that description into the presenter's own pauses. Blind and low-vision listeners hear the slides within seconds of live, on their own phone.
Median listener delay is 4 seconds, measured with real listeners connected. Not a transcript after the talk.
Slide changes are detected automatically. Build-up animations and repeated slides are recognised, so nothing is read twice.
Descriptions are placed in the presenter's natural pauses. The held speech is returned faster than real time with its pitch intact.
Listeners open a link on any phone or laptop. The page works with screen readers and starts with a single keystroke.
Works with any slide software and any meeting tool, because it reads the screen and system audio directly.
Three published studies with blind and low-vision participants, plus a live deployment at an international conference.
In lectures, conferences and webinars, the chart, the diagram and the bullet list are where the meaning lives. Presenters rarely describe them. Human describers are scarce and expensive, and recorded descriptions arrive days later. ASPIRE closes that gap while the talk is still happening.
See the studiesASPIRE runs on the presenter's Mac. It captures the screen and the system audio, decides when a new slide has appeared, writes a one-sentence description, turns it into speech, and mixes it into the live audio that listeners receive.
Screen video and system audio, straight from macOS. No virtual audio drivers, no plug-ins.
ScreenCaptureKit · ffmpegPerceptual hashing spots a visual change; a short stability window lets build-ups finish; text and description matching stop repeats.
pHash · OCR-text dedupOn-device OCR extracts the text; a vision-language model writes a short spoken-form description of what matters on the slide.
Apple Vision OCR · Claude Haiku 4.5A local neural voice renders the description in under a second, with numbers normalised so they are read correctly.
Kokoro TTS, on deviceA 50 fps mixer places the description in the next natural pause and streams the result to listeners' phones.
WebRTC · HLS fallbackAnimation runs at 2× speed. Timing comes from the same audio sample you can play below.
A presenter moves to a new slide and keeps talking. Without ASPIRE a blind listener hears only the voice. With ASPIRE the slide is described in the first pause, and the talk catches back up to live within half a minute.
So let's move on to the next slide. This is one of my favourite charts in the whole talk. What you're seeing here is the cost of solar electricity over the last ten years, and the trend is remarkable. In 2010, utility-scale solar cost about 38 cents per kilowatt-hour. By 2020, it was down to roughly 5 cents. That is an 87 percent drop in a single decade, faster than almost anyone predicted. And this is the point I really want you to take away: cost, not technology, is what changed the energy conversation. Once solar became the cheapest option on the table, the debate stopped being about whether, and started being about how fast. Keep that in mind, because the next slide looks at wind, and the story there is strikingly similar.
press play
So let's move on to the next slide. This is one of my favourite charts in the whole talk.
ASPIRENew slide. Line chart titled "Cost of solar electricity", falling from 38 cents to 5 cents per kilowatt-hour, 2010 to 2020.
What you're seeing here is the cost of solar electricity over the last ten years, and the trend is remarkable. In 2010, utility-scale solar cost about 38 cents per kilowatt-hour. By 2020, it was down to roughly 5 cents. That is an 87 percent drop in a single decade, faster than almost anyone predicted. And this is the point I really want you to take away: cost, not technology, is what changed the energy conversation. Once solar became the cheapest option on the table, the debate stopped being about whether, and started being about how fast. Keep that in mind, because the next slide looks at wind, and the story there is strikingly similar.
About this sample: the insertion point and the catch-up were computed with the production mixer's algorithm and July 2026 settings (0.3 s pause trigger, 2.5 s maximum hold, 1.0–1.5× pitch-preserving catch-up). The voices are synthetic stand-ins so that no real lecture is redistributed; in live use the description is written by Claude Haiku from the actual slide and spoken by the on-device Kokoro voice.
Two kinds of evidence: controlled studies with blind and low-vision participants, and instrumented engineering sessions of the live system with real listeners connected. Every number below traces to a published paper or to a logged session.
Share of comprehension questions answered correctly
Sources: DIS 2026 full paper (DOI 10.1145/3800645.3812965); ASSETS 2026 demo paper (DOI 10.1145/3797867.3841220).
Seconds, before and after the July 2026 engineering work
Source: ASPIRE system architecture and latency performance report, July 2026. Six instrumented sessions, about 3.5 hours of talk audio, ~300 logged events per second.
Up from roughly 0%. Two thirds of the session is within 5 seconds.
From 184 to 39 per session, through full-frame analysis and three layers of de-duplication. Lower cost per hour.
The vision call takes 1.8 s. The delay came from the audio pipeline, which is now fixed. A local model would have been slower.
All by Azizul Haque and Dr. Jonggi Hong at the Interactive Computing Lab, Stevens Institute of Technology. A fourth paper, the first live field study of the system, is under review.
Four blind and low-vision participants watched talks whose slide descriptions were inserted by hand. The study established what listeners want described, how long descriptions should be, and where in the talk they should land.
Result: the design requirements behind everything that followed.
Lab publications →The first automated pipeline, evaluated on recorded presentations with twelve blind and low-vision participants.
Result: comprehension rose from 50.0% to 88.9%.
Read the paper (ACM DL) →The real-time system, running live on a laptop and streaming to listeners' phones, piloted with four blind and low-vision participants. Presented in Vila Nova de Gaia, Portugal, October 2026.
Result: recall rose from 16.7% to 91.7%.
Read the paper (ACM DL) →Under review: a live field study in which twelve blind and low-vision participants each attended real conference sessions remotely, once with ASPIRE and once without. Results will be published here when the paper is out.
A Wizard-of-Oz study with four blind and low-vision participants: talks with human-edited slide descriptions. Listeners wanted short, factual descriptions placed in the pauses, not over the speaker. Presented as a poster at ASSETS 2025.
Slide detection, AI description and speech synthesis run end to end on recorded talks. Twelve participants, comprehension from 50.0% to 88.9%. Published at DIS 2026 with a Best Paper Honorable Mention.
First real-time version: native macOS capture, Claude Haiku vision, an on-device voice, and WebRTC streaming to phones. The reference implementation is released as open source.
An instrumented measurement campaign finds the real bottleneck in the audio pipeline, not the AI model. Pitch-preserving catch-up, WebRTC-by-default and deadline pacing bring the median delay to 4 s with zero dropouts. The system is then deployed in a live field study at an international HCI conference with twelve blind and low-vision participants.
The real-time system is presented to the accessibility research community in Portugal. Pilot recall: 16.7% to 91.7%.
We are forming the company, lining up university and conference pilots, and raising a pre-seed round to build the one-click product and the integrations customers ask for.
Lectures, conferences and all-hands meetings are streamed by default now, and the obligation to make them accessible is shifting from goodwill to regulation. Live captions solved the audio side. Nobody has solved the slides.
people worldwide live with a near or distance vision impairment.
World Health OrganizationUS public universities and governments must meet WCAG 2.1 AA for web content under the ADA Title II rule, including live and recorded media; smaller entities follow in April 2027.
US Department of JusticeThe European Accessibility Act began to apply, extending accessibility duties to digital products and services across the EU.
European CommissionDisability-services offices need a scalable way to make live lectures and recorded courses accessible. ASPIRE runs on the instructor's laptop and needs no change to how they teach.
Organisers can offer every remote attendee a described stream with a link, instead of hiring describers for a handful of sessions.
The description engine fits naturally alongside live captions in meeting and webinar platforms, as an integration or a licensed component.
First author of every ASPIRE paper. Designed and engineered the live system, from macOS capture and slide detection to the real-time mixer and the latency work that brought it to 4 seconds. M.Eng. in Applied Artificial Intelligence and research assistant at the Interactive Computing Lab, Stevens Institute of Technology.
Assistant Professor of Computer Science at Stevens Institute of Technology and Director of the Interactive Computing Lab, which designs and evaluates accessible interactions with emerging technology. Human-computer interaction and accessibility researcher; senior author of the ASPIRE research line.
ASPIRE grew out of the Interactive Computing Lab at Stevens Institute of Technology in Hoboken, New Jersey. The studies were approved by the university's Institutional Review Board and conducted with blind and low-vision participants throughout.
We are raising a pre-seed round to turn a peer-reviewed, field-tested research system into a product that universities, event organisers and video platforms can switch on.
We reply within two business days and are happy to run a live demo over a video call.