AccessPatch: breaking down web barriers for people with disabilities

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend.

What I Built

A visually impaired friend had told me about accessibility issues he encounters online. A website can look perfectly usable while leaving someone who relies on a screen reader or keyboard unable to understand or reach its controls.

When I saw this challenge, I wanted to explore whether local AI could help reduce those barriers directly in the browser. That became AccessPatch.

In my test recording, Windows Narrator reaches a search control and announces: “Search the catalog region. Button.” A sighted user can recognize the magnifying glass. The screen reader announces a button without a name. The next barrier is quieter: a clickable menu that keyboard navigation skips entirely.

AccessPatch is a Chrome extension that repairs supported controls in the user's copy of a page. It uses Gemma 3 4B, running locally through Ollama, to propose meaningful accessible names. A separate set of rules adds keyboard focus and activation to supported mouse-only controls.

There are three modes:

• Automatic: visits are analyzed and supported repairs applied quietly after the model responds. There is no page panel or focus change.
• Debug: before/after cards explain each issue, proposed repair, and evidence. A numbered highlight connects the card to its control. You choose which repairs to apply.
• Paused: automatic visit scans stop; manual analysis remains available.

The website's server stays untouched. Changes are local to the browser and can be undone.

Demo

The 2:26 demo includes genuine Windows Narrator audio from my recorded test session.

The most useful comparisons start around 1:33:

The video also shows a search-field repair on Deque's deliberately inaccessible training site. Both that page and my local lab are practice fixtures. The recording establishes behavior in these examples; broader user testing is still ahead.

Code

Download AccessPatch v0.1.0 · Installation guide

The release includes an unpacked extension ZIP and a SHA-256 checksum. Extract the ZIP and select its directory through Chrome's Load unpacked option.

The repository contains the unpacked Manifest V3 extension, the before/after lab, automated tests, and a model evaluation script. The extension has no production package dependencies or build step.

For the local setup:

ollama pull gemma3:4b
ollama run gemma3:4b

Then load the extension directory through Chrome's Load unpacked option and configure:

API base URL: http://localhost:11434/v1
Model:        gemma3:4b
API key:      empty for a normal local Ollama setup

Website access enables visit scanning. Ollama may also need the extension's specific origin in its OLLAMA_ORIGINS allowlist. Same-computer use works with the server bound to localhost.

How I Built It

Give the model a bounded job

The content script finds visible candidate controls and extracts nearby text, labels, placeholders, and icon metadata. Entered form values, editable text, script contents, and URL query strings are excluded from the scan payload. Nearby page prose can still contain sensitive information.

Gemma receives one unnamed control per request and returns a proposed name, a quoted piece of evidence, and an explanation in schema-constrained JSON. The extension validates the target, output bounds, and evidence before using the proposal.

For example, a supported proposal becomes:

<button aria-label="Search catalog">
  <!-- Existing icon and application behavior remain in place. -->
</button>

The inference input is text and metadata. This version does not send screenshots to a vision model.

Keyboard behavior belongs in explicit rules

The menu fixture is a clickable div. The keyboard repair adds a button role, a tab stop, and listeners that invoke its existing click action. Enter activates on keydown; Space activates on keyup. Repeated keydown events do not repeatedly activate the control.

That behavior is implemented in extension code. Gemma supplies the proposed accessible name. It never supplies executable repair scripts.

A parseable answer can still be wrong

I started with Gemma 1B. The first prompt produced responses that echoed the input instead of returning repairs. Constraining the JSON shape solved the format problem, but the smaller model still guessed labels and followed instructions embedded in page context.

Separating evidence fields, adding examples of abstention, and sending one target per request helped. I then compared the final prompt across both models:

This is a small synthetic smoke test. It guided the choice of 4B; real-world accuracy and resistance to malicious page text need wider evaluation.

Keep repairs reversible

Proposals are bound to actual elements and a snapshot of their context. A changed control or navigation can invalidate them. Undo restores only attributes that still match AccessPatch's applied values, preserving later changes made by the website.

The project has 22 passing automated tests, covering output validation, private form-value exclusion, keyboard activation and cleanup, stale targets, navigation, automatic/debug modes, and stopping auto-apply when the user pauses during inference.

Current limits

The prototype focuses on accessible names and a conservative class of custom-button keyboard repairs. A scan covers up to 24 candidate controls. It works in the top-level document and does not currently generate image descriptions or repair canvas interfaces, cross-origin frames, shadow-root content, or complete custom-widget interaction patterns.

Arbitrary same-route rerenders can require another scan. Model proposals can still be wrong, including in Automatic mode. Debug provides a way to inspect them and undo changes.

Why Does Open Innovation Matter?

An accessibility repair model sees text near the controls someone is using. On account pages or forms, that context can be personal even when typed values are excluded. Running Gemma locally keeps the analyzed context on the user's machine in this configuration.

Open weights also let me test model sizes against the same tasks, choose the one that behaves better, and keep the model replaceable. The extension accepts configurable OpenAI-compatible endpoints, so its repair pipeline can use another compatible model or runtime.

Local inference can continue without access to a remote inference API once the model is installed. The websites themselves still need whatever connectivity they normally require. There is no per-request inference-service bill in the local setup; hardware and electricity remain costs.

The practical benefit is control over where inference happens, which model performs it, and how its proposals become page changes.

What's Next

My goal is to give people who encounter inaccessible interfaces more control over their browsing experience. AccessPatch starts with two concrete barriers: unnamed controls and mouse-only buttons. The next step is testing it with people who rely on assistive technology and learning which repairs genuinely help them complete everyday tasks.

My Agent Session

I used OpenCode during implementation and testing. The development process included inspecting Chrome's accessibility tree, exercising real keyboard events, and checking model responses against the synthetic fixtures.

Prize Categories

Best Use of Gemma

Gemma 3 4B performs the project's accessible-name inference locally through Ollama. Its evidence-grounded proposals become bounded, reversible DOM repairs. The model comparison and recorded demo show its role in the finished extension.

Credits

Thanks to the Gemma and Ollama teams, and to Deque for the accessibility practice site shown in the demo.

AI tools assisted with the implementation and preparation of this write-up.