I am a software developer. A couple of years ago I started learning mobile development to build small personal apps for the phone. A hobby, really.
My wife is a physician who spends her day with an ultrasound probe. A couple of evenings a week she has late appointments, and on those evenings I noticed a pattern: she came home very late, and I spent that hour on the couch with the cat, waiting.
The reason is that after the last patient she sits down and types reports for a long time. If you have never seen an echocardiography report, let me describe it: a table of about fifty numbers. Chamber dimensions, wall thickness, valves, flow velocities, ejection fraction. Each number has its unit and its reference range, and a few are calculated from the others. During the exam the physician measures all of it on the machine and keeps it in her head or on a scrap of paper. After the exam she types it all up and prints it.
I looked at the scrap of paper and at the cat and thought: I am a programmer.
My first idea was the wrong one
There are dozens of speech-to-text services. Let her dictate, paste the text into the report, done.
We tried. We got a paragraph like "LVIDd forty seven LVIDs thirty two IVS nine PW nine LA thirty seven fifty two twenty eight". My wife looked at the paragraph and said she would rather type. Because now she had to move all of it into the table again. The time was not saved, it was moved.
That was when it clicked: this is not a speech problem, it is a report problem. She does not dictate text, she dictates measurements. So the software has to know which measurements exist in an echo, what units they come in and how they relate. "LA 37, 52, 28" is not three random numbers, it is the anteroposterior dimension, the length and the width of the left atrium. "Mild regurgitation" belongs to the valve she named last. Nobody dictates ejection fraction, you calculate it from LVIDd and LVIDs with Teichholz.
What it looks like now
She scans with the probe, the phone sits in her coat pocket, she says the numbers as she goes, the way she used to say them to an assistant, except this assistant does not get tired. A minute after the last sentence the report is waiting in her workspace: a table by sections, every number in its own field with its unit and reference range next to it, formulas done, a Word and a PDF document. She reads it through, corrects what needs correcting and prints.
Her favourite button appeared late. Next to every measurement there is a small "play". Tap it and the recording starts exactly one second before she said that number. Doubt about one value used to mean scrubbing through four minutes of dictation. Now it is one tap. Under the hood the software keeps the timestamp of every recognised word and matches them to the fields.
The hard part was not speech
It was the vocabulary.
Take "tricuspid aortic valve". The first version flagged it as an error: tricuspid is the valve on the right side, aortic is on the left, pick one. My wife patiently explained that a normal aortic valve has three cusps, so "tricuspid" there describes its structure and a bicuspid aortic valve is the finding. The software had to learn that the same word means different things in different places of the report.
Or pulmonary artery pressure. She reads it off the machine and dictates "twenty five". The formula, from other measurements, gives thirty. Whom to trust? We argued for a while. The answer we settled on: the software is on the physician's side. A dictated value always wins, formulas only fill what was not dictated, and if you really want the calculated one there is a separate button for that, so it is a deliberate act and not a silent override.
Or numbers themselves. People say "point nine five", "two and a half", "one point six". The machine shows 0.95. All of those have to be understood and not confused with two separate measurements.
Or silence. A real dictation is not a radio host. It is "so... let me see... mitral... leaflets thin... regurgitation... mild", with long pauses while the probe moves. We cut the pauses out automatically, recordings get about a quarter shorter on average, and nobody pays for silence.
Some numbers, because I am still a programmer
I compared several speech recognition models on 17 real dictations, 106 minutes in total. Not by "words correct" but by how many measurements ended up in the right field of the report. The spread between models was 48 to 82 percent. The choice of model mattered a lot. But the model also had to be taught the specific terms: without the vocabulary of the exam, even the best one keeps guessing.
About the hour
The hour is back. An echo report now takes three or four minutes of talking during the exam and a minute of checking. It used to take ten to fifteen minutes of typing afterwards.
The program grew into a service. It is called SonoForm and works in eight languages. Twenty one exam types so far: heart, abdomen, kidneys, thyroid, breast, pelvis, vessels, pregnancy and a few more. There is an Android app, and on an iPhone you can dictate straight from the browser.
One thing we decided on day one: there is no place in the system for a patient's name or date of birth. Those fields simply do not exist. The physician adds them to the printed report herself. I have worked with enough data systems to know how this goes otherwise.
If there is a physician with a probe in your family, show them this. If you are one and something landed in the wrong field, write to me. I have time now: nobody is late anymore.
