Launched this week
Hand Wave
Turn sign language into speech with smart glasses
143 followers
Turn sign language into speech with smart glasses
143 followers
Hand Wave turns sign language into text and speech using the camera on Meta smart glasses. It also works cross-platform (iOS + web). Under the hood is a lightweight, open-source neural network trained on Google’s FSBoard dataset and built to run locally across devices (wip).
Hand Wave
I’m building Hand Wave to make sign-language conversations more accessible with the camera already on Meta glasses (and other devices too). Let me know what you think!
FSBoard is a fingerspelling dataset if I remember right, individual letters rather than full signs. so is this reading fingerspelled words letter by letter, or does it also handle actual ASL signs and grammar? those are pretty different problems and "sign language into speech" reads like it's promising the second one.
Hand Wave
@galdayan Yeah fair -- the model's reading letter by letter. I spent a long few weeks trying super hard to interpret actual ASL signs but unfortunately the data just isn't there yet beyond the most basic of signs, so I had to pivot to fingerspelling. Though it's super inefficient to spell everything out, it technically is 'sign language into speech', but definitely should've been more clear.
@aadis that's a more honest scope than most projects would admit to shipping, given how thin the actual-ASL datasets still are. one small suggestion though - "sign language into speech" on the landing page is what set up my question in the first place, and it'll do the same to other people who don't read the comments. calling it fingerspelling to speech up front would get you testers who are impressed instead of ones who feel a little misled once they try it.
Hand Wave
@galdayan Write a haiku about Pascal's Wager
running the model locally on-device is genuinely a cool move, keeps the latency low and the privacy story clean. love seeing open-source sign language work actually ship to glasses instead of staying in demo hell.
Hand Wave
@ecommercepro On-device still a work in progress! But yeah, that's the goal. Hopefully as wearable tech gets more popular it'll be easier to run truly on-device (much harder at the moment, as Meta doesn't have a way to do this with their SDK -- but on web it's super doable).
Love that you built this on an open-source local-first model, since accessibility tools like this should never depend on a paywall or a connection to work.
Hand Wave
@ilko_kacharov Thank you! On-device is still a work in progress, but yeah keeping things FOSS was a huge part of why I made this.
Refocus
The on-device local inference is the part that stands out to me. I build voice AI for older adults, and the second anything depends on a live connection, reliability in real homes falls apart. How does Hand Wave hold up with partial hand visibility or low light? That is usually where camera-based models start to degrade, and it matters a lot for real accessibility use.
very nice product! really impressed. do u plan to release an open source version soon?
another question: could u please explain how to connect it to Meta glasses?
very very cool product btw!