Augmented Reality Virtual Keyboard using HTML 5 video
I’ve been pretty impressed with some of the stuff ARKit can do, as well as the Microsoft Surface demos.
I decided that we need to do that on the web, too, so I build a virtual keypad, using HTML5 video and a webcam.
In HTML5, a common (example 1, example 2) pattern is to take an image from a video, copy it to a canvas and manipulate it here using Javascript. That’s the general pattern I’ve used.
I use a colored marker (setup in a calibration phase) and track its location. In order to take account of brightness changes, etc, I do cosine similarity tests on average color vectors for each “key” on the keypad using the Sylvester Javascript vector library.
I think there’s a fair bit of potential for video based interfaces. I’d like to see a combination of this and the HTML5 drumkit, for example.
You can try it out here: http://demo.nicklothian.com/keypad/. You’ll need to be using Firefox 3.5+.
Here’s a demo video (ironically as an embedded flash video):
Anees Uddin Said,
November 19, 2009 @ 5:02 am
Does the camera need to be static for that to remain workable?
Ed Spencer Said,
November 19, 2009 @ 2:15 pm
Nice work, though the embedded video seems too wide for the page and gets clipped behind the left nav (on FF3.5 anyway).
Would be even better if you’d chuck it up on Github so people can fork it :)
Nick Lothian Said,
November 19, 2009 @ 4:14 pm
@Ed – yes, good idea.
@Anees – it shouldn’t – it relies on color matching rather than position. I can’t say I’ve tried it, though.