Bunny doodles might soon be recognized by search engines
Apps such as DrawSomething are turning everyday people into miniature Picasso’s, but for every well defined line drawing you’ll get some confusing blobs that bear as much resemblance to a sailboat as they do a Rorschach inkblot test. It would be pretty handy if you could somehow take those random shapes and identify what they were actually meant to represent, and this could have more ramifications than just allowing you to cheat your way to a high score.
Computer researchers from Brown University and the Technical University of Berlin have created a computer application that can recognize simple sketches of objects almost as well as humans, with a 56 percent success rate (humans have a 73% rate). This isn’t a first for computers- they can already recognize detailed sketches, such as those used in crime scene mockups, but this is new ground in terms of very simple drawings- such as those of the bunnies shown above. Here you have very few identifiers to go on, and the programme is still able to recognize key traits such as ear shape and whiskers to call the sketches a ‘bunny’. If this evolves, sketch based search applications might be more prevalent and could enable people with limited literacy to find information quickly, as well as aid children using search tools who can’t vocalize what they are looking for.
Some examples of what the sketch based search program recognizes.. and doesn’t.
This idea- and tool- is still in its infancy, but there is a related iPhone app you can play with and there are currently a library of 250 sketches that the program uses as a database, which includes animals, rainbows and revolvers.
James Hays, assistant professor of computer science at Brown believes that many things we recognize are due to our cultural conditioning. ‘It might be that we only recognize it as a rabbit because we all grew up that way. Whoever got the ball rolling on caricaturing rabbits like that, that’s just how we all draw them now, ‘ he says.
The database of objects was created by analyzing items that are most frequently drawn, and used label frequency in annotating photographs. Other items were added based on the researchers interests- such as rainbows- which is how the 250 subset came about. 20,000 sketches were collected and placed into the machine so it could learn to recognize different drawing algorithms, and this lead to the machine manging to *mostly* identify new sketches that corresponded to what it had already.
It will be interesting to see how the is progresses and if the machine gets mote sophisticated as all levels of artists are added to the mix- I’d love to see how it reacts to images created by children!
For more information on the project read about it on the Brown University Website.
