This is interesting:

http://www.seas.upenn.edu/media/feature-taskar.php

Not sure what the results are like. I wonder what the results would be for something more stylised that also has a huge fanbase, like anime. It's possible you could get more accurate results and use that as a middle step to more complex images.