iOS 10 Photos App

Behind Apple’s Advanced Computer Vision for Photos App→

Photos app recognises and distinguishes the following 7 facial expressions. Expressions are distinguished after forming a “faceprint”. These distinction are used for searching. They are also rated and indexed for generation Memories and montages.

Greedy, Disgust, Neutral, Scream, Smiling, Surprise, Suspicious

Photos app will generate Moments that falls within the following 33 categories. Default name of the moment will be automatically generated using metadata from the photos and tags from analysis of photos.

Memories from areas of interest, Best of past memories, Memories that break out of routine, Celebration in history, Contextual memories, Crowd, Day in history, Holiday in history, Location of interest, Nearby, New contextual memories, New memories, Person’s Birthdays, Person’s memories, Recent events (calendar, crowd, holiday, people, person, social, trip, weekend), Region of interest, Social group memories, Sometime memories, Special memories, Favourited, Trips, Week in history, Weekend, Year summary, Last week, Last Weekend

Kay Yin

I’m impressed to see what the Photos app is able to generate and distinguish, only computing data locally. The Photos app is also pretty accurate when recognising faces and objects. This compliments nicely the memories it is able to put together in nicely theme-matched videos. In addition to the above, Kay also provides a complete list of things the Photos app can be queried for as well as the categories to which they belong to. This list will continue to grow over time and it could easily be part of OTA updates. As Craig Federighi said in John Gruber’s The Talk Show (53:40)

There is this idea that if you don’t have the data, how would you ever learn? As it turns out, if you wanna get pictures of mountains, you don’t need to get it out of people’s personal photo libraries.