A person who uses Instagram.
Lorenzo Di Cola | NurPhoto via Getty Images
Pugs, Ferraris, mountains, brunches, beaches and babies – Instagram is full of it. In fact, it has become one of the largest image databases in the world over the past decade, and the company’s owner, Facebook, uses this treasure trove to teach machines what is in a photo.
Facebook announced Thursday that it has built an artificial intelligence program that can “see” what it is looking at. It did this by sending it over 1 billion public images from Instagram.
The “computer vision” program, nicknamed SEER, outperformed existing AI models in an object recognition test, Facebook said.
It achieved a “classification accuracy score” of 84.2% when it tested ImageNet, a large visual database designed for use in visual object recognition software. In effect, it tests whether an AI program can identify what’s in a photo.
New Approach
While many AI models are trained on carefully labeled datasets, Facebook said SEER has learned how to identify objects in photos by analyzing random, untagged, and uncured Instagram images. This AI technique is known as independent learning (SEER is a game on SElf-supERvised).
“The future of AI lies in creating systems that can learn directly from any information they get – be it text, images, or any other type of data – without relying on carefully curated and labeled datasets to teach them how. they create objects in a photo, interpret a block of text, or perform any of the myriad of other tasks we ask for it, ”the Facebook researchers wrote in a blog post.
“SEER’s performance shows that self-directed learning can excel in real-life computer vision tasks,” she added. “This is a breakthrough that will ultimately pave the way for more flexible, accurate and adaptable computer vision models in the future.”
While this is just a research project, a Facebook spokesperson said the potential uses were relatively broad. They include improved auto-generated text for describing images to visually impaired people, better auto-categorization of items sold in Facebook Marketplace, and better systems for keeping malicious images away from the Facebook platform, the company said.
Privacy problem?
But many Instagram users may be surprised to learn that their images are used to train Facebook AI systems.
“We inform Instagram account holders in our data policy that we are using the information we have to support research and innovation, including technology advancements such as these,” Priya Goyal, a software engineer at Facebook AI Research, told CNBC.
Facebook said it will open some of its software for other researchers to experiment with.
“While we are sharing the details of our research and creating an open-source library that will allow other researchers to use self-study to train models on uncured images, we are not sharing the images or the SEER mode,” said Goyal.
Other major tech companies, including Google and Microsoft, are also trying to push the boundaries of computer vision. Last summer, Google released the SimCLRv2 computer vision model, while OpenAI released iGPT 2.