Feb. 5 (UPI) — Facebook users can now search for photos based on the content of the images by moving beyond text-driven technology to incorporating artificial intelligence.
“Whether an image was discoverable was dependent on whether it was sufficiently tagged or had the right caption — until now,” according to a Facebook update by Joaquin Quiñonero Candela.
“That’s changing because we’ve pushed computer vision to the next stage with the goal of understanding images at the pixel level,” Candela wrote. “This helps our systems do things like recognize what’s in an image, what type of scene it is, if it’s a well-known landmark, and so on. This, in turn, helps us better describe photos for the visually impaired and provide better search results for posts with images and videos.”
Facebook developed a system called FBLearner Flow and is currently using it to run 1.2 million AI experiments per month.
“When these techniques are applied at Facebook, photos pass through a deep-learning engine that can segment the image and identify objects and scenes, and attach even more meaning to the photo,” according to the post.
Lumos, built on top of FBLearner Flow, is the “platform we built for image and video understanding.”
“Thanks to the neural networks, the system already knows what’s depicted in the billions of photos that have been uploaded to Facebook,” he wrote. “So a search for ‘black shirt,’ for instance, would return photos of black shirts, even if the person who uploaded it didn’t add any tags that say ‘black shirt.’ ”
PC Mag noted the commercial possibilities: “If you stumble across a black shirt or a wedding dress you really like, for instance, Facebook could alert the company that posted the image and encourage it to target you with ads.”
The new technology will have images described aloud to the visually impaired.
Candela said: “While these new developments are noteworthy, we have a long and exciting road ahead and are just scratching the surface of what is possible with a self-serve computer vision platform.”
