Visual search allows people to use images to search. For more information on our previous work, please refer to our KDD’15 paper. We’ll be releasing a paper describing our findings in building a large scale visual search system using deep learning features in the near future.

In order to do this task efficiently, we built a distributed index and search system (using open-source tools) that allows us to scale to billions of images and find thousands of visually similar results in a fraction of a second. To find visually similar results for a Pin, we consider the similarity scores of a given feature to billions of other features. For the past couple of months, we’ve been experimenting with improving Related Pins with these visual signals, as detailed in our latest white paper, released today. These features can then be used to compute a similarity score between any two images. With close collaboration with members of the Berkeley Vision and Learning Center, we use deep learning to learn powerful image features by utilizing our richly annotated dataset of billions of Pins curated by Pinners. The core of our visual search system is how we represent images, and was built in just a few months by a team of four engineers.

Pinterest search by image