Reverse Image Search

The other day, when I was searching for fish images to use in our aquarium programming assignment, I suddenly wondered if there exists a search engine where you could search by images to find matching results instead.  It turned out that the Google already implemented the Reverse Image Search a while ago, but the common internet users (including me) might not know of this feature. Now, every time you see a picture of a celebrity and wonder who that is, the Reverse Image Search would definitely be helpful!

How does this technology work actually? Google’s algorithms are not posted online as open-sourced code, so it’s difficult to know the exact algorithm they used to make Google Reverse Image Search. It’s likely that Google combined the following techniques, then ranked the results using a proprietary algorithm (Quora).

Feature detection is an essential component of computer vision applications, and it’s a popular approach for reverse image search. Computer algorithms would examine every pixel and detect the most “interesting” features in a picture such as blur, rotation, scale and illumination change. There commonly used algorithms for feature detection are SIFT, PCA-SIFT and SURF - each has its own merits and flaws. For example, “SIFT is slow and not good at illumination changes, but it is invariant to rotation, scale changes and affine transformations”, while SURF is fast but it’s not good with rotation and illumination changes. PCA-SIFT is the best but it has problems with image blur(Quora). Usually, the programmer would consider all of these information, and decide to make what kinds of trade-offs depending on the application. 

Search by color is another interesting method for reverse image source. It would create digital signatures for each image based on its color, store all of the signatures in a database, and then the search would return images containing the colors user selected. Visual similarity, on the other hand, analyzes the main attributes of images such as colour, shape, texture, luminoity, complexity, objects and regions (Quora).  The algorithm would map images into an n-dimensional Euclidean vector space (R^n) (for any one who hasn’t taken Linear Algebra, Khan Academy has great resources for this). Similar images are mapped to points close to each other and dissimilar images are mapped to points far away from each other. Then the algorithm would calculate the distance between each point to decide which images are more similar to each other. The result list would be generated as a sorted array containing all the images by their distance to the specified search image. (Mayflower)

On the internet many people could not give a definite answer regarding what combination of algorithms Google used for its reverse image search. The technologies I discussed above are the most common approaches to this problem, but if you have any further insight into this matter, please comment on my blog! Thank you! 

Sources: https://www.quora.com/What-is-the-algorithm-used-by-Googles-reverse-image-search-i-e-search-by-image

https://blog.mayflower.de/1755-Image-similarity-search-with-LIRE.html

Image Source: https://ongoingpro.com/copyright-find-stolen-photos-online-reverse-image-search/

 https://www.quora.com/What-is-the-algorithm-used-by-Googles-reverse-image-search-i-e-search-by-image