Working with Embeddings: Closed versus Open Supply | by Ida Silfverskiöld | Sep, 2024

Working with Retrieval

Utilizing strategies to enhance semantic search

Demonstration of clustering earlier than performing semantic search | Picture by creator

If you happen to’re not a member however need to learn this text, see this buddy hyperlink right here.

Embeddings are a cornerstone of pure language processing. You are able to do rather a lot with embeddings, however one of many extra well-liked makes use of is semantic search utilized in retrieval functions.

Though all the tech neighborhood is abuzz with understanding how information graph retrieval pipelines work, utilizing customary vector retrieval isn’t out of fashion.

You’ll discover a number of articles exhibiting you tips on how to filter out irrelevant outcomes from semantic searches, one thing we’ll even be specializing in right here utilizing strategies resembling clustering and re-ranking.

The fundamental focus of this text, although, is to examine open supply and closed supply embedding fashions of varied sizes.