Search

The Modus Collections API provides a robust way to store, retrieve, and search through data using both natural language and vector-based search methods. By leveraging embeddings, developers can enable semantic and similarity-based searches, improving the relevance of search results within their applications.

For example, with natural language similarity, if you search for a product description like ‘sleek red sports car’, the search method returns similar product descriptions such as “luxury sports car in red” or ‘high-speed car with sleek design’.

​

Understanding key components

Collections: a collection is a structured storage that organizes and stores textual data and associated metadata. Collections enable sophisticated search, retrieval, and classification tasks using vector embeddings.

Search Methods: a search method associated with a collection, defines how to convert collection items into a vector representation and provides indexing parameters.

Vector embeddings: for vector-based search and comparison, Modus converts each item in the collection into a vector representation called embedding. By embedding data, you enable powerful natural language and similarity-based searches.

​

Initializing your collection

Before implementing search, ensure you have defined a collection in the app manifest. In this example, myProducts is the collection used to store product descriptions.

First, we need to populate the collection with items (for example, product descriptions). You can insert individual or multiple items using the upsert and upsertBatch methods, respectively.

Use upsert to insert a product description into the collection. If you don’t specify a key, Modus generates a unique key for you.

​

Configure your search method

The search capability relies on a search method and embedding function. To configure your search method.

​

Create an embedding function

An embedding function is any API function that transforms text into vectors that represent their meaning in a high-dimensional space.

Embeddings functions must have the following signature:

Modus computes vectors using embedding models. Here are a few examples:

• all-MiniLM-L6-v2
• OpenAI

  "models": {
    // model card: https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2
    "minilm": {
      "sourceModel": "sentence-transformers/all-MiniLM-L6-v2", // model name on the provider
      "provider": "hugging-face", // provider for this model
      "connection": "hypermode" // host where the model is running
    }
  }

​

Declare the search method

With an embedding function in place, declare a search method in the collection properties.

  "collections": {
    "myProducts": {
        "searchMethods": {
            "searchMethod1": {
                "embedder": "embed" // embedding function name
            }
        }
    }
  }

​

Implement semantic similarity search

With the products stored, you can now search the collection by semantic similarity. The search] API computes an embedding for the provided text, compares it with the embeddings of the items in the collection, and returns the most similar items.

​

Search result format

The search response is a CollectionSearchResult containing the following fields:

• collection: the name of the collection.
• status: the status of the operation.
• objects: the search result items with their text, distance, and score values.
  • distance: a lower value indicates a closer match between the search query and the item in the collection
  • score: a higher value (closer to 1) represents a better match

{
  "collection": "myProducts",
  "status": "success",
  "objects": [
    {
      "key": "item-key-123",
      "text": "Sample product description",
      "distance": 0.05,
      "score": 0.95
    }
  ]
}

​

Search for similar Items

When you need to search similar items to a given item, use the searchByVector API. Retrieve the vector associated with the given item by its key, then perform a search using that vector.

​

Develop locally with Collections

While Collections expose a key-value interface for working with data, a PostgreSQL database instance persists the data. When using Collections in a Modus app deployed to the Hypermode platform, Hypermode manages this PostgreSQL database and users don’t need to perform any additional setup. However, when developing with Modus locally or outside of the Hypermode platform, you need a PostgreSQL instance for the Collections persistence layer.

Any hosting method for this PostgreSQL database is sufficient, so feel free to use your favorite method of installing and running PostgreSQL locally or in the cloud. You must apply a database migration after you start the PostgreSQL database (Step 3 below) and set the MODUS_DB environment variable with your PostgreSQL database connection string (Step 4 below). The steps below describe using Docker to start and configure a PostgreSQL instance for local development with Collections.

To start and configure a local PostgreSQL instance for working with Collections locally using Docker Compose, follow these steps:

git clone https://github.com/hypermodeinc/modus.git

cd modus/runtime/tools/local
docker-compose up

brew install golang-migrate

export POSTGRESQL_URL='postgresql://postgres:postgres@localhost:5433/my-runtime-db?sslmode=disable'
migrate -database ${POSTGRESQL_URL} -path ../../db/migrations up

export MODUS_DB=postgresql://postgres:postgres@localhost:5433/my-runtime-db?sslmode=disable

You can now use Collections locally in your Modus app.