Using the Retrieval Endpoint
Last updated
Was this helpful?
Last updated
Was this helpful?
This guide shows how to access and use the retrieval endpoint for a RAG Pipeline in Vectorize.
Navigate to the RAG Pipelines section in the Vectorize dashboard.
Click on the name of your desired pipeline (e.g., "friends-scripts").
In the pipeline details view, click the Connect tab.
In the Connect tab, you'll find:
Use the "Manage access tokens" button to create or manage authentication tokens.
The unique URL for your pipeline's retrieval endpoint is displayed here.
Click "Copy to Clipboard" to copy the URL.
The endpoint calculates a vector for the question text and performs a similarity search on the vector DB.
If rerank is set to true, which is the default, the results are passed to a reranking model and returned in the order by the rerank model's relevance score. Otherwise, results are ordered by similarity score.
Metadata filtering can be applied to the results by passing a metadata-filters object. See below for more information.
You can enable query rewriting by passing a context object. The endpoint automatically reformulates the question to improve search relevance. See Advanced Retrieval for details.
An example cURL request is provided:
Metadata filtering in a retrieval endpoint allows you to narrow down the data retrieved from your vector database based on specific metadata attributes. This allows you to filter data not just by similarity to a query, but also by tags, categories, or other metadata properties, improving the precision of context provided to the large language model (LLM).
Vectorize will automatically add system metadata to the records it writes to vector database. For details on these metadata fields see System metadata.
Some sources, like Google Cloud Storage support path metadata which allow you to define a regular expression to convert the path of the file into metadata. For example, if your files are organized in folders like /user/john, /user/mary, etc, you can convert the username to metadata. For examples on how to do this, see Path Metadata Regex & Path Regex Group Name.
You can also configure user-defined metadata, which can be specified on a per-document basis on some source connectors. For more information, see Using User-defined Metadata.
You can use all these types of metadata with the retrieval endpoint.
Pinecone supports multiple value matches per key, so the values you're filtering on must be formatted as a list.
Example using cURL:
All other databases support a single value per key.
Example using cURL:
If you'd like to learn more about metadata, go to Understanding Metadata in RAG.
OR
Learn how to configure user-defined metadata for your indexes in Using User-defined Metadata, then use these fields for filtering in your AI application.
