AI Platforms
AI Platforms are essential components in RAG (Retrieval-Augmented Generation) pipelines within Vectorize. They are responsible for generating text embeddings, which are crucial for transforming unstructured text data into vector representations that can be efficiently stored and searched in vector databases.
What are AI Platform Integrations?
AI Platform integrations in Vectorize allow you to connect to various AI services that provide advanced natural language processing capabilities, particularly for generating text embeddings. These embeddings capture the semantic meaning of text, enabling more accurate and context-aware information retrieval in RAG pipelines.
Available AI Platform Integrations
Vectorize currently supports the following AI Platform integrations:
OpenAI: Leverage OpenAI's powerful language models for generating high-quality text embeddings.
Voyage AI: Utilize Voyage AI's specialized embedding models for various natural language processing tasks.
Configuring AI Platform Integrations
You can configure AI Platform integrations in two ways:
1. From the AI Platforms Section
Navigate to the Vectorize dashboard.
In the left sidebar, under "Integrations," click on "AI Platforms."
You'll see a list of currently configured AI Platform integrations in your workspace.
To add a new integration, click the "New AI Platform" button.
Choose from the list of available AI Platform options.
Follow the prompts to configure the selected AI Platform integration.
2. While Creating a RAG Pipeline
During the RAG Pipeline creation process, you'll reach a step to configure the AI Platform.
You can either select an existing AI Platform integration or create a new one.
If creating a new integration, choose from the available options and follow the configuration steps.
Select the specific embedding model you want to use for this pipeline.
Note: AI Platform integrations configured as part of a RAG Pipeline will automatically appear in the AI Platforms list for your organization and can be reused in future RAG pipelines.
Configuration Details
When setting up an AI Platform integration, you'll typically need to provide:
API keys or access tokens for authentication
Model selection (if the platform offers multiple embedding models)
Any additional configuration options specific to the chosen AI Platform
Using AI Platforms in RAG Pipelines
In a RAG Pipeline, the integrated AI Platform is used to:
Generate embeddings for the documents ingested through source connectors
Create embeddings for user queries during the retrieval process
These embeddings enable semantic similarity searches in the vector database, allowing for more accurate and context-aware information retrieval.
For detailed information on configuring specific AI Platform integrations, please refer to their individual documentation pages linked above.
Last updated
