Source Connectors

Source connectors are a crucial component of RAG (Retrieval-Augmented Generation) pipelines in Vectorize. They enable the ingestion of unstructured data from various external sources, allowing you to leverage diverse information in your AI-powered applications.

What are Source Connectors?

Source connectors are integrations that allow Vectorize to connect to and retrieve data from different data storage systems and platforms. These connectors serve as bridges between your data sources and the RAG pipeline, facilitating the seamless flow of information into your vector database.

Available Source Connectors

Vectorize supports various source connectors to accommodate different data storage preferences:

• Amazon S3: Connect to Amazon's scalable object storage service.

• Azure Blob Storage: Integrate with Microsoft Azure's cloud storage solution.

• Google Cloud Storage: Access data stored in Google's cloud platform.

• Discord: Retrieve information from Discord channels and servers.

• Web Crawler: Automatically gather data from specified websites.

Additionally, there are several connectors coming soon, including FileUpload, Salesforce, Google Drive, Confluence, Dropbox, OneDrive, and Zendesk.

Configuring Source Connectors

You can configure source connectors in two ways:

1. From the Source Connectors Section

1. Navigate to the Vectorize dashboard.

2. In the left sidebar, under "Integrations," click on "Source Connectors."

Source Connector Menu

3. You'll see a list of currently configured connectors in your workspace.

Source Connector Section

4. To add a new connector, click the "New Source Connector" button.

5. A list of available and upcoming connectors will appear. Select the desired connector type.

   

   Source Connector Options
6. Follow the prompts to configure the selected connector.

2. While Creating a RAG Pipeline

1. During the RAG Pipeline creation process, you'll reach a step to configure source connectors.

2. Click on the "Add source connector" button.

RAG Pipeline Source Connector

3. Choose from the list of available connectors.

RAG Pipeline Source Connector Options

4. Configure the selected connector according to the provided instructions.

Note: Source connectors configured as part of a RAG Pipeline will automatically appear in the Source Connectors list for your organization and can be reused in future RAG pipelines.

Connector Status

Connectors in Vectorize have different status indicators:

• Beta: These connectors are available for use but may have limited features or require additional testing.

• Paid Plans Only: Some connectors, like Discord, are only available for paid plans.

• Coming Soon: These connectors are under development and will be available in future updates.

For detailed information on configuring specific connectors, please refer to their individual documentation pages linked above.