Updates

2025-02-03

Features

Extraction Tester

The Extraction Tester lets you test how different extraction methods process your documents before creating a RAG pipeline. This helps you choose the best extraction method for your specific documents and use case.

Documentation

• Extraction Tester

Vectorize Iris

Vectorize Iris is a model-based extraction solution that combines extraction and chunking into one streamlined process, making it easier than ever to get clean, usable text from complex documents.

Documentation

• Understanding Vectorize Iris

2025-02-02

Features

Vectorize API (Beta)

Manage your connectors, AI plaforms, vector databases, and pipelines using the Vectorize API. The API is Beta and may change.

Documentation

• Vectorize API

2025-01-23

Features

Query rewriting

Query rewriting uses conversation history to improve retrieval relevance. Before retrieving relevant documents, the system reformulates the user query based on the context of the conversation. This can help you provide more accurate answers to user queries.

Documentation

• Advanced Retrieval

2025-01-16

Features

Performance metrics for RAG pipelines are now available in the Vectorize UI users on the Pro plan. The dashboard displays key metrics to help you evaluate retrieval performance, including:

• Non-Rewritten Relevance

• Rewritten Question Relevance

• Overall Retrieval Health

New pipeline status: HIBERNATING

If a pipeline has been inactive for 14 days (no data processed, no use of the retrieval endpoint), it will hibernate. In order to use the pipeline it must be manually restarted.

If the pipeline remains inactive after restart, after 14 days it will hibernate again.

To prevent a pipeline from hibernating:

• Perform a single retrieval on the pipeline's endpoint.

• A new document being written/updated will also prevent hibernation.

Documentation

• Understanding RAG Pipeline Status

• Retrieval Performance Metrics

Integrations

New Vector Database: Weaviate

Documentation

• Weaviate integration docs

• Quickstart: Create a RAG Pipeline with Weaviate and a Web Crawler

2025-01-10 🎊

New Integrations

• New Vector Databases

  • Qdrant

2024-12-26

New Integrations

• New Vector Databases

  • PostgreSQL

2024-12-06

Features

• Added a context recall metric to RAG evaluations, which measures whether the retrieved context contains the necessary information to answer the provided question. The higher the value, the better the system is at retrieving relevant context.

New Integrations

• New Vector Databases

  • Milvus / Zilliz Cloud

2024-11-29

• Added the ability to select multiple Dropbox folders to read data from.

New Integrations

• Connectors

  • SharePoint

• New AI Platforms

  • Vertex AI

2024-11-21

Features

• Added support for Firecrawl's /scrape endpoint.

• Integrations not in use by a RAG pipeline can now be edited.

New Integrations

• Connectors

  • Firecrawl

  • Google Drive

  • Dropbox

  • OneDrive

• New Vector Databases

  • SingleStore

• New AI Platforms

  • Amazon Bedrock

2024-10-31 🎃

Features

• Connectors and AI Platforms not being used by a pipeline can now be edited.

• Added a tab which displays configuration for the source connector(s), AI platform, and vector database for the selected RAG pipeline.

Pipeline Configuration Tab

• Added an AI Assistant which will answer context-specific questions as you use Vectorize. You can optionally give the answer a thumbs-up or thumbs-down.

AI Assistant

New Embedding Models

• voyage-3

• voyage-3-lite

• voyage-finance-2

• voyage-multilingual-2

• voyage-law-2

• voyage-code-2

---