Creating a RAG Evaluation
The Create RAG Evaluation page allows you to set up and configure an evaluation of various vectorization strategies and embedding models for your data. This tool helps assess the accuracy and relevancy of different strategies and provides insights into optimizing your Retrieval-Augmented Generation (RAG) applications.
Available Options
1. RAG Evaluation Name
Description: Provide a name for your RAG evaluation. This will help you identify and track different evaluations.
Input Type: Text Field
2. Upload Documents
Description: You can upload up to 5 files to be used for your RAG Evaluation. These files will be chunked and embedded as part of the evaluation process.
Supported File Types:
PDFs
*.pdf
Markdown
*.md
Text
*.txt
HTML
*.html, *.htm
Microsoft Word Docs
*.doc, *.docx
File Size Limits:
PDF, HTML, and Doc/Docx: Maximum file size of 5MB
Text and Markdown: Maximum file size of 500KB
Additional Note: Empty files or files that contain only images will abort the RAG evaluation.
3. Select Vectorization Strategy
You can choose between two types of vectorization strategies:
Default:
This option evaluates 4 vectorization strategies using the following embedding models:
OpenAI Ada v2 (1536 dimensions)
OpenAI v3 Large (3072 dimensions)
OpenAI v3 Small (1536 dimensions)
Voyage AI Instruct v2 (1024 dimensions)
The chunking strategy used is paragraph-based.
Custom:
Allows you to select up to 4 custom embedding models and chunking strategies to test. This option provides more flexibility in determining the best-performing strategy for your data.
4. Select Vector Database
The vector database stores and retrieves the vector embeddings. Vectorize provides a database engine for each of the available database options. You do not need to supply your own database or bring your own API key. You can select from the following cloud-based databases:
Couchbase Capella: A modern cloud database for real-time applications.
DataStax Astra: A cloud database based on Apache Cassandra.
Elastic Cloud: Elastic's serverless cloud instance with vector support.
Pinecone: A top-rated cloud vector database optimized for vector search.
5. Configure RAG Evaluation (Custom Strategy Only)
For each of the 4 strategies, you can customize the following options:
Embedding Model:
Select from available models such as OpenAI Ada v2, OpenAI v3 Small, OpenAI v3 Large, VoyageAI 2, etc.
Chunking Strategy:
Choose a chunking strategy to segment your document, with the option to chunk by paragraphs.
Top K:
Select the number of top similar vectors (k) to retrieve from the database.
Chunk Size (Tokens):
Specify the maximum number of tokens in each chunk (e.g., 500 tokens).
Last updated