Endpoints

Creating a RAG Endpoint

To create a RAG Endpoint,

1. Click on New RAG Endpoint button in the top right corner of the FloTorch Console.
2. Provide a Name, Description, Vector Storage (optional) and Generating Model for the RAG Endpoint. RAG Endpoint name should be unique.
3. Once Generating Model is selected, we will automatically fetch the list of Generating Model Versions from the provider.
4. Select the Generating Model Version you want to add to the RAG Endpoint. If you don’t see the Generating Model Version you want to add, you can manually type the Generating Model Version name.
5. You can optionally add System Prompt.
6. Click on Submit button

Once a RAG Endpoint is created, you can see the RAG Endpoint in the RAG Endpoints page. You can access this RAG Endpoint using the name of the RAG Endpoint.

You should make a POST request to the following endpoint to generate a response using RAG Technique.

https://<your-flotorch-gateway-instance-url>/api/v1/rag/<your-rag-endpoint-name>

The request body should be a JSON object with the following fields:

{
    "query": "What is the capital of France?"
}

RAG endpoints can also be created directly from FloTorch Experiment Studio., When you choose to deploy an experiment, it create a RAG Endpoint and you can access it using the name of the RAG Endpoint.

RAG Endpoints once generated, cannot be deleted for now.