Skip to content
FloTorch

ADK Plugin: LLM

The FlotorchADKLLM provides an ADK-compatible interface for accessing language models through FloTorch Gateway. It acts as a wrapper that seamlessly integrates FloTorch’s managed model infrastructure with Google ADK’s agent framework, handling complexities such as tool call conversion, response parsing, and structured output generation.

Prerequisites

Section titled “Prerequisites”

Before using FlotorchADKLLM, ensure you have completed the general prerequisites outlined in the ADK Plugin Overview, Including installation and environment configuration.

Configuration

Section titled “Configuration”

Parameters

Section titled “Parameters”

Configure your LLM instance with the following parameters:

FlotorchADKLLM(
    model_id: str,      # Model identifier from FloTorch Console (required)
    api_key: str,       # FloTorch API key for authentication (required)
    base_url: str       # FloTorch Gateway endpoint URL (required)
)

Parameter Details:

  • model_id - The unique identifier of the model configured in FloTorch Console
  • api_key - Authentication key for accessing FloTorch Gateway (can be set via environment variable)
  • base_url - The FloTorch Gateway endpoint URL (can be set via environment variable)

Features

Section titled “Features”

Tool Call Support

Section titled “Tool Call Support”

Automatically handles tool call conversion and management:

  • Format Conversion - Seamlessly converts between OpenAI-format and ADK-format tool calls
  • Argument Parsing - Handles function call arguments and parameter validation
  • Response Management - Processes function responses and integrates them into the conversation flow
  • Metadata Handling - Preserves tool call IDs and metadata throughout the execution

Structured Response Generation

Section titled “Structured Response Generation”

Provides comprehensive support for structured outputs:

  • Pydantic Integration - Uses Pydantic models for schema definition and validation
  • Automatic Schema Conversion - Converts Pydantic schemas to ADK-compatible formats
  • Type Validation - Ensures response data conforms to defined schemas
  • Parsing Support - Automatically parses structured outputs from model responses

Error Handling

Section titled “Error Handling”

Implements robust error handling mechanisms:

  • Tool Call Errors - Gracefully handles tool call parsing and execution failures
  • Response Format Issues - Manages unexpected response formats with fallback strategies
  • Network Resilience - Handles connectivity issues with appropriate retry logic and error messages
  • Validation Errors - Provides clear feedback on schema validation failures

Usage Example

Section titled “Usage Example”

Basic LLM Usage

Section titled “Basic LLM Usage”
from flotorch.adk.llm import FlotorchADKLLM
from google.adk.agents import LlmAgent


# Initialize FloTorch LLM
llm = FlotorchADKLLM(
    model_id="your-model-id",
    api_key="your_api_key",
    base_url="https://gateway.flotorch.cloud"
)


# Create an ADK agent with FloTorch LLM
agent = LlmAgent(
    name="my-agent",
    model=llm,
    description="An AI assistant",
    instruction="You are a helpful assistant."
)

Best Practices

Section titled “Best Practices”
  1. Environment Variables - Use environment variables for credentials to enhance security
  2. Model Selection - Choose appropriate models based on your task requirements
  3. Error Handling - Implement proper error handling for production environments

Next Steps

Section titled “Next Steps”