Agent Studio
Adr

ADR-014: Prompt Library

Tenant-managed prompt templates for customizable agent content

ADR-014: Prompt Library

Status

Proposed

Context

ADR-013 established a hybrid approach for dynamic prompt generation where:

  • Backend computes context values
  • Agent templates reference those values via {{path}} syntax

However, ADR-013 places greeting scripts and language-specific content in the backend code, which has limitations:

  1. Greeting changes require backend deployment - Content updates need code releases
  2. No per-tenant customization - All tenants get the same scripts
  3. Mixed concerns - Backend (notification service) owns greeting content that belongs in the agent platform
  4. No A/B testing - Can't experiment with different greetings per tenant
  5. Scaling issues - Adding new languages requires backend code changes

The Opportunity

Since we're early in agent-studio adoption (only meal-logging agent migrated), we can design the architecture properly now rather than accumulate technical debt.

Requirements

  1. Dashboard-managed content - Tenants edit greetings, prompts, messages via UI
  2. Multi-language support - Store language variants of each template
  3. Version control - Track changes, rollback if needed
  4. Variable interpolation - Templates can reference user context
  5. Inheritance - Platform defaults with tenant overrides
  6. No backend content - Backend sends only raw context, not computed greetings

Decision

Introduce a Prompt Library feature in Agent Studio where prompt templates are first-class entities managed per-tenant.

Data Model

Tenant
├── PromptTemplates
│   ├── id: uuid
│   ├── tenant_id: uuid
│   ├── slug: string (unique per tenant)
│   ├── name: string
│   ├── description: string
│   ├── category: enum (greeting, closing, instruction, error)
│   ├── content: jsonb  # Language variants
│   │   {
│   │     "en": "Hello {{user.name}}! What did you have for {{meal.current}} today?",
│   │     "hi": "Namaste {{user.name}}! Aaj {{meal.current}} mein kya khaya?",
│   │     "ta": "வணக்கம் {{user.name}}! இன்று {{meal.current}} என்ன சாப்பிட்டீர்கள்?"
│   │   }
│   ├── variables: jsonb  # Variable definitions with defaults
│   │   [
│   │     {"name": "user.name", "type": "string", "default": "there"},
│   │     {"name": "meal.current", "type": "string", "required": true}
│   │   ]
│   ├── metadata: jsonb  # Additional config
│   │   {
│   │     "interruptible": true,
│   │     "voice_speed": 1.0
│   │   }
│   ├── is_system: bool  # Platform-provided template
│   ├── created_at: timestamp
│   └── updated_at: timestamp
│
├── Agents
│   └── config.prompt
│       ├── greeting: "{{prompt.returning_user_greeting}}"  # Reference to template
│       ├── greeting_interruptible: "{{prompt.returning_user_greeting.interruptible}}"
│       └── system: "{{prompt.meal_coach_system}}"

Template Categories

CategoryPurposeExamples
greetingInitial agent greetingWelcome, returning user greeting
closingEnd of conversationSign-off, next steps
instructionSystem promptsPersona, rules, guidelines
errorError messagesFallback, retry prompts

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                    Backend System (Notification)                 │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  Sends ONLY raw context (no greeting generation):               │
│  {                                                               │
│    "user_context": {                                            │
│      "name": "Rahul",                                           │
│      "language": "hi",                                          │
│      "is_new_user": false,                                      │
│      "timezone": "Asia/Kolkata",                                │
│      "pending_meals": ["Breakfast", "Lunch"],                   │
│      "logged_meals": ["Dinner"],                                │
│      "scheduled_time": "8 PM",                                  │
│      "goal": "HBA1C_REDUCTION",                                 │
│      "start_value": "8.5",                                      │
│      "target_value": "6.5"                                      │
│    }                                                            │
│  }                                                               │
│                                                                  │
└───────────────────────────────────────────────────────────────────┘
                    │
                    ▼
┌─────────────────────────────────────────────────────────────────┐
│                    Agent Studio Worker                           │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│   1. Load Agent Config                                          │
│      greeting: "{{prompt.returning_user_greeting}}"             │
│                                                                  │
│   2. Resolve Prompt Reference                                   │
│      → Fetch PromptTemplate(slug="returning_user_greeting")     │
│      → Select language variant: content["hi"]                   │
│                                                                  │
│   3. Interpolate Variables                                      │
│      "Namaste {{user.name}}! Aaj {{meal.current}} mein kya     │
│       khaya?" → "Namaste Rahul! Aaj Breakfast mein kya khaya?" │
│                                                                  │
│   4. Apply Metadata                                             │
│      interruptible: true (from template metadata)               │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘

Conditional Template Selection

For different user types (new vs returning), use the agent's input context mapping:

{
  "config": {
    "input_context_mapping": [
      {
        "source": "user.is_new_user",
        "target": "greeting_template",
        "transform": {
          "type": "conditional",
          "conditions": [
            { "when": true, "then": "welcome_new_user" },
            { "when": false, "then": "returning_user_greeting" }
          ]
        }
      }
    ],
    "prompt": {
      "greeting": "{{prompt[greeting_template]}}",
      "greeting_interruptible": "{{prompt[greeting_template].interruptible}}"
    }
  }
}

Or simpler - let the worker handle it:

# Worker determines which template to use
if context.get("user.is_new_user"):
    greeting_slug = "welcome_new_user"
    interruptible = False
else:
    greeting_slug = "returning_user_greeting"
    interruptible = True

template = await self.get_prompt_template(greeting_slug)
greeting = template.resolve(context)

API Endpoints

# Prompt Template CRUD
GET    /api/v1/prompts                    # List templates
POST   /api/v1/prompts                    # Create template
GET    /api/v1/prompts/:slug              # Get template
PATCH  /api/v1/prompts/:slug              # Update template
DELETE /api/v1/prompts/:slug              # Delete template

# Bulk operations
POST   /api/v1/prompts/import             # Import templates (JSON)
GET    /api/v1/prompts/export             # Export all templates

# Template preview
POST   /api/v1/prompts/:slug/preview      # Preview with sample context

Dashboard UI

┌─────────────────────────────────────────────────────────────────┐
│  Prompt Library                                    [+ New Prompt]│
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  ┌─────────────────────────────────────────────────────────────┐│
│  │ 🎤 Greetings                                                ││
│  ├─────────────────────────────────────────────────────────────┤│
│  │ welcome_new_user          Welcome script for new users      ││
│  │ returning_user_greeting   Quick greeting for returning...   ││
│  └─────────────────────────────────────────────────────────────┘│
│                                                                  │
│  ┌─────────────────────────────────────────────────────────────┐│
│  │ 👋 Closings                                                 ││
│  ├─────────────────────────────────────────────────────────────┤│
│  │ meal_logging_complete     Sign-off after logging meals      ││
│  │ call_timeout_closing      Message when call times out       ││
│  └─────────────────────────────────────────────────────────────┘│
│                                                                  │
│  ┌─────────────────────────────────────────────────────────────┐│
│  │ 📝 Instructions                                             ││
│  ├─────────────────────────────────────────────────────────────┤│
│  │ meal_coach_system         System prompt for meal coach      ││
│  │ meal_coach_persona        Persona and tone guidelines       ││
│  └─────────────────────────────────────────────────────────────┘│
│                                                                  │
└─────────────────────────────────────────────────────────────────┘

Template Editor

┌─────────────────────────────────────────────────────────────────┐
│  Edit: returning_user_greeting                         [Save]   │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  Name: Returning User Greeting                                  │
│  Slug: returning_user_greeting                                  │
│  Category: [Greeting ▼]                                         │
│                                                                  │
│  ┌─ Languages ──────────────────────────────────────────────────┐
│  │ [English] [Hindi ✓] [Tamil] [Telugu] [+ Add]                │
│  └──────────────────────────────────────────────────────────────┘
│                                                                  │
│  Content (Hindi):                                                │
│  ┌──────────────────────────────────────────────────────────────┐
│  │ Namaste {{user.name}}! Aaj {{meal.current}} mein kya khaya? │
│  └──────────────────────────────────────────────────────────────┘
│                                                                  │
│  Variables:                                                      │
│  ┌──────────────────────────────────────────────────────────────┐
│  │ user.name     string   Default: "there"                     │
│  │ meal.current  string   Required                              │
│  └──────────────────────────────────────────────────────────────┘
│                                                                  │
│  Metadata:                                                       │
│  ┌──────────────────────────────────────────────────────────────┐
│  │ Interruptible: [✓]                                          │
│  │ Voice Speed:   [1.0]                                         │
│  └──────────────────────────────────────────────────────────────┘
│                                                                  │
│  Preview:                                                        │
│  ┌──────────────────────────────────────────────────────────────┐
│  │ Sample context: {"user": {"name": "Rahul"}, "meal":         │
│  │                  {"current": "Breakfast"}}                   │
│  │                                                              │
│  │ Result: "Namaste Rahul! Aaj Breakfast mein kya khaya?"      │
│  └──────────────────────────────────────────────────────────────┘
│                                                                  │
└─────────────────────────────────────────────────────────────────┘

Platform Default Templates

Agent Studio ships with default templates that tenants can override:

PLATFORM_DEFAULTS = [
    {
        "slug": "welcome_new_user",
        "name": "Welcome New User",
        "category": "greeting",
        "content": {
            "en": "Welcome {{user.name}}! I'm your AI health coach...",
            "hi": "Namaste {{user.name}}! Main aapki AI health coach hoon..."
        },
        "metadata": {"interruptible": False},
        "is_system": True
    },
    {
        "slug": "returning_user_greeting", 
        "name": "Returning User Greeting",
        "category": "greeting",
        "content": {
            "en": "Hi {{user.name}}! What did you have for {{meal.current}} today?",
            "hi": "Namaste {{user.name}}! Aaj {{meal.current}} mein kya khaya?"
        },
        "metadata": {"interruptible": True},
        "is_system": True
    }
]

Tenants can:

  1. Use platform defaults as-is
  2. Override specific languages
  3. Create entirely new templates

Worker Integration

class CallSession:
    async def _get_greeting(self, agent: Agent, context: CallContext) -> tuple[str, bool]:
        """Get greeting text and interruptibility from prompt library."""
        
        # Determine which template to use based on user type
        is_new_user = context.get("user.is_new_user", False)
        
        if is_new_user:
            template_slug = "welcome_new_user"
        else:
            template_slug = "returning_user_greeting"
        
        # Check if agent overrides the template
        agent_greeting = agent.prompt.greeting
        if agent_greeting and agent_greeting.startswith("{{prompt."):
            # Extract slug from {{prompt.template_slug}}
            template_slug = agent_greeting[9:-2]
        
        # Fetch template from database
        template = await self._repo.get_prompt_template(
            tenant_id=self._tenant_id,
            slug=template_slug
        )
        
        if not template:
            # Fallback to platform default
            template = await self._repo.get_platform_default_template(template_slug)
        
        # Select language variant
        language = context.get("user.language", "en")
        content = template.content.get(language) or template.content.get("en")
        
        # Resolve variables
        greeting = context.resolve(content)
        interruptible = template.metadata.get("interruptible", True)
        
        return greeting, interruptible

Database Schema

CREATE TABLE prompt_templates (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    tenant_id UUID REFERENCES tenants(id) ON DELETE CASCADE,
    slug VARCHAR(100) NOT NULL,
    name VARCHAR(255) NOT NULL,
    description TEXT,
    category VARCHAR(50) NOT NULL CHECK (category IN ('greeting', 'closing', 'instruction', 'error')),
    content JSONB NOT NULL DEFAULT '{}',  -- Language -> text mapping
    variables JSONB NOT NULL DEFAULT '[]', -- Variable definitions
    metadata JSONB NOT NULL DEFAULT '{}',  -- interruptible, voice_speed, etc.
    is_system BOOLEAN NOT NULL DEFAULT FALSE,
    created_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT now(),
    updated_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT now(),
    
    CONSTRAINT unique_tenant_slug UNIQUE (tenant_id, slug)
);

-- Index for fast lookups
CREATE INDEX idx_prompt_templates_tenant_slug ON prompt_templates(tenant_id, slug);
CREATE INDEX idx_prompt_templates_category ON prompt_templates(tenant_id, category);

-- Platform defaults have NULL tenant_id
CREATE INDEX idx_prompt_templates_system ON prompt_templates(is_system) WHERE is_system = TRUE;

Implementation Plan

Phase 1: Core Infrastructure (Week 1)

  1. Database Model

    • Add prompt_templates table
    • Create PromptTemplateRepository with tenant scoping
    • Add migration

  2. API Layer

    • CRUD endpoints for prompt templates
    • Pydantic schemas for request/response
    • Validation for template syntax

  3. Seed Data

    • Create platform default templates
    • Migration to seed welcome/returning greetings in 10 languages

Phase 2: Worker Integration (Week 2)

  1. Template Resolution

    • Update CallContext to support {{prompt.slug}} syntax
    • Add PromptTemplateResolver class
    • Cache templates per-session

  2. Greeting Logic

    • Update _speak_greeting() to use prompt library
    • Handle new vs returning user template selection
    • Apply metadata (interruptible, voice_speed)

  3. Fallback Handling

    • Platform default fallback
    • Missing language fallback to English
    • Error handling for missing templates

Phase 3: Dashboard UI (Week 3)

  1. Prompt Library Page

    • List view with categories
    • Search and filter
    • Create/Edit/Delete actions

  2. Template Editor

    • Multi-language content editor
    • Variable definition UI
    • Metadata configuration
    • Live preview with sample context

  3. Agent Integration

    • Template selector in agent prompt config
    • Show available variables from selected template

Phase 4: Migration & Testing (Week 4)

  1. Migrate Existing Content

    • Import welcome scripts from voice-agent
    • Create templates for all 10 languages
    • Verify variable mapping

  2. Update Backend

    • Remove greeting generation from notification backend
    • Send only raw context data
    • Update AgentMetadata struct

  3. Testing

    • Unit tests for template resolution
    • Integration tests for greeting flow
    • E2E tests with real calls

Consequences

Positive

  1. Dashboard-managed content - Edit greetings without code deployment
  2. Per-tenant customization - Each tenant can have unique greetings
  3. Multi-language support - All variants in one place
  4. Separation of concerns - Backend sends context, agent-studio owns content
  5. A/B testing ready - Easy to create variant templates
  6. Version history - Track changes via updated_at, can add versioning later

Negative

  1. Initial development effort - New feature to build
  2. Migration work - Move scripts from backend to templates
  3. Learning curve - Users need to understand template syntax

Neutral

  1. Caching strategy needed - Templates should be cached in worker
  2. Template syntax limits - No complex conditionals (by design)

Migration from ADR-013

ADR-013's backend greeting generation becomes optional:

  1. Short-term: Backend can still send computed greeting field
  2. Long-term: Backend sends only raw context, worker uses prompt library
  3. Backward compatible: If user.greeting exists in context, use it; otherwise resolve from library

References

On this page

ADR-014: Prompt LibraryStatusContextThe OpportunityRequirementsDecisionData ModelTemplate CategoriesArchitectureConditional Template SelectionAPI EndpointsDashboard UITemplate EditorPlatform Default TemplatesWorker IntegrationDatabase SchemaImplementation PlanPhase 1: Core Infrastructure (Week 1)Phase 2: Worker Integration (Week 2)Phase 3: Dashboard UI (Week 3)Phase 4: Migration & Testing (Week 4)ConsequencesPositiveNegativeNeutralMigration from ADR-013References