Agent Studio
Adr

ADR-013: Dynamic Prompt Generation

Architecture for context-aware prompt generation in multi-tenant voice agents

ADR-013: Dynamic Prompt Generation

Status

Accepted

Context

Voice AI agents need prompts that adapt to:

  1. User Context - Name, language, preferences, current state
  2. Temporal Context - Time of day, pending tasks, history
  3. Business Logic - Meals logged, goals achieved, conditions met
  4. Language - Multilingual support with native greetings/closings

The original voice-agent implementation used Python functions (get_meal_prompt(), COMMON_INSTRUCTIONS_AGENT) to dynamically construct prompts based on runtime state. Migrating to Agent Studio's multi-tenant architecture requires maintaining this flexibility while supporting:

  • Multiple tenants with different prompt needs
  • Dashboard-based prompt editing
  • No code changes for prompt updates
  • Multi-language support per tenant

Approaches Considered

Option 1: Code-Based Prompt Generation (Current voice-agent approach)

def get_meal_prompt(pending_meals, logged_meals, user_name, language):
    return f"""
    You are helping {user_name} log meals.
    Pending: {', '.join(pending_meals)}
    Language: {LANGUAGE_NAMES[language]}
    """

Pros:

  • Full programming flexibility
  • Type safety
  • Easy to test

Cons:

  • Requires code deployment for changes
  • Not editable via dashboard
  • Per-tenant customization requires code branches

Option 2: Template-Based Resolution (Agent Studio approach)

{
  "system": "You are helping {{user.name}} log meals.\nPending: {{user.pending_meals_display}}\nLanguage: {{user.language_name}}"
}

Pros:

  • Editable via dashboard
  • No code deployment for changes
  • Multi-tenant customization built-in

Cons:

  • Limited to variable substitution
  • Complex logic requires backend pre-computation
  • No conditionals in templates

Combine both approaches:

  • Backend computes complex/dynamic values
  • Templates reference computed values
  • Dashboard edits template structure

Decision

Adopt the Hybrid Approach where:

  1. Backend pre-computes dynamic values when dispatching calls
  2. Agent prompts use template syntax ({{path}}) for variable injection
  3. CallContext resolves templates at runtime
  4. Dashboard allows prompt editing without code changes

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                    Backend System                                │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  User DB     Business Logic       Call Dispatch                 │
│    │              │                    │                        │
│    └──────────────┼────────────────────┘                        │
│                   ▼                                              │
│     ┌─────────────────────────────┐                             │
│     │   Compute user_context      │                             │
│     │   - name, language          │                             │
│     │   - pending_meals_display   │                             │
│     │   - greeting (pre-computed) │                             │
│     │   - closing_message         │                             │
│     └─────────────────────────────┘                             │
│                   │                                              │
└───────────────────┼──────────────────────────────────────────────┘
                    ▼
┌─────────────────────────────────────────────────────────────────┐
│                    Agent Studio API                              │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│   POST /api/v1/calls                                            │
│   {                                                              │
│     "workflow_slug": "meal-logging",                            │
│     "user_context": {                                           │
│       "name": "Rahul",                                          │
│       "language": "hi",                                         │
│       "language_name": "Hindi",                                 │
│       "pending_meals_display": "Breakfast, Lunch",              │
│       "greeting": "नमस्ते Rahul!"                               │
│     }                                                           │
│   }                                                              │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘
                    │
                    ▼
┌─────────────────────────────────────────────────────────────────┐
│                    Voice Worker                                  │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│   Agent Config (from Dashboard):                                │
│   {                                                              │
│     "prompt": {                                                  │
│       "system": "Speak in {{user.language_name}}...",           │
│       "greeting": "{{user.greeting}}"                           │
│     }                                                           │
│   }                                                              │
│                   │                                              │
│                   ▼                                              │
│   ┌─────────────────────────────┐                               │
│   │   CallContext.resolve()     │                               │
│   │   Template → Final Prompt   │                               │
│   └─────────────────────────────┘                               │
│                   │                                              │
│                   ▼                                              │
│   Final: "Speak in Hindi..."                                    │
│   Greeting: "नमस्ते Rahul!"                                     │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘

Implementation Details

1. Backend Computes Complex Values

# Backend application (not Agent Studio)
def get_user_call_context(user_id: str) -> dict:
    user = get_user(user_id)
    language = user.preferred_language or "hi"
    pending_meals = get_pending_meals(user_id)
    logged_meals = get_logged_meals(user_id)
    
    return {
        # Static user data
        "name": user.name,
        "language": language,
        "language_name": LANGUAGE_NAMES[language],
        
        # Pre-computed display strings
        "pending_meals_display": format_meals(pending_meals, language),
        "logged_meals_display": format_meals(logged_meals, language),
        
        # Pre-computed language-specific messages
        "greeting": generate_greeting(language, user.name),
        "closing_message": get_closing_message(language),
        
        # Business state
        "pending_meals": pending_meals,
        "logged_meals": logged_meals,
        "all_meals_logged": len(pending_meals) == 0
    }

2. Dashboard-Editable Templates

{
  "prompt": {
    "system": "You are Tap Health Coach.\n\nLANGUAGE: Speak ONLY in {{user.language_name}}.\n\nCONTEXT:\n- User: {{user.name}}\n- Pending: {{user.pending_meals_display}}\n- Logged: {{user.logged_meals_display}}",
    "greeting": "{{user.greeting}}",
    "greeting_interruptible": false,
    "variables": [
      { "name": "user.name", "default": "there" },
      { "name": "user.language_name", "default": "Hindi" }
    ]
  }
}

3. CallContext Resolution

# Worker resolves templates at runtime
def resolve_prompt(agent_config: AgentConfig, context: CallContext) -> str:
    system = agent_config.prompt.system
    
    # {{user.name}} → "Rahul"
    # {{user.pending_meals_display}} → "Breakfast, Lunch"
    return context.resolve(system)

Dynamic Greeting Based on User Type

The greeting and its interruptibility are dynamically determined by the backend based on whether the user is new or returning:

New Users - Full Onboarding (Non-interruptible)

New users receive a comprehensive welcome that explains the service. This greeting is non-interruptible to ensure they hear the full introduction.

# Backend determines user type and provides appropriate greeting
def get_user_call_context(user_id: str) -> dict:
    user = get_user(user_id)
    is_new_user = user.call_count == 0
    
    if is_new_user:
        greeting = get_welcome_script(user.name, user.language)
        greeting_interruptible = False  # Full onboarding, don't interrupt
    else:
        greeting = get_returning_greeting(user.name, user.language, pending_meals[0])
        greeting_interruptible = True   # Quick greeting, can interrupt
    
    return {
        "name": user.name,
        "is_new_user": is_new_user,
        "greeting": greeting,
        "greeting_interruptible": greeting_interruptible,
        # ... other fields
    }

Returning Users - Quick Greeting (Interruptible)

Returning users get a short greeting and can interrupt immediately to start logging:

# Returning user greeting (short, interruptible)
RETURNING_GREETINGS = {
    "hi": "Hi {name}! Aaj {meal} mein kya khaya?",
    "en": "Hi {name}! What did you have for {meal} today?",
    "ta": "வணக்கம் {name}! இன்று {meal} என்ன சாப்பிட்டீர்கள்?",
}

Agent Config Uses Dynamic Values

The agent config references the backend-provided values:

{
  "prompt": {
    "greeting": "{{user.greeting}}",
    "greeting_interruptible": "{{user.greeting_interruptible}}"
  }
}

Worker Implementation

async def _speak_greeting(self, agent_config: AgentConfig, context: CallContext):
    greeting = context.resolve(agent_config.prompt.greeting)
    
    # Get interruptibility from context (backend decides)
    interruptible = context.get("user.greeting_interruptible", True)
    
    await self._agent_session.say(
        greeting,
        allow_interruptions=interruptible
    )

This approach keeps the business logic (new vs returning user) in the backend while Agent Studio handles the voice delivery.

Multi-Tenant Support

Each tenant can have:

  • Different prompt templates
  • Different supported languages
  • Different voice mappings
  • Different greeting/closing messages

Tenant isolation is maintained:

  • user_context scoped to tenant's call
  • Templates stored per-tenant agent config
  • No cross-tenant data leakage

Consequences

Positive

  1. Dashboard-editable prompts - Non-technical users can modify prompts
  2. No code deployment for prompt changes - Faster iteration
  3. Multi-tenant support - Each tenant customizes independently
  4. Language flexibility - Backend handles localization complexity
  5. Separation of concerns - Business logic in backend, templates in Agent Studio

Negative

  1. Backend complexity - Business systems must compute context
  2. No template conditionals - Complex branching needs backend logic
  3. Learning curve - Template syntax needs documentation

Neutral

  1. Testing approach changes - Test context computation separately from templates
  2. Debugging - Need to inspect both computed context and template resolution

Migration Path

For existing voice-agent implementations:

  1. Extract dynamic values from get_meal_prompt() into user_context
  2. Convert prompts to template format with {{path}} variables
  3. Create agent config in Dashboard with templates
  4. Update backend to dispatch calls with computed context

Example migration:

# Before (voice-agent)
prompt = get_meal_prompt(
    pending_meals=["Breakfast", "Lunch"],
    logged_meals=["Dinner"],
    user_name="Rahul",
    language="hi"
)

# After (Agent Studio)
# Backend computes:
user_context = {
    "name": "Rahul",
    "language": "hi",
    "language_name": "Hindi",
    "pending_meals_display": "Breakfast, Lunch",
    "logged_meals_display": "Dinner"
}

# Dashboard template:
# "Pending: {{user.pending_meals_display}}\nLogged: {{user.logged_meals_display}}"

References

On this page

ADR-013: Dynamic Prompt GenerationStatusContextApproaches ConsideredOption 1: Code-Based Prompt Generation (Current voice-agent approach)Option 2: Template-Based Resolution (Agent Studio approach)Option 3: Hybrid Approach (Recommended)DecisionArchitectureImplementation Details1. Backend Computes Complex Values2. Dashboard-Editable Templates3. CallContext ResolutionDynamic Greeting Based on User TypeNew Users - Full Onboarding (Non-interruptible)Returning Users - Quick Greeting (Interruptible)Agent Config Uses Dynamic ValuesWorker ImplementationMulti-Tenant SupportConsequencesPositiveNegativeNeutralMigration PathReferences