Guide to testing Agent Studio - unit tests, integration tests, and test infrastructure

Testing

Agent Studio uses a pragmatic testing approach focused on high-value tests that catch real bugs without over-engineering.

Test Structure

tests/
├── conftest.py              # Shared fixtures (DB, factories, mocks)
├── unit/                    # Fast, isolated unit tests
│   ├── api/
│   │   └── test_auth.py
│   └── core/
│       ├── test_executor.py
│       ├── test_context.py
│       └── test_models.py
├── integration/             # Tests with real database
│   ├── test_api_health.py
│   └── test_calls.py
└── e2e/                     # End-to-end tests (future)
    └── __init__.py

Running Tests

All Tests

# Run all tests
pytest

# With verbose output
pytest -v

# With coverage
pytest --cov=agent_studio --cov-report=html

Specific Test Categories

# Unit tests only (fast)
pytest tests/unit -v

# Integration tests (requires Docker)
pytest tests/integration -v

# Single test file
pytest tests/integration/test_calls.py -v

# Single test
pytest tests/integration/test_calls.py::TestCallRepository::test_create_call -v

Test Infrastructure

Database Fixtures

Integration tests use PostgreSQL via testcontainers. When you run tests, a PostgreSQL container is automatically started.

Requirements:

Docker must be running
First run may take longer (pulls postgres:15-alpine image)

How it works:

# tests/conftest.py

@pytest.fixture(scope="session")
def postgres_container():
    """Start PostgreSQL container for the test session."""
    with PostgresContainer("postgres:15-alpine") as postgres:
        yield postgres

@pytest_asyncio.fixture
async def async_engine(postgres_container):
    """Create async PostgreSQL engine for testing."""
    database_url = (
        f"postgresql+asyncpg://{postgres_container.username}:"
        f"{postgres_container.password}@{postgres_container.get_container_host_ip()}:"
        f"{postgres_container.get_exposed_port(5432)}/{postgres_container.dbname}"
    )
    # ... create engine, tables

Using External Database

For CI or when you have a dedicated test database:

# Use external database instead of testcontainers
export TEST_DATABASE_URL="postgresql+asyncpg://user:pass@localhost:5432/test_db"
pytest tests/integration -v

Available Fixtures

Database Fixtures

Fixture	Scope	Description
`postgres_container`	session	PostgreSQL testcontainer instance
`async_engine`	function	SQLAlchemy async engine with tables created
`db_session`	function	Async database session (rolled back after test)

Factory Fixtures

Fixture	Description
`tenant_factory`	Create test tenants
`agent_factory`	Create test agents

Context Fixtures

Fixture	Description
`call_context`	Fresh CallContext instance
`call_context_with_data`	CallContext with pre-populated data

Config Fixtures

Fixture	Description
`sample_tool_config`	Example tool configuration
`sample_agent_config`	Example agent configuration
`sample_workflow_config`	Multi-agent workflow config
`simple_workflow_config`	Single-agent workflow config

Mock Fixtures

Fixture	Description
`mock_webhook_client`	Mock HTTP client that tracks requests
`mock_endpoint_resolver`	Mock endpoint name resolver

Writing Tests

Unit Tests

Unit tests should be fast, isolated, and test single functions/classes:

# tests/unit/core/test_context.py

import pytest
from agent_studio.core.context import CallContext

class TestCallContext:
    def test_get_nested_value(self, call_context):
        """Test getting nested values from context."""
        call_context.set("user.profile.age", 30)
        assert call_context.get("user.profile.age") == 30

    def test_get_missing_returns_default(self, call_context):
        """Test default value for missing keys."""
        assert call_context.get("missing.key", "default") == "default"

Integration Tests

Integration tests use the real database to test component interactions:

# tests/integration/test_calls.py

import pytest
from agent_studio.db.models.call import Call, CallStatus
from agent_studio.db.repositories.call import CallRepository

class TestCallRepository:
    @pytest.mark.asyncio
    async def test_create_call(
        self,
        db_session,
        call_repo,
        workflow,
    ):
        """Test creating a call."""
        call = Call(
            tenant_id=call_repo.tenant_id,
            workflow_id=workflow.id,
            room_name="room-123",
            user_id="user-456",
            status=CallStatus.PENDING,
        )
        db_session.add(call)
        await db_session.flush()

        result = await call_repo.get(call.id)
        assert result is not None
        assert result.status == CallStatus.PENDING

Testing Async Code

Use pytest-asyncio for async tests:

import pytest

class TestAsyncOperations:
    @pytest.mark.asyncio
    async def test_async_operation(self, db_session):
        """Test async database operation."""
        # Your async test code
        result = await some_async_function()
        assert result is not None

Test Coverage

Current Coverage Areas

Area	Coverage	Tests
Call Repository	High	CRUD, queries, filters
Call Model	High	Properties, helpers, lifecycle
Tool Executor	Medium	Context operations, actions
API Auth	Medium	Token validation

Running Coverage Report

# Generate HTML coverage report
pytest --cov=agent_studio --cov-report=html

# Open report
open htmlcov/index.html

CI/CD Integration

GitHub Actions Example

# .github/workflows/test.yml
name: Tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest

    services:
      postgres:
        image: postgres:15-alpine
        env:
          POSTGRES_USER: postgres
          POSTGRES_PASSWORD: postgres
          POSTGRES_DB: test_db
        ports:
          - 5432:5432
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5

    steps:
      - uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.11"

      - name: Install uv
        uses: astral-sh/setup-uv@v4

      - name: Install dependencies
        run: uv sync --dev

      - name: Run tests
        env:
          TEST_DATABASE_URL: postgresql+asyncpg://postgres:postgres@localhost:5432/test_db
        run: uv run pytest -v --cov=agent_studio

Best Practices

Do

Use fixtures for common setup
Test one thing per test
Use descriptive test names
Clean up after tests (fixtures handle this)
Test edge cases and error conditions

Don't

Mock everything (use real DB for integration tests)
Write tests that depend on test order
Test implementation details
Skip writing tests for bug fixes

Test Naming Convention

def test_<what>_<condition>_<expected>():
    """Test description."""
    pass

# Examples:
def test_create_call_with_valid_data_succeeds():
def test_get_call_with_invalid_id_returns_none():
def test_update_status_to_completed_sets_ended_at():

Troubleshooting

Docker Not Running

ERROR: Cannot connect to Docker daemon

Solution: Start Docker Desktop or Docker daemon.

Port Already in Use

ERROR: Port 5432 is already in use

Solution: Use TEST_DATABASE_URL to point to your existing database, or stop the conflicting service.

Slow First Run

First run downloads the PostgreSQL Docker image (~80MB). Subsequent runs are faster as the image is cached.

Database State Issues

Each test uses a transaction that's rolled back, but if you see stale data:

# Force recreate tables
pytest tests/integration/test_calls.py -v --tb=short

Testing

On this page