Supervisor Configuration Guide

Overview

The Lobster supervisor agent now features a dynamic, configuration-driven prompt system that automatically discovers agents and their capabilities, eliminating the need for manual updates when adding new agents.

Key Features

• Automatic Agent Discovery: Supervisor automatically knows about all registered agents
• Configurable Behavior: Customize interaction style via environment variables
• Dynamic Prompt Generation: Prompt adapts based on active agents and configuration
• Backward Compatible: Default settings maintain existing behavior
• Multiple Operation Modes: Research, Production, and Development modes

Configuration Options

Environment Variables

Configuration Modes

Research Mode (Interactive)

Best for exploratory analysis and development.

export SUPERVISOR_ASK_QUESTIONS=true
export SUPERVISOR_WORKFLOW_GUIDANCE=detailed
export SUPERVISOR_REQUIRE_CONFIRMATION=true

Characteristics:

• Asks clarification questions (up to 3)
• Provides detailed workflow guidance
• Requires explicit confirmation for downloads
• Includes examples and detailed instructions

Production Mode (Automated)

Optimized for automated workflows and pipelines.

export SUPERVISOR_ASK_QUESTIONS=false
export SUPERVISOR_WORKFLOW_GUIDANCE=minimal
export SUPERVISOR_REQUIRE_CONFIRMATION=false

Characteristics:

• No clarification questions
• Minimal workflow guidance
• Proceeds with downloads automatically
• Streamlined prompt for efficiency

Development Mode (Verbose)

Useful for debugging and understanding system behavior.

export SUPERVISOR_VERBOSE=true
export SUPERVISOR_WORKFLOW_GUIDANCE=detailed
export SUPERVISOR_INCLUDE_SYSTEM=true

Characteristics:

• Detailed delegation explanations
• Full workflow documentation
• System information included
• Maximum transparency

How It Works

1. Agent Discovery (v1.0.0+)

The system automatically discovers all registered agents via Python entry points:

# Each agent module defines AGENT_CONFIG at module top
from lobster.core.registry import AgentRegistryConfig

AGENT_CONFIG = AgentRegistryConfig(
    name='new_agent',
    display_name='New Agent',
    description='Agent purpose',
    factory_function='your_package.agents.new_agent:new_agent',
    tier_requirement='free'
)

2. Dynamic Prompt Building

The supervisor prompt is built dynamically based on:

• Active agents discovered via ComponentRegistry
• Configuration settings
• Current data context
• Workspace status

3. Capability Extraction

The system can optionally extract and display agent capabilities:

• Discovers @tool decorated functions
• Parses docstrings for descriptions
• Lists available tools per agent

Adding New Agents

To add a new agent to the system (v1.0.0+):

1. Create the agent module with AGENT_CONFIG at module top
2. Register via entry points in pyproject.toml:

   [project.entry-points."lobster.agents"]
   new_agent = "your_package.agents.new_agent:AGENT_CONFIG"

3. That's it! The supervisor automatically discovers and includes the new agent

Programmatic Usage

Python API

from lobster.config.supervisor_config import SupervisorConfig
from lobster.agents.supervisor import create_supervisor_prompt
from lobster.core.data_manager_v2 import DataManagerV2

# Create custom configuration
config = SupervisorConfig(
    ask_clarification_questions=False,
    workflow_guidance_level='minimal',
    verbose_delegation=True
)

# Generate prompt
data_manager = DataManagerV2()
prompt = create_supervisor_prompt(
    data_manager=data_manager,
    config=config,
    active_agents=['data_expert_agent', 'transcriptomics_expert']
)

Integration with Graph

from lobster.agents.graph import create_bioinformatics_graph
from lobster.config.supervisor_config import SupervisorConfig

# Create custom supervisor configuration
supervisor_config = SupervisorConfig(
    ask_clarification_questions=True,
    max_clarification_questions=5
)

# Create graph with custom config
graph = create_bioinformatics_graph(
    data_manager=data_manager,
    supervisor_config=supervisor_config
)

Configuration Validation

The system validates configuration values:

• workflow_guidance_level: Must be 'minimal', 'standard', or 'detailed'
• delegation_strategy: Must be 'auto', 'conservative', or 'aggressive'
• error_handling: Must be 'silent', 'informative', or 'verbose'
• max_clarification_questions: Clamped between 0 and 10
• max_tools_per_agent: Clamped between 0 and 20

Invalid values automatically fall back to defaults with a warning.

Monitoring Configuration

Check Current Configuration

from lobster.config.settings import get_settings

settings = get_settings()
config = settings.get_supervisor_config()

print(f"Mode: {config.get_prompt_mode()}")
print(f"Ask questions: {config.ask_clarification_questions}")
print(f"Guidance level: {config.workflow_guidance_level}")

View Configuration as Dictionary

config_dict = config.to_dict()
print(config_dict)

Performance Considerations

• Prompt Size: Dynamic generation keeps prompt size manageable

  • Production mode: ~8,000 characters
  • Standard mode: ~9,500 characters
  • Detailed mode: ~11,000 characters

• Caching: Agent capabilities are cached using LRU cache

  • Clear cache: AgentCapabilityExtractor.clear_cache()

• Startup Time: Minimal impact (<100ms) for prompt generation

Troubleshooting

Agents Not Appearing in Prompt

1. Verify entry points are registered in pyproject.toml
2. Run lobster agents list to verify discovery
3. Check factory function path is correct
4. Ensure agent module can be imported

Configuration Not Taking Effect

1. Check environment variable names (must start with SUPERVISOR_)
2. Verify boolean values are 'true' or 'false' (lowercase)
3. Check for typos in variable names

Performance Issues

1. Use minimal workflow guidance for production
2. Disable agent tool discovery if not needed
3. Consider caching prompt for repeated use

Migration from Static Prompt

The system maintains backward compatibility:

• Default configuration matches previous behavior exactly
• create_supervisor_prompt(data_manager) works without changes
• Existing workflows continue functioning

To leverage new features, simply:

1. Set environment variables for desired behavior
2. Or pass a SupervisorConfig object to create_supervisor_prompt()

Future Enhancements

Planned improvements include:

• Per-agent configuration overrides
• Dynamic capability extraction from running agents
• Prompt templates for specific domains
• Configuration profiles (save/load configurations)
• Web UI for configuration management