Entry Points

Reference Document: This page provides detailed reference for Python entry point registration.

For step-by-step agent creation, see Creating Agent Packages.

Lobster uses Python entry points for automatic plugin discovery. This guide explains how to register your agents, states, and services so they integrate seamlessly with the Lobster ecosystem.

How Entry Points Work

Python entry points (defined in pyproject.toml) allow packages to advertise components that other packages can discover. When Lobster starts, it scans all installed packages for entry points in specific groups.

# At runtime, Lobster discovers your agent automatically
from lobster.core.component_registry import component_registry

# All agents from all installed packages
all_agents = component_registry.list_agents()

# Get a specific agent
config = component_registry.get_agent("transcriptomics_expert")

Entry Point Groups

Lobster defines seven entry point groups organized into two categories: agent/service discovery and omics plugin architecture.

Agent and Service Groups

lobster.agents

[project.entry-points."lobster.agents"]
my_agent = "lobster.agents.my_domain.my_agent:AGENT_CONFIG"

The value must resolve to an AgentRegistryConfig instance defined at module level.

lobster.services

[project.entry-points."lobster.services"]
my_service = "my_package.services.my_service:MyService"

lobster.agent_configs

[project.entry-points."lobster.agent_configs"]
my_agent = "lobster.agents.my_domain.config:MY_AGENT_LLM_CONFIG"

Omics Plugin Groups

These four groups enable the modular omics plugin architecture, allowing new omics types and data sources to be added via external packages with zero changes to core.

lobster.adapters

[project.entry-points."lobster.adapters"]
metabolomics_lc_ms = "my_package.adapter:create_adapter"

Adapter entry points must be callables that return configured adapter instances. DataManagerV2 discovers adapters via this group before falling back to hardcoded adapters.

lobster.providers

[project.entry-points."lobster.providers"]
metabolights = "my_package.provider:MetaboLightsProvider"

Providers implement the database access interface. ContentAccessService discovers providers via this group before falling back to built-in providers.

lobster.download_services

[project.entry-points."lobster.download_services"]
metabolights_download = "my_package.download:MetaboLightsDownloadService"

Download services handle file retrieval from specific databases. DownloadOrchestrator discovers these before falling back to built-in services.

lobster.queue_preparers

[project.entry-points."lobster.queue_preparers"]
metabolights_preparer = "my_package.queue:MetaboLightsQueuePreparer"

Queue preparers map accession IDs (e.g., MTBLS*, ST*) to download queue entries. QueuePreparationService discovers these before falling back to built-in preparers.

lobster.omics_types

[project.entry-points."lobster.omics_types"]
metabolomics = "my_package.config:METABOLOMICS_CONFIG"

The value must resolve to an OmicsTypeConfig instance that defines detection keywords, feature count ranges, preferred databases, and QC thresholds. The OmicsTypeRegistry discovers these at startup.

ComponentRegistry Discovery

The ComponentRegistry singleton discovers and caches all entry points:

from lobster.core.component_registry import component_registry

# Trigger discovery (idempotent - safe to call multiple times)
component_registry.load_components()

# Query agents
if component_registry.has_agent("my_agent"):
    config = component_registry.get_agent("my_agent")
    print(f"Found: {config.display_name}")

# List all discovered agents
for name, config in component_registry.list_agents().items():
    print(f"  {name}: {config.description}")

# Get registry info for diagnostics
info = component_registry.get_info()
print(f"Total agents: {info['agents']['count']}")
print(f"Total services: {info['services']['count']}")

Discovery Flow

First access: When you call list_agents() or get_agent(), ComponentRegistry triggers discovery
Entry point scan: Python's importlib.metadata.entry_points() finds all lobster.* groups
Version check: External packages have their lobster-ai version constraint validated
Load and cache: Valid entry points are loaded and cached
Subsequent calls: Return cached results (no re-scanning)

Performance Considerations

Target: Entry point discovery should complete in <50ms. Heavy imports in your module will slow down discovery for all users.

The AGENT_CONFIG Rule

Entry point discovery imports your agent module to access AGENT_CONFIG. If your module has heavy imports at the top level, every user pays that cost at startup.

Slow pattern (avoid):

# lobster/agents/my_agent/my_agent.py - BAD: Heavy imports before AGENT_CONFIG
import scanpy as sc  # Takes ~2 seconds to import
import torch  # Takes ~1 second
from lobster.config.agent_registry import AgentRegistryConfig

AGENT_CONFIG = AgentRegistryConfig(...)  # Too late!

Fast pattern (recommended):

# lobster/agents/my_agent/my_agent.py - GOOD: AGENT_CONFIG first
from lobster.config.agent_registry import AgentRegistryConfig

AGENT_CONFIG = AgentRegistryConfig(
    name="my_agent",
    factory_function="lobster.agents.my_agent.my_agent.my_agent",
    # ...
)  # ~5ms to access

# Heavy imports below - only loaded when factory is called
# These don't affect discovery time
from pathlib import Path
import scanpy as sc

The __init__.py file imports and re-exports from the agent file:

# lobster/agents/my_agent/__init__.py - Exports only
from lobster.agents.my_agent.my_agent import AGENT_CONFIG, my_agent

__all__ = ["AGENT_CONFIG", "my_agent"]

Measuring Discovery Time

import time
from lobster.core.component_registry import component_registry

# Reset for accurate measurement
component_registry.reset()

start = time.perf_counter()
agents = component_registry.list_agents()
elapsed = (time.perf_counter() - start) * 1000

print(f"Discovery took {elapsed:.1f}ms for {len(agents)} agents")
# Target: <50ms

Multi-Agent Packages

A single package can register multiple agents. Each agent needs its own entry point:

[project.entry-points."lobster.agents"]
transcriptomics_expert = "lobster.agents.transcriptomics.transcriptomics_expert:AGENT_CONFIG"
annotation_expert = "lobster.agents.transcriptomics.annotation_expert:AGENT_CONFIG"
de_analysis_expert = "lobster.agents.transcriptomics.de_analysis_expert:AGENT_CONFIG"

[project.entry-points."lobster.states"]
TranscriptomicsExpertState = "lobster.agents.transcriptomics.state:TranscriptomicsExpertState"
AnnotationExpertState = "lobster.agents.transcriptomics.state:AnnotationExpertState"
DEAnalysisExpertState = "lobster.agents.transcriptomics.state:DEAnalysisExpertState"

Parent-Child Agent Relationships

Use child_agents in the parent's AGENT_CONFIG to define delegation:

# Parent agent
AGENT_CONFIG = AgentRegistryConfig(
    name="transcriptomics_expert",
    child_agents=["annotation_expert", "de_analysis_expert"],
    # ... other fields
)

Child agents should set supervisor_accessible=False to hide from direct supervisor routing:

# Child agent (accessed via parent only)
AGENT_CONFIG = AgentRegistryConfig(
    name="annotation_expert",
    supervisor_accessible=False,  # Hidden from supervisor
    # ... other fields
)

Verifying Registration

After installing your package, verify registration using the CLI:

# List all discovered agents
lobster agents list

# Get detailed info about your agent
lobster agents info my_agent

# Check if agent is available
lobster agents info my_agent --json

Programmatic Verification

from lobster.core.component_registry import component_registry

# Reset to force re-discovery (useful after pip install)
component_registry.reset()

# Check your agent is registered
if component_registry.has_agent("my_agent"):
    config = component_registry.get_agent("my_agent")
    print(f"Success! Agent '{config.display_name}' registered")
    print(f"  - Tier: {config.tier_requirement}")
    print(f"  - Factory: {config.factory_function}")
else:
    print("Error: Agent not found!")
    print("Check:")
    print("  1. Package is installed (pip install -e .)")
    print("  2. Entry points are in pyproject.toml")
    print("  3. AGENT_CONFIG is defined at module top")

Troubleshooting

Agent Not Discovered

Symptoms: lobster agents list doesn't show your agent

Common causes:

Package not installed: Run pip install -e . in your package directory
Entry point syntax error: Check pyproject.toml for typos
Import error in module: Test with python -c "from your.module import AGENT_CONFIG"
Missing AGENT_CONFIG: Ensure it's defined at module level

Debug steps:

# Check if entry points are registered
python -c "from importlib.metadata import entry_points; eps = entry_points(group='lobster.agents'); print([e.name for e in eps])"

# Test module import
python -c "from lobster.agents.my_domain.my_agent import AGENT_CONFIG; print(AGENT_CONFIG.name)"

Version Incompatibility Warning

Symptoms: Log shows "Skipping incompatible plugin"

Cause: Your package's lobster-ai version constraint doesn't match installed version

Fix: Update your pyproject.toml:

dependencies = [
    "lobster-ai~=1.0.0",  # Compatible with 1.0.x
]

Slow Discovery Time

Symptoms: lobster chat or lobster query takes >1 second to start

Cause: Heavy imports before AGENT_CONFIG

Fix: Move AGENT_CONFIG to top of module, before any heavy imports (see Package Structure)

Duplicate Agent Name

Symptoms: ComponentConflictError on startup

Cause: Two packages define agents with the same name

Fix: Use unique, descriptive names prefixed with your domain:

AGENT_CONFIG = AgentRegistryConfig(
    name="mycompany_genomics_expert",  # Prefixed to avoid conflicts
    # ...
)

Adding a New Omics Type via Plugins

The omics plugin architecture allows you to add support for an entirely new omics type (e.g., lipidomics, glycomics) without modifying Lobster core. Here is the complete process:

Step 1: Define OmicsTypeConfig

Create an OmicsTypeConfig with detection keywords, feature count ranges, and preferred databases:

from lobster.core.omics_registry import OmicsTypeConfig, OmicsDetectionConfig

MY_OMICS_CONFIG = OmicsTypeConfig(
    name="lipidomics",
    display_name="Lipidomics",
    schema_class="my_package.schema:LipidomicsSchema",
    adapter_names=["lipidomics_lc_ms"],
    preferred_databases=["metabolights", "metabolomics_workbench", "geo"],
    detection=OmicsDetectionConfig(
        keywords=["lipid", "fatty acid", "sphingolipid", "phospholipid"],
        platform_patterns=["LC-MS/MS", "shotgun lipidomics"],
        feature_range=(50, 3000),
        weight=11,
    ),
    qc_thresholds={"min_lipids": 50, "max_cv": 0.3},
    data_type_aliases=["lipid", "lipidomic"],
)

Step 2: Create adapter factory

def create_lipidomics_adapter(**kwargs):
    """Factory function returning a configured adapter instance."""
    return LipidomicsAdapter(**kwargs)

Step 3: Create provider class

class LipidMapsProvider:
    """Database provider for LIPID MAPS."""
    def search(self, query, **kwargs):
        ...

Step 4: Create download service

class LipidMapsDownloadService:
    """Download service for LIPID MAPS datasets."""
    def download(self, accession, **kwargs):
        ...

Step 5: Create queue preparer

class LipidMapsQueuePreparer:
    """Maps accession IDs to download queue entries."""
    def prepare(self, accession, **kwargs):
        ...

Step 6: Register all entry points

Add all entry points to your pyproject.toml:

[project.entry-points."lobster.omics_types"]
lipidomics = "my_package.config:MY_OMICS_CONFIG"

[project.entry-points."lobster.adapters"]
lipidomics_lc_ms = "my_package.adapter:create_lipidomics_adapter"

[project.entry-points."lobster.providers"]
lipid_maps = "my_package.provider:LipidMapsProvider"

[project.entry-points."lobster.download_services"]
lipid_maps_download = "my_package.download:LipidMapsDownloadService"

[project.entry-points."lobster.queue_preparers"]
lipid_maps_preparer = "my_package.queue:LipidMapsQueuePreparer"

After installing your package (pip install -e .), Lobster automatically discovers and integrates the new omics type. The research agent's routing table updates dynamically, the DataTypeDetector recognizes the new data type, and the download orchestrator routes to your service.

Best Practices

Unique names: Prefix agent names with your domain/company
AGENT_CONFIG first: Always define before heavy imports
Compatible versions: Use ~= for version constraints
Test discovery: Verify with lobster agents list after install
Minimal top-level imports: Only import AgentRegistryConfig before AGENT_CONFIG

Next Steps

Review Package Structure for directory layout
Follow the Plugin Contract for API compliance
Set up Testing with AgentContractTestMixin

PreviousTesting Guide - Lobster AI Testing Framework

NextExtending Lobster

Entry Points

On this page