Package Structure

Reference Document: This page provides detailed reference for package directory layout and PEP 420 namespace packages.

For step-by-step agent creation, see Creating Agent Packages.

This guide explains the standard directory layout and configuration for Lobster agent packages. Following this structure ensures your package integrates seamlessly with the Lobster ecosystem.

PEP 420 Namespace Packages

Lobster uses PEP 420 implicit namespace packages to allow multiple packages to contribute to the lobster namespace. This enables users to install combinations of packages that merge into a unified lobster.* import structure.

Critical Rule: Never create lobster/__init__.py or lobster/agents/__init__.py in your package. These directories must remain as implicit namespaces to allow package merging.

Standard Directory Layout

Every agent package follows this structure:

lobster-{domain}/
├── pyproject.toml              # Package metadata and entry points
├── README.md                   # PyPI description
├── lobster/                    # NO __init__.py here!
│   └── agents/                 # NO __init__.py here!
│       └── {domain}/           # Your domain namespace
│           ├── __init__.py     # AGENT_CONFIG defined here (before imports)
│           ├── {agent}_expert.py
│           ├── config.py       # Domain-specific configuration
│           ├── prompts.py      # Agent prompt templates
│           ├── state.py        # State class definitions
│           └── shared_tools.py # Tools shared across agents
└── tests/
    ├── __init__.py
    ├── conftest.py             # Test fixtures
    └── test_{agent}.py

Reference Implementation: lobster-transcriptomics

The lobster-transcriptomics package demonstrates the canonical structure:

lobster-transcriptomics/
├── pyproject.toml
├── README.md
├── lobster/
│   └── agents/
│       └── transcriptomics/
│           ├── __init__.py           # Exports AGENT_CONFIG
│           ├── transcriptomics_expert.py
│           ├── annotation_expert.py
│           ├── de_analysis_expert.py
│           ├── config.py
│           ├── prompts.py
│           ├── state.py
│           └── shared_tools.py
└── tests/

The `init.py` File

The __init__.py serves as the public API for your domain module. For simple packages with a single agent, it imports and re-exports:

# lobster/agents/my_domain/__init__.py
from lobster.agents.my_domain.my_agent import AGENT_CONFIG, my_agent

__all__ = ["AGENT_CONFIG", "my_agent"]

For complex packages with multiple agents (like lobster-transcriptomics), it exports more:

# lobster/agents/transcriptomics/__init__.py

from lobster.agents.transcriptomics.annotation_expert import AGENT_CONFIG as annotation_config, annotation_expert
from lobster.agents.transcriptomics.de_analysis_expert import AGENT_CONFIG as de_config, de_analysis_expert
from lobster.agents.transcriptomics.transcriptomics_expert import AGENT_CONFIG, transcriptomics_expert
from lobster.agents.transcriptomics.config import detect_data_type, get_qc_defaults
from lobster.agents.transcriptomics.prompts import (
    create_annotation_expert_prompt,
    create_de_analysis_expert_prompt,
    create_transcriptomics_expert_prompt,
)
from lobster.agents.transcriptomics.shared_tools import create_shared_tools
from lobster.agents.transcriptomics.state import (
    AnnotationExpertState,
    DEAnalysisExpertState,
    TranscriptomicsExpertState,
)

__all__ = [
    # Main agents
    "transcriptomics_expert",
    "annotation_expert",
    "de_analysis_expert",
    # Configuration
    "detect_data_type",
    "get_qc_defaults",
    # Prompts
    "create_transcriptomics_expert_prompt",
    "create_annotation_expert_prompt",
    "create_de_analysis_expert_prompt",
    # Shared tools
    "create_shared_tools",
    # State classes
    "TranscriptomicsExpertState",
    "AnnotationExpertState",
    "DEAnalysisExpertState",
]

Important: AGENT_CONFIG is defined in each agent file (e.g., transcriptomics_expert.py), not in __init__.py.

AGENT_CONFIG Placement Rule

Performance Critical: AGENT_CONFIG must be defined at the TOP of your agent file, BEFORE any heavy imports. This enables fast entry point discovery (<50ms target).

The correct pattern (define in agent file, not __init__.py):

# lobster/agents/transcriptomics/transcriptomics_expert.py

"""
Transcriptomics Expert Agent for single-cell and bulk RNA-seq analysis.
"""

# =============================================================================
# AGENT_CONFIG FIRST (before heavy imports)
# =============================================================================
from lobster.config.agent_registry import AgentRegistryConfig

AGENT_CONFIG = AgentRegistryConfig(
    name="transcriptomics_expert",
    display_name="Transcriptomics Expert",
    description="Unified expert for single-cell AND bulk RNA-seq analysis.",
    factory_function="lobster.agents.transcriptomics.transcriptomics_expert.transcriptomics_expert",
    handoff_tool_name="handoff_to_transcriptomics_expert",
    handoff_tool_description="Assign ALL transcriptomics analysis tasks",
    child_agents=["annotation_expert", "de_analysis_expert"],
    tier_requirement="free",
    package_name="lobster-transcriptomics",
)

# =============================================================================
# Heavy imports AFTER (these may take time to load)
# =============================================================================
from datetime import date
from pathlib import Path
from typing import List, Optional

from langchain_core.tools import tool
from langgraph.prebuilt import create_react_agent

from lobster.config.llm_factory import create_llm
from lobster.core.data_manager_v2 import DataManagerV2
# ... more imports

def transcriptomics_expert(...):
    """Factory function."""
    ...

Then the __init__.py imports and re-exports:

# lobster/agents/transcriptomics/__init__.py
from lobster.agents.transcriptomics.transcriptomics_expert import (
    AGENT_CONFIG,
    transcriptomics_expert,
)

__all__ = ["AGENT_CONFIG", "transcriptomics_expert"]

pyproject.toml Structure

The pyproject.toml defines package metadata and entry points:

[build-system]
requires = ["setuptools>=61.0", "wheel"]
build-backend = "setuptools.build_meta"

[project]
name = "lobster-transcriptomics"
version = "1.0.0"
description = "Transcriptomics agents for Lobster AI"
readme = "README.md"
license = {text = "MIT"}
authors = [
    {name = "Your Name", email = "you@example.com"}
]
keywords = ["bioinformatics", "single-cell", "RNA-seq"]
classifiers = [
    "Development Status :: 5 - Production/Stable",
    "Intended Audience :: Science/Research",
    "License :: OSI Approved :: MIT License",
    "Programming Language :: Python :: 3.11",
    "Programming Language :: Python :: 3.12",
    "Programming Language :: Python :: 3.13",
    "Programming Language :: Python :: 3.14",
    "Topic :: Scientific/Engineering :: Bio-Informatics",
]
requires-python = ">=3.11"
dependencies = [
    "lobster-ai~=1.0.0",  # Compatible version constraint
    "scanpy",
    "leidenalg",
]

[project.urls]
Homepage = "https://omics-os.com"
Documentation = "https://docs.omics-os.com"
Repository = "https://github.com/the-omics-os/lobster-ai"

# Entry points for agent discovery
[project.entry-points."lobster.agents"]
transcriptomics_expert = "lobster.agents.transcriptomics.transcriptomics_expert:AGENT_CONFIG"
annotation_expert = "lobster.agents.transcriptomics.annotation_expert:AGENT_CONFIG"
de_analysis_expert = "lobster.agents.transcriptomics.de_analysis_expert:AGENT_CONFIG"

# Entry points for state class discovery
[project.entry-points."lobster.states"]
TranscriptomicsExpertState = "lobster.agents.transcriptomics.state:TranscriptomicsExpertState"
AnnotationExpertState = "lobster.agents.transcriptomics.state:AnnotationExpertState"
DEAnalysisExpertState = "lobster.agents.transcriptomics.state:DEAnalysisExpertState"

# Setuptools configuration for namespace packages
[tool.setuptools]
packages.find = {where = ["."], include = ["lobster*"], namespaces = true}

[tool.setuptools.package-data]
"*" = ["py.typed"]

Entry Point Groups

Lobster uses three entry point groups:

Group	Purpose	Value Format
`lobster.agents`	Agent discovery	`module.path:AGENT_CONFIG`
`lobster.states`	State class discovery	`module.path:StateClass`
`lobster.services`	Service discovery	`module.path:ServiceClass`

See Entry Points for detailed registration documentation.

Services Colocation

Services that are specific to your domain should live within your package. Services that are shared across domains stay in lobster-ai core.

Guideline: If only your agent uses a service, include it in your package. If multiple agents need it, it belongs in core.

lobster-{domain}/
└── lobster/
    ├── agents/
    │   └── {domain}/
    │       └── ...
    └── services/           # Optional: domain-specific services
        └── {domain}/
            ├── __init__.py
            └── {service}_service.py

README.md for PyPI

Your README.md becomes the PyPI package description. Include:

What the package provides - which agents and capabilities
Installation - pip install lobster-{domain}
Quick example - show basic usage
Requirements - any domain-specific dependencies

# lobster-transcriptomics

Transcriptomics agents for Lobster AI - single-cell and bulk RNA-seq analysis.

## Installation

\`\`\`bash
pip install lobster-transcriptomics
\`\`\`

## Agents Included

- **transcriptomics_expert**: QC, clustering, marker genes
- **annotation_expert**: Cell type annotation
- **de_analysis_expert**: Differential expression analysis

## Requirements

- Python 3.11-3.14
- lobster-ai >= 1.0.0
- scanpy, leidenalg

Common Mistakes

Mistake 1: Creating lobster/__init__.py

This breaks PEP 420 namespace merging. Other packages cannot contribute to the lobster namespace.

Mistake 2: Placing AGENT_CONFIG after heavy imports

Entry point discovery loads your module to find AGENT_CONFIG. Heavy imports cause slow startup (>50ms).

Mistake 3: Using handoff_tools instead of delegation_tools

The parameter was renamed in Phase 2. Always use delegation_tools in factory signatures.

Mistake 4: Hardcoding version constraints

Use ~=1.0.0 (compatible release) instead of ==1.0.0 (exact). This allows patch updates.

NextPlugin Contract

Package Structure

Package Structure

PEP 420 Namespace Packages

Standard Directory Layout

Reference Implementation: lobster-transcriptomics

The `init.py` File

AGENT_CONFIG Placement Rule

pyproject.toml Structure

Entry Point Groups

Services Colocation

README.md for PyPI

Common Mistakes

What's Next?

Entry Points

Plugin Contract

Testing Agents

On this page