Skip to content

Meta-Repository Pattern

Deep dive into the meta-repository pattern and how provide-workenv implements it.

What is a Meta-Repository?

A meta-repository is a coordination repository that manages multiple independent repositories as a unified workspace, without merging them into a single repository (monorepo).

Think of it as: - Not a monorepo: Code stays in separate repositories - Not individual repos: Coordinated setup and development - A workspace manager: Orchestrates multiple repos for unified development

How provide-workenv Implements It

1. Sibling Directory Structure

All repositories are cloned as siblings:

parent-directory/
├── provide-workenv/      # Meta-repository (coordinator)
├── provide-foundation/   # Independent repository
├── pyvider/              # Independent repository
└── ... (13+ repos)

Why siblings? - Clean separation between workspace and packages - Standard relative paths (../provide-foundation) - Easy to work with individual repos when needed - No nested .git directories conflicts

2. Unified Virtual Environment

One .venv in the workspace root:

provide-workenv/.venv/
├── bin/python3          # Python interpreter
├── lib/...              # Installed packages
└── pyvreate-workspace

All packages installed with -e (editable mode): 

pip install -e ../provide-foundation
pip install -e ../pyvider
# ... etc

This makes changes immediately available without reinstall.

3. Configuration-Driven Setup

wrknv.toml defines the workspace:

[[siblings]]
name = "provide-foundation"
path = "../provide-foundation"
repo = "https://github.com/provide-io/provide-foundation.git"

[[siblings]]
name = "pyvider"
path = "../pyvider"
repo = "https://github.com/provide-io/pyvider.git"

The bootstrap script reads this configuration and clones/verifies all repositories.

4. Automated Coordination

Three scripts coordinate the workspace:

  • bootstrap.sh: Clones all repositories based on configuration
  • setup.sh: Installs all packages in correct dependency order
  • validate.sh: Verifies environment is correctly configured

Benefits Over Alternatives

vs. Monorepo

Aspect Meta-Repository Monorepo
Git History Clean, per-package Mixed, all packages
Repository Size Small, focused Large, grows forever
Clone Time Fast per repo Slow, one huge clone
Tooling Standard Git Custom (Bazel, Nx, etc.)
CI/CD Per-package, focused Global or complex splitting
Releases Independent Coordinated or complex
Learning Curve Git knowledge New tooling required

When monorepo wins: Atomic cross-package changes, guaranteed consistency

When meta-repository wins: Independent release cycles, standard tooling, clear ownership

vs. Manual Individual Clones

Aspect Meta-Repository Individual Clones
Setup 3 commands 13+ clone + setup commands
Consistency Enforced Manual
Cross-Package Dev Seamless editable installs Manual pip install -e everywhere
Documentation Integrated Fragmented across repos
Onboarding Quick, automated Slow, error-prone

When individual wins: Working on single package only

When meta-repository wins: Cross-package development, new contributors, consistency

Trade-offs

Advantages

  1. Independent Repositories
  2. Each package has its own git history
  3. Clear ownership and boundaries

  4. Standard GitHub workflows (PRs, issues, releases)

  5. Flexible Development

  6. Work with all packages or subset
  7. Clone what you need

  8. Symlink existing clones

  9. Standard Tooling

  10. Works with standard Git
  11. Compatible with all Python tools

  12. No custom build systems required

  13. Per-Package CI/CD

  14. Each package can optimize its own pipeline
  15. Faster CI (only test what changed)
  16. Clear failure attribution

Disadvantages

  1. Coordination Overhead
  2. Cross-package changes need multiple PRs
  3. Must coordinate breaking changes

  4. Version compatibility management

  5. Initial Clone Time

  6. 13+ git clone operations
  7. Though automated, takes longer than one clone

  8. Network-dependent

  9. Dependency Management

  10. Must explicitly manage inter-package versions
  11. Can have version conflicts

  12. Requires careful release coordination

  13. Discovery

  14. Changes scattered across repositories
  15. Need to check multiple repos for issues
  16. Can miss related changes

Implementation Pattern

The meta-repository pattern requires:

  1. Workspace Configuration (wrknv.toml)
  2. List all sibling repositories
  3. Define paths and URLs

  4. Specify any workspace-level settings

  5. Bootstrap Script (bootstrap.sh)

  6. Clone missing repositories
  7. Skip existing directories

  8. Support symbolic links

  9. Setup Script (setup.sh)

  10. Create shared virtual environment
  11. Install packages in dependency order

  12. Use editable installs

  13. Validation Script (validate.sh)

  14. Verify Python version
  15. Check virtual environment
  16. Test package imports

When to Use Meta-Repository

Good fit when: - Multiple related packages with independent release cycles - Want standard Git/GitHub workflows - Team familiar with Python packaging - Packages can be used independently - Clear package boundaries

Not a good fit when: - Single monolithic application - Require atomic cross-package changes - Complex build dependencies between packages - Team prefers monorepo workflows - Need guaranteed version consistency

Real-World Example

provide-workenv manages: - 13+ repositories - 3 architectural layers - Independent release cycles - Cross-language tooling (Python + Terraform) - Shared development environment

This would be difficult in a monorepo (too many unrelated changes) or individual repos (too much manual coordination).

Next Steps