Skip to content

Documentation Refactor Plan

Last updated: 2026-03-31 Repository: /home/daniyal/workspace/substrate

Objective

Refactor the documentation into a single canonical structure with one source of truth per topic, reduce maintenance cost, remove duplicated markdown trees, and establish clear agent instructions for Codex, Claude Code, Gemini CLI, and Kimi Code.

Summary of current state

The repository currently contains extensive markdown duplication across multiple trees:

  • docs/
  • docs/blueprint/
  • docs/pitch/
  • docs/superpowers/
  • ui/
  • ui/docs/
  • ui/design/
  • ui/roadmap/
  • ui/services/
  • ui/specs/
  • ui/plans/

There are also duplicate agent-instruction files:

  • AGENTS.md
  • agents.md
  • ui/AGENTS.md
  • ui/agents.md

This creates multiple failure modes:

  1. No clear canonical source for many topics.
  2. High risk of drift when one copy is edited and the others are not.
  3. Confusion for humans and coding agents about where to read and update docs.
  4. Redundant review burden for documentation changes.
  5. Mismatch between docs structure and actual repository structure.

Canonical target structure

The documentation should converge on four top-level sections inside docs/, organized by intent rather than by folder history:

  1. Product — why Substrate exists and why it matters
  2. Architecture — how the system is designed
  3. Roadmap — where the product is going
  4. Delivery — the concrete plans and specs being implemented

The practical target structure is:

/
  README.md
  AGENTS.md
  backend/README.md
  ui/README.md
  api/README.md
  docs/
    index.md
    overview.md
    usps.md
    competition.md
    pricing.md
    persona-benefits.md
    marketplace.md
    architecture.md
    modules.md
    api/
      overview.md
    design/
      infrastructure.md
      knowledge-base.md
      non-llm-functions.md
      security-auth.md
      ui-ux.md
    services/
      agent-orchestration.md
      governance.md
      ingestion.md
      proactive-maintenance.md
      reasoning.md
      simulation.md
    roadmap/
      milestones.md
      definition-of-done.md
      use-cases.md
      role-workflows-user-stories.md
      nfrs.md
    specs/
      *.md
    plans/
      *.md

Canonical ownership rules

Root files

  • README.md = repository entry point, quickstart, repo layout, links to deeper docs
  • AGENTS.md = canonical agent guidance for all repo-wide coding agents

Subsystem readmes

  • backend/README.md = backend-only developer instructions
  • ui/README.md = frontend-only developer instructions
  • api/README.md = API contract generation and ownership instructions

These files must not mirror the entire product/architecture docs tree.

docs/ ownership

The top-level docs sections are:

  • Product = docs/overview.md, docs/usps.md, docs/competition.md, docs/pricing.md, docs/persona-benefits.md, docs/marketplace.md
  • Architecture = docs/architecture.md, docs/modules.md, docs/api/overview.md, docs/design/*, docs/services/*
  • Roadmap = docs/roadmap/*
  • Delivery = docs/specs/* and docs/plans/*

Duplicate groups identified

The following duplicate groups were detected directly from the repository inventory.

5x duplicate groups

  • docs/blueprint/design/*
  • docs/design/*
  • ui/design/*
  • ui/docs/blueprint/design/*

  • ui/docs/design/*

  • docs/blueprint/roadmap/*

  • docs/roadmap/*
  • ui/roadmap/*
  • ui/docs/blueprint/roadmap/*

  • ui/docs/roadmap/*

  • docs/blueprint/services/*

  • docs/services/*
  • ui/services/*
  • ui/docs/blueprint/services/*

  • ui/docs/services/*

  • docs/competition.md

  • docs/pitch/competition.md
  • ui/competition.md
  • ui/docs/competition.md

  • ui/docs/pitch/competition.md

  • docs/marketplace.md

  • docs/pitch/marketplace.md
  • ui/marketplace.md
  • ui/docs/marketplace.md

  • ui/docs/pitch/marketplace.md

  • docs/persona-benefits.md

  • docs/pitch/persona-benefits.md
  • ui/persona-benefits.md
  • ui/docs/persona-benefits.md

  • ui/docs/pitch/persona-benefits.md

  • docs/pricing.md

  • docs/pitch/pricing.md
  • ui/pricing.md
  • ui/docs/pricing.md

  • ui/docs/pitch/pricing.md

  • docs/usps.md

  • docs/pitch/usps.md
  • ui/usps.md
  • ui/docs/usps.md

  • ui/docs/pitch/usps.md

  • docs/specs/* selected files also duplicated into docs/superpowers/specs/*, ui/specs/*, ui/docs/specs/*, ui/docs/superpowers/specs/*

  • docs/plans/* selected files duplicated into docs/superpowers/plans/*, ui/plans/*, ui/docs/plans/*, ui/docs/superpowers/plans/*

Additional duplicate groups

  • docs/architecture.md, ui/architecture.md, ui/docs/architecture.md
  • docs/modules.md, ui/modules.md, ui/docs/modules.md
  • docs/overview.md, ui/overview.md, ui/docs/overview.md
  • docs/setup.md, ui/setup.md, ui/docs/setup.md
  • agents.md and ui/agents.md

Refactor principles

  1. One source of truth per topic.
  2. No mirrored markdown trees under ui/ unless the content is truly UI-specific.
  3. No parallel lowercase and uppercase agent docs with overlapping purpose.
  4. Historical specs and plans may remain dated, but only in one canonical location.
  5. Moves and merges should be separated from substantive prose rewrites where possible.
  6. Link fixes and navigation updates should happen in the same change as file moves.

Planned file actions

Keep

  • README.md
  • AGENTS.md
  • backend/README.md
  • ui/README.md
  • api/README.md
  • docs/index.md
  • canonical files under docs/
  • infra-local README files that are specific to that subsystem

Rewrite

  • README.md
  • AGENTS.md
  • ui/README.md
  • docs/index.md
  • mkdocs.yml

Delete after migration

  • agents.md
  • ui/agents.md
  • docs/blueprint/**
  • docs/pitch/** after merge into canonical product docs
  • docs/superpowers/**
  • ui/docs/**
  • ui/design/**
  • ui/roadmap/**
  • ui/services/**
  • ui/specs/**
  • ui/plans/**
  • duplicate top-level UI markdown mirrors such as ui/competition.md, ui/pricing.md, ui/usps.md, etc. when their content is not UI-specific

Migration phases

Phase 1 — inventory and ownership

Deliverables: - this plan - canonical topic map - explicit keep/delete decisions

Phase 2 — agent guidance consolidation

Actions: - make root AGENTS.md the canonical repo-wide instructions file - merge useful content from agents.md - reduce ui/AGENTS.md to UI-specific local guidance only, or remove it if unnecessary - remove lowercase duplicates once canonical guidance is stable

Phase 3 — documentation tree normalization

Actions: - create canonical subtrees under docs/ - migrate product docs into docs/product/ - migrate architecture docs into docs/architecture/ - migrate roadmap docs into docs/roadmap/ - keep dated implementation documents in docs/specs/ and docs/plans/

Actions: - update mkdocs.yml to point only at canonical docs - update internal markdown links after file moves - ensure root README and subsystem READMEs link to canonical docs

Phase 5 — duplicate removal

Actions: - remove obsolete duplicate trees only after canonical files are verified - verify no links or docs build references still point to deleted paths

Phase 6 — guardrails

Actions: - add AGENTS rules to prohibit mirrored docs trees - optionally add CI checks for duplicate markdown by path pattern or exact hash - document docs ownership in root README or docs index

AGENTS.md plan

AGENTS.md should become the single canonical agent-instruction file for the repository.

It should include:

  1. repository overview
  2. canonical docs ownership rules
  3. backend rules
  4. frontend rules
  5. API contract rules
  6. docs update rules
  7. per-agent notes for:
  8. Codex
  9. Claude Code
  10. Gemini CLI
  11. Kimi Code
  12. commit conventions
  13. definition of done

Codex

  • Best for repo-wide refactors and implementation tasks.
  • Must read canonical docs before editing.
  • Must not create duplicate markdown mirrors.
  • If moving docs, must update links and MkDocs nav in the same change.

Claude Code

  • Best for long-form synthesis and multi-file documentation refactors.
  • Must preserve heading hierarchy and cross-links.
  • Must not create alternative duplicated docs “for clarity”.

Gemini CLI

  • Best for one-shot summaries, review support, and draft generation.
  • Output must be folded back into canonical files only.
  • Must not create sidecar markdown copies of canonical docs.

Kimi Code

  • Best for repository-aware coding and refactor support.
  • Same canonical-doc rules apply.
  • No temporary mirrored doc trees.

Risks

  1. Broken links after file moves.
  2. MkDocs nav drift during transition.
  3. Temporary confusion if canonical and duplicate files coexist too long.
  4. Review difficulty if structural moves and prose rewrites are mixed together.

Risk mitigation

  1. Separate structural migration from deeper prose cleanup.
  2. Update nav and links in the same PR as moves.
  3. Remove duplicates quickly once canonical paths are stable.
  4. Keep commit scopes narrow and coherent.
  1. Rewrite AGENTS.md
  2. Rewrite ui/README.md
  3. Normalize MkDocs navigation toward canonical paths
  4. Move/merge product docs
  5. Move/merge architecture docs
  6. Move/merge roadmap docs
  7. Remove duplicated specs/plans mirrors
  8. Delete obsolete duplicate trees
  9. Add CI/docs guardrails

Definition of done

The docs refactor is complete when:

  • every topic has exactly one canonical markdown source
  • root and subsystem READMEs point to canonical docs only
  • MkDocs navigation references canonical paths only
  • obsolete duplicate trees are removed
  • agent guidance is consolidated under root AGENTS.md
  • no lowercase/uppercase duplicate agent brief remains with overlapping purpose
  • coding agents can determine where to read and where to update without ambiguity