Refactor: SOLID/DRY Backend + Spec-Driven Frontend Wiring¶

Date: 2026-03-28 Status: Approved Scope: Backend refactoring (SOLID/DRY), OpenAPI spec-driven typed client generation, frontend migration from mock data to real API calls

1. Problem Statement¶

The codebase has two categories of technical debt:

Backend DRY violations — 250+ instances of duplicated code across 20 modules: identical row-to-dict conversions (60+), identical DI factory boilerplate (20 modules), identical response envelope formatting (20+), ad-hoc error handling (18+ RuntimeError throws).
Frontend/backend disconnect — 11 of 21 pages use mock JSON data files instead of calling real backend endpoints. API client layer is hand-written across 6 files with inconsistent patterns. No typed contract between frontend and backend.

2. Goals¶

Eliminate backend code duplication via base classes and shared utilities
Establish consistent error handling hierarchy
Generate a typed API client from the OpenAPI spec (single source of truth)
Replace all mock data pages with real backend API calls
Delete dead mock JSON files and hand-written API client files

3. Non-Goals¶

Wiring unused infrastructure connections (NATS, pgvector, vLLM) — follow-up work
Adding new features or endpoints
Changing database schemas
Refactoring the plugin system's discovery mechanism (noted but deferred)

4. Backend SOLID/DRY Refactoring¶

4.1 BaseRepository¶

A base class for all 15+ repositories that absorbs repeated query execution, row conversion, and error handling.

Location: app/core/database/base_repository.py

class BaseRepository:
    def __init__(self, session: AsyncSession):
        self.session = session

    async def _fetch_one(self, query: str, params: dict, *, entity: str = "record") -> dict:
        """Execute query, return single row as dict. Raises EntityNotFoundError if no row."""
        row = (await self.session.execute(text(query), params)).first()
        if row is None:
            raise EntityNotFoundError(entity)
        return dict(row._mapping)

    async def _fetch_one_or_none(self, query: str, params: dict) -> dict | None:
        """Execute query, return single row as dict or None."""
        row = (await self.session.execute(text(query), params)).first()
        return dict(row._mapping) if row else None

    async def _fetch_all(self, query: str, params: dict | None = None) -> list[dict]:
        """Execute query, return all rows as list of dicts."""
        result = await self.session.execute(text(query), params or {})
        return [dict(r._mapping) for r in result]

    async def _insert(self, query: str, params: dict, *, entity: str = "record") -> dict:
        """Execute INSERT/UPSERT, return row. Raises EntityCreationError on failure."""
        row = (await self.session.execute(text(query), params)).first()
        if row is None:
            raise EntityCreationError(entity)
        return dict(row._mapping)

Impact: Eliminates ~60 dict(row._mapping) patterns and ~18 RuntimeError patterns across all repositories.

Migration: Each repository changes self.session.execute(text(q), p) + manual row handling → self._fetch_all(q, p) or self._insert(q, p, entity="X").

4.2 DI Factory Generators¶

Replace 20 identical dependencies.py files with reusable factory functions.

Location: app/core/dependencies.py (extend existing)

def repo_provider(repo_class: type):
    """Create a FastAPI dependency that instantiates a repository with a DB session."""
    def factory(session: AsyncSession = Depends(get_session)):
        return repo_class(session)
    return factory

def service_provider(service_class: type, repo_factory):
    """Create a FastAPI dependency that instantiates a service with its repository."""
    def factory(repo=Depends(repo_factory)):
        return service_class(repo)
    return factory

Per-module usage (replaces entire dependencies.py body):

from app.core.dependencies import repo_provider, service_provider

get_community_repo = repo_provider(CommunityRepository)
get_community_service = service_provider(CommunityService, get_community_repo)

Exceptions: Modules with extra DI needs (marketplace needs session + repo + settings, billing needs a payment provider) keep custom factories but inherit the pattern where possible.

4.3 Response Envelope Helper¶

Location: app/core/responses.py

def list_response(items: list) -> dict:
    """Standard list response envelope."""
    return {"data": items, "meta": {"total": len(items)}}

Impact: Replaces 20+ identical {"data": items, "meta": {"total": len(items)}} constructions in routers and services.

4.4 Error Hierarchy¶

Extend the existing SubstrateError base in app/core/exceptions.py:

class EntityNotFoundError(SubstrateError):
    """Raised when a database lookup returns no rows."""
    def __init__(self, entity_type: str, identifier: str = ""):
        detail = f"{entity_type} not found"
        if identifier:
            detail = f"{entity_type} not found: {identifier}"
        super().__init__(detail, status_code=404)

class EntityCreationError(SubstrateError):
    """Raised when an INSERT/UPSERT returns no rows."""
    def __init__(self, entity_type: str):
        super().__init__(f"Failed to create {entity_type}", status_code=500)

These are caught by the existing SubstrateError handler in main.py and returned as proper HTTP responses.

5. OpenAPI Spec & Typed Client Generation¶

5.1 Regenerate OpenAPI Spec¶

The api/openapi.yml is regenerated from the FastAPI app to ensure it reflects the current backend state:

# In generate-types.sh
cd ../backend
uv run python -c "
import json
from app.main import create_app
app = create_app()
with open('../api/openapi.json', 'w') as f:
    json.dump(app.openapi(), f, indent=2)
"

Output: api/openapi.json (JSON format, consumed by openapi-typescript).

5.2 Type Generation Pipeline¶

Dependencies: - openapi-typescript (devDependency) — generates TypeScript types from OpenAPI spec - openapi-fetch (dependency) — typed fetch client using those types

Script in ui/package.json:

{
  "scripts": {
    "generate:api": "openapi-typescript ../api/openapi.json -o src/types/api.generated.ts"
  }
}

5.3 Typed Client with Auth Middleware¶

Location: ui/src/api/client.ts (rewritten)

import createClient, { type Middleware } from "openapi-fetch";
import type { paths } from "../types/api.generated";

let tokenAccessor: (() => string | undefined) | null = null;

export function setTokenAccessor(fn: () => string | undefined) {
  tokenAccessor = fn;
}

const authMiddleware: Middleware = {
  async onRequest({ request }) {
    const token = tokenAccessor?.();
    if (token) request.headers.set("Authorization", `Bearer ${token}`);
    return request;
  },
};

export const api = createClient<paths>({
  baseUrl: import.meta.env.VITE_BACKEND_URL || "/api/v1",
});
api.use(authMiddleware);

All pages import api from this single file. Every call is fully typed against the OpenAPI spec.

5.4 AuthBridge¶

ui/src/api/authBridge.ts remains unchanged — it calls setTokenAccessor() with the OIDC token accessor.

6. Frontend Page Migration¶

6.1 Pages Switching from Mock Data to Real API (11 pages)¶

Page	Mock function removed	New typed call
DashboardPage	`substrateApi.getDashboardData(p)`	`api.GET("/dashboard")`
CommunitiesPage	`substrateApi.getCommunities()`	`api.GET("/communities")`
GraphPage	`substrateApi.getNodes()` / `getEdges()`	`api.GET("/communities/{slug}/graph", { params: { path: { slug } } })`
MemoryPage	`substrateApi.getMemoryEntries()` / `getMeta()`	`api.GET("/memory")` / `api.GET("/memory/meta")`
SearchPage	`substrateApi.getSearchData()`	`api.GET("/search/data")`
QueuePage	`substrateApi.getQueueItems()`	`api.GET("/queue")`
SimulationPage	`substrateApi.getSimulationData()`	`api.GET("/simulation")`
PullRequestsPage	`substrateApi.getPullRequests()` / `getHistory()`	`api.GET("/pull-requests")` / `api.GET("/pull-requests/history")`
CreateCommunityPage	`substrateApi.getCreateCommunityData()`	`api.GET("/communities/create-data")`
CreatePolicyPage	`substrateApi.getCreatePolicyData()`	`api.GET("/policies/create-data")`
ContributeMemoryPage	`substrateApi.getContributeMemoryData()`	`api.GET("/memory/contribute")`

6.2 Pages Switching Client Only (10 pages)¶

These already call real endpoints. They switch from hand-written clients to the typed api client:

ApiTokensPage: substrateApi.listApiTokens() → api.GET("/auth/tokens")
ProfileSettingsPage: substrateApi.getMyProfile() → api.GET("/auth/me")
TeamSettingsPage: orgApi.getCurrentOrg() → api.GET("/iam/me/access")
BillingPage: billingApi.getAccount() → api.GET("/billing/account")
LlmConnectionsPage: configApi.getConfig(d) → api.GET("/config/{domain}")
OrgSettingsPage: configApi.getConfig(d) / updateConfig(d, v) → api.GET / api.PATCH
PlatformDataPage: same pattern as above
PreferencesPage: same pattern as above
MarketplacePage: substrateApi.listMarketplaceCatalog() → api.GET("/marketplace/catalog")
SettingsPage: all substrateApi.* calls → corresponding api.* calls

Topbar notifications: mock import → api.GET("/notifications")
Sidebar engines: mock import → include engine data in the dashboard endpoint response (no separate endpoint needed)

6.4 Files Deleted¶

Mock data files removed (12 files): - ui/src/data/dashboard.json - ui/src/data/communities.json - ui/src/data/graph.json - ui/src/data/memory.json - ui/src/data/search.json - ui/src/data/queue.json - ui/src/data/simulation.json - ui/src/data/pull-requests.json - ui/src/data/pull-request-blast-radius.json - ui/src/data/notifications.json - ui/src/data/policies.json - ui/src/data/engines.json

Hand-written API clients removed (6 files): - ui/src/api/substrateApi.ts - ui/src/api/billingApi.ts - ui/src/api/configApi.ts - ui/src/api/licensingApi.ts - ui/src/api/orgApi.ts - ui/src/api/pluginsApi.ts

Old generated types removed: - ui/src/types/openapi.generated.ts (replaced by api.generated.ts)

Files kept (UI structure, not API data): - ui/src/data/navigation.json — sidebar nav structure - ui/src/data/settings.json — settings page layout - ui/src/data/settings-org-fields.json — org form field definitions - ui/src/data/settings-platform-domains.json — platform config domain list - ui/src/data/users.json — dev-mode fallback users

7. Migration Order¶

Backend: Add BaseRepository, error classes, list_response(), DI factories
Backend: Migrate all repositories to inherit BaseRepository
Backend: Migrate all dependencies.py to use factory generators
Backend: Replace response envelope boilerplate in routers/services
Regenerate openapi.json from updated FastAPI app
Frontend: Install openapi-fetch + openapi-typescript, generate types
Frontend: Rewrite client.ts with openapi-fetch + auth middleware
Frontend: Migrate all pages to use typed api client
Frontend: Delete mock data files and hand-written API clients
Verify all endpoints respond correctly end-to-end