Xiber NetOS — Roadmap & Known Gaps
What's built, what's missing, and the recommended path forward.
Current Phase: Operational Beta
NetOS has moved well past the initial prototype. It is a working, RBAC-enforced platform covering circuits, infrastructure, customers, RF links, electrical services, vendor quotes, and their financial attribution — running live on the internal LAN behind SSO. The codebase went through a full-stack refactor (service extraction, N+1 query fixes, web API-client split), added a real integration test harness (~140 API tests) and web typecheck/lint/build gates, and shipped explicit redundancy/attribution modeling, searchable entity pickers, weekly SMTP digests, quote intake, and PDF agreement extraction.
The remaining blockers to calling it "production-grade" are narrower than before: identity is still terminated at the proxy rather than validated in-API (no native JWT), modeled revenue/margin is still a placeholder until Sonar is connected, renewal states are computed on the fly rather than persisted, and several external connectors (invoice ingestion, vendor portals) are unbuilt. There is also no CI pipeline yet.
Feature Inventory
What's Built
| Area | Details |
|---|---|
| Monorepo scaffold | apps/api, apps/web, apps/mcp, infra/docker |
| Database schema | 13+ tables, PostGIS geometry, pgvector embeddings, materialized financial view |
| Migrations | Alembic baseline + infrastructure + agreements migrations |
| API | FastAPI with OpenAPI 3.1, circuit CRUD, bulk actions, map, dashboard, import, providers, infrastructure |
| Circuit table | Search, filter by carrier/status/type, bulk edit/delete, click-through to detail |
| Circuit detail | Full attributes, endpoints, carrier info, NOC contacts, contract terms, lifecycle history |
| Map | A→Z circuit lines, endpoint dots, infrastructure markers, colored by provider, thickness by bandwidth |
| Dashboard | 17 KPI cards/panels including spend by carrier, service mix, renewal pipeline, at-risk circuits |
| Timeline | Contract-term Gantt with install markers, renewal deadlines, today line, state colors |
| Topology | Endpoint graph with logical/geographic layouts, SPOF detection, carrier-colored edges |
| Service providers | Portal links, NOC phones, account managers, circuit/infra counts, ETF exposure |
| Infrastructure | Data centers, towers, colos, rooftops, carrier hotels, POPs, offices, and aggregation facilities with costs, terms, and bulk edit/delete |
| Electrical | Utility service accounts attached to infrastructure or customer sites, with provider, meter, service details, average monthly cost, monitoring link, and documents |
| Financial / P&L | Per-circuit modeled revenue, margin, ETF exposure, carrier spend rollups, and infrastructure attribution foundation |
| Address lookup | Search circuits by address, endpoint, city, state, carrier, or circuit ID |
| Mobile field view | Phone-friendly circuit lookup with NOC call, copy ID, and map links |
| Admin / audit | Recent activity trail plus bug/feature queue with priority, status, and progress comments |
| CSV/XLSX import | Heuristic column mapping, staging, in-app staging-row editing, upsert by natural key, auto-stub carriers/endpoints — for circuits, infrastructure, and customers |
| Vendor quotes | Dedicated quote intake page, quote scope/target types, pricing options with term escalators, contact/colo fields, attachments (PDF/image/pasted-text), background AI analysis, draft-inbox review queue, win marking, and site-level aggregation |
| PDF agreement extraction | Upload contract PDFs against a circuit, text extraction with OCR fallback, AI structured field extraction, and an apply-to-record flow (/ai/extract-pdf, /circuits/{id}/agreement-extractions) |
| Notifications | In-process weekly scheduler sends SMTP renewal reports and business-readable activity digests; send-now + preview + SMTP test endpoints |
| Redundancy & attribution | Explicit redundancy_role (primary/secondary/standalone) on subtended links, circuits, and customer RF links; secondary/backup paths excluded from revenue but retained as cost; attribution-conflict 409 with retry-as-redundant action |
| Sample data | 4 carriers, 10 endpoints, 8 circuits with contracts and lifecycle events |
| MCP tools | 22 token-aware read/context tools for search, circuits, infrastructure, customers, RF links, quotes, quote drafts, carrier summary, renewal pipeline, and site documents/photos |
| RBAC | 7 default roles, permission-per-route enforcement, field-level masking, DB-backed roles/permissions/user-overrides, and API access tokens; dev shim accepts x-user-role header |
| Frontend UX | Searchable entity comboboxes replacing large scroll-selects; error/loading boundaries; deferred MapLibre load |
| Engineering baseline | Full-stack refactor (shared service extraction, N+1 query fixes, split web API client), ~140-test pytest+httpx integration harness, and web typecheck/lint/build gates |
| Docker dev stack | PostgreSQL 16, Redis 7, API, Web with hot reload |
Gap Analysis
1. Authentication & Authorization
| Detail | |
|---|---|
| - | -------- |
| Current | Three identity sources: API access tokens (ntos_...), trusted reverse-proxy identity headers (gated by TRUST_PROXY_AUTH_HEADERS + trusted CIDRs — this is how oauth2-proxy/Entra SSO reaches the API), and the dev-auth header shim (x-user-role, gated by ALLOW_DEV_AUTH). Permissions, roles, and user overrides are DB-backed with per-route enforcement and field-level masking. Live behind SSO on the internal LAN. |
| Impact | Usable today behind a trusted proxy, but the API itself never validates a JWT — it trusts headers from an in-front proxy. A misconfigured proxy/CIDR boundary is the main risk. |
| Needed | Native Microsoft Entra OIDC JWT validation inside the API (the ENTRA_*/JWT_AUDIENCE settings exist but are unused). Role assignment from Entra groups/claims. Server-side session handling and token refresh in Next.js. |
| Depends on | Entra app registration (see Public Hostname & SSO) |
2. Audit Logging
| Detail | |
|---|---|
| - | -------- |
| Current | Activity logging exists for request/write activity, feedback, and bulk actions. created_by/updated_by are populated on newer write paths. |
| Impact | Basic accountability exists, but sensitive read tracking and field-level redaction are incomplete. |
| Needed | Complete field-level before/after coverage for all write paths. Field-level read audit for sensitive data (ETF formulas, banking, MD5 keys). Sensitive field redaction on read by role. |
3. CSV/XLSX Import Enhancements
| Detail | |
|---|---|
| - | -------- |
| Current | Heuristic column mapping. Staging rows with an in-app staging-row editor (PATCH /imports/{entity}/{job}/rows). Commit valid rows. Upsert by natural key. Now covers circuits, infrastructure, and customers. |
| Impact | Column mapping is still fragile for non-standard headers, and there is no saved-mapping memory or pre-commit diff preview. |
| Needed | AI-assisted mapping (send headers + sample rows, get canonical mapping; POST /ai/import-preview exists as a starting point). Saved mappings by (user, source_signature). Diff preview before updating existing records. Explicit confirmation dialog before overwriting financial fields. |
4. PDF Contract Ingestion
| Detail | |
|---|---|
| - | -------- |
| Current | Built. PDF upload against a circuit (POST /circuits/{id}/agreement-extractions), text extraction with OCR fallback, AI structured field extraction (POST /ai/extract-pdf), stored extraction records, and an apply-to-record flow. AI provider is pluggable (AI_PROVIDER, default deterministic/no-op). |
| Impact | Contracts can be ingested from PDF rather than keyed by hand. |
| Needed | Object storage (S3/MinIO) instead of local filesystem paths. Richer confidence scoring with source quotes and page numbers. Hardened side-by-side approval UI with mandatory human approval for MRC, ETF, and term fields before commit. |
5. Renewal Engine
| Detail | |
|---|---|
| - | -------- |
| Current | Renewal states computed on-the-fly in API/frontend (/timeline/renewal-pipeline). Weekly SMTP renewal report + activity digest now ship via an in-process asyncio scheduler in the API (not Celery), with send-now, preview, and settings endpoints. States are still not persisted. |
| Impact | Stakeholders get a proactive weekly email, but there is no per-transition alerting or decision workflow, and nothing is persisted for historical state tracking. |
| Needed | Persist renewal states (stable → watch → active → critical → overdue → decided) via a scheduled job or materialized state table. Per-transition Slack/Monday.com notifications. Decision workflow UI (renew / negotiate / terminate / port). DocuSeal termination notice generation. Test-mode fast-forward for QA. |
| State machine | stable (>180d) → watch (90–180d) → active (30–90d) → critical (<30d) → overdue (past deadline) → decided |
6. Dashboard & Financial Accuracy
| Detail | |
|---|---|
| - | -------- |
| Current | Circuit-level modeled MRR uses MRC * 1.35 placeholder multiplier. Infrastructure detail now has subtended links, direct/downstream revenue and cost rollups, relationship effective dates, and circuit-linked attribution records. |
| Impact | Circuit margin remains illustrative until Sonar revenue is connected. Infrastructure economics can be modeled manually now, but allocation methods, period reporting, and actual invoice/revenue integrations are still needed for production financial reporting. |
| Needed | Sonar integration to pull actual customer revenue attribution. Real margin calculation using customer MRR, carrier MRC, facility MRC, transport allocation, and invoice actuals. Full infrastructure waterfall view by asset, parent/child relationship, market, provider, service type, and accounting period. ETF exposure formulas and renewal projection forecasts using contract escalators. |
7. Map Enhancements
| Detail | |
|---|---|
| - | -------- |
| Current | A→Z lines with provider coloring and bandwidth thickness. Provider/type/status filters. |
| Impact | Map is useful but isolated from table view. No external data overlays. |
| Needed | Polygon/lasso selection tool. Shared filter state with circuit table. Wisdm tower layer overlay. Sonar property/coverage layer overlay. Tile hosting decision (MapTiler vs. self-hosted). Cluster markers for dense endpoint areas. |
8. Topology Enhancements
| Detail | |
|---|---|
| - | -------- |
| Current | Endpoint graph with logical/geographic layouts, SPOF detection by node degree. |
| Impact | Layout resets on reload. No true networking detail. |
| Needed | Drag/drop layout with save per user and shared default. True networking fields: VLANs, BGP sessions, router interfaces, handoff ports, IP blocks. Better redundancy modeling (path diversity, not just node degree). |
9. Financial / P&L View
| Detail | |
|---|---|
| - | -------- |
| Current | Implemented with per-circuit modeled P&L, carrier spend, margin, ETF exposure, and the first infrastructure subtended-link economics model. |
| Needed | Sonar revenue attribution, invoice actuals, carrier/customer/market/service rollups, infrastructure waterfall reports, renewal projection forecast, allocation calculators, and export to XLSX. |
10. Address Lookup
| Detail | |
|---|---|
| - | -------- |
| Current | Implemented text search across circuit, carrier, endpoint, city, state, and address fields. |
| Needed | Integration with Xiber Service Address Availability Checker. Nearby endpoint/circuit search by radius. Quote-history cost-to-serve estimate. |
11. Mobile Field View
| Detail | |
|---|---|
| - | -------- |
| Current | Implemented responsive field view with search, one-tap NOC call, copy circuit ID, and map app links. |
| Needed | Add infrastructure asset support, recent events, paired-circuit context, and offline-friendly behavior. |
12. Invoice Ingestion
| Detail | |
|---|---|
| - | -------- |
| Current | Invoice and InvoiceLineItem schemas exist. No ingestion pipeline. |
| Needed | Mailbox polling (M365 Graph) or manual upload. PDF invoice parsing with line item extraction. Automatic matching to circuit by carrier circuit ID. Variance detection (invoiced MRC vs. contracted MRC). Auto-create Monday.com task for finance team on variance. |
13. Vendor Portal Connectors
| Detail | |
|---|---|
| - | -------- |
| Current | Not implemented. |
| Needed | Lumen Control Center API connector. Zayo Tranzact API connector. Cogent portal connector. Vault/Doppler credential storage. Playwright scraping fallback where APIs are unavailable. |
14. MCP Full Coverage
| Detail | |
|---|---|
| - | -------- |
| Current | Token-aware read/context tools exist for search, circuits, infrastructure, customers, RF links, quotes, quote draft queue, carrier summary, renewal pipeline, and site document/photo metadata. Quote draft MCP tools can stage vendor quotes for human review without committing them to quote history. |
| Needed | get_circuit_pl, get_outage_history, broader write tools with explicit agent confirmation flow, and XOS MCP gateway registration/runtime management. |
Recommended Build Phases
| Phase | Focus | Why First |
|---|---|---|
| 1 | Real Entra authentication | Cannot deploy without identity. Blocks audit logging. |
| 2 | Audit logging | Required for compliance and accountability. |
| 3 | Renewal engine + notifications | Highest business value — prevents missed deadlines and auto-renew traps. |
| 4 | Contract CRUD + PDF ingestion hardening (object storage, confidence, approval UI) | Ingestion pipeline exists; harden storage and human-approval before relying on it. |
| 5 | Financial/P&L + Sonar integration + infrastructure waterfall | Real margin visibility, accurate business intelligence across circuits, structures, child facilities, and customer endpoints. |
| 6 | Import enhancements (AI mapping, editor) | Reduces friction for bulk data onboarding. |
| 7 | Invoice ingestion + variance detection | Catches billing errors, automates finance workflows. |
| 8 | MCP full coverage | Enables XOS agents to query NetOS data for automated reporting and decisions. |
| 9 | Map + topology enhancements | Improved operational visualization. |
| 10 | Mobile, address lookup, exports | Quality-of-life features for field and sales teams. |
| 11 | Vendor portal connectors | Automated circuit status sync from carrier portals. |
Technical Debt
| Item | Notes |
|---|---|
| Celery / Redis | No Celery app or worker exists; scheduled work (renewal report, activity digest) runs in an in-process asyncio loop inside the API. Redis is provisioned in the dev stack but effectively dormant — no broker/queue or cache use yet. A real worker is still needed before heavier background jobs (invoice polling, persisted renewal states). |
| pgvector | Extension installed but no embeddings generated yet. Future use for semantic circuit search. |
| Error handling | Errors follow FastAPI {"detail": ...} with 401/403/404/409/422; some edge cases can still surface generic 500s. |
| Test coverage | ~140 API integration tests (pytest + httpx) covering RBAC, field masking, attribution/redundancy rules, imports, documents, and economics. Frontend has typecheck/lint/build gates. Coverage of import edge cases and renewal logic could still deepen. |
| CI/CD | Still no pipeline (.github/ absent). Gates are run locally/manually. Need GitHub Actions or similar for lint, typecheck, test, build, deploy. |