Xiber NetOS — Data & Import Guide
Data model, system-of-record boundaries, CSV/XLSX import workflow, and validation rules.
System of Record Boundaries
What NetOS Owns
| Domain | Details |
|---|---|
| Carriers | Provider names, portal URLs, NOC contacts, account managers |
| Endpoints | Location names, addresses, coordinates, facility types |
| Circuits | Technical attributes, bandwidth, status, purpose, handoff |
| Financials | MRC, NRC, PO numbers, GL codes |
| Contracts | Terms, renewal windows, ETF formulas, escalators, governing law |
| Renewals | Renewal state machine, decision history |
| Lifecycle | Event timeline (orders, installs, upgrades, outages, decomms) |
| Invoices | Carrier invoices and variance detection (future) |
| Infrastructure | Towers, rooftops, data centers, POPs, carrier hotels, offices, aggregation sites, facility costs, rack/power commitments, and tower leases |
| Customers | Commercial and MFC customer sites used as customer endpoints for circuits, RF Links, and attribution |
| RF Links | Wireless backhaul and customer endlinks with RF system, capacity, CAPEX, MRC/rent, frequency, FCC license, and depreciation fields |
| Electrical Services | Utility accounts attached to infrastructure or customer sites, including provider, meter, service details, average monthly cost, monitoring link, and documents |
| Infrastructure Attribution | Explicit subtended relationships between parent sites, downstream structures, customer endpoints, circuits, and RF Links |
What NetOS Does NOT Own
| Domain | Source System |
|---|---|
| End-customer billing | Sonar |
| Device telemetry & monitoring | LibreNMS / Prometheus / Loki |
| Physical network design | Wisdm |
| Document signatures | DocuSeal |
| Project task management | Monday.com |
NetOS will pull from these systems (Sonar revenue, LibreNMS monitoring links, Wisdm geometry) and push to them (Monday.com renewal tasks, DocuSeal termination notices) — but those systems remain the source of truth for their respective domains.
Data Model
Core Entities
Carrier ── Circuit ── Contract
│
├── Endpoint snapshots copied from selected sites
├── LifecycleEvent
├── Invoice ── InvoiceLineItem
└── RenewalDecision
InfrastructureAsset ── SubtendedLink ── Customer
│ │
│ ├── Circuit cost source
│ └── RFLink cost source
├── ElectricalService
└── InfrastructureDocument
Customer ── ElectricalService
RFSystemCatalog ── RFLink ── RFLinkDocumentCircuit Natural Key
(carrier_id, carrier_circuit_id)This composite key prevents duplicate circuits when importing the same carrier inventory multiple times. If a row matches an existing natural key, the circuit is updated rather than duplicated.
Manual circuit entry should use Infrastructure and Customer site selectors. Imported endpoint fields remain supported for bulk onboarding, carrier inventory loads, and legacy records, but the preferred operating workflow is to create or correct the Infrastructure/Customer site first and then select that site on the circuit.
Key Enums
| Enum | Values |
|---|---|
| Service Type | wave, epl, evpl, dia, ip_transit, dark_fiber, mpls, cross_connect, cellular_backup, broadband |
| Status | quoted, ordered, installing, active, suspended, decommissioned |
| Purpose | backhaul, backup, customer_dedicated, internet, dr, management |
| Redundancy | primary, secondary, standalone |
| Endpoint Type | mdu, tower, data_center, colo, pop, customer_premise, carrier_hotel |
| Renewal State | stable, watch, active, critical, overdue, decided |
CSV/XLSX Import
Import Templates
The import page now has separate downloadable templates:
| Template | Use For | Import Endpoint |
|---|---|---|
/templates/netos-circuit-import-template.csv | Wholesale circuits, A/Z endpoints, provider details, agreement terms | /api/v1/imports/circuits/* |
/templates/netos-infrastructure-import-template.csv | Data centers, towers, rooftops, colocation, POPs, offices, and aggregation facilities | /api/v1/imports/infrastructure/* |
Use separate files when the records have different lifecycles. Circuits are matched by provider plus carrier circuit ID; infrastructure is matched by Xiber infrastructure label.
Circuit Required Fields
| Field | Description |
|---|---|
carrier_name | Upstream provider (e.g., "Lumen", "Zayo") |
carrier_circuit_id | The carrier's reference number |
These two fields form the circuit natural key. All other fields are optional but recommended.
If xiber_circuit_label is omitted, NetOS generates it using Provider Short Name Service Type A Loc -> Z Loc when the provider has a short name, otherwise Carrier Service Type A Loc -> Z Loc. DIA and broadband use the same pattern without a Z location because they are single-site services. Generated labels are truncated to fit the circuit label field and receive a numeric suffix only when needed for uniqueness.
Recommended Fields
| Field | Type | Example |
|---|---|---|
xiber_circuit_label | string | Lumen WAVE Site A -> Site Z |
service_type | enum | wave, dia, dark_fiber, broadband |
bandwidth_mbps | number | 10000 |
status | enum | active |
mrc_usd | decimal | 2500.00 |
nrc_usd | decimal | 5000.00 |
ban_id | string | BAN-100245 |
billing_account | string | BA-44129 |
protocol_handoff | string | 10GBase-LR |
monitoring_url | URL | https://librenms.xiber.local/device/123 |
install_date | date | 2024-01-15 |
a_endpoint_name | string | Xiber HQ - 123 Main St |
a_city | string | Columbus |
a_state | string | OH |
z_endpoint_name | string | Cologix COL1 |
z_city | string | Columbus |
z_state | string | OH |
Contract Fields (Optional)
| Field | Type | Example |
|---|---|---|
contract_effective_date | date | 2024-01-15 |
contract_term_end_date | date | 2027-01-14 |
contract_initial_term_months | integer | 36 |
contract_renewal_type | enum | auto_renew, manual, month_to_month, evergreen |
contract_renewal_notice_days | integer | 90 |
contract_msa_terms_text | text | 36 month initial term... |
contract_etf_formula_text | text | 100% of remaining MRC |
contract_price_escalator_pct | decimal | 3.0 |
contract_pdf_url | URL | /api/v1/circuits/.../agreement/file/... |
Infrastructure Fields
Infrastructure imports cover facilities and structures that have their own cost, agreement, and renewal lifecycle. Electrical utility accounts are tracked separately under Electrical Services after the infrastructure or customer site exists.
| Field | Type | Example |
|---|---|---|
xiber_infrastructure_label | string | INF-IND-PWR-0012 |
provider_name | string | AES Indiana |
site_type | string | data_center, tower, rooftop, colo, pop, office, other |
display_name | string | Indy MDU 42 Electrical Service |
facility_account_id | string | METER-100245 |
latitude, longitude | decimal | 39.773350, -86.158050 |
service_description | text | Electrical utility service for network room |
power_commit | string | 400A 120/208V 3 phase |
monitoring_url | URL | Exact monitoring, graph, or utility-status page |
mrc_usd, nrc_usd | decimal | 1450.00, 0.00 |
price_escalator_pct | decimal | 3.0 |
effective_date, term_end_date | date | 2024-01-01, 2024-12-31 |
msa_terms_text, etf_formula_text | text | Agreement language |
Manual Bulk Maintenance
After import, users can make broad operational corrections from the UI.
Circuit Bulk Actions
URL: /circuits
Select rows with the checkboxes in the first column. The header checkbox selects all currently visible filtered rows.
| Bulk Field | Canonical Field |
|---|---|
| Status | status |
| Provider | carrier_name |
| Type | service_type |
| MRC | mrc_usd |
| NRC | nrc_usd |
| Term End | contract_term_end_date |
| Notes | notes |
Infrastructure Bulk Actions
URL: /infrastructure
Select rows with the checkboxes in the first column. The header checkbox selects all visible rows.
| Bulk Field | Canonical Field |
|---|---|
| Status | status |
| Provider | provider_name |
| Type | site_type |
| MRC | mrc_usd |
| Escalator % | price_escalator_pct |
| Term End | term_end_date |
| Notes | notes |
Bulk deletes are soft-deletes. They set deleted_at and preserve the record for audit/history.
Import Workflow
| Step | What Happens |
|---|---|
| 1. Upload | User uploads a CSV or XLSX file via /import or the API |
| 2. Signature | Raw file hash is calculated for deduplication |
| 3. Mapping | Headers are mapped to canonical fields using heuristic matching |
| 4. Staging | All rows are written to import_staging_rows with validation results |
| 5. Validation | Each row is checked for required fields, type coercion, enum matching |
| 6. Review | User sees first 5 rows with mapped columns and any validation errors |
| 7. Commit | Valid rows are committed; invalid rows are skipped with error details |
| 8. Carrier stubs | Missing carriers are auto-created with name only (fill in details later) |
| 9. Endpoint stubs | Missing circuit endpoints are auto-created; coordinates are populated when latitude/longitude are present |
What Happens on Re-Import
When a row matches an existing circuit by natural key (carrier, carrier_circuit_id), or existing infrastructure by xiber_infrastructure_label:
- All provided fields are updated to the new values
- Fields not present in the import are left unchanged
- This is an upsert, not a replace
Validation Rules
| Field | Validation |
|---|---|
carrier_name | Required, non-empty |
carrier_circuit_id | Required, non-empty |
service_type | Coerced to enum (case-insensitive, underscore-tolerant) |
status | Coerced to enum |
mrc_usd | Must be numeric if provided |
nrc_usd | Must be numeric if provided |
bandwidth_mbps | Must be numeric if provided |
install_date, decom_date, contract dates, infrastructure term dates | Must be ISO date if provided |
Current limitations:
- No inline editing of staged rows (must fix CSV and re-upload)
- No diff preview before updating existing circuits
- No explicit confirmation for financial field overwrites from CSV/XLSX
- Column mapping is heuristic-only (AI-assisted mapping planned)
API Import Examples
Preview (upload and detect mapping):
curl -H "x-user-role: exec" \
-F "file=@examples/sample_circuits.csv" \
http://localhost:8000/api/v1/imports/circuits/previewResponse:
{
"import_job_id": "abc123",
"detected_mapping": {
"Carrier": "carrier_name",
"Circuit ID": "carrier_circuit_id",
"MRC": "mrc_usd"
},
"headers": ["Carrier", "Circuit ID", "Type", "BW", "MRC", ...],
"sample_rows": [...],
"total_rows": 45,
"valid_rows": 42,
"invalid_rows": 3
}Commit (import valid rows):
curl -H "x-user-role: exec" \
-X POST \
http://localhost:8000/api/v1/imports/circuits/{import_job_id}/commitInfrastructure imports use the same pattern:
curl -H "x-user-role: exec" \
-F "file=@netos-infrastructure-import-template.csv" \
http://localhost:8000/api/v1/imports/infrastructure/preview
curl -H "x-user-role: exec" \
-X POST \
http://localhost:8000/api/v1/imports/infrastructure/{import_job_id}/commitResponse:
{
"inserted": 38,
"updated": 4,
"invalid": 3,
"errors": [
{"row": 12, "field": "mrc_usd", "error": "not a valid number"},
{"row": 23, "field": "carrier_name", "error": "required field missing"},
{"row": 31, "field": "service_type", "error": "unknown value: 'wavelength'"}
]
}Sample CSV
Files:
/templates/netos-circuit-import-template.csv/templates/netos-infrastructure-import-template.csv
carrier_name,carrier_circuit_id,xiber_circuit_label,service_type,bandwidth_mbps,status,mrc_usd,nrc_usd,a_endpoint_name,a_city,a_state,z_endpoint_name,z_city,z_state
Lumen,DHEC.123456,XIB-LUM-0001,wave,10000,active,2500.00,5000.00,Xiber HQ,Columbus,OH,Cologix COL1,Columbus,OH
Zayo,ZY-789012,XIB-ZAY-0001,dark_fiber,100000,active,4200.00,15000.00,56 Marietta St NW,Atlanta,GA,Equinix AT1,Atlanta,GACommon circuit purpose values include backhaul, backup, customer_dedicated, internet, dr, and management. Use backup for MFC backup circuits, such as broadband service backing up a primary fiber or RF path.
Endpoint Coordinates
Map rendering and geographic topology require endpoint coordinates.
| Field | Type | Format |
|---|---|---|
endpoints.geom | PostGIS POINT | SRID 4326 (WGS84 lat/lng) |
Coordinates are stored as PostGIS geometry. The seed data includes coordinates for all sample endpoints. When importing circuits, endpoints are auto-created as stubs without coordinates — add coordinates later via the API or UI to make them appear on the map.
Contract Data
Contracts are linked to circuits and/or infrastructure assets. Key fields:
| Field | Description |
|---|---|
carrier | Provider this contract is with |
contract_type | Master, circuit-specific, facility |
parent_contract | Link to master agreement (if applicable) |
effective_date | When the contract began |
initial_term_months | Original commitment length |
term_end_date | Calculated end date |
renewal_type | auto, manual, evergreen |
renewal_term_months | Length of auto-renewal period |
renewal_notice_days | Days before end to provide notice |
renewal_decision_deadline | Calculated: term_end - notice_days |
price_escalator | Annual price increase formula |
etf_formula | Early termination fee calculation |
etf_current_value | Current estimated ETF |
governing_law | Jurisdiction |
pdf_url | Link to signed contract PDF |
Current state: Sample contracts are created by the seed script. PDF extraction and side-by-side approval are not yet implemented. See Roadmap for the PDF ingestion plan.
