NetOSField accessMenu
Signed in user unavailable
Documentation/Data & Import Guide
Download PDF

Xiber NetOS — Data & Import Guide

Data model, system-of-record boundaries, CSV/XLSX import workflow, and validation rules.


System of Record Boundaries

What NetOS Owns

DomainDetails
CarriersProvider names, portal URLs, NOC contacts, account managers
EndpointsLocation names, addresses, coordinates, facility types
CircuitsTechnical attributes, bandwidth, status, purpose, handoff
FinancialsMRC, NRC, PO numbers, GL codes
ContractsTerms, renewal windows, ETF formulas, escalators, governing law
RenewalsRenewal state machine, decision history
LifecycleEvent timeline (orders, installs, upgrades, outages, decomms)
InvoicesCarrier invoices and variance detection (future)
InfrastructureTowers, rooftops, data centers, POPs, carrier hotels, offices, aggregation sites, facility costs, rack/power commitments, and tower leases
CustomersCommercial and MFC customer sites used as customer endpoints for circuits, RF Links, and attribution
RF LinksWireless backhaul and customer endlinks with RF system, capacity, CAPEX, MRC/rent, frequency, FCC license, and depreciation fields
Electrical ServicesUtility accounts attached to infrastructure or customer sites, including provider, meter, service details, average monthly cost, monitoring link, and documents
Infrastructure AttributionExplicit subtended relationships between parent sites, downstream structures, customer endpoints, circuits, and RF Links

What NetOS Does NOT Own

DomainSource System
End-customer billingSonar
Device telemetry & monitoringLibreNMS / Prometheus / Loki
Physical network designWisdm
Document signaturesDocuSeal
Project task managementMonday.com

NetOS will pull from these systems (Sonar revenue, LibreNMS monitoring links, Wisdm geometry) and push to them (Monday.com renewal tasks, DocuSeal termination notices) — but those systems remain the source of truth for their respective domains.


Data Model

Core Entities

Carrier ── Circuit ── Contract
              │
              ├── Endpoint snapshots copied from selected sites
              ├── LifecycleEvent
              ├── Invoice ── InvoiceLineItem
              └── RenewalDecision

InfrastructureAsset ── SubtendedLink ── Customer
        │                  │
        │                  ├── Circuit cost source
        │                  └── RFLink cost source
        ├── ElectricalService
        └── InfrastructureDocument

Customer ── ElectricalService
RFSystemCatalog ── RFLink ── RFLinkDocument

Circuit Natural Key

(carrier_id, carrier_circuit_id)

This composite key prevents duplicate circuits when importing the same carrier inventory multiple times. If a row matches an existing natural key, the circuit is updated rather than duplicated.

Manual circuit entry should use Infrastructure and Customer site selectors. Imported endpoint fields remain supported for bulk onboarding, carrier inventory loads, and legacy records, but the preferred operating workflow is to create or correct the Infrastructure/Customer site first and then select that site on the circuit.

Key Enums

EnumValues
Service Typewave, epl, evpl, dia, ip_transit, dark_fiber, mpls, cross_connect, cellular_backup, broadband
Statusquoted, ordered, installing, active, suspended, decommissioned
Purposebackhaul, backup, customer_dedicated, internet, dr, management
Redundancyprimary, secondary, standalone
Endpoint Typemdu, tower, data_center, colo, pop, customer_premise, carrier_hotel
Renewal Statestable, watch, active, critical, overdue, decided

CSV/XLSX Import

Import Templates

The import page now has separate downloadable templates:

TemplateUse ForImport Endpoint
/templates/netos-circuit-import-template.csvWholesale circuits, A/Z endpoints, provider details, agreement terms/api/v1/imports/circuits/*
/templates/netos-infrastructure-import-template.csvData centers, towers, rooftops, colocation, POPs, offices, and aggregation facilities/api/v1/imports/infrastructure/*

Use separate files when the records have different lifecycles. Circuits are matched by provider plus carrier circuit ID; infrastructure is matched by Xiber infrastructure label.

Circuit Required Fields

FieldDescription
carrier_nameUpstream provider (e.g., "Lumen", "Zayo")
carrier_circuit_idThe carrier's reference number

These two fields form the circuit natural key. All other fields are optional but recommended.

If xiber_circuit_label is omitted, NetOS generates it using Provider Short Name Service Type A Loc -> Z Loc when the provider has a short name, otherwise Carrier Service Type A Loc -> Z Loc. DIA and broadband use the same pattern without a Z location because they are single-site services. Generated labels are truncated to fit the circuit label field and receive a numeric suffix only when needed for uniqueness.

Recommended Fields

FieldTypeExample
xiber_circuit_labelstringLumen WAVE Site A -> Site Z
service_typeenumwave, dia, dark_fiber, broadband
bandwidth_mbpsnumber10000
statusenumactive
mrc_usddecimal2500.00
nrc_usddecimal5000.00
ban_idstringBAN-100245
billing_accountstringBA-44129
protocol_handoffstring10GBase-LR
monitoring_urlURLhttps://librenms.xiber.local/device/123
install_datedate2024-01-15
a_endpoint_namestringXiber HQ - 123 Main St
a_citystringColumbus
a_statestringOH
z_endpoint_namestringCologix COL1
z_citystringColumbus
z_statestringOH

Contract Fields (Optional)

FieldTypeExample
contract_effective_datedate2024-01-15
contract_term_end_datedate2027-01-14
contract_initial_term_monthsinteger36
contract_renewal_typeenumauto_renew, manual, month_to_month, evergreen
contract_renewal_notice_daysinteger90
contract_msa_terms_texttext36 month initial term...
contract_etf_formula_texttext100% of remaining MRC
contract_price_escalator_pctdecimal3.0
contract_pdf_urlURL/api/v1/circuits/.../agreement/file/...

Infrastructure Fields

Infrastructure imports cover facilities and structures that have their own cost, agreement, and renewal lifecycle. Electrical utility accounts are tracked separately under Electrical Services after the infrastructure or customer site exists.

FieldTypeExample
xiber_infrastructure_labelstringINF-IND-PWR-0012
provider_namestringAES Indiana
site_typestringdata_center, tower, rooftop, colo, pop, office, other
display_namestringIndy MDU 42 Electrical Service
facility_account_idstringMETER-100245
latitude, longitudedecimal39.773350, -86.158050
service_descriptiontextElectrical utility service for network room
power_commitstring400A 120/208V 3 phase
monitoring_urlURLExact monitoring, graph, or utility-status page
mrc_usd, nrc_usddecimal1450.00, 0.00
price_escalator_pctdecimal3.0
effective_date, term_end_datedate2024-01-01, 2024-12-31
msa_terms_text, etf_formula_texttextAgreement language

Manual Bulk Maintenance

After import, users can make broad operational corrections from the UI.

Circuit Bulk Actions

URL: /circuits

Select rows with the checkboxes in the first column. The header checkbox selects all currently visible filtered rows.

Bulk FieldCanonical Field
Statusstatus
Providercarrier_name
Typeservice_type
MRCmrc_usd
NRCnrc_usd
Term Endcontract_term_end_date
Notesnotes

Infrastructure Bulk Actions

URL: /infrastructure

Select rows with the checkboxes in the first column. The header checkbox selects all visible rows.

Bulk FieldCanonical Field
Statusstatus
Providerprovider_name
Typesite_type
MRCmrc_usd
Escalator %price_escalator_pct
Term Endterm_end_date
Notesnotes

Bulk deletes are soft-deletes. They set deleted_at and preserve the record for audit/history.


Import Workflow

StepWhat Happens
1. UploadUser uploads a CSV or XLSX file via /import or the API
2. SignatureRaw file hash is calculated for deduplication
3. MappingHeaders are mapped to canonical fields using heuristic matching
4. StagingAll rows are written to import_staging_rows with validation results
5. ValidationEach row is checked for required fields, type coercion, enum matching
6. ReviewUser sees first 5 rows with mapped columns and any validation errors
7. CommitValid rows are committed; invalid rows are skipped with error details
8. Carrier stubsMissing carriers are auto-created with name only (fill in details later)
9. Endpoint stubsMissing circuit endpoints are auto-created; coordinates are populated when latitude/longitude are present

What Happens on Re-Import

When a row matches an existing circuit by natural key (carrier, carrier_circuit_id), or existing infrastructure by xiber_infrastructure_label:

  • All provided fields are updated to the new values
  • Fields not present in the import are left unchanged
  • This is an upsert, not a replace

Validation Rules

FieldValidation
carrier_nameRequired, non-empty
carrier_circuit_idRequired, non-empty
service_typeCoerced to enum (case-insensitive, underscore-tolerant)
statusCoerced to enum
mrc_usdMust be numeric if provided
nrc_usdMust be numeric if provided
bandwidth_mbpsMust be numeric if provided
install_date, decom_date, contract dates, infrastructure term datesMust be ISO date if provided

Current limitations:

  • No inline editing of staged rows (must fix CSV and re-upload)
  • No diff preview before updating existing circuits
  • No explicit confirmation for financial field overwrites from CSV/XLSX
  • Column mapping is heuristic-only (AI-assisted mapping planned)

API Import Examples

Preview (upload and detect mapping):

curl -H "x-user-role: exec" \
  -F "file=@examples/sample_circuits.csv" \
  http://localhost:8000/api/v1/imports/circuits/preview

Response:

{
  "import_job_id": "abc123",
  "detected_mapping": {
    "Carrier": "carrier_name",
    "Circuit ID": "carrier_circuit_id",
    "MRC": "mrc_usd"
  },
  "headers": ["Carrier", "Circuit ID", "Type", "BW", "MRC", ...],
  "sample_rows": [...],
  "total_rows": 45,
  "valid_rows": 42,
  "invalid_rows": 3
}

Commit (import valid rows):

curl -H "x-user-role: exec" \
  -X POST \
http://localhost:8000/api/v1/imports/circuits/{import_job_id}/commit

Infrastructure imports use the same pattern:

curl -H "x-user-role: exec" \
  -F "file=@netos-infrastructure-import-template.csv" \
  http://localhost:8000/api/v1/imports/infrastructure/preview

curl -H "x-user-role: exec" \
  -X POST \
  http://localhost:8000/api/v1/imports/infrastructure/{import_job_id}/commit

Response:

{
  "inserted": 38,
  "updated": 4,
  "invalid": 3,
  "errors": [
    {"row": 12, "field": "mrc_usd", "error": "not a valid number"},
    {"row": 23, "field": "carrier_name", "error": "required field missing"},
    {"row": 31, "field": "service_type", "error": "unknown value: 'wavelength'"}
  ]
}

Sample CSV

Files:

  • /templates/netos-circuit-import-template.csv
  • /templates/netos-infrastructure-import-template.csv
carrier_name,carrier_circuit_id,xiber_circuit_label,service_type,bandwidth_mbps,status,mrc_usd,nrc_usd,a_endpoint_name,a_city,a_state,z_endpoint_name,z_city,z_state
Lumen,DHEC.123456,XIB-LUM-0001,wave,10000,active,2500.00,5000.00,Xiber HQ,Columbus,OH,Cologix COL1,Columbus,OH
Zayo,ZY-789012,XIB-ZAY-0001,dark_fiber,100000,active,4200.00,15000.00,56 Marietta St NW,Atlanta,GA,Equinix AT1,Atlanta,GA

Common circuit purpose values include backhaul, backup, customer_dedicated, internet, dr, and management. Use backup for MFC backup circuits, such as broadband service backing up a primary fiber or RF path.


Endpoint Coordinates

Map rendering and geographic topology require endpoint coordinates.

FieldTypeFormat
endpoints.geomPostGIS POINTSRID 4326 (WGS84 lat/lng)

Coordinates are stored as PostGIS geometry. The seed data includes coordinates for all sample endpoints. When importing circuits, endpoints are auto-created as stubs without coordinates — add coordinates later via the API or UI to make them appear on the map.


Contract Data

Contracts are linked to circuits and/or infrastructure assets. Key fields:

FieldDescription
carrierProvider this contract is with
contract_typeMaster, circuit-specific, facility
parent_contractLink to master agreement (if applicable)
effective_dateWhen the contract began
initial_term_monthsOriginal commitment length
term_end_dateCalculated end date
renewal_typeauto, manual, evergreen
renewal_term_monthsLength of auto-renewal period
renewal_notice_daysDays before end to provide notice
renewal_decision_deadlineCalculated: term_end - notice_days
price_escalatorAnnual price increase formula
etf_formulaEarly termination fee calculation
etf_current_valueCurrent estimated ETF
governing_lawJurisdiction
pdf_urlLink to signed contract PDF

Current state: Sample contracts are created by the seed script. PDF extraction and side-by-side approval are not yet implemented. See Roadmap for the PDF ingestion plan.