documentation md files

documentation/backend/source-reference.md

@@ -0,0 +1,70 @@
+# Source reference
+
+This index covers every Python module under `backend/src`, including helper and
+unmounted-router code. Function names prefixed with `_` are private implementation
+details; they are described by their owning module's responsibility rather than as
+separate public contracts.
+
+## Application and configuration
+
+| Module | Public surface and role |
+| --- | --- |
+| `main.py` | Creates FastAPI, enables permissive CORS, registers startup/shutdown handlers, provides `/hello` and `/versions`, includes live routers, and registers script lifecycle hooks. |
+| `config.py` | Parses environment strings/integers/floats/booleans; immutable `BackendSettings`; `load_settings`; module-global `settings`. See [configuration](configuration.md). |
+| `shared_objects.py` | Process-global `db`, `web_loop`, packet broadcaster, and network broadcaster references initialized by `main`. |
+
+## Models
+
+| Module | Public surface and role |
+| --- | --- |
+| `Models/packets.py` | `PacketObservationModel` and `PacketDBModel`, the normalized persisted/API packet schemas. |
+| `Models/ip_protocol.py` | `IPProtocolEnum` and `protocol_from_number`, translating IANA protocol numbers to labels. |
+| `Models/etherType.py` | `EtherTypeEnum` and `ethertype_from_int`, translating Ethernet type values to labels. |
+| `Models/netplan.py` | `Nameservers`, `EthernetConfig`, `BridgeConfig`, and `NetworkConfig` Pydantic schemas for Netplan-shaped network data. |
+
+## API routers
+
+| Module | Public surface and role |
+| --- | --- |
+| `api/network_api.py` | Network inspection, bridge create/remove, default reset, link-state watcher control, and network-state WebSocket. Holds shared `IPRoute`/`NDB`; converts netlink messages to Pydantic interface/route/bridge models; publishes state after mutations. |
+| `api/sniffer_api.py` | Pydantic start/stop/status models and endpoints. Validates one capture target and calls the capture-session API. |
+| `api/packet_api.py` | Latest packet retrieval, packet-history deletion, and packet-update WebSocket. Serialization handles database records and Pydantic values safely for JSON. |
+| `api/analysis_api.py` | Pydantic evidence/response models for attachment, protocols, paths, conversations, flow detail, hosts, discovery and anomaly views; delegates each endpoint to `DatabasePool`. |
+| `api/nft_manager.py` | **Mounted.** `NftManager` wrapper, normalized ruleset models and functions to list rules, delete by handle, and run textual nft. It parses JSON and textual output to enrich rule data. |
+| `api/packet_scripting_api.py` | **Mounted with doubled prefix.** Name/path validation, example deployment, systemd unit management, venv/pip operations, script status models, and upload/download/enable/disable/delete endpoints. |
+| `api/nft_api.py` | **Not mounted.** Bridge nftables typed expression model, command generator, handle mapping, and CRUD/preview endpoint functions. `RuleModel.only_bridge` rejects other families. |
+| `api/nftables_api.py` | **Not mounted.** Generic typed match/action models, resilient binding/CLI wrapper selection, rule reconstruction, and list/replace endpoint functions. |
+
+## Capture, telemetry, and broadcasting utilities
+
+| Module | Public surface and role |
+| --- | --- |
+| `network_sniffer.py` | Defines flexible `PacketInfo`; parses packet objects/bytes; opens/closes AF_PACKET sockets; owns session reader loops; coordinates telemetry; exposes `start_capture_session`, `stop_capture_session`, status and debug accessors. Legacy `*_afpacket_sniffer` functions delegate to current session functions. |
+| `utilities/packet_tracker.py` | `PacketTracker` observes capture or telemetry events, aggregates observations, schedules persistence, stops/discards state, and exposes diagnostics. The module-global tracker is the correlation entrypoint. |
+| `utilities/packet_identity.py` | Builds deterministic fallback packet UID and the minimum fields used to calculate it. |
+| `utilities/packet_mark.py` | Decodes numeric skb marks into a normalized mark, packet ID, and verdict hint according to the shared mark layout. |
+| `utilities/tshark_manager.py` | `TsharkManager` owns optional worker processes and caches. Parsing helpers safely coerce nested tshark JSON, extract protocol/HTTP/TLS/DNS/TCP data, derive stream context, and merge enrichment. Module-global `tshark_manager` is used by capture. |
+| `utilities/bridge_telemetry.py` | `BridgeTelemetryManager` starts/reconciles/stops the eBPF helper and reports subprocess/queue state. Module-global manager is invoked by sniffer lifecycle. |
+| `utilities/ebpf_bridge_events.py` | Standalone helper program: ctypes event format, BPF-source construction, tc attach/cleanup, perf callbacks, JSON output, signal handling, and `main`. |
+| `utilities/packet_broadcaster.py` | `PacketBroadcaster` manages subscriber queues. `subscribe`/`unsubscribe`, async `publish`, cross-thread `sync_publish`, and async `close` provide the WebSocket transport primitive. |
+
+## Network and persistence utilities
+
+| Module | Public surface and role |
+| --- | --- |
+| `utilities/interface_bridge_helpers.py` | Interface existence/up tests; sysfs readers for operational/carrier/admin/MTU state; Ethernet profile retrieval/cache; bridge-port discovery. |
+| `utilities/bridge_link_state_manager.py` | `EthernetProfile` and `MemberLinkState` data objects; `BridgeLinkStateWatcher` start/stop/status; `BridgeLinkStateManager` enable/disable/query/stop. It embodies debounce, failure, recovery, and profile propagation logic. |
+| `utilities/database.py` | `DatabasePool` initialization/closure, upsert/batch-upsert, enrichment backfills, latest-packet query, all analysis SQL, and history clearing. Internal helpers normalize values, derive protocol/flow/analysis fields, serialize outgoing rows, and classify discovery activity. |
+
+## Extension points and maintenance notes
+
+- New API functionality should live in an `APIRouter`, use Pydantic request and
+  response models, and be explicitly included from `main.py`; otherwise it is not
+  live.
+- New capture fields must be updated consistently in packet parsing, tracker merge,
+  database upsert SQL, `PacketDBModel`, broadcaster serialization, and analysis
+  queries where relevant.
+- Any new Linux side effect belongs in [host integration](host-integration.md),
+  including required binary/capability, rollback behavior, and its API exposure.
+- If an unmounted nft router is adopted, document the migration and remove or
+  version conflicting endpoints instead of silently mounting another implementation.