add backlog document

Network establishment

BACKLOG.md

@@ -0,0 +1,174 @@
+# gsio-net Backlog
+
+This document outlines the current tasks, planned features, known issues, and potential improvements for the GSIO-Net project. Items are prioritized based on their importance and alignment with the project's goals. Subject to change without notice.
+
+## Current Pending Tasks
+
+1. **User Personas Development** (Priority: High)
+   - ~~Hypothesize personas~~
+   - ~~Complete the PERSONAS_BLUEPRINT.md document with detailed user profiles (TinyTroupe)~~
+   - ~~Validate personas with stakeholders~~
+   - Run Tinytroupe on existing documentation 
+   - ~~Finalize personas~~
+
+2. **Project Documentation** (Priority: High)
+   - ~~Enhance README.md with comprehensive project description, architecture overview, and setup instructions~~ 
+   - Document existing Rust code
+   - Complete project document blueprints 
+     - ~~[PROBLEM_STATEMENT_BLUEPRINT.md](project/PROBLEM_STATEMENT_BLUEPRINT.md)~~
+     - ~~[HUMAN_FACTORS_BLUEPRINT.md](project/HUMAN_FACTORS_BLUEPRINT.md)~~
+     - ~~[PERSONAS_BLUEPRINT.md](project/PERSONAS_BLUEPRINT.md)~~
+     - ~~[SECURITY_BLUEPRINT.md](project/SECURITY_BLUEPRINT.md)~~
+     - ~~[STORY_MAPPING_BLUEPRINT.md](project/STORY_MAPPING_BLUEPRINT.md)~~
+     - ~~[CRITIQUE.md](project/CRITIQUE.md)~~
+
+
+### Trust & Provenance Backlog
+
+1. **Ledger Persistence** (Priority: Critical)
+   - Implement persistent storage for ledger entries
+   - Add data recovery mechanisms
+   - Ensure data integrity across restarts
+
+2. **Enhanced Validation** (Priority: High)
+   - Implement multi-signature validation for ledger entries
+   - Add cryptographic verification of entry content
+   - Create validation rules engine
+
+3. **Audit Trail** (Priority: Medium)
+   - Implement comprehensive audit logging
+   - Create audit report generation
+   - Add tamper-evident audit trails
+
+### Edge-Cloud Performance Backlog
+
+1. **Advanced P2P Networking** (Priority: High)
+   - Complete Iroh integration for improved peer discovery
+   - Implement NAT traversal techniques
+   - Add bandwidth optimization for constrained environments
+
+2. **Edge Optimization** (Priority: High)
+   - Implement local-first operations
+   - Add offline operation support
+   - Create intelligent sync strategies for intermittent connectivity
+
+3. **Performance Monitoring** (Priority: Medium)
+   - Add comprehensive metrics collection
+   - Implement performance dashboards
+   - Create alerting for performance degradation
+
+### Autonomous System Governance Backlog
+
+1. **Consensus Mechanism** (Priority: Critical)
+   - Implement robust consensus algorithm
+   - Add conflict resolution strategies
+   - Create governance rules for network operation
+
+2. **Explainable Operations** (Priority: High)
+   - Add operation tracing
+   - Implement decision logging
+   - Create visualization tools for system behavior
+
+3. **Controlled Rollback** (Priority: Medium)
+   - Implement safe rollback mechanisms
+   - Add checkpoint system
+   - Create recovery procedures
+
+### Regulatory Compliance Backlog
+
+1. **Policy Engine** (Priority: High)
+   - Implement policy-as-code framework
+   - Add jurisdiction-aware rule processing
+   - Create compliance reporting tools
+
+2. **Data Privacy** (Priority: High)
+   - Implement data encryption at rest and in transit
+   - Add access control mechanisms
+   - Create data minimization strategies
+
+3. **Automated Reporting** (Priority: Medium)
+   - Implement report generation for compliance requirements
+   - Add scheduled reporting
+   - Create audit-ready data exports
+
+### Human-System Integration Backlog
+
+1. **Client Libraries** (Priority: High)
+   - Complete gsio-client implementation
+   - Add language-specific SDKs
+   - Create comprehensive client documentation
+
+2. **Developer Tools** (Priority: Medium)
+   - Implement CLI tools for network interaction
+   - Add development environment setup scripts
+   - Create debugging and monitoring tools
+
+3. **User Interfaces** (Priority: Medium)
+   - Implement admin dashboard
+   - Add visualization tools for network activity
+   - Create user-friendly client applications
+
+## Critical Issue Backlog
+
+1. **Ledger Implementation** (Priority: High)
+   - In-memory only storage lacks persistence
+   - Limited validation (only checks hash correctness)
+   - No conflict resolution mechanism
+
+2. **P2P Networking** (Priority: High)
+   - Limited error handling and recovery
+   - No explicit security measures for node authentication
+   - Basic synchronization mechanism (full ledger sync)
+
+3. **Relay Component** (Priority: Medium)
+   - Very basic implementation (echo server only)
+   - No integration with ledger or p2p components
+   - No security measures
+   - iroh ecosystem lock-in
+
+## Improvements Backlog
+
+1. **Architecture** (Priority: High)
+   - Create detailed architecture documentation
+   - Implement modular plugin system
+   - Add more service discovery mechanisms
+
+2. **Testing** (Priority: High)
+   - Increase test coverage
+   - Add integration tests
+   - Implement performance benchmarks
+
+3. **Security** (Priority: Critical)
+   - Conduct security audit
+   - Implement secure node authentication
+   - Add encryption for all communications
+
+4. **Scalability** (Priority: Medium)
+   - Optimize for high-volume ledger operations
+   - Implement sharding for large networks
+   - Add load balancing for relay nodes
+
+5. **Usability** (Priority: Medium)
+   - Improve error messages and handling
+   - Add comprehensive logging
+   - Create user-friendly documentation
+
+## Roadmap
+
+### Phase 1: Networking + Edge
+- Ledger persistence implementation
+- Enhanced validation
+- Advanced P2P networking
+- Edge optimization
+
+### Phase 2: Autonomy + UX
+- Consensus mechanism
+- Explainable operations
+- Client libraries
+- Developer tools
+
+### Phase 3: Regulation + Cohesion
+- Policy engine
+- Data privacy
+- Automated reporting
+- System integration and cohesion

Cargo.lock

@@ -2,6 +2,20 @@
 # It is not intended for manual editing.
 version = 4
 
+[[package]]
+name = "acto"
+version = "0.7.4"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "a026259da4f1a13b4af60cda453c392de64c58c12d239c560923e0382f42f2b9"
+dependencies = [
+ "parking_lot",
+ "pin-project-lite",
+ "rustc_version",
+ "smol_str",
+ "tokio",
+ "tracing",
+]
+
 [[package]]
 name = "addr2line"
 version = "0.24.2"
@@ -1590,6 +1604,7 @@ dependencies = [
  "futures",
  "iroh",
  "iroh-blobs",
+ "iroh-relay",
  "libp2p",
  "rmpv",
  "serde",
@@ -1600,6 +1615,7 @@ dependencies = [
  "tower-http",
  "tracing",
  "tracing-subscriber",
+ "url",
  "uuid",
 ]
 
@@ -2338,6 +2354,7 @@ dependencies = [
  "strum",
  "stun-rs",
  "surge-ping",
+ "swarm-discovery",
  "thiserror 2.0.12",
  "time",
  "tokio",
@@ -5235,6 +5252,12 @@ dependencies = [
  "serde",
 ]
 
+[[package]]
+name = "smol_str"
+version = "0.1.24"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "fad6c857cbab2627dcf01ec85a623ca4e7dcb5691cbaa3d7fb7653671f0d09c9"
+
 [[package]]
 name = "snafu"
 version = "0.8.6"
@@ -5483,6 +5506,21 @@ dependencies = [
  "tracing",
 ]
 
+[[package]]
+name = "swarm-discovery"
+version = "0.3.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d3a95032b94c1dc318f55e0b130e3d2176cda022310a65c3df0092764ea69562"
+dependencies = [
+ "acto",
+ "anyhow",
+ "hickory-proto 0.25.2",
+ "rand 0.8.5",
+ "socket2",
+ "tokio",
+ "tracing",
+]
+
 [[package]]
 name = "syn"
 version = "1.0.109"

README.md

@@ -4,12 +4,11 @@
 
 Warning: This API is unstable.
 
-## run a network 
-```yaml
+```shell
 docker compose build
 docker compose up
 ```
 
 ## License
 
-This project is licensed under the MIT License - see the LICENSE file for details.
+This project is licensed under the MIT License - See [LICENSE](./LICENSE) for details.

crates/gsio-node/Cargo.toml

@@ -22,5 +22,7 @@ serde_json = "1.0"
 uuid = { version = "1.7.0", features = ["v4", "serde"] }
 chrono = { version = "0.4.35", features = ["serde"] }
 sha2 = "0.10.8"
-iroh = { version = "0.35.0", features = ["discovery-pkarr-dht"] }
+iroh = { version = "0.35.0", features = ["discovery-pkarr-dht", "discovery-local-network"] }
 iroh-blobs = { version = "0.35.0", features = ["rpc"] }
+url = "2.5.4"
+iroh-relay = "0.35.0"

crates/gsio-node/src/main.rs

@@ -6,7 +6,7 @@
 // - Each node is an autonomous sync unit
 
 use axum::{routing::get, Router};
-use iroh::{protocol::Router as IrohRouter, Endpoint};
+use iroh::{protocol::Router as IrohRouter, Endpoint, RelayMap, RelayMode, RelayUrl};
 use iroh_blobs::{
     net_protocol::Blobs,
     rpc::client::blobs::MemClient,
@@ -30,6 +30,9 @@ mod p2p;
 
 use ledger::{LedgerEntry, SharedLedger};
 use p2p::P2PManager;
+use url::Url;
+
+// assuming 'localhost' resolves to 127.0.0.1
 
 /// ========== Socket.io namespace helpers ==========
 fn register_root_namespace(io: &SocketIo, p2p: Arc<P2PManager>) {
@@ -355,8 +358,22 @@ async fn handle_blob_available(socket: SocketRef, p2p: Arc<P2PManager>, data: &J
 async fn main() -> Result<(), Box<dyn std::error::Error>> {
     tracing::subscriber::set_global_default(FmtSubscriber::default())?;
 
+
+    let relay_address = std::env::var("RELAY_ADDRESS").expect("RELAY_ADDRESS must be set");
+
+
+    let relay_url = RelayUrl::from_str(&*relay_address).unwrap();
+
+
+    let relays = RelayMap::from(relay_url);
+
+
+    
     // --- IROH SETUP --------------------------------------------------------
-    let endpoint = Endpoint::builder().discovery_n0().bind().await?;
+    let endpoint = Endpoint::builder().discovery_n0()
+        .relay_conn_protocol(iroh_relay::http::Protocol::Websocket)
+        .discovery_local_network()
+        .relay_mode(RelayMode::Custom(relays)).bind().await?;
     // Concrete store type inferred from the builder
     let blobs = Arc::new(Blobs::memory().build(&endpoint));
     let router = IrohRouter::builder(endpoint.clone())

crates/gsio-relay/Dockerfile

@@ -0,0 +1,33 @@
+################################################################
+#  Stage 1 – build rustls-cert-gen and generate the certs
+################################################################
+FROM rust:bookworm AS ssl-step
+
+# ↓ Allow override of SAN / output directory at build time
+ARG CERT_DOMAIN=relay.local
+ARG OUT_DIR=/app/ssl
+
+# ── deps we need only for the build ───────────────────────────
+RUN apt-get update -qq && \
+    apt-get install -y --no-install-recommends git ca-certificates && \
+    rm -rf /var/lib/apt/lists/*
+
+# ── fetch the rcgen repo (contains the CLI) and build once ────
+WORKDIR /src
+RUN git clone --depth 1 https://github.com/rustls/rcgen.git
+WORKDIR /src/rcgen
+RUN cargo run -- -o /app/ssl
+
+################################################################
+#  Stage 2 – minimal runtime with the iroh relay
+################################################################
+FROM n0computer/iroh-relay:v0.28.2
+
+# copy the certs produced in stage 1
+COPY --from=ssl-step /app/ssl /app/ssl
+
+# your relay configuration
+COPY ./relay-config.toml /app/
+
+# hand off control to the relay
+CMD ["--dev"]

crates/gsio-relay/relay-config.toml

@@ -0,0 +1 @@
+stun_only = false

docker-compose.yml

@@ -1,25 +1,25 @@
-version: '3.8'
-# GSIO-Net Docker Compose Configuration
-# 
-# This file defines a network of GSIO-Net nodes that can communicate with each other.
-# It creates three nodes, each exposing the API on a different host port:
-#   - node1: http://localhost:3001
-#   - node2: http://localhost:3002
-#   - node3: http://localhost:3003
-#
-# Usage:
-#   - Start the network: docker-compose up -d
-#   - View logs: docker-compose logs -f
-#   - Stop the network: docker-compose down
-#   - Stop and remove volumes: docker-compose down -v
-
 services:
+  relay:
+    container_name: gsio-relay
+    build:
+      context: ./crates/gsio-relay
+      dockerfile: Dockerfile
+      args:
+        CERT_DOMAIN: "gsio-relay."
+    networks:
+      - gsio-network
+    ports:
+      - "3340:3340"
+      - "7824:7824"
+
   # Node 1
   node1:
     build:
       context: .
       dockerfile: Dockerfile
     container_name: gsio-node1
+    environment:
+      RELAY_ADDRESS: "ws:///gsio-relay:3340"
     ports:
       - "3001:3000"  # Map to different host ports to avoid conflicts
     volumes:
@@ -40,6 +40,8 @@ services:
       context: .
       dockerfile: Dockerfile
     container_name: gsio-node2
+    environment:
+      RELAY_ADDRESS: "ws:///gsio-relay:3340"
     ports:
       - "3002:3000"
     volumes:
@@ -60,6 +62,8 @@ services:
       context: .
       dockerfile: Dockerfile
     container_name: gsio-node3
+    environment:
+      RELAY_ADDRESS: "ws:///gsio-relay:3340"
     ports:
       - "3003:3000"
     volumes:
@@ -83,4 +87,4 @@ volumes:
 # Define a custom network for the nodes to communicate
 networks:
   gsio-network:
-    driver: bridge
+    name: gsio-network