Documentation

Overview

Space Data Network (SDN) is a decentralized peer-to-peer network for exchanging standardized space situational awareness data. Built on libp2p with FlatBuffers serialization, SDN enables real-time sharing of orbital data, conjunction warnings, and tracking information.

Network Ecosystem

The interactive sandbox below shows providers publishing signed SDS data, PNMs announcing current artifacts, DPMs describing authoritative product sets, PLG module listings and delivery, peer subscriptions, signature verification, encryption grants, and pinning, archive, and consumer nodes moving artifacts by CID.

Network Components

Quick Start

# Clone and build from source
git clone https://github.com/DigitalArsenal/space-data-network.git
cd space-data-network
npm run install:all
npm run desktop

# Initialize configuration
./spacedatanetwork init

# Start the node
./spacedatanetwork daemon

import { SDNNode } from './sdn-js/dist/esm/index.js';

const node = new SDNNode();
await node.start();

// Subscribe to Orbital Mean-Elements Messages
node.subscribe('OMM', (data, peerId) => {
  console.log(`Received OMM from ${peerId}:`, data);
});

// Publish data
await node.publish('OMM', {
  OBJECT_NAME: 'ISS (ZARYA)',
  OBJECT_ID: '1998-067A',
  EPOCH: '2025-01-24T12:00:00.000Z',
  MEAN_MOTION: 15.5,
  ECCENTRICITY: 0.0001,
  INCLINATION: 51.6,
  RA_OF_ASC_NODE: 200.0,
  ARG_OF_PERICENTER: 100.0,
  MEAN_ANOMALY: 50.0
});

Installation

JavaScript SDK

cd sdn-js
npm install

Native Releases

Use the one-line installers for the self-contained CLI. They install under the current user's ~/.spacedatanetwork directory by default, initialize the local identity, and do not require administrator privileges.

# macOS/Linux
curl -fsSL https://spacedatanetwork.org/install.sh | bash

# Windows PowerShell
irm https://spacedatanetwork.org/install.ps1 | iex

CLI Operations

The first-version CLI and Desktop use the same identity, lifecycle, search, and signed update contracts. The CLI can manage the persistent user-scoped service, remove the active installed bundle while preserving data by default, search the same provider/data indexes as the UI, export public identity records in every supported format, and update through the SDN-owned provider feed.

# Persistent service lifecycle
spacedatanetwork start
spacedatanetwork stop
spacedatanetwork restart
spacedatanetwork service status
spacedatanetwork service install
spacedatanetwork service uninstall

# Remove the active install; data purge is explicit
spacedatanetwork remove --dry-run
spacedatanetwork remove --purge-data

# Provider, standard, and data search
spacedatanetwork search providers celestrak --schema OMM
spacedatanetwork search standards OMM --format json
spacedatanetwork search data --schema CAT --provider-id space-data-network-02 --format csv

# EPM/vCard identity setup and export
spacedatanetwork identity wizard
spacedatanetwork identity export --format flatbuffer --output epm.fbs
spacedatanetwork identity export --format qrcode

# SDN-owned signed updates
spacedatanetwork update check
spacedatanetwork update stage
spacedatanetwork update apply

Identity export supports text, JSON, CSV, FlatBuffer, and QR code output. The signed update provider is rooted at updates.spacedatanetwork.org; running-daemon updates are staged, applied in place, health checked, and rolled back on failure.

Native artifacts are attached to versioned SDN release tags, not SDK-only tags such as sdn-js-v.... Release v1.0.3-beta.1 currently publishes self-contained CLI archives, Desktop installers, Linux amd64 full-node packages, a Docker image tarball, a Linux VM bundle, and a macOS ARM64 full-node bundle.

Versioned Release

Pick the newest numbered release from Space Data Network v1.0.3-beta.1 release, then use that release number for package and SDK downloads.

SDN_VERSION=v1.0.3-beta.1
SDN_NATIVE_PACKAGE_VERSION=1.0.3.beta.1

# Full node package
curl -LO "https://github.com/DigitalArsenal/space-data-network/releases/download/${SDN_VERSION}/spacedatanetwork-full_${SDN_NATIVE_PACKAGE_VERSION}_amd64.deb"
curl -LO "https://github.com/DigitalArsenal/space-data-network/releases/download/${SDN_VERSION}/spacedatanetwork-full-${SDN_NATIVE_PACKAGE_VERSION}-1.x86_64.rpm"

# Linux VM bundle
curl -LO "https://github.com/DigitalArsenal/space-data-network/releases/download/${SDN_VERSION}/spacedatanetwork-linux-vm-${SDN_NATIVE_PACKAGE_VERSION}.tar.gz"

# Docker image tarball
curl -LO "https://github.com/DigitalArsenal/space-data-network/releases/download/${SDN_VERSION}/spacedatanetwork-container-${SDN_NATIVE_PACKAGE_VERSION}-linux-amd64.tar.gz"
docker load --input "spacedatanetwork-container-${SDN_NATIVE_PACKAGE_VERSION}-linux-amd64.tar.gz"
docker pull dockerdigitalarsenal/space-data-network:${SDN_VERSION}

# macOS ARM64 full-node bundle
curl -LO "https://github.com/DigitalArsenal/space-data-network/releases/download/${SDN_VERSION}/spacedatanetwork-darwin-arm64.tar.gz"

# Windows AMD64 self-contained CLI archive
curl -LO "https://github.com/DigitalArsenal/space-data-network/releases/download/${SDN_VERSION}/spacedatanetwork-1.0.3-beta.1-windows-amd64.zip"

# Desktop app installers
curl -LO "https://github.com/DigitalArsenal/space-data-network/releases/download/${SDN_VERSION}/space-data-network-desktop-0.47.0-mac.dmg"
curl -LO "https://github.com/DigitalArsenal/space-data-network/releases/download/${SDN_VERSION}/space-data-network-desktop-setup-0.47.0-windows-x64.exe"
curl -LO "https://github.com/DigitalArsenal/space-data-network/releases/download/${SDN_VERSION}/space-data-network-desktop-0.47.0-linux-x86_64.AppImage"

# Browser and Node JavaScript SDK tarball
curl -LO "https://github.com/DigitalArsenal/space-data-network/releases/download/${SDN_VERSION}/spacedatanetwork-sdn-js-<sdk-version>.tgz"

Full Node Setup

Run a full SDN node to contribute to the network's routing, relay traffic for firewalled peers, and store pinned content.

Requirements

• Public IP address - Required for other peers to connect directly
• Open ports: 4001/tcp (libp2p), 4001/udp (QUIC), 8080/tcp (HTTP API)
• Storage: Minimum 1GB recommended
• Memory: 512MB minimum, 2GB+ for busy nodes

Configuration

# Initialize default config
./spacedatanetwork init

This creates ~/.spacedatanetwork/ with:

• config.yaml — Node configuration (YAML format)
• keys/mnemonic — BIP-39 mnemonic, encrypted at rest with XChaCha20-Poly1305 (see Key Security)
• keys/node.key — Serialized libp2p private key (backward compatibility)
• data/ — FlatSQL record streams (append-only FlatBuffer bytes, plus SQL metadata/index tables) and, if a local Kubo node is configured, its pinned content

Configuration File

The server uses YAML configuration at ~/.spacedatanetwork/config.yaml (override with --config flag).

mode: full

network:
  listen:
    - /ip4/0.0.0.0/tcp/4001
    - /ip4/0.0.0.0/tcp/8080/ws
    - /ip4/0.0.0.0/udp/4001/quic-v1
  bootstrap:
    - /dnsaddr/bootstrap.spacedatanetwork.org/p2p/16Uiu2HAm1LbvwjEHW2GDP2ZQZvwHLZrz2jbYoRLQmJEQ3wZ5Fm45
    - /dnsaddr/bootstrap.spacedatanetwork.org/p2p/16Uiu2HAm9oK2jAeVC2RMESFcYfq7BKGp2K2CCDxzoKhB5s9vpbj3
    - /ip4/159.203.150.8/tcp/4001/p2p/16Uiu2HAm1LbvwjEHW2GDP2ZQZvwHLZrz2jbYoRLQmJEQ3wZ5Fm45
    - /ip4/167.172.219.213/tcp/4001/p2p/16Uiu2HAm9oK2jAeVC2RMESFcYfq7BKGp2K2CCDxzoKhB5s9vpbj3
  enable_relay: true
  max_connections: 1000

storage:
  path: /home/sdn/.spacedatanetwork/data
  max_size: 10GB
  gc_interval: 1h

security:
  # key_password: ""            # Optional: explicit mnemonic password
                                # Default: machine-derived via Argon2
                                # Override: SDN_KEY_PASSWORD env var

admin:
  enabled: true
  listen_addr: "0.0.0.0:443"
  require_auth: true
  session_expiry: 24h
  tls_enabled: true
  tls_cert_file: /etc/ssl/certs/your-cert.pem
  tls_key_file: /etc/ssl/private/your-key.pem
  ipfs_api_url: "http://127.0.0.1:5002"

# HD wallet users (xpub-based authentication)
users:
  - xpub: "xpub6..."
    trust_level: admin
    name: Operator

Running as a Service

Create /etc/systemd/system/spacedatanetwork.service:

[Unit]
Description=Space Data Network Node
After=network.target

[Service]
Type=simple
User=sdn
ExecStart=/usr/local/bin/spacedatanetwork daemon
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target

sudo systemctl enable spacedatanetwork
sudo systemctl start spacedatanetwork

CLI Commands

The spacedatanetwork binary provides several subcommands for node management and identity inspection.

daemon

Start the SDN server node (default command).

spacedatanetwork daemon --config /path/to/config.yaml

show-identity

Display the node’s cryptographic identity derived from its BIP-39 mnemonic. Prints PeerID to stdout (for scripting) and full details to stderr.

spacedatanetwork show-identity \
  --config /path/to/config.yaml \
  --wasm /path/to/hd-wallet.wasm

Example output:

$ spacedatanetwork show-identity --wasm /opt/sdn/wasm/hd-wallet.wasm

=== SDN Node Identity ===
XPub:            xpub6DKCyLbCHZLFR...FZ3Wa
PeerID:          16Uiu2HAm1LbvwjEH...Fm45
Signing Key:     0d80e1fd5f...42fc8c (Ed25519)
Encryption Key:  08ea56d043...191741 (X25519)

Derivation Paths:
  Identity:   m/44'/0'/0'        (secp256k1 → PeerID + XPub)
  Signing:    m/44'/0'/0'/0'/0'  (Ed25519, SLIP-10)
  Encryption: m/44'/0'/0'/1'/0'  (X25519, SLIP-10)

# Get just the PeerID for scripting:
PEER_ID=$(spacedatanetwork show-identity --wasm /opt/sdn/wasm/hd-wallet.wasm 2>/dev/null)

derive-xpub

Derive an xpub from a BIP-39 mnemonic for use in the users config section. The xpub is the node’s public identity — safe to share and used for authentication.

spacedatanetwork derive-xpub --wasm /path/to/hd-wallet.wasm

You will be prompted to enter a mnemonic. The derived xpub is printed to stdout.

init

Initialize a new SDN data directory with default configuration and a fresh BIP-39 mnemonic (encrypted at rest).

spacedatanetwork init --config /path/to/config.yaml

Edge Relay Deployment

Edge relays bridge browser clients (which can only use WebSocket/WebRTC) to the full libp2p network.

Command Line Options

Production with TLS (nginx)

server {
    listen 443 ssl;
    server_name relay.example.com;

    ssl_certificate /path/to/cert.pem;
    ssl_certificate_key /path/to/key.pem;

    location /ws {
        proxy_pass http://localhost:8080;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_read_timeout 86400;
    }
}

Client Configuration

import { SDNNode } from './sdn-js/dist/esm/index.js';

const node = new SDNNode({
  relays: [
    'wss://relay.digitalarsenal.io/ws',
    'wss://relay2.digitalarsenal.io/ws'
  ],
  relayStrategy: 'failover' // or 'round-robin'
});

await node.start();

Docker Deployment

Full Node

docker run -d \
  --name spacedatanetwork \
  -p 4001:4001 \
  -p 4001:4001/udp \
  -p 5001:5001 \
  -v sdn-data:/app/data \
  dockerdigitalarsenal/space-data-network:v1.0.3-beta.1

Edge Relay (Kubernetes)

apiVersion: apps/v1
kind: Deployment
metadata:
  name: sdn-edge
spec:
  replicas: 3
  selector:
    matchLabels:
      app: sdn-edge
  template:
    spec:
      containers:
      - name: edge
        image: dockerdigitalarsenal/space-data-network:v1.0.3-beta.1
        command: ["/app/spacedatanetwork-edge"]
        args: ["--listen", "/ip4/0.0.0.0/tcp/8080/ws", "--health-port", "8081"]
        ports:
        - containerPort: 8080
        resources:
          requests:
            memory: "256Mi"
            cpu: "250m"
---
apiVersion: v1
kind: Service
metadata:
  name: sdn-edge
spec:
  type: LoadBalancer
  sessionAffinity: ClientIP
  ports:
  - port: 443
    targetPort: 8080
  selector:
    app: sdn-edge

JavaScript SDK

Installation

cd sdn-js
npm install

Browser (ES Modules)

<script type="module">
  import { SDNNode } from './sdn-js/dist/esm/index.js';

  const node = new SDNNode();
  await node.start();
</script>

Browser (UMD)

<script src="./sdn-js/dist/umd/sdn.min.js"></script>
<script>
  const node = new SDN.SDNNode();
  node.start().then(() => console.log('Connected!'));
</script>

SDNNode Options

Methods

// Start/stop
await node.start();
await node.stop();

// Subscribe to schema
node.subscribe('OMM', (data, peerId, signature) => {
  console.log(data.OBJECT_NAME);
});

// Unsubscribe
node.unsubscribe('OMM');

// Publish data
await node.publish('OMM', { ...ommData });

// Get connected peers
const peers = await node.getPeers();

// Get network stats
const stats = await node.getStats();

Events

node.on('connected', () => console.log('Connected'));
node.on('disconnected', () => console.log('Disconnected'));
node.on('peer:connect', (peerId) => console.log(`Peer: ${peerId}`));
node.on('error', (error) => console.error(error));

Storefront & Encrypted Module Delivery

Third-party browser and Node applications can discover a PLG marketplace listing, purchase access, request encrypted WASM delivery, decrypt locally, load the module through the SDK browser harness, and invoke it using only public sdn-js package exports. No src/* imports or helper service is required.

Public Package Surfaces

End-to-End Browser Flow

import { SDNNode, identityFromMnemonic, initHDWallet } from '@spacedatanetwork/sdn-js';
import {
  decryptEncryptedModuleBundle,
  invokeLoadedModule,
  loadDecryptedModule,
  loadMarketplaceListingsFromServer,
  unwrapGrantContentKey,
} from '@spacedatanetwork/sdn-js/ui';
import { PaymentMethod, createStorefrontClient } from '@spacedatanetwork/sdn-js/storefront';

await initHDWallet();
const identity = await identityFromMnemonic(mnemonic);
const node = await SDNNode.create({ identity, enableRelayProbing: true });

const listings = await loadMarketplaceListingsFromServer('https://spaceaware.io');
const listing = listings.find((entry) => entry.pluginId === 'com.example.analytics');

const storefront = createStorefrontClient({
  apiBaseUrl: 'https://spaceaware.io',
  peerId: identity.peerId,
  encryptionPubkey: identity.encryptionKey.publicKey,
  keyAlgorithm: 'x25519',
});

const purchase = await storefront.createPurchase({
  listingId: listing.pluginId,
  tierName: 'default',
  paymentMethod: PaymentMethod.SDNCredits,
  encryptionPubkey: identity.encryptionKey.publicKey,
  keyAlgorithm: 'x25519',
  preferredDeliveryMethod: 'IPFSPin',
});
await storefront.payWithCredits(purchase.requestId);

const delivery = await node.requestEncryptedModuleBundle({
  serverDescriptor: {
    publicKey: providerPublicKey,
    relayAddresses: providerRelayAddresses,
  },
  moduleId: listing.pluginId,
  moduleVersion: listing.version,
  requesterDomain: globalThis.location.origin,
  requestedTimeoutMs: 300000,
});

const contentKey = await unwrapGrantContentKey(
  delivery.grant.wrappedContentKey,
  identity.encryptionKey.privateKey,
);
const wasmBytes = await decryptEncryptedModuleBundle(
  delivery.encryptedBundleBytes,
  contentKey,
);
const harness = await loadDecryptedModule(wasmBytes);
const result = await invokeLoadedModule(harness, {
  methodId: 'invoke',
  inputs: [],
});
harness.destroy?.();

Protocol Boundaries

• Public encrypted module delivery stays on /space-data-network/module-delivery/1.0.0.
• Provider discovery is anchored by the provider compressed secp256k1 public key and the DHT namespace space-data-network/module-delivery/provider-pubkey.
• Clients trust relay addresses advertised by the provider descriptor, seed from the approved demo relay, and expand discovery with libp2p and the DHT.
• Module bundles remain encrypted in transit and at rest, and decrypt only locally after the requester receives the wrapped content key in the grant response.

See the complete TypeScript example in sdn-js/examples/purchase-encrypted-wasm-delivery.ts.

Identity Management

import { Identity } from './sdn-js/dist/esm/index.js';

// Generate new identity
const identity = await Identity.generate();
console.log(identity.peerId);     // Qm...
console.log(identity.publicKey);  // Base64 public key

// From seed phrase (BIP-39)
const identity = await Identity.fromMnemonic(
  'abandon abandon abandon...'
);

// Export/Import (encrypted)
const exported = await identity.export('password123');
const imported = await Identity.import(exported, 'password123');

Astrodynamics Library

The SDK includes a complete astrodynamics library for propagation and conjunction assessment.

import { propagate, sgp4 } from './sdn-js/dist/esm/astro/index.js';

// SGP4 propagation from TLE
const state = sgp4(tle, epochMinutes);
// { position: [x, y, z], velocity: [vx, vy, vz] }

// Propagate OMM
const states = propagate(omm, {
  start: new Date('2025-01-24T00:00:00Z'),
  end: new Date('2025-01-25T00:00:00Z'),
  step: 60 // seconds
});

import { screenConjunctions, computePc } from './sdn-js/dist/esm/astro/index.js';

// Screen for close approaches
const conjunctions = screenConjunctions(primaryOMM, secondaryOMMs, {
  threshold: 5000, // meters
  timeWindow: 7 * 24 * 3600 // 7 days
});

// Compute collision probability
const pc = computePc(cdm);

import { eciToEcef, ecefToLla } from './sdn-js/dist/esm/astro/index.js';

// ECI to ECEF
const ecef = eciToEcef(eci, gmst);

// ECEF to Lat/Lon/Alt
const lla = ecefToLla(ecef);
// { lat: 51.5, lon: -0.1, alt: 400000 }

Identity & Key Derivation

Every SDN node derives its full cryptographic identity from a BIP-39 mnemonic generated client-side. No central authority issues keys — identity is fully decentralized. Using SLIP-10 hierarchical deterministic derivation with the standard BIP-44 Bitcoin derivation path (coin type 0), one seed produces all the keys a node needs.

Derivation Tree

BIP-39 Mnemonic (generated client-side)
    │
    ▼ PBKDF2
  512-bit Seed
    │
    ├── BIP-32 secp256k1 Master Key
    │     │
    │     └── m/44'/0'/0'  →  Identity Key (secp256k1)
    │           ├── xpub   →  Master Network Identity
    │           └── PeerID →  libp2p address (16Uiu2...)
    │
    └── SLIP-10 Ed25519 Master Key
          │
          ├── m/44'/0'/0'/0'/0'  →  Ed25519 Signing Key (auth, data signatures)
          │
          └── m/44'/0'/0'/1'/0'  →  X25519 Encryption Key (ECDH, ECIES)

The xpub (extended public key) at m/44'/0'/account' serves as the node's master network identity. It encodes a secp256k1 public key from which the libp2p PeerID is deterministically derived. Anyone with the xpub can compute the PeerID without access to private key material.

The Ed25519 signing key is bound to the xpub identity on first login via TOFU (Trust On First Use) and is used for authentication and data signatures. The X25519 encryption key provides end-to-end encryption between peers.

Why BIP-44?

SDN reuses the standard BIP-44 HD wallet path structure with Bitcoin's coin type 0:

• Deterministic, wallet-native identity. By using a BIP-44-style path (m/44'/0'/account'/change'/index'), SDN nodes can derive signing and encryption keys from the same BIP-39 mnemonic a user already has for their cryptocurrency wallets. One seed, many independent key trees.
• Multi-account support. The account' segment enables a single mnemonic to manage multiple SDN identities (e.g., account 0 for an operator, account 1 for a sensor, account 2 for an analytics service), each with its own independent signing and encryption key pair.

JavaScript SDK

import { initHDWallet, deriveIdentity, mnemonicToSeed } from './sdn-js/dist/esm/index.js';

// Initialize WASM module
await initHDWallet();

// Generate or load mnemonic
const mnemonic = 'abandon abandon abandon ...'; // generated client-side
const seed = await mnemonicToSeed(mnemonic);

// Derive full identity (signing + encryption keys)
const identity = await deriveIdentity(seed, 0); // account 0

console.log(identity.signingKey.publicKey);    // Ed25519 pub (32 bytes)
console.log(identity.encryptionKey.publicKey); // X25519 pub (32 bytes)
console.log(identity.signingKeyPath);          // m/44'/0'/0'/0'/0'
console.log(identity.encryptionKeyPath);       // m/44'/0'/0'/1'/0'

Go Server

// The server automatically derives identity from its encrypted mnemonic on startup.
// The mnemonic is encrypted at rest with XChaCha20-Poly1305 (see Key Security).
//
// Files in ~/.spacedatanetwork/keys/:
//   mnemonic     - BIP-39 mnemonic (encrypted at rest)
//   node.key     - Serialized libp2p private key (backward compat)
//
// Inspect the node identity without starting the daemon:
//   spacedatanetwork show-identity --wasm /path/to/hd-wallet.wasm

hw, _ := wasm.NewHDWalletModule(ctx, "/path/to/hd-wallet.wasm")
seed, _ := hw.MnemonicToSeed(ctx, mnemonic, "")
identity, _ := hw.DeriveIdentity(ctx, seed, 0)
xpub, _ := hw.DeriveXPub(ctx, seed, 0)

fmt.Println(xpub)                       // BIP-32 xpub (secp256k1)
fmt.Println(identity.PeerID)            // libp2p PeerID (from secp256k1)
fmt.Println(identity.SigningKeyPath)    // m/44'/0'/0'/0'/0'
fmt.Println(identity.EncryptionKeyPath) // m/44'/0'/0'/1'/0'

Secp256k1 Identity (xpub & PeerID)

The secp256k1 identity key at m/44'/0'/account' is the root of a node's network identity. It is derived using standard BIP-32 hierarchical deterministic derivation on the secp256k1 curve — the same curve and derivation used by Bitcoin, Ethereum, and most cryptocurrency wallets.

xpub (Extended Public Key)

The xpub is a Base58Check-encoded extended public key (prefix xpub6...) that encodes:

• The 33-byte compressed secp256k1 public key
• A 32-byte chain code for child key derivation
• Depth, parent fingerprint, and child index metadata

The xpub serves as the node's portable identity. It is safe to share publicly and is used in the users config section for authentication:

users:
  - xpub: "xpub6DKCyLbCHZLFR4XpFg26royZdkxExSMHTjNorEgkn1kgvQbLF5sts9RfNt3PbGhphVUh7WsFQ5H6GJBh4LhmRL27oSPt1qDkJ5mAr6FZ3Wa"
    trust_level: admin
    name: Operator

PeerID from secp256k1

The libp2p PeerID is deterministically derived from the compressed secp256k1 public key:

secp256k1 pubkey (33 bytes, compressed)
    │
    ▼ Protobuf encode (libp2p crypto key format)
  PublicKey { Type: Secp256k1, Data: [33 bytes] }
    │
    ▼ SHA-256 multihash (identity CID for keys ≤42 bytes uses inline)
  PeerID: 16Uiu2HAm1LbvwjEH...Fm45

Secp256k1 PeerIDs have a 16Uiu2... prefix, distinguishing them from Ed25519 PeerIDs (12D3Koo...). Because the PeerID is derived purely from the xpub's public key, anyone with an xpub can compute the corresponding PeerID without any private key material.

Why secp256k1 for Identity?

• Wallet compatibility — The same BIP-39 mnemonic used for Bitcoin/Ethereum wallets produces the SDN identity key. No separate key management needed.
• xpub portability — BIP-32 xpubs are a widely-understood, standardized format. Operators can share an xpub to grant access, and any SDN node can verify identity from it.
• 1:1 xpub↔PeerID mapping — Every xpub maps to exactly one PeerID and vice versa, enabling deterministic peer discovery from identity alone.
• Multi-account isolation — Different account' indices produce completely independent identities from the same seed, each with its own xpub, PeerID, signing key, and encryption key.

Computing PeerID from xpub

// JavaScript (hd-wallet-wasm)
import init from 'hd-wallet-wasm';
const { libp2p } = await init();
const peerId = libp2p.peerIdFromXpub('xpub6DKCyLb...');
// → "16Uiu2HAm1LbvwjEH...Fm45"

// Go server CLI
spacedatanetwork show-identity --wasm /path/to/hd-wallet.wasm
# PeerID printed to stdout

// Go (programmatic)
hw, _ := wasm.NewHDWalletModule(ctx, wasmPath)
seed, _ := hw.MnemonicToSeed(ctx, mnemonic, "")
identity, _ := hw.DeriveIdentity(ctx, seed, 0) // account 0
fmt.Println(identity.PeerID()) // "16Uiu2HAm..."

Signing & Authentication

SDN uses a two-layer identity model:

• Network Identity (secp256k1) — The secp256k1 key at m/44'/0'/account' produces the xpub and libp2p PeerID. The PeerID is a multihash of the compressed secp256k1 public key, giving every xpub a unique, deterministic network address (prefix 16Uiu2...).
• Digital Signatures (Ed25519) — The Ed25519 key at m/44'/0'/account'/0'/0' signs data (SDS schemas, messages, authentication challenges). Recipients verify signatures using the corresponding 32-byte public key.

The Ed25519 signing key is bound to the secp256k1 identity via TOFU (Trust On First Use): on the first wallet-based login, the server records the Ed25519 public key alongside the xpub. Subsequent logins require both the correct xpub and a valid Ed25519 challenge-response signature.

// Sign data with the identity's Ed25519 key
const { sign, verify } = await import('./sdn-js/dist/esm/index.js');

const message = new TextEncoder().encode('ISS conjunction alert');
const signature = await sign(identity.signingKey.privateKey, message);

// Anyone with the public key (or PeerID) can verify
const valid = await verify(identity.signingKey.publicKey, message, signature);
console.log(valid); // true

// Go: sign and verify using the DerivedIdentity
sig, _ := identity.Sign([]byte("ISS conjunction alert"))
ok, _  := identity.Verify([]byte("ISS conjunction alert"), sig)
// ok == true

Signed EPMs and Portable vCards

Entity Profile Messages (EPMs) are portable identity records for nodes and users. Because EPMs can be exported as vCards and stored in ordinary contact systems, the contact itself must carry a tamper-evident signature.

Embedded EPM Signature

Each node EPM includes an embedded Ed25519 signature and signature timestamp. The signature covers a deterministic canonical payload built from the EPM fields, excluding only the signature field itself. The signature timestamp is included in the signed payload so it cannot be changed without invalidating the record.

canonical EPM fields + SIGNATURE_TIMESTAMP
  -> Ed25519 signature using m/44'/0'/account'/0'/0'
  -> EPM.SIGNATURE

vCard Export

When SDN exports a node profile as a vCard, it includes both human-readable contact fields and SDN verification extensions:

On import, SDN prefers the embedded signed EPM payload over spoofable vCard display fields. A modified FN or ORG line does not change the trusted identity record unless the embedded EPM also verifies.

PNM Announcement

PNM remains the network announcement layer. Every time a node EPM is updated, the node computes a new EPM CID and publishes a PNM with FILE_ID=EPM. The PNM signature signs the CID announcement, while the embedded EPM signature signs the portable identity content.

EPM bytes
  -> EPM CID
  -> PNM { FILE_ID: "EPM", CID, SIGNATURE: sign(CID) }

Verification Order

1. Decode the EPM from vCard, EPM exchange, or CID fetch.
2. Verify the embedded EPM signature over canonical EPM content.
3. Confirm the signed EPM advertises the expected peer ID for network discovery.
4. Compute the EPM CID from the exact bytes.
5. Verify the PNM signature over that CID when the record arrived via PubSub.
6. Store the verified record in the FlatSQL-backed directory.

Encryption (X25519 ECDH)

The encryption key at path m/44'/0'/account'/1'/0' is derived from the same seed as the signing key but at a different derivation path (change index 1 instead of 0). This produces an X25519 key pair used for:

• ECDH Key Exchange — Two peers combine their X25519 keys to derive a shared secret without transmitting it.
• AES-256-GCM Encryption — The shared secret encrypts data end-to-end. Only the intended recipient can decrypt.

import { encryptBytes, decryptBytes, deriveIdentity } from './sdn-js/dist/esm/index.js';

// Alice and Bob each derive their identity
const alice = await deriveIdentity(aliceSeed, 0);
const bob   = await deriveIdentity(bobSeed, 0);

// Alice encrypts a message for Bob using his X25519 public key
const plaintext = new TextEncoder().encode('Classified orbital data');
const encrypted = await encryptBytes(
  plaintext,
  bob.encryptionKey.publicKey,    // Bob's X25519 public key
  alice.encryptionKey.privateKey  // Alice's X25519 private key
);

// Bob decrypts using Alice's public key and his private key
const decrypted = await decryptBytes(
  encrypted,
  alice.encryptionKey.publicKey,  // Alice's X25519 public key
  bob.encryptionKey.privateKey    // Bob's X25519 private key
);

console.log(new TextDecoder().decode(decrypted)); // 'Classified orbital data'

Key Separation

Using separate derivation paths for signing and encryption follows cryptographic best practice. A compromised encryption key does not affect the signing key (and vice versa), even though both derive from the same master seed.

ECIES (Elliptic Curve Integrated Encryption)

The underlying hd-wallet-wasm library provides a unified ECIES API that combines ECDH key agreement, HKDF-SHA256 key derivation, and AES-256-GCM authenticated encryption into a single encrypt/decrypt interface.

• Supported curves: secp256k1, P-256 (NIST), P-384 (NIST), X25519
• FIPS mode: When enabled, restricts to P-256/P-384 and routes HKDF + AES-GCM through OpenSSL FIPS provider
• Wire format: [ephemeral_pubkey][iv (12B)][ciphertext][tag (16B)]

AES-CTR Mode

Counter mode encryption is available for streaming use cases. Unlike AES-GCM, CTR mode does not provide authentication — pair with HMAC-SHA256 if message integrity is required. Supports AES-128, AES-192, and AES-256 (key size selects variant). Uses 16-byte IV.

FlatBuffers Field Encryption

The flatc-wasm package provides per-field AES-256-CTR encryption for FlatBuffer data via the encryption.mjs module. Fields marked with the (encrypted) attribute are transparently encrypted with unique keys derived per field and per record.

import {
  loadEncryptionWasm, EncryptionContext,
  x25519GenerateKeyPair, encryptionHeaderFromJSON,
} from 'flatc-wasm';

await loadEncryptionWasm();

// Sender creates an ECIES encryption session
const ctx = EncryptionContext.forEncryption(recipientPubKey, {
  algorithm: 'x25519',    // or 'secp256k1'
  context: 'sdn-orbit-v1' // HKDF domain separation
});

// Encrypt fields in a FlatBuffer
ctx.encryptScalar(buffer, offset, length, fieldId, recordIndex);
const headerJSON = ctx.getHeaderJSON(); // send to recipient

// Recipient decrypts
const decCtx = EncryptionContext.forDecryption(
  myPrivateKey,
  encryptionHeaderFromJSON(headerJSON),
  'sdn-orbit-v1'
);
decCtx.decryptScalar(buffer, offset, length, fieldId, recordIndex);

The module also provides authenticated encryption (AES-CTR + HMAC-SHA256), standalone HKDF, SHA-256, and signature operations for secp256k1, P-256, P-384, and Ed25519. P-256/P-384 operations use the Web Crypto API for FIPS-grade implementations.

Key Security

The BIP-39 mnemonic is the root secret from which all node keys are derived. SDN encrypts it at rest using XChaCha20-Poly1305 authenticated encryption with a key derived from Argon2id.

Encryption at Rest

File format:  salt (32 bytes) || nonce (24 bytes) || ciphertext
Permissions:  0600 (owner-only read/write)
Cipher:       XChaCha20-Poly1305 (AEAD)
KDF:          Argon2id (time=3, memory=64 MiB, threads=4, keyLen=32)

On first startup (or when a plaintext mnemonic is detected), the server automatically encrypts the mnemonic file in place. Subsequent reads decrypt transparently.

Password Resolution

The encryption password is resolved in order of priority:

1. SDN_KEY_PASSWORD environment variable (highest priority)
2. security.key_password in config.yaml
3. Machine-derived password — deterministic Argon2id hash of hostname:GOARCH:GOOS with the user's home directory as salt (allows unattended startup)

Configuration Example

# Option 1: Environment variable (recommended for production)
export SDN_KEY_PASSWORD="your-strong-passphrase"
spacedatanetwork daemon

# Option 2: Config file
security:
  key_password: "your-strong-passphrase"

# Option 3: Machine-derived (default, for unattended startup)
# No configuration needed — password is derived automatically

Auto-Migration

If the server detects a plaintext mnemonic file (from an older installation), it automatically encrypts it using the resolved password. The original plaintext is overwritten with the encrypted blob. This migration is logged as a warning at startup.

IPFS Pinning Mechanics

SDN uses IPFS-style content addressing to identify data. Every piece of data published on the network gets a Content Identifier (CID): a cryptographic hash that uniquely identifies the content.

How Content Addressing Works

1. Data (e.g., an OMM FlatBuffer)  →  SHA-256 hash  →  CID
   bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi

2. The CID is announced on the DHT (Kademlia distributed hash table)
3. Any node can retrieve the data by requesting the CID from the network
4. The hash guarantees integrity: if the data changes, the CID changes

Pinning = Persistent Availability

By default, data flowing through the network is transient — nodes process and forward it but don't store it permanently. Pinning is the act of committing to store a CID and serve it to the network on demand.

• Self-hosted nodes can pin any data for free — you control your own storage.
• SpaceAware.io (our hosted premium node) charges a per-GB/month fee for persistent pinning, providing guaranteed availability and SLA-backed uptime.
• Operators running their own marketplace nodes can set their own pinning fees or offer pinning for free.

Signed Pinning

When a node pins data, it signs the pin announcement with its Ed25519 signing key. This creates a verifiable chain:

Author signs data  →  CID (content hash)  →  Pin announcement (signed)

Verifiers can confirm:
  1. The data matches the CID (content integrity)
  2. The CID was published by a specific PeerID (author attribution)
  3. The pin was committed by a specific node (availability commitment)

Because the signing key and PeerID are the same key (see Signing & IPFS), author attribution is built into the protocol — no separate identity layer is needed.

REST API

The gateway HTTP API is served on the configured admin port (default 127.0.0.1:5001; see admin.listen_addr in Full Node Setup). All paths below are absolute. A machine-readable, always-current reference is generated at runtime and served at GET /api/v1/openapi.json — treat that endpoint as the source of truth over any table on this page.

Node information

GET /api/v1/id        # Node identity: peer_id, listen_addresses, version fields  (public)
GET /api/v1/version   # Software versions: agent_version, suite_version, standards_version  (public)
GET /api/v1/stats     # Runtime stats: connected_peers, total_records, total_bytes, schemas  (public)

Example GET /api/v1/id response:

{
  "peer_id": "12D3KooW...",
  "listen_addresses": ["/ip4/0.0.0.0/tcp/4001"],
  "agent_version": "spacedatanetwork/1.0.3",
  "suite_version": "1.0.3",
  "standards_version": "1.100.0"
}

Peer management — /api/v1/peers/

These endpoints operate on the libp2p network layer (currently connected peers), which is separate from the trusted-peer registry at /api/peers (below) and from the public-DHT advertisement-flag discovery that finds peers in the first place.

Trusted-peer registry — /api/peers

A separate handler manages the operator-curated trusted-peer registry that the admin UI reads and writes (trust level, groups, EPM/vCard export, blocklist). This registry is what PNM materialization checks before importing a peer's published data.

PubSub — /api/v1/pubsub/

Data query & retrieval — /api/v1/data/

Record bytes are content-addressed and stored in the per-(producer, standard) FlatSQL tables described in FlatSQL Storage. A small set of routes are native Go and always on; per-schema convenience routes are served by the default-mounted data-retrieval flow module (a sandboxed FlatSQL-WASM bundle, not hand-written Go).

Record-stream endpoints accept ?format=json to return a bare top-level JSON array of records (the default encoding is the binary FlatBuffer stream). Responses carry X-SDN-Record-Count and X-SDN-Schema headers; other endpoint families (channels, storefront, peer/EPM export) use their own X-SDN-* headers documented inline below.

Wallet authentication — /api/auth/

SDN has no username/password login on the gateway API. A wallet proves control of its xpub identity with a signed challenge:

1. The client requests a single-use, 60-second challenge for an xpub (or a bare Ed25519 public key on first-run bootstrap).
2. The wallet signs the raw challenge bytes with its Ed25519 signing key and posts the signature back.
3. The server verifies the Ed25519 signature; on the very first successful verify for an xpub the signing key is bound to it via TOFU (see Signing & Authentication).
4. A session token is issued as an HttpOnly cookie (sdn_wallet_session).

Channels — /api/v1/channels/

Channels carry PNM/DPM publication updates, encrypted marketplace delivery, and shard import for a given dataset or grant.

Module delivery — /api/module-delivery/

These two GET endpoints are public HTTP discovery only:

The actual purchase/grant/decrypt exchange is not HTTP — it runs over the libp2p stream protocol /space-data-network/module-delivery/1.0.0, as shown in Storefront & Encrypted Module Delivery.

WebSocket API

A JSON WebSocket bridge is available at /ws on the admin port. It provides real-time subscribe/publish over the same GossipSub layer used by libp2p peers.

Connect

ws://127.0.0.1:5001/ws

Client → Server: subscribe

{
  "type": "subscribe",
  "schema": "OMM.fbs"
}

Server responds with {"type":"subscribed","schema":"OMM.fbs"}. The connection will then receive message frames whenever data for that schema is published (locally or via libp2p).

Client → Server: unsubscribe

{
  "type": "unsubscribe",
  "schema": "OMM.fbs"
}

Client → Server: publish

{
  "type": "publish",
  "schema": "OMM.fbs",
  "data": "<base64-encoded FlatBuffer bytes>"
}

Data is validated against the named SDS schema before publication. Invalid data returns an error frame. On success the message is broadcast to local subscribers and forwarded to the libp2p topic /spacedatanetwork/sds/{schema}.

Server → Client: message

{
  "type": "message",
  "schema": "OMM.fbs",
  "data": "<base64-encoded FlatBuffer bytes>",
  "from": "12D3KooW...",
  "ts": "2025-01-24T12:00:00Z"
}

Server → Client: error

{
  "type": "error",
  "error": "validation failed: ..."
}

Error Handling

Errors from the /api/v1/id, /api/v1/version, /api/v1/stats, /api/v1/peers/, and /api/v1/pubsub/ endpoints use this envelope:

{
  "error": {
    "code": "INVALID_REQUEST",
    "message": "schema query parameter is required"
  }
}

The native /api/v1/data/* query endpoints (health, summary, query, records/{schema}/{cid}, and the datasync surface) use a simpler envelope, {"error":{"message":"..."}}, without a code field; /api/v1/data/publish/{schema} returns a flat {"error":"..."} string. Both predate the structured {"error":{"code":...}} envelope above and are kept for compatibility with existing clients.

Auth rejections from the existing middleware return {"code":"unauthorized"} (401) or {"code":"forbidden"} (403) — these originate from internal/auth/middleware.go and are not part of the new error envelope.

Rate Limiting

The /api/v1/id, /api/v1/version, /api/v1/stats, /api/v1/peers, /api/v1/pubsub/topics, /api/v1/pubsub/messages, and /api/v1/pubsub/publish endpoints are rate-limited per client IP: 100 requests/minute for read methods (GET), 10 requests/minute for write methods (POST, PUT, PATCH, DELETE).

X-RateLimit-Limit: 100
X-RateLimit-Remaining: 95
X-RateLimit-Reset: 1706097660   # Unix timestamp when the bucket refills

When the limit is exceeded the server responds with HTTP 429 and the RATE_LIMITED error body. The X-RateLimit-Reset header tells the client when to retry.

Schema Reference

SDN registers every Space Data Standards (SDS) schema — 160+ standards — plus the current SDN coordination schemas used by the runtime. All data on the network must conform to these schemas. The full registered list:

OMM - Orbit Mean-Elements Message

Standard format for mean orbital elements (TLE-equivalent).

interface OMM {
  OBJECT_NAME: string;           // Satellite name
  OBJECT_ID: string;             // International designator
  CENTER_NAME: string;           // Central body (EARTH)
  REF_FRAME: string;             // Reference frame (TEME)
  TIME_SYSTEM: string;           // Time system (UTC)
  MEAN_ELEMENT_THEORY: string;   // Theory (SGP4)
  EPOCH: string;                 // Epoch (ISO 8601)
  MEAN_MOTION: number;           // Rev/day
  ECCENTRICITY: number;          // Dimensionless
  INCLINATION: number;           // Degrees
  RA_OF_ASC_NODE: number;        // Degrees
  ARG_OF_PERICENTER: number;     // Degrees
  MEAN_ANOMALY: number;          // Degrees
  // ... additional optional fields
}

OEM - Orbit Ephemeris Message

High-precision state vectors over time.

interface OEM {
  OBJECT_NAME: string;
  OBJECT_ID: string;
  CENTER_NAME: string;
  REF_FRAME: string;
  START_TIME: string;
  STOP_TIME: string;
  EPHEMERIS: EphemerisDataLine[];
}

interface EphemerisDataLine {
  EPOCH: string;
  X: number;      // km
  Y: number;      // km
  Z: number;      // km
  X_DOT: number;  // km/s
  Y_DOT: number;  // km/s
  Z_DOT: number;  // km/s
}

CDM - Conjunction Data Message

Standard format for close approach warnings.

interface CDM {
  CCSDS_CDM_VERS: string;
  CREATION_DATE: string;
  ORIGINATOR: string;
  MESSAGE_ID: string;

  // Conjunction info
  TCA: string;                    // Time of Closest Approach
  MISS_DISTANCE: number;          // meters
  RELATIVE_SPEED: number;         // m/s
  COLLISION_PROBABILITY?: number; // 0-1

  // Object 1
  OBJECT1_OBJECT_DESIGNATOR: string;
  OBJECT1_OBJECT_NAME: string;
  OBJECT1_INTERNATIONAL_DESIGNATOR: string;
  // ... state and covariance

  // Object 2
  OBJECT2_OBJECT_DESIGNATOR: string;
  // ... same fields as Object 1
}

EPM - Entity Profile Manifest

Organization/entity identification with cryptographic keys.

interface EPM {
  ENTITY_ID: string;              // Unique identifier
  ENTITY_NAME: string;            // Organization name
  ENTITY_TYPE: string;            // OPERATOR, AGENCY, etc.
  LEGAL_NAME?: string;
  CONTACT_EMAIL?: string;
  PUBLIC_KEY: string;             // Ed25519 public key (base64)
  KEY_ALGORITHM: string;          // ED25519
  CREATED: string;                // ISO 8601
  UPDATED: string;                // ISO 8601
}

Schema Validation

import { SchemaRegistry } from './sdn-js/dist/esm/index.js';

// Validate data against schema
const errors = SchemaRegistry.validate('OMM', data);
if (errors.length > 0) {
  console.error('Validation failed:', errors);
}

// List all schemas
const schemas = SchemaRegistry.list();
// ['OMM', 'OEM', 'CDM', 'EPM', ...]

Converting from TLE

import { tleToOMM } from './sdn-js/dist/esm/index.js';

const tle = [
  '1 25544U 98067A   25024.50000000  .00016717  00000-0  10270-3 0  9029',
  '2 25544  51.6400 200.0000 0001000 100.0000  50.0000 15.50000000000000'
];

const omm = tleToOMM(tle);

Go FlatBuffer Builders

The internal/sds package provides fluent builders for creating FlatBuffer messages that conform to Space Data Standards.

Available Builders

OMM Builder Example

import "github.com/spacedatanetwork/sdn-server/internal/sds"

data := sds.NewOMMBuilder().
    WithObjectName("ISS (ZARYA)").
    WithObjectID("1998-067A").
    WithNoradCatID(25544).
    WithEpoch("2024-01-15T12:00:00.000Z").
    WithMeanMotion(15.49).
    WithEccentricity(0.0001215).
    WithInclination(51.6434).
    WithRaOfAscNode(178.1234).
    WithArgOfPericenter(85.5678).
    WithMeanAnomaly(274.9012).
    Build()

EPM Builder Example

data := sds.NewEPMBuilder().
    WithDN("John Doe").
    WithLegalName("Acme Corporation").
    WithFamilyName("Doe").
    WithGivenName("John").
    WithEmail("john@acme.com").
    WithTelephone("+1-555-0100").
    WithAddress("123 Main St", "Springfield", "IL", "62701", "USA").
    WithJobTitle("Chief Engineer").
    WithKeys("signingKey123", "encryptionKey456").
    Build()

Performance

FlatBuffers provide zero-copy deserialization with excellent performance:

For complete throughput benchmarks including PNM and CAT schemas, batch processing, and parallel tests, see Throughput Benchmarks.

vCard/QR Code Conversion

The internal/vcard package provides bidirectional conversion between EPM FlatBuffers, vCard 4.0 format, and QR codes.

EPM to vCard Field Mapping

Conversion Examples

import "github.com/spacedatanetwork/sdn-server/internal/vcard"

// EPM to vCard string
vcardStr, err := vcard.EPMToVCard(epmBytes)

// vCard string to EPM
epmBytes, err := vcard.VCardToEPM(vcardStr)

// EPM to QR Code (PNG)
pngData, err := vcard.EPMToQR(epmBytes, 256)  // 256x256 pixels

// QR Code to EPM
epmBytes, err := vcard.QRToEPM(pngData)

// Full roundtrip: EPM -> vCard -> QR -> vCard -> EPM
qrPNG, _ := vcard.EPMToQR(originalEPM, 512)
recoveredEPM, _ := vcard.QRToEPM(qrPNG)

For complete test coverage of the EPM/vCard/QR conversion pipeline, see EPM/QR Conversion Tests.

PNM Tip/Queue System

The tip/queue system uses Publish Notification Messages (PNM) as the core messaging mechanism for content discovery. This is a fundamental design principle: not all pinned data should be broadcast. Instead, nodes announce content availability via lightweight PNM messages, allowing subscribers to selectively fetch and pin content based on configurable policies.

Why PNM-Based Messaging?

• Bandwidth efficiency: PNM messages are tiny (~100-200 bytes) compared to actual content (KB to MB)
• Selective consumption: Nodes only fetch content they're interested in
• Trust-based filtering: Configure which peers and schema types to auto-fetch
• Spam resistance: Malicious nodes can't flood the network with large payloads
• Verifiable announcements: PNMs include cryptographic signatures for authenticity

Architecture

Publisher                           Subscriber
    |                                   |
    |-- Pin content locally             |
    |-- Create PNM with CID + sig       |
    |-- Broadcast PNM on /sdn/PNM ------|--> Receive PNM (lightweight notification)
    |                                   |-- Check config for peer + schema
    |                                   |-- If trusted + autoFetch: fetch by CID
    |                                   |-- If autoPin: pin with TTL
    |                                   |
    |   [Content NOT sent unless        |   [Content fetched on-demand
    |    explicitly requested]          |    based on subscriber policy]

Configuration Levels

Configuration supports per-source AND per-schema settings with priority resolution:

Configuration Example

import "github.com/spacedatanetwork/sdn-server/internal/pubsub"

config := pubsub.NewTipQueueConfig()

// System defaults
config.DefaultAutoFetch = false
config.DefaultAutoPin = false
config.DefaultTTL = 24 * time.Hour

// Per-schema: always fetch conjunction data
config.SetSchemaDefault("CDM", &pubsub.SchemaConfig{
    AutoFetch: true,
    AutoPin:   true,
    TTL:       48 * time.Hour,
    Priority:  10,  // Critical data
})

// Per-source: trust a partner
config.SetSourceOverride("trusted-peer-id", &pubsub.SourceConfig{
    Trusted:   true,
    AutoFetch: pubsub.BoolPtr(true),
    TTL:       pubsub.DurationPtr(72 * time.Hour),
})

// Per-source+schema: custom handling
config.SetSourceSchemaOverride("trusted-peer-id", "OMM", &pubsub.SchemaConfig{
    AutoFetch: true,
    AutoPin:   true,
    TTL:       168 * time.Hour,  // 1 week
})

TipQueue Usage

// Create and configure
tq := pubsub.NewTipQueue(config)
tq.SetTopicManager(topicManager)
tq.SetFetcher(fetcher)
tq.SetPinner(pinner)

// Handle received tips
tq.OnTip(func(tip *pubsub.Tip, cfg pubsub.ResolvedConfig) {
    log.Printf("Tip from %s: CID=%s Schema=%s",
        tip.PeerID, tip.CID, tip.SchemaType)
})

// Start listening
tq.Subscribe()

// Publish your own content
tq.PublishTip(ctx, pubsub.PublishOptions{
    CID:        "bafybei...",
    SchemaType: "OMM",
    Signature:  "0x...",
})

Configurable Options

Data Flow Architecture

SDN dataset delivery is built on FlatBuffers for zero-copy binary records, FlatSQL for durable local materialization, DPM manifests for signed publication contracts, and PNM messages for compact network announcements. The current latest-product path is PNM -> DPM -> data shard/query-index assets or provider-mediated query, followed by verification and import.

Peer Discovery — Public IPFS/Amino DHT

SDN nodes find each other on the same public IPFS/Amino DHT that every stock IPFS/Kubo node joins — the standard Kademlia protocol /ipfs/kad/1.0.0. There is no private /spacedatanetwork DHT swarm; at the DHT-transport layer an SDN node is indistinguishable from any other public IPFS peer, and a regression test in the server (TestPublicDHTOptionsJoinStockIPFSProtocol) asserts the legacy private protocol string is never registered.

The advertisement flag

Using libp2p's routing-discovery convention (Provide / FindPeers over the DHT), each node advertises and searches for a CID derived from a namespaced string:

namespace = "space-data-network/discovery/advertisement-flag/<flag>"
                                                                  # current flag: "spacedatanetwork/1.0.0"
cid       = CIDv1(raw, sha2-256, SHA-256(UTF-8 bytes of namespace))

Advertise: drouting.NewRoutingDiscovery(dht).Advertise(ctx, namespace)   # ~every 30s
Discover:  drouting.NewRoutingDiscovery(dht).FindPeers(ctx, namespace)

A peer only counts as SDN-discovered when it is found via (or itself advertises) this namespace — never merely by DHT routing-table adjacency or an inbound/outbound libp2p connection. Multiple flags can be supported simultaneously across a rolling upgrade; a node advertises its current flag and discovers peers under every flag it still recognizes.

Same derivation, Go and browser

The browser SDK computes the identical namespace-to-CID derivation (sdn-advertisement-discovery.ts) and runs libp2p's Kademlia DHT client mode with no protocol-prefix override, so it speaks the same public swarm as a Go daemon. A browser tab can therefore discover full nodes directly through the DHT; edge relays are still useful for bridging transports a browser cannot dial on its own (e.g. raw TCP), but they are not required for basic peer discovery.

Bootstrap peers

A node's effective bootstrap set combines SDN's own production peers with the standard public IPFS/libp2p defaults — SDN augments the public bootstrap set, it does not replace it:

network:
  bootstrap:
    - /dnsaddr/bootstrap.spacedatanetwork.org/p2p/16Uiu2HAm1LbvwjEHW2GDP2ZQZvwHLZrz2jbYoRLQmJEQ3wZ5Fm45
    - /dnsaddr/bootstrap.spacedatanetwork.org/p2p/16Uiu2HAm9oK2jAeVC2RMESFcYfq7BKGp2K2CCDxzoKhB5s9vpbj3
    # ...plus the standard go-libp2p-kad-dht default bootstrap peers (bootstrap.libp2p.io),
    # appended automatically unless bootstrap is overridden.

FlatBuffers Binary Format

All SDN data uses Google FlatBuffers — a zero-copy binary serialization format. Unlike JSON or Protocol Buffers, FlatBuffers can be read directly from the binary without parsing or memory allocation.

Binary Layout

Offset 0-3:  Root table offset (uint32 little-endian)
Offset 4-7:  File identifier (4 ASCII bytes: "$PNM", "$DPM", "$OMM", etc.)
Offset 8+:   VTable + field data + string/vector payloads

File Identifiers

The 4-byte file identifier at bytes 4-7 tells you the message type without parsing:

Reading the File Identifier

// JavaScript — identify message type from raw bytes
const fileId = String.fromCharCode(bytes[4], bytes[5], bytes[6], bytes[7]);
// "$PNM" -> Publish Notification, "$DPM" -> Dataset Publication Manifest

// Go — same concept
fileID := string(data[4:8])

Why FlatBuffers?

• Zero-copy reads — read fields directly from the buffer, no deserialization step
• Deterministic hashing — same data produces same binary, enabling reliable SHA-256 CIDs
• Schema evolution — new fields can be added without breaking old readers
• 160+ schemas — every Space Data Standards schema (OMM, CDM, EPM, etc.) plus SDN-internal schemas

FlatSQL Storage

FlatSQL is a streaming SQL-over-FlatBuffers engine: a C++/SQLite core compiled to WebAssembly and hosted in-process (WasmEdge), queried through database/sql via a Go driver so existing SQL-shaped code doesn't need rewriting. The engine itself runs in memory — there is no on-disk mattn/go-sqlite3 or other CGO SQLite dependency behind the record store, and record payload bytes are never stored as SQL BLOB columns. Durability comes from append-only flatsql-streams/*.flatsql files that hold the raw FlatBuffer bytes; SQL tables hold only pointers into those streams plus indexes. (A small, separate pure-Go modernc.org/sqlite file-backed database is still used for a few unrelated standalone subsystems — admin, audit, auth, storefront — each with its own private .db file; those are not part of the FlatSQL record path.) Browser nodes use the same concepts in the sdn-js local (IndexedDB) store.

Per-Producer, Per-Standard Tables

Every record is routed into a physical table named for its publisher and Space Data Standard:

sds_p_<sanitized-producer-id>__<standard>

# examples
sds_p_16Uiu2HAm1LbvwjEHW2GDP2ZQZvwHLZrz2jbYoRLQmJEQ3wZ5Fm45__OMM
sds_p_unattributed__CAT   # records with no resolvable publisher identity

The producer segment is the publisher's libp2p peer ID (or hex pubkey) with non-alphanumeric characters replaced by _; the standard segment is the SDS schema name with its .fbs suffix stripped (OMM.fbs → OMM). A table is created on demand the first time a given producer publishes a given standard — there is no up-front schema registration step. This is the v1 storage layout: it is routed-only, meaning new data is never written to the older, non-producer-scoped per-standard tables (e.g. a bare OMM table); those remain readable for continuity with data imported before this loop, via a read-side union across both layouts.

Table Schema

-- Shared metadata shape for every sds_p_<producer>__<standard> table
CREATE TABLE sds_p_<producer>__<standard> (
  cid            TEXT PRIMARY KEY,  -- real CIDv1: raw codec, sha2-256 multihash
  peer_id        TEXT NOT NULL,     -- publisher's libp2p peer ID
  timestamp      INTEGER NOT NULL,  -- unix seconds
  stream_path    TEXT NOT NULL,     -- which flatsql-streams/*.flatsql file holds the bytes
  stream_offset  INTEGER NOT NULL,  -- byte offset of the record within that file
  record_length  INTEGER NOT NULL,  -- byte length of the record
  signature_hex  TEXT,              -- Ed25519 signature, hex-encoded
  created_at     INTEGER DEFAULT (strftime('%s', 'now')),
  UNIQUE(cid)
);

-- Fast query index (NORAD ID, epoch, entity/object filtering) across all tables
CREATE TABLE sdn_record_index (
  schema_name       TEXT NOT NULL,
  cid               TEXT NOT NULL,
  norad_cat_id      INTEGER,        -- for orbital data (OMM, CDM, ...)
  entity_id         TEXT,
  object_type       TEXT,
  ops_status_code   TEXT,
  epoch_unix        INTEGER,
  epoch_day         TEXT,           -- "YYYY-MM-DD" for time queries
  source_timestamp  INTEGER NOT NULL,
  created_at        INTEGER DEFAULT (strftime('%s', 'now')),
  PRIMARY KEY (schema_name, cid)
);

Additional supporting tables: sdn_record_source_tags / sdn_record_source_summary (provenance), sdn_directory (identity/EPM directory), sdn_log_index (hash-chained PLG logs), and sdn_pin_ledger (what this node has pinned and why).

Content Addressing

Every record gets a real CIDv1 — raw codec (0x55), sha2-256 multihash, default base32 multibase — computed the same way a stock IPFS client computes it (bafybei...). This is not a bespoke hash format: any IPFS/Kubo node can fetch and pin an SDN record CID directly, and by default an SDN node pins published/imported records itself through a local Kubo RPC endpoint (admin.ipfs_api_url, default http://127.0.0.1:5001; pinning is on unless an operator explicitly sets it to an empty string). Some rows written before this content-addressing scheme landed may still carry legacy bare-hex digests instead of CIDs; those remain readable but nothing writes them anymore.

Data Lifecycle

Incoming FlatBuffer bytes
    → Validate file_identifier (bytes 4-7) against schema registry
    → Compute CID = CIDv1(raw, sha2-256, bytes)
    → Append bytes to flatsql-streams/*.flatsql; record (stream_path, stream_offset, record_length)
    → INSERT into sds_p_<producer>__<standard>(cid, peer_id, timestamp, stream_path, stream_offset, record_length, signature_hex)
    → Extract index fields (NORAD_CAT_ID, epoch, entity_id) → INSERT into sdn_record_index
    → Pin the CID via the local Kubo RPC (if admin.ipfs_api_url is set)
    → Export latest product shard/index when publishing a dataset
    → Build signed DPM and signed PNM for the publication

PNM — Publish Notification Messages

PNM is a lightweight signed announcement. For a dataset update, the PNM normally points at the DPM CID and carries the canonical FILE_ID for that publication. It does not contain the dataset and it is not the authoritative manifest; it tells subscribers which DPM to fetch and verify.

PNM Fields

Latest Dataset Resolution

1. Select provider peer ID and standard, e.g. celestrak.eth + OMM.fbs.
2. Resolve the latest signed PNM from PubSub/history, peer API, or feed-head cache.
3. Verify the PNM signature with the provider public key.
4. Fetch the PNM CID as a DPM.
5. Verify DPM signature, FILE_ID, source hashes, byte hashes, and schema hash.
6. Fetch DPM-listed DATA_SHARD / QUERY_INDEX assets by CID, or open the DPM query protocol.
7. Import only records that verify against the DPM roots and hashes.

Topic Map

PNM announcements use the canonical per-schema PubSub topic. Dataset assets are fetched by CID or query protocol; modules and private deliveries use their own storefront/grant channels.

/spacedatanetwork/sds/PNM.fbs                     ← signed PNM announcements (and one such topic per SDS schema, e.g. /spacedatanetwork/sds/OMM.fbs)
/space-data-network/feed-heads/1.0.0/{standard}   ← dataset feed-head announcements where enabled
/spacedatanetwork/edge-relays                     ← edge relay announcements
/sdn/storefront/listings                          ← PLG marketplace listings
/space-data-network/module-delivery/1.0.0         ← encrypted module grant/delivery protocol (libp2p stream protocol, not PubSub)

DPM — Dataset Publication Manifests

A DPM is the immutable signed publication contract for one dataset update. It binds the provider identity, dataset/update IDs, canonical FILE_ID, source batch provenance, data shard and query-index CIDs, byte lengths, SHA-256 hashes, schema hash, query replay metadata, optional encryption metadata, and provider signature.

What a Subscriber Verifies

• The PNM signature was made by the expected provider and its CID resolves to the DPM.
• PNM.FILE_ID, DPM.FILE_ID, and every DPMAsset.FILE_ID match.
• The DPM provider signature verifies over the canonical manifest bytes or digest.
• Each fetched shard/index byte stream matches the DPM CID, byte length, and SHA-256 fields.
• Imported records match the DPM schema, query result hash, and declared completeness roots when provided.

CelesTrak Latest Product

The CelesTrak provider node should keep the newest OMM product as the authoritative live dataset, publish a new DPM every time the source changes, and announce that DPM through PNM. Core pinning nodes can materialize and pin the latest product. Archive nodes can retain historical DPM/shard sets and serve historical queries through REST/query endpoints.

Streaming & Channels

SDN subscriptions support three delivery modes for real-time and batch data consumption:

Subscription Configuration

// JavaScript — create a filtered subscription
const sub = subscriptionManager.createSubscription({
  dataTypes: ['CDM', 'OMM'],          // Which SDS record codes
  sourcePeers: ['all'],                 // From any peer (or specific IDs)
  encrypted: false,                     // Plaintext or encrypted
  streaming: true,                      // Real-time delivery
  filters: [{                           // Optional field filters
    field: 'COLLISION_PROBABILITY',
    operator: 'gt',
    value: 1e-4
  }],
  rateLimit: 100,                       // Messages per minute
  ttl: 3600000                          // 1 hour TTL
});

Routing Headers

Each message carries a binary RoutingHeader with metadata for efficient routing:

Encrypted Messages Between Nodes

SDN supports end-to-end encryption for private data delivery. Encryption uses the recipient's X25519 public key (derived from their xpub via SLIP-10).

Encrypted conjunction assessment uses those private channels for maneuver ephemeris screening without broadcasting planned maneuvers to competitors. Operators can keep MPE/EPM inputs encrypted, choose the grant or channel used for screening, and export CA results with provenance without revealing maneuver intent to the public DHT.

Encryption Modes

ECIES Encryption Flow

Sender:
  1. Look up recipient's X25519 public key (from xpub)
  2. Generate ephemeral X25519 keypair
  3. shared_secret = X25519(ephemeral_private, recipient_public)
  4. key = HKDF-SHA256(shared_secret)
  5. ciphertext = ChaCha20-Poly1305(key, nonce, plaintext)
  6. envelope = [ephemeral_pubkey | nonce | ciphertext | auth_tag]

Recipient:
  1. Extract ephemeral public key from envelope
  2. shared_secret = X25519(own_private, ephemeral_public)
  3. key = HKDF-SHA256(shared_secret)
  4. plaintext = ChaCha20-Poly1305.decrypt(key, nonce, ciphertext)

Encrypted Delivery Channels

The marketplace uses per-buyer encrypted GossipSub topics for data delivery:

/sdn/data/{listing_id}/{buyer_peer_id}

Only the buyer (with the matching X25519 private key) can decrypt the data.

Wallet Identity — Users Are Keys

SDN has no separate concept of "user accounts". A user IS their HD wallet key. The same BIP-39 mnemonic derives identical keys whether used in a Go server node, a browser via sdn-js, or the desktop app. Logging in = deriving your keys.

Key Derivation

Multi-Account

One mnemonic can manage multiple SDN identities by varying the account' segment:

• m/44'/0'/0'/... — Operator identity
• m/44'/0'/1'/... — Sensor identity
• m/44'/0'/2'/... — Analytics service identity

TOFU Key Binding

On first login, the Ed25519 signing public key is bound to the xpub. Subsequent logins verify this binding. This prevents key substitution attacks while allowing passwordless key rotation.

Browser Nodes with sdn-js

The sdn-js SDK turns any browser or Node.js process into a full SDN peer. A browser tab running sdn-js has the same cryptographic identity model as a Go server node — the only difference is transport (WebSocket/WebTransport instead of raw TCP).

Code Example

import { SDNNode, identityFromMnemonic } from '@spacedatanetwork/sdn-js';

// Same mnemonic as your server node = same identity
const identity = await identityFromMnemonic('abandon abandon ...');

const node = await SDNNode.create({
  identity,
  edgeRelays: ['wss://relay.example.invalid/...'],
  enableStorage: true,  // IndexedDB
});

// Subscribe to conjunction warnings
await node.subscribe('CDM', (data, peerId) => {
  console.log('Conjunction alert from', peerId);
});

// Publish orbital data
await node.publish('OMM', flatBufferBytes);

// Query remote server via REST
import { SDNClient } from '@spacedatanetwork/sdn-js';
const client = await SDNClient.resolve('spaceaware.io');
const records = await client.query({
  schema: 'OMM.fbs',
  noradCatId: 25544,  // ISS
  day: '2026-03-09'
});

Plugin System

SDN loads plugins as WASI-compiled WebAssembly modules via Wazero (pure Go runtime). Plugins run in a sandboxed 32MB memory space and communicate with the host via shared memory and FlatBuffer messages.

Required Plugin Exports

Host Functions Available

Plugin Lifecycle

Load WASM → Wazero runtime (32MB limit)
  → Instantiate WASI imports
  → Register sdn.* host functions
  → Call _initialize (if present)
  → Call plugin_init(identity_seed)
  → Call plugin_get_metadata() → register protocols
  → Register HTTP routes: /api/v1/plugins/{id}/*
  → Register libp2p stream handlers
  → Plugin is running, handles requests via FlatBuffer I/O

See the plugin-demo directory for a complete annotated example plugin with build instructions.

Throughput Benchmarks

The SDN server includes comprehensive benchmark tests for measuring SpaceDataStandards message throughput. All benchmarks use the current SDS FlatBuffer schemas.

Running Benchmarks

# Run all SDS benchmarks
cd sdn-server
go test -bench=. ./internal/sds/

# Run specific schema benchmarks
go test -bench=BenchmarkOMM ./internal/sds/
go test -bench=BenchmarkEPM ./internal/sds/
go test -bench=BenchmarkPNM ./internal/sds/
go test -bench=BenchmarkCAT ./internal/sds/

# Run with memory allocation stats
go test -bench=. -benchmem ./internal/sds/

# Run schema comparison suite
go test -bench=BenchmarkSchemaComparison ./internal/sds/

Benchmark Results

Actual throughput on Apple M3 Ultra (results vary by hardware):

Batch Processing

# Test batch message processing (100 messages)
go test -bench=BenchmarkBatchProcessing ./internal/sds/

# Test parallel processing
go test -bench=BenchmarkParallelProcessing ./internal/sds/

Message Sizes

# Report message sizes in bytes
go test -bench=BenchmarkMessageSize ./internal/sds/

Roundtrip Tests

Roundtrip tests verify that data survives serialization and deserialization without loss.

Running Roundtrip Tests

# Run all SDS roundtrip tests
go test -v -run=Roundtrip ./internal/sds/

# Run vCard roundtrip tests
go test -v -run=Roundtrip ./internal/vcard/

# Run with race detection
go test -race -run=Roundtrip ./internal/...

What's Tested

• OMM roundtrip: All orbital elements preserved through serialize/deserialize
• EPM roundtrip: Entity profiles with addresses, keys, alternate names
• PNM roundtrip: Publish notifications with CID, signatures, timestamps
• CAT roundtrip: Catalog entries with orbital parameters
• vCard roundtrip: EPM to vCard to EPM conversion
• QR code roundtrip: EPM to QR to EPM conversion

Example Test Output

=== RUN   TestOMMRoundtrip
--- PASS: TestOMMRoundtrip (0.00s)
=== RUN   TestEPMRoundtripWithAddress
--- PASS: TestEPMRoundtripWithAddress (0.00s)
=== RUN   TestEPMVCardRoundtrip
--- PASS: TestEPMVCardRoundtrip (0.00s)
=== RUN   TestEPMQRFullRoundtrip
--- PASS: TestEPMQRFullRoundtrip (0.02s)
PASS

EPM / vCard / QR Code Conversion Tests

The complete conversion pipeline enables Entity Profile Messages to be shared as standard vCards and embedded in QR codes for easy scanning.

Conversion Flow

┌─────────────────────────────────────────────────────────────────┐
│                    Full Conversion Pipeline                      │
└─────────────────────────────────────────────────────────────────┘

     EPM (FlatBuffer)                    vCard 4.0 String
    ┌──────────────┐                   ┌──────────────────┐
    │ DN           │                   │ BEGIN:VCARD      │
    │ LEGAL_NAME   │  ──EPMToVCard──►  │ VERSION:4.0      │
    │ FAMILY_NAME  │                   │ FN:John Doe      │
    │ GIVEN_NAME   │                   │ N:Doe;John;;;    │
    │ EMAIL        │  ◄──VCardToEPM──  │ ORG:Acme Corp    │
    │ TELEPHONE    │                   │ EMAIL:john@...   │
    │ ADDRESS      │                   │ ADR:;;123 Main...│
    │ KEYS[]       │                   │ X-SIGNING-KEY:...│
    └──────────────┘                   └──────────────────┘
           │                                    │
           │                                    │
           ▼                                    ▼
    ┌──────────────┐                   ┌──────────────────┐
    │   EPMToQR    │                   │   VCardToQR      │
    │   (direct)   │                   │                  │
    └──────────────┘                   └──────────────────┘
           │                                    │
           └────────────┬───────────────────────┘
                        ▼
               ┌──────────────────┐
               │   QR Code PNG    │
               │   (scannable)    │
               └──────────────────┘
                        │
           ┌────────────┴───────────────────────┐
           ▼                                    ▼
    ┌──────────────┐                   ┌──────────────────┐
    │   QRToEPM    │                   │   QRToVCard      │
    │   (direct)   │                   │                  │
    └──────────────┘                   └──────────────────┘

Running EPM/QR Tests

# Run all vCard tests
go test -v ./internal/vcard/

# Run QR-specific tests
go test -v -run=QR ./internal/vcard/

# Run the full roundtrip test
go test -v -run=TestEPMQRFullRoundtrip ./internal/vcard/

# Run benchmarks
go test -bench=. ./internal/vcard/

Test Coverage

Field Mapping

Complete mapping between EPM FlatBuffer fields and vCard 4.0 properties:

QR Code Specifications

• Default size: 256x256 pixels
• Maximum size: 4096x4096 pixels
• Error correction: Medium level (recovers ~15% damage)
• Format: PNG
• Typical capacity: ~1KB of vCard data

Conversion Throughput

Benchmark results on Apple M3 Ultra:

Code Example

import "github.com/spacedatanetwork/sdn-server/internal/vcard"
import "github.com/spacedatanetwork/sdn-server/internal/sds"

// Create an Entity Profile
epmBytes := sds.NewEPMBuilder().
    WithDN("Dr. Jane Smith").
    WithLegalName("Space Agency").
    WithFamilyName("Smith").
    WithGivenName("Jane").
    WithEmail("jane@spaceagency.gov").
    WithTelephone("+1-555-0199").
    WithAddress("1 Rocket Way", "Houston", "TX", "77058", "USA").
    WithJobTitle("Chief Scientist").
    WithKeys("0xed25519signingkey", "0xed25519encryptionkey").
    Build()

// Convert to vCard string
vcardStr, err := vcard.EPMToVCard(epmBytes)
// Result:
// BEGIN:VCARD
// VERSION:4.0
// FN:Dr. Jane Smith
// N:Smith;Jane;;;
// ORG:Space Agency
// EMAIL:jane@spaceagency.gov
// TEL:+1-555-0199
// TITLE:Chief Scientist
// ADR:;;1 Rocket Way;Houston;TX;77058;USA
// X-SIGNING-KEY:0xed25519signingkey
// X-ENCRYPTION-KEY:0xed25519encryptionkey
// END:VCARD

// Generate QR code (256x256 PNG)
pngData, err := vcard.EPMToQR(epmBytes, 256)
os.WriteFile("entity-profile.png", pngData, 0644)

// Scan QR code back to EPM
scannedEPM, err := vcard.QRToEPM(pngData)
// scannedEPM contains the original entity profile