From 184a3be4139a08142e3f7ec633aac33dbb895030 Mon Sep 17 00:00:00 2001 From: Stephan D Date: Sat, 12 Sep 2026 21:04:42 +0200 Subject: [PATCH 01/11] Build local product stack with Vault-managed secrets (#42) --- .dockerignore | 16 ++ .gitignore | 6 + AGENTS.md | 10 + Makefile | 39 +++- README.md | 17 ++ .../notifications/service/integration_test.go | 4 +- backend/product/app/runtime/bootstrap_test.go | 2 +- .../public/verification_integration_test.go | 2 +- .../customers/internal/customer_test.go | 2 +- backend/product/twofactor/service_test.go | 2 +- compose.yml | 145 ++++++++++++++ dev/agents.py | 43 +++++ dev/bootstrap.py | 105 ++++++++++ dev/config/bff.yml | 39 ++++ dev/config/nginx.conf | 17 ++ dev/config/notifications.yml | 27 +++ dev/images.py | 72 +++++++ dev/images/backend.Dockerfile | 21 ++ dev/images/frontend.Dockerfile | 17 ++ dev/images/nats.Dockerfile | 10 + dev/local.py | 47 +++++ dev/start-mongo.sh | 6 + dev/start-nats.sh | 13 ++ dev/start-service.sh | 14 ++ dev/start-vault.sh | 16 ++ dev/test_local.py | 62 ++++++ dev/vault.py | 117 +++++++++++ dev/vault/agent.hcl | 14 ++ dev/vault/server.hcl | 10 + dev/verify.py | 72 +++++++ docs/implementation.md | 52 +++++ docs/local-development.md | 182 ++++++++++++++++++ 32 files changed, 1191 insertions(+), 10 deletions(-) create mode 100644 .dockerignore create mode 100644 compose.yml create mode 100644 dev/agents.py create mode 100644 dev/bootstrap.py create mode 100644 dev/config/bff.yml create mode 100644 dev/config/nginx.conf create mode 100644 dev/config/notifications.yml create mode 100644 dev/images.py create mode 100644 dev/images/backend.Dockerfile create mode 100644 dev/images/frontend.Dockerfile create mode 100644 dev/images/nats.Dockerfile create mode 100644 dev/local.py create mode 100755 dev/start-mongo.sh create mode 100755 dev/start-nats.sh create mode 100755 dev/start-service.sh create mode 100755 dev/start-vault.sh create mode 100644 dev/test_local.py create mode 100644 dev/vault.py create mode 100644 dev/vault/agent.hcl create mode 100644 dev/vault/server.hcl create mode 100644 dev/verify.py create mode 100644 docs/local-development.md diff --git a/.dockerignore b/.dockerignore new file mode 100644 index 0000000..0942622 --- /dev/null +++ b/.dockerignore @@ -0,0 +1,16 @@ +.git +.local +**/.dart_tool +**/build +**/.env* +**/node_modules +**/*.pb.go +frontend/app/lib/generated +**/.DS_Store +dist +website +**/config.yml +**/tls +**/*.pem +**/*.key +**/__pycache__ diff --git a/.gitignore b/.gitignore index e999375..9c09e99 100644 --- a/.gitignore +++ b/.gitignore @@ -34,3 +34,9 @@ pnpm-debug.log* /backend/notifications/interface/notifications/**/*.pb.go /backend/notifications/config.yml /backend/notifications/notifications + +# Local container pins, Vault bootstrap material and service identities +/.local/ + +# Python bytecode +__pycache__/ diff --git a/AGENTS.md b/AGENTS.md index a2f2a0e..687ac94 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -22,6 +22,16 @@ generated outputs. Document the generation workflow so a clean checkout can build without committed generated code. +## Container versions + +Use only the latest stable upstream releases for infrastructure, product base +images, build stages and disposable test containers. Exclude prereleases and +vendor mainline channels when a separate stable channel exists. Resolve stable +versions when building, pull fresh manifests, and pin the resolved image digests +for that local build. Fail when stable release resolution fails; never silently +fall back to an older cached version. Keep the reproducible resolver tracked and +its generated local pins ignored by Git. + ## Data and state architecture Preserve explicit **DTO → mapper → service → data model → provider → UI** diff --git a/Makefile b/Makefile index 0877454..bc85743 100644 --- a/Makefile +++ b/Makefile @@ -17,10 +17,12 @@ test-frontend: .PHONY: test-backend lint-backend run-backend test-backend: generate-notifications - cd backend/shared && go test -race ./... - cd backend/notifications/interface && go test -race ./... - cd backend/notifications && go test -race ./... - cd backend/product && go test -race ./... + docker pull mongo:latest + docker pull nats:latest + cd backend/shared && go test -race -count=1 ./... + cd backend/notifications/interface && go test -race -count=1 ./... + cd backend/notifications && go test -race -count=1 ./... + cd backend/product && go test -race -count=1 ./... lint-backend: generate-notifications cd backend/product && go test ./architecture @@ -49,3 +51,32 @@ package-notifications-interface: generate-notifications mkdir -p dist tar -czf dist/notifications-proto-v1.tar.gz -C backend/notifications/interface proto README.md tar -czf dist/notifications-go-v1.tar.gz -C backend/notifications/interface go.mod go.sum notifications README.md + +# Local product stack. Releases are resolved afresh, then pinned by digest. +COMPOSE = docker compose --env-file .local/dev/images.env +.PHONY: build up down logs ps test-dev +build: + python3 dev/images.py + $(COMPOSE) pull vault mongo + $(COMPOSE) build --pull + +up: + python3 dev/local.py up + +down: + $(COMPOSE) down + +logs: + $(COMPOSE) logs --tail=100 -f + +ps: + $(COMPOSE) ps + +test-dev: + python3 -m unittest discover -s dev -p 'test_*.py' + python3 dev/local.py verify + +.PHONY: test-dev-infra +test-dev-infra: + python3 -m unittest discover -s dev -p 'test_*.py' + python3 dev/local.py verify-infra diff --git a/README.md b/README.md index de8f711..468814f 100644 --- a/README.md +++ b/README.md @@ -32,6 +32,23 @@ Each application owns its dependencies and build configuration. Shared Dart and Flutter packages can be reused by a future mobile app; mobile runners are not currently included. Local `.vscode/` configuration is ignored. +## Local Docker development + +```sh +make build && make up +``` + +This builds the product images and starts an isolated Vault, authenticated MongoDB +replica set and NATS JetStream. First startup intentionally stops after bootstrap +until you set the Resend API key and verified sender in Vault's `kv/email` secret. +Then rerun `make up` to start notifications, BFF and the frontend at +[https://localhost:5126](https://localhost:5126). + +All image inputs resolve to current stable releases and are pinned by digest for +the build. Services fetch secrets from Vault; Mongo application credentials come +from Vault's database plugin. See the [local development guide](docs/local-development.md) +for Vault provisioning, local HTTPS trust, lifecycle commands and verification. + ## Run the consumer app Requires Flutter **3.47.0 or later** with Dart **3.13.0 or later** (below Dart 4), matching diff --git a/backend/notifications/service/integration_test.go b/backend/notifications/service/integration_test.go index 1be0f83..599304e 100644 --- a/backend/notifications/service/integration_test.go +++ b/backend/notifications/service/integration_test.go @@ -50,7 +50,7 @@ func (s *sender) Send(_ context.Context, id string, message model.Email) email.O func TestDurableDelivery(t *testing.T) { ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute) defer cancel() - container, err := mongodb.Run(ctx, "mongo:8.0") + container, err := mongodb.Run(ctx, "mongo:latest") if err != nil { t.Fatal(err) } @@ -192,7 +192,7 @@ func TestDurableDelivery(t *testing.T) { } } // Verify the real JetStream publisher/consumer path with an isolated broker. - ncContainer, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{ContainerRequest: testcontainers.ContainerRequest{Image: "nats:2.11-alpine", ExposedPorts: []string{"4222/tcp"}, Cmd: []string{"-js"}, WaitingFor: wait.ForListeningPort("4222/tcp")}, Started: true}) + ncContainer, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{ContainerRequest: testcontainers.ContainerRequest{Image: "nats:latest", ExposedPorts: []string{"4222/tcp"}, Cmd: []string{"-js"}, WaitingFor: wait.ForListeningPort("4222/tcp")}, Started: true}) if err != nil { t.Fatal(err) } diff --git a/backend/product/app/runtime/bootstrap_test.go b/backend/product/app/runtime/bootstrap_test.go index 5da7bec..bbacf2c 100644 --- a/backend/product/app/runtime/bootstrap_test.go +++ b/backend/product/app/runtime/bootstrap_test.go @@ -24,7 +24,7 @@ import ( func TestProfileStartsOnEmptyDatabase(t *testing.T) { ctx, cancel := context.WithTimeout(context.Background(), time.Minute) defer cancel() - container, err := mongodb.Run(ctx, "mongo:8", mongodb.WithReplicaSet("rs")) + container, err := mongodb.Run(ctx, "mongo:latest", mongodb.WithReplicaSet("rs")) require.NoError(t, err) t.Cleanup(func() { require.NoError(t, container.Terminate(context.Background())) }) uri, err := container.ConnectionString(ctx) diff --git a/backend/product/auth/http/public/verification_integration_test.go b/backend/product/auth/http/public/verification_integration_test.go index cfd2e9b..b89ce5a 100644 --- a/backend/product/auth/http/public/verification_integration_test.go +++ b/backend/product/auth/http/public/verification_integration_test.go @@ -44,7 +44,7 @@ func (m *capturedMail) Send(_ context.Context, r notifications.Message) error { func TestSignupActivationAndLoginEmailFlow(t *testing.T) { ctx, cancel := context.WithTimeout(context.Background(), time.Minute) defer cancel() - container, err := mongodb.Run(ctx, "mongo:8", mongodb.WithReplicaSet("rs")) + container, err := mongodb.Run(ctx, "mongo:latest", mongodb.WithReplicaSet("rs")) require.NoError(t, err) t.Cleanup(func() { require.NoError(t, container.Terminate(context.Background())) }) uri, err := container.ConnectionString(ctx) diff --git a/backend/product/customers/internal/customer_test.go b/backend/product/customers/internal/customer_test.go index 16fec93..bc5eec4 100644 --- a/backend/product/customers/internal/customer_test.go +++ b/backend/product/customers/internal/customer_test.go @@ -29,7 +29,7 @@ func customer(login string) *customermodel.Account { func TestCustomerLifecycleWithoutTenancy(t *testing.T) { ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute) defer cancel() - container, err := mongodb.Run(ctx, "mongo:8", mongodb.WithReplicaSet("rs")) + container, err := mongodb.Run(ctx, "mongo:latest", mongodb.WithReplicaSet("rs")) require.NoError(t, err) t.Cleanup(func() { require.NoError(t, container.Terminate(context.Background())) }) uri, err := container.ConnectionString(ctx) diff --git a/backend/product/twofactor/service_test.go b/backend/product/twofactor/service_test.go index 7f74473..059cd7f 100644 --- a/backend/product/twofactor/service_test.go +++ b/backend/product/twofactor/service_test.go @@ -43,7 +43,7 @@ func (d *capturedDriver) Send(_ context.Context, m model.Delivery) error { func TestServiceTransactionsAndFactors(t *testing.T) { ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute) defer cancel() - container, err := mongodb.Run(ctx, "mongo:8", mongodb.WithReplicaSet("rs")) + container, err := mongodb.Run(ctx, "mongo:latest", mongodb.WithReplicaSet("rs")) require.NoError(t, err) t.Cleanup(func() { require.NoError(t, container.Terminate(context.Background())) }) uri, err := container.ConnectionString(ctx) diff --git a/compose.yml b/compose.yml new file mode 100644 index 0000000..169d6fa --- /dev/null +++ b/compose.yml @@ -0,0 +1,145 @@ +name: sxchnge-local + +x-agent: &agent + restart: unless-stopped + environment: + VAULT_ADDR: http://vault:8200 + tmpfs: + - /run/sxchnge:uid=100,gid=1000,mode=0700 + +x-backend-build: &backend-build + context: . + dockerfile: dev/images/backend.Dockerfile + +services: + vault: + image: ${VAULT_IMAGE:?Run make build first} + user: root + entrypoint: [/bin/sh, /etc/vault/start.sh] + restart: unless-stopped + ports: ['127.0.0.1:18200:8200'] + volumes: + - vault-data:/vault/file + - ./dev/vault/server.hcl:/etc/vault/server.hcl:ro + - ./dev/start-vault.sh:/etc/vault/start.sh:ro + - ./.local/dev/vault:/bootstrap:ro + healthcheck: + test: [CMD-SHELL, 'VAULT_ADDR=http://127.0.0.1:8200 vault status >/dev/null'] + interval: 3s + timeout: 3s + retries: 40 + + mongo: + image: ${MONGO_IMAGE:?Run make build first} + entrypoint: [/bin/sh, /etc/mongo/start.sh] + command: [mongod, --replSet, rs0, --bind_ip_all, --keyFile, /tmp/mongo-keyfile] + restart: unless-stopped + environment: + MONGO_INITDB_ROOT_USERNAME: local-admin + MONGO_INITDB_ROOT_PASSWORD_FILE: /run/mongo/password + volumes: + - ./dev/start-mongo.sh:/etc/mongo/start.sh:ro + - mongo-data:/data/db + - ./.local/dev/mongo:/run/mongo:ro + healthcheck: + test: [CMD, mongosh, --quiet, --eval, 'quit(db.hello().isWritablePrimary ? 0 : 1)'] + interval: 3s + timeout: 5s + retries: 40 + + nats: + <<: *agent + image: sxchnge-local/nats:dev + build: + context: . + dockerfile: dev/images/nats.Dockerfile + args: + NATS_IMAGE: ${NATS_IMAGE:?Run make build first} + VAULT_IMAGE: ${VAULT_IMAGE:?Run make build first} + volumes: + - nats-data:/data + - ./.local/dev/agents/nats:/etc/agent:ro + depends_on: + vault: {condition: service_healthy} + healthcheck: + test: [CMD, wget, -q, -O, /dev/null, 'http://127.0.0.1:8222/healthz?js-enabled-only=true'] + interval: 3s + timeout: 3s + retries: 40 + + notifications: + <<: *agent + image: sxchnge-local/notifications:dev + build: + <<: *backend-build + args: + GO_IMAGE: ${GO_IMAGE:?Run make build first} + VAULT_IMAGE: ${VAULT_IMAGE:?Run make build first} + MODULE: notifications + environment: + VAULT_ADDR: http://vault:8200 + SXCHNGE_SERVICE: notifications + volumes: + - ./.local/dev/agents/notifications:/etc/agent:ro + depends_on: + vault: {condition: service_healthy} + mongo: {condition: service_healthy} + nats: {condition: service_healthy} + healthcheck: + test: [CMD, nc, -z, 127.0.0.1, '8083'] + interval: 3s + timeout: 3s + retries: 40 + + bff: + <<: *agent + image: sxchnge-local/bff:dev + build: + <<: *backend-build + args: + GO_IMAGE: ${GO_IMAGE:?Run make build first} + VAULT_IMAGE: ${VAULT_IMAGE:?Run make build first} + MODULE: product + environment: + VAULT_ADDR: http://vault:8200 + SXCHNGE_SERVICE: bff + SXCHNGE_DOMAIN: localhost:5126 + SXCHNGE_API_PROTOCOL: https + SXCHNGE_API_ENDPOINT: /api/v1 + volumes: + - ./.local/dev/agents/bff:/etc/agent:ro + depends_on: + notifications: {condition: service_healthy} + mongo: {condition: service_healthy} + healthcheck: + test: [CMD, wget, -q, -O, /dev/null, 'http://127.0.0.1:8082/api/v1/health'] + interval: 3s + timeout: 3s + retries: 40 + + frontend: + image: sxchnge-local/frontend:dev + build: + context: . + dockerfile: dev/images/frontend.Dockerfile + args: + UBUNTU_IMAGE: ${UBUNTU_IMAGE:?Run make build first} + NGINX_IMAGE: ${NGINX_IMAGE:?Run make build first} + FLUTTER_VERSION: ${FLUTTER_VERSION:?Run make build first} + FLUTTER_COMMIT: ${FLUTTER_COMMIT:?Run make build first} + restart: unless-stopped + ports: ['127.0.0.1:5126:8443'] + volumes: + - ./.local/dev/frontend:/etc/tls:ro + depends_on: + bff: {condition: service_healthy} + healthcheck: + test: [CMD, wget, -q, --no-check-certificate, -O, /dev/null, 'https://127.0.0.1:8443/'] + interval: 3s + timeout: 3s + retries: 40 + +volumes: + vault-data: + mongo-data: + nats-data: diff --git a/dev/agents.py b/dev/agents.py new file mode 100644 index 0000000..70e71fe --- /dev/null +++ b/dev/agents.py @@ -0,0 +1,43 @@ +"""Build per-service Vault Agent configs (no secret values in configs).""" +import json +from pathlib import Path + +BASE = (Path(__file__).parent / 'vault' / 'agent.hcl').read_text() + + +def kv(path, field): + return '{{ with secret "kv/data/' + path + '" }}{{ .Data.data.' + field + ' }}{{ end }}' + + +def config(service): + env = {} + if service == 'nats': + env['NATS_TOKEN'] = kv('nats', 'token') + else: + database = 'sxchnge' if service == 'bff' else 'sxchnge_notifications' + env['SXCHNGE_MONGO_URI'] = ('{{ with secret "database/creds/' + service + '" }}' + 'mongodb://{{ .Data.username | urlquery }}:{{ .Data.password | urlquery }}' + '@mongo:27017/' + database + '?authSource=admin&replicaSet=rs0{{ end }}') + env['TLS_CA'] = kv('tls', 'ca') + if service == 'bff': + env['SXCHNGE_JWT_SECRET'] = kv('bff', 'jwt') + env['SXCHNGE_NOTIFICATIONS_TOKEN'] = kv('caller', 'token') + else: + env['SXCHNGE_NOTIFICATIONS_PRODUCT_TOKEN'] = kv('caller', 'token') + env['SXCHNGE_NATS_URL'] = ('{{ with secret "kv/data/nats" }}' + 'nats://{{ .Data.data.token }}@nats:4222{{ end }}') + env['TLS_CERT'] = kv('notifications-tls', 'certificate') + env['TLS_KEY'] = kv('notifications-tls', 'private_key') + env['EMAIL_API_KEY'] = kv('email', 'api_key') + # JSON is valid YAML and escapes arbitrary provider configuration. + env['EMAIL_CONFIG'] = ('{{ with secret "kv/data/email" }}' + 'email:\n driver: {{ .Data.data.driver | toJSON }}\n' + ' settings:\n api_key_env: EMAIL_API_KEY\n' + ' sender_address: {{ .Data.data.sender_address | toJSON }}\n' + ' sender_name: {{ .Data.data.sender_name | toJSON }}\n{{ end }}') + result = BASE + for name, template in env.items(): + result += '\nenv_template ' + json.dumps(name) + ' {\n contents = ' + json.dumps(template) + '\n error_on_missing_key = true\n}\n' + command = '/usr/local/bin/start-nats' if service == 'nats' else '/usr/local/bin/start-service' + result += '\nexec {\n command = [' + json.dumps(command) + ']\n restart_on_secret_changes = "always"\n restart_stop_signal = "SIGTERM"\n}\n' + return result diff --git a/dev/bootstrap.py b/dev/bootstrap.py new file mode 100644 index 0000000..a89e719 --- /dev/null +++ b/dev/bootstrap.py @@ -0,0 +1,105 @@ +"""Idempotent provisioning for the isolated local Compose project.""" +import json +import os +import subprocess +import time + +from agents import config +from vault import STATE, write + +ROOT = STATE.parents[1] +COMPOSE = ['docker', 'compose', '--env-file', str(STATE / 'images.env')] + + +def compose(*args, capture=False, input=None): + result = subprocess.run(COMPOSE + list(args), cwd=ROOT, input=input, text=True, + capture_output=capture) + if result.returncode: + # Bootstrap stdin and child errors may contain credentials. Never echo them. + raise RuntimeError('Compose command failed: ' + ' '.join(args[:3])) + return result.stdout if capture else None + + +def mongo(script, password): + auth = 'const admin = db.getSiblingDB("admin"); admin.auth("local-admin", ' + json.dumps(password) + ');\n' + return compose('exec', '-T', 'mongo', 'mongosh', '--quiet', '--file', '/dev/stdin', capture=True, input=auth + script) + + +def provision_mongo(vault): + secret = vault.ensure('bootstrap/mongo', lambda: {'password': vault.random(), 'keyfile': vault.random(384), 'manager_password': vault.random()}) + write(STATE / 'mongo' / 'password', secret['password']) + write(STATE / 'mongo' / 'keyfile', secret['keyfile']) + compose('up', '-d', 'mongo') + for attempt in range(90): + try: + mongo('try { rs.status(); } catch(e) { if(e.code === 94) rs.initiate({_id:"rs0",members:[{_id:0,host:"mongo:27017"}]}); else throw e; }', secret['password']) + if 'true' in mongo('print(db.hello().isWritablePrimary)', secret['password']): + break + except RuntimeError: + pass + time.sleep(2) + else: + raise RuntimeError('Mongo replica set did not become primary') + if vault.call('database/config/mongo', missing=True) is None: + mongo('if (!admin.getUser("vault-manager")) admin.createUser({user:"vault-manager",pwd:' + json.dumps(secret['manager_password']) + ',roles:[{role:"userAdminAnyDatabase",db:"admin"}]});', secret['password']) + vault.call('database/config/mongo', {'plugin_name': 'mongodb-database-plugin', + 'allowed_roles': ['bff', 'notifications'], + 'connection_url': 'mongodb://{{username}}:{{password}}@mongo:27017/admin?replicaSet=rs0', + 'username': 'vault-manager', 'password': secret['manager_password']}) + vault.call('database/rotate-root/mongo', {}) + for role, database in [('bff', 'sxchnge'), ('notifications', 'sxchnge_notifications')]: + vault.call('database/roles/' + role, {'db_name': 'mongo', + 'creation_statements': json.dumps({'db': 'admin', 'roles': [{'role': 'readWrite', 'db': database}]}), + 'default_ttl': '1h', 'max_ttl': '24h'}) + # Public browser client: an identifier, never a client secret in the web bundle. + mongo('db.getSiblingDB("sxchnge").clients.updateOne({clientId:"sxchnge-local-web"}, {$setOnInsert: {clientName:"Local browser",allowedCIDRs:["0.0.0.0/0","::/0"],allowedScopes:["account"],redirectURIs:["https://localhost:5126"],grantTypes:["password","refresh_token"],tokenEndpointAuthMethod:"none",isRevoked:false}}, {upsert:true});', secret['password']) + + +def provision_agents(vault): + paths = {'bff': ['bff', 'caller', 'tls'], + 'notifications': ['caller', 'tls', 'notifications-tls', 'nats', 'email'], + 'nats': ['nats']} + for service, secrets in paths.items(): + directory = STATE / 'agents' / service + directory.mkdir(parents=True, exist_ok=True, mode=0o755) + os.chmod(directory, 0o755) + policy = ''.join('path "kv/data/' + p + '" { capabilities = ["read"] }\n' for p in secrets) + if service != 'nats': + policy += 'path "database/creds/' + service + '" { capabilities = ["read"] }\n' + vault.call('sys/policies/acl/' + service, {'policy': policy}) + vault.call('auth/approle/role/' + service, {'token_policies': [service], + 'token_period': '1h', 'secret_id_ttl': '0', 'secret_id_num_uses': 0}) + role_id = vault.call('auth/approle/role/' + service + '/role-id')['data']['role_id'] + write(directory / 'role-id', role_id, 0o444) + if not (directory / 'secret-id').exists(): + secret_id = vault.call('auth/approle/role/' + service + '/secret-id', {})['data']['secret_id'] + write(directory / 'secret-id', secret_id, 0o444) + write(directory / 'config.hcl', config(service), 0o444) + + +def up(vault): + if not (STATE / 'images.env').exists(): + raise RuntimeError('Run make build before make up') + (STATE / 'vault').mkdir(parents=True, exist_ok=True, mode=0o755) + compose('up', '-d', 'vault') + vault.initialize() + vault.mounts() + vault.ensure('nats', lambda: {'token': vault.random()}) + vault.ensure('caller', lambda: {'token': vault.random()}) + vault.ensure('bff', lambda: {'jwt': vault.random()}) + vault.ensure('email', lambda: {'driver': 'resend', 'api_key': '', + 'sender_address': 'notifications@localhost.test', 'sender_name': 'SXCHNGE local'}) + vault.certificates() + provision_mongo(vault) + provision_agents(vault) + compose('up', '-d', '--wait', '--wait-timeout', '120', 'nats') + email = vault.kv('email') + key, sender = email.get('api_key'), email.get('sender_address') + if (not isinstance(key, str) or not key.strip() + or not isinstance(sender, str) or not sender.strip() + or sender == 'notifications@localhost.test'): + raise RuntimeError('Vault kv/email needs a Resend api_key and verified sender_address. Run python3 dev/local.py email, then make up.') + compose('up', '-d', '--wait', '--wait-timeout', '180') + compose('exec', '-T', 'frontend', 'nginx', '-s', 'reload', capture=True) + print('Local stack ready: https://localhost:5126; Vault: http://127.0.0.1:18200') + print('Local CA: .local/dev/ca.pem. Email driver: ' + vault.kv('email')['driver']) diff --git a/dev/config/bff.yml b/dev/config/bff.yml new file mode 100644 index 0000000..c2b493c --- /dev/null +++ b/dev/config/bff.yml @@ -0,0 +1,39 @@ +# SXCHNGE-owned Mongo database and JWT key. Never point these at Sendico. +http_server: + listen_address: 0.0.0.0:8082 + read_header_timeout: 5 + shutdown_timeout: 10 +database: + driver: mongodb + settings: + uri_env: SXCHNGE_MONGO_URI + database: sxchnge +api: + middleware: + domain_env: SXCHNGE_DOMAIN + api_endpoint_env: SXCHNGE_API_ENDPOINT + api_protocol_env: SXCHNGE_API_PROTOCOL + signature: + algorithm: HS256 + secret_key_env: SXCHNGE_JWT_SECRET + CORS: + allowed_origins: ["https://localhost:5126"] + allowed_methods: ["GET", "POST", "PATCH", "DELETE", "OPTIONS"] + allowed_headers: ["Content-Type", "Authorization", "X-Auth-Transport", "X-CSRF-Protection"] + allow_credentials: true + max_age: 300 + token: + expiration_hours: + account: 1 + refresh: 720 + length: 32 + password: + token_length: 32 + check: + min_length: 12 + notifications: + address: notifications:8083 + token_env: SXCHNGE_NOTIFICATIONS_TOKEN + ca_file: /run/sxchnge/ca.pem + server_name: notifications + activation_url: https://localhost:5126/verify-email diff --git a/dev/config/nginx.conf b/dev/config/nginx.conf new file mode 100644 index 0000000..463a631 --- /dev/null +++ b/dev/config/nginx.conf @@ -0,0 +1,17 @@ +server { + listen 8443 ssl; + ssl_certificate /etc/tls/server.crt; + ssl_certificate_key /etc/tls/server.key; + ssl_protocols TLSv1.2 TLSv1.3; + root /usr/share/nginx/html; + index index.html; + location /api/ { + resolver 127.0.0.11 valid=10s ipv6=off; + set $bff http://bff:8082; + proxy_pass $bff; + proxy_set_header Host $http_host; + proxy_set_header X-Forwarded-Proto https; + proxy_set_header X-Forwarded-For $remote_addr; + } + location / { try_files $uri $uri/ /index.html; } +} diff --git a/dev/config/notifications.yml b/dev/config/notifications.yml new file mode 100644 index 0000000..687274a --- /dev/null +++ b/dev/config/notifications.yml @@ -0,0 +1,27 @@ +listen: "0.0.0.0:8083" +tls_cert: "/run/sxchnge/server.crt" +tls_key: "/run/sxchnge/server.key" +database: + driver: mongodb + settings: + uri_env: SXCHNGE_MONGO_URI + database: sxchnge_notifications +messaging: + driver: NATS + settings: + url_env: SXCHNGE_NATS_URL + broker_name: sxchnge-notifications + max_reconnects: -1 + reconnect_wait: 2 +templates: "/etc/sxchnge/templates.json" +callers: + product: "SXCHNGE_NOTIFICATIONS_PRODUCT_TOKEN" +# Use replicas: 3 and environment: production for a three-node NATS cluster. +# mode: ops validates resources without modifying them; app provisions them. +queue: + stream: + replicas: 1 + max_age_hours: 48 + provisioning: + mode: app + environment: non-production diff --git a/dev/images.py b/dev/images.py new file mode 100644 index 0000000..f884728 --- /dev/null +++ b/dev/images.py @@ -0,0 +1,72 @@ +#!/usr/bin/env python3 +"""Resolve stable upstream releases on every build; pin their registry digests.""" +import concurrent.futures +import json +import os +from pathlib import Path +import re +import subprocess +import urllib.request + +ROOT = Path(__file__).resolve().parents[1] +STATE = ROOT / '.local' / 'dev' + + +def get(url): + with urllib.request.urlopen(url, timeout=45) as response: + return json.load(response) + + +def image(repo, alias): + url = f'https://hub.docker.com/v2/repositories/{repo}/tags' + selected = get(f'{url}/{alias}') + digest = selected['digest'] + if not re.fullmatch(r'sha256:[a-f0-9]{64}', digest): + raise ValueError('Registry returned an invalid digest') + # Find the exact numeric tag represented by the maintained stable alias. + candidates = get(f'{url}?page_size=100')['results'] + names = [x['name'] for x in candidates if x.get('digest') == digest + and re.fullmatch(r'\d+\.\d+(?:\.\d+)?(?:-alpine)?', x['name'])] + if not names: + raise ValueError(f'No stable version tag for {repo}:{alias}') + tag = max(names, key=lambda x: tuple(map(int, x.split('-')[0].split('.')))) + return f'{repo}:{tag}@{digest}' + + +def flutter(): + commit = get('https://api.github.com/repos/flutter/flutter/commits/stable')['sha'] + tags = subprocess.check_output(['git', 'ls-remote', '--tags', + 'https://github.com/flutter/flutter.git'], text=True) + names = [line.split('refs/tags/')[1] for line in tags.splitlines() + if line.startswith(commit + '\trefs/tags/') + and re.fullmatch(r'\d+\.\d+\.\d+', line.split('refs/tags/')[1])] + if not names: + raise ValueError('Flutter stable does not point at a stable release tag') + return max(names, key=lambda x: tuple(map(int, x.split('.')))), commit + + +def main(): + go = get('https://go.dev/dl/?mode=json')[0] + if not go['stable'] or not re.fullmatch(r'go\d+\.\d+\.\d+', go['version']): + raise ValueError('Go release is not stable') + specs = {'VAULT_IMAGE': ('hashicorp/vault', 'latest'), + 'MONGO_IMAGE': ('library/mongo', 'latest'), + 'NATS_IMAGE': ('library/nats', 'latest'), + 'NGINX_IMAGE': ('library/nginx', 'stable-alpine'), + 'GO_IMAGE': ('library/golang', go['version'][2:] + '-alpine'), + 'UBUNTU_IMAGE': ('library/ubuntu', 'latest')} + with concurrent.futures.ThreadPoolExecutor() as pool: + futures = {key: pool.submit(image, *spec) for key, spec in specs.items()} + refs = {key: future.result() for key, future in futures.items()} + refs['FLUTTER_VERSION'], refs['FLUTTER_COMMIT'] = flutter() + STATE.mkdir(parents=True, exist_ok=True, mode=0o700) + target = STATE / 'images.env' + temporary = target.with_suffix('.tmp') + temporary.write_text(''.join(f'{key}={value}\n' for key, value in refs.items())) + os.replace(temporary, target) + for key, value in refs.items(): + print(f'{key}={value}') + + +if __name__ == '__main__': + main() diff --git a/dev/images/backend.Dockerfile b/dev/images/backend.Dockerfile new file mode 100644 index 0000000..dd3a7d3 --- /dev/null +++ b/dev/images/backend.Dockerfile @@ -0,0 +1,21 @@ +ARG GO_IMAGE +ARG VAULT_IMAGE +FROM ${GO_IMAGE} AS build +RUN apk add --no-cache protobuf protobuf-dev make +WORKDIR /src +COPY Makefile ./ +COPY backend ./backend +RUN make generate-notifications +ARG MODULE +WORKDIR /src/backend/${MODULE} +RUN --mount=type=cache,target=/root/.cache/go-build --mount=type=cache,target=/go/pkg/mod CGO_ENABLED=0 go build -trimpath -o /out/service . + +FROM ${VAULT_IMAGE} +USER root +COPY --from=build /out/service /usr/local/bin/service +COPY dev/start-service.sh /usr/local/bin/start-service +COPY dev/config /etc/sxchnge +COPY backend/notifications/templates.example.json /etc/sxchnge/templates.json +RUN mkdir -p /run/sxchnge && chown 100:1000 /run/sxchnge +USER 100:1000 +ENTRYPOINT ["vault", "agent", "-config=/etc/agent/config.hcl"] diff --git a/dev/images/frontend.Dockerfile b/dev/images/frontend.Dockerfile new file mode 100644 index 0000000..51c5b2c --- /dev/null +++ b/dev/images/frontend.Dockerfile @@ -0,0 +1,17 @@ +ARG UBUNTU_IMAGE +ARG NGINX_IMAGE +FROM ${UBUNTU_IMAGE} AS build +RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates curl git unzip xz-utils libglu1-mesa && rm -rf /var/lib/apt/lists/* +ARG FLUTTER_VERSION +ARG FLUTTER_COMMIT +RUN git clone --depth 1 --branch "${FLUTTER_VERSION}" https://github.com/flutter/flutter.git /opt/flutter && test "$(git -C /opt/flutter rev-parse HEAD)" = "${FLUTTER_COMMIT}" +ENV PATH="/opt/flutter/bin:${PATH}" +RUN flutter config --no-analytics && flutter precache --web +WORKDIR /src +COPY frontend ./frontend +WORKDIR /src/frontend/app +ARG FRONTEND_URL=https://localhost:5126 +RUN flutter pub get && flutter gen-l10n && flutter build web --no-wasm-dry-run --dart-define=SXCHNGE_API_URL=${FRONTEND_URL}/api/v1 --dart-define=SXCHNGE_CLIENT_ID=sxchnge-local-web +FROM ${NGINX_IMAGE} +COPY dev/config/nginx.conf /etc/nginx/conf.d/default.conf +COPY --from=build /src/frontend/app/build/web /usr/share/nginx/html diff --git a/dev/images/nats.Dockerfile b/dev/images/nats.Dockerfile new file mode 100644 index 0000000..9fe15c9 --- /dev/null +++ b/dev/images/nats.Dockerfile @@ -0,0 +1,10 @@ +ARG NATS_IMAGE +ARG VAULT_IMAGE +FROM ${NATS_IMAGE} AS nats +FROM ${VAULT_IMAGE} +USER root +COPY --from=nats /nats-server /usr/local/bin/nats-server +COPY dev/start-nats.sh /usr/local/bin/start-nats +RUN mkdir -p /data /run/sxchnge && chown -R 100:1000 /data /run/sxchnge +USER 100:1000 +ENTRYPOINT ["vault", "agent", "-config=/etc/agent/config.hcl"] diff --git a/dev/local.py b/dev/local.py new file mode 100644 index 0000000..17d3a48 --- /dev/null +++ b/dev/local.py @@ -0,0 +1,47 @@ +#!/usr/bin/env python3 +"""Local stack lifecycle. Requires Python 3 and Docker Compose.""" +import sys +from bootstrap import up +from vault import Vault + + +def main(): + vault = Vault() + if len(sys.argv) == 2 and sys.argv[1] == 'up': + import fcntl + from vault import STATE + STATE.mkdir(parents=True, exist_ok=True, mode=0o700) + with (STATE / 'bootstrap.lock').open('w') as lock: + try: + fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB) + except BlockingIOError: + raise RuntimeError('Another local bootstrap is running') from None + up(vault) + elif len(sys.argv) == 2 and sys.argv[1] == 'verify': + from verify import verify + vault.existing() + verify(vault) + elif len(sys.argv) == 2 and sys.argv[1] == 'verify-infra': + from verify import infrastructure + vault.existing() + infrastructure(vault) + elif len(sys.argv) == 2 and sys.argv[1] == 'email': + import getpass + vault.existing() + address = input('Verified Resend sender address: ').strip() + key = getpass.getpass('Resend API key (hidden): ').strip() + if not key or '@' not in address: + raise RuntimeError('A key and sender address are required') + vault.put('email', {'driver': 'resend', 'api_key': key, + 'sender_address': address, 'sender_name': 'SXCHNGE'}) + print('Email configuration saved in Vault. Run make up.') + else: + raise RuntimeError('Usage: python3 dev/local.py up|verify|verify-infra|email') + + +if __name__ == '__main__': + try: + main() + except (RuntimeError, OSError) as error: + print('Local development setup failed: ' + str(error), file=sys.stderr) + sys.exit(1) diff --git a/dev/start-mongo.sh b/dev/start-mongo.sh new file mode 100755 index 0000000..61cf6d3 --- /dev/null +++ b/dev/start-mongo.sh @@ -0,0 +1,6 @@ +#!/bin/sh +set -eu +cp /run/mongo/keyfile /tmp/mongo-keyfile +chown mongodb:mongodb /tmp/mongo-keyfile +chmod 400 /tmp/mongo-keyfile +exec docker-entrypoint.sh "$@" diff --git a/dev/start-nats.sh b/dev/start-nats.sh new file mode 100755 index 0000000..f41344f --- /dev/null +++ b/dev/start-nats.sh @@ -0,0 +1,13 @@ +#!/bin/sh +set -eu +umask 077 +# Generated by Vault with hexadecimal encoding; never interpolates user input. +case "$NATS_TOKEN" in ''|*[!0-9a-f]*) exit 1;; esac +cat > /run/sxchnge/nats.conf < /run/sxchnge/ca.pem +unset TLS_CA +if [ "$SXCHNGE_SERVICE" = notifications ]; then + printf '%s\n' "$TLS_CERT" > /run/sxchnge/server.crt + printf '%s\n' "$TLS_KEY" > /run/sxchnge/server.key + printf '%s\n' "$EMAIL_CONFIG" > /run/sxchnge/email.yml + unset TLS_CERT TLS_KEY EMAIL_CONFIG + cat /etc/sxchnge/notifications.yml /run/sxchnge/email.yml > /run/sxchnge/config.yml + exec service -config.file=/run/sxchnge/config.yml +fi +exec service -config.file=/etc/sxchnge/bff.yml diff --git a/dev/start-vault.sh b/dev/start-vault.sh new file mode 100755 index 0000000..7d15f6b --- /dev/null +++ b/dev/start-vault.sh @@ -0,0 +1,16 @@ +#!/bin/sh +# Local-only automatic unseal. The root token is never mounted in this container. +set -eu +export VAULT_ADDR=http://127.0.0.1:8200 +vault server -config=/etc/vault/server.hcl & +pid=$! +trap 'kill -TERM "$pid"; wait "$pid"' TERM INT +until [ -s /bootstrap/unseal ]; do + kill -0 "$pid" || exit 1 + sleep 1 +done +until vault operator unseal "$(cat /bootstrap/unseal)" >/dev/null 2>&1; do + kill -0 "$pid" || exit 1 + sleep 1 +done +wait "$pid" diff --git a/dev/test_local.py b/dev/test_local.py new file mode 100644 index 0000000..4af9e93 --- /dev/null +++ b/dev/test_local.py @@ -0,0 +1,62 @@ +"""Focused regression tests for release selection and bootstrap failure paths.""" +from pathlib import Path +import tempfile +import unittest +from unittest.mock import patch + +import bootstrap +import images +from vault import Vault + + +class Releases(unittest.TestCase): + def test_selects_exact_stable_digest_not_prerelease(self): + digest = 'sha256:' + 'a' * 64 + versions = [{'name': name, 'digest': digest} for name in ['latest', '2', '2.1', '2.1.0', '2.2.0-rc1']] + with patch.object(images, 'get', side_effect=[{'digest': digest}, {'results': versions}]): + self.assertEqual(images.image('hashicorp/vault', 'latest'), 'hashicorp/vault:2.1.0@' + digest) + + def test_missing_stable_mapping_fails(self): + digest = 'sha256:' + 'a' * 64 + with patch.object(images, 'get', side_effect=[{'digest': digest}, {'results': [{'name': '3.0.0-rc1', 'digest': digest}]}]): + with self.assertRaises(ValueError): + images.image('hashicorp/vault', 'latest') + + def test_upstream_failure_does_not_replace_existing_pins(self): + with tempfile.TemporaryDirectory() as temporary: + state = Path(temporary) + target = state / 'images.env' + target.write_text('previous build') + with patch.object(images, 'STATE', state), patch.object(images, 'get', side_effect=OSError('offline')): + with self.assertRaises(OSError): + images.main() + self.assertEqual(target.read_text(), 'previous build') + + +class Bootstrap(unittest.TestCase): + def test_missing_resend_credentials_stops_before_product_start(self): + with tempfile.TemporaryDirectory() as temporary: + state = Path(temporary) + (state / 'images.env').touch() + from unittest.mock import Mock + vault = Mock() + vault.kv.return_value = {'driver': 'resend', 'api_key': '', 'sender_address': 'notifications@localhost.test'} + with patch.object(bootstrap, 'STATE', state), patch.object(bootstrap, 'compose') as compose, \ + patch.object(bootstrap, 'provision_mongo'), patch.object(bootstrap, 'provision_agents'): + with self.assertRaisesRegex(RuntimeError, 'Resend api_key'): + bootstrap.up(vault) + self.assertEqual(compose.call_args_list[-1].args[-1], 'nats') + email_seed = next(c for c in vault.ensure.call_args_list if c.args[0] == 'email').args[1]() + self.assertEqual(email_seed['driver'], 'resend') + self.assertEqual(email_seed['api_key'], '') + + def test_initialized_vault_without_local_keys_fails(self): + with tempfile.TemporaryDirectory() as temporary: + with patch('vault.STATE', Path(temporary)), patch.object(Vault, 'call', return_value={'initialized': True}): + with self.assertRaisesRegex(RuntimeError, 'Missing .local'): + Vault().initialize() + + + +if __name__ == '__main__': + unittest.main() diff --git a/dev/vault.py b/dev/vault.py new file mode 100644 index 0000000..5dcc0c0 --- /dev/null +++ b/dev/vault.py @@ -0,0 +1,117 @@ +"""Local Vault bootstrap API; response bodies with secrets never reach logs.""" +import json +import os +from pathlib import Path +import time +import urllib.error +import urllib.request + +STATE = Path(__file__).resolve().parents[1] / '.local' / 'dev' +URL = 'http://127.0.0.1:18200/v1/' + + +def write(path, value, mode=0o600): + path.parent.mkdir(parents=True, exist_ok=True, mode=0o700) + tmp = path.with_name(path.name + '.tmp') + fd = os.open(tmp, os.O_WRONLY | os.O_CREAT | os.O_TRUNC, mode) + with os.fdopen(fd, 'w') as out: + out.write(value) + os.chmod(tmp, mode) + os.replace(tmp, path) + + +class Vault: + def __init__(self): + self.token = None + + def call(self, path, data=None, method=None, missing=False): + headers = {'Content-Type': 'application/json'} + if self.token: + headers['X-Vault-Token'] = self.token + req = urllib.request.Request(URL + path, headers=headers, + data=json.dumps(data).encode() if data is not None else None, method=method) + try: + with urllib.request.urlopen(req, timeout=30) as response: + body = response.read() + return json.loads(body) if body else {} + except urllib.error.HTTPError as error: + if error.code == 404 and missing: + return None + raise RuntimeError(f'Vault {path} returned HTTP {error.code}') from None + + def initialize(self): + for attempt in range(90): + try: + initialized = self.call('sys/init')['initialized'] + break + except (urllib.error.URLError, ConnectionError): + time.sleep(1) + else: + raise RuntimeError('Vault did not start within 90 seconds') + initfile = STATE / 'admin' / 'init.json' + if not initialized: + if initfile.exists(): + raise RuntimeError('Vault data is missing but bootstrap material exists; restore the matching volume or explicitly reset both') + init = self.call('sys/init', {'secret_shares': 1, 'secret_threshold': 1}) + write(initfile, json.dumps(init)) + if not initfile.exists(): + raise RuntimeError('Missing .local/dev/admin/init.json for initialized Vault; restore the matching bootstrap material') + init = json.loads(initfile.read_text()) + self.token = init['root_token'] + write(STATE / 'vault' / 'unseal', init['keys_base64'][0], 0o444) + for attempt in range(60): + if not self.call('sys/seal-status')['sealed']: + return + time.sleep(1) + raise RuntimeError('Vault did not unseal within 60 seconds') + + def existing(self): + self.token = json.loads((STATE / 'admin' / 'init.json').read_text())['root_token'] + + def kv(self, path): + result = self.call('kv/data/' + path, missing=True) + return result['data']['data'] if result else None + + def put(self, path, data): + self.call('kv/data/' + path, {'data': data}) + + def ensure(self, path, factory): + value = self.kv(path) + if value is None: + value = factory() + self.put(path, value) + return value + + def random(self, length=32): + return self.call('sys/tools/random', {'bytes': length, 'format': 'hex'})['data']['random_bytes'] + + def mounts(self): + mounts = self.call('sys/mounts')['data'] + for name, kind in [('kv', 'kv'), ('database', 'database'), ('pki', 'pki')]: + if name + '/' not in mounts: + settings = {'type': kind} + if kind == 'kv': + settings['options'] = {'version': '2'} + self.call('sys/mounts/' + name, settings) + auth = self.call('sys/auth')['data'] + if 'approle/' not in auth: + self.call('sys/auth/approle', {'type': 'approle'}) + + def certificates(self): + self.call('sys/mounts/pki/tune', {'max_lease_ttl': '87600h'}) + ca = self.kv('tls') + if ca is None: + result = self.call('pki/root/generate/internal', {'common_name': 'SXCHNGE local development CA', 'ttl': '87600h'})['data'] + ca = {'ca': result['certificate']} + self.put('tls', ca) + self.call('pki/roles/local', {'allowed_domains': ['localhost', 'notifications'], + 'allow_bare_domains': True, 'allow_ip_sans': True, 'max_ttl': '720h'}) + for name, host in [('notifications-tls', 'notifications'), ('frontend-tls', 'localhost')]: + cert = self.kv(name) + if cert is None or cert['expiration'] < time.time() + 7 * 86400: + cert = self.call('pki/issue/local', {'common_name': host, 'ip_sans': '127.0.0.1', 'ttl': '720h'})['data'] + self.put(name, cert) + if name == 'frontend-tls': + write(STATE / 'frontend' / 'server.crt', cert['certificate']) + write(STATE / 'frontend' / 'server.key', cert['private_key']) + write(STATE / 'ca.pem', ca['ca'], 0o644) diff --git a/dev/vault/agent.hcl b/dev/vault/agent.hcl new file mode 100644 index 0000000..67c228e --- /dev/null +++ b/dev/vault/agent.hcl @@ -0,0 +1,14 @@ +vault { address = "http://vault:8200" } +auto_auth { + method "approle" { + config = { + role_id_file_path = "/etc/agent/role-id" + secret_id_file_path = "/etc/agent/secret-id" + remove_secret_id_file_after_reading = false + } + } +} +template_config { + static_secret_render_interval = "10s" + exit_on_retry_failure = true +} diff --git a/dev/vault/server.hcl b/dev/vault/server.hcl new file mode 100644 index 0000000..7f9e357 --- /dev/null +++ b/dev/vault/server.hcl @@ -0,0 +1,10 @@ +ui = true +disable_mlock = true +storage "file" { + path = "/vault/file" +} +listener "tcp" { + address = "0.0.0.0:8200" + tls_disable = true +} +api_addr = "http://vault:8200" diff --git a/dev/verify.py b/dev/verify.py new file mode 100644 index 0000000..d448797 --- /dev/null +++ b/dev/verify.py @@ -0,0 +1,72 @@ +"""Integration checks against this project's running stack; never print secrets.""" +import json +import ssl +import urllib.request + +from bootstrap import compose, mongo +from vault import STATE + + +def require(condition, message): + if not condition: + raise RuntimeError(message) + + +def database_access(vault): + for role, database, forbidden in [('bff', 'sxchnge', 'sxchnge_notifications'), + ('notifications', 'sxchnge_notifications', 'sxchnge')]: + lease = vault.call('database/creds/' + role) + credentials = lease['data'] + auth = 'db.getSiblingDB("admin").auth(' + json.dumps(credentials['username']) + ',' + json.dumps(credentials['password']) + ');' + try: + result = compose('exec', '-T', 'mongo', 'mongosh', '--quiet', '--file', '/dev/stdin', capture=True, + input=auth + 'const own=db.getSiblingDB(' + json.dumps(database) + '); own.local_smoke.insertOne({_id:"vault-check"}); own.local_smoke.deleteOne({_id:"vault-check"});\n' + 'let denied=false; try { db.getSiblingDB(' + json.dumps(forbidden) + ').runCommand({find:"local_smoke",filter:{}}); } catch(e) { denied=e.code===13; } if(!denied) throw new Error("cross-database access allowed"); print("scope-ok");') + require('scope-ok' in result, 'Mongo service scope check failed') + renewed = vault.call('sys/leases/renew', {'lease_id': lease['lease_id'], 'increment': '1h'}) + require(renewed['lease_duration'] > 0, 'Mongo lease renewal failed') + finally: + vault.call('sys/leases/revoke', {'lease_id': lease['lease_id']}) + admin = vault.kv('bootstrap/mongo')['password'] + result = mongo('print(admin.getUser(' + json.dumps(credentials['username']) + ')===null)', admin) + require('true' in result, 'Revoked Mongo user still exists') + print('PASS: database-plugin issuance, separate database permissions, lease renewal and revocation') + + +def policies(vault): + for service in ['bff', 'notifications', 'nats']: + root = STATE / 'agents' / service + login = vault.call('auth/approle/login', {'role_id': (root / 'role-id').read_text(), + 'secret_id': (root / 'secret-id').read_text()}) + token = login['auth']['client_token'] + try: + result = vault.call('sys/capabilities', {'token': token, 'paths': ['kv/data/bootstrap/mongo', 'sys/init']})['data'] + require(all(result[path] == ['deny'] for path in ['kv/data/bootstrap/mongo', 'sys/init']), 'Service can access bootstrap credentials') + finally: + vault.call('auth/token/revoke', {'token': token}) + print('PASS: service identities cannot read bootstrap credentials') + + +def infrastructure(vault): + status = compose('ps', '--format', 'json', capture=True) + rows = [json.loads(line) for line in status.splitlines() if line.strip()] + infra = {r['Service']: r for r in rows if r['Service'] in {'vault', 'mongo', 'nats'}} + require(set(infra) == {'vault', 'mongo', 'nats'}, 'Expected Vault, Mongo and NATS') + require(all(r['Health'] == 'healthy' for r in infra.values()), 'Infrastructure is unhealthy') + print('PASS: Vault, Mongo replica set and NATS JetStream are healthy') + policies(vault) + database_access(vault) + + +def verify(vault): + status = compose('ps', '--format', 'json', capture=True) + rows = [json.loads(line) for line in status.splitlines() if line.strip()] + require({r['Service'] for r in rows} == {'vault', 'mongo', 'nats', 'notifications', 'bff', 'frontend'}, 'Expected six running services') + require(all(r['Health'] == 'healthy' for r in rows), 'Some services are unhealthy') + context = ssl.create_default_context(cafile=str(STATE / 'ca.pem')) + for path in ['/', '/api/v1/health']: + with urllib.request.urlopen('https://localhost:5126' + path, context=context, timeout=10) as response: + require(response.status == 200, 'Frontend or BFF health failed') + print('PASS: all six services healthy; frontend and proxied BFF use verified TLS') + infrastructure(vault) + print('Resend credentials are configured; no live email sent by this check') diff --git a/docs/implementation.md b/docs/implementation.md index 095b7fa..d23a272 100644 --- a/docs/implementation.md +++ b/docs/implementation.md @@ -583,3 +583,55 @@ notification queue acceptance is not inbox delivery. Delivered in [PR #41](https://git.sendico.io/tech/sxchnge/pulls/41) against `main`. Branch `twofactor-40` is pushed. The PR is left open for review; no merge was performed. + +## Vault-backed local development (#42) + +Task [#42](https://git.sendico.io/tech/sxchnge/issues/42), branch `dev-env-42`. +Primary module: dev environment; secondary scope: container packaging, root build +commands, current-stable backend test images and repository container policy. + +- Added a dedicated `sxchnge-local` Compose project with persistent Vault, an + authenticated MongoDB replica set, NATS JetStream and build definitions for + notifications, BFF and Flutter/Nginx. Existing Sendico containers and volumes + remain independent. +- `make build` resolves current stable releases and pins manifest digests in + ignored local state. Flutter is built from its stable source tag/commit. All + protobuf bindings and frontend output are generated within the build, never + committed. Root `AGENTS.md` now records the stable-container policy. +- Bootstrap configures Vault KV/PKI/AppRole and `mongodb-database-plugin`, rotates + its Mongo manager credential and issues separate database roles for the BFF + and notifications. Vault Agent renews credentials and restarts child processes + on changes. Existing shared database/messaging paths remain intact. +- Resend is the default. The user explicitly chose to provision its API key and + verified sender manually after bootstrap. `make up` leaves healthy infrastructure + running and intentionally exits nonzero before product startup while those + fields are empty. No placeholder credential or disabled driver was retained. +- Frontend HTTPS and notification TLS use Vault PKI. The frontend proxies the BFF + at the same origin and builds with the existing real auth adapter and a seeded + public browser client. No UI or business service code changed. + +Validation: `make build` built all product images; `make test` and +`make lint-backend` passed. Backend tests pull fresh Mongo/NATS stable images and +use `-count=1` to prevent Go's test cache from hiding an image update. +`make test-dev-infra` passed its five setup regression tests and live checks for +AppRole restrictions, database issuance, cross-database denial, renewal and +revocation. `docker compose ... config --quiet` and `git diff --check` passed. +The initial image build exposed missing protobuf headers; installing +`protobuf-dev` in the build stage fixed it. + +Additional live checks passed: a down/up cycle preserved a Mongo marker and Vault +secrets and automatically unsealed Vault; changing/restoring the NATS token +restarted the NATS child; a temporary 10-second/20-second BFF database role proved +replacement credentials restart a healthy BFF (normal TTLs restored afterward). +Temporary BFF/frontend containers served the web bundle and proxied BFF health +successfully over HTTPS verified with the generated CA; those test containers +were removed. The notifications image rendered its Agent configuration and exited +with the expected missing email API key error. No real email was sent. + +The final `make build && make up` is expected to exit at the Resend provisioning +check until the user populates `kv/email`. Full six-service readiness and a live +Resend signup/login were not asserted without that credential. Vault, Mongo and +NATS remain running. No optional security scan or merge was performed. See the +[local development guide](local-development.md) for exact provisioning fields, +certificate trust, restart behavior, local bootstrap-material handling and +verification commands. diff --git a/docs/local-development.md b/docs/local-development.md new file mode 100644 index 0000000..46c57b5 --- /dev/null +++ b/docs/local-development.md @@ -0,0 +1,182 @@ +# Local development + +Task [#42](https://git.sendico.io/tech/sxchnge/issues/42), branch `dev-env-42`. + +## Build and bootstrap + +Requires Docker with Compose v2 and BuildKit, Python 3.10+ and Git. Docker builds +Go, protobuf bindings and Flutter web; host Go, protoc and Flutter installations +are not required for `make build && make up`. The first Flutter SDK download can +take several minutes. Both ARM64 and AMD64 build from the same source definitions. + +```sh +make build && make up +``` + +`make build` resolves the current stable upstream releases, pulls infrastructure +and builds NATS packaging, notifications, BFF and frontend images. Vault and Mongo +use upstream images directly. Resolution excludes prereleases and selects Nginx's +stable channel. The exact versions and manifest digests are saved in the ignored +`.local/dev/images.env`; Flutter's stable release is pinned by Git commit. Build +fails if upstream metadata cannot be resolved. Run `make build` again to refresh +versions. Review database release upgrade requirements before updating a populated +local volume across major versions; the tooling does not perform data migrations. + +`make up` starts persistent Vault, initializes/unseals it, provisions secrets and +AppRoles, initializes an authenticated single-member MongoDB replica set, configures +Vault's MongoDB database plugin, and starts authenticated NATS JetStream. It then +starts notifications, BFF and the frontend when the required email settings exist. + +**On the first run, `make up` intentionally exits with an error after infrastructure +bootstrap because Resend credentials have not been supplied.** No placeholder API +key, mock delivery mode or embedded credential is installed. The three infrastructure +services remain running so you can configure Vault, then rerun `make up`. + +## Configure Resend in Vault + +The default email driver is `resend`. In the Vault UI at +[http://127.0.0.1:18200](http://127.0.0.1:18200), edit the KV v2 secret **`kv/email`** +(API path `kv/data/email`): + +| Field | Value | +| --- | --- | +| `driver` | `resend` | +| `api_key` | Your Resend API key | +| `sender_address` | Your verified Resend sender email | +| `sender_name` | `SXCHNGE` or your desired display name | + +Local administrative login material is in the private, ignored +`.local/dev/admin/init.json` (`root_token`). Keep that file private. Product +containers never mount it. You can also provision the same secret with a hidden +terminal prompt, without placing the API key in shell history: + +```sh +python3 dev/local.py email +make up +``` + +The startup check rejects an empty key or the bootstrap sender placeholder. It +does not validate the key against Resend or prove domain verification; an invalid +key fails at delivery. Vault Agent fetches the key and email settings and injects +them into the notifications process. There is no `.env` secret fallback. Changing +`kv/email` restarts the notification process automatically within about 10 seconds. +Bootstrapping again preserves your existing email secret. + +## Open the product + +After providing Resend credentials: + +- Frontend and proxied API: [https://localhost:5126](https://localhost:5126). +- BFF health: [https://localhost:5126/api/v1/health](https://localhost:5126/api/v1/health). +- Vault UI/API: [http://127.0.0.1:18200](http://127.0.0.1:18200). + +Trust `.local/dev/ca.pem` in your local browser/OS certificate store to use HTTPS +without a certificate error. Trust installation is a separate local action, not +performed by the bootstrap. For a command-line check without changing trust: + +```sh +curl --cacert .local/dev/ca.pem https://localhost:5126/api/v1/health +``` + +HTTPS keeps the existing Secure/HttpOnly refresh cookie contract. Nginx serves +Flutter and proxies `/api/` to the BFF on the same origin. The bundle selects the +existing backend auth adapter with the public client ID `sxchnge-local-web`. +Bootstrap inserts this public browser client if absent; it has no client secret. +No frontend widget, service or provider architecture changes are needed. + +The rest of the app retains its existing capabilities and limitations: wallet, +KYC and settings fixtures are not turned into production integrations by this +container setup. Full email signup/login requires a working Resend account. + +## Secret and connection lifecycle + +| Consumer | Vault access | +| --- | --- | +| BFF | `database/creds/bff`, JWT key, notification caller token and CA | +| Notifications | `database/creds/notifications`, NATS token, caller token, TLS certificate/key and email settings | +| NATS | NATS authentication token only | +| Bootstrap | Infrastructure administration, PKI and initial identities | + +The database engine is configured at `database/config/mongo` with +`mongodb-database-plugin`. It uses its own `vault-manager` Mongo user, whose +password is rotated by Vault immediately after configuration. The manager has +`userAdminAnyDatabase`; applications never receive that credential. The independent +local Mongo administrator and replica-set key come from Vault's random generator +and are retained in `kv/bootstrap/mongo` for local provisioning. + +The BFF role has `readWrite` only on `sxchnge`; notifications has `readWrite` only +on `sxchnge_notifications`. Mongo URIs come from dynamic Vault credentials, not +static application users. Leases default to one hour with a 24-hour maximum. +Vault Agent renews leases, obtains replacement credentials, and restarts its child +process when credentials change. Apps retain their existing shared database and +messaging connection wrappers. No new direct application database path is added. + +Each agent authenticates using its own AppRole with a renewable periodic token. +Local SecretIDs are reusable for container restarts; each container mounts only +its own identity directory under `.local/dev/agents/`. Revoking an AppRole SecretID +requires removing its matching local `secret-id` file and rerunning `make up` to +issue a replacement. Revoking an active token/lease terminates its credentials; +Agent's retries and the container restart policy recover when Vault is reachable. + +The notification gRPC connection uses a Vault-issued certificate for +`notifications` and the BFF validates the Vault CA and hostname. Frontend HTTPS +certificates also come from Vault PKI. Leaf certificates last 30 days; `make up` +refreshes them when less than seven days remain and reloads Nginx. Run it regularly +for a development stack kept online for weeks. Agents restart when their TLS KV +secret changes. Application secrets are never put into the frontend bundle. + +Vault, Mongo and JetStream data use project-scoped named Docker volumes. Local +Vault unseal material, administrator login, per-service bootstrap identities and +frontend TLS material are under ignored `.local/dev/`, outside image build contexts. +The local Vault automatically unseals using its local key on container restart. +These convenience credentials and loopback HTTP Vault access are for this local +machine; this is not a production Vault deployment. Do not share `.local/dev/`. +Keep it paired with its matching Docker volumes for recovery. + +Mongo and NATS expose no host ports. The `sxchnge-local` network, volumes and +container names are independent of existing Sendico stacks. Only the frontend +and Vault bind host loopback ports. No Sendico database, message bus or secrets +are consumed. + +## Manage and verify + +```sh +make ps +make logs +make down # preserves named volumes and local bootstrap material +make up # idempotent provisioning and readiness checks +make test-dev-infra # works before Resend provisioning +make test-dev # requires all six services running +``` + +The infrastructure check exercises AppRole access restrictions, real Vault +MongoDB credential issuance, per-database access, lease renewal and user deletion +on revocation. The complete check also verifies service health and frontend/API +HTTPS with the generated CA. These checks never send a real email. + +Backend regression tests use fresh `mongo:latest` and `nats:latest` official +stable images, pulled before `make test-backend`. Normal validation remains +`make test` and `make lint-backend` and requires the host SDKs described in the +repository guides. + +Deleting Docker volumes removes local data; `make down` deliberately keeps them. +If starting from scratch, stop the stack and explicitly remove this project's +three volumes **and** its matching `.local/dev/` bootstrap state together. Never +reuse old unseal material with a newly initialized Vault volume, or retain Mongo +users while discarding the Vault data that manages their leases. + +## Sources and checked releases + +On 2026-09-12 the resolver selected Vault 2.1.0, MongoDB 8.3.9, NATS 2.14.6, +Go 1.27.1, Nginx stable 1.30.4, Ubuntu 26.04 and Flutter stable 3.47.4. These are +verification records, not fixed future defaults. Release selection follows the +[Vault releases](https://releases.hashicorp.com/vault/), +[MongoDB stable release notes](https://www.mongodb.com/docs/manual/release-notes/), +[NATS releases](https://github.com/nats-io/nats-server/releases/latest), +[Nginx downloads](https://nginx.org/en/download.html), +[Go releases](https://go.dev/dl/) and +[Flutter stable channel](https://github.com/flutter/flutter/tree/stable). +The implementation follows HashiCorp's +[MongoDB database plugin](https://developer.hashicorp.com/vault/docs/secrets/databases/mongodb) +and [Agent process supervisor](https://developer.hashicorp.com/vault/docs/agent-and-proxy/agent/process-supervisor) +contracts. From 80d6917aa76e46e5cf0546c230b7129512fb61ce Mon Sep 17 00:00:00 2001 From: Stephan D Date: Sat, 12 Sep 2026 21:05:39 +0200 Subject: [PATCH 02/11] Record local environment delivery PR (#42) --- docs/implementation.md | 3 ++- docs/local-development.md | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/implementation.md b/docs/implementation.md index d23a272..68d3004 100644 --- a/docs/implementation.md +++ b/docs/implementation.md @@ -586,7 +586,8 @@ Branch `twofactor-40` is pushed. The PR is left open for review; no merge was pe ## Vault-backed local development (#42) -Task [#42](https://git.sendico.io/tech/sxchnge/issues/42), branch `dev-env-42`. +Task [#42](https://git.sendico.io/tech/sxchnge/issues/42), +[PR #43](https://git.sendico.io/tech/sxchnge/pulls/43), branch `dev-env-42`. Primary module: dev environment; secondary scope: container packaging, root build commands, current-stable backend test images and repository container policy. diff --git a/docs/local-development.md b/docs/local-development.md index 46c57b5..ff108ca 100644 --- a/docs/local-development.md +++ b/docs/local-development.md @@ -1,6 +1,7 @@ # Local development -Task [#42](https://git.sendico.io/tech/sxchnge/issues/42), branch `dev-env-42`. +Task [#42](https://git.sendico.io/tech/sxchnge/issues/42), +[PR #43](https://git.sendico.io/tech/sxchnge/pulls/43), branch `dev-env-42`. ## Build and bootstrap From 23f7b78c3b23c97c9b317a772d89d4bd600f222e Mon Sep 17 00:00:00 2001 From: Stephan D Date: Sat, 12 Sep 2026 22:19:36 +0200 Subject: [PATCH 03/11] Keep Mongo operations online across Vault credential rotation --- Makefile | 4 + backend/notifications/store/mongo.go | 4 +- .../customers/internal/customer_test.go | 14 +- backend/product/customers/store/mongo/db.go | 8 +- backend/product/sessions/store/db.go | 10 +- .../sessions/store/refreshtokens_test.go | 5 +- backend/product/verification/store/db.go | 8 +- backend/shared/database/connection.go | 56 ++-- .../database/dependencies/dependencies.go | 6 +- backend/shared/database/internal/mongo/db.go | 126 ++++---- .../database/internal/repository/index.go | 33 ++- .../internal/repository/repository.go | 143 +++++++-- .../internal/repository/repository_filter.go | 24 +- .../database/internal/transaction/factory.go | 15 +- .../internal/transaction/transaction.go | 31 +- .../shared/database/repository/repository.go | 8 +- backend/shared/database/rotation_test.go | 275 ++++++++++++++++++ backend/shared/database/source/manager.go | 253 ++++++++++++++++ backend/shared/database/source/source.go | 30 ++ backend/shared/database/template/template.go | 6 +- backend/shared/database/vault/issuer.go | 140 +++++++++ backend/shared/database/vault/issuer_test.go | 79 +++++ backend/shared/ratelimit/store/db.go | 6 +- dev/agents.py | 4 - dev/config/bff.yml | 7 +- dev/config/notifications.yml | 7 +- dev/rotation.py | 92 ++++++ dev/test_local.py | 9 + docs/implementation.md | 48 ++- docs/local-development.md | 41 ++- 30 files changed, 1290 insertions(+), 202 deletions(-) create mode 100644 backend/shared/database/rotation_test.go create mode 100644 backend/shared/database/source/manager.go create mode 100644 backend/shared/database/source/source.go create mode 100644 backend/shared/database/vault/issuer.go create mode 100644 backend/shared/database/vault/issuer_test.go create mode 100644 dev/rotation.py diff --git a/Makefile b/Makefile index bc85743..170c1d5 100644 --- a/Makefile +++ b/Makefile @@ -80,3 +80,7 @@ test-dev: test-dev-infra: python3 -m unittest discover -s dev -p 'test_*.py' python3 dev/local.py verify-infra + +.PHONY: test-dev-rotation +test-dev-rotation: + python3 dev/rotation.py diff --git a/backend/notifications/store/mongo.go b/backend/notifications/store/mongo.go index 70bffd3..81b7c65 100644 --- a/backend/notifications/store/mongo.go +++ b/backend/notifications/store/mongo.go @@ -9,11 +9,11 @@ import ( "git.sendico.io/tech/sxchnge/shared/database/repository" "git.sendico.io/tech/sxchnge/shared/database/repository/builder" ri "git.sendico.io/tech/sxchnge/shared/database/repository/index" + "git.sendico.io/tech/sxchnge/shared/database/source" "git.sendico.io/tech/sxchnge/shared/database/template" merrors "git.sendico.io/tech/sxchnge/shared/errors" mlogger "git.sendico.io/tech/sxchnge/shared/logging" "go.mongodb.org/mongo-driver/v2/bson" - "go.mongodb.org/mongo-driver/v2/mongo" "go.uber.org/zap" ) @@ -21,7 +21,7 @@ type Store struct { template.DBImp[*model.Notification] } -func New(logger mlogger.Logger, db *mongo.Database) (*Store, error) { +func New(logger mlogger.Logger, db source.Source) (*Store, error) { s := &Store{*template.Create[*model.Notification](mlogger.Named(logger, "store"), "notifications", db)} ttl := int32(0) for _, index := range []*ri.Definition{ diff --git a/backend/product/customers/internal/customer_test.go b/backend/product/customers/internal/customer_test.go index bc5eec4..1607a8b 100644 --- a/backend/product/customers/internal/customer_test.go +++ b/backend/product/customers/internal/customer_test.go @@ -2,6 +2,10 @@ package accountserviceimp import ( "context" + "net/url" + "testing" + "time" + "git.sendico.io/tech/sxchnge/product/auth/config" customermodel "git.sendico.io/tech/sxchnge/product/customers/model" sessionmodel "git.sendico.io/tech/sxchnge/product/sessions/model" @@ -15,9 +19,6 @@ import ( "github.com/testcontainers/testcontainers-go/modules/mongodb" "go.mongodb.org/mongo-driver/v2/bson" "go.uber.org/zap" - "net/url" - "testing" - "time" ) func customer(login string) *customermodel.Account { @@ -86,7 +87,10 @@ func TestCustomerLifecycleWithoutTenancy(t *testing.T) { deps, err := edgemongo.ResolveDependencies(factory) require.NoError(t, err) - collections, err := deps.Database.ListCollectionNames(ctx, bson.D{}) + raw, rawContext, release, err := deps.Database.Acquire(ctx) + require.NoError(t, err) + defer release() + collections, err := raw.ListCollectionNames(rawContext, bson.D{}) require.NoError(t, err) for _, forbidden := range []string{"organizations", "roles", "policies", "policy_assignments", "invitations", "accounts"} { require.NotContains(t, collections, forbidden) @@ -116,7 +120,7 @@ func TestCustomerLifecycleWithoutTenancy(t *testing.T) { require.ErrorIs(t, err, outerFailure) _, err = svc.accountDB.GetByEmail(ctx, "outer@example.test") require.ErrorIs(t, err, merrors.ErrNoData) - count, err := deps.Database.Collection(config.VerificationTokensCollection).CountDocuments(ctx, bson.D{}) + count, err := raw.Collection(config.VerificationTokensCollection).CountDocuments(rawContext, bson.D{}) require.NoError(t, err) require.EqualValues(t, 2, count) } diff --git a/backend/product/customers/store/mongo/db.go b/backend/product/customers/store/mongo/db.go index 58dd464..58aed01 100644 --- a/backend/product/customers/store/mongo/db.go +++ b/backend/product/customers/store/mongo/db.go @@ -1,14 +1,14 @@ package accountdb import ( - customermodel "git.sendico.io/tech/sxchnge/product/customers/model" "strings" + customermodel "git.sendico.io/tech/sxchnge/product/customers/model" ri "git.sendico.io/tech/sxchnge/shared/database/repository/index" + "git.sendico.io/tech/sxchnge/shared/database/source" "git.sendico.io/tech/sxchnge/shared/database/template" "git.sendico.io/tech/sxchnge/shared/logging" "git.sendico.io/tech/sxchnge/shared/service" - "go.mongodb.org/mongo-driver/v2/mongo" "go.uber.org/zap" ) @@ -16,11 +16,11 @@ type AccountDB struct { template.DBImp[*customermodel.Account] } -func Create(logger mlogger.Logger, db *mongo.Database) (*AccountDB, error) { +func Create(logger mlogger.Logger, db source.Source) (*AccountDB, error) { return CreateWithCollection(logger, db, mservice.Accounts) } -func CreateWithCollection(logger mlogger.Logger, db *mongo.Database, collection string) (*AccountDB, error) { +func CreateWithCollection(logger mlogger.Logger, db source.Source, collection string) (*AccountDB, error) { if normalized := strings.TrimSpace(collection); normalized != "" { collection = normalized } else { diff --git a/backend/product/sessions/store/db.go b/backend/product/sessions/store/db.go index 8b9549a..444068a 100644 --- a/backend/product/sessions/store/db.go +++ b/backend/product/sessions/store/db.go @@ -1,15 +1,15 @@ package refreshtokens import ( - sessionmodel "git.sendico.io/tech/sxchnge/product/sessions/model" "strings" + sessionmodel "git.sendico.io/tech/sxchnge/product/sessions/model" "git.sendico.io/tech/sxchnge/shared/database/repository" ri "git.sendico.io/tech/sxchnge/shared/database/repository/index" + "git.sendico.io/tech/sxchnge/shared/database/source" "git.sendico.io/tech/sxchnge/shared/database/template" "git.sendico.io/tech/sxchnge/shared/logging" "git.sendico.io/tech/sxchnge/shared/service" - "go.mongodb.org/mongo-driver/v2/mongo" "go.uber.org/zap" ) @@ -24,7 +24,7 @@ type RefreshTokenDB struct { collection Collections } -func Create(logger mlogger.Logger, db *mongo.Database) (*RefreshTokenDB, error) { +func Create(logger mlogger.Logger, db source.Source) (*RefreshTokenDB, error) { return CreateWithCollections(logger, db, Collections{}) } @@ -38,11 +38,11 @@ func (c Collections) WithDefaults() Collections { return c } -func CreateWithCollections(logger mlogger.Logger, db *mongo.Database, collections Collections) (*RefreshTokenDB, error) { +func CreateWithCollections(logger mlogger.Logger, db source.Source, collections Collections) (*RefreshTokenDB, error) { collections = collections.WithDefaults() p := &RefreshTokenDB{ DBImp: *template.Create[*sessionmodel.RefreshToken](logger, mservice.Type(collections.RefreshTokens), db), - clients: repository.CreateMongoRepository(db, collections.Clients), + clients: repository.CreateRepository(db, collections.Clients), collection: collections, } diff --git a/backend/product/sessions/store/refreshtokens_test.go b/backend/product/sessions/store/refreshtokens_test.go index a3b707e..24b12bf 100644 --- a/backend/product/sessions/store/refreshtokens_test.go +++ b/backend/product/sessions/store/refreshtokens_test.go @@ -8,11 +8,12 @@ import ( "crypto/sha1" "errors" "fmt" - sessionmodel "git.sendico.io/tech/sxchnge/product/sessions/model" "testing" "time" + sessionmodel "git.sendico.io/tech/sxchnge/product/sessions/model" "git.sendico.io/tech/sxchnge/product/sessions/store" + "git.sendico.io/tech/sxchnge/shared/database/source" "git.sendico.io/tech/sxchnge/shared/errors" factory "git.sendico.io/tech/sxchnge/shared/logging/factory" "github.com/stretchr/testify/assert" @@ -55,7 +56,7 @@ func setupTestDBWithMongo(t *testing.T) (*refreshtokens.RefreshTokenDB, *mongo.D database := client.Database(fmt.Sprintf("test_refresh_tokens_%x", sha1.Sum([]byte(t.Name())))) logger := factory.NewLogger(true) - db, err := refreshtokens.Create(logger, database) + db, err := refreshtokens.Create(logger, source.Fixed(database)) require.NoError(t, err, "failed to create refresh tokens db") cleanup := func() { diff --git a/backend/product/verification/store/db.go b/backend/product/verification/store/db.go index e495739..a163daf 100644 --- a/backend/product/verification/store/db.go +++ b/backend/product/verification/store/db.go @@ -1,16 +1,16 @@ package verification import ( - verificationmodel "git.sendico.io/tech/sxchnge/product/verification/model" "strings" + verificationmodel "git.sendico.io/tech/sxchnge/product/verification/model" ri "git.sendico.io/tech/sxchnge/shared/database/repository/index" + "git.sendico.io/tech/sxchnge/shared/database/source" "git.sendico.io/tech/sxchnge/shared/database/template" "git.sendico.io/tech/sxchnge/shared/database/transaction" "git.sendico.io/tech/sxchnge/shared/logging" "git.sendico.io/tech/sxchnge/shared/ratelimit/store" "git.sendico.io/tech/sxchnge/shared/service" - "go.mongodb.org/mongo-driver/v2/mongo" "go.uber.org/zap" ) @@ -29,7 +29,7 @@ const verificationContextIdempotencyIndexName = "uniq_verification_context_idemp func Create( logger mlogger.Logger, - db *mongo.Database, + db source.Source, tf transaction.Factory, ) (*verificationDB, error) { return CreateWithCollection(logger, db, tf, "") @@ -37,7 +37,7 @@ func Create( func CreateWithCollection( logger mlogger.Logger, - db *mongo.Database, + db source.Source, tf transaction.Factory, collection string, ) (*verificationDB, error) { diff --git a/backend/shared/database/connection.go b/backend/shared/database/connection.go index 52092fd..2be6c7f 100644 --- a/backend/shared/database/connection.go +++ b/backend/shared/database/connection.go @@ -4,56 +4,38 @@ import ( "context" mongoimpl "git.sendico.io/tech/sxchnge/shared/database/internal/mongo" - "git.sendico.io/tech/sxchnge/shared/errors" - "git.sendico.io/tech/sxchnge/shared/logging" - "go.mongodb.org/mongo-driver/v2/mongo" + "git.sendico.io/tech/sxchnge/shared/database/source" + merrors "git.sendico.io/tech/sxchnge/shared/errors" + mlogger "git.sendico.io/tech/sxchnge/shared/logging" "go.mongodb.org/mongo-driver/v2/mongo/readpref" ) -// Connection represents a low-level database connection lifecycle. type Connection interface { - Disconnect(ctx context.Context) error - Ping(ctx context.Context) error + Disconnect(context.Context) error + Ping(context.Context) error } - -// MongoConnection provides direct access to the underlying mongo client. type MongoConnection struct { - client *mongo.Client - database string -} - -func (c *MongoConnection) Client() *mongo.Client { - return c.client -} - -func (c *MongoConnection) Database() *mongo.Database { - return c.client.Database(c.database) -} - -func (c *MongoConnection) Disconnect(ctx context.Context) error { - return c.client.Disconnect(ctx) + database source.Source + close func(context.Context) error } +func (c *MongoConnection) Database() source.Source { return c.database } +func (c *MongoConnection) Disconnect(ctx context.Context) error { return c.close(ctx) } func (c *MongoConnection) Ping(ctx context.Context) error { - return c.client.Ping(ctx, readpref.Primary()) + database, ctx, release, err := c.database.Acquire(ctx) + if err != nil { + return err + } + defer release() + return database.Client().Ping(ctx, readpref.Primary()) } - -// ConnectMongo returns a low-level MongoDB connection without constructing repositories. func ConnectMongo(logger mlogger.Logger, config *Config) (*MongoConnection, error) { - if config == nil { - return nil, merrors.InvalidArgument("database configuration is nil", "config") + if config == nil || config.Driver != Mongo { + return nil, merrors.InvalidArgument("MongoDB configuration is required") } - if config.Driver != Mongo { - return nil, merrors.InvalidArgument("unsupported database driver: "+string(config.Driver), "config.driver") - } - - client, _, settings, err := mongoimpl.ConnectClient(logger, config.Settings) + database, close, err := mongoimpl.OpenSource(logger, config.Settings) if err != nil { return nil, err } - - return &MongoConnection{ - client: client, - database: settings.Database, - }, nil + return &MongoConnection{database: database, close: close}, nil } diff --git a/backend/shared/database/dependencies/dependencies.go b/backend/shared/database/dependencies/dependencies.go index e8c4dc2..49013aa 100644 --- a/backend/shared/database/dependencies/dependencies.go +++ b/backend/shared/database/dependencies/dependencies.go @@ -2,19 +2,19 @@ package mongo import ( "git.sendico.io/tech/sxchnge/shared/database" + "git.sendico.io/tech/sxchnge/shared/database/source" "git.sendico.io/tech/sxchnge/shared/database/transaction" "git.sendico.io/tech/sxchnge/shared/errors" - "go.mongodb.org/mongo-driver/v2/mongo" ) // Dependencies resolves local DB/transaction primitives for edge-local store wiring only. type Dependencies struct { - Database *mongo.Database + Database source.Source Transactions transaction.Factory } type DatabaseProvider interface { - MongoDatabase() *mongo.Database + MongoDatabase() source.Source } func ResolveDependencies(dbf db.Factory) (*Dependencies, error) { diff --git a/backend/shared/database/internal/mongo/db.go b/backend/shared/database/internal/mongo/db.go index 1cbd093..170173b 100644 --- a/backend/shared/database/internal/mongo/db.go +++ b/backend/shared/database/internal/mongo/db.go @@ -3,15 +3,18 @@ package mongo import ( "context" "fmt" - sharedmodel "git.sendico.io/tech/sxchnge/shared/model" "net" "os" "strings" + "time" mutil "git.sendico.io/tech/sxchnge/shared/config" "git.sendico.io/tech/sxchnge/shared/database/internal/transaction" + "git.sendico.io/tech/sxchnge/shared/database/source" "git.sendico.io/tech/sxchnge/shared/database/transaction" + dbvault "git.sendico.io/tech/sxchnge/shared/database/vault" "git.sendico.io/tech/sxchnge/shared/logging" + sharedmodel "git.sendico.io/tech/sxchnge/shared/model" "github.com/go-viper/mapstructure/v2" "go.mongodb.org/mongo-driver/v2/mongo" "go.mongodb.org/mongo-driver/v2/mongo/readpref" @@ -20,28 +23,29 @@ import ( // Config represents configuration type Config struct { - Port *string `mapstructure:"port"` - PortEnv *string `mapstructure:"port_env"` - User *string `mapstructure:"user"` - UserEnv *string `mapstructure:"user_env"` - PasswordEnv string `mapstructure:"password_env"` - Database *string `mapstructure:"database"` - DatabaseEnv *string `mapstructure:"database_env"` - Host *string `mapstructure:"host"` - HostEnv *string `mapstructure:"host_env"` - Hosts []string `mapstructure:"hosts,omitempty"` - HostsEnvPrefix *string `mapstructure:"hosts_env_prefix,omitempty"` - HostsEnvPrefixEnv *string `mapstructure:"hosts_env_prefix_env,omitempty"` - PortsEnvPrefix *string `mapstructure:"ports_env_prefix,omitempty"` - PortsEnvPrefixEnv *string `mapstructure:"ports_env_prefix_env,omitempty"` - URI *string `mapstructure:"uri,omitempty"` - URIEnv *string `mapstructure:"uri_env,omitempty"` - AuthSource *string `mapstructure:"auth_source,omitempty"` - AuthSourceEnv *string `mapstructure:"auth_source_env,omitempty"` - AuthMechanism *string `mapstructure:"auth_mechanism,omitempty"` - AuthMechanismEnv *string `mapstructure:"auth_mechanism_env,omitempty"` - ReplicaSet *string `mapstructure:"replica_set,omitempty"` - ReplicaSetEnv *string `mapstructure:"replica_set_env,omitempty"` + Vault *dbvault.Config `mapstructure:"vault"` + Port *string `mapstructure:"port"` + PortEnv *string `mapstructure:"port_env"` + User *string `mapstructure:"user"` + UserEnv *string `mapstructure:"user_env"` + PasswordEnv string `mapstructure:"password_env"` + Database *string `mapstructure:"database"` + DatabaseEnv *string `mapstructure:"database_env"` + Host *string `mapstructure:"host"` + HostEnv *string `mapstructure:"host_env"` + Hosts []string `mapstructure:"hosts,omitempty"` + HostsEnvPrefix *string `mapstructure:"hosts_env_prefix,omitempty"` + HostsEnvPrefixEnv *string `mapstructure:"hosts_env_prefix_env,omitempty"` + PortsEnvPrefix *string `mapstructure:"ports_env_prefix,omitempty"` + PortsEnvPrefixEnv *string `mapstructure:"ports_env_prefix_env,omitempty"` + URI *string `mapstructure:"uri,omitempty"` + URIEnv *string `mapstructure:"uri_env,omitempty"` + AuthSource *string `mapstructure:"auth_source,omitempty"` + AuthSourceEnv *string `mapstructure:"auth_source_env,omitempty"` + AuthMechanism *string `mapstructure:"auth_mechanism,omitempty"` + AuthMechanismEnv *string `mapstructure:"auth_mechanism_env,omitempty"` + ReplicaSet *string `mapstructure:"replica_set,omitempty"` + ReplicaSetEnv *string `mapstructure:"replica_set_env,omitempty"` } type DBSettings struct { @@ -119,37 +123,42 @@ func dialMongo(logger mlogger.Logger, dbSettings *DBSettings) (*mongo.Client, er return client, nil } -func ConnectClient(logger mlogger.Logger, settings sharedmodel.SettingsT) (*mongo.Client, *Config, *DBSettings, error) { - config, dbSettings, err := decodeConfig(logger, settings) - if err != nil { - return nil, nil, nil, err - } - - client, err := dialMongo(logger, dbSettings) - if err != nil { - return nil, nil, nil, err - } - - return client, config, dbSettings, nil -} - // DB represents the structure of the database type DB struct { - logger mlogger.Logger - config *DBSettings - client *mongo.Client -} - -func (db *DB) db() *mongo.Database { - return db.client.Database(db.config.Database) -} - -func (db *DB) MongoDatabase() *mongo.Database { - return db.db() + logger mlogger.Logger + database source.Source + close func(context.Context) error } +func (db *DB) MongoDatabase() source.Source { return db.database } func (db *DB) TransactionFactory() transaction.Factory { - return transactionimp.CreateFactory(db.client) + return transactionimp.CreateSourceFactory(db.database) +} + +// OpenSource owns either a fixed connection or a rotating Vault-backed source. +func OpenSource(logger mlogger.Logger, settings sharedmodel.SettingsT) (source.Source, func(context.Context) error, error) { + config, dbSettings, err := decodeConfig(logger, settings) + if err != nil { + return nil, nil, err + } + if config.Vault != nil { + issuer, err := dbvault.New(*config.Vault, dbSettings.URI) + if err != nil { + return nil, nil, err + } + ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second) + defer cancel() + manager, err := source.New(ctx, logger.Named("credentials"), dbSettings.Database, issuer) + if err != nil { + return nil, nil, err + } + return manager, manager.Close, nil + } + client, err := dialMongo(logger, dbSettings) + if err != nil { + return nil, nil, err + } + return source.Fixed(client.Database(dbSettings.Database)), client.Disconnect, nil } func collectReplicaHosts(configuredHosts []string, replicaSet, defaultPort, hostPrefix, portPrefix string) []string { @@ -217,24 +226,17 @@ func collectReplicaHosts(configuredHosts []string, replicaSet, defaultPort, host } func (db *DB) CloseConnection() { - if err := db.client.Disconnect(context.Background()); err != nil { - db.logger.Warn("Failed to close connection", zap.Error(err)) + ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second) + defer cancel() + if err := db.close(ctx); err != nil { + db.logger.Warn("Failed to drain database connection", zap.Error(err)) } - db.logger.Info("Database connection closed") } -// NewConnection creates a new database connection func NewConnection(logger mlogger.Logger, settings sharedmodel.SettingsT) (*DB, error) { - client, _, dbSettings, err := ConnectClient(logger, settings) + database, close, err := OpenSource(logger, settings) if err != nil { return nil, err } - - db := &DB{ - logger: logger.Named("db"), - config: dbSettings, - client: client, - } - - return db, nil + return &DB{logger: logger, database: database, close: close}, nil } diff --git a/backend/shared/database/internal/repository/index.go b/backend/shared/database/internal/repository/index.go index 0c4a39c..6e59ed9 100644 --- a/backend/shared/database/internal/repository/index.go +++ b/backend/shared/database/internal/repository/index.go @@ -12,7 +12,14 @@ import ( ) func (r *MongoRepository) CreateIndex(def *ri.Definition) error { - if r.collection == nil { + ctx := context.Background() + collection, ctx, release, err := r.acquire(ctx) + if err != nil { + return err + } + defer release() + + if collection == nil { return merrors.NoData("data collection is not set") } if len(def.Keys) == 0 { @@ -49,28 +56,40 @@ func (r *MongoRepository) CreateIndex(def *ri.Definition) error { opts.SetSparse(def.Sparse) } - _, err := r.collection.Indexes().CreateOne( - context.Background(), + _, err = collection.Indexes().CreateOne( + ctx, mongo.IndexModel{Keys: keys, Options: opts}, ) return err } func (r *MongoRepository) DropIndex(ctx context.Context, name string) error { - if r.collection == nil { + collection, ctx, release, err := r.acquire(ctx) + if err != nil { + return err + } + defer release() + + if collection == nil { return merrors.NoData("data collection is not set") } if name == "" { return merrors.InvalidArgument("index name is required", "index.name") } - return r.collection.Indexes().DropOne(ctx, name) + return collection.Indexes().DropOne(ctx, name) } func (r *MongoRepository) ListIndexDefinitions(ctx context.Context) ([]*ri.Definition, error) { - if r.collection == nil { + collection, ctx, release, err := r.acquire(ctx) + if err != nil { + return nil, err + } + defer release() + + if collection == nil { return nil, merrors.NoData("data collection is not set") } - cursor, err := r.collection.Indexes().List(ctx) + cursor, err := collection.Indexes().List(ctx) if err != nil { return nil, err } diff --git a/backend/shared/database/internal/repository/repository.go b/backend/shared/database/internal/repository/repository.go index f11e249..c937364 100644 --- a/backend/shared/database/internal/repository/repository.go +++ b/backend/shared/database/internal/repository/repository.go @@ -8,6 +8,7 @@ import ( "git.sendico.io/tech/sxchnge/shared/database/record" "git.sendico.io/tech/sxchnge/shared/database/repository/builder" rd "git.sendico.io/tech/sxchnge/shared/database/repository/decoder" + "git.sendico.io/tech/sxchnge/shared/database/source" "git.sendico.io/tech/sxchnge/shared/database/transaction" "git.sendico.io/tech/sxchnge/shared/errors" "go.mongodb.org/mongo-driver/v2/bson" @@ -17,7 +18,7 @@ import ( type MongoRepository struct { collectionName string - collection *mongo.Collection + source source.Source } func idFilter(id bson.ObjectID) bson.D { @@ -27,10 +28,23 @@ func idFilter(id bson.ObjectID) bson.D { } func NewMongoRepository(db *mongo.Database, collection string) *MongoRepository { - return &MongoRepository{ - collectionName: collection, - collection: db.Collection(collection), + return NewRepository(source.Fixed(db), collection) +} + +func NewRepository(database source.Source, collection string) *MongoRepository { + return &MongoRepository{collectionName: collection, source: database} +} + +func (r *MongoRepository) acquire(ctx context.Context) (*mongo.Collection, context.Context, func(), error) { + if r.source == nil { + return nil, ctx, nil, merrors.FailedPrecondition("database source is not configured") } + + database, scoped, release, err := r.source.Acquire(ctx) + if err != nil { + return nil, ctx, nil, err + } + return database.Collection(r.collectionName), scoped, release, nil } func (r *MongoRepository) Collection() string { @@ -48,8 +62,14 @@ func prepareForWrite(obj storable.Storable) bson.ObjectID { } func (r *MongoRepository) Insert(ctx context.Context, obj storable.Storable, getFilter builder.Query) error { + collection, ctx, release, err := r.acquire(ctx) + if err != nil { + return err + } + defer release() + prepareForWrite(obj) - _, err := r.collection.InsertOne(ctx, obj) + _, err = collection.InsertOne(ctx, obj) if mongo.IsDuplicateKeyError(err) { if getFilter != nil { if err = r.FindOneByFilter(ctx, getFilter, obj); err != nil { @@ -62,6 +82,12 @@ func (r *MongoRepository) Insert(ctx context.Context, obj storable.Storable, get } func (r *MongoRepository) InsertMany(ctx context.Context, objects []storable.Storable) error { + collection, ctx, release, err := r.acquire(ctx) + if err != nil { + return err + } + defer release() + if len(objects) == 0 { return nil } @@ -72,12 +98,18 @@ func (r *MongoRepository) InsertMany(ctx context.Context, objects []storable.Sto docs[i] = obj } - _, err := r.collection.InsertMany(ctx, docs) + _, err = collection.InsertMany(ctx, docs) return err } func (r *MongoRepository) findOneByFilterImp(ctx context.Context, filter bson.D, errMessage string, result storable.Storable) error { - err := r.collection.FindOne(ctx, filter).Decode(result) + collection, ctx, release, err := r.acquire(ctx) + if err != nil { + return err + } + defer release() + + err = collection.FindOne(ctx, filter).Decode(result) if errors.Is(err, mongo.ErrNoDocuments) { return merrors.NoData(errMessage) } @@ -94,7 +126,13 @@ func (r *MongoRepository) Get(ctx context.Context, id bson.ObjectID, result stor type QueryFunc func(ctx context.Context, collection *mongo.Collection) (*mongo.Cursor, error) func (r *MongoRepository) executeQuery(ctx context.Context, queryFunc QueryFunc, decoder rd.DecodingFunc) error { - cursor, err := queryFunc(ctx, r.collection) + collection, ctx, release, err := r.acquire(ctx) + if err != nil { + return err + } + defer release() + + cursor, err := queryFunc(ctx, collection) if errors.Is(err, mongo.ErrNoDocuments) { return merrors.NoData("no_items_in_array") } @@ -111,7 +149,7 @@ func (r *MongoRepository) executeQuery(ctx context.Context, queryFunc QueryFunc, } } - return nil + return cursor.Err() } func (r *MongoRepository) Aggregate(ctx context.Context, pipeline builder.Pipeline, decoder rd.DecodingFunc) error { @@ -129,8 +167,14 @@ func (r *MongoRepository) FindManyByFilter(ctx context.Context, query builder.Qu } func (r *MongoRepository) FindAllByFilter(ctx context.Context, query builder.Query, result any) error { + collection, ctx, release, err := r.acquire(ctx) + if err != nil { + return err + } + defer release() + if query == nil { - cursor, err := r.collection.Find(ctx, bson.D{}) + cursor, err := collection.Find(ctx, bson.D{}) if err != nil { return err } @@ -139,7 +183,7 @@ func (r *MongoRepository) FindAllByFilter(ctx context.Context, query builder.Que }() return cursor.All(ctx, result) } - cursor, err := r.collection.Find(ctx, query.BuildQuery(), query.BuildOptions()) + cursor, err := collection.Find(ctx, query.BuildQuery(), query.BuildOptions()) if err != nil { return err } @@ -154,18 +198,30 @@ func (r *MongoRepository) FindOneByFilter(ctx context.Context, query builder.Que } func (r *MongoRepository) Update(ctx context.Context, obj storable.Storable) error { + collection, ctx, release, err := r.acquire(ctx) + if err != nil { + return err + } + defer release() + obj.Update() - return r.collection.FindOneAndReplace(ctx, idFilter(*obj.GetID()), obj).Err() + return collection.FindOneAndReplace(ctx, idFilter(*obj.GetID()), obj).Err() } func (r *MongoRepository) ReplaceOneByFilter(ctx context.Context, query builder.Query, obj storable.Storable) (bool, error) { + collection, ctx, release, err := r.acquire(ctx) + if err != nil { + return false, err + } + defer release() + if query == nil { return false, merrors.InvalidArgument("filter is required", "filter") } if obj == nil { return false, merrors.InvalidArgument("object is required", "obj") } - result, err := r.collection.ReplaceOne(ctx, query.BuildQuery(), obj) + result, err := collection.ReplaceOne(ctx, query.BuildQuery(), obj) if mongo.IsDuplicateKeyError(err) { return false, merrors.DataConflict("duplicate_key") } @@ -176,21 +232,39 @@ func (r *MongoRepository) ReplaceOneByFilter(ctx context.Context, query builder. } func (r *MongoRepository) Upsert(ctx context.Context, obj storable.Storable) error { + collection, ctx, release, err := r.acquire(ctx) + if err != nil { + return err + } + defer release() + id := prepareForWrite(obj) - _, err := r.collection.ReplaceOne(ctx, idFilter(id), obj, options.Replace().SetUpsert(true)) + _, err = collection.ReplaceOne(ctx, idFilter(id), obj, options.Replace().SetUpsert(true)) return err } func (r *MongoRepository) Patch(ctx context.Context, id bson.ObjectID, patch builder.Patch) error { + collection, ctx, release, err := r.acquire(ctx) + if err != nil { + return err + } + defer release() + if id.IsZero() { return merrors.InvalidArgument("zero id provided while patching", "id") } - _, err := r.collection.UpdateByID(ctx, id, patch.Build()) + _, err = collection.UpdateByID(ctx, id, patch.Build()) return err } func (r *MongoRepository) PatchMany(ctx context.Context, query builder.Query, patch builder.Patch) (int, error) { - result, err := r.collection.UpdateMany(ctx, query.BuildQuery(), patch.Build()) + collection, ctx, release, err := r.acquire(ctx) + if err != nil { + return 0, err + } + defer release() + + result, err := collection.UpdateMany(ctx, query.BuildQuery(), patch.Build()) if err != nil { return 0, err } @@ -198,10 +272,16 @@ func (r *MongoRepository) PatchMany(ctx context.Context, query builder.Query, pa } func (r *MongoRepository) ListIDs(ctx context.Context, query builder.Query) ([]bson.ObjectID, error) { + collection, ctx, release, err := r.acquire(ctx) + if err != nil { + return nil, err + } + defer release() + filter := query.BuildQuery() findOptions := query.BuildOptions().SetProjection(bson.M{storable.IDField: 1}) - cursor, err := r.collection.Find(ctx, filter, findOptions) + cursor, err := collection.Find(ctx, filter, findOptions) if err != nil { return nil, err } @@ -227,24 +307,45 @@ func (r *MongoRepository) ListIDs(ctx context.Context, query builder.Query) ([]b } func (r *MongoRepository) Delete(ctx context.Context, id bson.ObjectID) error { - _, err := r.collection.DeleteOne(ctx, idFilter(id)) + collection, ctx, release, err := r.acquire(ctx) + if err != nil { + return err + } + defer release() + + _, err = collection.DeleteOne(ctx, idFilter(id)) return err } func (r *MongoRepository) DeleteMany(ctx context.Context, query builder.Query) error { - _, err := r.collection.DeleteMany(ctx, query.BuildQuery()) + collection, ctx, release, err := r.acquire(ctx) + if err != nil { + return err + } + defer release() + + _, err = collection.DeleteMany(ctx, query.BuildQuery()) return err } func (r *MongoRepository) Name() string { - return r.collection.Name() + return r.collectionName } func (r *MongoRepository) WithTransaction(ctx context.Context, cb transaction.Callback) (any, error) { + collection, ctx, release, err := r.acquire(ctx) + if err != nil { + return nil, err + } + defer release() + if cb == nil { return nil, merrors.InvalidArgument("transaction callback is required", "callback") } - session, err := r.collection.Database().Client().StartSession() + if transaction.ActiveMongoTransaction(ctx) { + return cb(ctx) + } + session, err := collection.Database().Client().StartSession() if err != nil { return nil, err } diff --git a/backend/shared/database/internal/repository/repository_filter.go b/backend/shared/database/internal/repository/repository_filter.go index 381bb1e..420faf9 100644 --- a/backend/shared/database/internal/repository/repository_filter.go +++ b/backend/shared/database/internal/repository/repository_filter.go @@ -12,8 +12,14 @@ import ( ) func (r *MongoRepository) FindFirstByFilter(ctx context.Context, query builder.Query, result storable.Storable) error { + collection, ctx, release, err := r.acquire(ctx) + if err != nil { + return err + } + defer release() + opts := query.BuildOptions().SetLimit(1) - cursor, err := r.collection.Find(ctx, query.BuildQuery(), opts) + cursor, err := collection.Find(ctx, query.BuildQuery(), opts) if err != nil { return err } @@ -30,10 +36,22 @@ func (r *MongoRepository) FindFirstByFilter(ctx context.Context, query builder.Q } func (r *MongoRepository) CountByFilter(ctx context.Context, query builder.Query) (int64, error) { - return r.collection.CountDocuments(ctx, query.BuildQuery()) + collection, ctx, release, err := r.acquire(ctx) + if err != nil { + return 0, err + } + defer release() + + return collection.CountDocuments(ctx, query.BuildQuery()) } func (r *MongoRepository) PatchOneByFilter(ctx context.Context, query builder.Query, patch builder.Patch, result storable.Storable) (bool, error) { + collection, ctx, release, err := r.acquire(ctx) + if err != nil { + return false, err + } + defer release() + if query == nil { return false, merrors.InvalidArgument("filter is required", "filter") } @@ -43,7 +61,7 @@ func (r *MongoRepository) PatchOneByFilter(ctx context.Context, query builder.Qu if result == nil { return false, merrors.InvalidArgument("result is required", "result") } - err := r.collection.FindOneAndUpdate( + err = collection.FindOneAndUpdate( ctx, query.BuildQuery(), patch.Build(), diff --git a/backend/shared/database/internal/transaction/factory.go b/backend/shared/database/internal/transaction/factory.go index 344b312..f84c736 100644 --- a/backend/shared/database/internal/transaction/factory.go +++ b/backend/shared/database/internal/transaction/factory.go @@ -1,18 +1,15 @@ package transactionimp import ( + "git.sendico.io/tech/sxchnge/shared/database/source" "git.sendico.io/tech/sxchnge/shared/database/transaction" - "go.mongodb.org/mongo-driver/v2/mongo" ) -type MongoTransactionFactory struct { - client *mongo.Client -} +type MongoTransactionFactory struct{ database source.Source } -func (mtf *MongoTransactionFactory) CreateTransaction() transaction.Transaction { - return Create(mtf.client) +func (f *MongoTransactionFactory) CreateTransaction() transaction.Transaction { + return &MongoTransaction{database: f.database} } - -func CreateFactory(client *mongo.Client) transaction.Factory { - return &MongoTransactionFactory{client: client} +func CreateSourceFactory(database source.Source) transaction.Factory { + return &MongoTransactionFactory{database: database} } diff --git a/backend/shared/database/internal/transaction/transaction.go b/backend/shared/database/internal/transaction/transaction.go index 458d6c3..b31ff10 100644 --- a/backend/shared/database/internal/transaction/transaction.go +++ b/backend/shared/database/internal/transaction/transaction.go @@ -3,32 +3,29 @@ package transactionimp import ( "context" + "git.sendico.io/tech/sxchnge/shared/database/source" "git.sendico.io/tech/sxchnge/shared/database/transaction" - "go.mongodb.org/mongo-driver/v2/mongo" + merrors "git.sendico.io/tech/sxchnge/shared/errors" ) -type MongoTransaction struct { - client *mongo.Client -} +type MongoTransaction struct{ database source.Source } -func (mt *MongoTransaction) Execute(ctx context.Context, cb transaction.Callback) (any, error) { - // Reuse an outer customer workflow transaction when a token store joins it. +func (t *MongoTransaction) Execute(ctx context.Context, cb transaction.Callback) (any, error) { + if cb == nil { + return nil, merrors.InvalidArgument("transaction callback is required") + } + database, ctx, release, err := t.database.Acquire(ctx) + if err != nil { + return nil, err + } + defer release() if transaction.ActiveMongoTransaction(ctx) { return cb(ctx) } - session, err := mt.client.StartSession() + session, err := database.Client().StartSession() if err != nil { return nil, err } defer session.EndSession(ctx) - - callback := func(c context.Context) (any, error) { - return cb(c) - } - - return session.WithTransaction(ctx, callback) -} - -func Create(client *mongo.Client) *MongoTransaction { - return &MongoTransaction{client: client} + return session.WithTransaction(ctx, cb) } diff --git a/backend/shared/database/repository/repository.go b/backend/shared/database/repository/repository.go index f53bc95..2a34c32 100644 --- a/backend/shared/database/repository/repository.go +++ b/backend/shared/database/repository/repository.go @@ -8,6 +8,7 @@ import ( "git.sendico.io/tech/sxchnge/shared/database/repository/builder" rd "git.sendico.io/tech/sxchnge/shared/database/repository/decoder" ri "git.sendico.io/tech/sxchnge/shared/database/repository/index" + "git.sendico.io/tech/sxchnge/shared/database/source" "git.sendico.io/tech/sxchnge/shared/database/transaction" "go.mongodb.org/mongo-driver/v2/bson" "go.mongodb.org/mongo-driver/v2/mongo" @@ -53,6 +54,11 @@ type Repository interface { Collection() string } +func CreateRepository(db source.Source, collection string) Repository { + return repositoryimp.NewRepository(db, collection) +} + +// CreateMongoRepository adapts an externally owned fixed database. func CreateMongoRepository(db *mongo.Database, collection string) Repository { - return repositoryimp.NewMongoRepository(db, collection) + return CreateRepository(source.Fixed(db), collection) } diff --git a/backend/shared/database/rotation_test.go b/backend/shared/database/rotation_test.go new file mode 100644 index 0000000..433e498 --- /dev/null +++ b/backend/shared/database/rotation_test.go @@ -0,0 +1,275 @@ +package db_test + +import ( + "context" + "encoding/json" + "fmt" + "net/http" + "net/http/httptest" + "net/url" + "os" + "path/filepath" + "strings" + "sync" + "sync/atomic" + "testing" + "time" + + db "git.sendico.io/tech/sxchnge/shared/database" + dependencies "git.sendico.io/tech/sxchnge/shared/database/dependencies" + storable "git.sendico.io/tech/sxchnge/shared/database/record" + "git.sendico.io/tech/sxchnge/shared/database/repository" + merrors "git.sendico.io/tech/sxchnge/shared/errors" + "git.sendico.io/tech/sxchnge/shared/model" + "github.com/stretchr/testify/require" + "github.com/testcontainers/testcontainers-go/modules/mongodb" + "go.mongodb.org/mongo-driver/v2/bson" + "go.mongodb.org/mongo-driver/v2/mongo" + "go.mongodb.org/mongo-driver/v2/mongo/options" + "go.uber.org/zap" +) + +type rotationRecord struct { + storable.Base `bson:",inline"` + Value string `bson:"value"` +} + +func (*rotationRecord) Collection() string { return "rotation" } + +type leaseServer struct { + admin *mongo.Database + issued atomic.Int64 + revoked sync.Map + outage atomic.Bool + badPassword atomic.Bool +} + +func (s *leaseServer) ServeHTTP(w http.ResponseWriter, r *http.Request) { + if s.outage.Load() { + w.WriteHeader(503) + return + } + w.Header().Set("Content-Type", "application/json") + switch r.URL.Path { + case "/v1/auth/approle/login": + // A unique token owns each credential in this fake Vault HTTP boundary. + token := fmt.Sprintf("test-token-%d", s.issued.Load()+1) + _ = json.NewEncoder(w).Encode(map[string]any{"auth": map[string]any{"client_token": token, "lease_duration": 60}}) + case "/v1/database/creds/bff": + n := s.issued.Add(1) + user := fmt.Sprintf("lease%d", n) + err := s.admin.RunCommand(r.Context(), bson.D{{Key: "createUser", Value: user}, {Key: "pwd", Value: "test-password"}, {Key: "roles", Value: bson.A{bson.M{"role": "readWrite", "db": "rotation"}}}}).Err() + if err != nil { + w.WriteHeader(500) + return + } + password := "test-password" + if s.badPassword.Load() { + password = "wrong-password" + } + _ = json.NewEncoder(w).Encode(map[string]any{"lease_id": user, "lease_duration": 8, "data": map[string]string{"username": user, "password": password}}) + case "/v1/auth/token/revoke-self": + user := "lease" + strings.TrimPrefix(r.Header.Get("X-Vault-Token"), "test-token-") + _ = s.admin.RunCommand(r.Context(), bson.D{{Key: "dropUser", Value: user}}).Err() + s.revoked.Store(user, true) + w.WriteHeader(204) + default: + w.WriteHeader(404) + } +} + +func TestRotationPreservesRequestsCursorsAndTransactions(t *testing.T) { + ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second) + defer cancel() + container, err := mongodb.Run(ctx, "mongo:latest", mongodb.WithReplicaSet("rs"), mongodb.WithUsername("root"), mongodb.WithPassword("test-admin-password")) + require.NoError(t, err) + t.Cleanup(func() { _ = container.Terminate(context.Background()) }) + raw, err := container.ConnectionString(ctx) + require.NoError(t, err) + uri, err := url.Parse(raw) + require.NoError(t, err) + q := uri.Query() + q.Set("directConnection", "true") + q.Set("authSource", "admin") + uri.RawQuery = q.Encode() + admin, err := mongo.Connect(options.Client().ApplyURI(uri.String())) + require.NoError(t, err) + defer admin.Disconnect(context.Background()) + fake := &leaseServer{admin: admin.Database("admin")} + vault := httptest.NewServer(fake) + defer vault.Close() + directory := t.TempDir() + role := filepath.Join(directory, "role") + secret := filepath.Join(directory, "secret") + require.NoError(t, os.WriteFile(role, []byte("test-role"), 0600)) + require.NoError(t, os.WriteFile(secret, []byte("test-secret"), 0600)) + uri.User = nil + uri.Path = "/rotation" + factory, err := db.NewConnection(zap.NewNop(), &db.Config{Driver: db.Mongo, Settings: model.SettingsT{"database": "rotation", "uri": uri.String(), "vault": map[string]any{"address": vault.URL, "role": "bff", "role_id_file": role, "secret_id_file": secret}}}) + require.NoError(t, err) + defer factory.CloseConnection() + deps, err := dependencies.ResolveDependencies(factory) + require.NoError(t, err) + repo := repository.CreateRepository(deps.Database, "rotation") + require.NoError(t, repo.Insert(ctx, &rotationRecord{Value: "cursor-fixture"}, nil)) + + var failures, requests atomic.Int64 + httpServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if _, e := repo.CountByFilter(r.Context(), repository.Query()); e != nil { + failures.Add(1) + w.WriteHeader(503) + return + } + requests.Add(1) + w.WriteHeader(200) + })) + defer httpServer.Close() + loadCtx, stopLoad := context.WithCancel(ctx) + var load sync.WaitGroup + defer func() { stopLoad(); load.Wait() }() + for range 4 { + load.Add(1) + go func() { + defer load.Done() + for loadCtx.Err() == nil { + res, e := http.Get(httpServer.URL) + if e != nil { + failures.Add(1) + } else { + res.Body.Close() + if res.StatusCode != 200 { + failures.Add(1) + } + } + time.Sleep(5 * time.Millisecond) + } + }() + } + + // A cursor and a transaction must keep generation 1 alive through publication. + first, scope, release, err := deps.Database.Acquire(ctx) + require.NoError(t, err) + firstClient := first.Client() + release() + _, _, _, err = deps.Database.Acquire(scope) + require.Error(t, err, "escaped operation scope must be rejected") + txEntered := make(chan struct{}) + finishTx := make(chan struct{}) + txDone := make(chan error, 1) + var entered sync.Once + go func() { + _, e := factory.TransactionFactory().CreateTransaction().Execute(ctx, func(tx context.Context) (any, error) { + if e := repo.Insert(tx, &rotationRecord{Value: "transaction-before"}, nil); e != nil { + return nil, e + } + entered.Do(func() { close(txEntered) }) + select { + case <-finishTx: + case <-tx.Done(): + return nil, tx.Err() + } + // Nested transaction must share the same session/generation. + return repo.WithTransaction(tx, func(nested context.Context) (any, error) { + return nil, repo.Insert(nested, &rotationRecord{Value: "transaction-after"}, nil) + }) + }) + txDone <- e + }() + select { + case <-txEntered: + case err := <-txDone: + t.Fatalf("transaction did not enter: %v", err) + case <-ctx.Done(): + t.Fatal(ctx.Err()) + } + cursorEntered := make(chan struct{}) + finishCursor := make(chan struct{}) + cursorDone := make(chan error, 1) + go func() { + var once sync.Once + cursorDone <- repo.FindManyByFilter(ctx, repository.Query(), func(d *mongo.Cursor) error { + once.Do(func() { close(cursorEntered) }) + select { + case <-finishCursor: + return nil + case <-ctx.Done(): + return ctx.Err() + } + }) + }() + select { + case <-cursorEntered: + case err := <-cursorDone: + t.Fatalf("cursor did not enter: %v", err) + case <-ctx.Done(): + t.Fatal(ctx.Err()) + } + require.Eventually(t, func() bool { + database, _, free, e := deps.Database.Acquire(ctx) + if e != nil { + return false + } + defer free() + return database.Client() != firstClient + }, 6*time.Second, 20*time.Millisecond) + _, revoked := fake.revoked.Load("lease1") + require.False(t, revoked, "in-flight work lost its credentials") + close(finishTx) + require.NoError(t, <-txDone) + _, revoked = fake.revoked.Load("lease1") + require.False(t, revoked, "cursor must retain old pool") + close(finishCursor) + require.NoError(t, <-cursorDone) + require.Eventually(t, func() bool { _, ok := fake.revoked.Load("lease1"); return ok }, 2*time.Second, 20*time.Millisecond) + count, err := repo.CountByFilter(ctx, repository.Filter("value", "transaction-after")) + require.NoError(t, err) + require.EqualValues(t, 1, count) + + // Reject a candidate with invalid credentials while retaining the valid pool. + fake.badPassword.Store(true) + before := fake.issued.Load() + require.Eventually(t, func() bool { return fake.issued.Load() > before }, 5*time.Second, 20*time.Millisecond) + require.Eventually(t, func() bool { _, ok := fake.revoked.Load(fmt.Sprintf("lease%d", before+1)); return ok }, time.Second, 10*time.Millisecond) + require.NoError(t, repo.Insert(ctx, &rotationRecord{Value: "during-bad-refresh"}, nil)) + fake.badPassword.Store(false) + fake.outage.Store(true) + require.NoError(t, repo.Insert(ctx, &rotationRecord{Value: "during-vault-outage"}, nil)) + time.Sleep(300 * time.Millisecond) + fake.outage.Store(false) + require.Eventually(t, func() bool { return fake.issued.Load() > before+1 }, 3*time.Second, 20*time.Millisecond) + stopLoad() + load.Wait() + require.Greater(t, requests.Load(), int64(100)) + require.Zero(t, failures.Load(), "HTTP requests failed during rotation") + + // If Vault stays unavailable beyond the lease, fail closed without restarting. + fake.outage.Store(true) + require.Eventually(t, func() bool { _, e := repo.CountByFilter(ctx, repository.Query()); return e != nil }, 10*time.Second, 50*time.Millisecond) + fake.outage.Store(false) + require.Eventually(t, func() bool { _, e := repo.CountByFilter(ctx, repository.Query()); return e == nil }, 6*time.Second, 20*time.Millisecond) + // Transaction errors still roll back all writes after generation changes. + _, err = repo.WithTransaction(ctx, func(tx context.Context) (any, error) { + if e := repo.Insert(tx, &rotationRecord{Value: "rollback"}, nil); e != nil { + return nil, e + } + return nil, merrors.Internal("injected rollback") + }) + require.Error(t, err) + count, err = repo.CountByFilter(ctx, repository.Filter("value", "rollback")) + require.NoError(t, err) + require.Zero(t, count) + held, heldContext, releaseHeld, err := deps.Database.Acquire(ctx) + require.NoError(t, err) + manager := deps.Database.(interface{ Close(context.Context) error }) + short, stop := context.WithTimeout(ctx, 30*time.Millisecond) + require.ErrorIs(t, manager.Close(short), context.DeadlineExceeded) + stop() + _, _, _, err = deps.Database.Acquire(ctx) + require.Error(t, err) + require.NoError(t, held.Client().Ping(heldContext, nil), "shutdown must drain, not kill held work") + releaseHeld() + closed, stop := context.WithTimeout(ctx, 5*time.Second) + defer stop() + require.NoError(t, manager.Close(closed)) + +} diff --git a/backend/shared/database/source/manager.go b/backend/shared/database/source/manager.go new file mode 100644 index 0000000..0c35d53 --- /dev/null +++ b/backend/shared/database/source/manager.go @@ -0,0 +1,253 @@ +package source + +import ( + "context" + "sync" + "sync/atomic" + "time" + + merrors "git.sendico.io/tech/sxchnge/shared/errors" + mlogger "git.sendico.io/tech/sxchnge/shared/logging" + "go.mongodb.org/mongo-driver/v2/mongo" + "go.mongodb.org/mongo-driver/v2/mongo/options" + "go.mongodb.org/mongo-driver/v2/mongo/readpref" + "go.uber.org/zap" +) + +// Lease owns a credential and its revocation capability. Neither may be logged. +type Lease struct { + URI string + ExpiresAt time.Time + Revoke func(context.Context) error +} +type Issuer interface { + Issue(context.Context) (Lease, error) +} + +type generation struct { + client *mongo.Client + lease Lease + refs int + retired bool + drained chan struct{} +} +type pin struct { + manager *Manager + generation *generation + active atomic.Bool +} +type pinKey struct{} + +// Manager publishes a verified replacement before retiring the previous pool. +// Every cursor, operation and transaction retains its originating generation. +type Manager struct { + mu sync.Mutex + current *generation + closed bool + issuer Issuer + database string + logger mlogger.Logger + cancel context.CancelFunc + workerDone chan struct{} + draining sync.WaitGroup +} + +func New(ctx context.Context, logger mlogger.Logger, database string, issuer Issuer) (*Manager, error) { + if database == "" || issuer == nil { + return nil, merrors.InvalidArgument("database and credential issuer are required") + } + life, cancel := context.WithCancel(context.Background()) + m := &Manager{issuer: issuer, database: database, logger: logger, cancel: cancel, workerDone: make(chan struct{})} + g, err := m.prepare(ctx) + if err != nil { + cancel() + return nil, err + } + m.current = g + go m.run(life) + return m, nil +} + +func (m *Manager) Acquire(ctx context.Context) (*mongo.Database, context.Context, func(), error) { + if err := ctx.Err(); err != nil { + return nil, ctx, nil, err + } + if p, ok := ctx.Value(pinKey{}).(*pin); ok && p.manager == m { + if !p.active.Load() { + return nil, ctx, nil, merrors.FailedPrecondition("database operation scope has ended") + } + return p.generation.client.Database(m.database), ctx, func() {}, nil + } + m.mu.Lock() + g := m.current + if m.closed || g == nil || !time.Now().Before(g.lease.ExpiresAt) { + m.mu.Unlock() + return nil, ctx, nil, merrors.ServiceUnavailable("no valid database credential is available") + } + if session := mongo.SessionFromContext(ctx); session != nil && session.Client() != g.client { + m.mu.Unlock() + return nil, ctx, nil, merrors.FailedPrecondition("transaction has no matching database operation scope") + } + g.refs++ + m.mu.Unlock() + p := &pin{manager: m, generation: g} + p.active.Store(true) + bounded, cancel := context.WithDeadline(ctx, g.lease.ExpiresAt) + scoped := context.WithValue(bounded, pinKey{}, p) + release := func() { + if !p.active.CompareAndSwap(true, false) { + return + } + cancel() + m.mu.Lock() + g.refs-- + if g.retired && g.refs == 0 { + close(g.drained) + } + m.mu.Unlock() + } + return g.client.Database(m.database), scoped, release, nil +} + +func (m *Manager) prepare(ctx context.Context) (*generation, error) { + ctx, cancel := context.WithTimeout(ctx, 10*time.Second) + defer cancel() + lease, err := m.issuer.Issue(ctx) + if err != nil { + return nil, err + } + var client *mongo.Client + success := false + defer func() { + if !success { + if client != nil { + cleanup, cancel := context.WithTimeout(context.Background(), 5*time.Second) + _ = client.Disconnect(cleanup) + cancel() + } + m.draining.Add(1) + go func() { defer m.draining.Done(); m.revoke(lease) }() + } + }() + if lease.Revoke == nil || time.Until(lease.ExpiresAt) < 2*time.Second { + return nil, merrors.FailedPrecondition("database credential lifetime is too short") + } + client, err = mongo.Connect(options.Client().ApplyURI(lease.URI).SetServerSelectionTimeout(5 * time.Second)) + if err != nil { + return nil, merrors.OpaqueWrap(merrors.ErrServiceUnavailable, err, "create replacement database pool") + } + if err = client.Ping(ctx, readpref.Primary()); err != nil { + return nil, merrors.OpaqueWrap(merrors.ErrServiceUnavailable, err, "validate replacement database pool") + } + if time.Until(lease.ExpiresAt) < time.Second { + return nil, merrors.FailedPrecondition("replacement database credential expired during validation") + } + success = true + return &generation{client: client, lease: lease, drained: make(chan struct{})}, nil +} + +func (m *Manager) retireLocked(g *generation) { + g.retired = true + if g.refs == 0 { + close(g.drained) + } + m.draining.Add(1) + go func() { + defer m.draining.Done() + <-g.drained + ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) + defer cancel() + if err := g.client.Disconnect(ctx); err != nil { + m.logger.Warn("Retired database pool close failed") + } + m.revoke(g.lease) + m.logger.Info("Retired database pool drained") + }() +} + +func (m *Manager) revoke(lease Lease) { + if lease.Revoke == nil { + return + } + // Expiration is the backstop when Vault is unavailable during bounded cleanup. + ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second) + defer cancel() + if err := lease.Revoke(ctx); err != nil { + m.logger.Warn("Database credential cleanup failed; Vault expiry will revoke it") + } + +} + +func (m *Manager) run(ctx context.Context) { + defer close(m.workerDone) + m.mu.Lock() + wait := time.Until(m.current.lease.ExpiresAt) / 2 + m.mu.Unlock() + backoff := 250 * time.Millisecond + failureReported, expiryReported := false, false + for { + timer := time.NewTimer(wait) + select { + case <-ctx.Done(): + timer.Stop() + return + case <-timer.C: + } + next, err := m.prepare(ctx) + if err != nil { + if ctx.Err() != nil { + return + } + m.mu.Lock() + remaining := time.Until(m.current.lease.ExpiresAt) + m.mu.Unlock() + if !failureReported { + m.logger.Warn("Database credential refresh failed; retaining current pool", zap.Duration("valid_for", max(remaining, 0))) + failureReported = true + } + if remaining <= 0 && !expiryReported { + m.logger.Error("Database credentials expired; rejecting new operations until refresh succeeds") + expiryReported = true + } + wait = backoff + if remaining > 0 { + wait = max(100*time.Millisecond, min(wait, remaining/4)) + } + backoff = min(5*time.Second, backoff*2) + + continue + } + m.mu.Lock() + if m.closed { + m.retireLocked(next) + m.mu.Unlock() + return + } + old := m.current + m.current = next + m.retireLocked(old) + m.mu.Unlock() + failureReported, expiryReported = false, false + backoff = 250 * time.Millisecond + m.logger.Info("Database pool replaced without restarting service") + wait = time.Until(next.lease.ExpiresAt) / 2 + } +} + +func (m *Manager) Close(ctx context.Context) error { + m.mu.Lock() + if !m.closed { + m.closed = true + m.cancel() + m.retireLocked(m.current) + } + m.mu.Unlock() + done := make(chan struct{}) + go func() { <-m.workerDone; m.draining.Wait(); close(done) }() + select { + case <-done: + return nil + case <-ctx.Done(): + return ctx.Err() + } +} diff --git a/backend/shared/database/source/source.go b/backend/shared/database/source/source.go new file mode 100644 index 0000000..b372796 --- /dev/null +++ b/backend/shared/database/source/source.go @@ -0,0 +1,30 @@ +// Package source owns Mongo connection generations and operation lifetimes. +package source + +import ( + "context" + + merrors "git.sendico.io/tech/sxchnge/shared/errors" + "go.mongodb.org/mongo-driver/v2/mongo" +) + +// Source pins one database/client until release. The returned context must be +// passed to nested operations so transactions never cross client generations. +type Source interface { + Acquire(context.Context) (*mongo.Database, context.Context, func(), error) +} + +type fixed struct{ database *mongo.Database } + +// Fixed adapts an externally owned database, e.g. a test fixture. It never closes it. +func Fixed(database *mongo.Database) Source { return &fixed{database: database} } + +func (s *fixed) Acquire(ctx context.Context) (*mongo.Database, context.Context, func(), error) { + if s.database == nil { + return nil, ctx, nil, merrors.FailedPrecondition("database is not configured") + } + if session := mongo.SessionFromContext(ctx); session != nil && session.Client() != s.database.Client() { + return nil, ctx, nil, merrors.FailedPrecondition("transaction belongs to another database client") + } + return s.database, ctx, func() {}, nil +} diff --git a/backend/shared/database/template/template.go b/backend/shared/database/template/template.go index 93b02cc..4dd10bd 100644 --- a/backend/shared/database/template/template.go +++ b/backend/shared/database/template/template.go @@ -6,12 +6,12 @@ import ( "git.sendico.io/tech/sxchnge/shared/database/record" "git.sendico.io/tech/sxchnge/shared/database/repository" "git.sendico.io/tech/sxchnge/shared/database/repository/builder" + "git.sendico.io/tech/sxchnge/shared/database/source" "git.sendico.io/tech/sxchnge/shared/errors" "git.sendico.io/tech/sxchnge/shared/logging" "git.sendico.io/tech/sxchnge/shared/logging/fields" "git.sendico.io/tech/sxchnge/shared/service" "go.mongodb.org/mongo-driver/v2/bson" - "go.mongodb.org/mongo-driver/v2/mongo" ) type CascadeDeleterT = func(ctx context.Context, objectRef bson.ObjectID) error @@ -88,11 +88,11 @@ func (db *DBImp[T]) DeleteCascade(ctx context.Context, objectRef bson.ObjectID) func Create[T storable.Storable]( logger mlogger.Logger, collection mservice.Type, - db *mongo.Database, + db source.Source, ) *DBImp[T] { res := &DBImp[T]{ Logger: logger.Named(collection), - Repository: repository.CreateMongoRepository(db, collection), + Repository: repository.CreateRepository(db, collection), } res.cdeleter = res.deleteCascadeStub return res diff --git a/backend/shared/database/vault/issuer.go b/backend/shared/database/vault/issuer.go new file mode 100644 index 0000000..73e8d88 --- /dev/null +++ b/backend/shared/database/vault/issuer.go @@ -0,0 +1,140 @@ +// Package vault issues independently owned MongoDB leases through Vault AppRole. +package vault + +import ( + "bytes" + "context" + "encoding/json" + "io" + "net/http" + "net/url" + "os" + "regexp" + "strings" + "time" + + "git.sendico.io/tech/sxchnge/shared/database/source" + merrors "git.sendico.io/tech/sxchnge/shared/errors" +) + +type Config struct { + Address string `mapstructure:"address"` + DatabaseRole string `mapstructure:"role"` + RoleIDFile string `mapstructure:"role_id_file"` + SecretIDFile string `mapstructure:"secret_id_file"` +} +type Issuer struct { + config Config + uri *url.URL + http *http.Client +} + +func New(config Config, uri string) (*Issuer, error) { + endpoint, err := url.Parse(config.Address) + if err != nil || endpoint.Host == "" || (endpoint.Scheme != "http" && endpoint.Scheme != "https") || endpoint.User != nil || endpoint.RawQuery != "" || endpoint.Fragment != "" { + return nil, merrors.InvalidArgument("invalid Vault address") + } + if !regexp.MustCompile(`^[a-zA-Z0-9_-]+$`).MatchString(config.DatabaseRole) || config.RoleIDFile == "" || config.SecretIDFile == "" { + return nil, merrors.InvalidArgument("Vault database role and AppRole files are required") + } + database, err := url.Parse(uri) + if err != nil || database.Scheme != "mongodb" || database.Host == "" || database.User != nil { + return nil, merrors.InvalidArgument("Vault requires a MongoDB URI without embedded credentials") + } + return &Issuer{config: config, uri: database, http: &http.Client{Timeout: 10 * time.Second, CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse }}}, nil +} + +type response struct { + Auth struct { + Token string `json:"client_token"` + TTL int `json:"lease_duration"` + } `json:"auth"` + TTL int `json:"lease_duration"` + LeaseID string `json:"lease_id"` + Data struct { + Username string `json:"username"` + Password string `json:"password"` + } `json:"data"` +} + +func (i *Issuer) request(ctx context.Context, path, token string, body any) (response, error) { + var out response + var input io.Reader + method := http.MethodGet + if body != nil { + raw, err := json.Marshal(body) + if err != nil { + return out, merrors.InternalWrap(err, "encode Vault request") + } + input = bytes.NewReader(raw) + method = http.MethodPost + } + req, err := http.NewRequestWithContext(ctx, method, strings.TrimRight(i.config.Address, "/")+"/v1/"+path, input) + if err != nil { + return out, merrors.OpaqueWrap(merrors.ErrInvalidArg, err, "create Vault request") + } + req.Header.Set("Content-Type", "application/json") + if token != "" { + req.Header.Set("X-Vault-Token", token) + } + res, err := i.http.Do(req) + if err != nil { + return out, merrors.OpaqueWrap(merrors.ErrServiceUnavailable, err, "Vault request failed") + } + defer res.Body.Close() + if res.StatusCode < 200 || res.StatusCode >= 300 { + return out, merrors.ServiceUnavailable("Vault request was rejected") + } + if res.StatusCode == http.StatusNoContent { + return out, nil + } + if err = json.NewDecoder(io.LimitReader(res.Body, 1<<20)).Decode(&out); err != nil { + return out, merrors.OpaqueWrap(merrors.ErrServiceUnavailable, err, "invalid Vault response") + } + return out, nil +} + +func (i *Issuer) Issue(ctx context.Context) (source.Lease, error) { + var lease source.Lease + role, err := os.ReadFile(i.config.RoleIDFile) + if err != nil { + return lease, merrors.OpaqueWrap(merrors.ErrServiceUnavailable, err, "read Vault AppRole identity") + } + secret, err := os.ReadFile(i.config.SecretIDFile) + if err != nil { + return lease, merrors.OpaqueWrap(merrors.ErrServiceUnavailable, err, "read Vault AppRole credential") + } + started := time.Now() + login, err := i.request(ctx, "auth/approle/login", "", map[string]string{"role_id": strings.TrimSpace(string(role)), "secret_id": strings.TrimSpace(string(secret))}) + if err != nil { + return lease, err + } + if login.Auth.Token == "" { + return lease, merrors.FailedPrecondition("Vault returned no application identity") + } + // One token owns one database lease. Revoking it never touches Agent's token + // or the replacement generation, and requires only the default self policy. + revoke := func(ctx context.Context) error { + _, err := i.request(ctx, "auth/token/revoke-self", login.Auth.Token, map[string]any{}) + return err + } + if login.Auth.TTL <= 0 { + cleanup, cancel := context.WithTimeout(context.Background(), 3*time.Second) + defer cancel() + _ = revoke(cleanup) + return lease, merrors.FailedPrecondition("Vault returned no bounded identity lifetime") + } + credential, err := i.request(ctx, "database/creds/"+i.config.DatabaseRole, login.Auth.Token, nil) + if err != nil || credential.Data.Username == "" || credential.Data.Password == "" || credential.TTL <= 0 || credential.LeaseID == "" { + cleanup, cancel := context.WithTimeout(context.Background(), 3*time.Second) + defer cancel() + _ = revoke(cleanup) + if err != nil { + return lease, err + } + return lease, merrors.FailedPrecondition("Vault returned incomplete database credentials") + } + uri := *i.uri + uri.User = url.UserPassword(credential.Data.Username, credential.Data.Password) + return source.Lease{URI: uri.String(), ExpiresAt: started.Add(time.Duration(min(login.Auth.TTL, credential.TTL)) * time.Second), Revoke: revoke}, nil +} diff --git a/backend/shared/database/vault/issuer_test.go b/backend/shared/database/vault/issuer_test.go new file mode 100644 index 0000000..3e08815 --- /dev/null +++ b/backend/shared/database/vault/issuer_test.go @@ -0,0 +1,79 @@ +package vault + +import ( + "context" + "encoding/json" + "net/http" + "net/http/httptest" + "net/url" + "os" + "path/filepath" + "sync/atomic" + "testing" + "time" + + "github.com/stretchr/testify/require" +) + +func TestIssuerLifetimeEncodingAndCleanup(t *testing.T) { + for _, test := range []struct { + name string + identityTTL, databaseTTL int + reject bool + wantError bool + }{ + {"identity limits lease", 4, 20, false, false}, + {"database limits lease", 20, 4, false, false}, + {"invalid identity", 0, 20, false, true}, + {"invalid database lifetime", 20, 0, false, true}, + {"rejected issuance", 20, 20, true, true}, + } { + t.Run(test.name, func(t *testing.T) { + var revoked atomic.Bool + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + switch r.URL.Path { + case "/v1/auth/approle/login": + _ = json.NewEncoder(w).Encode(map[string]any{"auth": map[string]any{"client_token": "private-token", "lease_duration": test.identityTTL}}) + case "/v1/database/creds/bff": + if test.reject { + http.Error(w, "private-password", 403) + return + } + _ = json.NewEncoder(w).Encode(map[string]any{"lease_id": "lease", "lease_duration": test.databaseTTL, "data": map[string]string{"username": "user@host", "password": "p@ss:/?#"}}) + case "/v1/auth/token/revoke-self": + if r.Header.Get("X-Vault-Token") == "private-token" { + revoked.Store(true) + } + w.WriteHeader(204) + default: + w.WriteHeader(404) + } + })) + defer server.Close() + directory := t.TempDir() + identity, secret := filepath.Join(directory, "identity"), filepath.Join(directory, "secret") + require.NoError(t, os.WriteFile(identity, []byte("identity"), 0600)) + require.NoError(t, os.WriteFile(secret, []byte("secret"), 0600)) + issuer, err := New(Config{Address: server.URL, DatabaseRole: "bff", RoleIDFile: identity, SecretIDFile: secret}, "mongodb://localhost/db?authSource=admin") + require.NoError(t, err) + started := time.Now() + lease, err := issuer.Issue(context.Background()) + if test.wantError { + require.Error(t, err) + require.NotContains(t, err.Error(), "private-") + require.True(t, revoked.Load()) + return + } + require.NoError(t, err) + require.WithinDuration(t, started.Add(4*time.Second), lease.ExpiresAt, time.Second) + parsed, err := url.Parse(lease.URI) + require.NoError(t, err) + require.Equal(t, "user@host", parsed.User.Username()) + password, _ := parsed.User.Password() + require.Equal(t, "p@ss:/?#", password) + require.False(t, revoked.Load()) + require.NoError(t, lease.Revoke(context.Background())) + require.True(t, revoked.Load()) + }) + } +} diff --git a/backend/shared/ratelimit/store/db.go b/backend/shared/ratelimit/store/db.go index 7069e54..a45cd87 100644 --- a/backend/shared/ratelimit/store/db.go +++ b/backend/shared/ratelimit/store/db.go @@ -11,9 +11,9 @@ import ( "git.sendico.io/tech/sxchnge/shared/database/record" "git.sendico.io/tech/sxchnge/shared/database/repository" ri "git.sendico.io/tech/sxchnge/shared/database/repository/index" + "git.sendico.io/tech/sxchnge/shared/database/source" "git.sendico.io/tech/sxchnge/shared/errors" "git.sendico.io/tech/sxchnge/shared/logging" - "go.mongodb.org/mongo-driver/v2/mongo" "go.uber.org/zap" ) @@ -55,7 +55,7 @@ type DB struct { logger mlogger.Logger } -func Create(logger mlogger.Logger, database *mongo.Database, collection string) (*DB, error) { +func Create(logger mlogger.Logger, database source.Source, collection string) (*DB, error) { if database == nil { return nil, merrors.InvalidArgument("database is required", "database") } @@ -64,7 +64,7 @@ func Create(logger mlogger.Logger, database *mongo.Database, collection string) collection = defaultCollection } db := &DB{ - repo: repository.CreateMongoRepository(database, collection), + repo: repository.CreateRepository(database, collection), logger: logger.Named("rate_limits"), } if err := db.ensureIndexes(); err != nil { diff --git a/dev/agents.py b/dev/agents.py index 70e71fe..5fecdf7 100644 --- a/dev/agents.py +++ b/dev/agents.py @@ -14,10 +14,6 @@ def config(service): if service == 'nats': env['NATS_TOKEN'] = kv('nats', 'token') else: - database = 'sxchnge' if service == 'bff' else 'sxchnge_notifications' - env['SXCHNGE_MONGO_URI'] = ('{{ with secret "database/creds/' + service + '" }}' - 'mongodb://{{ .Data.username | urlquery }}:{{ .Data.password | urlquery }}' - '@mongo:27017/' + database + '?authSource=admin&replicaSet=rs0{{ end }}') env['TLS_CA'] = kv('tls', 'ca') if service == 'bff': env['SXCHNGE_JWT_SECRET'] = kv('bff', 'jwt') diff --git a/dev/config/bff.yml b/dev/config/bff.yml index c2b493c..ca914db 100644 --- a/dev/config/bff.yml +++ b/dev/config/bff.yml @@ -6,7 +6,12 @@ http_server: database: driver: mongodb settings: - uri_env: SXCHNGE_MONGO_URI + uri: mongodb://mongo:27017/sxchnge?authSource=admin&replicaSet=rs0 + vault: + address: http://vault:8200 + role: bff + role_id_file: /etc/agent/role-id + secret_id_file: /etc/agent/secret-id database: sxchnge api: middleware: diff --git a/dev/config/notifications.yml b/dev/config/notifications.yml index 687274a..a5a3882 100644 --- a/dev/config/notifications.yml +++ b/dev/config/notifications.yml @@ -4,7 +4,12 @@ tls_key: "/run/sxchnge/server.key" database: driver: mongodb settings: - uri_env: SXCHNGE_MONGO_URI + uri: mongodb://mongo:27017/sxchnge_notifications?authSource=admin&replicaSet=rs0 + vault: + address: http://vault:8200 + role: notifications + role_id_file: /etc/agent/role-id + secret_id_file: /etc/agent/secret-id database: sxchnge_notifications messaging: driver: NATS diff --git a/dev/rotation.py b/dev/rotation.py new file mode 100644 index 0000000..8a37c46 --- /dev/null +++ b/dev/rotation.py @@ -0,0 +1,92 @@ +"""Exercise real Vault database rotation under BFF HTTP transaction traffic. + +Run after make build/bootstrap. No Resend credential or live email is needed. +Temporarily shortens the BFF database role TTL, restores it on every exit, and +preserves whether BFF/frontend were running before this check. +""" +import concurrent.futures +import json +import ssl +import threading +import time +import urllib.error +import urllib.request +import uuid + +from bootstrap import compose +from vault import STATE, Vault + + +def verify_rotation(): + vault = Vault() + vault.existing() + role = 'database/roles/bff' + original = vault.call(role)['data'] + running = {json.loads(line)['Service'] for line in compose('ps', '--format', 'json', capture=True).splitlines()} + stop = threading.Event() + successes = [] + failures = [] + lock = threading.Lock() + context = ssl.create_default_context(cafile=str(STATE / 'ca.pem')) + + def requests(): + while not stop.is_set(): + # Exactly 43 characters invokes the activation transaction and token + # lookup, then returns the expected invalid-token response. No user + # or notification is created and the test never sends an email. + token = uuid.uuid4().hex + uuid.uuid4().hex[:11] + request = urllib.request.Request('https://localhost:5126/api/v1/accounts/verification/confirm', + data=json.dumps({'token': token}).encode(), headers={ + 'Content-Type': 'application/json', 'Origin': 'https://localhost:5126', + 'X-Auth-Transport': 'cookie', 'X-CSRF-Protection': '1'}) + try: + with urllib.request.urlopen(request, context=context, timeout=5) as response: + code = response.status + body = response.read() + except urllib.error.HTTPError as error: + code = error.code + body = error.read() + except Exception: + code = 0 + body = b'' + with lock: + (successes if code == 400 and b'Verification is invalid or already used' in body else failures).append(code) + stop.wait(0.03) + + try: + vault.call(role, dict(original, default_ttl='12s', max_ttl='24s')) + compose('up', '-d', '--no-deps', '--force-recreate', '--wait', '--wait-timeout', '90', 'bff', 'frontend') + process = compose('exec', '-T', 'bff', 'pidof', 'service', capture=True).strip() + with concurrent.futures.ThreadPoolExecutor(max_workers=4) as pool: + workers = [pool.submit(requests) for _ in range(4)] + try: + deadline = time.monotonic() + 40 + while time.monotonic() < deadline: + time.sleep(1) + current = compose('exec', '-T', 'bff', 'pidof', 'service', capture=True).strip() + if current != process: + raise RuntimeError('BFF process restarted during credential rotation') + logs = compose('logs', '--no-color', 'bff', capture=True) + if logs.count('Database pool replaced without restarting service') >= 3: + break + else: + raise RuntimeError('Three database pool rotations were not observed') + finally: + stop.set() + for worker in workers: + worker.result() + if failures or len(successes) < 100: + raise RuntimeError(f'Rotation traffic check: {len(successes)} expected responses, {len(failures)} failures') + print(f'PASS: three real Vault rotations, unchanged BFF PID, {len(successes)} transaction-backed HTTP requests, zero failures') + finally: + stop.set() + vault.call(role, original) + newly_started = [name for name in ['frontend', 'bff'] if name not in running] + if newly_started: + compose('stop', *newly_started) + compose('rm', '-f', *newly_started) + # Existing services stay online; their next refresh picks up normal TTLs. + + +if __name__ == '__main__': + verify_rotation() diff --git a/dev/test_local.py b/dev/test_local.py index 4af9e93..3a60194 100644 --- a/dev/test_local.py +++ b/dev/test_local.py @@ -4,6 +4,7 @@ import tempfile import unittest from unittest.mock import patch +import agents import bootstrap import images from vault import Vault @@ -33,6 +34,14 @@ class Releases(unittest.TestCase): self.assertEqual(target.read_text(), 'previous build') +class AgentContract(unittest.TestCase): + def test_database_lease_does_not_trigger_process_restart(self): + for service in ['bff', 'notifications']: + config = agents.config(service) + self.assertNotIn('database/creds/', config) + self.assertNotIn('SXCHNGE_MONGO_URI', config) + + class Bootstrap(unittest.TestCase): def test_missing_resend_credentials_stops_before_product_start(self): with tempfile.TemporaryDirectory() as temporary: diff --git a/docs/implementation.md b/docs/implementation.md index 68d3004..e2deefa 100644 --- a/docs/implementation.md +++ b/docs/implementation.md @@ -601,8 +601,9 @@ commands, current-stable backend test images and repository container policy. committed. Root `AGENTS.md` now records the stable-container policy. - Bootstrap configures Vault KV/PKI/AppRole and `mongodb-database-plugin`, rotates its Mongo manager credential and issues separate database roles for the BFF - and notifications. Vault Agent renews credentials and restarts child processes - on changes. Existing shared database/messaging paths remain intact. + and notifications. The shared database wrapper replaces credential pools in + process and drains transactions/cursors before revoking retired credentials. + Agent handles non-database secrets. Shared messaging paths remain intact. - Resend is the default. The user explicitly chose to provision its API key and verified sender manually after bootstrap. `make up` leaves healthy infrastructure running and intentionally exits nonzero before product startup while those @@ -622,8 +623,8 @@ The initial image build exposed missing protobuf headers; installing Additional live checks passed: a down/up cycle preserved a Mongo marker and Vault secrets and automatically unsealed Vault; changing/restoring the NATS token -restarted the NATS child; a temporary 10-second/20-second BFF database role proved -replacement credentials restart a healthy BFF (normal TTLs restored afterward). +restarted the NATS child; the initial restart-based database credential lifecycle was superseded by +the uninterrupted pool rotation implementation described below. Temporary BFF/frontend containers served the web bundle and proxied BFF health successfully over HTTPS verified with the generated CA; those test containers were removed. The notifications image rendered its Agent configuration and exited @@ -636,3 +637,42 @@ NATS remain running. No optional security scan or merge was performed. See the [local development guide](local-development.md) for exact provisioning fields, certificate trust, restart behavior, local bootstrap-material handling and verification commands. + +### Uninterrupted Mongo credential rotation + +BFF and notifications now acquire database operation scopes from the shared +wrapper. A replacement is issued at half the effective token/lease lifetime, +validated before publication, and independently revoked after draining. Whole +transactions and cursors retain their original pool. Failed refreshes retry while +the current credential remains valid. Expiry rejects new operations until recovery; +there is no scheduled service restart. Non-database Agent template changes still +restart their child process. This is planned-rotation continuity, not infrastructure +high availability or a guarantee beyond Vault credential expiry. + +The repository and store constructor contract now uses `source.Source` rather +than a concrete Mongo database. Static test integrations use `source.Fixed`. +Application code continues through shared repositories and transaction factories. +The new `make test-dev-rotation` command proves real Vault plugin replacements +under BFF transaction traffic and checks process identity. It restores shortened +role TTLs and removes temporary product containers afterward. + +Validation of the revision: `make build`, `make test`, `make lint-backend`, +`make test-dev-infra` (six setup tests and live checks), and +`make test-dev-rotation` passed. The live rotation check observed three replacements +with an unchanged BFF PID and 1,155 expected invalid-token HTTP responses, zero +failures. Product integration-tag packages also compiled. The race-enabled +rotation suite covers active transactions/cursors, rejected credentials, transient +and extended outages, automatic recovery, rollback and shutdown; issuer tests +cover effective lifetime, credential escaping and failed-issuance cleanup. + +An additional `go test -race -tags integration ./database/internal/repository/... +./database/vault/...` run in `backend/shared` found two existing test issues: the +partial-index test expects nested BSON maps instead of ordered documents, and the +patch timestamp assertion can require a strict increase within the same Mongo +millisecond. The BSON mismatch reproduces against `origin/main`; five isolated baseline +repetitions of the timestamp test pass, consistent with its timing-dependent +assertion. Neither assertion nor the timestamp update logic changes in this task; +the supplementary failures are recorded and are not suppressed. The required +normal test and lint commands pass. No optional security scan or email delivery +was performed. Resend remains intentionally unconfigured; infrastructure stays up +and product startup awaits manual `kv/email` provisioning. diff --git a/docs/local-development.md b/docs/local-development.md index ff108ca..b4b1937 100644 --- a/docs/local-development.md +++ b/docs/local-development.md @@ -108,16 +108,39 @@ and are retained in `kv/bootstrap/mongo` for local provisioning. The BFF role has `readWrite` only on `sxchnge`; notifications has `readWrite` only on `sxchnge_notifications`. Mongo URIs come from dynamic Vault credentials, not static application users. Leases default to one hour with a 24-hour maximum. -Vault Agent renews leases, obtains replacement credentials, and restarts its child -process when credentials change. Apps retain their existing shared database and -messaging connection wrappers. No new direct application database path is added. +The shared database wrapper obtains credentials directly from Vault using the +service's AppRole. At half the effective lifetime (the shorter of the database +lease and owning token TTL), it authenticates independently and prepares a new +Mongo pool. Only a successful primary ping permits publication. New operations +use that pool; active operations, cursors and complete transactions, including +nested stores and commit retries, retain the old pool until they finish. The old +pool then closes and its dedicated token is revoked. Mongo credential replacement +never restarts BFF or notifications. Each generation has a separate token so +cleanup cannot revoke the replacement or Vault Agent's identity. + +Refresh failures keep the current valid pool and retry with bounded backoff. +Operations are bounded by credential expiry. If Vault remains unavailable beyond +that deadline, new database operations fail closed; refresh continues and access +recovers automatically without restarting the service. Administrative revocation, +operations exceeding their lease and infrastructure failures cannot be made +uninterrupted by pool rotation. Shutdown rejects new scopes and drains existing +work. Failed cleanup relies on Vault expiration as a backstop. + +Repositories and transaction factories share `source.Source` rather than retaining +concrete Mongo collections. Low-level consumers must acquire a scope, use its +returned context and release it after all work completes; never retain its client +or context beyond release. `source.Fixed` adapts externally owned test/static +connections. Database credentials are no longer Agent environment templates; +Agent continues to fetch non-database secrets and may restart children when those +secrets change. Each agent authenticates using its own AppRole with a renewable periodic token. Local SecretIDs are reusable for container restarts; each container mounts only its own identity directory under `.local/dev/agents/`. Revoking an AppRole SecretID requires removing its matching local `secret-id` file and rerunning `make up` to issue a replacement. Revoking an active token/lease terminates its credentials; -Agent's retries and the container restart policy recover when Vault is reachable. +The database wrapper retries with a fresh identity; Agent independently retries +its non-database secrets when Vault is reachable. The notification gRPC connection uses a Vault-issued certificate for `notifications` and the BFF validates the Vault CA and hostname. Frontend HTTPS @@ -147,6 +170,7 @@ make logs make down # preserves named volumes and local bootstrap material make up # idempotent provisioning and readiness checks make test-dev-infra # works before Resend provisioning +make test-dev-rotation # real Vault rotations under BFF HTTP traffic make test-dev # requires all six services running ``` @@ -154,6 +178,15 @@ The infrastructure check exercises AppRole access restrictions, real Vault MongoDB credential issuance, per-database access, lease renewal and user deletion on revocation. The complete check also verifies service health and frontend/API HTTPS with the generated CA. These checks never send a real email. +`test-dev-rotation` temporarily sets a 12-second BFF lease, deploys the built BFF +and frontend, and requires three pool replacements with unchanged process ID and +zero unexpected HTTP responses. Its invalid-token requests exercise Mongo-backed +transactions without sending email. The check restores the role TTL and removes +services it started; it recreates BFF/frontend at the beginning, so run it as a +development validation command, not against active work you need to preserve. +Race-enabled backend tests additionally hold a transaction and cursor across a +replacement and exercise failed candidate validation, outage recovery, rollback, +expiry rejection and graceful shutdown. Backend regression tests use fresh `mongo:latest` and `nats:latest` official stable images, pulled before `make test-backend`. Normal validation remains From 1b8800ee73e3368a5b4beb4cad965fc1a548952b Mon Sep 17 00:00:00 2001 From: Stephan D Date: Sat, 12 Sep 2026 23:14:34 +0200 Subject: [PATCH 04/11] Replace Python development tooling with Make and containerized shell --- AGENTS.md | 6 ++ Makefile | 22 +++-- README.md | 3 +- dev/agents.py | 39 -------- dev/bootstrap.py | 105 ---------------------- dev/bootstrap.sh | 143 ++++++++++++++++++++++++++++++ dev/common.sh | 60 +++++++++++++ dev/dispatch.sh | 13 +++ dev/images.py | 72 --------------- dev/images.sh | 24 +++++ dev/images/tools.Dockerfile | 3 + dev/local.py | 47 ---------- dev/mongo-check.js | 32 +++++++ dev/mongo-client.js | 9 ++ dev/rotation.py | 92 ------------------- dev/rotation.sh | 60 +++++++++++++ dev/test.sh | 39 ++++++++ dev/test_local.py | 71 --------------- dev/tools.sh | 25 ++++++ dev/vault.py | 117 ------------------------ dev/vault/bff.hcl | 35 ++++++++ dev/vault/{agent.hcl => nats.hcl} | 11 +++ dev/vault/notifications.hcl | 55 ++++++++++++ dev/verify.py | 72 --------------- dev/verify.sh | 50 +++++++++++ docs/implementation.md | 29 ++++++ docs/local-development.md | 13 ++- 27 files changed, 620 insertions(+), 627 deletions(-) delete mode 100644 dev/agents.py delete mode 100644 dev/bootstrap.py create mode 100644 dev/bootstrap.sh create mode 100644 dev/common.sh create mode 100644 dev/dispatch.sh delete mode 100644 dev/images.py create mode 100644 dev/images.sh create mode 100644 dev/images/tools.Dockerfile delete mode 100644 dev/local.py create mode 100644 dev/mongo-check.js create mode 100644 dev/mongo-client.js delete mode 100644 dev/rotation.py create mode 100644 dev/rotation.sh create mode 100644 dev/test.sh delete mode 100644 dev/test_local.py create mode 100644 dev/tools.sh delete mode 100644 dev/vault.py create mode 100644 dev/vault/bff.hcl rename dev/vault/{agent.hcl => nats.hcl} (55%) create mode 100644 dev/vault/notifications.hcl delete mode 100644 dev/verify.py create mode 100644 dev/verify.sh diff --git a/AGENTS.md b/AGENTS.md index 687ac94..fd3976c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -22,6 +22,12 @@ generated outputs. Document the generation workflow so a clean checkout can build without committed generated code. +## Local development tooling + +Use Make and shell for local development orchestration. Do not introduce Python +scripts or a Python runtime dependency. Keep bootstrap tools inside Docker so +the host needs only Make, a POSIX shell and Docker with Compose/BuildKit. + ## Container versions Use only the latest stable upstream releases for infrastructure, product base diff --git a/Makefile b/Makefile index 170c1d5..f0d88a2 100644 --- a/Makefile +++ b/Makefile @@ -56,12 +56,13 @@ package-notifications-interface: generate-notifications COMPOSE = docker compose --env-file .local/dev/images.env .PHONY: build up down logs ps test-dev build: - python3 dev/images.py + sh dev/tools.sh build + sh dev/tools.sh images $(COMPOSE) pull vault mongo $(COMPOSE) build --pull up: - python3 dev/local.py up + sh dev/tools.sh up down: $(COMPOSE) down @@ -73,14 +74,21 @@ ps: $(COMPOSE) ps test-dev: - python3 -m unittest discover -s dev -p 'test_*.py' - python3 dev/local.py verify + sh dev/tools.sh test + sh dev/tools.sh verify .PHONY: test-dev-infra test-dev-infra: - python3 -m unittest discover -s dev -p 'test_*.py' - python3 dev/local.py verify-infra + sh dev/tools.sh test + sh dev/tools.sh verify-infra .PHONY: test-dev-rotation test-dev-rotation: - python3 dev/rotation.py + sh dev/tools.sh rotation + +.PHONY: email test-dev-tooling +email: + sh dev/tools.sh email + +test-dev-tooling: + sh dev/tools.sh test diff --git a/README.md b/README.md index 468814f..1468cbc 100644 --- a/README.md +++ b/README.md @@ -71,7 +71,8 @@ Output is `frontend/app/build/web/`. To inspect that build locally, from the repository root: ```sh -python3 -m http.server 5123 --bind 127.0.0.1 --directory frontend/app/build/web +docker run --rm --pull always -p 127.0.0.1:5123:80 \ + -v "$PWD/frontend/app/build/web:/usr/share/nginx/html:ro" nginx:stable-alpine ``` Open [the local app](http://127.0.0.1:5123/). The language selector supports EN/RU diff --git a/dev/agents.py b/dev/agents.py deleted file mode 100644 index 5fecdf7..0000000 --- a/dev/agents.py +++ /dev/null @@ -1,39 +0,0 @@ -"""Build per-service Vault Agent configs (no secret values in configs).""" -import json -from pathlib import Path - -BASE = (Path(__file__).parent / 'vault' / 'agent.hcl').read_text() - - -def kv(path, field): - return '{{ with secret "kv/data/' + path + '" }}{{ .Data.data.' + field + ' }}{{ end }}' - - -def config(service): - env = {} - if service == 'nats': - env['NATS_TOKEN'] = kv('nats', 'token') - else: - env['TLS_CA'] = kv('tls', 'ca') - if service == 'bff': - env['SXCHNGE_JWT_SECRET'] = kv('bff', 'jwt') - env['SXCHNGE_NOTIFICATIONS_TOKEN'] = kv('caller', 'token') - else: - env['SXCHNGE_NOTIFICATIONS_PRODUCT_TOKEN'] = kv('caller', 'token') - env['SXCHNGE_NATS_URL'] = ('{{ with secret "kv/data/nats" }}' - 'nats://{{ .Data.data.token }}@nats:4222{{ end }}') - env['TLS_CERT'] = kv('notifications-tls', 'certificate') - env['TLS_KEY'] = kv('notifications-tls', 'private_key') - env['EMAIL_API_KEY'] = kv('email', 'api_key') - # JSON is valid YAML and escapes arbitrary provider configuration. - env['EMAIL_CONFIG'] = ('{{ with secret "kv/data/email" }}' - 'email:\n driver: {{ .Data.data.driver | toJSON }}\n' - ' settings:\n api_key_env: EMAIL_API_KEY\n' - ' sender_address: {{ .Data.data.sender_address | toJSON }}\n' - ' sender_name: {{ .Data.data.sender_name | toJSON }}\n{{ end }}') - result = BASE - for name, template in env.items(): - result += '\nenv_template ' + json.dumps(name) + ' {\n contents = ' + json.dumps(template) + '\n error_on_missing_key = true\n}\n' - command = '/usr/local/bin/start-nats' if service == 'nats' else '/usr/local/bin/start-service' - result += '\nexec {\n command = [' + json.dumps(command) + ']\n restart_on_secret_changes = "always"\n restart_stop_signal = "SIGTERM"\n}\n' - return result diff --git a/dev/bootstrap.py b/dev/bootstrap.py deleted file mode 100644 index a89e719..0000000 --- a/dev/bootstrap.py +++ /dev/null @@ -1,105 +0,0 @@ -"""Idempotent provisioning for the isolated local Compose project.""" -import json -import os -import subprocess -import time - -from agents import config -from vault import STATE, write - -ROOT = STATE.parents[1] -COMPOSE = ['docker', 'compose', '--env-file', str(STATE / 'images.env')] - - -def compose(*args, capture=False, input=None): - result = subprocess.run(COMPOSE + list(args), cwd=ROOT, input=input, text=True, - capture_output=capture) - if result.returncode: - # Bootstrap stdin and child errors may contain credentials. Never echo them. - raise RuntimeError('Compose command failed: ' + ' '.join(args[:3])) - return result.stdout if capture else None - - -def mongo(script, password): - auth = 'const admin = db.getSiblingDB("admin"); admin.auth("local-admin", ' + json.dumps(password) + ');\n' - return compose('exec', '-T', 'mongo', 'mongosh', '--quiet', '--file', '/dev/stdin', capture=True, input=auth + script) - - -def provision_mongo(vault): - secret = vault.ensure('bootstrap/mongo', lambda: {'password': vault.random(), 'keyfile': vault.random(384), 'manager_password': vault.random()}) - write(STATE / 'mongo' / 'password', secret['password']) - write(STATE / 'mongo' / 'keyfile', secret['keyfile']) - compose('up', '-d', 'mongo') - for attempt in range(90): - try: - mongo('try { rs.status(); } catch(e) { if(e.code === 94) rs.initiate({_id:"rs0",members:[{_id:0,host:"mongo:27017"}]}); else throw e; }', secret['password']) - if 'true' in mongo('print(db.hello().isWritablePrimary)', secret['password']): - break - except RuntimeError: - pass - time.sleep(2) - else: - raise RuntimeError('Mongo replica set did not become primary') - if vault.call('database/config/mongo', missing=True) is None: - mongo('if (!admin.getUser("vault-manager")) admin.createUser({user:"vault-manager",pwd:' + json.dumps(secret['manager_password']) + ',roles:[{role:"userAdminAnyDatabase",db:"admin"}]});', secret['password']) - vault.call('database/config/mongo', {'plugin_name': 'mongodb-database-plugin', - 'allowed_roles': ['bff', 'notifications'], - 'connection_url': 'mongodb://{{username}}:{{password}}@mongo:27017/admin?replicaSet=rs0', - 'username': 'vault-manager', 'password': secret['manager_password']}) - vault.call('database/rotate-root/mongo', {}) - for role, database in [('bff', 'sxchnge'), ('notifications', 'sxchnge_notifications')]: - vault.call('database/roles/' + role, {'db_name': 'mongo', - 'creation_statements': json.dumps({'db': 'admin', 'roles': [{'role': 'readWrite', 'db': database}]}), - 'default_ttl': '1h', 'max_ttl': '24h'}) - # Public browser client: an identifier, never a client secret in the web bundle. - mongo('db.getSiblingDB("sxchnge").clients.updateOne({clientId:"sxchnge-local-web"}, {$setOnInsert: {clientName:"Local browser",allowedCIDRs:["0.0.0.0/0","::/0"],allowedScopes:["account"],redirectURIs:["https://localhost:5126"],grantTypes:["password","refresh_token"],tokenEndpointAuthMethod:"none",isRevoked:false}}, {upsert:true});', secret['password']) - - -def provision_agents(vault): - paths = {'bff': ['bff', 'caller', 'tls'], - 'notifications': ['caller', 'tls', 'notifications-tls', 'nats', 'email'], - 'nats': ['nats']} - for service, secrets in paths.items(): - directory = STATE / 'agents' / service - directory.mkdir(parents=True, exist_ok=True, mode=0o755) - os.chmod(directory, 0o755) - policy = ''.join('path "kv/data/' + p + '" { capabilities = ["read"] }\n' for p in secrets) - if service != 'nats': - policy += 'path "database/creds/' + service + '" { capabilities = ["read"] }\n' - vault.call('sys/policies/acl/' + service, {'policy': policy}) - vault.call('auth/approle/role/' + service, {'token_policies': [service], - 'token_period': '1h', 'secret_id_ttl': '0', 'secret_id_num_uses': 0}) - role_id = vault.call('auth/approle/role/' + service + '/role-id')['data']['role_id'] - write(directory / 'role-id', role_id, 0o444) - if not (directory / 'secret-id').exists(): - secret_id = vault.call('auth/approle/role/' + service + '/secret-id', {})['data']['secret_id'] - write(directory / 'secret-id', secret_id, 0o444) - write(directory / 'config.hcl', config(service), 0o444) - - -def up(vault): - if not (STATE / 'images.env').exists(): - raise RuntimeError('Run make build before make up') - (STATE / 'vault').mkdir(parents=True, exist_ok=True, mode=0o755) - compose('up', '-d', 'vault') - vault.initialize() - vault.mounts() - vault.ensure('nats', lambda: {'token': vault.random()}) - vault.ensure('caller', lambda: {'token': vault.random()}) - vault.ensure('bff', lambda: {'jwt': vault.random()}) - vault.ensure('email', lambda: {'driver': 'resend', 'api_key': '', - 'sender_address': 'notifications@localhost.test', 'sender_name': 'SXCHNGE local'}) - vault.certificates() - provision_mongo(vault) - provision_agents(vault) - compose('up', '-d', '--wait', '--wait-timeout', '120', 'nats') - email = vault.kv('email') - key, sender = email.get('api_key'), email.get('sender_address') - if (not isinstance(key, str) or not key.strip() - or not isinstance(sender, str) or not sender.strip() - or sender == 'notifications@localhost.test'): - raise RuntimeError('Vault kv/email needs a Resend api_key and verified sender_address. Run python3 dev/local.py email, then make up.') - compose('up', '-d', '--wait', '--wait-timeout', '180') - compose('exec', '-T', 'frontend', 'nginx', '-s', 'reload', capture=True) - print('Local stack ready: https://localhost:5126; Vault: http://127.0.0.1:18200') - print('Local CA: .local/dev/ca.pem. Email driver: ' + vault.kv('email')['driver']) diff --git a/dev/bootstrap.sh b/dev/bootstrap.sh new file mode 100644 index 0000000..cd6da49 --- /dev/null +++ b/dev/bootstrap.sh @@ -0,0 +1,143 @@ +initialize() { + attempt=0 + until initialized=$(vault sys/init | jq -er '.initialized|tostring'); do + attempt=$((attempt+1)); [ "$attempt" -lt 90 ] || fail 'Vault did not start'; sleep 1 + done + if [ "$initialized" = false ]; then + [ ! -e "$STATE/admin/init.json" ] || fail 'Vault data missing but bootstrap material exists; restore matching volume or reset both' + printf '{"secret_shares":1,"secret_threshold":1}' | vault sys/init POST | write "$STATE/admin/init.json" + fi + existing + jq -r '.keys_base64[0]' "$STATE/admin/init.json" | write "$STATE/vault/unseal" 444 + attempt=0 + until vault sys/seal-status | jq -e '.sealed==false' >/dev/null; do + attempt=$((attempt+1)); [ "$attempt" -lt 60 ] || fail 'Vault did not unseal'; sleep 1 + done +} +mounts() { + vault sys/mounts > "$WORK/mounts" + for name in kv database pki; do + if ! jq -e --arg key "$name/" 'has($key)' < "$WORK/mounts" >/dev/null; then + if ! jq -e --arg key "$name/" '.data|has($key)' < "$WORK/mounts" >/dev/null; then + jq -n --arg name "$name" '{type:$name} + (if $name=="kv" then {options:{version:"2"}} else {} end)' | vault "sys/mounts/$name" POST >/dev/null + fi + fi + done + if ! vault sys/auth | jq -e '.data|has("approle/")' >/dev/null; then + printf '{"type":"approle"}' | vault sys/auth/approle POST >/dev/null + fi +} +certificates() { + printf '{"max_lease_ttl":"87600h"}' | vault sys/mounts/pki/tune POST >/dev/null + ca=$(kv tls) + if [ "$ca" = null ]; then + printf '{"common_name":"SXCHNGE local development CA","ttl":"87600h"}' | vault pki/root/generate/internal POST | jq '{ca:.data.certificate}' | put tls + fi + printf '{"allowed_domains":["localhost","notifications"],"allow_bare_domains":true,"allow_ip_sans":true,"max_ttl":"720h"}' | vault pki/roles/local POST >/dev/null + for name in notifications-tls frontend-tls; do + host=notifications; [ "$name" != frontend-tls ] || host=localhost + kv "$name" > "$WORK/cert" + if ! jq -e '.!=null and .expiration>(now+604800)' "$WORK/cert" >/dev/null; then + jq -n --arg host "$host" '{common_name:$host,ip_sans:"127.0.0.1",ttl:"720h"}' | vault pki/issue/local POST | jq '.data' > "$WORK/cert" + put "$name" < "$WORK/cert" + fi + if [ "$name" = frontend-tls ]; then + jq -r '.certificate' "$WORK/cert" | write "$STATE/frontend/server.crt" + jq -r '.private_key' "$WORK/cert" | write "$STATE/frontend/server.key" + fi + done + kv tls | jq -r '.ca' | write "$STATE/ca.pem" 644 +} +provision_mongo() { + if [ "$(kv bootstrap/mongo)" = null ]; then + random > "$WORK/password"; random 384 > "$WORK/keyfile"; random > "$WORK/manager" + jq -n --rawfile password "$WORK/password" --rawfile keyfile "$WORK/keyfile" --rawfile manager "$WORK/manager" '{password:($password|rtrimstr("\n")),keyfile:($keyfile|rtrimstr("\n")),manager_password:($manager|rtrimstr("\n"))}' | put bootstrap/mongo + fi + kv bootstrap/mongo > "$WORK/mongo-secret" + jq -r '.password' "$WORK/mongo-secret" | write "$STATE/mongo/password" + jq -r '.keyfile' "$WORK/mongo-secret" | write "$STATE/mongo/keyfile" + compose up -d mongo + admin_auth > "$WORK/admin.js" + attempt=0 + while :; do + if { cat "$WORK/admin.js"; printf 'try {rs.status()} catch(e) {if(e.code===94) rs.initiate({_id:"rs0",members:[{_id:0,host:"mongo:27017"}]}); else throw e;} print(db.hello().isWritablePrimary);'; } | mongo > "$WORK/primary" && grep -q true "$WORK/primary"; then break; fi + attempt=$((attempt+1)); [ "$attempt" -lt 90 ] || fail 'Mongo replica set did not become primary'; sleep 2 + done + if [ "$(vault database/config/mongo GET yes)" = null ]; then + { cat "$WORK/admin.js"; jq -r '"if(!admin.getUser(\"vault-manager\")) admin.createUser({user:\"vault-manager\",pwd:"+(.manager_password|tojson)+",roles:[{role:\"userAdminAnyDatabase\",db:\"admin\"}]});"' "$WORK/mongo-secret"; } | mongo >/dev/null || fail 'Mongo manager provisioning failed' + jq '{plugin_name:"mongodb-database-plugin",allowed_roles:["bff","notifications"],connection_url:"mongodb://{{username}}:{{password}}@mongo:27017/admin?replicaSet=rs0",username:"vault-manager",password:.manager_password}' "$WORK/mongo-secret" | vault database/config/mongo POST >/dev/null + printf '{}' | vault database/rotate-root/mongo POST >/dev/null + fi + for role in bff notifications; do + database=sxchnge; [ "$role" != notifications ] || database=sxchnge_notifications + jq -n --arg database "$database" '{db_name:"mongo",creation_statements:({db:"admin",roles:[{role:"readWrite",db:$database}]}|tojson),default_ttl:"1h",max_ttl:"24h"}' | vault "database/roles/$role" POST >/dev/null + done + { cat "$WORK/admin.js"; cat dev/mongo-client.js; } | mongo >/dev/null || fail 'Browser client provisioning failed' +} +provision_agents() { + for service in bff notifications nats; do + directory="$STATE/agents/$service" + mkdir -p "$directory"; chmod 755 "$directory" + case "$service" in + bff) paths='bff caller tls';; + notifications) paths='caller tls notifications-tls nats email';; + nats) paths=nats;; + esac + : > "$WORK/policy" + for path in $paths; do printf 'path "kv/data/%s" { capabilities = ["read"] }\n' "$path" >> "$WORK/policy"; done + if [ "$service" != nats ]; then printf 'path "database/creds/%s" { capabilities = ["read"] }\n' "$service" >> "$WORK/policy"; fi + jq -n --rawfile policy "$WORK/policy" '{policy:$policy}' | vault "sys/policies/acl/$service" POST >/dev/null + jq -n --arg service "$service" '{token_policies:[$service],token_period:"1h",secret_id_ttl:"0",secret_id_num_uses:0}' | vault "auth/approle/role/$service" POST >/dev/null + vault "auth/approle/role/$service/role-id" | jq -r '.data.role_id' | write "$directory/role-id" 444 + if [ ! -f "$directory/secret-id" ]; then + printf '{}' | vault "auth/approle/role/$service/secret-id" POST | jq -r '.data.secret_id' | write "$directory/secret-id" 444 + fi + cat "dev/vault/$service.hcl" | write "$directory/config.hcl" 444 + done +} +up() { + [ -f "$STATE/images.env" ] || fail 'Run make build before make up' + exec 9> "$STATE/bootstrap.lock" + flock -n 9 || fail 'Another local bootstrap is running' + mkdir -p "$STATE/vault"; chmod 755 "$STATE/vault" + compose up -d vault + # On first bootstrap Compose creates the network after this tool container starts. + if ! docker inspect "$HOSTNAME" --format '{{json .NetworkSettings.Networks}}' | jq -e 'has("sxchnge-local_default")' >/dev/null; then + docker network connect sxchnge-local_default "$HOSTNAME" + fi + initialize + mounts + for path in nats caller bff; do + if [ "$(kv "$path")" = null ]; then + field=token; [ "$path" != bff ] || field=jwt + random | jq -Rs --arg field "$field" '{($field):rtrimstr("\n")}' | put "$path" + fi + done + printf '{"driver":"resend","api_key":"","sender_address":"notifications@localhost.test","sender_name":"SXCHNGE local"}' | ensure email + certificates + provision_mongo + provision_agents + compose up -d --wait --wait-timeout 120 nats + kv email | email_valid || fail 'Vault kv/email needs a Resend api_key and verified sender_address. Run make email, then make up.' + compose up -d --wait --wait-timeout 180 + compose exec -T frontend nginx -s reload + echo 'Local stack ready: https://localhost:5126; Vault: http://127.0.0.1:18200' +} +email() { + existing + printf 'Verified Resend sender address: ' >&2 + IFS= read -r address + printf 'Resend API key (hidden): ' >&2 + tty_state=$(stty -g) + trap 'stty "$tty_state"; cleanup' EXIT + stty -echo + IFS= read -r key + stty "$tty_state" + printf '\n' >&2 + [ -n "$key" ] || fail 'A key is required' + case "$address" in *@*) ;; *) fail 'A sender email address is required';; esac + printf '%s' "$key" > "$WORK/email-key" + printf '%s' "$address" > "$WORK/email-address" + jq -n --rawfile key "$WORK/email-key" --rawfile address "$WORK/email-address" '{driver:"resend",api_key:$key,sender_address:$address,sender_name:"SXCHNGE"}' | put email + echo 'Email configuration saved in Vault. Run make up.' +} diff --git a/dev/common.sh b/dev/common.sh new file mode 100644 index 0000000..45ef441 --- /dev/null +++ b/dev/common.sh @@ -0,0 +1,60 @@ +# Shared container-side primitives. Secret JSON travels through stdin or private files. +set -eu +set -o pipefail +umask 077 +STATE="$PWD/.local/dev" +TOOL_STATE=$STATE +mkdir -p "$STATE" +WORK=$(mktemp -d) +cleanup() { + rm -rf "$WORK" + chown -R "${LOCAL_UID:-0}:${LOCAL_GID:-0}" "$TOOL_STATE" +} +trap cleanup EXIT +trap 'exit 130' INT +trap 'exit 143' TERM +fail() { echo "$*" >&2; exit 1; } +compose() { docker compose --env-file "$STATE/images.env" "$@"; } +write() { + target=$1; mode=${2:-600} + mkdir -p "$(dirname "$target")" + cat > "$target.tmp" + chmod "$mode" "$target.tmp" + mv "$target.tmp" "$target" +} +# Token is never placed in curl's arguments, errors or response logs. +vault() ( + path=$1; method=${2:-GET}; missing=${3:-no} + header=$(mktemp "$WORK/header.XXXXXX") + response=$(mktemp "$WORK/response.XXXXXX") + trap 'rm -f "$header" "$response"' EXIT + printf 'Content-Type: application/json\n' > "$header" + if [ -n "${VAULT_TOKEN:-}" ]; then printf 'X-Vault-Token: %s\n' "$VAULT_TOKEN" >> "$header"; fi + if [ "$method" = GET ]; then + code=$(curl -s --connect-timeout 5 --max-time 30 -H "@$header" -o "$response" -w '%{http_code}' "http://vault:8200/v1/$path") || exit 1 + else + code=$(curl -s --connect-timeout 5 --max-time 30 -H "@$header" -X "$method" --data-binary @- -o "$response" -w '%{http_code}' "http://vault:8200/v1/$path") || exit 1 + fi + case "$code" in + 200|204) if [ -s "$response" ]; then cat "$response"; else printf '{}'; fi;; + 404) if [ "$missing" = yes ]; then printf 'null'; else echo "Vault $path returned HTTP $code" >&2; exit 1; fi;; + *) echo "Vault $path returned HTTP $code" >&2; exit 1;; + esac +) +existing() { + [ -f "$STATE/admin/init.json" ] || fail 'Missing .local/dev/admin/init.json; restore matching bootstrap material' + VAULT_TOKEN=$(jq -er '.root_token' "$STATE/admin/init.json") + export VAULT_TOKEN +} +kv() { vault "kv/data/$1" GET yes | jq 'if . == null then null else .data.data end'; } +put() { jq '{data:.}' | vault "kv/data/$1" POST >/dev/null; } +random() { printf '{"bytes":%s,"format":"hex"}' "${1:-32}" | vault sys/tools/random POST | jq -er '.data.random_bytes'; } +ensure() ( + path=$1 + value=$(kv "$path") + if [ "$value" = null ]; then put "$path"; fi +) +# Never forward mongosh errors: authentication/command errors may include secrets. +mongo() { compose exec -T mongo mongosh --quiet --file /dev/stdin > "$WORK/mongo-result" 2> "$WORK/mongo-error" || return 1; cat "$WORK/mongo-result"; } +admin_auth() { kv bootstrap/mongo | jq -r '"const admin=db.getSiblingDB(\"admin\"); admin.auth(\"local-admin\","+(.password|tojson)+");"'; } +email_valid() { jq -e '(.api_key|type=="string") and (.api_key|test("\\S")) and (.sender_address|type=="string") and (.sender_address|test("\\S")) and .sender_address!="notifications@localhost.test"' >/dev/null; } diff --git a/dev/dispatch.sh b/dev/dispatch.sh new file mode 100644 index 0000000..eba2e24 --- /dev/null +++ b/dev/dispatch.sh @@ -0,0 +1,13 @@ +#!/bin/sh +set -eu +set -o pipefail +. dev/common.sh +case "${1:-}" in + images) . dev/images.sh; images;; + up) . dev/bootstrap.sh; up;; + email) . dev/bootstrap.sh; email;; + verify|verify-infra) . dev/verify.sh; existing; infrastructure; if [ "$1" = verify ]; then product; fi;; + rotation) . dev/verify.sh; . dev/rotation.sh; existing; rotation;; + test) sh dev/test.sh;; + *) fail 'Unknown local tooling command';; +esac diff --git a/dev/images.py b/dev/images.py deleted file mode 100644 index f884728..0000000 --- a/dev/images.py +++ /dev/null @@ -1,72 +0,0 @@ -#!/usr/bin/env python3 -"""Resolve stable upstream releases on every build; pin their registry digests.""" -import concurrent.futures -import json -import os -from pathlib import Path -import re -import subprocess -import urllib.request - -ROOT = Path(__file__).resolve().parents[1] -STATE = ROOT / '.local' / 'dev' - - -def get(url): - with urllib.request.urlopen(url, timeout=45) as response: - return json.load(response) - - -def image(repo, alias): - url = f'https://hub.docker.com/v2/repositories/{repo}/tags' - selected = get(f'{url}/{alias}') - digest = selected['digest'] - if not re.fullmatch(r'sha256:[a-f0-9]{64}', digest): - raise ValueError('Registry returned an invalid digest') - # Find the exact numeric tag represented by the maintained stable alias. - candidates = get(f'{url}?page_size=100')['results'] - names = [x['name'] for x in candidates if x.get('digest') == digest - and re.fullmatch(r'\d+\.\d+(?:\.\d+)?(?:-alpine)?', x['name'])] - if not names: - raise ValueError(f'No stable version tag for {repo}:{alias}') - tag = max(names, key=lambda x: tuple(map(int, x.split('-')[0].split('.')))) - return f'{repo}:{tag}@{digest}' - - -def flutter(): - commit = get('https://api.github.com/repos/flutter/flutter/commits/stable')['sha'] - tags = subprocess.check_output(['git', 'ls-remote', '--tags', - 'https://github.com/flutter/flutter.git'], text=True) - names = [line.split('refs/tags/')[1] for line in tags.splitlines() - if line.startswith(commit + '\trefs/tags/') - and re.fullmatch(r'\d+\.\d+\.\d+', line.split('refs/tags/')[1])] - if not names: - raise ValueError('Flutter stable does not point at a stable release tag') - return max(names, key=lambda x: tuple(map(int, x.split('.')))), commit - - -def main(): - go = get('https://go.dev/dl/?mode=json')[0] - if not go['stable'] or not re.fullmatch(r'go\d+\.\d+\.\d+', go['version']): - raise ValueError('Go release is not stable') - specs = {'VAULT_IMAGE': ('hashicorp/vault', 'latest'), - 'MONGO_IMAGE': ('library/mongo', 'latest'), - 'NATS_IMAGE': ('library/nats', 'latest'), - 'NGINX_IMAGE': ('library/nginx', 'stable-alpine'), - 'GO_IMAGE': ('library/golang', go['version'][2:] + '-alpine'), - 'UBUNTU_IMAGE': ('library/ubuntu', 'latest')} - with concurrent.futures.ThreadPoolExecutor() as pool: - futures = {key: pool.submit(image, *spec) for key, spec in specs.items()} - refs = {key: future.result() for key, future in futures.items()} - refs['FLUTTER_VERSION'], refs['FLUTTER_COMMIT'] = flutter() - STATE.mkdir(parents=True, exist_ok=True, mode=0o700) - target = STATE / 'images.env' - temporary = target.with_suffix('.tmp') - temporary.write_text(''.join(f'{key}={value}\n' for key, value in refs.items())) - os.replace(temporary, target) - for key, value in refs.items(): - print(f'{key}={value}') - - -if __name__ == '__main__': - main() diff --git a/dev/images.sh b/dev/images.sh new file mode 100644 index 0000000..4238fbb --- /dev/null +++ b/dev/images.sh @@ -0,0 +1,24 @@ +# Resolve maintained stable aliases to exact numeric tags and digests. +get() { curl -fsS --connect-timeout 10 --max-time 60 "$1"; } +image() ( + repo=$1; alias=$2 + digest=$(get "https://hub.docker.com/v2/repositories/$repo/tags/$alias" | jq -er '.digest | select(test("^sha256:[a-f0-9]{64}$"))') || exit 1 + tag=$(get "https://hub.docker.com/v2/repositories/$repo/tags?page_size=100" | jq -er --arg digest "$digest" ' + [.results[] | select(.digest==$digest) | .name | select(test("^[0-9]+\\.[0-9]+(\\.[0-9]+)?(-alpine)?$"))] + | sort_by(split("-")[0] | split(".") | map(tonumber)) | last // error("No stable numeric tag matches digest")') || exit 1 + printf '%s:%s@%s\n' "$repo" "$tag" "$digest" +) +images() ( + go=$(get 'https://go.dev/dl/?mode=json' | jq -er '.[0] | select(.stable) | .version | select(test("^go[0-9]+\\.[0-9]+\\.[0-9]+$"))') || exit 1 + for spec in VAULT_IMAGE:hashicorp/vault:latest MONGO_IMAGE:library/mongo:latest NATS_IMAGE:library/nats:latest NGINX_IMAGE:library/nginx:stable-alpine "GO_IMAGE:library/golang:${go#go}-alpine" UBUNTU_IMAGE:library/ubuntu:latest; do + key=${spec%%:*}; rest=${spec#*:}; repo=${rest%:*}; alias=${rest##*:} + ref=$(image "$repo" "$alias") || exit 1 + printf '%s=%s\n' "$key" "$ref" >> "$WORK/images.env" + done + commit=$(get https://api.github.com/repos/flutter/flutter/commits/stable | jq -er '.sha | select(test("^[a-f0-9]{40}$"))') || exit 1 + git ls-remote --tags https://github.com/flutter/flutter.git > "$WORK/flutter-tags" || exit 1 + tag=$(jq -Rrs --arg commit "$commit" '[split("\n")[] | split("\t") | select(.[0]==$commit) | .[1] | sub("^refs/tags/";"") | select(test("^[0-9]+\\.[0-9]+\\.[0-9]+$"))] | sort_by(split(".")|map(tonumber)) | last // error("Flutter stable has no stable release tag")' "$WORK/flutter-tags") || exit 1 + printf 'FLUTTER_VERSION=%s\nFLUTTER_COMMIT=%s\n' "$tag" "$commit" >> "$WORK/images.env" + cat "$WORK/images.env" | write "$STATE/images.env" + cat "$STATE/images.env" +) diff --git a/dev/images/tools.Dockerfile b/dev/images/tools.Dockerfile new file mode 100644 index 0000000..39e2ea9 --- /dev/null +++ b/dev/images/tools.Dockerfile @@ -0,0 +1,3 @@ +ARG DOCKER_IMAGE +FROM ${DOCKER_IMAGE} +RUN apk add --no-cache curl jq git openssl util-linux shellcheck diff --git a/dev/local.py b/dev/local.py deleted file mode 100644 index 17d3a48..0000000 --- a/dev/local.py +++ /dev/null @@ -1,47 +0,0 @@ -#!/usr/bin/env python3 -"""Local stack lifecycle. Requires Python 3 and Docker Compose.""" -import sys -from bootstrap import up -from vault import Vault - - -def main(): - vault = Vault() - if len(sys.argv) == 2 and sys.argv[1] == 'up': - import fcntl - from vault import STATE - STATE.mkdir(parents=True, exist_ok=True, mode=0o700) - with (STATE / 'bootstrap.lock').open('w') as lock: - try: - fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB) - except BlockingIOError: - raise RuntimeError('Another local bootstrap is running') from None - up(vault) - elif len(sys.argv) == 2 and sys.argv[1] == 'verify': - from verify import verify - vault.existing() - verify(vault) - elif len(sys.argv) == 2 and sys.argv[1] == 'verify-infra': - from verify import infrastructure - vault.existing() - infrastructure(vault) - elif len(sys.argv) == 2 and sys.argv[1] == 'email': - import getpass - vault.existing() - address = input('Verified Resend sender address: ').strip() - key = getpass.getpass('Resend API key (hidden): ').strip() - if not key or '@' not in address: - raise RuntimeError('A key and sender address are required') - vault.put('email', {'driver': 'resend', 'api_key': key, - 'sender_address': address, 'sender_name': 'SXCHNGE'}) - print('Email configuration saved in Vault. Run make up.') - else: - raise RuntimeError('Usage: python3 dev/local.py up|verify|verify-infra|email') - - -if __name__ == '__main__': - try: - main() - except (RuntimeError, OSError) as error: - print('Local development setup failed: ' + str(error), file=sys.stderr) - sys.exit(1) diff --git a/dev/mongo-check.js b/dev/mongo-check.js new file mode 100644 index 0000000..a48b60e --- /dev/null +++ b/dev/mongo-check.js @@ -0,0 +1,32 @@ +// Run using a leased application identity, never the bootstrap administrator. +if (db.hello().setName !== "rs0" || !db.hello().isWritablePrimary) { + throw new Error("replica set is not primary"); +} +const own = db.getSiblingDB(database); +const name = "local_check_" + new ObjectId().toHexString(); +const session = db.getMongo().startSession(); +try { + own.createCollection(name); + const collection = session.getDatabase(database).getCollection(name); + session.startTransaction(); + collection.insertMany([{_id: "first"}, {_id: "second"}]); + session.commitTransaction(); + if (own.getCollection(name).countDocuments({}) !== 2) throw new Error("commit failed"); + session.startTransaction(); + collection.deleteOne({_id: "first"}); + collection.insertOne({_id: "aborted"}); + session.abortTransaction(); + if (own.getCollection(name).countDocuments({}) !== 2 || + !own.getCollection(name).findOne({_id: "first"}) || + own.getCollection(name).findOne({_id: "aborted"})) throw new Error("rollback failed"); + let denied = false; + try { + const result = db.getSiblingDB(forbidden).runCommand({find: name, filter: {}}); + denied = result.code === 13; + } catch (error) { denied = error.code === 13; } + if (!denied) throw new Error("cross-database access allowed"); + print("scope-ok"); +} finally { + session.endSession(); + own.getCollection(name).drop(); +} diff --git a/dev/mongo-client.js b/dev/mongo-client.js new file mode 100644 index 0000000..4e7922a --- /dev/null +++ b/dev/mongo-client.js @@ -0,0 +1,9 @@ +// Public browser client: this identifier is not a client secret. +db.getSiblingDB("sxchnge").clients.updateOne( + {clientId: "sxchnge-local-web"}, + {$setOnInsert: { + clientName: "Local browser", allowedCIDRs: ["0.0.0.0/0", "::/0"], + allowedScopes: ["account"], redirectURIs: ["https://localhost:5126"], + grantTypes: ["password", "refresh_token"], tokenEndpointAuthMethod: "none", isRevoked: false + }}, {upsert: true} +); diff --git a/dev/rotation.py b/dev/rotation.py deleted file mode 100644 index 8a37c46..0000000 --- a/dev/rotation.py +++ /dev/null @@ -1,92 +0,0 @@ -"""Exercise real Vault database rotation under BFF HTTP transaction traffic. - -Run after make build/bootstrap. No Resend credential or live email is needed. -Temporarily shortens the BFF database role TTL, restores it on every exit, and -preserves whether BFF/frontend were running before this check. -""" -import concurrent.futures -import json -import ssl -import threading -import time -import urllib.error -import urllib.request -import uuid - -from bootstrap import compose -from vault import STATE, Vault - - -def verify_rotation(): - vault = Vault() - vault.existing() - role = 'database/roles/bff' - original = vault.call(role)['data'] - running = {json.loads(line)['Service'] for line in compose('ps', '--format', 'json', capture=True).splitlines()} - stop = threading.Event() - successes = [] - failures = [] - lock = threading.Lock() - context = ssl.create_default_context(cafile=str(STATE / 'ca.pem')) - - def requests(): - while not stop.is_set(): - # Exactly 43 characters invokes the activation transaction and token - # lookup, then returns the expected invalid-token response. No user - # or notification is created and the test never sends an email. - token = uuid.uuid4().hex + uuid.uuid4().hex[:11] - request = urllib.request.Request('https://localhost:5126/api/v1/accounts/verification/confirm', - data=json.dumps({'token': token}).encode(), headers={ - 'Content-Type': 'application/json', 'Origin': 'https://localhost:5126', - 'X-Auth-Transport': 'cookie', 'X-CSRF-Protection': '1'}) - try: - with urllib.request.urlopen(request, context=context, timeout=5) as response: - code = response.status - body = response.read() - except urllib.error.HTTPError as error: - code = error.code - body = error.read() - except Exception: - code = 0 - body = b'' - with lock: - (successes if code == 400 and b'Verification is invalid or already used' in body else failures).append(code) - stop.wait(0.03) - - try: - vault.call(role, dict(original, default_ttl='12s', max_ttl='24s')) - compose('up', '-d', '--no-deps', '--force-recreate', '--wait', '--wait-timeout', '90', 'bff', 'frontend') - process = compose('exec', '-T', 'bff', 'pidof', 'service', capture=True).strip() - with concurrent.futures.ThreadPoolExecutor(max_workers=4) as pool: - workers = [pool.submit(requests) for _ in range(4)] - try: - deadline = time.monotonic() + 40 - while time.monotonic() < deadline: - time.sleep(1) - current = compose('exec', '-T', 'bff', 'pidof', 'service', capture=True).strip() - if current != process: - raise RuntimeError('BFF process restarted during credential rotation') - logs = compose('logs', '--no-color', 'bff', capture=True) - if logs.count('Database pool replaced without restarting service') >= 3: - break - else: - raise RuntimeError('Three database pool rotations were not observed') - finally: - stop.set() - for worker in workers: - worker.result() - if failures or len(successes) < 100: - raise RuntimeError(f'Rotation traffic check: {len(successes)} expected responses, {len(failures)} failures') - print(f'PASS: three real Vault rotations, unchanged BFF PID, {len(successes)} transaction-backed HTTP requests, zero failures') - finally: - stop.set() - vault.call(role, original) - newly_started = [name for name in ['frontend', 'bff'] if name not in running] - if newly_started: - compose('stop', *newly_started) - compose('rm', '-f', *newly_started) - # Existing services stay online; their next refresh picks up normal TTLs. - - -if __name__ == '__main__': - verify_rotation() diff --git a/dev/rotation.sh b/dev/rotation.sh new file mode 100644 index 0000000..9f7ce2f --- /dev/null +++ b/dev/rotation.sh @@ -0,0 +1,60 @@ +# Temporarily deploy BFF/frontend and shorten the role lifetime. Always restore it. +rotation() ( + exec 9> "$STATE/bootstrap.lock" + flock -n 9 || fail 'Another local bootstrap or rotation check is running' + vault database/roles/bff | jq '.data' > "$WORK/original-role" + compose ps --services --status running > "$WORK/running" + workers='' + restore() { + touch "$WORK/stop" + for pid in $workers; do wait "$pid" || true; done + restore_failed=0 + vault database/roles/bff POST < "$WORK/original-role" >/dev/null || { echo 'ERROR: restore BFF role TTL manually' >&2; restore_failed=1; } + for service in frontend bff; do + if ! grep -qx "$service" "$WORK/running"; then compose stop "$service"; compose rm -f "$service"; fi + done + return "$restore_failed" + } + trap restore EXIT + trap 'exit 130' INT + trap 'exit 143' TERM + jq '.+{default_ttl:"12s",max_ttl:"24s"}' "$WORK/original-role" | vault database/roles/bff POST >/dev/null + compose up -d --no-deps --force-recreate --wait --wait-timeout 90 bff frontend + process=$(compose exec -T bff pidof service) + for worker in 1 2 3 4; do + ( + while [ ! -f "$WORK/stop" ]; do + # 43 characters passes syntax validation, executes a Mongo transaction/read, + # and returns invalid-token without creating users or sending email. + token=$(openssl rand -hex 22 | cut -c1-43) + code=$(printf '{"token":"%s"}' "$token" | https -o "$WORK/body-$worker" -w '%{http_code}' \ + -H 'Content-Type: application/json' -H 'Origin: https://localhost:5126' \ + -H 'X-Auth-Transport: cookie' -H 'X-CSRF-Protection: 1' --data-binary @- \ + https://localhost:5126/api/v1/accounts/verification/confirm) || code=0 + if [ "$code" = 400 ] && grep -q 'Verification is invalid or already used' "$WORK/body-$worker"; then + echo ok >> "$WORK/results-$worker" + else echo fail >> "$WORK/results-$worker"; fi + sleep 0.03 + done + ) & + workers="$workers $!" + done + count=0; attempt=0 + while [ "$attempt" -lt 40 ]; do + sleep 1 + [ "$(compose exec -T bff pidof service)" = "$process" ] || fail 'BFF restarted during rotation' + compose logs --no-color bff > "$WORK/logs" + count=$(grep -c 'Database pool replaced without restarting service' "$WORK/logs" || true) + [ "$count" -lt 3 ] || break + attempt=$((attempt+1)) + done + touch "$WORK/stop" + for pid in $workers; do wait "$pid"; done + workers='' + [ "$count" -ge 3 ] || fail 'Three pool rotations were not observed' + cat "$WORK"/results-* > "$WORK/results" + ! grep -q fail "$WORK/results" || fail 'Unexpected HTTP responses during rotation' + count=$(wc -l < "$WORK/results") + [ "$count" -ge 100 ] || fail 'Insufficient rotation traffic' + echo "PASS: three real Vault rotations, unchanged BFF PID, $count transaction-backed HTTP requests, zero failures" +) diff --git a/dev/test.sh b/dev/test.sh new file mode 100644 index 0000000..c0307ed --- /dev/null +++ b/dev/test.sh @@ -0,0 +1,39 @@ +#!/bin/sh +set -eu +set -o pipefail +# Shell syntax and static analysis cover the actual container shell scripts. +shellcheck -s ash -x -S warning dev/dispatch.sh dev/test.sh +shellcheck -x -S warning dev/tools.sh dev/start-*.sh +. dev/common.sh +. dev/images.sh +. dev/bootstrap.sh +# Offline fixtures exercise stable selection, reject prereleases, and preserve pins. +get() { + case "$1" in + */tags/latest) printf '{"digest":"sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}';; + *page_size*) cat <<'JSON' +{"results":[{"name":"2.1.0","digest":"sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"},{"name":"2.2.0-rc1","digest":"sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}]} +JSON + ;; + *) return 1;; + esac +} +[ "$(image hashicorp/vault latest)" = 'hashicorp/vault:2.1.0@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' ] || fail 'Stable release selection failed' +get() { printf '{"digest":"sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa","results":[{"name":"3.0.0-rc1","digest":"sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}]}'; } +if image hashicorp/vault latest >/dev/null 2>&1; then fail 'Prerelease accepted'; fi +# Isolate state from the developer stack during failure tests. +STATE="$WORK/state"; mkdir "$STATE"; printf 'previous build' > "$STATE/images.env" +get() { return 1; } +if images >/dev/null 2>&1; then fail 'Offline resolution succeeded'; fi +[ "$(cat "$STATE/images.env")" = 'previous build' ] || fail 'Failed resolution replaced pins' +if printf '{"api_key":"","sender_address":"notifications@localhost.test"}' | email_valid; then fail 'Empty email accepted'; fi +printf '{"api_key":"fixture-only","sender_address":"sender@example.test"}' | email_valid +# Initialized Vault without matching local keys must fail without initializing it again. +vault() { printf '{"initialized":true}'; } +if (initialize) > "$WORK/initialize-error" 2>&1; then fail 'Missing local keys accepted'; fi +grep -q 'Missing .local' "$WORK/initialize-error" +for service in bff notifications; do + if grep -Eq 'database/creds/|SXCHNGE_MONGO_URI' "dev/vault/$service.hcl"; then fail 'Database credentials trigger Agent restart'; fi +done +echo 'PASS: shell analysis, stable selection, failed-resolution preservation, email validation, missing-key protection and Agent contract' +if command -v python >/dev/null 2>&1 || command -v python3 >/dev/null 2>&1; then fail 'Tool image unexpectedly contains Python'; fi diff --git a/dev/test_local.py b/dev/test_local.py deleted file mode 100644 index 3a60194..0000000 --- a/dev/test_local.py +++ /dev/null @@ -1,71 +0,0 @@ -"""Focused regression tests for release selection and bootstrap failure paths.""" -from pathlib import Path -import tempfile -import unittest -from unittest.mock import patch - -import agents -import bootstrap -import images -from vault import Vault - - -class Releases(unittest.TestCase): - def test_selects_exact_stable_digest_not_prerelease(self): - digest = 'sha256:' + 'a' * 64 - versions = [{'name': name, 'digest': digest} for name in ['latest', '2', '2.1', '2.1.0', '2.2.0-rc1']] - with patch.object(images, 'get', side_effect=[{'digest': digest}, {'results': versions}]): - self.assertEqual(images.image('hashicorp/vault', 'latest'), 'hashicorp/vault:2.1.0@' + digest) - - def test_missing_stable_mapping_fails(self): - digest = 'sha256:' + 'a' * 64 - with patch.object(images, 'get', side_effect=[{'digest': digest}, {'results': [{'name': '3.0.0-rc1', 'digest': digest}]}]): - with self.assertRaises(ValueError): - images.image('hashicorp/vault', 'latest') - - def test_upstream_failure_does_not_replace_existing_pins(self): - with tempfile.TemporaryDirectory() as temporary: - state = Path(temporary) - target = state / 'images.env' - target.write_text('previous build') - with patch.object(images, 'STATE', state), patch.object(images, 'get', side_effect=OSError('offline')): - with self.assertRaises(OSError): - images.main() - self.assertEqual(target.read_text(), 'previous build') - - -class AgentContract(unittest.TestCase): - def test_database_lease_does_not_trigger_process_restart(self): - for service in ['bff', 'notifications']: - config = agents.config(service) - self.assertNotIn('database/creds/', config) - self.assertNotIn('SXCHNGE_MONGO_URI', config) - - -class Bootstrap(unittest.TestCase): - def test_missing_resend_credentials_stops_before_product_start(self): - with tempfile.TemporaryDirectory() as temporary: - state = Path(temporary) - (state / 'images.env').touch() - from unittest.mock import Mock - vault = Mock() - vault.kv.return_value = {'driver': 'resend', 'api_key': '', 'sender_address': 'notifications@localhost.test'} - with patch.object(bootstrap, 'STATE', state), patch.object(bootstrap, 'compose') as compose, \ - patch.object(bootstrap, 'provision_mongo'), patch.object(bootstrap, 'provision_agents'): - with self.assertRaisesRegex(RuntimeError, 'Resend api_key'): - bootstrap.up(vault) - self.assertEqual(compose.call_args_list[-1].args[-1], 'nats') - email_seed = next(c for c in vault.ensure.call_args_list if c.args[0] == 'email').args[1]() - self.assertEqual(email_seed['driver'], 'resend') - self.assertEqual(email_seed['api_key'], '') - - def test_initialized_vault_without_local_keys_fails(self): - with tempfile.TemporaryDirectory() as temporary: - with patch('vault.STATE', Path(temporary)), patch.object(Vault, 'call', return_value={'initialized': True}): - with self.assertRaisesRegex(RuntimeError, 'Missing .local'): - Vault().initialize() - - - -if __name__ == '__main__': - unittest.main() diff --git a/dev/tools.sh b/dev/tools.sh new file mode 100644 index 0000000..ff74fc1 --- /dev/null +++ b/dev/tools.sh @@ -0,0 +1,25 @@ +#!/bin/sh +# Host entry point: only POSIX shell and Docker are required. +set -eu +cd "$(dirname "$0")/.." +case "${1:-}" in + build) + docker pull docker:cli + ref=$(docker image inspect docker:cli --format '{{index .RepoDigests 0}}') + case "$ref" in docker@sha256:*) ;; *) echo 'Cannot pin stable Docker CLI image' >&2; exit 1;; esac + docker build --pull --build-arg "DOCKER_IMAGE=$ref" -f dev/images/tools.Dockerfile -t sxchnge-local/tools:dev dev/images + exit + ;; +esac +docker image inspect sxchnge-local/tools:dev >/dev/null 2>&1 || { echo 'Run make build first' >&2; exit 1; } +endpoint=${DOCKER_HOST:-$(docker context inspect --format '{{.Endpoints.docker.Host}}')} +case "$endpoint" in unix://*) socket=${endpoint#unix://};; *) echo 'Local tooling requires a local Unix Docker socket' >&2; exit 1;; esac +network_args= +if docker network inspect sxchnge-local_default >/dev/null 2>&1; then network_args='--network sxchnge-local_default'; fi +terminal_args=-i +if [ "${1:-}" = email ]; then terminal_args=-it; fi +# These two argument lists contain only fixed strings controlled above. +# shellcheck disable=SC2086 +exec docker run --rm $terminal_args $network_args -e "LOCAL_UID=$(id -u)" -e "LOCAL_GID=$(id -g)" \ + -v "$socket:/var/run/docker.sock" -v "$PWD:$PWD" -w "$PWD" \ + sxchnge-local/tools:dev sh dev/dispatch.sh "$@" diff --git a/dev/vault.py b/dev/vault.py deleted file mode 100644 index 5dcc0c0..0000000 --- a/dev/vault.py +++ /dev/null @@ -1,117 +0,0 @@ -"""Local Vault bootstrap API; response bodies with secrets never reach logs.""" -import json -import os -from pathlib import Path -import time -import urllib.error -import urllib.request - -STATE = Path(__file__).resolve().parents[1] / '.local' / 'dev' -URL = 'http://127.0.0.1:18200/v1/' - - -def write(path, value, mode=0o600): - path.parent.mkdir(parents=True, exist_ok=True, mode=0o700) - tmp = path.with_name(path.name + '.tmp') - fd = os.open(tmp, os.O_WRONLY | os.O_CREAT | os.O_TRUNC, mode) - with os.fdopen(fd, 'w') as out: - out.write(value) - os.chmod(tmp, mode) - os.replace(tmp, path) - - -class Vault: - def __init__(self): - self.token = None - - def call(self, path, data=None, method=None, missing=False): - headers = {'Content-Type': 'application/json'} - if self.token: - headers['X-Vault-Token'] = self.token - req = urllib.request.Request(URL + path, headers=headers, - data=json.dumps(data).encode() if data is not None else None, method=method) - try: - with urllib.request.urlopen(req, timeout=30) as response: - body = response.read() - return json.loads(body) if body else {} - except urllib.error.HTTPError as error: - if error.code == 404 and missing: - return None - raise RuntimeError(f'Vault {path} returned HTTP {error.code}') from None - - def initialize(self): - for attempt in range(90): - try: - initialized = self.call('sys/init')['initialized'] - break - except (urllib.error.URLError, ConnectionError): - time.sleep(1) - else: - raise RuntimeError('Vault did not start within 90 seconds') - initfile = STATE / 'admin' / 'init.json' - if not initialized: - if initfile.exists(): - raise RuntimeError('Vault data is missing but bootstrap material exists; restore the matching volume or explicitly reset both') - init = self.call('sys/init', {'secret_shares': 1, 'secret_threshold': 1}) - write(initfile, json.dumps(init)) - if not initfile.exists(): - raise RuntimeError('Missing .local/dev/admin/init.json for initialized Vault; restore the matching bootstrap material') - init = json.loads(initfile.read_text()) - self.token = init['root_token'] - write(STATE / 'vault' / 'unseal', init['keys_base64'][0], 0o444) - for attempt in range(60): - if not self.call('sys/seal-status')['sealed']: - return - time.sleep(1) - raise RuntimeError('Vault did not unseal within 60 seconds') - - def existing(self): - self.token = json.loads((STATE / 'admin' / 'init.json').read_text())['root_token'] - - def kv(self, path): - result = self.call('kv/data/' + path, missing=True) - return result['data']['data'] if result else None - - def put(self, path, data): - self.call('kv/data/' + path, {'data': data}) - - def ensure(self, path, factory): - value = self.kv(path) - if value is None: - value = factory() - self.put(path, value) - return value - - def random(self, length=32): - return self.call('sys/tools/random', {'bytes': length, 'format': 'hex'})['data']['random_bytes'] - - def mounts(self): - mounts = self.call('sys/mounts')['data'] - for name, kind in [('kv', 'kv'), ('database', 'database'), ('pki', 'pki')]: - if name + '/' not in mounts: - settings = {'type': kind} - if kind == 'kv': - settings['options'] = {'version': '2'} - self.call('sys/mounts/' + name, settings) - auth = self.call('sys/auth')['data'] - if 'approle/' not in auth: - self.call('sys/auth/approle', {'type': 'approle'}) - - def certificates(self): - self.call('sys/mounts/pki/tune', {'max_lease_ttl': '87600h'}) - ca = self.kv('tls') - if ca is None: - result = self.call('pki/root/generate/internal', {'common_name': 'SXCHNGE local development CA', 'ttl': '87600h'})['data'] - ca = {'ca': result['certificate']} - self.put('tls', ca) - self.call('pki/roles/local', {'allowed_domains': ['localhost', 'notifications'], - 'allow_bare_domains': True, 'allow_ip_sans': True, 'max_ttl': '720h'}) - for name, host in [('notifications-tls', 'notifications'), ('frontend-tls', 'localhost')]: - cert = self.kv(name) - if cert is None or cert['expiration'] < time.time() + 7 * 86400: - cert = self.call('pki/issue/local', {'common_name': host, 'ip_sans': '127.0.0.1', 'ttl': '720h'})['data'] - self.put(name, cert) - if name == 'frontend-tls': - write(STATE / 'frontend' / 'server.crt', cert['certificate']) - write(STATE / 'frontend' / 'server.key', cert['private_key']) - write(STATE / 'ca.pem', ca['ca'], 0o644) diff --git a/dev/vault/bff.hcl b/dev/vault/bff.hcl new file mode 100644 index 0000000..37e62ef --- /dev/null +++ b/dev/vault/bff.hcl @@ -0,0 +1,35 @@ +vault { address = "http://vault:8200" } +auto_auth { + method "approle" { + config = { + role_id_file_path = "/etc/agent/role-id" + secret_id_file_path = "/etc/agent/secret-id" + remove_secret_id_file_after_reading = false + } + } +} +template_config { + static_secret_render_interval = "10s" + exit_on_retry_failure = true +} + +env_template "TLS_CA" { + contents = "{{ with secret \"kv/data/tls\" }}{{ .Data.data.ca }}{{ end }}" + error_on_missing_key = true +} + +env_template "SXCHNGE_JWT_SECRET" { + contents = "{{ with secret \"kv/data/bff\" }}{{ .Data.data.jwt }}{{ end }}" + error_on_missing_key = true +} + +env_template "SXCHNGE_NOTIFICATIONS_TOKEN" { + contents = "{{ with secret \"kv/data/caller\" }}{{ .Data.data.token }}{{ end }}" + error_on_missing_key = true +} + +exec { + command = ["/usr/local/bin/start-service"] + restart_on_secret_changes = "always" + restart_stop_signal = "SIGTERM" +} diff --git a/dev/vault/agent.hcl b/dev/vault/nats.hcl similarity index 55% rename from dev/vault/agent.hcl rename to dev/vault/nats.hcl index 67c228e..2c2aea3 100644 --- a/dev/vault/agent.hcl +++ b/dev/vault/nats.hcl @@ -12,3 +12,14 @@ template_config { static_secret_render_interval = "10s" exit_on_retry_failure = true } + +env_template "NATS_TOKEN" { + contents = "{{ with secret \"kv/data/nats\" }}{{ .Data.data.token }}{{ end }}" + error_on_missing_key = true +} + +exec { + command = ["/usr/local/bin/start-nats"] + restart_on_secret_changes = "always" + restart_stop_signal = "SIGTERM" +} diff --git a/dev/vault/notifications.hcl b/dev/vault/notifications.hcl new file mode 100644 index 0000000..63c99ee --- /dev/null +++ b/dev/vault/notifications.hcl @@ -0,0 +1,55 @@ +vault { address = "http://vault:8200" } +auto_auth { + method "approle" { + config = { + role_id_file_path = "/etc/agent/role-id" + secret_id_file_path = "/etc/agent/secret-id" + remove_secret_id_file_after_reading = false + } + } +} +template_config { + static_secret_render_interval = "10s" + exit_on_retry_failure = true +} + +env_template "TLS_CA" { + contents = "{{ with secret \"kv/data/tls\" }}{{ .Data.data.ca }}{{ end }}" + error_on_missing_key = true +} + +env_template "SXCHNGE_NOTIFICATIONS_PRODUCT_TOKEN" { + contents = "{{ with secret \"kv/data/caller\" }}{{ .Data.data.token }}{{ end }}" + error_on_missing_key = true +} + +env_template "SXCHNGE_NATS_URL" { + contents = "{{ with secret \"kv/data/nats\" }}nats://{{ .Data.data.token }}@nats:4222{{ end }}" + error_on_missing_key = true +} + +env_template "TLS_CERT" { + contents = "{{ with secret \"kv/data/notifications-tls\" }}{{ .Data.data.certificate }}{{ end }}" + error_on_missing_key = true +} + +env_template "TLS_KEY" { + contents = "{{ with secret \"kv/data/notifications-tls\" }}{{ .Data.data.private_key }}{{ end }}" + error_on_missing_key = true +} + +env_template "EMAIL_API_KEY" { + contents = "{{ with secret \"kv/data/email\" }}{{ .Data.data.api_key }}{{ end }}" + error_on_missing_key = true +} + +env_template "EMAIL_CONFIG" { + contents = "{{ with secret \"kv/data/email\" }}email:\n driver: {{ .Data.data.driver | toJSON }}\n settings:\n api_key_env: EMAIL_API_KEY\n sender_address: {{ .Data.data.sender_address | toJSON }}\n sender_name: {{ .Data.data.sender_name | toJSON }}\n{{ end }}" + error_on_missing_key = true +} + +exec { + command = ["/usr/local/bin/start-service"] + restart_on_secret_changes = "always" + restart_stop_signal = "SIGTERM" +} diff --git a/dev/verify.py b/dev/verify.py deleted file mode 100644 index d448797..0000000 --- a/dev/verify.py +++ /dev/null @@ -1,72 +0,0 @@ -"""Integration checks against this project's running stack; never print secrets.""" -import json -import ssl -import urllib.request - -from bootstrap import compose, mongo -from vault import STATE - - -def require(condition, message): - if not condition: - raise RuntimeError(message) - - -def database_access(vault): - for role, database, forbidden in [('bff', 'sxchnge', 'sxchnge_notifications'), - ('notifications', 'sxchnge_notifications', 'sxchnge')]: - lease = vault.call('database/creds/' + role) - credentials = lease['data'] - auth = 'db.getSiblingDB("admin").auth(' + json.dumps(credentials['username']) + ',' + json.dumps(credentials['password']) + ');' - try: - result = compose('exec', '-T', 'mongo', 'mongosh', '--quiet', '--file', '/dev/stdin', capture=True, - input=auth + 'const own=db.getSiblingDB(' + json.dumps(database) + '); own.local_smoke.insertOne({_id:"vault-check"}); own.local_smoke.deleteOne({_id:"vault-check"});\n' - 'let denied=false; try { db.getSiblingDB(' + json.dumps(forbidden) + ').runCommand({find:"local_smoke",filter:{}}); } catch(e) { denied=e.code===13; } if(!denied) throw new Error("cross-database access allowed"); print("scope-ok");') - require('scope-ok' in result, 'Mongo service scope check failed') - renewed = vault.call('sys/leases/renew', {'lease_id': lease['lease_id'], 'increment': '1h'}) - require(renewed['lease_duration'] > 0, 'Mongo lease renewal failed') - finally: - vault.call('sys/leases/revoke', {'lease_id': lease['lease_id']}) - admin = vault.kv('bootstrap/mongo')['password'] - result = mongo('print(admin.getUser(' + json.dumps(credentials['username']) + ')===null)', admin) - require('true' in result, 'Revoked Mongo user still exists') - print('PASS: database-plugin issuance, separate database permissions, lease renewal and revocation') - - -def policies(vault): - for service in ['bff', 'notifications', 'nats']: - root = STATE / 'agents' / service - login = vault.call('auth/approle/login', {'role_id': (root / 'role-id').read_text(), - 'secret_id': (root / 'secret-id').read_text()}) - token = login['auth']['client_token'] - try: - result = vault.call('sys/capabilities', {'token': token, 'paths': ['kv/data/bootstrap/mongo', 'sys/init']})['data'] - require(all(result[path] == ['deny'] for path in ['kv/data/bootstrap/mongo', 'sys/init']), 'Service can access bootstrap credentials') - finally: - vault.call('auth/token/revoke', {'token': token}) - print('PASS: service identities cannot read bootstrap credentials') - - -def infrastructure(vault): - status = compose('ps', '--format', 'json', capture=True) - rows = [json.loads(line) for line in status.splitlines() if line.strip()] - infra = {r['Service']: r for r in rows if r['Service'] in {'vault', 'mongo', 'nats'}} - require(set(infra) == {'vault', 'mongo', 'nats'}, 'Expected Vault, Mongo and NATS') - require(all(r['Health'] == 'healthy' for r in infra.values()), 'Infrastructure is unhealthy') - print('PASS: Vault, Mongo replica set and NATS JetStream are healthy') - policies(vault) - database_access(vault) - - -def verify(vault): - status = compose('ps', '--format', 'json', capture=True) - rows = [json.loads(line) for line in status.splitlines() if line.strip()] - require({r['Service'] for r in rows} == {'vault', 'mongo', 'nats', 'notifications', 'bff', 'frontend'}, 'Expected six running services') - require(all(r['Health'] == 'healthy' for r in rows), 'Some services are unhealthy') - context = ssl.create_default_context(cafile=str(STATE / 'ca.pem')) - for path in ['/', '/api/v1/health']: - with urllib.request.urlopen('https://localhost:5126' + path, context=context, timeout=10) as response: - require(response.status == 200, 'Frontend or BFF health failed') - print('PASS: all six services healthy; frontend and proxied BFF use verified TLS') - infrastructure(vault) - print('Resend credentials are configured; no live email sent by this check') diff --git a/dev/verify.sh b/dev/verify.sh new file mode 100644 index 0000000..a0d636f --- /dev/null +++ b/dev/verify.sh @@ -0,0 +1,50 @@ +policies() ( + root_token=$VAULT_TOKEN + identity='' + trap 'if [ -n "$identity" ]; then printf "{}" | VAULT_TOKEN=$identity vault auth/token/revoke-self POST >/dev/null; fi' EXIT + for service in bff notifications nats; do + directory="$STATE/agents/$service" + login=$(jq -n --rawfile role "$directory/role-id" --rawfile secret "$directory/secret-id" '{role_id:($role|rtrimstr("\n")),secret_id:($secret|rtrimstr("\n"))}' | vault auth/approle/login POST) + identity=$(printf '%s' "$login" | jq -er '.auth.client_token') + printf '%s' "$login" | jq '{token:.auth.client_token,paths:["kv/data/bootstrap/mongo","sys/init"]}' | vault sys/capabilities POST | jq -e '.data | all(.[]; .==["deny"])' >/dev/null || fail 'Service can access bootstrap credentials' + VAULT_TOKEN=$(printf '%s' "$login" | jq -er '.auth.client_token') + printf '{}' | vault auth/token/revoke-self POST >/dev/null + VAULT_TOKEN=$root_token + identity='' + done + echo 'PASS: service identities cannot read bootstrap credentials' +) +database_access() ( + for role in bff notifications; do + database=sxchnge; forbidden=sxchnge_notifications + if [ "$role" = notifications ]; then database=sxchnge_notifications; forbidden=sxchnge; fi + vault "database/creds/$role" > "$WORK/lease" + trap 'jq "{lease_id:.lease_id}" "$WORK/lease" | vault sys/leases/revoke POST >/dev/null' EXIT + { + jq -r '"db.getSiblingDB(\"admin\").auth("+(.data.username|tojson)+","+(.data.password|tojson)+");"' "$WORK/lease" + printf 'const database="%s", forbidden="%s";\n' "$database" "$forbidden" + cat dev/mongo-check.js + } | mongo > "$WORK/check" || fail 'Mongo access/transaction check failed' + grep -q scope-ok "$WORK/check" || fail 'Mongo check incomplete' + jq '{lease_id:.lease_id,increment:"1h"}' "$WORK/lease" | vault sys/leases/renew POST | jq -e '.lease_duration>0' >/dev/null || fail 'Mongo renewal failed' + jq '{lease_id:.lease_id}' "$WORK/lease" | vault sys/leases/revoke POST >/dev/null + trap - EXIT + { admin_auth; jq -r '"print(admin.getUser("+(.data.username|tojson)+")===null);"' "$WORK/lease"; } | mongo > "$WORK/revoked" || fail 'Mongo revocation check failed' + grep -q true "$WORK/revoked" || fail 'Revoked Mongo user still exists' + done + echo 'PASS: Vault database issuance, isolated access, transaction commit/rollback, renewal and revocation' +) +infrastructure() { + compose ps --format json | jq -se '[.[] | select(.Service=="vault" or .Service=="mongo" or .Service=="nats")] | length==3 and all(.[];.Health=="healthy")' >/dev/null || fail 'Infrastructure is not healthy' + echo 'PASS: Vault, Mongo replica set and NATS JetStream are healthy' + policies + database_access +} +# Resolve through Docker DNS while verifying the certificate for localhost. +https() { curl -sS --max-time 5 --cacert "$STATE/ca.pem" --connect-to localhost:5126:frontend:8443 "$@"; } +product() { + compose ps --format json | jq -se 'length==6 and all(.[];.Health=="healthy")' >/dev/null || fail 'Expected six healthy services' + https -f https://localhost:5126/ >/dev/null + https -f https://localhost:5126/api/v1/health >/dev/null + echo 'PASS: six healthy services, frontend and proxied BFF use verified TLS; no email sent' +} diff --git a/docs/implementation.md b/docs/implementation.md index e2deefa..f2e3b6f 100644 --- a/docs/implementation.md +++ b/docs/implementation.md @@ -676,3 +676,32 @@ the supplementary failures are recorded and are not suppressed. The required normal test and lint commands pass. No optional security scan or email delivery was performed. Resend remains intentionally unconfigured; infrastructure stays up and product startup awaits manual `kv/email` provisioning. + +### Make and shell development tooling + +The local development tooling now runs POSIX-style shell scripts in a Docker +image containing Docker CLI, curl, jq, Git, OpenSSL, flock and ShellCheck. All eight +Python scripts were removed. Make remains the entry point; `make email` replaces +the Python credential prompt. Host requirements are Make, shell and local Docker +with Compose/BuildKit. The tool image is built from a freshly pulled stable Docker +CLI digest; application/infrastructure stable resolution and atomic local pins +remain intact. Tooling mounts the local Docker socket and workspace at its host +path; remote Docker engines are unsupported. + +Static Agent HCL sources replace configuration generation and contain only Vault +template expressions. Bootstrap preserves existing Vault state and credentials, +protects its private files, and fails when Resend credentials are absent. The shell +infrastructure check now explicitly validates real Mongo transaction commit and +rollback with each application's Vault-issued identity. The rotation command +retains its unchanged-process and expected-response checks and TTL restoration. + +Validation of the shell replacement passed: `make build`, `make test-dev-tooling`, +`make test-dev-infra`, Compose configuration validation and `git diff --check`. +An isolated `sxchnge-shell-check` project passed first bootstrap, live transaction +and lease checks, and repeated bootstrap; its containers, volumes and temporary +state were removed. `make test-dev-rotation` passed three real Vault replacements +with unchanged BFF PID, 1,048 expected HTTP responses and zero failures; role TTLs +were restored and temporary BFF/frontend containers removed. `make up` exits at +the intentional missing-Resend check, with infrastructure healthy. Backend and +frontend code did not change in this revision; their earlier test/lint results +and supplementary legacy test limitations above still apply. No email was sent. diff --git a/docs/local-development.md b/docs/local-development.md index b4b1937..298b9c2 100644 --- a/docs/local-development.md +++ b/docs/local-development.md @@ -5,9 +5,13 @@ Task [#42](https://git.sendico.io/tech/sxchnge/issues/42), ## Build and bootstrap -Requires Docker with Compose v2 and BuildKit, Python 3.10+ and Git. Docker builds -Go, protobuf bindings and Flutter web; host Go, protoc and Flutter installations -are not required for `make build && make up`. The first Flutter SDK download can +Requires Make, a POSIX shell and a local Docker engine with Compose v2 and +BuildKit. Docker builds Go, protobuf bindings and Flutter web; host Go, protoc +and Flutter installations are not required for `make build && make up`. The +tooling container includes Docker CLI, curl, jq, Git and OpenSSL. No host Python, +jq, curl or Git installation is needed for these Make targets. It mounts +the local Docker socket and workspace to orchestrate Compose; a remote Docker +daemon is not supported. The first Flutter SDK download can take several minutes. Both ARM64 and AMD64 build from the same source definitions. ```sh @@ -52,7 +56,7 @@ containers never mount it. You can also provision the same secret with a hidden terminal prompt, without placing the API key in shell history: ```sh -python3 dev/local.py email +make email make up ``` @@ -169,6 +173,7 @@ make ps make logs make down # preserves named volumes and local bootstrap material make up # idempotent provisioning and readiness checks +make test-dev-tooling # offline shell and failure-path checks make test-dev-infra # works before Resend provisioning make test-dev-rotation # real Vault rotations under BFF HTTP traffic make test-dev # requires all six services running From 9dbf0bd80c34e681357103718b717f990e76845b Mon Sep 17 00:00:00 2001 From: Stephan D Date: Sat, 12 Sep 2026 23:30:18 +0200 Subject: [PATCH 05/11] Organize local infrastructure and Vault secrets by service ownership --- AGENTS.md | 12 + Makefile | 22 +- README.md | 4 +- backend/notifications/README.md | 5 +- backend/notifications/app/config.go | 5 +- backend/notifications/app/config_test.go | 28 ++ backend/notifications/app/run.go | 18 +- backend/notifications/config.example.yml | 2 - backend/product/config.example.yml | 4 +- backend/product/notifications/client.go | 7 +- backend/product/notifications/client_test.go | 16 + dev/config/nginx.conf | 17 -- dev/dispatch.sh | 13 - dev/vault/notifications.hcl | 55 ---- docs/implementation.md | 38 +++ docs/local-development.md | 284 +++++++----------- infra/README.md | 20 ++ {dev => infra}/images/backend.Dockerfile | 5 +- {dev => infra}/images/frontend.Dockerfile | 8 +- {dev => infra}/images/nats.Dockerfile | 2 +- {dev => infra}/images/tools.Dockerfile | 0 .../bff.yml => infra/local/bff/runtime.yml | 2 - compose.yml => infra/local/compose.yml | 21 +- infra/local/frontend/Caddyfile | 16 + {dev => infra/local/mongo}/mongo-check.js | 0 {dev => infra/local/mongo}/mongo-client.js | 0 .../local/mongo/start.sh | 0 .../local/nats/start.sh | 0 .../local/notifications/runtime.yml | 2 - {dev => infra/local/scripts}/bootstrap.sh | 53 ++-- {dev => infra/local/scripts}/common.sh | 5 +- infra/local/scripts/dispatch.sh | 14 + {dev => infra/local/scripts}/images.sh | 2 +- {dev => infra/local/scripts}/rotation.sh | 6 + infra/local/scripts/secrets.sh | 56 ++++ {dev => infra/local/scripts}/start-service.sh | 6 +- {dev => infra/local/scripts}/test.sh | 34 ++- {dev => infra/local/scripts}/verify.sh | 14 +- {dev => infra/local}/tools.sh | 6 +- {dev => infra/local}/vault/bff.hcl | 8 +- {dev => infra/local}/vault/nats.hcl | 2 +- infra/local/vault/notifications.hcl | 43 +++ {dev => infra/local}/vault/server.hcl | 0 .../local/vault/start.sh | 0 44 files changed, 496 insertions(+), 359 deletions(-) delete mode 100644 dev/config/nginx.conf delete mode 100644 dev/dispatch.sh delete mode 100644 dev/vault/notifications.hcl create mode 100644 infra/README.md rename {dev => infra}/images/backend.Dockerfile (77%) rename {dev => infra}/images/frontend.Dockerfile (83%) rename {dev => infra}/images/nats.Dockerfile (83%) rename {dev => infra}/images/tools.Dockerfile (100%) rename dev/config/bff.yml => infra/local/bff/runtime.yml (95%) rename compose.yml => infra/local/compose.yml (88%) create mode 100644 infra/local/frontend/Caddyfile rename {dev => infra/local/mongo}/mongo-check.js (100%) rename {dev => infra/local/mongo}/mongo-client.js (100%) rename dev/start-mongo.sh => infra/local/mongo/start.sh (100%) rename dev/start-nats.sh => infra/local/nats/start.sh (100%) rename dev/config/notifications.yml => infra/local/notifications/runtime.yml (92%) rename {dev => infra/local/scripts}/bootstrap.sh (77%) rename {dev => infra/local/scripts}/common.sh (92%) create mode 100644 infra/local/scripts/dispatch.sh rename {dev => infra/local/scripts}/images.sh (91%) rename {dev => infra/local/scripts}/rotation.sh (89%) create mode 100644 infra/local/scripts/secrets.sh rename {dev => infra/local/scripts}/start-service.sh (60%) rename {dev => infra/local/scripts}/test.sh (61%) rename {dev => infra/local/scripts}/verify.sh (78%) rename {dev => infra/local}/tools.sh (83%) rename {dev => infra/local}/vault/bff.hcl (67%) rename {dev => infra/local}/vault/nats.hcl (83%) create mode 100644 infra/local/vault/notifications.hcl rename {dev => infra/local}/vault/server.hcl (100%) rename dev/start-vault.sh => infra/local/vault/start.sh (100%) diff --git a/AGENTS.md b/AGENTS.md index fd3976c..8fc574f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -169,3 +169,15 @@ an uncommitted working tree. Preserve unrelated changes and avoid mixing them into task commits. Record scope, validation, limitations, and PR links in the task and durable repository documentation. Merge only when authorized by the user. If work is delivered as stacked PRs, document dependencies and merge order. + +## Local infrastructure ownership + +Keep infrastructure Dockerfiles under `infra/images/` and local Compose, service +configuration and shell tooling under `infra/local/`. Keep product source under +`backend/` and `frontend/`. Organize Vault KV paths by owning service under +`services//`, infrastructure under `infrastructure//`, +and bootstrap administration under `bootstrap/`. Caller credentials +belong to the receiving service at `services//callers/`. +Use Caddy for local browser HTTPS and certificate management. Treat the private +local Docker network as trusted; do not add internal gRPC TLS or Vault-managed +local TLS certificates without an explicit requirement. diff --git a/Makefile b/Makefile index f0d88a2..756e50f 100644 --- a/Makefile +++ b/Makefile @@ -53,16 +53,16 @@ package-notifications-interface: generate-notifications tar -czf dist/notifications-go-v1.tar.gz -C backend/notifications/interface go.mod go.sum notifications README.md # Local product stack. Releases are resolved afresh, then pinned by digest. -COMPOSE = docker compose --env-file .local/dev/images.env +COMPOSE = docker compose --project-directory . -f infra/local/compose.yml --env-file .local/dev/images.env .PHONY: build up down logs ps test-dev build: - sh dev/tools.sh build - sh dev/tools.sh images + sh infra/local/tools.sh build + sh infra/local/tools.sh images $(COMPOSE) pull vault mongo $(COMPOSE) build --pull up: - sh dev/tools.sh up + sh infra/local/tools.sh up down: $(COMPOSE) down @@ -74,21 +74,21 @@ ps: $(COMPOSE) ps test-dev: - sh dev/tools.sh test - sh dev/tools.sh verify + sh infra/local/tools.sh test + sh infra/local/tools.sh verify .PHONY: test-dev-infra test-dev-infra: - sh dev/tools.sh test - sh dev/tools.sh verify-infra + sh infra/local/tools.sh test + sh infra/local/tools.sh verify-infra .PHONY: test-dev-rotation test-dev-rotation: - sh dev/tools.sh rotation + sh infra/local/tools.sh rotation .PHONY: email test-dev-tooling email: - sh dev/tools.sh email + sh infra/local/tools.sh email test-dev-tooling: - sh dev/tools.sh test + sh infra/local/tools.sh test diff --git a/README.md b/README.md index 1468cbc..054aa99 100644 --- a/README.md +++ b/README.md @@ -40,7 +40,7 @@ make build && make up This builds the product images and starts an isolated Vault, authenticated MongoDB replica set and NATS JetStream. First startup intentionally stops after bootstrap -until you set the Resend API key and verified sender in Vault's `kv/email` secret. +until you set the Resend API key and verified sender in Vault's `kv/services/notifications/email/resend` secret. Then rerun `make up` to start notifications, BFF and the frontend at [https://localhost:5126](https://localhost:5126). @@ -72,7 +72,7 @@ repository root: ```sh docker run --rm --pull always -p 127.0.0.1:5123:80 \ - -v "$PWD/frontend/app/build/web:/usr/share/nginx/html:ro" nginx:stable-alpine + -v "$PWD/frontend/app/build/web:/srv:ro" caddy:alpine caddy file-server --root /srv --listen :80 ``` Open [the local app](http://127.0.0.1:5123/). The language selector supports EN/RU diff --git a/backend/notifications/README.md b/backend/notifications/README.md index 3cffe8d..6cb0f45 100644 --- a/backend/notifications/README.md +++ b/backend/notifications/README.md @@ -24,10 +24,11 @@ Requires Go 1.27, protoc, an SXCHNGE MongoDB database and a NATS server with JetStream enabled. Copy `config.example.yml` to an untracked `config.yml` here. Set `SXCHNGE_MONGO_URI`, `SXCHNGE_NATS_URL`, a unique token of at least 32 bytes for each configured caller, and the selected email provider's API key environment -variable. Configure a verified sender address, TLS certificate/key and template +variable. Configure a verified sender address and template catalog. Paths are relative to the process working directory. -From the root, run `make run-notifications`. The gRPC listener always uses TLS. +From the root, run `make run-notifications`. The gRPC listener uses plaintext on the private local network. Configure both +`tls_cert` and `tls_key` only when TLS is explicitly required. Use a private authenticated NATS endpoint; `tls://` URLs and credentials in the URL are supported by the shared driver. Mongo and messaging both use shared `DriverConfig` settings; email settings reject unknown keys. diff --git a/backend/notifications/app/config.go b/backend/notifications/app/config.go index 8359077..e4f5dff 100644 --- a/backend/notifications/app/config.go +++ b/backend/notifications/app/config.go @@ -40,9 +40,12 @@ func Load(path string) (Config, error) { if err = decoder.Decode(new(any)); err != io.EOF { return c, merrors.InvalidArgument("unexpected configuration content") } - if c.Listen == "" || c.TLSCert == "" || c.TLSKey == "" || c.Database.Driver != db.Mongo || c.Messaging.Driver != messaging.BBNats || c.Templates == "" || len(c.Callers) == 0 { + if c.Listen == "" || c.Database.Driver != db.Mongo || c.Messaging.Driver != messaging.BBNats || c.Templates == "" || len(c.Callers) == 0 { return c, merrors.InvalidArgument("missing notification configuration") } + if (c.TLSCert == "") != (c.TLSKey == "") { + return c, merrors.InvalidArgument("both TLS certificate and key must be configured together") + } if err = c.Queue.Validate(); err != nil { return c, err } diff --git a/backend/notifications/app/config_test.go b/backend/notifications/app/config_test.go index 6a424be..5643ee7 100644 --- a/backend/notifications/app/config_test.go +++ b/backend/notifications/app/config_test.go @@ -33,3 +33,31 @@ func TestExampleConfigAndSecrets(t *testing.T) { t.Fatal("extra YAML document accepted") } } + +func TestPlaintextConfigAndIncompleteTLS(t *testing.T) { + t.Setenv("SXCHNGE_NOTIFICATIONS_PRODUCT_TOKEN", strings.Repeat("a", 32)) + c, err := Load("../config.example.yml") + if err != nil { + t.Fatal(err) + } + options, err := serverOptions(c) + if err != nil || len(options) != 0 { + t.Fatal("plain gRPC must not require TLS material") + } + data, err := os.ReadFile("../config.example.yml") + if err != nil { + t.Fatal(err) + } + for _, partial := range []string{"\ntls_cert: missing.crt\n", "\ntls_key: missing.key\n"} { + path := filepath.Join(t.TempDir(), "config.yml") + if err := os.WriteFile(path, append(data, []byte(partial)...), 0600); err != nil { + t.Fatal(err) + } + if _, err := Load(path); err == nil { + t.Fatal("incomplete TLS configuration accepted") + } + } + if _, err := serverOptions(Config{TLSCert: "missing.crt", TLSKey: "missing.key"}); err == nil { + t.Fatal("bad explicit certificate accepted") + } +} diff --git a/backend/notifications/app/run.go b/backend/notifications/app/run.go index da24ff1..08b8110 100644 --- a/backend/notifications/app/run.go +++ b/backend/notifications/app/run.go @@ -33,9 +33,9 @@ func Run(ctx context.Context, c Config, logger mlogger.Logger) error { if err != nil { return err } - cert, err := tls.LoadX509KeyPair(c.TLSCert, c.TLSKey) + options, err := serverOptions(c) if err != nil { - return merrors.OpaqueWrap(merrors.ErrInvalidArg, err, "invalid gRPC TLS certificate") + return err } client, err := db.ConnectMongo(logger, &c.Database) if err != nil { @@ -67,7 +67,7 @@ func Run(ctx context.Context, c Config, logger mlogger.Logger) error { for caller, env := range c.Callers { tokens[caller] = os.Getenv(env) } - server := grpc.NewServer(grpc.Creds(credentials.NewTLS(&tls.Config{Certificates: []tls.Certificate{cert}, MinVersion: tls.VersionTLS12})), grpc.UnaryInterceptor(transport.Authenticate(tokens)), grpc.MaxRecvMsgSize(128<<10), grpc.MaxConcurrentStreams(64)) + server := grpc.NewServer(append(options, grpc.UnaryInterceptor(transport.Authenticate(tokens)), grpc.MaxRecvMsgSize(128<<10), grpc.MaxConcurrentStreams(64))...) pb.RegisterNotificationsServiceServer(server, transport.New(svc)) listener, err := net.Listen("tcp", c.Listen) if err != nil { @@ -103,3 +103,15 @@ func Run(ctx context.Context, c Config, logger mlogger.Logger) error { logger.Info("Notifications service stopped") return nil } + +// An omitted certificate pair uses plaintext gRPC on the private service network. +func serverOptions(c Config) ([]grpc.ServerOption, error) { + if c.TLSCert == "" && c.TLSKey == "" { + return nil, nil + } + cert, err := tls.LoadX509KeyPair(c.TLSCert, c.TLSKey) + if err != nil { + return nil, merrors.OpaqueWrap(merrors.ErrInvalidArg, err, "invalid gRPC TLS certificate") + } + return []grpc.ServerOption{grpc.Creds(credentials.NewTLS(&tls.Config{Certificates: []tls.Certificate{cert}, MinVersion: tls.VersionTLS12}))}, nil +} diff --git a/backend/notifications/config.example.yml b/backend/notifications/config.example.yml index 373e531..02343a3 100644 --- a/backend/notifications/config.example.yml +++ b/backend/notifications/config.example.yml @@ -1,6 +1,4 @@ listen: "127.0.0.1:8083" -tls_cert: "server.crt" -tls_key: "server.key" database: driver: mongodb settings: diff --git a/backend/product/config.example.yml b/backend/product/config.example.yml index 3ca2ecb..2a49aef 100644 --- a/backend/product/config.example.yml +++ b/backend/product/config.example.yml @@ -32,8 +32,6 @@ api: check: min_length: 12 notifications: - address: localhost:8443 + address: localhost:8083 token_env: SXCHNGE_NOTIFICATIONS_TOKEN - ca_file: ../notifications/tls/ca.pem - server_name: localhost activation_url: http://localhost:5126/verify-email diff --git a/backend/product/notifications/client.go b/backend/product/notifications/client.go index a53159b..9174f79 100644 --- a/backend/product/notifications/client.go +++ b/backend/product/notifications/client.go @@ -12,6 +12,7 @@ import ( merrors "git.sendico.io/tech/sxchnge/shared/errors" "google.golang.org/grpc" "google.golang.org/grpc/credentials" + "google.golang.org/grpc/credentials/insecure" "google.golang.org/grpc/metadata" "google.golang.org/protobuf/types/known/timestamppb" ) @@ -57,7 +58,11 @@ func Open(c Config) (*Client, error) { } tlsConfig.RootCAs = pool } - conn, err := grpc.NewClient(c.Address, grpc.WithTransportCredentials(credentials.NewTLS(tlsConfig))) + var transport credentials.TransportCredentials = insecure.NewCredentials() + if c.CAFile != "" || c.ServerName != "" { + transport = credentials.NewTLS(tlsConfig) + } + conn, err := grpc.NewClient(c.Address, grpc.WithTransportCredentials(transport)) if err != nil { return nil, merrors.InternalWrap(err, "create notification client") } diff --git a/backend/product/notifications/client_test.go b/backend/product/notifications/client_test.go index 7b45734..bc7f5fb 100644 --- a/backend/product/notifications/client_test.go +++ b/backend/product/notifications/client_test.go @@ -88,3 +88,19 @@ func TestOpenUsesVerifiedTLSAndCallerAuthentication(t *testing.T) { require.NoError(t, client.Send(context.Background(), Message{ExpiresAt: time.Now().Add(time.Minute)})) require.Equal(t, "Bearer test-service-token-with-at-least-32-bytes", <-rpc.auth) } + +func TestOpenUsesPlaintextAndCallerTokenOnPrivateNetwork(t *testing.T) { + listener, err := net.Listen("tcp", "127.0.0.1:0") + require.NoError(t, err) + server := grpc.NewServer() + rpc := &tlsRPC{auth: make(chan string, 1)} + pb.RegisterNotificationsServiceServer(server, rpc) + go func() { _ = server.Serve(listener) }() + t.Cleanup(server.Stop) + t.Setenv("TEST_PLAIN_CALLER_TOKEN", "test-private-service-token-at-least-32-bytes") + client, err := Open(Config{Address: listener.Addr().String(), TokenEnv: "TEST_PLAIN_CALLER_TOKEN"}) + require.NoError(t, err) + t.Cleanup(func() { _ = client.Close() }) + require.NoError(t, client.Send(context.Background(), Message{ExpiresAt: time.Now().Add(time.Minute)})) + require.Equal(t, "Bearer test-private-service-token-at-least-32-bytes", <-rpc.auth) +} diff --git a/dev/config/nginx.conf b/dev/config/nginx.conf deleted file mode 100644 index 463a631..0000000 --- a/dev/config/nginx.conf +++ /dev/null @@ -1,17 +0,0 @@ -server { - listen 8443 ssl; - ssl_certificate /etc/tls/server.crt; - ssl_certificate_key /etc/tls/server.key; - ssl_protocols TLSv1.2 TLSv1.3; - root /usr/share/nginx/html; - index index.html; - location /api/ { - resolver 127.0.0.11 valid=10s ipv6=off; - set $bff http://bff:8082; - proxy_pass $bff; - proxy_set_header Host $http_host; - proxy_set_header X-Forwarded-Proto https; - proxy_set_header X-Forwarded-For $remote_addr; - } - location / { try_files $uri $uri/ /index.html; } -} diff --git a/dev/dispatch.sh b/dev/dispatch.sh deleted file mode 100644 index eba2e24..0000000 --- a/dev/dispatch.sh +++ /dev/null @@ -1,13 +0,0 @@ -#!/bin/sh -set -eu -set -o pipefail -. dev/common.sh -case "${1:-}" in - images) . dev/images.sh; images;; - up) . dev/bootstrap.sh; up;; - email) . dev/bootstrap.sh; email;; - verify|verify-infra) . dev/verify.sh; existing; infrastructure; if [ "$1" = verify ]; then product; fi;; - rotation) . dev/verify.sh; . dev/rotation.sh; existing; rotation;; - test) sh dev/test.sh;; - *) fail 'Unknown local tooling command';; -esac diff --git a/dev/vault/notifications.hcl b/dev/vault/notifications.hcl deleted file mode 100644 index 63c99ee..0000000 --- a/dev/vault/notifications.hcl +++ /dev/null @@ -1,55 +0,0 @@ -vault { address = "http://vault:8200" } -auto_auth { - method "approle" { - config = { - role_id_file_path = "/etc/agent/role-id" - secret_id_file_path = "/etc/agent/secret-id" - remove_secret_id_file_after_reading = false - } - } -} -template_config { - static_secret_render_interval = "10s" - exit_on_retry_failure = true -} - -env_template "TLS_CA" { - contents = "{{ with secret \"kv/data/tls\" }}{{ .Data.data.ca }}{{ end }}" - error_on_missing_key = true -} - -env_template "SXCHNGE_NOTIFICATIONS_PRODUCT_TOKEN" { - contents = "{{ with secret \"kv/data/caller\" }}{{ .Data.data.token }}{{ end }}" - error_on_missing_key = true -} - -env_template "SXCHNGE_NATS_URL" { - contents = "{{ with secret \"kv/data/nats\" }}nats://{{ .Data.data.token }}@nats:4222{{ end }}" - error_on_missing_key = true -} - -env_template "TLS_CERT" { - contents = "{{ with secret \"kv/data/notifications-tls\" }}{{ .Data.data.certificate }}{{ end }}" - error_on_missing_key = true -} - -env_template "TLS_KEY" { - contents = "{{ with secret \"kv/data/notifications-tls\" }}{{ .Data.data.private_key }}{{ end }}" - error_on_missing_key = true -} - -env_template "EMAIL_API_KEY" { - contents = "{{ with secret \"kv/data/email\" }}{{ .Data.data.api_key }}{{ end }}" - error_on_missing_key = true -} - -env_template "EMAIL_CONFIG" { - contents = "{{ with secret \"kv/data/email\" }}email:\n driver: {{ .Data.data.driver | toJSON }}\n settings:\n api_key_env: EMAIL_API_KEY\n sender_address: {{ .Data.data.sender_address | toJSON }}\n sender_name: {{ .Data.data.sender_name | toJSON }}\n{{ end }}" - error_on_missing_key = true -} - -exec { - command = ["/usr/local/bin/start-service"] - restart_on_secret_changes = "always" - restart_stop_signal = "SIGTERM" -} diff --git a/docs/implementation.md b/docs/implementation.md index f2e3b6f..349e31d 100644 --- a/docs/implementation.md +++ b/docs/implementation.md @@ -705,3 +705,41 @@ were restored and temporary BFF/frontend containers removed. `make up` exits at the intentional missing-Resend check, with infrastructure healthy. Backend and frontend code did not change in this revision; their earlier test/lint results and supplementary legacy test limitations above still apply. No email was sent. + +### Organized infrastructure and Vault ownership + +This revision supersedes the earlier local Nginx/Vault-certificate arrangement. +Local Compose, runtime configuration, startup scripts and verification now live +under `infra/local/`, grouped by service and responsibility. Dockerfiles live in +`infra/images/`; the root Make targets remain the entry point. Product source +stays in backend/frontend. Generated state remains in ignored `.local/dev/`. + +Vault KV now contains `services/bff/jwt`, +`services/notifications/callers/bff`, `services/notifications/email/resend`, +`infrastructure/nats/auth` and `bootstrap/mongo`. Dynamic database credentials +remain in the database engine. Migration copies with CAS protection, compares +existing values, archives readable old versions and metadata privately, and then +removes old flat/TLS KV entries. Conflicting destinations stop migration without +removing sources. Policies and Agent templates reference only the new paths. +The unused existing PKI mount is retained to avoid destroying its signing key; +new bootstrap neither creates nor uses it. + +Caddy replaces Nginx, serving Flutter and proxying `/api` to BFF. Caddy owns browser +certificates and its CA in persistent volumes; only its public CA is exported to +`.local/dev/ca.pem`. Existing browser trust must be updated to that new CA. Local +BFF-to-notifications uses plaintext authenticated gRPC. Optional explicitly +configured TLS remains available in the reusable client/server; incomplete TLS +configuration is rejected. No local service consumes TLS secrets from Vault. + +Validation passed: `make build`, `make test`, `make lint-backend`, +`make test-dev-tooling`, `make test-dev-infra`, and Compose/diff checks. Tests cover +plaintext caller-token transport, optional verified TLS, invalid certificate +configuration and secret migration conflicts. The existing Vault was migrated +and canonical paths/absence of old entries verified. An isolated fresh project +passed transaction/lease checks and repeated bootstrap; disposable volumes and +state were removed. The final rotation check also verified Caddy static serving, +SPA fallback and API health, then three pool replacements with unchanged BFF PID +and 1,083 expected HTTP responses, zero failures. Role TTLs were restored and +temporary BFF/frontend containers removed. `make up` intentionally stops at the +missing Resend credential check; no real email, optional security scan or merge +was performed. Earlier supplementary legacy test limitations remain recorded. diff --git a/docs/local-development.md b/docs/local-development.md index 298b9c2..5639c9d 100644 --- a/docs/local-development.md +++ b/docs/local-development.md @@ -3,219 +3,163 @@ Task [#42](https://git.sendico.io/tech/sxchnge/issues/42), [PR #43](https://git.sendico.io/tech/sxchnge/pulls/43), branch `dev-env-42`. -## Build and bootstrap +## Build and start Requires Make, a POSIX shell and a local Docker engine with Compose v2 and -BuildKit. Docker builds Go, protobuf bindings and Flutter web; host Go, protoc -and Flutter installations are not required for `make build && make up`. The -tooling container includes Docker CLI, curl, jq, Git and OpenSSL. No host Python, -jq, curl or Git installation is needed for these Make targets. It mounts -the local Docker socket and workspace to orchestrate Compose; a remote Docker -daemon is not supported. The first Flutter SDK download can -take several minutes. Both ARM64 and AMD64 build from the same source definitions. +BuildKit. No host Python, Go, Flutter, jq or curl is required. Make runs bootstrap +tooling in Docker with the local Docker socket and workspace mounted. Remote +Docker engines are unsupported. ```sh make build && make up ``` -`make build` resolves the current stable upstream releases, pulls infrastructure -and builds NATS packaging, notifications, BFF and frontend images. Vault and Mongo -use upstream images directly. Resolution excludes prereleases and selects Nginx's -stable channel. The exact versions and manifest digests are saved in the ignored -`.local/dev/images.env`; Flutter's stable release is pinned by Git commit. Build -fails if upstream metadata cannot be resolved. Run `make build` again to refresh -versions. Review database release upgrade requirements before updating a populated -local volume across major versions; the tooling does not perform data migrations. +The build resolves the latest stable releases and pins their digests in ignored +`.local/dev/images.env`. Prereleases are excluded, resolution failures stop the +build, and previous pins are retained on failure. Flutter is pinned to the stable +release commit. Tooling uses a freshly pulled stable Docker CLI digest. Generated +protobuf bindings and Flutter output are built locally/in Docker and never tracked. -`make up` starts persistent Vault, initializes/unseals it, provisions secrets and -AppRoles, initializes an authenticated single-member MongoDB replica set, configures -Vault's MongoDB database plugin, and starts authenticated NATS JetStream. It then -starts notifications, BFF and the frontend when the required email settings exist. +Compose is at `infra/local/compose.yml`; Make supplies the repository root as its +project directory. Dockerfiles are in `infra/images/`. Runtime and startup files +are grouped by service under `infra/local/`. The stack consists of Vault, Mongo, +NATS, notifications, BFF and Caddy serving the frontend. -**On the first run, `make up` intentionally exits with an error after infrastructure -bootstrap because Resend credentials have not been supplied.** No placeholder API -key, mock delivery mode or embedded credential is installed. The three infrastructure -services remain running so you can configure Vault, then rerun `make up`. +On first startup, bootstrap initializes/unseals persistent Vault, provisions +AppRoles and secrets, initializes Mongo replica set `rs0`, configures Vault's Mongo +plugin, and starts authenticated NATS JetStream. Product startup fails explicitly +until Resend credentials exist. Infrastructure remains available for provisioning. -## Configure Resend in Vault +## Vault layout -The default email driver is `resend`. In the Vault UI at -[http://127.0.0.1:18200](http://127.0.0.1:18200), edit the KV v2 secret **`kv/email`** -(API path `kv/data/email`): +Paths below are inside the KV v2 mount `kv`; the API inserts `data/` after `kv/`. + +```text +kv/ +├── services/ +│ ├── bff/jwt # jwt +│ └── notifications/ +│ ├── callers/bff # token; read by BFF and notifications +│ └── email/resend # driver, api_key, sender_address, sender_name +├── infrastructure/ +│ └── nats/auth # token; read by NATS and notifications +└── bootstrap/ + └── mongo # local administrator and replica-set bootstrap material +``` + +Every secret has an owning service or infrastructure component. Service policies +permit only the exact paths they consume. Applications cannot read bootstrap +credentials. Mongo application users are dynamic secrets under +`database/creds/bff` and `database/creds/notifications`, not KV entries. They have +`readWrite` only on their respective `sxchnge` / `sxchnge_notifications` databases. +Vault uses `mongodb-database-plugin` and its own `vault-manager` account, whose +password is rotated immediately after initial configuration. + +There are no frontend certificates, notification certificates or generic `tls` +entries in KV. Browser certificates belong to Caddy; internal gRPC is plaintext +on the private Docker network and retains caller-token authentication. + +## Resend + +Open [Vault](http://127.0.0.1:18200) and edit +`kv/services/notifications/email/resend`: | Field | Value | | --- | --- | | `driver` | `resend` | | `api_key` | Your Resend API key | -| `sender_address` | Your verified Resend sender email | -| `sender_name` | `SXCHNGE` or your desired display name | +| `sender_address` | Your verified sender address | +| `sender_name` | Your display name | -Local administrative login material is in the private, ignored -`.local/dev/admin/init.json` (`root_token`). Keep that file private. Product -containers never mount it. You can also provision the same secret with a hidden -terminal prompt, without placing the API key in shell history: +Local administrator login material is in private `.local/dev/admin/init.json`. +Product containers cannot access it. Alternatively use the hidden terminal prompt: ```sh make email make up ``` -The startup check rejects an empty key or the bootstrap sender placeholder. It -does not validate the key against Resend or prove domain verification; an invalid -key fails at delivery. Vault Agent fetches the key and email settings and injects -them into the notifications process. There is no `.env` secret fallback. Changing -`kv/email` restarts the notification process automatically within about 10 seconds. -Bootstrapping again preserves your existing email secret. +No placeholder key or delivery fallback is installed. Startup validates presence; +it does not send an email or validate domain ownership with Resend. Updating email +settings causes Agent to restart notifications. Existing values survive bootstrap. -## Open the product +## Browser HTTPS and internal connections -After providing Resend credentials: +Caddy serves Flutter files and proxies `/api/*` to BFF over HTTP. Open +[https://localhost:5126](https://localhost:5126). HTTPS preserves Secure/HttpOnly +cookies and same-origin API access. Caddy uses `tls internal` and manages its local +CA, certificates and renewal in the `caddy-data` volume. The `caddy-config` volume +stores Caddy runtime configuration. Neither volume nor Caddy has Vault access. -- Frontend and proxied API: [https://localhost:5126](https://localhost:5126). -- BFF health: [https://localhost:5126/api/v1/health](https://localhost:5126/api/v1/health). -- Vault UI/API: [http://127.0.0.1:18200](http://127.0.0.1:18200). - -Trust `.local/dev/ca.pem` in your local browser/OS certificate store to use HTTPS -without a certificate error. Trust installation is a separate local action, not -performed by the bootstrap. For a command-line check without changing trust: +After frontend starts, Make exports only the public Caddy CA to `.local/dev/ca.pem`. +Trust it in your browser/OS to avoid certificate errors. This is a new CA when +migrating from Vault-issued browser certificates; the old CA will not trust it. +Trust installation is not automated. A CLI check can use: ```sh curl --cacert .local/dev/ca.pem https://localhost:5126/api/v1/health ``` -HTTPS keeps the existing Secure/HttpOnly refresh cookie contract. Nginx serves -Flutter and proxies `/api/` to the BFF on the same origin. The bundle selects the -existing backend auth adapter with the public client ID `sxchnge-local-web`. -Bootstrap inserts this public browser client if absent; it has no client secret. -No frontend widget, service or provider architecture changes are needed. +BFF calls notifications over plaintext gRPC with its caller token. Mongo and NATS +also stay within the private Compose network; no internal service publishes a host +port. Only frontend and Vault bind host loopback ports. TLS remains optional in +the reusable notification client/server for deployments that explicitly configure +it; local runtime configuration contains no certificate settings. -The rest of the app retains its existing capabilities and limitations: wallet, -KYC and settings fixtures are not turned into production integrations by this -container setup. Full email signup/login requires a working Resend account. +Caddy behavior follows its [internal TLS](https://caddyserver.com/docs/caddyfile/directives/tls) +and [automatic HTTPS](https://caddyserver.com/docs/automatic-https) documentation. -## Secret and connection lifecycle +## Mongo transactions and credential rotation -| Consumer | Vault access | -| --- | --- | -| BFF | `database/creds/bff`, JWT key, notification caller token and CA | -| Notifications | `database/creds/notifications`, NATS token, caller token, TLS certificate/key and email settings | -| NATS | NATS authentication token only | -| Bootstrap | Infrastructure administration, PKI and initial identities | +Mongo is an authenticated single-node replica set (`rs0`), supporting multi-document +transactions. It has no failover or redundant data copy. A Mongo restart interrupts +access even though application credential replacement does not. -The database engine is configured at `database/config/mongo` with -`mongodb-database-plugin`. It uses its own `vault-manager` Mongo user, whose -password is rotated by Vault immediately after configuration. The manager has -`userAdminAnyDatabase`; applications never receive that credential. The independent -local Mongo administrator and replica-set key come from Vault's random generator -and are retained in `kv/bootstrap/mongo` for local provisioning. +The shared database wrapper authenticates with each service's AppRole and obtains +an independent token/lease for each pool. At half the shorter token/lease lifetime, +it verifies a replacement pool before publishing it. Active operations, cursors +and full transactions retain the old pool until they finish, then it closes and +its token is revoked. Nested stores and commit retries keep the same pool. -The BFF role has `readWrite` only on `sxchnge`; notifications has `readWrite` only -on `sxchnge_notifications`. Mongo URIs come from dynamic Vault credentials, not -static application users. Leases default to one hour with a 24-hour maximum. -The shared database wrapper obtains credentials directly from Vault using the -service's AppRole. At half the effective lifetime (the shorter of the database -lease and owning token TTL), it authenticates independently and prepares a new -Mongo pool. Only a successful primary ping permits publication. New operations -use that pool; active operations, cursors and complete transactions, including -nested stores and commit retries, retain the old pool until they finish. The old -pool then closes and its dedicated token is revoked. Mongo credential replacement -never restarts BFF or notifications. Each generation has a separate token so -cleanup cannot revoke the replacement or Vault Agent's identity. +Failed refreshes retain valid credentials and retry with bounded backoff. After +expiry, database operations fail closed until automatic recovery. Administrative +revocation, infrastructure outages and operations longer than their lease can +still interrupt work. Repository consumers use `source.Source` operation scopes; +`source.Fixed` adapts externally owned static/test connections. -Refresh failures keep the current valid pool and retry with bounded backoff. -Operations are bounded by credential expiry. If Vault remains unavailable beyond -that deadline, new database operations fail closed; refresh continues and access -recovers automatically without restarting the service. Administrative revocation, -operations exceeding their lease and infrastructure failures cannot be made -uninterrupted by pool rotation. Shutdown rejects new scopes and drains existing -work. Failed cleanup relies on Vault expiration as a backstop. +## Migration and persistence -Repositories and transaction factories share `source.Source` rather than retaining -concrete Mongo collections. Low-level consumers must acquire a scope, use its -returned context and release it after all work completes; never retain its client -or context beyond release. `source.Fixed` adapts externally owned test/static -connections. Database credentials are no longer Agent environment templates; -Agent continues to fetch non-database secrets and may restart children when those -secrets change. +Bootstrap migrates the old flat `bff`, `caller`, `email` and `nats` paths into the +hierarchy above. It copies with CAS protection, compares values, and refuses to +overwrite conflicting destinations. After consumers use the new paths, it removes +old KV metadata and obsolete `tls`, `frontend-tls`, `notifications-tls` entries. +Readable old version history and metadata are archived privately in +`.local/dev/migrations/organized-v1/` before removal. That archive is administrative +recovery material; applications do not mount it. Old deleted/destroyed versions +are not recreated. The old PKI engine is not used or provisioned by this stack; +an existing PKI mount is retained to avoid destroying its non-exportable signing +key and history as part of a KV reorganization. -Each agent authenticates using its own AppRole with a renewable periodic token. -Local SecretIDs are reusable for container restarts; each container mounts only -its own identity directory under `.local/dev/agents/`. Revoking an AppRole SecretID -requires removing its matching local `secret-id` file and rerunning `make up` to -issue a replacement. Revoking an active token/lease terminates its credentials; -The database wrapper retries with a fresh identity; Agent independently retries -its non-database secrets when Vault is reachable. +Vault, Mongo, JetStream and Caddy data use project-scoped named volumes. Private +bootstrap material lives in `.local/dev/`. `make down` preserves both. Vault's local +unseal file allows automatic unsealing on restart. This local convenience and +loopback HTTP Vault endpoint are for development. Keep bootstrap files paired +with their matching volumes; do not discard one side independently. -The notification gRPC connection uses a Vault-issued certificate for -`notifications` and the BFF validates the Vault CA and hostname. Frontend HTTPS -certificates also come from Vault PKI. Leaf certificates last 30 days; `make up` -refreshes them when less than seven days remain and reloads Nginx. Run it regularly -for a development stack kept online for weeks. Agents restart when their TLS KV -secret changes. Application secrets are never put into the frontend bundle. - -Vault, Mongo and JetStream data use project-scoped named Docker volumes. Local -Vault unseal material, administrator login, per-service bootstrap identities and -frontend TLS material are under ignored `.local/dev/`, outside image build contexts. -The local Vault automatically unseals using its local key on container restart. -These convenience credentials and loopback HTTP Vault access are for this local -machine; this is not a production Vault deployment. Do not share `.local/dev/`. -Keep it paired with its matching Docker volumes for recovery. - -Mongo and NATS expose no host ports. The `sxchnge-local` network, volumes and -container names are independent of existing Sendico stacks. Only the frontend -and Vault bind host loopback ports. No Sendico database, message bus or secrets -are consumed. - -## Manage and verify +## Verification ```sh make ps make logs -make down # preserves named volumes and local bootstrap material -make up # idempotent provisioning and readiness checks -make test-dev-tooling # offline shell and failure-path checks -make test-dev-infra # works before Resend provisioning -make test-dev-rotation # real Vault rotations under BFF HTTP traffic -make test-dev # requires all six services running +make down +make test-dev-tooling # shell analysis, failure paths, migration conflict checks +make test-dev-infra # policy isolation, real Mongo transactions and lease lifecycle +make test-dev-rotation # temporarily deploy BFF/frontend and exercise short leases +make test-dev # requires all six services; verified frontend/API HTTPS ``` -The infrastructure check exercises AppRole access restrictions, real Vault -MongoDB credential issuance, per-database access, lease renewal and user deletion -on revocation. The complete check also verifies service health and frontend/API -HTTPS with the generated CA. These checks never send a real email. -`test-dev-rotation` temporarily sets a 12-second BFF lease, deploys the built BFF -and frontend, and requires three pool replacements with unchanged process ID and -zero unexpected HTTP responses. Its invalid-token requests exercise Mongo-backed -transactions without sending email. The check restores the role TTL and removes -services it started; it recreates BFF/frontend at the beginning, so run it as a -development validation command, not against active work you need to preserve. -Race-enabled backend tests additionally hold a transaction and cursor across a -replacement and exercise failed candidate validation, outage recovery, rollback, -expiry rejection and graceful shutdown. - -Backend regression tests use fresh `mongo:latest` and `nats:latest` official -stable images, pulled before `make test-backend`. Normal validation remains -`make test` and `make lint-backend` and requires the host SDKs described in the -repository guides. - -Deleting Docker volumes removes local data; `make down` deliberately keeps them. -If starting from scratch, stop the stack and explicitly remove this project's -three volumes **and** its matching `.local/dev/` bootstrap state together. Never -reuse old unseal material with a newly initialized Vault volume, or retain Mongo -users while discarding the Vault data that manages their leases. - -## Sources and checked releases - -On 2026-09-12 the resolver selected Vault 2.1.0, MongoDB 8.3.9, NATS 2.14.6, -Go 1.27.1, Nginx stable 1.30.4, Ubuntu 26.04 and Flutter stable 3.47.4. These are -verification records, not fixed future defaults. Release selection follows the -[Vault releases](https://releases.hashicorp.com/vault/), -[MongoDB stable release notes](https://www.mongodb.com/docs/manual/release-notes/), -[NATS releases](https://github.com/nats-io/nats-server/releases/latest), -[Nginx downloads](https://nginx.org/en/download.html), -[Go releases](https://go.dev/dl/) and -[Flutter stable channel](https://github.com/flutter/flutter/tree/stable). -The implementation follows HashiCorp's -[MongoDB database plugin](https://developer.hashicorp.com/vault/docs/secrets/databases/mongodb) -and [Agent process supervisor](https://developer.hashicorp.com/vault/docs/agent-and-proxy/agent/process-supervisor) -contracts. +Rotation validation temporarily shortens the BFF role TTL, recreates BFF/frontend, +and requires three replacements with unchanged BFF PID and zero unexpected +transaction-backed HTTP responses. It restores TTLs and removes services it +started. No verification command sends email. Backend validation remains +`make test` and `make lint-backend`, using the host SDKs documented in module guides. diff --git a/infra/README.md b/infra/README.md new file mode 100644 index 0000000..3289ec1 --- /dev/null +++ b/infra/README.md @@ -0,0 +1,20 @@ +# Infrastructure + +Run `make build && make up` from the repository root. Local tooling needs Make, +POSIX shell and Docker Compose/BuildKit; all other tools run in Docker. + +- `images/`: reusable Docker build definitions for product images and tooling. +- `local/compose.yml`: isolated local stack and persistent volumes. +- `local/bff/`, `local/notifications/`: application runtime configuration. +- `local/frontend/`: Caddy configuration for Flutter files, browser HTTPS and API proxying. +- `local/vault/`: server startup/configuration and per-service Agent templates. +- `local/mongo/`, `local/nats/`: infrastructure startup and Mongo provisioning checks. +- `local/scripts/`: bootstrap, migration, release resolution and verification. +- `local/tools.sh`: host entry point used by Make. + +Local generated pins, identities and administrator material remain under ignored +`.local/dev/`; they are never build inputs. Application source remains under +`backend/` and `frontend/`. + +See [local development](../docs/local-development.md) for Vault paths, commands, +Caddy trust, persistence and migration details. diff --git a/dev/images/backend.Dockerfile b/infra/images/backend.Dockerfile similarity index 77% rename from dev/images/backend.Dockerfile rename to infra/images/backend.Dockerfile index dd3a7d3..836195b 100644 --- a/dev/images/backend.Dockerfile +++ b/infra/images/backend.Dockerfile @@ -13,8 +13,9 @@ RUN --mount=type=cache,target=/root/.cache/go-build --mount=type=cache,target=/g FROM ${VAULT_IMAGE} USER root COPY --from=build /out/service /usr/local/bin/service -COPY dev/start-service.sh /usr/local/bin/start-service -COPY dev/config /etc/sxchnge +COPY infra/local/scripts/start-service.sh /usr/local/bin/start-service +COPY infra/local/bff/runtime.yml /etc/sxchnge/bff.yml +COPY infra/local/notifications/runtime.yml /etc/sxchnge/notifications.yml COPY backend/notifications/templates.example.json /etc/sxchnge/templates.json RUN mkdir -p /run/sxchnge && chown 100:1000 /run/sxchnge USER 100:1000 diff --git a/dev/images/frontend.Dockerfile b/infra/images/frontend.Dockerfile similarity index 83% rename from dev/images/frontend.Dockerfile rename to infra/images/frontend.Dockerfile index 51c5b2c..c0f7eb7 100644 --- a/dev/images/frontend.Dockerfile +++ b/infra/images/frontend.Dockerfile @@ -1,5 +1,5 @@ ARG UBUNTU_IMAGE -ARG NGINX_IMAGE +ARG CADDY_IMAGE FROM ${UBUNTU_IMAGE} AS build RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates curl git unzip xz-utils libglu1-mesa && rm -rf /var/lib/apt/lists/* ARG FLUTTER_VERSION @@ -12,6 +12,6 @@ COPY frontend ./frontend WORKDIR /src/frontend/app ARG FRONTEND_URL=https://localhost:5126 RUN flutter pub get && flutter gen-l10n && flutter build web --no-wasm-dry-run --dart-define=SXCHNGE_API_URL=${FRONTEND_URL}/api/v1 --dart-define=SXCHNGE_CLIENT_ID=sxchnge-local-web -FROM ${NGINX_IMAGE} -COPY dev/config/nginx.conf /etc/nginx/conf.d/default.conf -COPY --from=build /src/frontend/app/build/web /usr/share/nginx/html +FROM ${CADDY_IMAGE} +COPY infra/local/frontend/Caddyfile /etc/caddy/Caddyfile +COPY --from=build /src/frontend/app/build/web /srv diff --git a/dev/images/nats.Dockerfile b/infra/images/nats.Dockerfile similarity index 83% rename from dev/images/nats.Dockerfile rename to infra/images/nats.Dockerfile index 9fe15c9..3b5b1dc 100644 --- a/dev/images/nats.Dockerfile +++ b/infra/images/nats.Dockerfile @@ -4,7 +4,7 @@ FROM ${NATS_IMAGE} AS nats FROM ${VAULT_IMAGE} USER root COPY --from=nats /nats-server /usr/local/bin/nats-server -COPY dev/start-nats.sh /usr/local/bin/start-nats +COPY infra/local/nats/start.sh /usr/local/bin/start-nats RUN mkdir -p /data /run/sxchnge && chown -R 100:1000 /data /run/sxchnge USER 100:1000 ENTRYPOINT ["vault", "agent", "-config=/etc/agent/config.hcl"] diff --git a/dev/images/tools.Dockerfile b/infra/images/tools.Dockerfile similarity index 100% rename from dev/images/tools.Dockerfile rename to infra/images/tools.Dockerfile diff --git a/dev/config/bff.yml b/infra/local/bff/runtime.yml similarity index 95% rename from dev/config/bff.yml rename to infra/local/bff/runtime.yml index ca914db..e399832 100644 --- a/dev/config/bff.yml +++ b/infra/local/bff/runtime.yml @@ -39,6 +39,4 @@ api: notifications: address: notifications:8083 token_env: SXCHNGE_NOTIFICATIONS_TOKEN - ca_file: /run/sxchnge/ca.pem - server_name: notifications activation_url: https://localhost:5126/verify-email diff --git a/compose.yml b/infra/local/compose.yml similarity index 88% rename from compose.yml rename to infra/local/compose.yml index 169d6fa..a7bb6d3 100644 --- a/compose.yml +++ b/infra/local/compose.yml @@ -9,7 +9,7 @@ x-agent: &agent x-backend-build: &backend-build context: . - dockerfile: dev/images/backend.Dockerfile + dockerfile: infra/images/backend.Dockerfile services: vault: @@ -20,8 +20,8 @@ services: ports: ['127.0.0.1:18200:8200'] volumes: - vault-data:/vault/file - - ./dev/vault/server.hcl:/etc/vault/server.hcl:ro - - ./dev/start-vault.sh:/etc/vault/start.sh:ro + - ./infra/local/vault/server.hcl:/etc/vault/server.hcl:ro + - ./infra/local/vault/start.sh:/etc/vault/start.sh:ro - ./.local/dev/vault:/bootstrap:ro healthcheck: test: [CMD-SHELL, 'VAULT_ADDR=http://127.0.0.1:8200 vault status >/dev/null'] @@ -38,7 +38,7 @@ services: MONGO_INITDB_ROOT_USERNAME: local-admin MONGO_INITDB_ROOT_PASSWORD_FILE: /run/mongo/password volumes: - - ./dev/start-mongo.sh:/etc/mongo/start.sh:ro + - ./infra/local/mongo/start.sh:/etc/mongo/start.sh:ro - mongo-data:/data/db - ./.local/dev/mongo:/run/mongo:ro healthcheck: @@ -52,7 +52,7 @@ services: image: sxchnge-local/nats:dev build: context: . - dockerfile: dev/images/nats.Dockerfile + dockerfile: infra/images/nats.Dockerfile args: NATS_IMAGE: ${NATS_IMAGE:?Run make build first} VAULT_IMAGE: ${VAULT_IMAGE:?Run make build first} @@ -121,20 +121,21 @@ services: image: sxchnge-local/frontend:dev build: context: . - dockerfile: dev/images/frontend.Dockerfile + dockerfile: infra/images/frontend.Dockerfile args: UBUNTU_IMAGE: ${UBUNTU_IMAGE:?Run make build first} - NGINX_IMAGE: ${NGINX_IMAGE:?Run make build first} + CADDY_IMAGE: ${CADDY_IMAGE:?Run make build first} FLUTTER_VERSION: ${FLUTTER_VERSION:?Run make build first} FLUTTER_COMMIT: ${FLUTTER_COMMIT:?Run make build first} restart: unless-stopped ports: ['127.0.0.1:5126:8443'] volumes: - - ./.local/dev/frontend:/etc/tls:ro + - caddy-data:/data + - caddy-config:/config depends_on: bff: {condition: service_healthy} healthcheck: - test: [CMD, wget, -q, --no-check-certificate, -O, /dev/null, 'https://127.0.0.1:8443/'] + test: [CMD, wget, -q, --no-check-certificate, -O, /dev/null, 'https://localhost:8443/'] interval: 3s timeout: 3s retries: 40 @@ -143,3 +144,5 @@ volumes: vault-data: mongo-data: nats-data: + caddy-data: + caddy-config: diff --git a/infra/local/frontend/Caddyfile b/infra/local/frontend/Caddyfile new file mode 100644 index 0000000..8f39041 --- /dev/null +++ b/infra/local/frontend/Caddyfile @@ -0,0 +1,16 @@ +{ + skip_install_trust + auto_https disable_redirects +} + +https://localhost:8443 { + tls internal + root * /srv + handle /api/* { + reverse_proxy bff:8082 + } + handle { + try_files {path} /index.html + file_server + } +} diff --git a/dev/mongo-check.js b/infra/local/mongo/mongo-check.js similarity index 100% rename from dev/mongo-check.js rename to infra/local/mongo/mongo-check.js diff --git a/dev/mongo-client.js b/infra/local/mongo/mongo-client.js similarity index 100% rename from dev/mongo-client.js rename to infra/local/mongo/mongo-client.js diff --git a/dev/start-mongo.sh b/infra/local/mongo/start.sh similarity index 100% rename from dev/start-mongo.sh rename to infra/local/mongo/start.sh diff --git a/dev/start-nats.sh b/infra/local/nats/start.sh similarity index 100% rename from dev/start-nats.sh rename to infra/local/nats/start.sh diff --git a/dev/config/notifications.yml b/infra/local/notifications/runtime.yml similarity index 92% rename from dev/config/notifications.yml rename to infra/local/notifications/runtime.yml index a5a3882..7e4b060 100644 --- a/dev/config/notifications.yml +++ b/infra/local/notifications/runtime.yml @@ -1,6 +1,4 @@ listen: "0.0.0.0:8083" -tls_cert: "/run/sxchnge/server.crt" -tls_key: "/run/sxchnge/server.key" database: driver: mongodb settings: diff --git a/dev/bootstrap.sh b/infra/local/scripts/bootstrap.sh similarity index 77% rename from dev/bootstrap.sh rename to infra/local/scripts/bootstrap.sh index cd6da49..33e3585 100644 --- a/dev/bootstrap.sh +++ b/infra/local/scripts/bootstrap.sh @@ -16,7 +16,7 @@ initialize() { } mounts() { vault sys/mounts > "$WORK/mounts" - for name in kv database pki; do + for name in kv database; do if ! jq -e --arg key "$name/" 'has($key)' < "$WORK/mounts" >/dev/null; then if ! jq -e --arg key "$name/" '.data|has($key)' < "$WORK/mounts" >/dev/null; then jq -n --arg name "$name" '{type:$name} + (if $name=="kv" then {options:{version:"2"}} else {} end)' | vault "sys/mounts/$name" POST >/dev/null @@ -27,27 +27,6 @@ mounts() { printf '{"type":"approle"}' | vault sys/auth/approle POST >/dev/null fi } -certificates() { - printf '{"max_lease_ttl":"87600h"}' | vault sys/mounts/pki/tune POST >/dev/null - ca=$(kv tls) - if [ "$ca" = null ]; then - printf '{"common_name":"SXCHNGE local development CA","ttl":"87600h"}' | vault pki/root/generate/internal POST | jq '{ca:.data.certificate}' | put tls - fi - printf '{"allowed_domains":["localhost","notifications"],"allow_bare_domains":true,"allow_ip_sans":true,"max_ttl":"720h"}' | vault pki/roles/local POST >/dev/null - for name in notifications-tls frontend-tls; do - host=notifications; [ "$name" != frontend-tls ] || host=localhost - kv "$name" > "$WORK/cert" - if ! jq -e '.!=null and .expiration>(now+604800)' "$WORK/cert" >/dev/null; then - jq -n --arg host "$host" '{common_name:$host,ip_sans:"127.0.0.1",ttl:"720h"}' | vault pki/issue/local POST | jq '.data' > "$WORK/cert" - put "$name" < "$WORK/cert" - fi - if [ "$name" = frontend-tls ]; then - jq -r '.certificate' "$WORK/cert" | write "$STATE/frontend/server.crt" - jq -r '.private_key' "$WORK/cert" | write "$STATE/frontend/server.key" - fi - done - kv tls | jq -r '.ca' | write "$STATE/ca.pem" 644 -} provision_mongo() { if [ "$(kv bootstrap/mongo)" = null ]; then random > "$WORK/password"; random 384 > "$WORK/keyfile"; random > "$WORK/manager" @@ -72,16 +51,16 @@ provision_mongo() { database=sxchnge; [ "$role" != notifications ] || database=sxchnge_notifications jq -n --arg database "$database" '{db_name:"mongo",creation_statements:({db:"admin",roles:[{role:"readWrite",db:$database}]}|tojson),default_ttl:"1h",max_ttl:"24h"}' | vault "database/roles/$role" POST >/dev/null done - { cat "$WORK/admin.js"; cat dev/mongo-client.js; } | mongo >/dev/null || fail 'Browser client provisioning failed' + { cat "$WORK/admin.js"; cat infra/local/mongo/mongo-client.js; } | mongo >/dev/null || fail 'Browser client provisioning failed' } provision_agents() { for service in bff notifications nats; do directory="$STATE/agents/$service" mkdir -p "$directory"; chmod 755 "$directory" case "$service" in - bff) paths='bff caller tls';; - notifications) paths='caller tls notifications-tls nats email';; - nats) paths=nats;; + bff) paths='services/bff/jwt services/notifications/callers/bff';; + notifications) paths='services/notifications/callers/bff infrastructure/nats/auth services/notifications/email/resend';; + nats) paths=infrastructure/nats/auth;; esac : > "$WORK/policy" for path in $paths; do printf 'path "kv/data/%s" { capabilities = ["read"] }\n' "$path" >> "$WORK/policy"; done @@ -92,7 +71,12 @@ provision_agents() { if [ ! -f "$directory/secret-id" ]; then printf '{}' | vault "auth/approle/role/$service/secret-id" POST | jq -r '.data.secret_id' | write "$directory/secret-id" 444 fi - cat "dev/vault/$service.hcl" | write "$directory/config.hcl" 444 + if ! cmp -s "infra/local/vault/$service.hcl" "$directory/config.hcl"; then + cat "infra/local/vault/$service.hcl" | write "$directory/config.hcl" 444 + if compose ps --status running --services "$service" | grep -qx "$service"; then + compose up -d --no-deps --force-recreate --wait --wait-timeout 120 "$service" + fi + fi done } up() { @@ -107,20 +91,21 @@ up() { fi initialize mounts - for path in nats caller bff; do + migrate_secrets + for path in infrastructure/nats/auth services/notifications/callers/bff services/bff/jwt; do if [ "$(kv "$path")" = null ]; then - field=token; [ "$path" != bff ] || field=jwt + field=token; [ "$path" != services/bff/jwt ] || field=jwt random | jq -Rs --arg field "$field" '{($field):rtrimstr("\n")}' | put "$path" fi done - printf '{"driver":"resend","api_key":"","sender_address":"notifications@localhost.test","sender_name":"SXCHNGE local"}' | ensure email - certificates + printf '{"driver":"resend","api_key":"","sender_address":"notifications@localhost.test","sender_name":"SXCHNGE local"}' | ensure services/notifications/email/resend provision_mongo provision_agents compose up -d --wait --wait-timeout 120 nats - kv email | email_valid || fail 'Vault kv/email needs a Resend api_key and verified sender_address. Run make email, then make up.' + retire_secrets + kv services/notifications/email/resend | email_valid || fail 'Vault kv/services/notifications/email/resend needs a Resend api_key and verified sender_address. Run make email, then make up.' compose up -d --wait --wait-timeout 180 - compose exec -T frontend nginx -s reload + export_ca echo 'Local stack ready: https://localhost:5126; Vault: http://127.0.0.1:18200' } email() { @@ -138,6 +123,6 @@ email() { case "$address" in *@*) ;; *) fail 'A sender email address is required';; esac printf '%s' "$key" > "$WORK/email-key" printf '%s' "$address" > "$WORK/email-address" - jq -n --rawfile key "$WORK/email-key" --rawfile address "$WORK/email-address" '{driver:"resend",api_key:$key,sender_address:$address,sender_name:"SXCHNGE"}' | put email + jq -n --rawfile key "$WORK/email-key" --rawfile address "$WORK/email-address" '{driver:"resend",api_key:$key,sender_address:$address,sender_name:"SXCHNGE"}' | put services/notifications/email/resend echo 'Email configuration saved in Vault. Run make up.' } diff --git a/dev/common.sh b/infra/local/scripts/common.sh similarity index 92% rename from dev/common.sh rename to infra/local/scripts/common.sh index 45ef441..522bfb0 100644 --- a/dev/common.sh +++ b/infra/local/scripts/common.sh @@ -14,7 +14,7 @@ trap cleanup EXIT trap 'exit 130' INT trap 'exit 143' TERM fail() { echo "$*" >&2; exit 1; } -compose() { docker compose --env-file "$STATE/images.env" "$@"; } +compose() { docker compose --project-directory . -f infra/local/compose.yml --env-file "$STATE/images.env" "$@"; } write() { target=$1; mode=${2:-600} mkdir -p "$(dirname "$target")" @@ -58,3 +58,6 @@ ensure() ( mongo() { compose exec -T mongo mongosh --quiet --file /dev/stdin > "$WORK/mongo-result" 2> "$WORK/mongo-error" || return 1; cat "$WORK/mongo-result"; } admin_auth() { kv bootstrap/mongo | jq -r '"const admin=db.getSiblingDB(\"admin\"); admin.auth(\"local-admin\","+(.password|tojson)+");"'; } email_valid() { jq -e '(.api_key|type=="string") and (.api_key|test("\\S")) and (.sender_address|type=="string") and (.sender_address|test("\\S")) and .sender_address!="notifications@localhost.test"' >/dev/null; } +export_ca() { + compose exec -T frontend cat /data/caddy/pki/authorities/local/root.crt | write "$STATE/ca.pem" 644 +} diff --git a/infra/local/scripts/dispatch.sh b/infra/local/scripts/dispatch.sh new file mode 100644 index 0000000..1adbd42 --- /dev/null +++ b/infra/local/scripts/dispatch.sh @@ -0,0 +1,14 @@ +#!/bin/sh +set -eu +set -o pipefail +. infra/local/scripts/common.sh +. infra/local/scripts/secrets.sh +case "${1:-}" in + images) . infra/local/scripts/images.sh; images;; + up) . infra/local/scripts/bootstrap.sh; up;; + email) . infra/local/scripts/bootstrap.sh; email;; + verify|verify-infra) . infra/local/scripts/verify.sh; existing; infrastructure; if [ "$1" = verify ]; then product; fi;; + rotation) . infra/local/scripts/verify.sh; . infra/local/scripts/rotation.sh; existing; rotation;; + test) sh infra/local/scripts/test.sh;; + *) fail 'Unknown local tooling command';; +esac diff --git a/dev/images.sh b/infra/local/scripts/images.sh similarity index 91% rename from dev/images.sh rename to infra/local/scripts/images.sh index 4238fbb..7137117 100644 --- a/dev/images.sh +++ b/infra/local/scripts/images.sh @@ -10,7 +10,7 @@ image() ( ) images() ( go=$(get 'https://go.dev/dl/?mode=json' | jq -er '.[0] | select(.stable) | .version | select(test("^go[0-9]+\\.[0-9]+\\.[0-9]+$"))') || exit 1 - for spec in VAULT_IMAGE:hashicorp/vault:latest MONGO_IMAGE:library/mongo:latest NATS_IMAGE:library/nats:latest NGINX_IMAGE:library/nginx:stable-alpine "GO_IMAGE:library/golang:${go#go}-alpine" UBUNTU_IMAGE:library/ubuntu:latest; do + for spec in VAULT_IMAGE:hashicorp/vault:latest MONGO_IMAGE:library/mongo:latest NATS_IMAGE:library/nats:latest CADDY_IMAGE:library/caddy:alpine "GO_IMAGE:library/golang:${go#go}-alpine" UBUNTU_IMAGE:library/ubuntu:latest; do key=${spec%%:*}; rest=${spec#*:}; repo=${rest%:*}; alias=${rest##*:} ref=$(image "$repo" "$alias") || exit 1 printf '%s=%s\n' "$key" "$ref" >> "$WORK/images.env" diff --git a/dev/rotation.sh b/infra/local/scripts/rotation.sh similarity index 89% rename from dev/rotation.sh rename to infra/local/scripts/rotation.sh index 9f7ce2f..7355773 100644 --- a/dev/rotation.sh +++ b/infra/local/scripts/rotation.sh @@ -20,6 +20,12 @@ rotation() ( trap 'exit 143' TERM jq '.+{default_ttl:"12s",max_ttl:"24s"}' "$WORK/original-role" | vault database/roles/bff POST >/dev/null compose up -d --no-deps --force-recreate --wait --wait-timeout 90 bff frontend + export_ca + https -f https://localhost:5126/ > "$WORK/index" + grep -q flutter_bootstrap "$WORK/index" || fail 'Frontend bundle is not served' + https -f https://localhost:5126/settings > "$WORK/route" + cmp -s "$WORK/index" "$WORK/route" || fail 'Frontend route fallback failed' + https -f https://localhost:5126/api/v1/health >/dev/null process=$(compose exec -T bff pidof service) for worker in 1 2 3 4; do ( diff --git a/infra/local/scripts/secrets.sh b/infra/local/scripts/secrets.sh new file mode 100644 index 0000000..264a1d2 --- /dev/null +++ b/infra/local/scripts/secrets.sh @@ -0,0 +1,56 @@ +# Canonical paths follow ownership. Shared callers live under the receiving service. +secret_moves() { + cat <<'PATHS' +bff services/bff/jwt +caller services/notifications/callers/bff +email services/notifications/email/resend +nats infrastructure/nats/auth +PATHS +} +# Archive readable version history privately before removing an obsolete KV path. +archive_secret() ( + old=$1 + metadata=$(vault "kv/metadata/$old" GET yes) + [ "$metadata" != null ] || exit 0 + directory="$STATE/migrations/organized-v1/$old" + mkdir -p "$directory" + printf '%s' "$metadata" | write "$directory/metadata.json" + printf '%s' "$metadata" | jq -r '.data.versions|to_entries[]|select(.value.destroyed==false and .value.deletion_time=="")|.key' > "$WORK/versions" + while IFS= read -r version; do + vault "kv/data/$old?version=$version" | write "$directory/version-$version.json" + done < "$WORK/versions" +) +migrate_secrets() ( + secret_moves > "$WORK/moves" + while read -r old new; do + kv "$old" > "$WORK/old" + [ "$(cat "$WORK/old")" != null ] || continue + archive_secret "$old" + kv "$new" > "$WORK/new" + if [ "$(cat "$WORK/new")" = null ]; then + # CAS=0 prevents overwriting a destination created concurrently. + jq '{data:.,options:{cas:0}}' "$WORK/old" | vault "kv/data/$new" POST >/dev/null + fi + kv "$new" > "$WORK/new" + jq -e -s '.[0]==.[1]' "$WORK/old" "$WORK/new" >/dev/null || fail "Conflicting Vault values at $old and $new; no source removed" + done < "$WORK/moves" +) +retire_secrets() ( + # Call only after templates, policies and deployed images use canonical paths. + secret_moves > "$WORK/moves" + while read -r old new; do + kv "$old" > "$WORK/old" + [ "$(cat "$WORK/old")" != null ] || continue + kv "$new" > "$WORK/new" + jq -e -s '.[0]==.[1]' "$WORK/old" "$WORK/new" >/dev/null || fail "Vault migration verification failed for $old" + archive_secret "$old" + printf '{}' | vault "kv/metadata/$old" DELETE >/dev/null + done < "$WORK/moves" + for old in tls frontend-tls notifications-tls; do + archive_secret "$old" + if [ "$(vault "kv/metadata/$old" GET yes)" != null ]; then printf '{}' | vault "kv/metadata/$old" DELETE >/dev/null; fi + done + # The obsolete PKI mount is deliberately not disabled: its private signing key + # and history are retained until the owner explicitly chooses to destroy them. + echo 'PASS: Vault secrets migrated and verified; obsolete KV entries retired' +) diff --git a/dev/start-service.sh b/infra/local/scripts/start-service.sh similarity index 60% rename from dev/start-service.sh rename to infra/local/scripts/start-service.sh index 62bf205..dec163b 100755 --- a/dev/start-service.sh +++ b/infra/local/scripts/start-service.sh @@ -1,13 +1,9 @@ #!/bin/sh set -eu umask 077 -printf '%s\n' "$TLS_CA" > /run/sxchnge/ca.pem -unset TLS_CA if [ "$SXCHNGE_SERVICE" = notifications ]; then - printf '%s\n' "$TLS_CERT" > /run/sxchnge/server.crt - printf '%s\n' "$TLS_KEY" > /run/sxchnge/server.key printf '%s\n' "$EMAIL_CONFIG" > /run/sxchnge/email.yml - unset TLS_CERT TLS_KEY EMAIL_CONFIG + unset EMAIL_CONFIG cat /etc/sxchnge/notifications.yml /run/sxchnge/email.yml > /run/sxchnge/config.yml exec service -config.file=/run/sxchnge/config.yml fi diff --git a/dev/test.sh b/infra/local/scripts/test.sh similarity index 61% rename from dev/test.sh rename to infra/local/scripts/test.sh index c0307ed..7fb8a92 100644 --- a/dev/test.sh +++ b/infra/local/scripts/test.sh @@ -2,11 +2,11 @@ set -eu set -o pipefail # Shell syntax and static analysis cover the actual container shell scripts. -shellcheck -s ash -x -S warning dev/dispatch.sh dev/test.sh -shellcheck -x -S warning dev/tools.sh dev/start-*.sh -. dev/common.sh -. dev/images.sh -. dev/bootstrap.sh +shellcheck -s ash -x -S warning infra/local/scripts/dispatch.sh infra/local/scripts/test.sh +shellcheck -x -S warning infra/local/tools.sh infra/local/scripts/start-service.sh infra/local/mongo/start.sh infra/local/nats/start.sh infra/local/vault/start.sh +. infra/local/scripts/common.sh +. infra/local/scripts/images.sh +. infra/local/scripts/bootstrap.sh # Offline fixtures exercise stable selection, reject prereleases, and preserve pins. get() { case "$1" in @@ -33,7 +33,29 @@ vault() { printf '{"initialized":true}'; } if (initialize) > "$WORK/initialize-error" 2>&1; then fail 'Missing local keys accepted'; fi grep -q 'Missing .local' "$WORK/initialize-error" for service in bff notifications; do - if grep -Eq 'database/creds/|SXCHNGE_MONGO_URI' "dev/vault/$service.hcl"; then fail 'Database credentials trigger Agent restart'; fi + if grep -Eq 'database/creds/|SXCHNGE_MONGO_URI' "infra/local/vault/$service.hcl"; then fail 'Database credentials trigger Agent restart'; fi done echo 'PASS: shell analysis, stable selection, failed-resolution preservation, email validation, missing-key protection and Agent contract' if command -v python >/dev/null 2>&1 || command -v python3 >/dev/null 2>&1; then fail 'Tool image unexpectedly contains Python'; fi +# Migration never overwrites an independently provisioned destination. +. infra/local/scripts/secrets.sh +secret_moves() { echo 'email services/notifications/email/resend'; } +archive_secret() { :; } +kv() { + case "$1" in + email) printf '{"api_key":"existing-fixture"}';; + *) cat "$WORK/destination";; + esac +} +vault() { + [ "$1" = kv/data/services/notifications/email/resend ] || fail 'Unexpected migration mutation' + jq -e '.options.cas==0' >/dev/null + printf '{"api_key":"existing-fixture"}' > "$WORK/destination" +} +printf 'null' > "$WORK/destination" +migrate_secrets +migrate_secrets +printf '{"api_key":"different-fixture"}' > "$WORK/destination" +if (migrate_secrets) >/dev/null 2>&1; then fail 'Migration overwrote conflicting destination'; fi +jq -e '.api_key=="different-fixture"' "$WORK/destination" >/dev/null +echo 'PASS: secret migration is repeatable and refuses conflicting destination values' diff --git a/dev/verify.sh b/infra/local/scripts/verify.sh similarity index 78% rename from dev/verify.sh rename to infra/local/scripts/verify.sh index a0d636f..c378101 100644 --- a/dev/verify.sh +++ b/infra/local/scripts/verify.sh @@ -6,7 +6,7 @@ policies() ( directory="$STATE/agents/$service" login=$(jq -n --rawfile role "$directory/role-id" --rawfile secret "$directory/secret-id" '{role_id:($role|rtrimstr("\n")),secret_id:($secret|rtrimstr("\n"))}' | vault auth/approle/login POST) identity=$(printf '%s' "$login" | jq -er '.auth.client_token') - printf '%s' "$login" | jq '{token:.auth.client_token,paths:["kv/data/bootstrap/mongo","sys/init"]}' | vault sys/capabilities POST | jq -e '.data | all(.[]; .==["deny"])' >/dev/null || fail 'Service can access bootstrap credentials' + printf '%s' "$login" | jq '{token:.auth.client_token,paths:["kv/data/bootstrap/mongo","sys/init","kv/data/bff","kv/data/caller","kv/data/email","kv/data/nats","kv/data/tls"]}' | vault sys/capabilities POST | jq -e '.data | all(.[]; .==["deny"])' >/dev/null || fail 'Service can access bootstrap credentials' VAULT_TOKEN=$(printf '%s' "$login" | jq -er '.auth.client_token') printf '{}' | vault auth/token/revoke-self POST >/dev/null VAULT_TOKEN=$root_token @@ -23,7 +23,7 @@ database_access() ( { jq -r '"db.getSiblingDB(\"admin\").auth("+(.data.username|tojson)+","+(.data.password|tojson)+");"' "$WORK/lease" printf 'const database="%s", forbidden="%s";\n' "$database" "$forbidden" - cat dev/mongo-check.js + cat infra/local/mongo/mongo-check.js } | mongo > "$WORK/check" || fail 'Mongo access/transaction check failed' grep -q scope-ok "$WORK/check" || fail 'Mongo check incomplete' jq '{lease_id:.lease_id,increment:"1h"}' "$WORK/lease" | vault sys/leases/renew POST | jq -e '.lease_duration>0' >/dev/null || fail 'Mongo renewal failed' @@ -37,6 +37,7 @@ database_access() ( infrastructure() { compose ps --format json | jq -se '[.[] | select(.Service=="vault" or .Service=="mongo" or .Service=="nats")] | length==3 and all(.[];.Health=="healthy")' >/dev/null || fail 'Infrastructure is not healthy' echo 'PASS: Vault, Mongo replica set and NATS JetStream are healthy' + layout policies database_access } @@ -48,3 +49,12 @@ product() { https -f https://localhost:5126/api/v1/health >/dev/null echo 'PASS: six healthy services, frontend and proxied BFF use verified TLS; no email sent' } +layout() { + for old in bff caller email nats tls frontend-tls notifications-tls; do + [ "$(vault "kv/metadata/$old" GET yes)" = null ] || fail "Obsolete KV entry remains: $old" + done + for path in services/bff/jwt services/notifications/callers/bff services/notifications/email/resend infrastructure/nats/auth bootstrap/mongo; do + vault "kv/metadata/$path" | jq -e '.data.current_version>0' >/dev/null || fail "Missing canonical KV entry: $path" + done + echo 'PASS: canonical Vault hierarchy present; obsolete flat and TLS entries absent' +} diff --git a/dev/tools.sh b/infra/local/tools.sh similarity index 83% rename from dev/tools.sh rename to infra/local/tools.sh index ff74fc1..7fe4ed2 100644 --- a/dev/tools.sh +++ b/infra/local/tools.sh @@ -1,13 +1,13 @@ #!/bin/sh # Host entry point: only POSIX shell and Docker are required. set -eu -cd "$(dirname "$0")/.." +cd "$(dirname "$0")/../.." case "${1:-}" in build) docker pull docker:cli ref=$(docker image inspect docker:cli --format '{{index .RepoDigests 0}}') case "$ref" in docker@sha256:*) ;; *) echo 'Cannot pin stable Docker CLI image' >&2; exit 1;; esac - docker build --pull --build-arg "DOCKER_IMAGE=$ref" -f dev/images/tools.Dockerfile -t sxchnge-local/tools:dev dev/images + docker build --pull --build-arg "DOCKER_IMAGE=$ref" -f infra/images/tools.Dockerfile -t sxchnge-local/tools:dev infra/images exit ;; esac @@ -22,4 +22,4 @@ if [ "${1:-}" = email ]; then terminal_args=-it; fi # shellcheck disable=SC2086 exec docker run --rm $terminal_args $network_args -e "LOCAL_UID=$(id -u)" -e "LOCAL_GID=$(id -g)" \ -v "$socket:/var/run/docker.sock" -v "$PWD:$PWD" -w "$PWD" \ - sxchnge-local/tools:dev sh dev/dispatch.sh "$@" + sxchnge-local/tools:dev sh infra/local/scripts/dispatch.sh "$@" diff --git a/dev/vault/bff.hcl b/infra/local/vault/bff.hcl similarity index 67% rename from dev/vault/bff.hcl rename to infra/local/vault/bff.hcl index 37e62ef..1b1eebe 100644 --- a/dev/vault/bff.hcl +++ b/infra/local/vault/bff.hcl @@ -13,18 +13,14 @@ template_config { exit_on_retry_failure = true } -env_template "TLS_CA" { - contents = "{{ with secret \"kv/data/tls\" }}{{ .Data.data.ca }}{{ end }}" - error_on_missing_key = true -} env_template "SXCHNGE_JWT_SECRET" { - contents = "{{ with secret \"kv/data/bff\" }}{{ .Data.data.jwt }}{{ end }}" + contents = "{{ with secret \"kv/data/services/bff/jwt\" }}{{ .Data.data.jwt }}{{ end }}" error_on_missing_key = true } env_template "SXCHNGE_NOTIFICATIONS_TOKEN" { - contents = "{{ with secret \"kv/data/caller\" }}{{ .Data.data.token }}{{ end }}" + contents = "{{ with secret \"kv/data/services/notifications/callers/bff\" }}{{ .Data.data.token }}{{ end }}" error_on_missing_key = true } diff --git a/dev/vault/nats.hcl b/infra/local/vault/nats.hcl similarity index 83% rename from dev/vault/nats.hcl rename to infra/local/vault/nats.hcl index 2c2aea3..4e712a4 100644 --- a/dev/vault/nats.hcl +++ b/infra/local/vault/nats.hcl @@ -14,7 +14,7 @@ template_config { } env_template "NATS_TOKEN" { - contents = "{{ with secret \"kv/data/nats\" }}{{ .Data.data.token }}{{ end }}" + contents = "{{ with secret \"kv/data/infrastructure/nats/auth\" }}{{ .Data.data.token }}{{ end }}" error_on_missing_key = true } diff --git a/infra/local/vault/notifications.hcl b/infra/local/vault/notifications.hcl new file mode 100644 index 0000000..bb1a912 --- /dev/null +++ b/infra/local/vault/notifications.hcl @@ -0,0 +1,43 @@ +vault { address = "http://vault:8200" } +auto_auth { + method "approle" { + config = { + role_id_file_path = "/etc/agent/role-id" + secret_id_file_path = "/etc/agent/secret-id" + remove_secret_id_file_after_reading = false + } + } +} +template_config { + static_secret_render_interval = "10s" + exit_on_retry_failure = true +} + + +env_template "SXCHNGE_NOTIFICATIONS_PRODUCT_TOKEN" { + contents = "{{ with secret \"kv/data/services/notifications/callers/bff\" }}{{ .Data.data.token }}{{ end }}" + error_on_missing_key = true +} + +env_template "SXCHNGE_NATS_URL" { + contents = "{{ with secret \"kv/data/infrastructure/nats/auth\" }}nats://{{ .Data.data.token }}@nats:4222{{ end }}" + error_on_missing_key = true +} + + + +env_template "EMAIL_API_KEY" { + contents = "{{ with secret \"kv/data/services/notifications/email/resend\" }}{{ .Data.data.api_key }}{{ end }}" + error_on_missing_key = true +} + +env_template "EMAIL_CONFIG" { + contents = "{{ with secret \"kv/data/services/notifications/email/resend\" }}email:\n driver: {{ .Data.data.driver | toJSON }}\n settings:\n api_key_env: EMAIL_API_KEY\n sender_address: {{ .Data.data.sender_address | toJSON }}\n sender_name: {{ .Data.data.sender_name | toJSON }}\n{{ end }}" + error_on_missing_key = true +} + +exec { + command = ["/usr/local/bin/start-service"] + restart_on_secret_changes = "always" + restart_stop_signal = "SIGTERM" +} diff --git a/dev/vault/server.hcl b/infra/local/vault/server.hcl similarity index 100% rename from dev/vault/server.hcl rename to infra/local/vault/server.hcl diff --git a/dev/start-vault.sh b/infra/local/vault/start.sh similarity index 100% rename from dev/start-vault.sh rename to infra/local/vault/start.sh From 9d57a65010ac0b5eb97aaeb8fa84e56afec223dd Mon Sep 17 00:00:00 2001 From: Stephan D Date: Sat, 12 Sep 2026 23:36:47 +0200 Subject: [PATCH 06/11] Group Mongo bootstrap secrets under infrastructure ownership --- AGENTS.md | 2 +- docs/implementation.md | 8 ++++++++ docs/local-development.md | 10 +++++----- infra/local/scripts/bootstrap.sh | 6 +++--- infra/local/scripts/common.sh | 2 +- infra/local/scripts/secrets.sh | 1 + infra/local/scripts/verify.sh | 6 +++--- 7 files changed, 22 insertions(+), 13 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 8fc574f..8b145d7 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -176,7 +176,7 @@ Keep infrastructure Dockerfiles under `infra/images/` and local Compose, service configuration and shell tooling under `infra/local/`. Keep product source under `backend/` and `frontend/`. Organize Vault KV paths by owning service under `services//`, infrastructure under `infrastructure//`, -and bootstrap administration under `bootstrap/`. Caller credentials +and bootstrap administration under `infrastructure//bootstrap`. Caller credentials belong to the receiving service at `services//callers/`. Use Caddy for local browser HTTPS and certificate management. Treat the private local Docker network as trusted; do not add internal gRPC TLS or Vault-managed diff --git a/docs/implementation.md b/docs/implementation.md index 349e31d..b7955f8 100644 --- a/docs/implementation.md +++ b/docs/implementation.md @@ -743,3 +743,11 @@ and 1,083 expected HTTP responses, zero failures. Role TTLs were restored and temporary BFF/frontend containers removed. `make up` intentionally stops at the missing Resend credential check; no real email, optional security scan or merge was performed. Earlier supplementary legacy test limitations remain recorded. + +Mongo bootstrap credentials now live at `kv/infrastructure/mongo/bootstrap`, +consistent with component ownership. The live `kv/bootstrap/mongo` value was +copied, compared and archived before removing the old path. Bootstrap and admin +checks read the new path; application identities are denied both paths. Shell +checks and live infrastructure validation (including Mongo transactions, lease +renewal/revocation and obsolete-path absence) passed. `make up` still stops at the +expected missing-Resend check. No backend code or product behavior changed. diff --git a/docs/local-development.md b/docs/local-development.md index 5639c9d..fe030fc 100644 --- a/docs/local-development.md +++ b/docs/local-development.md @@ -41,10 +41,9 @@ kv/ │ └── notifications/ │ ├── callers/bff # token; read by BFF and notifications │ └── email/resend # driver, api_key, sender_address, sender_name -├── infrastructure/ -│ └── nats/auth # token; read by NATS and notifications -└── bootstrap/ - └── mongo # local administrator and replica-set bootstrap material +└── infrastructure/ + ├── nats/auth # token; read by NATS and notifications + └── mongo/bootstrap # local administrator and replica-set bootstrap material ``` Every secret has an owning service or infrastructure component. Service policies @@ -130,7 +129,8 @@ still interrupt work. Repository consumers use `source.Source` operation scopes; ## Migration and persistence Bootstrap migrates the old flat `bff`, `caller`, `email` and `nats` paths into the -hierarchy above. It copies with CAS protection, compares values, and refuses to +hierarchy above, and moves `bootstrap/mongo` to `infrastructure/mongo/bootstrap`. +It copies with CAS protection, compares values, and refuses to overwrite conflicting destinations. After consumers use the new paths, it removes old KV metadata and obsolete `tls`, `frontend-tls`, `notifications-tls` entries. Readable old version history and metadata are archived privately in diff --git a/infra/local/scripts/bootstrap.sh b/infra/local/scripts/bootstrap.sh index 33e3585..3a5ecc7 100644 --- a/infra/local/scripts/bootstrap.sh +++ b/infra/local/scripts/bootstrap.sh @@ -28,11 +28,11 @@ mounts() { fi } provision_mongo() { - if [ "$(kv bootstrap/mongo)" = null ]; then + if [ "$(kv infrastructure/mongo/bootstrap)" = null ]; then random > "$WORK/password"; random 384 > "$WORK/keyfile"; random > "$WORK/manager" - jq -n --rawfile password "$WORK/password" --rawfile keyfile "$WORK/keyfile" --rawfile manager "$WORK/manager" '{password:($password|rtrimstr("\n")),keyfile:($keyfile|rtrimstr("\n")),manager_password:($manager|rtrimstr("\n"))}' | put bootstrap/mongo + jq -n --rawfile password "$WORK/password" --rawfile keyfile "$WORK/keyfile" --rawfile manager "$WORK/manager" '{password:($password|rtrimstr("\n")),keyfile:($keyfile|rtrimstr("\n")),manager_password:($manager|rtrimstr("\n"))}' | put infrastructure/mongo/bootstrap fi - kv bootstrap/mongo > "$WORK/mongo-secret" + kv infrastructure/mongo/bootstrap > "$WORK/mongo-secret" jq -r '.password' "$WORK/mongo-secret" | write "$STATE/mongo/password" jq -r '.keyfile' "$WORK/mongo-secret" | write "$STATE/mongo/keyfile" compose up -d mongo diff --git a/infra/local/scripts/common.sh b/infra/local/scripts/common.sh index 522bfb0..8d8c004 100644 --- a/infra/local/scripts/common.sh +++ b/infra/local/scripts/common.sh @@ -56,7 +56,7 @@ ensure() ( ) # Never forward mongosh errors: authentication/command errors may include secrets. mongo() { compose exec -T mongo mongosh --quiet --file /dev/stdin > "$WORK/mongo-result" 2> "$WORK/mongo-error" || return 1; cat "$WORK/mongo-result"; } -admin_auth() { kv bootstrap/mongo | jq -r '"const admin=db.getSiblingDB(\"admin\"); admin.auth(\"local-admin\","+(.password|tojson)+");"'; } +admin_auth() { kv infrastructure/mongo/bootstrap | jq -r '"const admin=db.getSiblingDB(\"admin\"); admin.auth(\"local-admin\","+(.password|tojson)+");"'; } email_valid() { jq -e '(.api_key|type=="string") and (.api_key|test("\\S")) and (.sender_address|type=="string") and (.sender_address|test("\\S")) and .sender_address!="notifications@localhost.test"' >/dev/null; } export_ca() { compose exec -T frontend cat /data/caddy/pki/authorities/local/root.crt | write "$STATE/ca.pem" 644 diff --git a/infra/local/scripts/secrets.sh b/infra/local/scripts/secrets.sh index 264a1d2..d39b7cd 100644 --- a/infra/local/scripts/secrets.sh +++ b/infra/local/scripts/secrets.sh @@ -5,6 +5,7 @@ bff services/bff/jwt caller services/notifications/callers/bff email services/notifications/email/resend nats infrastructure/nats/auth +bootstrap/mongo infrastructure/mongo/bootstrap PATHS } # Archive readable version history privately before removing an obsolete KV path. diff --git a/infra/local/scripts/verify.sh b/infra/local/scripts/verify.sh index c378101..9441e1c 100644 --- a/infra/local/scripts/verify.sh +++ b/infra/local/scripts/verify.sh @@ -6,7 +6,7 @@ policies() ( directory="$STATE/agents/$service" login=$(jq -n --rawfile role "$directory/role-id" --rawfile secret "$directory/secret-id" '{role_id:($role|rtrimstr("\n")),secret_id:($secret|rtrimstr("\n"))}' | vault auth/approle/login POST) identity=$(printf '%s' "$login" | jq -er '.auth.client_token') - printf '%s' "$login" | jq '{token:.auth.client_token,paths:["kv/data/bootstrap/mongo","sys/init","kv/data/bff","kv/data/caller","kv/data/email","kv/data/nats","kv/data/tls"]}' | vault sys/capabilities POST | jq -e '.data | all(.[]; .==["deny"])' >/dev/null || fail 'Service can access bootstrap credentials' + printf '%s' "$login" | jq '{token:.auth.client_token,paths:["kv/data/bootstrap/mongo","kv/data/infrastructure/mongo/bootstrap","sys/init","kv/data/bff","kv/data/caller","kv/data/email","kv/data/nats","kv/data/tls"]}' | vault sys/capabilities POST | jq -e '.data | all(.[]; .==["deny"])' >/dev/null || fail 'Service can access bootstrap credentials' VAULT_TOKEN=$(printf '%s' "$login" | jq -er '.auth.client_token') printf '{}' | vault auth/token/revoke-self POST >/dev/null VAULT_TOKEN=$root_token @@ -50,10 +50,10 @@ product() { echo 'PASS: six healthy services, frontend and proxied BFF use verified TLS; no email sent' } layout() { - for old in bff caller email nats tls frontend-tls notifications-tls; do + for old in bootstrap/mongo bff caller email nats tls frontend-tls notifications-tls; do [ "$(vault "kv/metadata/$old" GET yes)" = null ] || fail "Obsolete KV entry remains: $old" done - for path in services/bff/jwt services/notifications/callers/bff services/notifications/email/resend infrastructure/nats/auth bootstrap/mongo; do + for path in services/bff/jwt services/notifications/callers/bff services/notifications/email/resend infrastructure/nats/auth infrastructure/mongo/bootstrap; do vault "kv/metadata/$path" | jq -e '.data.current_version>0' >/dev/null || fail "Missing canonical KV entry: $path" done echo 'PASS: canonical Vault hierarchy present; obsolete flat and TLS entries absent' From 1fe173397fdf5e5d5875a413ca1014f7e91b1185 Mon Sep 17 00:00:00 2001 From: Stephan D Date: Sat, 12 Sep 2026 23:45:58 +0200 Subject: [PATCH 07/11] Keep email runtime configuration out of Vault --- AGENTS.md | 2 ++ README.md | 3 ++- docs/implementation.md | 15 ++++++++++++++ docs/local-development.md | 29 ++++++++++++++++++++------- infra/local/notifications/runtime.yml | 6 ++++++ infra/local/scripts/bootstrap.sh | 14 ++++++------- infra/local/scripts/common.sh | 2 +- infra/local/scripts/secrets.sh | 13 ++++++++++++ infra/local/scripts/start-service.sh | 5 +---- infra/local/scripts/test.sh | 15 ++++++++++++++ infra/local/scripts/verify.sh | 1 + infra/local/vault/notifications.hcl | 4 ---- 12 files changed, 84 insertions(+), 25 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 8b145d7..b09c0e4 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -181,3 +181,5 @@ belong to the receiving service at `services//callers/`. Use Caddy for local browser HTTPS and certificate management. Treat the private local Docker network as trusted; do not add internal gRPC TLS or Vault-managed local TLS certificates without an explicit requirement. +Store only secret values in Vault. Driver selection, sender identities, endpoints +and other non-secret runtime settings belong in service configuration files. diff --git a/README.md b/README.md index 054aa99..df96951 100644 --- a/README.md +++ b/README.md @@ -40,7 +40,8 @@ make build && make up This builds the product images and starts an isolated Vault, authenticated MongoDB replica set and NATS JetStream. First startup intentionally stops after bootstrap -until you set the Resend API key and verified sender in Vault's `kv/services/notifications/email/resend` secret. +until you set `api_key` in Vault's `kv/services/notifications/email/resend` secret +and the verified sender in `infra/local/notifications/runtime.yml`. Then rerun `make up` to start notifications, BFF and the frontend at [https://localhost:5126](https://localhost:5126). diff --git a/docs/implementation.md b/docs/implementation.md index b7955f8..bf0f3a8 100644 --- a/docs/implementation.md +++ b/docs/implementation.md @@ -751,3 +751,18 @@ checks read the new path; application identities are denied both paths. Shell checks and live infrastructure validation (including Mongo transactions, lease renewal/revocation and obsolete-path absence) passed. `make up` still stops at the expected missing-Resend check. No backend code or product behavior changed. + +Email configuration is now separated from credentials. The current Vault secret +`kv/services/notifications/email/resend` contains only `api_key`; `driver`, +`sender_address` and `sender_name` live in +`infra/local/notifications/runtime.yml`. Sender address is empty until configured. +Agent injects only `EMAIL_API_KEY`, and notifications loads its runtime YAML +directly. `make email` prompts only for the API key. The migration preserves the +key with a CAS-checked update, archives the prior readable version, and refuses +unknown fields rather than discarding them. The live Vault entry was normalized. + +Validation passed: image build, shell/migration tests, live infrastructure checks +(including key-only secret shape and Mongo transactions/leases), and diff checks. +`make up` intentionally fails for the still-missing API key. Verified sender must +also be configured in runtime YAML before rebuilding and starting notifications. +No backend source changed and no email was sent. diff --git a/docs/local-development.md b/docs/local-development.md index fe030fc..3c0ee72 100644 --- a/docs/local-development.md +++ b/docs/local-development.md @@ -40,7 +40,7 @@ kv/ │ ├── bff/jwt # jwt │ └── notifications/ │ ├── callers/bff # token; read by BFF and notifications -│ └── email/resend # driver, api_key, sender_address, sender_name +│ └── email/resend # api_key only └── infrastructure/ ├── nats/auth # token; read by NATS and notifications └── mongo/bootstrap # local administrator and replica-set bootstrap material @@ -65,22 +65,37 @@ Open [Vault](http://127.0.0.1:18200) and edit | Field | Value | | --- | --- | -| `driver` | `resend` | | `api_key` | Your Resend API key | -| `sender_address` | Your verified sender address | -| `sender_name` | Your display name | + +The driver and sender settings belong in `infra/local/notifications/runtime.yml`: + +```yaml +email: + driver: resend + settings: + api_key_env: EMAIL_API_KEY + sender_address: "your-verified-sender@example.com" + sender_name: "SXCHNGE" +``` + +Set your verified address in that file and rebuild. It is intentionally empty by +default, so notifications cannot start with an unconfigured sender. Vault Agent +injects only the API key; it does not generate email configuration. Local administrator login material is in private `.local/dev/admin/init.json`. Product containers cannot access it. Alternatively use the hidden terminal prompt: ```sh make email -make up +make build && make up ``` No placeholder key or delivery fallback is installed. Startup validates presence; -it does not send an email or validate domain ownership with Resend. Updating email -settings causes Agent to restart notifications. Existing values survive bootstrap. +it does not send an email or validate domain ownership with Resend. Updating the +API key causes Agent to restart notifications. Driver/sender changes require an +image rebuild. Existing API keys survive bootstrap. Migration removes legacy +driver/sender fields from the current Vault secret with a version-checked write; +the preceding version is preserved in Vault history and the private archive. ## Browser HTTPS and internal connections diff --git a/infra/local/notifications/runtime.yml b/infra/local/notifications/runtime.yml index 7e4b060..80e5e78 100644 --- a/infra/local/notifications/runtime.yml +++ b/infra/local/notifications/runtime.yml @@ -19,6 +19,12 @@ messaging: templates: "/etc/sxchnge/templates.json" callers: product: "SXCHNGE_NOTIFICATIONS_PRODUCT_TOKEN" +email: + driver: resend + settings: + api_key_env: EMAIL_API_KEY + sender_address: "" # Set your verified Resend sender here, then run make build. + sender_name: "SXCHNGE" # Use replicas: 3 and environment: production for a three-node NATS cluster. # mode: ops validates resources without modifying them; app provisions them. queue: diff --git a/infra/local/scripts/bootstrap.sh b/infra/local/scripts/bootstrap.sh index 3a5ecc7..e5dd618 100644 --- a/infra/local/scripts/bootstrap.sh +++ b/infra/local/scripts/bootstrap.sh @@ -98,20 +98,19 @@ up() { random | jq -Rs --arg field "$field" '{($field):rtrimstr("\n")}' | put "$path" fi done - printf '{"driver":"resend","api_key":"","sender_address":"notifications@localhost.test","sender_name":"SXCHNGE local"}' | ensure services/notifications/email/resend + printf '{"api_key":""}' | ensure services/notifications/email/resend provision_mongo provision_agents compose up -d --wait --wait-timeout 120 nats retire_secrets - kv services/notifications/email/resend | email_valid || fail 'Vault kv/services/notifications/email/resend needs a Resend api_key and verified sender_address. Run make email, then make up.' + normalize_email_secret + kv services/notifications/email/resend | email_valid || fail 'Vault kv/services/notifications/email/resend needs api_key. Run make email; configure the verified sender in infra/local/notifications/runtime.yml, then make build && make up.' compose up -d --wait --wait-timeout 180 export_ca echo 'Local stack ready: https://localhost:5126; Vault: http://127.0.0.1:18200' } email() { existing - printf 'Verified Resend sender address: ' >&2 - IFS= read -r address printf 'Resend API key (hidden): ' >&2 tty_state=$(stty -g) trap 'stty "$tty_state"; cleanup' EXIT @@ -120,9 +119,8 @@ email() { stty "$tty_state" printf '\n' >&2 [ -n "$key" ] || fail 'A key is required' - case "$address" in *@*) ;; *) fail 'A sender email address is required';; esac printf '%s' "$key" > "$WORK/email-key" - printf '%s' "$address" > "$WORK/email-address" - jq -n --rawfile key "$WORK/email-key" --rawfile address "$WORK/email-address" '{driver:"resend",api_key:$key,sender_address:$address,sender_name:"SXCHNGE"}' | put services/notifications/email/resend - echo 'Email configuration saved in Vault. Run make up.' + normalize_email_secret + jq -n --rawfile key "$WORK/email-key" '{api_key:$key}' | put services/notifications/email/resend + echo 'API key saved in Vault. Set the verified sender in infra/local/notifications/runtime.yml, then make build && make up.' } diff --git a/infra/local/scripts/common.sh b/infra/local/scripts/common.sh index 8d8c004..b147db0 100644 --- a/infra/local/scripts/common.sh +++ b/infra/local/scripts/common.sh @@ -57,7 +57,7 @@ ensure() ( # Never forward mongosh errors: authentication/command errors may include secrets. mongo() { compose exec -T mongo mongosh --quiet --file /dev/stdin > "$WORK/mongo-result" 2> "$WORK/mongo-error" || return 1; cat "$WORK/mongo-result"; } admin_auth() { kv infrastructure/mongo/bootstrap | jq -r '"const admin=db.getSiblingDB(\"admin\"); admin.auth(\"local-admin\","+(.password|tojson)+");"'; } -email_valid() { jq -e '(.api_key|type=="string") and (.api_key|test("\\S")) and (.sender_address|type=="string") and (.sender_address|test("\\S")) and .sender_address!="notifications@localhost.test"' >/dev/null; } +email_valid() { jq -e '(.api_key|type=="string") and (.api_key|test("\\S"))' >/dev/null; } export_ca() { compose exec -T frontend cat /data/caddy/pki/authorities/local/root.crt | write "$STATE/ca.pem" 644 } diff --git a/infra/local/scripts/secrets.sh b/infra/local/scripts/secrets.sh index d39b7cd..1989442 100644 --- a/infra/local/scripts/secrets.sh +++ b/infra/local/scripts/secrets.sh @@ -55,3 +55,16 @@ retire_secrets() ( # and history are retained until the owner explicitly chooses to destroy them. echo 'PASS: Vault secrets migrated and verified; obsolete KV entries retired' ) + +# Only credentials belong in Vault; driver/sender settings are runtime YAML. +normalize_email_secret() ( + path=services/notifications/email/resend + vault "kv/data/$path" GET yes > "$WORK/email-current" + [ "$(cat "$WORK/email-current")" != null ] || exit 0 + jq -e '(.data.data|keys)-["api_key","driver","sender_address","sender_name"]|length==0' "$WORK/email-current" >/dev/null || fail 'Unexpected Resend secret fields; refusing to discard them' + if jq -e '.data.data|keys==["api_key"]' "$WORK/email-current" >/dev/null; then exit 0; fi + archive_secret "$path" + jq '{data:{api_key:(.data.data.api_key // "")},options:{cas:.data.metadata.version}}' "$WORK/email-current" | vault "kv/data/$path" POST >/dev/null + vault "kv/data/$path" | jq -e --slurpfile original "$WORK/email-current" '.data.data|keys==["api_key"] and .api_key==($original[0].data.data.api_key // "")' >/dev/null || fail 'Resend migration verification failed' + echo 'Resend secret now contains only api_key; configure driver/sender in notifications runtime.yml' +) diff --git a/infra/local/scripts/start-service.sh b/infra/local/scripts/start-service.sh index dec163b..fb1e6bd 100755 --- a/infra/local/scripts/start-service.sh +++ b/infra/local/scripts/start-service.sh @@ -2,9 +2,6 @@ set -eu umask 077 if [ "$SXCHNGE_SERVICE" = notifications ]; then - printf '%s\n' "$EMAIL_CONFIG" > /run/sxchnge/email.yml - unset EMAIL_CONFIG - cat /etc/sxchnge/notifications.yml /run/sxchnge/email.yml > /run/sxchnge/config.yml - exec service -config.file=/run/sxchnge/config.yml + exec service -config.file=/etc/sxchnge/notifications.yml fi exec service -config.file=/etc/sxchnge/bff.yml diff --git a/infra/local/scripts/test.sh b/infra/local/scripts/test.sh index 7fb8a92..2dd2f13 100644 --- a/infra/local/scripts/test.sh +++ b/infra/local/scripts/test.sh @@ -59,3 +59,18 @@ printf '{"api_key":"different-fixture"}' > "$WORK/destination" if (migrate_secrets) >/dev/null 2>&1; then fail 'Migration overwrote conflicting destination'; fi jq -e '.api_key=="different-fixture"' "$WORK/destination" >/dev/null echo 'PASS: secret migration is repeatable and refuses conflicting destination values' +# Removing non-secret fields retains the key and checks the current KV version. +vault() { + if [ "${2:-GET}" = GET ]; then cat "$WORK/email-fixture"; else + jq -e '.options.cas==42 and .data=={api_key:"preserved-fixture-key"}' > /dev/null || return 1 + printf '{"data":{"metadata":{"version":43},"data":{"api_key":"preserved-fixture-key"}}}' > "$WORK/email-fixture" + fi +} +printf '{"data":{"metadata":{"version":42},"data":{"api_key":"preserved-fixture-key","driver":"resend","sender_address":"sender@example.test","sender_name":"Fixture"}}}' > "$WORK/email-fixture" +normalize_email_secret +normalize_email_secret +jq -e '.data.data=={api_key:"preserved-fixture-key"}' "$WORK/email-fixture" >/dev/null +printf '{"data":{"metadata":{"version":43},"data":{"api_key":"preserved-fixture-key","unexpected":"must-not-drop"}}}' > "$WORK/email-fixture" +if (normalize_email_secret) >/dev/null 2>&1; then fail 'Unknown secret field was silently dropped'; fi +if grep -q EMAIL_CONFIG infra/local/vault/notifications.hcl infra/local/scripts/start-service.sh; then fail 'Email configuration is still rendered from Vault'; fi +echo 'PASS: email migration preserves API key, removes configuration fields and rejects unknown fields' diff --git a/infra/local/scripts/verify.sh b/infra/local/scripts/verify.sh index 9441e1c..d171fe4 100644 --- a/infra/local/scripts/verify.sh +++ b/infra/local/scripts/verify.sh @@ -50,6 +50,7 @@ product() { echo 'PASS: six healthy services, frontend and proxied BFF use verified TLS; no email sent' } layout() { + kv services/notifications/email/resend | jq -e 'keys==["api_key"]' >/dev/null || fail 'Non-secret email configuration remains in Vault' for old in bootstrap/mongo bff caller email nats tls frontend-tls notifications-tls; do [ "$(vault "kv/metadata/$old" GET yes)" = null ] || fail "Obsolete KV entry remains: $old" done diff --git a/infra/local/vault/notifications.hcl b/infra/local/vault/notifications.hcl index bb1a912..0708ff6 100644 --- a/infra/local/vault/notifications.hcl +++ b/infra/local/vault/notifications.hcl @@ -31,10 +31,6 @@ env_template "EMAIL_API_KEY" { error_on_missing_key = true } -env_template "EMAIL_CONFIG" { - contents = "{{ with secret \"kv/data/services/notifications/email/resend\" }}email:\n driver: {{ .Data.data.driver | toJSON }}\n settings:\n api_key_env: EMAIL_API_KEY\n sender_address: {{ .Data.data.sender_address | toJSON }}\n sender_name: {{ .Data.data.sender_name | toJSON }}\n{{ end }}" - error_on_missing_key = true -} exec { command = ["/usr/local/bin/start-service"] From caad7a3d5bf5515509df1611b13edae1155b69a9 Mon Sep 17 00:00:00 2001 From: Stephan D Date: Sat, 12 Sep 2026 23:59:30 +0200 Subject: [PATCH 08/11] Remove internal notification caller tokens --- AGENTS.md | 5 ++- backend/notifications/README.md | 3 +- backend/notifications/app/config.go | 28 +++++--------- backend/notifications/app/config_test.go | 8 ---- backend/notifications/app/run.go | 7 +--- backend/notifications/config.example.yml | 2 - backend/notifications/interface/README.md | 6 +-- .../interface/proto/v1/interface.proto | 4 +- backend/notifications/service/grpc_test.go | 9 +---- .../notifications/service/integration_test.go | 11 ++---- backend/notifications/transport/server.go | 38 ++----------------- .../notifications/transport/server_test.go | 11 ++---- backend/product/app/runtime/bootstrap_test.go | 1 - backend/product/config.example.yml | 1 - backend/product/notifications/client.go | 15 +++----- backend/product/notifications/client_test.go | 37 +++++++++--------- docs/auth-notifications.md | 12 +++--- docs/implementation.md | 22 +++++++++++ docs/local-development.md | 10 +++-- infra/local/bff/runtime.yml | 1 - infra/local/notifications/runtime.yml | 2 - infra/local/scripts/bootstrap.sh | 6 +-- infra/local/scripts/secrets.sh | 5 +-- infra/local/scripts/test.sh | 19 ++++++++++ infra/local/scripts/verify.sh | 6 +-- infra/local/vault/bff.hcl | 4 -- infra/local/vault/notifications.hcl | 4 -- 27 files changed, 112 insertions(+), 165 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index b09c0e4..03233d3 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -176,8 +176,9 @@ Keep infrastructure Dockerfiles under `infra/images/` and local Compose, service configuration and shell tooling under `infra/local/`. Keep product source under `backend/` and `frontend/`. Organize Vault KV paths by owning service under `services//`, infrastructure under `infrastructure//`, -and bootstrap administration under `infrastructure//bootstrap`. Caller credentials -belong to the receiving service at `services//callers/`. +and bootstrap administration under `infrastructure//bootstrap`. +Do not add BFF-to-notifications caller authentication on the trusted private network. +Preserve NATS authentication. Use Caddy for local browser HTTPS and certificate management. Treat the private local Docker network as trusted; do not add internal gRPC TLS or Vault-managed local TLS certificates without an explicit requirement. diff --git a/backend/notifications/README.md b/backend/notifications/README.md index 6cb0f45..645c9d2 100644 --- a/backend/notifications/README.md +++ b/backend/notifications/README.md @@ -22,8 +22,7 @@ subject, text and HTML, bounded timeouts and safe error classification. Requires Go 1.27, protoc, an SXCHNGE MongoDB database and a NATS server with JetStream enabled. Copy `config.example.yml` to an untracked `config.yml` here. -Set `SXCHNGE_MONGO_URI`, `SXCHNGE_NATS_URL`, a unique token of at least 32 bytes for -each configured caller, and the selected email provider's API key environment +Set `SXCHNGE_MONGO_URI`, `SXCHNGE_NATS_URL` and the selected email provider's API key environment variable. Configure a verified sender address and template catalog. Paths are relative to the process working directory. diff --git a/backend/notifications/app/config.go b/backend/notifications/app/config.go index e4f5dff..7a3fe32 100644 --- a/backend/notifications/app/config.go +++ b/backend/notifications/app/config.go @@ -4,7 +4,6 @@ import ( "bytes" "io" "os" - "strings" "git.sendico.io/tech/sxchnge/notifications/bus" "git.sendico.io/tech/sxchnge/notifications/email" @@ -15,15 +14,14 @@ import ( ) type Config struct { - Queue bus.Config `yaml:"queue"` - Messaging messaging.Config `yaml:"messaging"` - Listen string `yaml:"listen"` - TLSCert string `yaml:"tls_cert"` - TLSKey string `yaml:"tls_key"` - Database db.Config `yaml:"database"` - Templates string `yaml:"templates"` - Callers map[string]string `yaml:"callers"` // caller -> token environment variable - Email email.Config `yaml:"email"` + Queue bus.Config `yaml:"queue"` + Messaging messaging.Config `yaml:"messaging"` + Listen string `yaml:"listen"` + TLSCert string `yaml:"tls_cert"` + TLSKey string `yaml:"tls_key"` + Database db.Config `yaml:"database"` + Templates string `yaml:"templates"` + Email email.Config `yaml:"email"` } func Load(path string) (Config, error) { @@ -40,7 +38,7 @@ func Load(path string) (Config, error) { if err = decoder.Decode(new(any)); err != io.EOF { return c, merrors.InvalidArgument("unexpected configuration content") } - if c.Listen == "" || c.Database.Driver != db.Mongo || c.Messaging.Driver != messaging.BBNats || c.Templates == "" || len(c.Callers) == 0 { + if c.Listen == "" || c.Database.Driver != db.Mongo || c.Messaging.Driver != messaging.BBNats || c.Templates == "" { return c, merrors.InvalidArgument("missing notification configuration") } if (c.TLSCert == "") != (c.TLSKey == "") { @@ -49,13 +47,5 @@ func Load(path string) (Config, error) { if err = c.Queue.Validate(); err != nil { return c, err } - seen := map[string]bool{} - for caller, env := range c.Callers { - token := os.Getenv(env) - if strings.TrimSpace(caller) == "" || len(strings.TrimSpace(token)) < 32 || seen[token] { - return c, merrors.InvalidArgument("caller credentials must be unique and at least 32 bytes") - } - seen[token] = true - } return c, nil } diff --git a/backend/notifications/app/config_test.go b/backend/notifications/app/config_test.go index 5643ee7..424fbb7 100644 --- a/backend/notifications/app/config_test.go +++ b/backend/notifications/app/config_test.go @@ -3,12 +3,10 @@ package app import ( "os" "path/filepath" - "strings" "testing" ) func TestExampleConfigAndSecrets(t *testing.T) { - t.Setenv("SXCHNGE_NOTIFICATIONS_PRODUCT_TOKEN", strings.Repeat("a", 32)) c, err := Load("../config.example.yml") if err != nil { t.Fatal(err) @@ -16,11 +14,6 @@ func TestExampleConfigAndSecrets(t *testing.T) { if c.Email.Driver != "resend" || c.Messaging.Driver != "NATS" || c.Database.Driver != "mongodb" { t.Fatal("wrong driver configuration") } - t.Setenv("SXCHNGE_NOTIFICATIONS_PRODUCT_TOKEN", "") - if _, err = Load("../config.example.yml"); err == nil { - t.Fatal("missing credentials accepted") - } - t.Setenv("SXCHNGE_NOTIFICATIONS_PRODUCT_TOKEN", strings.Repeat("a", 32)) data, err := os.ReadFile("../config.example.yml") if err != nil { t.Fatal(err) @@ -35,7 +28,6 @@ func TestExampleConfigAndSecrets(t *testing.T) { } func TestPlaintextConfigAndIncompleteTLS(t *testing.T) { - t.Setenv("SXCHNGE_NOTIFICATIONS_PRODUCT_TOKEN", strings.Repeat("a", 32)) c, err := Load("../config.example.yml") if err != nil { t.Fatal(err) diff --git a/backend/notifications/app/run.go b/backend/notifications/app/run.go index 08b8110..118b0bb 100644 --- a/backend/notifications/app/run.go +++ b/backend/notifications/app/run.go @@ -4,7 +4,6 @@ import ( "context" "crypto/tls" "net" - "os" "sync" "time" @@ -63,11 +62,7 @@ func Run(ctx context.Context, c Config, logger mlogger.Logger) error { return err } defer broker.Close() - tokens := map[string]string{} - for caller, env := range c.Callers { - tokens[caller] = os.Getenv(env) - } - server := grpc.NewServer(append(options, grpc.UnaryInterceptor(transport.Authenticate(tokens)), grpc.MaxRecvMsgSize(128<<10), grpc.MaxConcurrentStreams(64))...) + server := grpc.NewServer(append(options, grpc.MaxRecvMsgSize(128<<10), grpc.MaxConcurrentStreams(64))...) pb.RegisterNotificationsServiceServer(server, transport.New(svc)) listener, err := net.Listen("tcp", c.Listen) if err != nil { diff --git a/backend/notifications/config.example.yml b/backend/notifications/config.example.yml index 02343a3..ff763e2 100644 --- a/backend/notifications/config.example.yml +++ b/backend/notifications/config.example.yml @@ -12,8 +12,6 @@ messaging: max_reconnects: -1 reconnect_wait: 2 templates: "templates.example.json" -callers: - product: "SXCHNGE_NOTIFICATIONS_PRODUCT_TOKEN" email: driver: resend # sendgrid or resend settings: diff --git a/backend/notifications/interface/README.md b/backend/notifications/interface/README.md index 52d2de9..d10824c 100644 --- a/backend/notifications/interface/README.md +++ b/backend/notifications/interface/README.md @@ -20,14 +20,14 @@ Internal Go consumers use the module `git.sendico.io/tech/sxchnge/notifications/interface/notifications/v1`. Within this checkout, use a local replace to this directory and generate before building. -Calls require TLS and `authorization: Bearer ` metadata. The service -maps tokens to configured caller identities. Callers authorize user/destination +Calls use the trusted private network without caller authentication. The service +uses the existing product storage namespace. Callers authorize user/destination ownership and preferences before sending; this contract is not a public user API. Parameters are string values keyed by template parameter name. The entire request is validated before persistence. An identical idempotency key/request returns the original resource, even after completion or expiry. A changed request conflicts. -Keys are scoped to the authenticated caller and retained at least seven days; +Keys are scoped to the product namespace and retained at least seven days; Mongo TTL deletion is asynchronous. Expiry defaults to 24 hours and cannot exceed 24 hours. One email target is accepted; other declared channels return UNIMPLEMENTED. diff --git a/backend/notifications/interface/proto/v1/interface.proto b/backend/notifications/interface/proto/v1/interface.proto index e8821de..f962235 100644 --- a/backend/notifications/interface/proto/v1/interface.proto +++ b/backend/notifications/interface/proto/v1/interface.proto @@ -3,13 +3,13 @@ package sxchnge.notifications.v1; option go_package = "git.sendico.io/tech/sxchnge/notifications/interface/notifications/v1;notificationsv1"; import "google/protobuf/timestamp.proto"; -// Internal service: authenticate callers over TLS. Send durably queues work. +// Internal service on the trusted private network; no caller authentication. Send durably queues work. service NotificationsService { rpc Send(SendRequest) returns (SendResponse); rpc Get(GetRequest) returns (GetResponse); } message SendRequest { - string idempotency_key = 1; // Required; scoped to authenticated caller, retained seven days. + string idempotency_key = 1; // Required; scoped to the product namespace, retained seven days. string user_id = 2; // Caller resolves and authorizes recipient ownership. string template_id = 3; string template_version = 4; diff --git a/backend/notifications/service/grpc_test.go b/backend/notifications/service/grpc_test.go index 93f7f15..b46e3ca 100644 --- a/backend/notifications/service/grpc_test.go +++ b/backend/notifications/service/grpc_test.go @@ -1,7 +1,6 @@ package service_test import ( - "context" "crypto/ecdsa" "crypto/elliptic" "crypto/rand" @@ -17,10 +16,9 @@ import ( "git.sendico.io/tech/sxchnge/notifications/transport" "google.golang.org/grpc" "google.golang.org/grpc/credentials" - "google.golang.org/grpc/metadata" ) -// Exercise authenticated TLS transport against the real service and storage. +// Exercise optional TLS transport against the real service and storage. func grpcClient(t *testing.T, svc *service.Service) pb.NotificationsServiceClient { t.Helper() key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader) @@ -42,7 +40,7 @@ func grpcClient(t *testing.T, svc *service.Service) pb.NotificationsServiceClien if err != nil { t.Fatal(err) } - server := grpc.NewServer(grpc.Creds(credentials.NewTLS(&tls.Config{MinVersion: tls.VersionTLS12, Certificates: []tls.Certificate{{Certificate: [][]byte{der}, PrivateKey: key}}})), grpc.UnaryInterceptor(transport.Authenticate(map[string]string{"product": "product-test-token", "other": "other-test-token"}))) + server := grpc.NewServer(grpc.Creds(credentials.NewTLS(&tls.Config{MinVersion: tls.VersionTLS12, Certificates: []tls.Certificate{{Certificate: [][]byte{der}, PrivateKey: key}}}))) pb.RegisterNotificationsServiceServer(server, transport.New(svc)) go func() { _ = server.Serve(listener) }() t.Cleanup(server.Stop) @@ -53,6 +51,3 @@ func grpcClient(t *testing.T, svc *service.Service) pb.NotificationsServiceClien t.Cleanup(func() { _ = conn.Close() }) return pb.NewNotificationsServiceClient(conn) } -func callerContext(ctx context.Context, caller string) context.Context { - return metadata.NewOutgoingContext(ctx, metadata.Pairs("authorization", "Bearer "+caller+"-test-token")) -} diff --git a/backend/notifications/service/integration_test.go b/backend/notifications/service/integration_test.go index 599304e..0c3cfa7 100644 --- a/backend/notifications/service/integration_test.go +++ b/backend/notifications/service/integration_test.go @@ -26,8 +26,6 @@ import ( "github.com/testcontainers/testcontainers-go/modules/mongodb" "github.com/testcontainers/testcontainers-go/wait" "go.uber.org/zap" - "google.golang.org/grpc/codes" - "google.golang.org/grpc/status" ) type sender struct { @@ -233,11 +231,8 @@ func TestDurableDelivery(t *testing.T) { } client := grpcClient(t, svc) - rpcCtx := callerContext(ctx, "product") + rpcCtx := ctx input := &pb.SendRequest{IdempotencyKey: "jetstream", UserId: "u", TemplateId: "test", TemplateVersion: "1", Parameters: map[string]string{"name": "End to end & escaped"}, Targets: []*pb.Target{{Destination: &pb.Target_Email{Email: &pb.EmailTarget{Address: "user@example.com"}}}}} - if _, e := client.Send(ctx, input); status.Code(e) != codes.Unauthenticated { - t.Fatal("unauthenticated request accepted", e) - } sent, err := client.Send(rpcCtx, input) if err != nil { t.Fatal(err) @@ -246,8 +241,8 @@ func TestDurableDelivery(t *testing.T) { if err != nil || replay.Notification.Id != sent.Notification.Id { t.Fatal("RPC idempotency failed", err) } - if _, e := client.Get(callerContext(ctx, "other"), &pb.GetRequest{Id: sent.Notification.Id}); status.Code(e) != codes.NotFound { - t.Fatal("RPC caller isolation failed", e) + if _, e := client.Get(ctx, &pb.GetRequest{Id: sent.Notification.Id}); e != nil { + t.Fatal("RPC retrieval without credentials failed", e) } if err = broker.PublishDue(ctx); err != nil { t.Fatal(err) diff --git a/backend/notifications/transport/server.go b/backend/notifications/transport/server.go index 57b1fe7..818fee3 100644 --- a/backend/notifications/transport/server.go +++ b/backend/notifications/transport/server.go @@ -2,21 +2,15 @@ package transport import ( "context" - "crypto/sha256" - "crypto/subtle" "errors" - "strings" pb "git.sendico.io/tech/sxchnge/notifications/interface/notifications/v1" "git.sendico.io/tech/sxchnge/notifications/model" merrors "git.sendico.io/tech/sxchnge/shared/errors" - "google.golang.org/grpc" "google.golang.org/grpc/codes" - "google.golang.org/grpc/metadata" "google.golang.org/grpc/status" ) -type callerKey struct{} type API interface { Send(context.Context, string, model.Request) (*model.Notification, error) Get(context.Context, string, string) (*model.Notification, error) @@ -26,14 +20,14 @@ type Server struct { service API } +// The private endpoint uses the existing product storage namespace without caller authentication. func New(service API) *Server { return &Server{service: service} } func (s *Server) Send(ctx context.Context, r *pb.SendRequest) (*pb.SendResponse, error) { input, err := request(r) if err != nil { return nil, rpcError(err) } - caller, _ := ctx.Value(callerKey{}).(string) - n, err := s.service.Send(ctx, caller, input) + n, err := s.service.Send(ctx, "product", input) if err != nil { return nil, rpcError(err) } @@ -43,39 +37,13 @@ func (s *Server) Get(ctx context.Context, r *pb.GetRequest) (*pb.GetResponse, er if r == nil { return nil, status.Error(codes.InvalidArgument, "missing request") } - caller, _ := ctx.Value(callerKey{}).(string) - n, err := s.service.Get(ctx, caller, r.Id) + n, err := s.service.Get(ctx, "product", r.Id) if err != nil { return nil, rpcError(err) } return &pb.GetResponse{Notification: response(n)}, nil } -// Tokens are mapped to stable server-configured caller identities, never supplied IDs. -func Authenticate(tokens map[string]string) grpc.UnaryServerInterceptor { - hashes := map[string][32]byte{} - for caller, token := range tokens { - hashes[caller] = sha256.Sum256([]byte(token)) - } - return func(ctx context.Context, req any, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (any, error) { - md, _ := metadata.FromIncomingContext(ctx) - values := md.Get("authorization") - if len(values) != 1 || !strings.HasPrefix(values[0], "Bearer ") { - return nil, status.Error(codes.Unauthenticated, "invalid service credentials") - } - hash := sha256.Sum256([]byte(strings.TrimPrefix(values[0], "Bearer "))) - identity := "" - for caller, expected := range hashes { - if subtle.ConstantTimeCompare(hash[:], expected[:]) == 1 { - identity = caller - } - } - if identity == "" { - return nil, status.Error(codes.Unauthenticated, "invalid service credentials") - } - return handler(context.WithValue(ctx, callerKey{}, identity), req) - } -} func rpcError(err error) error { if errors.Is(err, context.Canceled) { return status.Error(codes.Canceled, "notification operation canceled") diff --git a/backend/notifications/transport/server_test.go b/backend/notifications/transport/server_test.go index f6cc70b..f6ea23a 100644 --- a/backend/notifications/transport/server_test.go +++ b/backend/notifications/transport/server_test.go @@ -15,7 +15,6 @@ import ( "google.golang.org/grpc" "google.golang.org/grpc/codes" "google.golang.org/grpc/credentials/insecure" - "google.golang.org/grpc/metadata" "google.golang.org/grpc/status" "google.golang.org/grpc/test/bufconn" ) @@ -31,10 +30,9 @@ func (fakeAPI) Send(_ context.Context, caller string, r model.Request) (*model.N func (fakeAPI) Get(context.Context, string, string) (*model.Notification, error) { return nil, merrors.NoData("private details") } -func TestGRPCAuthenticationAndMapping(t *testing.T) { +func TestGRPCWithoutAuthenticationAndMapping(t *testing.T) { listener := bufconn.Listen(1 << 20) - token := strings.Repeat("a", 32) - server := grpc.NewServer(grpc.UnaryInterceptor(Authenticate(map[string]string{"product": token}))) + server := grpc.NewServer() pb.RegisterNotificationsServiceServer(server, New(fakeAPI{})) go func() { _ = server.Serve(listener) }() defer server.Stop() @@ -45,10 +43,7 @@ func TestGRPCAuthenticationAndMapping(t *testing.T) { defer conn.Close() client := pb.NewNotificationsServiceClient(conn) req := &pb.SendRequest{UserId: "u", Targets: []*pb.Target{{Destination: &pb.Target_Email{Email: &pb.EmailTarget{Address: "user@example.com"}}}}} - if _, err = client.Send(context.Background(), req); status.Code(err) != codes.Unauthenticated { - t.Fatal(err) - } - ctx := metadata.NewOutgoingContext(context.Background(), metadata.Pairs("authorization", "Bearer "+token)) + ctx := context.Background() result, err := client.Send(ctx, req) if err != nil { t.Fatal(err) diff --git a/backend/product/app/runtime/bootstrap_test.go b/backend/product/app/runtime/bootstrap_test.go index bbacf2c..9b66a61 100644 --- a/backend/product/app/runtime/bootstrap_test.go +++ b/backend/product/app/runtime/bootstrap_test.go @@ -48,7 +48,6 @@ func TestProfileStartsOnEmptyDatabase(t *testing.T) { t.Setenv("SXCHNGE_API_PROTOCOL", "http") t.Setenv("SXCHNGE_API_ENDPOINT", "/api/v1") t.Setenv("SXCHNGE_JWT_SECRET", "test-only-signing-key-at-least-32-bytes") - t.Setenv("SXCHNGE_NOTIFICATIONS_TOKEN", "test-notification-token-at-least-32-bytes") example.API.Notifications.CAFile = "" // This startup test makes no notification RPCs. root, routes := chi.NewRouter(), chi.NewRouter() runtime, err := sharedruntime.CreateAPI(ctx, zap.NewNop(), &example.API, &example.API, factory, root, routes, false, profile.Runtime()) diff --git a/backend/product/config.example.yml b/backend/product/config.example.yml index 2a49aef..b455b11 100644 --- a/backend/product/config.example.yml +++ b/backend/product/config.example.yml @@ -33,5 +33,4 @@ api: min_length: 12 notifications: address: localhost:8083 - token_env: SXCHNGE_NOTIFICATIONS_TOKEN activation_url: http://localhost:5126/verify-email diff --git a/backend/product/notifications/client.go b/backend/product/notifications/client.go index 9174f79..79c2756 100644 --- a/backend/product/notifications/client.go +++ b/backend/product/notifications/client.go @@ -13,13 +13,11 @@ import ( "google.golang.org/grpc" "google.golang.org/grpc/credentials" "google.golang.org/grpc/credentials/insecure" - "google.golang.org/grpc/metadata" "google.golang.org/protobuf/types/known/timestamppb" ) type Config struct { Address string `yaml:"address"` - TokenEnv string `yaml:"token_env"` CAFile string `yaml:"ca_file"` ServerName string `yaml:"server_name"` ActivationURL string `yaml:"activation_url"` @@ -33,15 +31,13 @@ type Sender interface { Send(context.Context, Message) error } type Client struct { - conn *grpc.ClientConn - rpc pb.NotificationsServiceClient - token string + conn *grpc.ClientConn + rpc pb.NotificationsServiceClient } func Open(c Config) (*Client, error) { - token := os.Getenv(c.TokenEnv) - if c.Address == "" || len(token) < 32 { - return nil, merrors.InvalidArgument("notification address and caller token of at least 32 bytes are required") + if c.Address == "" { + return nil, merrors.InvalidArgument("notification address is required") } tlsConfig := &tls.Config{MinVersion: tls.VersionTLS12, ServerName: c.ServerName} if c.CAFile != "" { @@ -66,13 +62,12 @@ func Open(c Config) (*Client, error) { if err != nil { return nil, merrors.InternalWrap(err, "create notification client") } - return &Client{conn: conn, rpc: pb.NewNotificationsServiceClient(conn), token: token}, nil + return &Client{conn: conn, rpc: pb.NewNotificationsServiceClient(conn)}, nil } func (c *Client) Close() error { return c.conn.Close() } func (c *Client) Send(ctx context.Context, m Message) error { ctx, cancel := context.WithTimeout(ctx, 5*time.Second) defer cancel() - ctx = metadata.AppendToOutgoingContext(ctx, "authorization", "Bearer "+c.token) res, err := c.rpc.Send(ctx, request(m)) if err != nil { return merrors.ServiceUnavailableWrap(err, "notification queue unavailable") diff --git a/backend/product/notifications/client_test.go b/backend/product/notifications/client_test.go index bc7f5fb..2ba9e99 100644 --- a/backend/product/notifications/client_test.go +++ b/backend/product/notifications/client_test.go @@ -23,7 +23,7 @@ import ( type rpcStub struct { pb.NotificationsServiceClient request *pb.SendRequest - auth string + auth []string result *pb.SendResponse err error } @@ -31,16 +31,16 @@ type rpcStub struct { func (s *rpcStub) Send(ctx context.Context, r *pb.SendRequest, _ ...grpc.CallOption) (*pb.SendResponse, error) { s.request = r md, _ := metadata.FromOutgoingContext(ctx) - s.auth = md.Get("authorization")[0] + s.auth = md.Get("authorization") return s.result, s.err } func TestSendMapsOwnedContractAndRejectsFailedReceipt(t *testing.T) { rpc := &rpcStub{result: &pb.SendResponse{Notification: &pb.Notification{Id: "job", Deliveries: []*pb.Delivery{{State: pb.DeliveryState_DELIVERY_STATE_QUEUED}}}}} - client := &Client{rpc: rpc, token: "service-secret"} + client := &Client{rpc: rpc} expiry := time.Now().Add(time.Minute) m := Message{Key: "key", UserID: "customer", Email: "user@example.com", Locale: "ru", Template: "auth.login", Parameters: map[string]string{"code": "123456"}, ExpiresAt: expiry} require.NoError(t, client.Send(context.Background(), m)) - require.Equal(t, "Bearer service-secret", rpc.auth) + require.Empty(t, rpc.auth) require.Equal(t, m.Email, rpc.request.Targets[0].GetEmail().Address) require.Equal(t, "1", rpc.request.TemplateVersion) require.Equal(t, m.Parameters, rpc.request.Parameters) @@ -50,57 +50,54 @@ func TestSendMapsOwnedContractAndRejectsFailedReceipt(t *testing.T) { rpc.result = nil require.ErrorIs(t, client.Send(context.Background(), m), merrors.ErrServiceUnavailable) } -func TestOpenRequiresCredentialsAndValidCA(t *testing.T) { - _, err := Open(Config{Address: "localhost:8443", TokenEnv: "ABSENT_NOTIFICATION_TEST_TOKEN"}) +func TestOpenRequiresAddressAndValidCA(t *testing.T) { + _, err := Open(Config{}) require.ErrorIs(t, err, merrors.ErrInvalidArg) - t.Setenv("TEST_NOTIFICATION_TOKEN", "test-token-that-is-at-least-32-bytes") - _, err = Open(Config{Address: "localhost:8443", TokenEnv: "TEST_NOTIFICATION_TOKEN", CAFile: "missing.pem"}) + _, err = Open(Config{Address: "localhost:8443", CAFile: "missing.pem"}) require.Error(t, err) } type tlsRPC struct { pb.UnimplementedNotificationsServiceServer - auth chan string + auth chan []string } func (s *tlsRPC) Send(ctx context.Context, _ *pb.SendRequest) (*pb.SendResponse, error) { md, _ := metadata.FromIncomingContext(ctx) - s.auth <- md.Get("authorization")[0] + s.auth <- md.Get("authorization") return &pb.SendResponse{Notification: &pb.Notification{Id: "tls-job", Deliveries: []*pb.Delivery{{State: pb.DeliveryState_DELIVERY_STATE_QUEUED}}}}, nil } -func TestOpenUsesVerifiedTLSAndCallerAuthentication(t *testing.T) { +func TestOpenUsesVerifiedTLSWithoutCallerToken(t *testing.T) { fixture := httptest.NewTLSServer(http.HandlerFunc(func(http.ResponseWriter, *http.Request) {})) cert := fixture.TLS.Certificates[0] fixture.Close() listener, err := net.Listen("tcp", "127.0.0.1:0") require.NoError(t, err) server := grpc.NewServer(grpc.Creds(credentials.NewTLS(&tls.Config{Certificates: []tls.Certificate{cert}, MinVersion: tls.VersionTLS12}))) - rpc := &tlsRPC{auth: make(chan string, 1)} + rpc := &tlsRPC{auth: make(chan []string, 1)} pb.RegisterNotificationsServiceServer(server, rpc) go func() { _ = server.Serve(listener) }() t.Cleanup(server.Stop) ca := filepath.Join(t.TempDir(), "ca.pem") require.NoError(t, os.WriteFile(ca, pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: cert.Certificate[0]}), 0600)) - t.Setenv("TEST_TLS_CALLER_TOKEN", "test-service-token-with-at-least-32-bytes") - client, err := Open(Config{Address: listener.Addr().String(), TokenEnv: "TEST_TLS_CALLER_TOKEN", CAFile: ca}) + client, err := Open(Config{Address: listener.Addr().String(), CAFile: ca}) require.NoError(t, err) t.Cleanup(func() { _ = client.Close() }) require.NoError(t, client.Send(context.Background(), Message{ExpiresAt: time.Now().Add(time.Minute)})) - require.Equal(t, "Bearer test-service-token-with-at-least-32-bytes", <-rpc.auth) + require.Empty(t, <-rpc.auth) } -func TestOpenUsesPlaintextAndCallerTokenOnPrivateNetwork(t *testing.T) { +func TestOpenUsesPlaintextWithoutCallerToken(t *testing.T) { listener, err := net.Listen("tcp", "127.0.0.1:0") require.NoError(t, err) server := grpc.NewServer() - rpc := &tlsRPC{auth: make(chan string, 1)} + rpc := &tlsRPC{auth: make(chan []string, 1)} pb.RegisterNotificationsServiceServer(server, rpc) go func() { _ = server.Serve(listener) }() t.Cleanup(server.Stop) - t.Setenv("TEST_PLAIN_CALLER_TOKEN", "test-private-service-token-at-least-32-bytes") - client, err := Open(Config{Address: listener.Addr().String(), TokenEnv: "TEST_PLAIN_CALLER_TOKEN"}) + client, err := Open(Config{Address: listener.Addr().String()}) require.NoError(t, err) t.Cleanup(func() { _ = client.Close() }) require.NoError(t, client.Send(context.Background(), Message{ExpiresAt: time.Now().Add(time.Minute)})) - require.Equal(t, "Bearer test-private-service-token-at-least-32-bytes", <-rpc.auth) + require.Empty(t, <-rpc.auth) } diff --git a/docs/auth-notifications.md b/docs/auth-notifications.md index c0cf038..6467100 100644 --- a/docs/auth-notifications.md +++ b/docs/auth-notifications.md @@ -16,23 +16,21 @@ changes under a new revision rather than mutating deployed content. Configure `api.notifications` in the product YAML: - `address`: internal notifications gRPC address. -- `token_env`: environment variable containing the configured caller's token - (at least 32 bytes); example `SXCHNGE_NOTIFICATIONS_TOKEN`. -- `ca_file`: trusted PEM CA bundle; omit to use system roots. +- `ca_file`: optional trusted PEM CA bundle for explicit TLS. Omit both TLS settings for plaintext. - `server_name`: optional TLS certificate hostname override, never a TLS bypass. - `activation_url`: absolute frontend `/verify-email` URL. HTTPS is required, except HTTP on localhost/127.0.0.1 for development. No user-supplied redirect is accepted. The token is appended as a query parameter. -The caller token must map to the product caller in the notification service's -configuration. Keep Mongo, notification caller, TLS and provider credentials out +The private notifications endpoint uses the existing product storage namespace +without caller authentication. Keep Mongo, TLS and provider credentials out of Git. JWT signing configuration must contain at least 32 bytes of private key material; HMAC-derived issuance keys depend on this key. Rotation changes replay keys and existing pending JWT validity. Omission of `api.notifications` preserves the bootstrap-only legacy routes; it does not enable signup or code completion. -The example config enables the integration and therefore requires credentials. +The example config enables the integration; configure Mongo, NATS and email credentials separately. -The notification service must be reachable over TLS, with its Mongo/NATS/email +The notification service must be reachable on the private network, with its Mongo/NATS/email provider configured. Product startup creates a lazy gRPC channel; dependency outages appear as HTTP 503 when attempting delivery. A successful Send means persisted queue acceptance, not inbox delivery or a provider receipt. diff --git a/docs/implementation.md b/docs/implementation.md index bf0f3a8..44f090d 100644 --- a/docs/implementation.md +++ b/docs/implementation.md @@ -766,3 +766,25 @@ Validation passed: image build, shell/migration tests, live infrastructure check `make up` intentionally fails for the still-missing API key. Verified sender must also be configured in runtime YAML before rebuilding and starting notifications. No backend source changed and no email was sent. + +### Task #42: remove internal notification caller authentication + +Removed BFF bearer metadata, token configuration/validation, and the notifications +authentication interceptor. The private gRPC endpoint uses the existing `product` +storage namespace, preserving notification receipts and idempotency without a +database migration. NATS authentication and application/user authentication remain. + +Removed the caller secret from bootstrap, Vault Agent templates and policies. +Bootstrap archives and retires both `caller` and +`services/notifications/callers/bff`; live Vault no longer contains either path. +Service policies deny both paths. The retirement regression fixture verifies +repeatability and preservation of the NATS authentication secret. + +Validation: `make build`, `make test`, `make lint-backend`, +`make test-dev-tooling`, `make test-dev-infra`, and `git diff --check`. +Tests cover credential-free Send/Get, idempotent replay through the real service, +and absence of outgoing bearer metadata over plaintext and optional TLS. +`make up` applied the live migration and intentionally stopped at the missing +Resend API key. No email was sent; full product deployment still awaits the API +key and configured sender. Delivery continues on `dev-env-42` in +[PR #43](https://git.sendico.io/tech/sxchnge/pulls/43). diff --git a/docs/local-development.md b/docs/local-development.md index 3c0ee72..3233008 100644 --- a/docs/local-development.md +++ b/docs/local-development.md @@ -39,7 +39,6 @@ kv/ ├── services/ │ ├── bff/jwt # jwt │ └── notifications/ -│ ├── callers/bff # token; read by BFF and notifications │ └── email/resend # api_key only └── infrastructure/ ├── nats/auth # token; read by NATS and notifications @@ -56,7 +55,7 @@ password is rotated immediately after initial configuration. There are no frontend certificates, notification certificates or generic `tls` entries in KV. Browser certificates belong to Caddy; internal gRPC is plaintext -on the private Docker network and retains caller-token authentication. +on the trusted private Docker network without caller authentication. ## Resend @@ -114,7 +113,7 @@ Trust installation is not automated. A CLI check can use: curl --cacert .local/dev/ca.pem https://localhost:5126/api/v1/health ``` -BFF calls notifications over plaintext gRPC with its caller token. Mongo and NATS +BFF calls notifications over plaintext gRPC without caller authentication. Mongo and NATS also stay within the private Compose network; no internal service publishes a host port. Only frontend and Vault bind host loopback ports. TLS remains optional in the reusable notification client/server for deployments that explicitly configure @@ -143,7 +142,7 @@ still interrupt work. Repository consumers use `source.Source` operation scopes; ## Migration and persistence -Bootstrap migrates the old flat `bff`, `caller`, `email` and `nats` paths into the +Bootstrap migrates the old flat `bff`, `email` and `nats` paths into the hierarchy above, and moves `bootstrap/mongo` to `infrastructure/mongo/bootstrap`. It copies with CAS protection, compares values, and refuses to overwrite conflicting destinations. After consumers use the new paths, it removes @@ -178,3 +177,6 @@ and requires three replacements with unchanged BFF PID and zero unexpected transaction-backed HTTP responses. It restores TTLs and removes services it started. No verification command sends email. Backend validation remains `make test` and `make lint-backend`, using the host SDKs documented in module guides. + +Bootstrap archives and removes obsolete `caller` and `services/notifications/callers/bff` +KV entries after updating Agent policies/templates. NATS authentication remains enabled. diff --git a/infra/local/bff/runtime.yml b/infra/local/bff/runtime.yml index e399832..1095f8e 100644 --- a/infra/local/bff/runtime.yml +++ b/infra/local/bff/runtime.yml @@ -38,5 +38,4 @@ api: min_length: 12 notifications: address: notifications:8083 - token_env: SXCHNGE_NOTIFICATIONS_TOKEN activation_url: https://localhost:5126/verify-email diff --git a/infra/local/notifications/runtime.yml b/infra/local/notifications/runtime.yml index 80e5e78..0e45830 100644 --- a/infra/local/notifications/runtime.yml +++ b/infra/local/notifications/runtime.yml @@ -17,8 +17,6 @@ messaging: max_reconnects: -1 reconnect_wait: 2 templates: "/etc/sxchnge/templates.json" -callers: - product: "SXCHNGE_NOTIFICATIONS_PRODUCT_TOKEN" email: driver: resend settings: diff --git a/infra/local/scripts/bootstrap.sh b/infra/local/scripts/bootstrap.sh index e5dd618..e12765e 100644 --- a/infra/local/scripts/bootstrap.sh +++ b/infra/local/scripts/bootstrap.sh @@ -58,8 +58,8 @@ provision_agents() { directory="$STATE/agents/$service" mkdir -p "$directory"; chmod 755 "$directory" case "$service" in - bff) paths='services/bff/jwt services/notifications/callers/bff';; - notifications) paths='services/notifications/callers/bff infrastructure/nats/auth services/notifications/email/resend';; + bff) paths='services/bff/jwt';; + notifications) paths='infrastructure/nats/auth services/notifications/email/resend';; nats) paths=infrastructure/nats/auth;; esac : > "$WORK/policy" @@ -92,7 +92,7 @@ up() { initialize mounts migrate_secrets - for path in infrastructure/nats/auth services/notifications/callers/bff services/bff/jwt; do + for path in infrastructure/nats/auth services/bff/jwt; do if [ "$(kv "$path")" = null ]; then field=token; [ "$path" != services/bff/jwt ] || field=jwt random | jq -Rs --arg field "$field" '{($field):rtrimstr("\n")}' | put "$path" diff --git a/infra/local/scripts/secrets.sh b/infra/local/scripts/secrets.sh index 1989442..b6b01c8 100644 --- a/infra/local/scripts/secrets.sh +++ b/infra/local/scripts/secrets.sh @@ -1,8 +1,7 @@ -# Canonical paths follow ownership. Shared callers live under the receiving service. +# Canonical paths follow ownership. secret_moves() { cat <<'PATHS' bff services/bff/jwt -caller services/notifications/callers/bff email services/notifications/email/resend nats infrastructure/nats/auth bootstrap/mongo infrastructure/mongo/bootstrap @@ -47,7 +46,7 @@ retire_secrets() ( archive_secret "$old" printf '{}' | vault "kv/metadata/$old" DELETE >/dev/null done < "$WORK/moves" - for old in tls frontend-tls notifications-tls; do + for old in caller services/notifications/callers/bff tls frontend-tls notifications-tls; do archive_secret "$old" if [ "$(vault "kv/metadata/$old" GET yes)" != null ]; then printf '{}' | vault "kv/metadata/$old" DELETE >/dev/null; fi done diff --git a/infra/local/scripts/test.sh b/infra/local/scripts/test.sh index 2dd2f13..ddcede0 100644 --- a/infra/local/scripts/test.sh +++ b/infra/local/scripts/test.sh @@ -74,3 +74,22 @@ printf '{"data":{"metadata":{"version":43},"data":{"api_key":"preserved-fixture- if (normalize_email_secret) >/dev/null 2>&1; then fail 'Unknown secret field was silently dropped'; fi if grep -q EMAIL_CONFIG infra/local/vault/notifications.hcl infra/local/scripts/start-service.sh; then fail 'Email configuration is still rendered from Vault'; fi echo 'PASS: email migration preserves API key, removes configuration fields and rejects unknown fields' +# Caller retirement is repeatable and must not remove NATS authentication. +secret_moves() { :; } +printf '{"caller":true,"services/notifications/callers/bff":true,"infrastructure/nats/auth":true}' > "$WORK/retirement" +vault() { + path=${1#kv/metadata/} + case "${2:-GET}" in + GET) jq -c --arg path "$path" '.[$path] // null' "$WORK/retirement";; + DELETE) + case "$path" in caller|services/notifications/callers/bff) ;; *) fail 'Unexpected secret retirement';; esac + jq --arg path "$path" 'del(.[$path])' "$WORK/retirement" > "$WORK/retirement-next" + mv "$WORK/retirement-next" "$WORK/retirement" + ;; + *) fail 'Unexpected retirement method';; + esac +} +retire_secrets +retire_secrets +jq -e '.=={"infrastructure/nats/auth":true}' "$WORK/retirement" >/dev/null +echo 'PASS: obsolete caller secrets retired idempotently; NATS authentication preserved' diff --git a/infra/local/scripts/verify.sh b/infra/local/scripts/verify.sh index d171fe4..7aaf702 100644 --- a/infra/local/scripts/verify.sh +++ b/infra/local/scripts/verify.sh @@ -6,7 +6,7 @@ policies() ( directory="$STATE/agents/$service" login=$(jq -n --rawfile role "$directory/role-id" --rawfile secret "$directory/secret-id" '{role_id:($role|rtrimstr("\n")),secret_id:($secret|rtrimstr("\n"))}' | vault auth/approle/login POST) identity=$(printf '%s' "$login" | jq -er '.auth.client_token') - printf '%s' "$login" | jq '{token:.auth.client_token,paths:["kv/data/bootstrap/mongo","kv/data/infrastructure/mongo/bootstrap","sys/init","kv/data/bff","kv/data/caller","kv/data/email","kv/data/nats","kv/data/tls"]}' | vault sys/capabilities POST | jq -e '.data | all(.[]; .==["deny"])' >/dev/null || fail 'Service can access bootstrap credentials' + printf '%s' "$login" | jq '{token:.auth.client_token,paths:["kv/data/bootstrap/mongo","kv/data/infrastructure/mongo/bootstrap","sys/init","kv/data/bff","kv/data/caller","kv/data/services/notifications/callers/bff","kv/data/email","kv/data/nats","kv/data/tls"]}' | vault sys/capabilities POST | jq -e '.data | all(.[]; .==["deny"])' >/dev/null || fail 'Service can access bootstrap credentials' VAULT_TOKEN=$(printf '%s' "$login" | jq -er '.auth.client_token') printf '{}' | vault auth/token/revoke-self POST >/dev/null VAULT_TOKEN=$root_token @@ -51,10 +51,10 @@ product() { } layout() { kv services/notifications/email/resend | jq -e 'keys==["api_key"]' >/dev/null || fail 'Non-secret email configuration remains in Vault' - for old in bootstrap/mongo bff caller email nats tls frontend-tls notifications-tls; do + for old in bootstrap/mongo bff caller services/notifications/callers/bff email nats tls frontend-tls notifications-tls; do [ "$(vault "kv/metadata/$old" GET yes)" = null ] || fail "Obsolete KV entry remains: $old" done - for path in services/bff/jwt services/notifications/callers/bff services/notifications/email/resend infrastructure/nats/auth infrastructure/mongo/bootstrap; do + for path in services/bff/jwt services/notifications/email/resend infrastructure/nats/auth infrastructure/mongo/bootstrap; do vault "kv/metadata/$path" | jq -e '.data.current_version>0' >/dev/null || fail "Missing canonical KV entry: $path" done echo 'PASS: canonical Vault hierarchy present; obsolete flat and TLS entries absent' diff --git a/infra/local/vault/bff.hcl b/infra/local/vault/bff.hcl index 1b1eebe..8b1b9f6 100644 --- a/infra/local/vault/bff.hcl +++ b/infra/local/vault/bff.hcl @@ -19,10 +19,6 @@ env_template "SXCHNGE_JWT_SECRET" { error_on_missing_key = true } -env_template "SXCHNGE_NOTIFICATIONS_TOKEN" { - contents = "{{ with secret \"kv/data/services/notifications/callers/bff\" }}{{ .Data.data.token }}{{ end }}" - error_on_missing_key = true -} exec { command = ["/usr/local/bin/start-service"] diff --git a/infra/local/vault/notifications.hcl b/infra/local/vault/notifications.hcl index 0708ff6..7e332ff 100644 --- a/infra/local/vault/notifications.hcl +++ b/infra/local/vault/notifications.hcl @@ -14,10 +14,6 @@ template_config { } -env_template "SXCHNGE_NOTIFICATIONS_PRODUCT_TOKEN" { - contents = "{{ with secret \"kv/data/services/notifications/callers/bff\" }}{{ .Data.data.token }}{{ end }}" - error_on_missing_key = true -} env_template "SXCHNGE_NATS_URL" { contents = "{{ with secret \"kv/data/infrastructure/nats/auth\" }}nats://{{ .Data.data.token }}@nats:4222{{ end }}" From ce6583f4547b18de5c5292d99e1968f5645084c0 Mon Sep 17 00:00:00 2001 From: Stephan D Date: Sun, 13 Sep 2026 08:40:24 +0200 Subject: [PATCH 09/11] Show service links and next steps after local startup --- docs/local-development.md | 15 +++++++++++++++ infra/local/scripts/bootstrap.sh | 18 +++++++++++++++++- 2 files changed, 32 insertions(+), 1 deletion(-) diff --git a/docs/local-development.md b/docs/local-development.md index 3233008..9b9c8b6 100644 --- a/docs/local-development.md +++ b/docs/local-development.md @@ -30,6 +30,21 @@ AppRoles and secrets, initializes Mongo replica set `rs0`, configures Vault's Mo plugin, and starts authenticated NATS JetStream. Product startup fails explicitly until Resend credentials exist. Infrastructure remains available for provisioning. +After all services pass their startup health checks and the public Caddy CA is +exported, `make up` prints a Sendico-style completion message with the frontend, +API, API health and Vault URLs, the CA trust reminder, and status/log/stop commands. +Failed startup does not print the success message. Mongo and NATS are private +Compose services, so the summary does not advertise host URLs or credentials for +them. This terminal-only change reuses Compose errors and the command exit status +as startup outcome signals; it adds no analytics or secret output. + +Completion-message validation: `make test-dev-tooling` and `git diff --check` +passed. A containerized shell preview with stubbed infrastructure calls confirmed +the success layout and that failed service startup or CA export exits nonzero +without the message. The UX review checked readable plain-text labels, actionable +URLs/commands and confirmed-success timing. This check did not restart the live +stack or verify application readiness. + ## Vault layout Paths below are inside the KV v2 mount `kv`; the API inserts `data/` after `kv/`. diff --git a/infra/local/scripts/bootstrap.sh b/infra/local/scripts/bootstrap.sh index e12765e..3fcee0d 100644 --- a/infra/local/scripts/bootstrap.sh +++ b/infra/local/scripts/bootstrap.sh @@ -107,7 +107,23 @@ up() { kv services/notifications/email/resend | email_valid || fail 'Vault kv/services/notifications/email/resend needs api_key. Run make email; configure the verified sender in infra/local/notifications/runtime.yml, then make build && make up.' compose up -d --wait --wait-timeout 180 export_ca - echo 'Local stack ready: https://localhost:5126; Vault: http://127.0.0.1:18200' + cat <<'MESSAGE' + +✅ SXCHNGE development environment started! + +Services: + Frontend: https://localhost:5126 + API: https://localhost:5126/api/v1 + API health: https://localhost:5126/api/v1/health + Vault: http://127.0.0.1:18200 + +HTTPS: Trust .local/dev/ca.pem in your browser or system trust store. + See docs/local-development.md for setup instructions. + +View status: make ps +View logs: make logs +Stop: make down +MESSAGE } email() { existing From d04c8b12abeb54c6b4d8a4d4058f262107f8ebc2 Mon Sep 17 00:00:00 2001 From: Stephan D Date: Sun, 13 Sep 2026 10:26:06 +0200 Subject: [PATCH 10/11] Make product API versions independently installable --- backend/product/README.md | 5 +- backend/product/app/api/api.go | 42 ++-- backend/product/app/api/config.go | 2 + backend/product/app/api/register.go | 13 -- backend/product/app/api/runtime.go | 30 ++- backend/product/app/api/v1/install.go | 26 +++ backend/product/app/http/middleware.go | 118 +---------- backend/product/app/profile/runtime.go | 17 +- backend/product/app/runtime/api.go | 66 +----- backend/product/app/runtime/bootstrap.go | 131 +++++------- backend/product/app/runtime/bootstrap_test.go | 2 +- backend/product/app/runtime/lifecycle.go | 32 ++- backend/product/app/runtime/mounts.go | 58 ++++++ backend/product/app/runtime/mounts_test.go | 190 ++++++++++++++++++ backend/product/app/runtime/services.go | 82 +++++--- .../app/server/internal/config_test.go | 32 +++ backend/product/app/server/internal/server.go | 2 +- backend/product/app/urls/domain_provider.go | 1 + .../app/urls/internal/domain_provider.go | 18 +- .../app/urls/internal/domain_provider_test.go | 27 +++ backend/product/architecture/versions_test.go | 41 ++++ .../http/{ => v1}/authorized/customer_test.go | 4 +- .../auth/http/{ => v1}/authorized/handler.go | 2 +- .../auth/http/{ => v1}/authorized/router.go | 4 +- .../http/{ => v1}/authorized/session_test.go | 0 .../product/auth/http/{ => v1}/dispatcher.go | 20 +- .../auth/http/{ => v1}/endpoint/endpoint.go | 2 +- .../auth/http/{ => v1}/endpoint/install.go | 4 +- .../auth/http/{ => v1}/endpoint/token.go | 2 +- .../auth/http/{ => v1}/public/login.go | 4 +- .../http/{ => v1}/public/login_rate_limit.go | 2 +- .../{ => v1}/public/login_rate_limit_test.go | 2 +- .../auth/http/{ => v1}/public/login_test.go | 4 +- .../public/login_test_support_test.go | 0 .../auth/http/{ => v1}/public/logout.go | 2 +- .../auth/http/{ => v1}/public/logout_test.go | 0 .../auth/http/{ => v1}/public/metrics.go | 2 +- .../auth/http/{ => v1}/public/refresh.go | 4 +- .../auth/http/{ => v1}/public/refresh_test.go | 2 +- .../auth/http/{ => v1}/public/respond.go | 2 +- .../auth/http/{ => v1}/public/rotate.go | 2 +- .../auth/http/{ => v1}/public/router.go | 6 +- .../auth/http/{ => v1}/public/validate.go | 4 +- .../auth/http/{ => v1}/public/verification.go | 2 +- .../public/verification_integration_test.go | 2 +- .../auth/{ => http/v1}/request/login.go | 0 .../auth/{ => http/v1}/request/refresh.go | 0 .../auth/{ => http/v1}/request/rotate.go | 0 .../{ => http/v1}/request/validateable.go | 0 .../{ => http/v1}/request/verification.go | 0 .../auth/{ => http/v1}/response/account.go | 0 .../auth/{ => http/v1}/response/authresp.go | 0 .../auth/{ => http/v1}/response/file.go | 0 .../auth/{ => http/v1}/response/login.go | 0 .../{ => http/v1}/response/login_pending.go | 0 .../auth/{ => http/v1}/response/login_test.go | 0 .../auth/{ => http/v1}/response/response.go | 0 .../auth/{ => http/v1}/response/result.go | 0 .../auth/{ => http/v1}/response/token.go | 0 .../{ => http/v1}/response/verification.go | 0 backend/product/auth/http/{ => v1}/router.go | 2 +- backend/product/config.example.yml | 4 + backend/product/customers/internal/service.go | 11 +- backend/product/customers/types.go | 9 +- backend/product/sessions/browser/browser.go | 19 +- .../product/sessions/browser/browser_test.go | 6 +- .../product/twofactor/http/{ => v1}/router.go | 2 +- .../twofactor/http/{ => v1}/router_test.go | 2 +- docs/api-versions.md | 91 +++++++++ 69 files changed, 755 insertions(+), 404 deletions(-) delete mode 100644 backend/product/app/api/register.go create mode 100644 backend/product/app/api/v1/install.go create mode 100644 backend/product/app/runtime/mounts.go create mode 100644 backend/product/app/runtime/mounts_test.go create mode 100644 backend/product/app/urls/internal/domain_provider_test.go create mode 100644 backend/product/architecture/versions_test.go rename backend/product/auth/http/{ => v1}/authorized/customer_test.go (91%) rename backend/product/auth/http/{ => v1}/authorized/handler.go (99%) rename backend/product/auth/http/{ => v1}/authorized/router.go (89%) rename backend/product/auth/http/{ => v1}/authorized/session_test.go (100%) rename backend/product/auth/http/{ => v1}/dispatcher.go (73%) rename backend/product/auth/http/{ => v1}/endpoint/endpoint.go (94%) rename backend/product/auth/http/{ => v1}/endpoint/install.go (93%) rename backend/product/auth/http/{ => v1}/endpoint/token.go (95%) rename backend/product/auth/http/{ => v1}/public/login.go (98%) rename backend/product/auth/http/{ => v1}/public/login_rate_limit.go (98%) rename backend/product/auth/http/{ => v1}/public/login_rate_limit_test.go (99%) rename backend/product/auth/http/{ => v1}/public/login_test.go (99%) rename backend/product/auth/http/{ => v1}/public/login_test_support_test.go (100%) rename backend/product/auth/http/{ => v1}/public/logout.go (97%) rename backend/product/auth/http/{ => v1}/public/logout_test.go (100%) rename backend/product/auth/http/{ => v1}/public/metrics.go (94%) rename backend/product/auth/http/{ => v1}/public/refresh.go (93%) rename backend/product/auth/http/{ => v1}/public/refresh_test.go (99%) rename backend/product/auth/http/{ => v1}/public/respond.go (96%) rename backend/product/auth/http/{ => v1}/public/rotate.go (95%) rename backend/product/auth/http/{ => v1}/public/router.go (95%) rename backend/product/auth/http/{ => v1}/public/validate.go (96%) rename backend/product/auth/http/{ => v1}/public/verification.go (99%) rename backend/product/auth/http/{ => v1}/public/verification_integration_test.go (95%) rename backend/product/auth/{ => http/v1}/request/login.go (100%) rename backend/product/auth/{ => http/v1}/request/refresh.go (100%) rename backend/product/auth/{ => http/v1}/request/rotate.go (100%) rename backend/product/auth/{ => http/v1}/request/validateable.go (100%) rename backend/product/auth/{ => http/v1}/request/verification.go (100%) rename backend/product/auth/{ => http/v1}/response/account.go (100%) rename backend/product/auth/{ => http/v1}/response/authresp.go (100%) rename backend/product/auth/{ => http/v1}/response/file.go (100%) rename backend/product/auth/{ => http/v1}/response/login.go (100%) rename backend/product/auth/{ => http/v1}/response/login_pending.go (100%) rename backend/product/auth/{ => http/v1}/response/login_test.go (100%) rename backend/product/auth/{ => http/v1}/response/response.go (100%) rename backend/product/auth/{ => http/v1}/response/result.go (100%) rename backend/product/auth/{ => http/v1}/response/token.go (100%) rename backend/product/auth/{ => http/v1}/response/verification.go (100%) rename backend/product/auth/http/{ => v1}/router.go (89%) rename backend/product/twofactor/http/{ => v1}/router.go (99%) rename backend/product/twofactor/http/{ => v1}/router_test.go (96%) create mode 100644 docs/api-versions.md diff --git a/backend/product/README.md b/backend/product/README.md index 8f45c07..da04be0 100644 --- a/backend/product/README.md +++ b/backend/product/README.md @@ -28,7 +28,10 @@ Signup and email verification delivery/completion are available when `api.notifications` is configured. Flutter selects the backend using `SXCHNGE_API_URL` and `SXCHNGE_CLIENT_ID`. See [auth notification setup and contracts](../../docs/auth-notifications.md). -Profile HTTP wiring remains follow-up work. +API versions are installed explicitly by the product composition. See +[version installation and compatibility](../../docs/api-versions.md) for selection, +mounts, shared lifecycle and adding a future version. Customer profile endpoint +wiring remains follow-up work. Sendico integration is not wired. Its organization context belongs only to the future public HTTP adapter. There is no unused public API snapshot in this module; diff --git a/backend/product/app/api/api.go b/backend/product/app/api/api.go index 8e156a4..4af05aa 100644 --- a/backend/product/app/api/api.go +++ b/backend/product/app/api/api.go @@ -1,23 +1,37 @@ package api import ( - "context" - "git.sendico.io/tech/sxchnge/product/app/urls" - "git.sendico.io/tech/sxchnge/shared/database" + "git.sendico.io/tech/sxchnge/product/auth/config" + "git.sendico.io/tech/sxchnge/product/auth/flow" + "git.sendico.io/tech/sxchnge/product/auth/storage" + "git.sendico.io/tech/sxchnge/product/customers" + "git.sendico.io/tech/sxchnge/product/twofactor" "git.sendico.io/tech/sxchnge/shared/logging" - "git.sendico.io/tech/sxchnge/shared/service" + "github.com/go-chi/chi/v5" ) -type API interface { - LifecycleContext() context.Context - Logger() mlogger.Logger - DomainProvider() domainprovider.DomainProvider - Config() *Config - ProfileConfig() ConfigProvider - DBFactory() db.Factory - RuntimeComponent(name string) any - Register() Register +// Dependencies are initialized once and shared by installed transports. +type Dependencies struct { + Stores *storage.Stores + Customers accountservice.AccountService + Auth *flow.Service + Factors *twofactor.Service + Token middleware.TokenConfig + Signature middleware.Signature + LoginLimits middleware.LoginRateLimits } -type MicroServiceFactoryT = func(API) (mservice.MicroService, error) +// Scope belongs to one installation. Router paths are relative to Mount. +type Scope struct { + Router chi.Router + Mount string + URLs domainprovider.DomainProvider + Logger mlogger.Logger +} +type Installer func(Scope, *Dependencies) error + +type Mount struct { + Version string `yaml:"version"` + Path string `yaml:"path"` +} diff --git a/backend/product/app/api/config.go b/backend/product/app/api/config.go index 5ce6ba6..338f69c 100644 --- a/backend/product/app/api/config.go +++ b/backend/product/app/api/config.go @@ -7,6 +7,8 @@ import ( ) type Config struct { + // nil preserves the legacy v1 mount; an explicit empty list installs no API. + Versions []Mount `yaml:"versions"` Notifications *notifications.Config `yaml:"notifications"` Mw *mwa.Config `yaml:"middleware"` AuthStorage *mwa.AuthStorageConfig `yaml:"auth_storage"` diff --git a/backend/product/app/api/register.go b/backend/product/app/api/register.go deleted file mode 100644 index 4361edd..0000000 --- a/backend/product/app/api/register.go +++ /dev/null @@ -1,13 +0,0 @@ -package api - -import ( - "git.sendico.io/tech/sxchnge/product/auth/response" - api "git.sendico.io/tech/sxchnge/shared/http" - "git.sendico.io/tech/sxchnge/shared/service" -) - -type Register interface { - Handler(service mservice.Type, endpoint string, method api.HTTPMethod, handler sresponse.HandlerFunc) - AccountHandler(service mservice.Type, endpoint string, method api.HTTPMethod, handler sresponse.AccountHandlerFunc) - PendingAccountHandler(service mservice.Type, endpoint string, method api.HTTPMethod, handler sresponse.PendingAccountHandlerFunc) -} diff --git a/backend/product/app/api/runtime.go b/backend/product/app/api/runtime.go index 1490caf..789418e 100644 --- a/backend/product/app/api/runtime.go +++ b/backend/product/app/api/runtime.go @@ -1,36 +1,32 @@ package api -import "git.sendico.io/tech/sxchnge/shared/service" +import ( + "context" -type ConfigProvider interface { - SharedConfig() *Config -} + "git.sendico.io/tech/sxchnge/shared/database" + "git.sendico.io/tech/sxchnge/shared/logging" +) +type ConfigProvider interface{ SharedConfig() *Config } type ConfigFactory func() ConfigProvider -type RuntimeComponents map[string]any -type RuntimeBootstrapFunc func(API) (RuntimeComponents, error) -type RuntimeFinishFunc func(API) error +// Bootstrap opens process-owned resources. Its cleanup is called once even if +// installation fails. On error, bootstrap must return cleanup or clean its own +// resources before returning, never both. +type Bootstrap func(context.Context, mlogger.Logger, *Config, db.Factory) (*Dependencies, func(context.Context) error, error) type RuntimeConfig struct { Name string - Services []ServiceFactory ConfigFactory ConfigFactory - Bootstrap RuntimeBootstrapFunc - Finish RuntimeFinishFunc -} - -type ServiceFactory struct { - Name mservice.Type - Factory MicroServiceFactoryT + Bootstrap Bootstrap + Versions map[string]Installer } func (cfg RuntimeConfig) RuntimeName() string { if cfg.Name == "" { - return "edge" + return "product" } return cfg.Name } - func (cfg RuntimeConfig) NewConfig() ConfigProvider { if cfg.ConfigFactory != nil { return cfg.ConfigFactory() diff --git a/backend/product/app/api/v1/install.go b/backend/product/app/api/v1/install.go new file mode 100644 index 0000000..72cf227 --- /dev/null +++ b/backend/product/app/api/v1/install.go @@ -0,0 +1,26 @@ +// Package v1 composes the product's first public HTTP contract. +package v1 + +import ( + "git.sendico.io/tech/sxchnge/product/app/api" + auth "git.sendico.io/tech/sxchnge/product/auth/http/v1" + "git.sendico.io/tech/sxchnge/product/auth/http/v1/public" + factors "git.sendico.io/tech/sxchnge/product/twofactor/http/v1" + "git.sendico.io/tech/sxchnge/shared/errors" +) + +func Install(scope api.Scope, deps *api.Dependencies) error { + if deps == nil || deps.Stores == nil { + return merrors.InvalidArgument("v1 auth dependencies are required") + } + flows := public.Flows{Onboarding: deps.Auth} + if deps.Factors != nil { + flows.Factors = deps.Factors + } + // Signature is resolved at bootstrap, not by each version from the environment. + routes := auth.Install(scope.Logger, scope.Router, "", deps.Stores.Accounts, deps.Stores.RefreshTokens, deps.Stores.LoginAttempts, &deps.Token, deps.Signature, deps.LoginLimits, flows) + if deps.Factors != nil { + factors.Install(routes, deps.Factors, scope.Logger) + } + return nil +} diff --git a/backend/product/app/http/middleware.go b/backend/product/app/http/middleware.go index 207916e..5bdb2f9 100644 --- a/backend/product/app/http/middleware.go +++ b/backend/product/app/http/middleware.go @@ -2,26 +2,12 @@ package httptransport import ( "net/http" - "os" "path" mr "git.sendico.io/tech/sxchnge/product/app/metrics" "git.sendico.io/tech/sxchnge/product/auth/config" - "git.sendico.io/tech/sxchnge/product/auth/flow" - httprouter "git.sendico.io/tech/sxchnge/product/auth/http" - public "git.sendico.io/tech/sxchnge/product/auth/http/public" - "git.sendico.io/tech/sxchnge/product/auth/response" - authstorage "git.sendico.io/tech/sxchnge/product/auth/storage" - accountservice "git.sendico.io/tech/sxchnge/product/customers" - "git.sendico.io/tech/sxchnge/product/notifications" "git.sendico.io/tech/sxchnge/product/sessions/browser" - "git.sendico.io/tech/sxchnge/product/twofactor" - delivery "git.sendico.io/tech/sxchnge/product/twofactor/delivery" - factorhttp "git.sendico.io/tech/sxchnge/product/twofactor/http" - factorstore "git.sendico.io/tech/sxchnge/product/twofactor/store" - "git.sendico.io/tech/sxchnge/shared/database" "git.sendico.io/tech/sxchnge/shared/errors" - api "git.sendico.io/tech/sxchnge/shared/http" "git.sendico.io/tech/sxchnge/shared/http/response" "git.sendico.io/tech/sxchnge/shared/logging" "git.sendico.io/tech/sxchnge/shared/service" @@ -34,33 +20,17 @@ import ( ) type Middleware struct { - notifications *notifications.Client - logger mlogger.Logger - router *chi.Mux - apiEndpoint string - metrics mr.Metrics - epdispatcher *httprouter.Dispatcher -} - -func (mw *Middleware) Handler(service mservice.Type, endpoint string, method api.HTTPMethod, handler sresponse.HandlerFunc) { - mw.epdispatcher.Handler(service, endpoint, method, handler) -} - -func (mw *Middleware) AccountHandler(service mservice.Type, endpoint string, method api.HTTPMethod, handler sresponse.AccountHandlerFunc) { - mw.epdispatcher.AccountHandler(service, endpoint, method, handler) -} - -func (mw *Middleware) PendingAccountHandler(service mservice.Type, endpoint string, method api.HTTPMethod, handler sresponse.PendingAccountHandlerFunc) { - mw.epdispatcher.PendingAccountHandler(service, endpoint, method, handler) + logger mlogger.Logger + router *chi.Mux + apiEndpoint string + metrics mr.Metrics } func (mw *Middleware) Finish() { - mw.metrics.Finish() - if mw.notifications != nil { - _ = mw.notifications.Close() + if mw.metrics != nil { + mw.metrics.Finish() } } - func (mw *Middleware) installMiddleware(config *middleware.Config, debug bool) { mw.logger.Debug("Installing middleware stack...") // Collect metrics for application traffic only. Scrapes and health probes are @@ -112,15 +82,7 @@ func isObservabilityRequest(apiEndpoint, requestPath string) bool { return requestPath == healthPath || requestPath == metricsPath || requestPath == "/metrics" } -func CreateMiddleware( - logger mlogger.Logger, - db db.Factory, - apiRouter *chi.Mux, - config *middleware.Config, - authStorage *middleware.AuthStorageConfig, - debug bool, - notificationConfig ...*notifications.Config, -) (*Middleware, error) { +func CreateMiddleware(logger mlogger.Logger, apiRouter *chi.Mux, config *middleware.Config, endpoint string, debug bool) (*Middleware, error) { if config == nil { return nil, merrors.InvalidArgument("middleware configuration is required", "middleware") } @@ -130,72 +92,12 @@ func CreateMiddleware( if err := config.CORS.Validate(); err != nil { return nil, err } - loginRateLimits, err := config.ResolveLoginRateLimits() - if err != nil { - logger.Error("Invalid login rate limit configuration", zap.Error(err)) - return nil, err - } - p := &Middleware{ - logger: logger.Named("middleware"), - router: apiRouter, - apiEndpoint: os.Getenv(config.EndPointEnv), - } - p.logger.Info("Set endpoint", zap.String("endpoint", p.apiEndpoint)) + p := &Middleware{logger: logger.Named("middleware"), router: apiRouter, apiEndpoint: endpoint} p.installMiddleware(config, debug) - if p.metrics, err = mr.NewMetricsRouter(p.logger, p.router, p.apiEndpoint); err != nil { - p.logger.Error("Failed to create metrics router", zap.Error(err), zap.String("api_endpoint", p.apiEndpoint)) - return nil, err - } - - stores, err := authstorage.OpenStores(p.logger, db, authStorage) + var err error + p.metrics, err = mr.NewMetricsRouter(p.logger, p.router, p.apiEndpoint) if err != nil { return nil, err } - - var authFlow *flow.Service - var factors *twofactor.Service - if len(notificationConfig) > 0 && notificationConfig[0] != nil { - c := notificationConfig[0] - customers, err := accountservice.NewAccountService(p.logger, db, &config.Password, authStorage) - if err != nil { - return nil, err - } - authFlow = &flow.Service{Accounts: stores.Accounts, Customers: customers, Tokens: stores.Verification, Transactions: db.TransactionFactory(), Attempts: stores.LoginAttempts, Secret: middleware.SignatureConf(config).PrivateKey, ActivationURL: c.ActivationURL} - if err = authFlow.Validate(); err != nil { - return nil, err - } - p.notifications, err = notifications.Open(*c) - if err != nil { - return nil, err - } - authFlow.Sender = p.notifications - records, err := factorstore.Open(p.logger, db) - if err != nil { - _ = p.notifications.Close() - return nil, err - } - factors, err = twofactor.New(stores.Accounts, stores.Verification, records, db.TransactionFactory(), middleware.SignatureConf(config).PrivateKey, p.logger, delivery.Email{Sender: p.notifications}) - if err != nil { - _ = p.notifications.Close() - return nil, err - } - } - flows := public.Flows{Onboarding: authFlow} - if factors != nil { - flows.Factors = factors - } - p.epdispatcher = httprouter.NewDispatcher( - p.logger, - p.router, - stores.Accounts, - stores.RefreshTokens, - stores.LoginAttempts, - config, - loginRateLimits, - flows, - ) - if factors != nil { - factorhttp.Install(p.epdispatcher, factors, p.logger) - } return p, nil } diff --git a/backend/product/app/profile/runtime.go b/backend/product/app/profile/runtime.go index b8bbba1..f334156 100644 --- a/backend/product/app/profile/runtime.go +++ b/backend/product/app/profile/runtime.go @@ -1,21 +1,12 @@ -// Package profile selects the locally owned components of the copied BFF. +// Package profile selects the product's supported API implementations. package profile import ( "git.sendico.io/tech/sxchnge/product/app/api" - "git.sendico.io/tech/sxchnge/product/customers" + "git.sendico.io/tech/sxchnge/product/app/api/v1" + "git.sendico.io/tech/sxchnge/product/app/runtime" ) -const Accounts = "accounts" - func Runtime() api.RuntimeConfig { - return api.RuntimeConfig{Name: "sxchnge.bff", Bootstrap: bootstrap} -} - -func bootstrap(a api.API) (api.RuntimeComponents, error) { - accounts, err := accountservice.NewAccountService(a.Logger(), a.DBFactory(), &a.Config().Mw.Password, a.Config().AuthStorage) - if err != nil { - return nil, err - } - return api.RuntimeComponents{Accounts: accounts}, nil + return api.RuntimeConfig{Name: "sxchnge.bff", Bootstrap: runtime.OpenDependencies, Versions: map[string]api.Installer{"v1": v1.Install}} } diff --git a/backend/product/app/runtime/api.go b/backend/product/app/runtime/api.go index 9b82f60..be25731 100644 --- a/backend/product/app/runtime/api.go +++ b/backend/product/app/runtime/api.go @@ -2,68 +2,20 @@ package runtime import ( "context" + "sync" - "git.sendico.io/tech/sxchnge/product/app/api" amr "git.sendico.io/tech/sxchnge/product/app/health" httptransport "git.sendico.io/tech/sxchnge/product/app/http" - "git.sendico.io/tech/sxchnge/product/app/urls" - "git.sendico.io/tech/sxchnge/shared/database" "git.sendico.io/tech/sxchnge/shared/logging" - "git.sendico.io/tech/sxchnge/shared/service" ) -type Microservices = []mservice.MicroService - -// APIImp represents the structure of the APIImp -type APIImp struct { - lifecycleCtx context.Context - logger mlogger.Logger - db db.Factory - domain domainprovider.DomainProvider - config *api.Config - profile api.ConfigProvider - runtime api.RuntimeConfig - components api.RuntimeComponents - services Microservices - health amr.Health - mw *httptransport.Middleware +type instance struct { + logger mlogger.Logger + health amr.Health + mw *httptransport.Middleware + cleanup func(context.Context) error + once sync.Once + finishErr error } -func (a *APIImp) LifecycleContext() context.Context { - return a.lifecycleCtx -} - -func (a *APIImp) Logger() mlogger.Logger { - return a.logger -} - -func (a *APIImp) Config() *api.Config { - return a.config -} - -func (a *APIImp) ProfileConfig() api.ConfigProvider { - return a.profile -} - -func (a *APIImp) DBFactory() db.Factory { - return a.db -} - -func (a *APIImp) DomainProvider() domainprovider.DomainProvider { - return a.domain -} - -func (a *APIImp) Register() api.Register { - return a.mw -} - -func (a *APIImp) RuntimeComponent(name string) any { - if a == nil || a.components == nil { - return nil - } - return a.components[name] -} - -func (a *APIImp) Name() string { - return "api" -} +func (a *instance) Name() string { return "api" } diff --git a/backend/product/app/runtime/bootstrap.go b/backend/product/app/runtime/bootstrap.go index 8aa651d..4ed64f4 100644 --- a/backend/product/app/runtime/bootstrap.go +++ b/backend/product/app/runtime/bootstrap.go @@ -6,9 +6,12 @@ import ( "git.sendico.io/tech/sxchnge/product/app/api" amr "git.sendico.io/tech/sxchnge/product/app/health" + "git.sendico.io/tech/sxchnge/product/app/health/status" httptransport "git.sendico.io/tech/sxchnge/product/app/http" "git.sendico.io/tech/sxchnge/product/app/urls" + "git.sendico.io/tech/sxchnge/product/sessions/browser" "git.sendico.io/tech/sxchnge/shared/database" + "git.sendico.io/tech/sxchnge/shared/errors" "git.sendico.io/tech/sxchnge/shared/logging" "git.sendico.io/tech/sxchnge/shared/service" "github.com/go-chi/chi/v5" @@ -16,93 +19,67 @@ import ( ) func CreateAPI( - lifecycleCtx context.Context, + ctx context.Context, logger mlogger.Logger, config *api.Config, - profileConfig api.ConfigProvider, db db.Factory, - rootRouter *chi.Mux, - apiRouter *chi.Mux, + rootRouter, apiRouter *chi.Mux, debug bool, runtime api.RuntimeConfig, -) (mservice.MicroService, error) { - p := &APIImp{ - lifecycleCtx: lifecycleCtx, - logger: logger.Named("api"), - config: config, - profile: profileConfig, - db: db, - runtime: runtime, +) (_ mservice.MicroService, result error) { + if config == nil || config.Mw == nil { + return nil, merrors.InvalidArgument("API middleware configuration is required") } - - if err := p.installRuntimeBasics(logger, db, rootRouter, apiRouter, debug); err != nil { + endpoint := os.Getenv(config.Mw.EndPointEnv) + mounts, err := resolveMounts(config.Versions, endpoint, runtime.Versions) + if err != nil { return nil, err } - if err := p.bootstrapConfiguredRuntime(config, db); err != nil { - return nil, err + if runtime.Bootstrap == nil { + return nil, merrors.InvalidArgument("runtime bootstrap is required") } - - p.logger.Info("Installing microservices...", zap.String("runtime", runtime.RuntimeName())) - if err := p.installServices(); err != nil { - p.logger.Error("Failed to install a microservice", zap.Error(err)) - return nil, err - } - p.logger.Info("Microservices installation complete", zap.Int("microservices", len(p.services))) - - return p, nil -} - -func (a *APIImp) installRuntimeBasics( - logger mlogger.Logger, - db db.Factory, - rootRouter *chi.Mux, - apiRouter *chi.Mux, - debug bool, -) error { - var err error - if a.domain, err = domainprovider.CreateDomainProvider( - a.logger, - a.config.Mw.DomainEnv, - a.config.Mw.APIProtocolEnv, - a.config.Mw.EndPointEnv, - ); err != nil { - a.logger.Error("Failed to initialize domain provider") - return err - } - a.logger.Info("Domain provider installed") - - apiEndpoint := os.Getenv(a.config.Mw.EndPointEnv) - if a.health, err = amr.NewHealthRouter(a.logger, rootRouter, apiEndpoint); err != nil { - a.logger.Error("Failed to create healthcheck router", zap.Error(err), zap.String("api_endpoint", apiEndpoint)) - return err - } - if a.mw, err = httptransport.CreateMiddleware( - logger, - db, - apiRouter, - a.config.Mw, - a.config.AuthStorage, - debug, - a.config.Notifications, - ); err != nil { - a.logger.Error("Failed to create middleware", zap.Error(err)) - return err - } - a.logger.Info("Middleware installed", zap.Bool("debug_mode", debug)) - return nil -} - -func (a *APIImp) bootstrapConfiguredRuntime(_ *api.Config, _ db.Factory) error { - if a.runtime.Bootstrap != nil { - components, err := a.runtime.Bootstrap(a) + // Resolve every URL before opening resources or publishing routes. + scopes := make([]api.Scope, len(mounts)) + for i, mount := range mounts { + urls, err := domainprovider.CreateDomainProvider(logger, os.Getenv(config.Mw.DomainEnv), os.Getenv(config.Mw.APIProtocolEnv), mount.Path) if err != nil { - a.logger.Error("Failed to bootstrap runtime profile", zap.Error(err), zap.String("runtime", a.runtime.RuntimeName())) - return err + return nil, err } - a.components = components - a.logger.Info("Runtime profile bootstrap complete", - zap.String("runtime", a.runtime.RuntimeName()), - zap.Int("components", len(components))) + scopes[i] = api.Scope{Router: chi.NewRouter(), Mount: mount.Path, URLs: urls, Logger: logger.With(zap.String("api_version", mount.Version), zap.String("mount", mount.Path))} + scopes[i].Router.Use(browser.Scope(mount.Path)) } - return nil + a := &instance{logger: logger.Named("api").With(zap.String("runtime", runtime.RuntimeName()))} + defer func() { + if result != nil { + if err := a.Finish(context.Background()); err != nil { + a.logger.Warn("Failed startup resource cleanup", zap.Error(err)) + } + } + }() + a.mw, err = httptransport.CreateMiddleware(logger, apiRouter, config.Mw, endpoint, debug) + if err != nil { + return nil, err + } + deps, cleanup, err := runtime.Bootstrap(ctx, a.logger, config, db) + a.cleanup = cleanup + if err != nil { + return nil, err + } + for i, mount := range mounts { + if err := runtime.Versions[mount.Version](scopes[i], deps); err != nil { + return nil, merrors.OpaqueWrap(merrors.ErrInternal, err, "API version installation failed: "+mount.Version) + } + } + // Attach only after every installer succeeds. + for i, mount := range mounts { + apiRouter.Mount(mount.Path, scopes[i].Router) + scopes[i].Logger.Info("API version installed") + } + a.health, err = amr.NewHealthRouter(a.logger, rootRouter, endpoint) + if err != nil { + return nil, err + } + a.logger.Info("API installation complete", zap.Int("versions", len(mounts))) + a.health.SetStatus(health.SSRunning) + return a, nil } diff --git a/backend/product/app/runtime/bootstrap_test.go b/backend/product/app/runtime/bootstrap_test.go index 9b66a61..1c90b76 100644 --- a/backend/product/app/runtime/bootstrap_test.go +++ b/backend/product/app/runtime/bootstrap_test.go @@ -50,7 +50,7 @@ func TestProfileStartsOnEmptyDatabase(t *testing.T) { t.Setenv("SXCHNGE_JWT_SECRET", "test-only-signing-key-at-least-32-bytes") example.API.Notifications.CAFile = "" // This startup test makes no notification RPCs. root, routes := chi.NewRouter(), chi.NewRouter() - runtime, err := sharedruntime.CreateAPI(ctx, zap.NewNop(), &example.API, &example.API, factory, root, routes, false, profile.Runtime()) + runtime, err := sharedruntime.CreateAPI(ctx, zap.NewNop(), &example.API, factory, root, routes, false, profile.Runtime()) require.NoError(t, err) t.Cleanup(func() { require.NoError(t, runtime.Finish(context.Background())) }) root.Mount("/", routes) diff --git a/backend/product/app/runtime/lifecycle.go b/backend/product/app/runtime/lifecycle.go index 141c483..43a450a 100644 --- a/backend/product/app/runtime/lifecycle.go +++ b/backend/product/app/runtime/lifecycle.go @@ -4,28 +4,20 @@ import ( "context" "git.sendico.io/tech/sxchnge/product/app/health/status" - "go.uber.org/zap" ) -func (a *APIImp) Finish(ctx context.Context) error { - a.health.SetStatus(health.SSTerminating) - a.health.Finish() - a.mw.Finish() - var lastError error - for i := len(a.services) - 1; i >= 0; i-- { - if err := (a.services[i]).Finish(ctx); err != nil { - lastError = err - a.logger.Warn("Error occurred when finishing service", - zap.Error(err), zap.String("service_name", (a.services[i]).Name())) - } else { - a.logger.Info("Microservice is down", zap.String("service_name", (a.services[i]).Name())) +func (a *instance) Finish(ctx context.Context) error { + a.once.Do(func() { + if a.health != nil { + a.health.SetStatus(health.SSTerminating) + a.health.Finish() } - } - if a.runtime.Finish != nil { - if err := a.runtime.Finish(a); err != nil { - lastError = err - a.logger.Warn("Error occurred when finishing runtime profile", zap.String("runtime", a.runtime.RuntimeName()), zap.Error(err)) + if a.cleanup != nil { + a.finishErr = a.cleanup(ctx) } - } - return lastError + if a.mw != nil { + a.mw.Finish() + } + }) + return a.finishErr } diff --git a/backend/product/app/runtime/mounts.go b/backend/product/app/runtime/mounts.go new file mode 100644 index 0000000..2a6722c --- /dev/null +++ b/backend/product/app/runtime/mounts.go @@ -0,0 +1,58 @@ +package runtime + +import ( + "path" + "strings" + + "git.sendico.io/tech/sxchnge/product/app/api" + "git.sendico.io/tech/sxchnge/shared/errors" +) + +func validMount(value string) bool { + if value == "" || value == "/" || !strings.HasPrefix(value, "/") || path.Clean(value) != value { + return false + } + for _, c := range value { + if c >= 'a' && c <= 'z' || c >= 'A' && c <= 'Z' || c >= '0' && c <= '9' || strings.ContainsRune("/-._~", c) { + continue + } + return false + } + return true +} +func overlaps(a, b string) bool { + return a == b || strings.HasPrefix(a, b+"/") || strings.HasPrefix(b, a+"/") +} + +func resolveMounts(configured []api.Mount, endpoint string, supported map[string]api.Installer) ([]api.Mount, error) { + if !validMount(endpoint) { + return nil, merrors.InvalidArgument("a canonical absolute API endpoint is required", "api_endpoint") + } + if configured == nil { + configured = []api.Mount{{Version: "v1", Path: endpoint}} + } + seen := map[string]bool{} + for i, mount := range configured { + if supported[mount.Version] == nil { + return nil, merrors.InvalidArgument("unknown or unavailable API version: "+mount.Version, "api.versions") + } + if seen[mount.Version] { + return nil, merrors.InvalidArgument("duplicate API version: "+mount.Version, "api.versions") + } + seen[mount.Version] = true + if !validMount(mount.Path) { + return nil, merrors.InvalidArgument("invalid API mount: "+mount.Path, "api.versions") + } + for _, reserved := range []string{"/metrics", endpoint + "/metrics", endpoint + "/health"} { + if mount.Path == reserved || strings.HasPrefix(mount.Path, reserved+"/") { + return nil, merrors.InvalidArgument("API mount conflicts with operational endpoint", "api.versions") + } + } + for _, previous := range configured[:i] { + if overlaps(mount.Path, previous.Path) { + return nil, merrors.InvalidArgument("overlapping API mounts", "api.versions") + } + } + } + return configured, nil +} diff --git a/backend/product/app/runtime/mounts_test.go b/backend/product/app/runtime/mounts_test.go new file mode 100644 index 0000000..bb8a8e7 --- /dev/null +++ b/backend/product/app/runtime/mounts_test.go @@ -0,0 +1,190 @@ +package runtime + +import ( + "context" + "errors" + "net/http" + "net/http/httptest" + "os" + "sync" + "testing" + "time" + + "git.sendico.io/tech/sxchnge/product/app/api" + "git.sendico.io/tech/sxchnge/product/sessions/browser" + "git.sendico.io/tech/sxchnge/shared/database" + merrors "git.sendico.io/tech/sxchnge/shared/errors" + "git.sendico.io/tech/sxchnge/shared/logging" + "github.com/go-chi/chi/v5" + "github.com/stretchr/testify/require" + "go.uber.org/zap" + "gopkg.in/yaml.v3" +) + +func testConfig(t *testing.T) *api.Config { + t.Helper() + data, err := os.ReadFile("../../config.example.yml") + require.NoError(t, err) + var raw struct { + API api.Config `yaml:"api"` + } + require.NoError(t, yaml.Unmarshal(data, &raw)) + t.Setenv(raw.API.Mw.EndPointEnv, "/api/v1") + t.Setenv(raw.API.Mw.DomainEnv, "example.test") + t.Setenv(raw.API.Mw.APIProtocolEnv, "https") + return &raw.API +} + +func TestInstalledVersionsShareDependenciesAndKeepScopes(t *testing.T) { + config := testConfig(t) + config.Versions = []api.Mount{{Version: "v1", Path: "/api/v1"}, {Version: "v2", Path: "/gateway/v2"}} + initialized, closed := 0, 0 + deps := &api.Dependencies{} + installer := func(version string) api.Installer { + return func(scope api.Scope, got *api.Dependencies) error { + require.Same(t, deps, got) + link, err := scope.URLs.GetAPILink("accounts") + require.NoError(t, err) + require.Equal(t, "https://example.test"+scope.Mount+"/accounts", link) + scope.Router.Get("/contract", func(w http.ResponseWriter, _ *http.Request) { _, _ = w.Write([]byte(version)) }) + scope.Router.Post("/accounts/session", func(w http.ResponseWriter, r *http.Request) { + browser.SetRefreshCookie(w, r, "credential", time.Now().Add(time.Hour)) + }) + scope.Router.Delete("/accounts/session", browser.ClearRefreshCookie) + return nil + } + } + runtime := api.RuntimeConfig{Versions: map[string]api.Installer{"v1": installer("old"), "v2": installer("new"), "v3": installer("disabled")}, Bootstrap: func(context.Context, mlogger.Logger, *api.Config, db.Factory) (*api.Dependencies, func(context.Context) error, error) { + initialized++ + return deps, func(context.Context) error { closed++; return nil }, nil + }} + root, routes := chi.NewRouter(), chi.NewRouter() + srv, err := CreateAPI(context.Background(), zap.NewNop(), config, nil, root, routes, false, runtime) + require.NoError(t, err) + root.Mount("/", routes) + for target, expected := range map[string]string{"/api/v1/contract": "old", "/gateway/v2/contract": "new"} { + result := httptest.NewRecorder() + root.ServeHTTP(result, httptest.NewRequest(http.MethodGet, target, nil)) + require.Equal(t, http.StatusOK, result.Code) + require.Equal(t, expected, result.Body.String()) + } + t.Setenv("API_ENDPOINT", "/wrong/global") + for _, mount := range config.Versions { + for _, method := range []string{http.MethodPost, http.MethodDelete} { + result := httptest.NewRecorder() + request := httptest.NewRequest(method, mount.Path+"/accounts/session", nil) + request.Header.Set(browser.TransportHeader, browser.TransportCookie) + request.Header.Set(browser.CSRFHeader, browser.CSRFValue) + request.Header.Set("Origin", config.Mw.CORS.AllowedOrigins[0]) + root.ServeHTTP(result, request) + require.Equal(t, http.StatusOK, result.Code) + cookies := result.Result().Cookies() + require.Len(t, cookies, 1) + require.Equal(t, mount.Path+"/accounts", cookies[0].Path) + if method == http.MethodDelete { + require.Equal(t, -1, cookies[0].MaxAge) + } + } + } + result := httptest.NewRecorder() + root.ServeHTTP(result, httptest.NewRequest(http.MethodGet, "/api/v3/contract", nil)) + require.Equal(t, 404, result.Code) + require.Equal(t, 1, initialized) + var wg sync.WaitGroup + results := make(chan error, 3) + for range 3 { + wg.Go(func() { results <- srv.Finish(context.Background()) }) + } + wg.Wait() + close(results) + for err := range results { + require.NoError(t, err) + } + require.Equal(t, 1, closed) +} + +func TestInvalidMountsFailBeforeBootstrap(t *testing.T) { + for name, mounts := range map[string][]api.Mount{ + "unknown": {{Version: "v9", Path: "/api/v9"}}, "duplicate": {{Version: "v1", Path: "/api/v1"}, {Version: "v1", Path: "/other"}}, + "same path": {{Version: "v1", Path: "/api/v1"}, {Version: "v2", Path: "/api/v1"}}, + "nested": {{Version: "v1", Path: "/api"}, {Version: "v2", Path: "/api/v2"}}, + "cookie delimiter": {{Version: "v1", Path: "/api;v1"}}, + "control": {{Version: "v1", Path: "/api/\x01"}}, + "relative": {{Version: "v1", Path: "api/v1"}}, "wildcard": {{Version: "v1", Path: "/api/*"}}, + "unclean": {{Version: "v1", Path: "/api/../v1"}}, "metrics": {{Version: "v1", Path: "/metrics"}}, + "health": {{Version: "v1", Path: "/api/v1/health"}}, "root": {{Version: "v1", Path: "/"}}, + } { + t.Run(name, func(t *testing.T) { + config := testConfig(t) + config.Versions = mounts + install := func(api.Scope, *api.Dependencies) error { t.Fatal("installer must not run"); return nil } + _, err := CreateAPI(context.Background(), zap.NewNop(), config, nil, chi.NewRouter(), chi.NewRouter(), false, api.RuntimeConfig{Versions: map[string]api.Installer{"v1": install, "v2": install}, Bootstrap: func(context.Context, mlogger.Logger, *api.Config, db.Factory) (*api.Dependencies, func(context.Context) error, error) { + t.Fatal("bootstrap must not run") + return nil, nil, nil + }}) + require.ErrorIs(t, err, merrors.ErrInvalidArg) + }) + } +} + +func TestInstallFailureCleansResourcesWithoutPublishingRoutes(t *testing.T) { + config := testConfig(t) + config.Versions = []api.Mount{{Version: "v1", Path: "/api/v1"}, {Version: "v2", Path: "/api/v2"}} + closed := 0 + cause := merrors.InvalidArgument("test installation failure") + routes := chi.NewRouter() + _, err := CreateAPI(context.Background(), zap.NewNop(), config, nil, chi.NewRouter(), routes, false, api.RuntimeConfig{ + Bootstrap: func(context.Context, mlogger.Logger, *api.Config, db.Factory) (*api.Dependencies, func(context.Context) error, error) { + return &api.Dependencies{}, func(context.Context) error { closed++; return nil }, nil + }, + Versions: map[string]api.Installer{ + "v1": func(scope api.Scope, _ *api.Dependencies) error { + scope.Router.Get("/ready", func(w http.ResponseWriter, _ *http.Request) { w.WriteHeader(200) }) + return nil + }, + "v2": func(api.Scope, *api.Dependencies) error { return cause }, + }}) + require.True(t, errors.Is(err, cause)) + require.Equal(t, 1, closed) + result := httptest.NewRecorder() + routes.ServeHTTP(result, httptest.NewRequest(http.MethodGet, "/api/v1/ready", nil)) + require.Equal(t, 404, result.Code) +} + +func TestExplicitEmptyVersionsInstallNoAPI(t *testing.T) { + config := testConfig(t) + config.Versions = []api.Mount{} + root, routes := chi.NewRouter(), chi.NewRouter() + srv, err := CreateAPI(context.Background(), zap.NewNop(), config, nil, root, routes, false, api.RuntimeConfig{Bootstrap: func(context.Context, mlogger.Logger, *api.Config, db.Factory) (*api.Dependencies, func(context.Context) error, error) { + return &api.Dependencies{}, nil, nil + }}) + require.NoError(t, err) + defer srv.Finish(context.Background()) + root.Mount("/", routes) + for target, status := range map[string]int{"/api/v1/accounts/login": 404, "/api/v1/health": 200, "/metrics": 200, "/api/v1/metrics": 200} { + result := httptest.NewRecorder() + root.ServeHTTP(result, httptest.NewRequest(http.MethodGet, target, nil)) + require.Equal(t, status, result.Code, target) + } +} + +func TestBootstrapFailureClosesReturnedResources(t *testing.T) { + config := testConfig(t) + config.Versions = []api.Mount{} + closed := 0 + cause := merrors.Internal("bootstrap failure") + _, err := CreateAPI(context.Background(), zap.NewNop(), config, nil, chi.NewRouter(), chi.NewRouter(), false, api.RuntimeConfig{Bootstrap: func(context.Context, mlogger.Logger, *api.Config, db.Factory) (*api.Dependencies, func(context.Context) error, error) { + return nil, func(context.Context) error { closed++; return nil }, cause + }}) + require.ErrorIs(t, err, cause) + require.Equal(t, 1, closed) +} + +func TestFinishPreservesCleanupErrorAndRunsOnce(t *testing.T) { + cause := merrors.Internal("cleanup failure") + closed := 0 + srv := &instance{logger: zap.NewNop(), cleanup: func(context.Context) error { closed++; return cause }} + require.ErrorIs(t, srv.Finish(context.Background()), cause) + require.ErrorIs(t, srv.Finish(context.Background()), cause) + require.Equal(t, 1, closed) +} diff --git a/backend/product/app/runtime/services.go b/backend/product/app/runtime/services.go index d0d9ef7..af94b82 100644 --- a/backend/product/app/runtime/services.go +++ b/backend/product/app/runtime/services.go @@ -1,39 +1,69 @@ package runtime import ( + "context" + "git.sendico.io/tech/sxchnge/product/app/api" - "git.sendico.io/tech/sxchnge/product/app/health/status" - "git.sendico.io/tech/sxchnge/shared/errors" - "git.sendico.io/tech/sxchnge/shared/service" + "git.sendico.io/tech/sxchnge/product/auth/config" + "git.sendico.io/tech/sxchnge/product/auth/flow" + "git.sendico.io/tech/sxchnge/product/auth/storage" + "git.sendico.io/tech/sxchnge/product/customers" + "git.sendico.io/tech/sxchnge/product/notifications" + "git.sendico.io/tech/sxchnge/product/twofactor" + "git.sendico.io/tech/sxchnge/product/twofactor/delivery" + factorstore "git.sendico.io/tech/sxchnge/product/twofactor/store" + "git.sendico.io/tech/sxchnge/shared/database" + "git.sendico.io/tech/sxchnge/shared/logging" "go.uber.org/zap" ) -func (a *APIImp) installMicroservice(srv mservice.MicroService) { - a.services = append(a.services, srv) - a.logger.Info("Microservice installed", zap.String("service", srv.Name())) -} - -func (a *APIImp) addMicroservice(srvf api.MicroServiceFactoryT) error { - srv, err := srvf(a) +func OpenDependencies(_ context.Context, logger mlogger.Logger, config *api.Config, db db.Factory) (_ *api.Dependencies, _ func(context.Context) error, result error) { + limits, err := config.Mw.ResolveLoginRateLimits() if err != nil { - a.logger.Error("Failed to install a microservice", zap.Error(err)) - return err + return nil, nil, err } - - a.installMicroservice(srv) - return nil -} - -func (a *APIImp) installServices() error { - for _, service := range a.runtime.Services { - if service.Factory == nil { - return merrors.InvalidArgument("microservice factory is required", "runtime.services") + stores, err := storage.OpenStores(logger, db, config.AuthStorage) + if err != nil { + return nil, nil, err + } + customers, err := accountservice.FromStores(logger, &config.Mw.Password, stores, db.TransactionFactory()) + if err != nil { + return nil, nil, err + } + deps := &api.Dependencies{Stores: stores, Customers: customers, Token: config.Mw.Token, Signature: middleware.SignatureConf(config.Mw), LoginLimits: limits} + var client *notifications.Client + cleanup := func(context.Context) error { + if client != nil { + return client.Close() } - if err := a.addMicroservice(service.Factory); err != nil { - return err + return nil + } + defer func() { + if result != nil { + if err := cleanup(context.Background()); err != nil { + logger.Warn("Partial bootstrap cleanup failed", zap.Error(err)) + } + } + }() + if c := config.Notifications; c != nil { + deps.Auth = &flow.Service{Accounts: stores.Accounts, Customers: customers, Tokens: stores.Verification, Transactions: db.TransactionFactory(), Attempts: stores.LoginAttempts, Secret: deps.Signature.PrivateKey, ActivationURL: c.ActivationURL} + if err := deps.Auth.Validate(); err != nil { + return nil, nil, err + } + client, err = notifications.Open(*c) + if err != nil { + return nil, nil, err + } + deps.Auth.Sender = client + records, err := factorstore.Open(logger, db) + if err != nil { + return nil, nil, err + } + deps.Factors, err = twofactor.New(stores.Accounts, stores.Verification, records, db.TransactionFactory(), deps.Signature.PrivateKey, logger, delivery.Email{Sender: client}) + if err != nil { + return nil, nil, err } } - - a.health.SetStatus(health.SSRunning) - return nil + logger.Info("Product dependencies initialized") + return deps, cleanup, nil } diff --git a/backend/product/app/server/internal/config_test.go b/backend/product/app/server/internal/config_test.go index e2d4e8e..f42a869 100644 --- a/backend/product/app/server/internal/config_test.go +++ b/backend/product/app/server/internal/config_test.go @@ -5,6 +5,7 @@ import ( "git.sendico.io/tech/sxchnge/product/app/api" "git.sendico.io/tech/sxchnge/shared/errors" "os" + "strings" "testing" ) @@ -32,3 +33,34 @@ func TestExampleConfigMatchesCopiedRuntime(t *testing.T) { t.Fatal(err) } } + +func TestVersionSelectionPreservesOmittedAndEmptyLists(t *testing.T) { + data, err := os.ReadFile("../../../config.example.yml") + if err != nil { + t.Fatal(err) + } + for _, tc := range []struct { + name, setting string + wantNil bool + wantLength int + }{ + {name: "omitted", wantNil: true}, + {name: "disabled", setting: " versions: []\n"}, + {name: "explicit", setting: " versions: [{version: v1, path: /gateway/v1}]\n", wantLength: 1}, + } { + t.Run(tc.name, func(t *testing.T) { + instance := &Imp{runtime: api.RuntimeConfig{}} + configured := strings.Replace(string(data), "api:\n", "api:\n"+tc.setting, 1) + if err := instance.decodeConfig([]byte(configured)); err != nil { + t.Fatal(err) + } + versions := instance.config.API.SharedConfig().Versions + if (versions == nil) != tc.wantNil || len(versions) != tc.wantLength { + t.Fatalf("unexpected versions: %#v", versions) + } + if tc.wantLength == 1 && (versions[0].Version != "v1" || versions[0].Path != "/gateway/v1") { + t.Fatalf("lost version configuration: %#v", versions) + } + }) + } +} diff --git a/backend/product/app/server/internal/server.go b/backend/product/app/server/internal/server.go index a1821ae..2f121b3 100644 --- a/backend/product/app/server/internal/server.go +++ b/backend/product/app/server/internal/server.go @@ -112,7 +112,7 @@ func (i *Imp) Start() error { rootRouter := chi.NewRouter() apiRouter := chi.NewRouter() - if i.api, err = sharedruntime.CreateAPI(i.lifecycleCtx, i.logger, i.config.API.SharedConfig(), i.config.API, i.db, rootRouter, apiRouter, i.debug, i.runtime); err != nil { + if i.api, err = sharedruntime.CreateAPI(i.lifecycleCtx, i.logger, i.config.API.SharedConfig(), i.db, rootRouter, apiRouter, i.debug, i.runtime); err != nil { i.db.CloseConnection() return err } diff --git a/backend/product/app/urls/domain_provider.go b/backend/product/app/urls/domain_provider.go index ca5eab4..2edab81 100644 --- a/backend/product/app/urls/domain_provider.go +++ b/backend/product/app/urls/domain_provider.go @@ -10,6 +10,7 @@ type DomainProvider interface { GetAPILink(linkElem ...string) (string, error) } +// CreateDomainProvider accepts a resolved host, protocol and installation mount. func CreateDomainProvider(logger mlogger.Logger, domain, protocol, apiEndpoint string) (DomainProvider, error) { return dimp.CreateDomainProviderImp(logger, domain, protocol, apiEndpoint) } diff --git a/backend/product/app/urls/internal/domain_provider.go b/backend/product/app/urls/internal/domain_provider.go index 0c1bc70..ab7eceb 100644 --- a/backend/product/app/urls/internal/domain_provider.go +++ b/backend/product/app/urls/internal/domain_provider.go @@ -2,8 +2,8 @@ package domainproviderimp import ( "net/url" - "os" + "git.sendico.io/tech/sxchnge/shared/errors" "git.sendico.io/tech/sxchnge/shared/logging" "go.uber.org/zap" ) @@ -21,11 +21,19 @@ func (dp *DomainProviderImp) GetAPILink(linkElem ...string) (string, error) { return url.JoinPath(dp.apiEndpoint, linkElem...) } -func CreateDomainProviderImp(logger mlogger.Logger, domain, protocol, apiPointEnv string) (*DomainProviderImp, error) { +func CreateDomainProviderImp(logger mlogger.Logger, domain, protocol, apiPath string) (*DomainProviderImp, error) { + if protocol != "http" && protocol != "https" { + return nil, merrors.InvalidArgument("API protocol must be http or https") + } + parsed, err := url.Parse(protocol + "://" + domain) + if err != nil { + return nil, merrors.OpaqueWrap(merrors.ErrInvalidArg, err, "invalid API origin") + } + if parsed.Hostname() == "" || parsed.Host != domain || parsed.User != nil || parsed.Path != "" || parsed.RawQuery != "" || parsed.Fragment != "" { + return nil, merrors.InvalidArgument("API domain must be a host with optional port") + } p := new(DomainProviderImp) - p.domain = os.Getenv(protocol) + "://" + os.Getenv(domain) - var err error - apiPath := os.Getenv(apiPointEnv) + p.domain = protocol + "://" + domain if p.apiEndpoint, err = url.JoinPath(p.domain, apiPath); err != nil { logger.Error("Failed to create domain provider", zap.String("domain", p.domain), zap.String("api_path", apiPath)) return nil, err diff --git a/backend/product/app/urls/internal/domain_provider_test.go b/backend/product/app/urls/internal/domain_provider_test.go new file mode 100644 index 0000000..7d358f9 --- /dev/null +++ b/backend/product/app/urls/internal/domain_provider_test.go @@ -0,0 +1,27 @@ +package domainproviderimp + +import ( + "testing" + + "github.com/stretchr/testify/require" + "go.uber.org/zap" +) + +func TestExplicitOriginAndMount(t *testing.T) { + provider, err := CreateDomainProviderImp(zap.NewNop(), "localhost:8082", "https", "/gateway/v2") + require.NoError(t, err) + link, err := provider.GetAPILink("accounts", "me") + require.NoError(t, err) + require.Equal(t, "https://localhost:8082/gateway/v2/accounts/me", link) + link, err = provider.GetFullLink("signup") + require.NoError(t, err) + require.Equal(t, "https://localhost:8082/signup", link) +} +func TestInvalidOrigin(t *testing.T) { + for _, domain := range []string{"", "host/path", "user@host", "host?query", "host?", "host#", "host#fragment", "bad host"} { + _, err := CreateDomainProviderImp(zap.NewNop(), domain, "https", "/v1") + require.Error(t, err, domain) + } + _, err := CreateDomainProviderImp(zap.NewNop(), "host", "ftp", "/v1") + require.Error(t, err) +} diff --git a/backend/product/architecture/versions_test.go b/backend/product/architecture/versions_test.go new file mode 100644 index 0000000..ad5480a --- /dev/null +++ b/backend/product/architecture/versions_test.go @@ -0,0 +1,41 @@ +package architecture + +import ( + "go/parser" + "go/token" + "io/fs" + "path/filepath" + "strconv" + "strings" + "testing" +) + +func TestGenericRuntimeDoesNotInstallConcreteContracts(t *testing.T) { + for _, dir := range []string{"../app/runtime", "../app/http"} { + err := filepath.WalkDir(dir, func(path string, entry fs.DirEntry, err error) error { + if err != nil { + return err + } + if entry.IsDir() || !strings.HasSuffix(path, ".go") || strings.HasSuffix(path, "_test.go") { + return nil + } + file, err := parser.ParseFile(token.NewFileSet(), path, nil, parser.ImportsOnly) + if err != nil { + return err + } + for _, im := range file.Imports { + name, err := strconv.Unquote(im.Path.Value) + if err != nil { + return err + } + if strings.Contains(name, "/http/v") || strings.Contains(name, "/app/api/v") || strings.HasSuffix(name, "/app/profile") { + t.Errorf("generic runtime imports a concrete API contract: %s in %s", name, path) + } + } + return nil + }) + if err != nil { + t.Fatal(err) + } + } +} diff --git a/backend/product/auth/http/authorized/customer_test.go b/backend/product/auth/http/v1/authorized/customer_test.go similarity index 91% rename from backend/product/auth/http/authorized/customer_test.go rename to backend/product/auth/http/v1/authorized/customer_test.go index 979ae33..63de01a 100644 --- a/backend/product/auth/http/authorized/customer_test.go +++ b/backend/product/auth/http/v1/authorized/customer_test.go @@ -9,8 +9,8 @@ import ( "testing" "git.sendico.io/tech/sxchnge/product/auth/config" + "git.sendico.io/tech/sxchnge/product/auth/http/v1/response" emodel "git.sendico.io/tech/sxchnge/product/auth/model" - "git.sendico.io/tech/sxchnge/product/auth/response" "git.sendico.io/tech/sxchnge/product/customers/store" "git.sendico.io/tech/sxchnge/shared/errors" api "git.sendico.io/tech/sxchnge/shared/http" @@ -48,7 +48,7 @@ func TestCustomerHandlerResolvesIdentityWithoutRoles(t *testing.T) { accounts.customer.Status = status routes := chi.NewRouter() signature := &middleware.Signature{Algorithm: "HS256", PrivateKey: []byte("test-only-signing-key-at-least-32-bytes")} - router := NewRouter(zap.NewNop(), "/api/v1", routes, accounts, &sessionDB{token: &sessionmodel.RefreshToken{}}, &middleware.TokenConfig{Expiration: middleware.TokenExpiration{Account: 1}}, signature) + router := Install(zap.NewNop(), "/api/v1", routes, accounts, &sessionDB{token: &sessionmodel.RefreshToken{}}, &middleware.TokenConfig{Expiration: middleware.TokenExpiration{Account: 1}}, signature) called := false router.AccountHandler(mservice.Accounts, "/me", api.Get, func(_ *http.Request, c *customermodel.Account, _ *sresponse.TokenData) http.HandlerFunc { called = true diff --git a/backend/product/auth/http/authorized/handler.go b/backend/product/auth/http/v1/authorized/handler.go similarity index 99% rename from backend/product/auth/http/authorized/handler.go rename to backend/product/auth/http/v1/authorized/handler.go index d670d26..a49b8dd 100644 --- a/backend/product/auth/http/authorized/handler.go +++ b/backend/product/auth/http/v1/authorized/handler.go @@ -6,9 +6,9 @@ import ( "net/http" "strings" + "git.sendico.io/tech/sxchnge/product/auth/http/v1/response" "git.sendico.io/tech/sxchnge/product/auth/ipguard" emodel "git.sendico.io/tech/sxchnge/product/auth/model" - "git.sendico.io/tech/sxchnge/product/auth/response" "git.sendico.io/tech/sxchnge/product/sessions/metrics" "git.sendico.io/tech/sxchnge/shared/errors" api "git.sendico.io/tech/sxchnge/shared/http" diff --git a/backend/product/auth/http/authorized/router.go b/backend/product/auth/http/v1/authorized/router.go similarity index 89% rename from backend/product/auth/http/authorized/router.go rename to backend/product/auth/http/v1/authorized/router.go index 17b1495..df9af97 100644 --- a/backend/product/auth/http/authorized/router.go +++ b/backend/product/auth/http/v1/authorized/router.go @@ -4,7 +4,7 @@ import ( "net/http" "git.sendico.io/tech/sxchnge/product/auth/config" - re "git.sendico.io/tech/sxchnge/product/auth/http/endpoint" + re "git.sendico.io/tech/sxchnge/product/auth/http/v1/endpoint" emodel "git.sendico.io/tech/sxchnge/product/auth/model" "git.sendico.io/tech/sxchnge/product/customers/store" "git.sendico.io/tech/sxchnge/product/sessions/metrics" @@ -27,7 +27,7 @@ type AuthorizedRouter struct { service mservice.Type } -func NewRouter(logger mlogger.Logger, apiEndpoint string, router chi.Router, db account.DB, rtdb refreshtokens.DB, config *middleware.TokenConfig, signature *middleware.Signature) *AuthorizedRouter { +func Install(logger mlogger.Logger, apiEndpoint string, router chi.Router, db account.DB, rtdb refreshtokens.DB, config *middleware.TokenConfig, signature *middleware.Signature) *AuthorizedRouter { ja := jwtauth.New(signature.Algorithm, signature.PrivateKey, signature.PublicKey) router.Use(jwtauth.Verifier(ja)) router.Use(jwtauth.Authenticator(ja)) diff --git a/backend/product/auth/http/authorized/session_test.go b/backend/product/auth/http/v1/authorized/session_test.go similarity index 100% rename from backend/product/auth/http/authorized/session_test.go rename to backend/product/auth/http/v1/authorized/session_test.go diff --git a/backend/product/auth/http/dispatcher.go b/backend/product/auth/http/v1/dispatcher.go similarity index 73% rename from backend/product/auth/http/dispatcher.go rename to backend/product/auth/http/v1/dispatcher.go index 01e383b..dd330dd 100644 --- a/backend/product/auth/http/dispatcher.go +++ b/backend/product/auth/http/v1/dispatcher.go @@ -1,12 +1,10 @@ package router import ( - "os" - "git.sendico.io/tech/sxchnge/product/auth/config" - rauthorized "git.sendico.io/tech/sxchnge/product/auth/http/authorized" - rpublic "git.sendico.io/tech/sxchnge/product/auth/http/public" - "git.sendico.io/tech/sxchnge/product/auth/response" + rauthorized "git.sendico.io/tech/sxchnge/product/auth/http/v1/authorized" + rpublic "git.sendico.io/tech/sxchnge/product/auth/http/v1/public" + "git.sendico.io/tech/sxchnge/product/auth/http/v1/response" "git.sendico.io/tech/sxchnge/product/customers/store" "git.sendico.io/tech/sxchnge/product/sessions/store" api "git.sendico.io/tech/sxchnge/shared/http" @@ -34,13 +32,15 @@ func (d *Dispatcher) PendingAccountHandler(service mservice.Type, endpoint strin d.protected.PendingAccountHandler(service, endpoint, method, handler) } -func NewDispatcher( +func Install( logger mlogger.Logger, router chi.Router, + endpoint string, db account.DB, rtdb refreshtokens.DB, attempts ratelimits.Store, - config *middleware.Config, + token *middleware.TokenConfig, + signature middleware.Signature, loginRateLimits middleware.LoginRateLimits, flows ...rpublic.Flows, ) *Dispatcher { @@ -49,13 +49,11 @@ func NewDispatcher( } d.logger.Debug("Installing endpoints middleware...") - endpoint := os.Getenv(config.EndPointEnv) - signature := middleware.SignatureConf(config) router.Group(func(r chi.Router) { - d.public = rpublic.NewRouter(d.logger, endpoint, db, rtdb, attempts, r, &config.Token, &signature, loginRateLimits, flows...) + d.public = rpublic.Install(d.logger, endpoint, db, rtdb, attempts, r, token, &signature, loginRateLimits, flows...) }) router.Group(func(r chi.Router) { - d.protected = rauthorized.NewRouter(d.logger, endpoint, r, db, rtdb, &config.Token, &signature) + d.protected = rauthorized.Install(d.logger, endpoint, r, db, rtdb, token, &signature) }) return d diff --git a/backend/product/auth/http/endpoint/endpoint.go b/backend/product/auth/http/v1/endpoint/endpoint.go similarity index 94% rename from backend/product/auth/http/endpoint/endpoint.go rename to backend/product/auth/http/v1/endpoint/endpoint.go index 8f7642d..d353f51 100644 --- a/backend/product/auth/http/endpoint/endpoint.go +++ b/backend/product/auth/http/v1/endpoint/endpoint.go @@ -7,7 +7,7 @@ import ( "net/http" "git.sendico.io/tech/sxchnge/product/auth/config" - "git.sendico.io/tech/sxchnge/product/auth/response" + "git.sendico.io/tech/sxchnge/product/auth/http/v1/response" "git.sendico.io/tech/sxchnge/shared/logging" "github.com/go-chi/chi/v5" ) diff --git a/backend/product/auth/http/endpoint/install.go b/backend/product/auth/http/v1/endpoint/install.go similarity index 93% rename from backend/product/auth/http/endpoint/install.go rename to backend/product/auth/http/v1/endpoint/install.go index 2c94d88..f3bdee1 100644 --- a/backend/product/auth/http/endpoint/install.go +++ b/backend/product/auth/http/v1/endpoint/install.go @@ -4,7 +4,7 @@ import ( "net/http" "path" - "git.sendico.io/tech/sxchnge/product/auth/response" + "git.sendico.io/tech/sxchnge/product/auth/http/v1/response" api "git.sendico.io/tech/sxchnge/shared/http" "git.sendico.io/tech/sxchnge/shared/service" "github.com/go-chi/chi/v5" @@ -33,7 +33,7 @@ func (er *HttpEndpointRouter) chooseMethod(method api.HTTPMethod) RegistratorT { } func (er *HttpEndpointRouter) endpoint(service mservice.Type, handler string) string { - return path.Join(er.apiEndpoint, service, handler) + return path.Join("/", er.apiEndpoint, service, handler) } func (er *HttpEndpointRouter) InstallHandler(service mservice.Type, endpoint string, method api.HTTPMethod, handler sresponse.HandlerFunc) { diff --git a/backend/product/auth/http/endpoint/token.go b/backend/product/auth/http/v1/endpoint/token.go similarity index 95% rename from backend/product/auth/http/endpoint/token.go rename to backend/product/auth/http/v1/endpoint/token.go index b6df9e8..f0c0c9e 100644 --- a/backend/product/auth/http/endpoint/token.go +++ b/backend/product/auth/http/v1/endpoint/token.go @@ -5,8 +5,8 @@ import ( sessionmodel "git.sendico.io/tech/sxchnge/product/sessions/model" "time" + "git.sendico.io/tech/sxchnge/product/auth/http/v1/response" emodel "git.sendico.io/tech/sxchnge/product/auth/model" - "git.sendico.io/tech/sxchnge/product/auth/response" "github.com/go-chi/jwtauth/v5" ) diff --git a/backend/product/auth/http/public/login.go b/backend/product/auth/http/v1/public/login.go similarity index 98% rename from backend/product/auth/http/public/login.go rename to backend/product/auth/http/v1/public/login.go index 7f25dce..bf863a9 100644 --- a/backend/product/auth/http/public/login.go +++ b/backend/product/auth/http/v1/public/login.go @@ -9,9 +9,9 @@ import ( "strings" clientauth "git.sendico.io/tech/sxchnge/product/auth/client" + "git.sendico.io/tech/sxchnge/product/auth/http/v1/request" + "git.sendico.io/tech/sxchnge/product/auth/http/v1/response" "git.sendico.io/tech/sxchnge/product/auth/ipguard" - "git.sendico.io/tech/sxchnge/product/auth/request" - "git.sendico.io/tech/sxchnge/product/auth/response" customermodel "git.sendico.io/tech/sxchnge/product/customers/model" sessionmodel "git.sendico.io/tech/sxchnge/product/sessions/model" "git.sendico.io/tech/sxchnge/shared/errors" diff --git a/backend/product/auth/http/public/login_rate_limit.go b/backend/product/auth/http/v1/public/login_rate_limit.go similarity index 98% rename from backend/product/auth/http/public/login_rate_limit.go rename to backend/product/auth/http/v1/public/login_rate_limit.go index e7520e6..a0ecb2f 100644 --- a/backend/product/auth/http/public/login_rate_limit.go +++ b/backend/product/auth/http/v1/public/login_rate_limit.go @@ -9,8 +9,8 @@ import ( "time" "git.sendico.io/tech/sxchnge/product/auth/config" + "git.sendico.io/tech/sxchnge/product/auth/http/v1/request" "git.sendico.io/tech/sxchnge/product/auth/ipguard" - "git.sendico.io/tech/sxchnge/product/auth/request" "git.sendico.io/tech/sxchnge/shared/errors" "git.sendico.io/tech/sxchnge/shared/http/response" "git.sendico.io/tech/sxchnge/shared/ratelimit/store" diff --git a/backend/product/auth/http/public/login_rate_limit_test.go b/backend/product/auth/http/v1/public/login_rate_limit_test.go similarity index 99% rename from backend/product/auth/http/public/login_rate_limit_test.go rename to backend/product/auth/http/v1/public/login_rate_limit_test.go index 25785c9..54cd0ca 100644 --- a/backend/product/auth/http/public/login_rate_limit_test.go +++ b/backend/product/auth/http/v1/public/login_rate_limit_test.go @@ -13,7 +13,7 @@ import ( "time" "git.sendico.io/tech/sxchnge/product/auth/config" - "git.sendico.io/tech/sxchnge/product/auth/request" + "git.sendico.io/tech/sxchnge/product/auth/http/v1/request" "git.sendico.io/tech/sxchnge/shared/errors" "git.sendico.io/tech/sxchnge/shared/service" "go.uber.org/zap" diff --git a/backend/product/auth/http/public/login_test.go b/backend/product/auth/http/v1/public/login_test.go similarity index 99% rename from backend/product/auth/http/public/login_test.go rename to backend/product/auth/http/v1/public/login_test.go index 1a589d8..cf381ca 100644 --- a/backend/product/auth/http/public/login_test.go +++ b/backend/product/auth/http/v1/public/login_test.go @@ -11,7 +11,7 @@ import ( "testing" "git.sendico.io/tech/sxchnge/product/auth/config" - "git.sendico.io/tech/sxchnge/product/auth/request" + "git.sendico.io/tech/sxchnge/product/auth/http/v1/request" "git.sendico.io/tech/sxchnge/shared/errors" "git.sendico.io/tech/sxchnge/shared/service" "github.com/go-chi/chi/v5" @@ -183,7 +183,7 @@ func TestAPILoginAllowsLoginAgainAfterRevokeSameSession(t *testing.T) { PrivateKey: []byte("test-secret"), } - router := NewRouter( + router := Install( zap.NewNop(), "/", &testAccountDB{result: account}, diff --git a/backend/product/auth/http/public/login_test_support_test.go b/backend/product/auth/http/v1/public/login_test_support_test.go similarity index 100% rename from backend/product/auth/http/public/login_test_support_test.go rename to backend/product/auth/http/v1/public/login_test_support_test.go diff --git a/backend/product/auth/http/public/logout.go b/backend/product/auth/http/v1/public/logout.go similarity index 97% rename from backend/product/auth/http/public/logout.go rename to backend/product/auth/http/v1/public/logout.go index c4fa2e1..a3c8368 100644 --- a/backend/product/auth/http/public/logout.go +++ b/backend/product/auth/http/v1/public/logout.go @@ -4,7 +4,7 @@ import ( "encoding/json" "net/http" - "git.sendico.io/tech/sxchnge/product/auth/request" + "git.sendico.io/tech/sxchnge/product/auth/http/v1/request" "git.sendico.io/tech/sxchnge/product/sessions/browser" "git.sendico.io/tech/sxchnge/product/sessions/metrics" "git.sendico.io/tech/sxchnge/shared/http/response" diff --git a/backend/product/auth/http/public/logout_test.go b/backend/product/auth/http/v1/public/logout_test.go similarity index 100% rename from backend/product/auth/http/public/logout_test.go rename to backend/product/auth/http/v1/public/logout_test.go diff --git a/backend/product/auth/http/public/metrics.go b/backend/product/auth/http/v1/public/metrics.go similarity index 94% rename from backend/product/auth/http/public/metrics.go rename to backend/product/auth/http/v1/public/metrics.go index c2baa27..4be3e7a 100644 --- a/backend/product/auth/http/public/metrics.go +++ b/backend/product/auth/http/v1/public/metrics.go @@ -4,7 +4,7 @@ import ( "net/http" "time" - sresponse "git.sendico.io/tech/sxchnge/product/auth/response" + sresponse "git.sendico.io/tech/sxchnge/product/auth/http/v1/response" "github.com/go-chi/chi/v5/middleware" "github.com/prometheus/client_golang/prometheus" "github.com/prometheus/client_golang/prometheus/promauto" diff --git a/backend/product/auth/http/public/refresh.go b/backend/product/auth/http/v1/public/refresh.go similarity index 93% rename from backend/product/auth/http/public/refresh.go rename to backend/product/auth/http/v1/public/refresh.go index 53f31a0..3278af2 100644 --- a/backend/product/auth/http/public/refresh.go +++ b/backend/product/auth/http/v1/public/refresh.go @@ -6,8 +6,8 @@ import ( "net/http" "time" - "git.sendico.io/tech/sxchnge/product/auth/request" - "git.sendico.io/tech/sxchnge/product/auth/response" + "git.sendico.io/tech/sxchnge/product/auth/http/v1/request" + "git.sendico.io/tech/sxchnge/product/auth/http/v1/response" "git.sendico.io/tech/sxchnge/product/sessions/browser" "git.sendico.io/tech/sxchnge/shared/http/response" "git.sendico.io/tech/sxchnge/shared/service" diff --git a/backend/product/auth/http/public/refresh_test.go b/backend/product/auth/http/v1/public/refresh_test.go similarity index 99% rename from backend/product/auth/http/public/refresh_test.go rename to backend/product/auth/http/v1/public/refresh_test.go index d74b3c4..d78941a 100644 --- a/backend/product/auth/http/public/refresh_test.go +++ b/backend/product/auth/http/v1/public/refresh_test.go @@ -47,7 +47,7 @@ func newBrowserRefreshRouter(t *testing.T, expiresAt time.Time) (*PublicRouter, Algorithm: "HS256", PrivateKey: []byte("browser-refresh-test-secret"), } - return NewRouter( + return Install( zap.NewNop(), "/api/v1", &testAccountDB{result: account}, diff --git a/backend/product/auth/http/public/respond.go b/backend/product/auth/http/v1/public/respond.go similarity index 96% rename from backend/product/auth/http/public/respond.go rename to backend/product/auth/http/v1/public/respond.go index e9d96e8..466f2b8 100644 --- a/backend/product/auth/http/public/respond.go +++ b/backend/product/auth/http/v1/public/respond.go @@ -6,7 +6,7 @@ import ( sessionmodel "git.sendico.io/tech/sxchnge/product/sessions/model" "net/http" - "git.sendico.io/tech/sxchnge/product/auth/response" + "git.sendico.io/tech/sxchnge/product/auth/http/v1/response" rtokens "git.sendico.io/tech/sxchnge/product/sessions/tokens" "git.sendico.io/tech/sxchnge/shared/http/response" "git.sendico.io/tech/sxchnge/shared/logging/fields" diff --git a/backend/product/auth/http/public/rotate.go b/backend/product/auth/http/v1/public/rotate.go similarity index 95% rename from backend/product/auth/http/public/rotate.go rename to backend/product/auth/http/v1/public/rotate.go index a72cbb9..b857680 100644 --- a/backend/product/auth/http/public/rotate.go +++ b/backend/product/auth/http/v1/public/rotate.go @@ -5,7 +5,7 @@ import ( "errors" "net/http" - "git.sendico.io/tech/sxchnge/product/auth/request" + "git.sendico.io/tech/sxchnge/product/auth/http/v1/request" "git.sendico.io/tech/sxchnge/shared/http/response" "git.sendico.io/tech/sxchnge/shared/service" "go.uber.org/zap" diff --git a/backend/product/auth/http/public/router.go b/backend/product/auth/http/v1/public/router.go similarity index 95% rename from backend/product/auth/http/public/router.go rename to backend/product/auth/http/v1/public/router.go index 47e60b2..0e00462 100644 --- a/backend/product/auth/http/public/router.go +++ b/backend/product/auth/http/v1/public/router.go @@ -5,8 +5,8 @@ import ( "git.sendico.io/tech/sxchnge/product/auth/config" "git.sendico.io/tech/sxchnge/product/auth/flow" - re "git.sendico.io/tech/sxchnge/product/auth/http/endpoint" - "git.sendico.io/tech/sxchnge/product/auth/response" + re "git.sendico.io/tech/sxchnge/product/auth/http/v1/endpoint" + "git.sendico.io/tech/sxchnge/product/auth/http/v1/response" "git.sendico.io/tech/sxchnge/product/customers/store" "git.sendico.io/tech/sxchnge/product/sessions/store" "git.sendico.io/tech/sxchnge/product/twofactor" @@ -41,7 +41,7 @@ func (pr *PublicRouter) InstallHandler(service mservice.Type, endpoint string, m pr.imp.InstallHandler(service, endpoint, method, handler) } -func NewRouter( +func Install( logger mlogger.Logger, apiEndpoint string, db account.DB, diff --git a/backend/product/auth/http/public/validate.go b/backend/product/auth/http/v1/public/validate.go similarity index 96% rename from backend/product/auth/http/public/validate.go rename to backend/product/auth/http/v1/public/validate.go index b8fab90..dbe5cb6 100644 --- a/backend/product/auth/http/public/validate.go +++ b/backend/product/auth/http/v1/public/validate.go @@ -8,8 +8,8 @@ import ( "net/http" "time" - "git.sendico.io/tech/sxchnge/product/auth/request" - "git.sendico.io/tech/sxchnge/product/auth/response" + "git.sendico.io/tech/sxchnge/product/auth/http/v1/request" + "git.sendico.io/tech/sxchnge/product/auth/http/v1/response" "git.sendico.io/tech/sxchnge/product/sessions/browser" "git.sendico.io/tech/sxchnge/shared/errors" "git.sendico.io/tech/sxchnge/shared/logging/fields" diff --git a/backend/product/auth/http/public/verification.go b/backend/product/auth/http/v1/public/verification.go similarity index 99% rename from backend/product/auth/http/public/verification.go rename to backend/product/auth/http/v1/public/verification.go index 209f9a9..2128473 100644 --- a/backend/product/auth/http/public/verification.go +++ b/backend/product/auth/http/v1/public/verification.go @@ -11,9 +11,9 @@ import ( "net/http" "time" + sresponse "git.sendico.io/tech/sxchnge/product/auth/http/v1/response" "git.sendico.io/tech/sxchnge/product/auth/ipguard" authmodel "git.sendico.io/tech/sxchnge/product/auth/model" - sresponse "git.sendico.io/tech/sxchnge/product/auth/response" customer "git.sendico.io/tech/sxchnge/product/customers/model" session "git.sendico.io/tech/sxchnge/product/sessions/model" factor "git.sendico.io/tech/sxchnge/product/twofactor/model" diff --git a/backend/product/auth/http/public/verification_integration_test.go b/backend/product/auth/http/v1/public/verification_integration_test.go similarity index 95% rename from backend/product/auth/http/public/verification_integration_test.go rename to backend/product/auth/http/v1/public/verification_integration_test.go index b89ce5a..7604d81 100644 --- a/backend/product/auth/http/public/verification_integration_test.go +++ b/backend/product/auth/http/v1/public/verification_integration_test.go @@ -71,7 +71,7 @@ func TestSignupActivationAndLoginEmailFlow(t *testing.T) { factors, err := twofactor.New(stores.Accounts, stores.Verification, records, factory.TransactionFactory(), service.Secret, zap.NewNop(), delivery.Email{Sender: mail}) require.NoError(t, err) routes := chi.NewRouter() - NewRouter(zap.NewNop(), "/api/v1", stores.Accounts, sessions, nil, routes, &middleware.TokenConfig{Length: 32, Expiration: middleware.TokenExpiration{Account: 1, Refresh: 24}}, &middleware.Signature{Algorithm: "HS256", PrivateKey: service.Secret}, middleware.LoginRateLimits{}, Flows{Onboarding: service, Factors: factors}) + Install(zap.NewNop(), "/api/v1", stores.Accounts, sessions, nil, routes, &middleware.TokenConfig{Length: 32, Expiration: middleware.TokenExpiration{Account: 1, Refresh: 24}}, &middleware.Signature{Algorithm: "HS256", PrivateKey: service.Secret}, middleware.LoginRateLimits{}, Flows{Onboarding: service, Factors: factors}) post := func(path string, body any) *httptest.ResponseRecorder { raw, err := json.Marshal(body) require.NoError(t, err) diff --git a/backend/product/auth/request/login.go b/backend/product/auth/http/v1/request/login.go similarity index 100% rename from backend/product/auth/request/login.go rename to backend/product/auth/http/v1/request/login.go diff --git a/backend/product/auth/request/refresh.go b/backend/product/auth/http/v1/request/refresh.go similarity index 100% rename from backend/product/auth/request/refresh.go rename to backend/product/auth/http/v1/request/refresh.go diff --git a/backend/product/auth/request/rotate.go b/backend/product/auth/http/v1/request/rotate.go similarity index 100% rename from backend/product/auth/request/rotate.go rename to backend/product/auth/http/v1/request/rotate.go diff --git a/backend/product/auth/request/validateable.go b/backend/product/auth/http/v1/request/validateable.go similarity index 100% rename from backend/product/auth/request/validateable.go rename to backend/product/auth/http/v1/request/validateable.go diff --git a/backend/product/auth/request/verification.go b/backend/product/auth/http/v1/request/verification.go similarity index 100% rename from backend/product/auth/request/verification.go rename to backend/product/auth/http/v1/request/verification.go diff --git a/backend/product/auth/response/account.go b/backend/product/auth/http/v1/response/account.go similarity index 100% rename from backend/product/auth/response/account.go rename to backend/product/auth/http/v1/response/account.go diff --git a/backend/product/auth/response/authresp.go b/backend/product/auth/http/v1/response/authresp.go similarity index 100% rename from backend/product/auth/response/authresp.go rename to backend/product/auth/http/v1/response/authresp.go diff --git a/backend/product/auth/response/file.go b/backend/product/auth/http/v1/response/file.go similarity index 100% rename from backend/product/auth/response/file.go rename to backend/product/auth/http/v1/response/file.go diff --git a/backend/product/auth/response/login.go b/backend/product/auth/http/v1/response/login.go similarity index 100% rename from backend/product/auth/response/login.go rename to backend/product/auth/http/v1/response/login.go diff --git a/backend/product/auth/response/login_pending.go b/backend/product/auth/http/v1/response/login_pending.go similarity index 100% rename from backend/product/auth/response/login_pending.go rename to backend/product/auth/http/v1/response/login_pending.go diff --git a/backend/product/auth/response/login_test.go b/backend/product/auth/http/v1/response/login_test.go similarity index 100% rename from backend/product/auth/response/login_test.go rename to backend/product/auth/http/v1/response/login_test.go diff --git a/backend/product/auth/response/response.go b/backend/product/auth/http/v1/response/response.go similarity index 100% rename from backend/product/auth/response/response.go rename to backend/product/auth/http/v1/response/response.go diff --git a/backend/product/auth/response/result.go b/backend/product/auth/http/v1/response/result.go similarity index 100% rename from backend/product/auth/response/result.go rename to backend/product/auth/http/v1/response/result.go diff --git a/backend/product/auth/response/token.go b/backend/product/auth/http/v1/response/token.go similarity index 100% rename from backend/product/auth/response/token.go rename to backend/product/auth/http/v1/response/token.go diff --git a/backend/product/auth/response/verification.go b/backend/product/auth/http/v1/response/verification.go similarity index 100% rename from backend/product/auth/response/verification.go rename to backend/product/auth/http/v1/response/verification.go diff --git a/backend/product/auth/http/router.go b/backend/product/auth/http/v1/router.go similarity index 89% rename from backend/product/auth/http/router.go rename to backend/product/auth/http/v1/router.go index 2580c5b..ccfecdb 100644 --- a/backend/product/auth/http/router.go +++ b/backend/product/auth/http/v1/router.go @@ -1,7 +1,7 @@ package router import ( - "git.sendico.io/tech/sxchnge/product/auth/response" + "git.sendico.io/tech/sxchnge/product/auth/http/v1/response" api "git.sendico.io/tech/sxchnge/shared/http" "git.sendico.io/tech/sxchnge/shared/service" ) diff --git a/backend/product/config.example.yml b/backend/product/config.example.yml index b455b11..899eb3b 100644 --- a/backend/product/config.example.yml +++ b/backend/product/config.example.yml @@ -9,6 +9,10 @@ database: uri_env: SXCHNGE_MONGO_URI database: sxchnge api: + # Optional: omit for v1 at SXCHNGE_API_ENDPOINT; [] disables application APIs. + # versions: + # - version: v1 + # path: /api/v1 middleware: domain_env: SXCHNGE_DOMAIN api_endpoint_env: SXCHNGE_API_ENDPOINT diff --git a/backend/product/customers/internal/service.go b/backend/product/customers/internal/service.go index cf73553..9d031de 100644 --- a/backend/product/customers/internal/service.go +++ b/backend/product/customers/internal/service.go @@ -243,6 +243,13 @@ func NewAccountService( if err != nil { return nil, err } - return &service{logger: logger, config: config, accountDB: stores.Accounts, - vdb: stores.Verification, sessions: stores.RefreshTokens, transactions: dbf.TransactionFactory()}, nil + return FromStores(l, config, stores, dbf.TransactionFactory()) +} + +func FromStores(l mlogger.Logger, config *middleware.PasswordConfig, stores *authstorage.Stores, transactions transaction.Factory) (*service, error) { + if config == nil || stores == nil || transactions == nil { + return nil, merrors.InvalidArgument("customer dependencies are required") + } + return &service{logger: l.Named("account_service"), config: config, accountDB: stores.Accounts, + vdb: stores.Verification, sessions: stores.RefreshTokens, transactions: transactions}, nil } diff --git a/backend/product/customers/types.go b/backend/product/customers/types.go index fa3b0e7..d885790 100644 --- a/backend/product/customers/types.go +++ b/backend/product/customers/types.go @@ -2,11 +2,13 @@ package accountservice import ( "context" - customermodel "git.sendico.io/tech/sxchnge/product/customers/model" "git.sendico.io/tech/sxchnge/product/auth/config" + authstorage "git.sendico.io/tech/sxchnge/product/auth/storage" accountserviceimp "git.sendico.io/tech/sxchnge/product/customers/internal" + customermodel "git.sendico.io/tech/sxchnge/product/customers/model" "git.sendico.io/tech/sxchnge/shared/database" + "git.sendico.io/tech/sxchnge/shared/database/transaction" "git.sendico.io/tech/sxchnge/shared/logging" "go.mongodb.org/mongo-driver/v2/bson" ) @@ -61,3 +63,8 @@ func NewAccountService( ) (AccountService, error) { return accountserviceimp.NewAccountService(logger, dbf, config, storageConfig) } + +// FromStores composes a customer service using already initialized storage. +func FromStores(logger mlogger.Logger, config *middleware.PasswordConfig, stores *authstorage.Stores, transactions transaction.Factory) (AccountService, error) { + return accountserviceimp.FromStores(logger, config, stores, transactions) +} diff --git a/backend/product/sessions/browser/browser.go b/backend/product/sessions/browser/browser.go index 29ec15a..9cb7aa1 100644 --- a/backend/product/sessions/browser/browser.go +++ b/backend/product/sessions/browser/browser.go @@ -1,9 +1,9 @@ package browser import ( + "context" "net/http" "net/url" - "os" "path" "strings" "time" @@ -86,8 +86,10 @@ func ValidateCSRF(r *http.Request, allowedOrigins []string) error { func cookiePath(r *http.Request) string { const accountsSegment = "/accounts" - if apiEndpoint := strings.TrimSpace(os.Getenv("API_ENDPOINT")); apiEndpoint != "" { - return path.Join("/", apiEndpoint, accountsSegment) + if r != nil { + if mount, ok := r.Context().Value(scopeKey{}).(string); ok { + return path.Join(mount, accountsSegment) + } } if r != nil && r.URL != nil { if index := strings.Index(r.URL.Path, accountsSegment); index >= 0 { @@ -119,3 +121,14 @@ func originAllowed(origin string, allowedOrigins []string) bool { } return false } + +type scopeKey struct{} + +// Scope binds cookie creation and deletion to a validated installed API mount. +func Scope(mount string) func(http.Handler) http.Handler { + return func(next http.Handler) http.Handler { + return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + next.ServeHTTP(w, r.WithContext(context.WithValue(r.Context(), scopeKey{}, mount))) + }) + } +} diff --git a/backend/product/sessions/browser/browser_test.go b/backend/product/sessions/browser/browser_test.go index d376b28..c0f7d74 100644 --- a/backend/product/sessions/browser/browser_test.go +++ b/backend/product/sessions/browser/browser_test.go @@ -31,13 +31,15 @@ func TestSetRefreshCookieUsesHardenedHostOnlyPolicy(t *testing.T) { require.Positive(t, cookie.MaxAge) } -func TestSetRefreshCookieUsesConfiguredAPIPathOutsideAccountsRouter(t *testing.T) { +func TestSetRefreshCookieUsesInstalledScopeOutsideAccountsRouter(t *testing.T) { t.Setenv("API_ENDPOINT", "/gateway/v2/") req := httptest.NewRequest(http.MethodPost, "https://app.sendico.io/gateway/v2/verification/login", nil) req.Header.Set(TransportHeader, TransportCookie) recorder := httptest.NewRecorder() - SetRefreshCookie(recorder, req, "refresh", time.Now().Add(time.Hour)) + Scope("/gateway/v2")(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + SetRefreshCookie(w, r, "refresh", time.Now().Add(time.Hour)) + })).ServeHTTP(recorder, req) require.Equal(t, "/gateway/v2/accounts", recorder.Result().Cookies()[0].Path) } diff --git a/backend/product/twofactor/http/router.go b/backend/product/twofactor/http/v1/router.go similarity index 99% rename from backend/product/twofactor/http/router.go rename to backend/product/twofactor/http/v1/router.go index 375ea3e..75f8ab3 100644 --- a/backend/product/twofactor/http/router.go +++ b/backend/product/twofactor/http/v1/router.go @@ -11,8 +11,8 @@ import ( "strconv" "time" + sresponse "git.sendico.io/tech/sxchnge/product/auth/http/v1/response" "git.sendico.io/tech/sxchnge/product/auth/ipguard" - sresponse "git.sendico.io/tech/sxchnge/product/auth/response" customer "git.sendico.io/tech/sxchnge/product/customers/model" "git.sendico.io/tech/sxchnge/product/twofactor" model "git.sendico.io/tech/sxchnge/product/twofactor/model" diff --git a/backend/product/twofactor/http/router_test.go b/backend/product/twofactor/http/v1/router_test.go similarity index 96% rename from backend/product/twofactor/http/router_test.go rename to backend/product/twofactor/http/v1/router_test.go index 1045cbf..c5a61e3 100644 --- a/backend/product/twofactor/http/router_test.go +++ b/backend/product/twofactor/http/v1/router_test.go @@ -6,7 +6,7 @@ import ( "strings" "testing" - sresponse "git.sendico.io/tech/sxchnge/product/auth/response" + sresponse "git.sendico.io/tech/sxchnge/product/auth/http/v1/response" customer "git.sendico.io/tech/sxchnge/product/customers/model" api "git.sendico.io/tech/sxchnge/shared/http" mservice "git.sendico.io/tech/sxchnge/shared/service" diff --git a/docs/api-versions.md b/docs/api-versions.md new file mode 100644 index 0000000..3d473ea --- /dev/null +++ b/docs/api-versions.md @@ -0,0 +1,91 @@ +# Installing product HTTP API versions + +The product executable selects supported implementations in +`backend/product/app/profile/runtime.go`. Each entry maps a version name to an +`api.Installer`; v1 composes the existing authentication and second-factor +contracts. Adding an implementation requires a package and a registry entry, +without changing runtime installation code. + +Select installations in the product YAML: + +```yaml +api: + versions: + - version: v1 + path: /api/v1 + # Existing middleware, storage and notification configuration follows. +``` + +Only v1 ships today. Once a v2 implementation is registered, a second entry with +`version: v2` and `path: /api/v2` serves both versions in the same process. Each +version can be selected once. Unknown/unavailable versions, duplicate versions, +overlapping mounts, noncanonical paths and mounts occupying operational endpoints +fail startup. Paths must be absolute, non-root, without trailing slashes, URL +escapes, queries, fragments or router pattern syntax. Mounts use ASCII letters, +digits, slashes, hyphens, underscores, periods and tildes. + +Omitting `versions` retains compatibility: v1 mounts at the path in the environment +variable named by `api.middleware.api_endpoint_env` (normally +`SXCHNGE_API_ENDPOINT=/api/v1`). An explicit `versions: []` installs no application +routes; process infrastructure and operational endpoints remain available. + +The endpoint environment variable continues to locate the process health and +metrics compatibility endpoints, including when explicit application mounts are +used. With `/api/v1`, these are `/api/v1/health`, `/api/v1/metrics` and `/metrics`. +Keep this value unchanged when adding a version. Health is outside application +middleware; the existing HTTP collector excludes operational traffic. These +operational endpoints are reserved and are never version-owned contracts. + +## Ownership and lifecycle + +Runtime bootstrap opens shared stores once and injects those stores into the +customer service, auth flow and second-factor service. Notification connections +are opened once. Every installer receives the same typed dependencies and its +own router, validated mount, URL provider and logger. Installers register routes +relative to their mount and do not open or close process resources. + +Auth v1 handlers, request DTOs and response mappers live under +`backend/product/auth/http/v1`; factor transport lives under +`backend/product/twofactor/http/v1`. Models, services and storage remain in their +owning features. A future version owns its wire contract and reuses domain +services, so changing its DTOs does not silently change v1. The generic runtime +and HTTP middleware do not import concrete API versions. + +All version routers are built before any application router is attached. An +installer error aborts startup, preserves the cause and closes shared resources. +Bootstrap must either return a cleanup callback for resources it opened or close +them itself before returning an error; it must not do both. The built-in bootstrap +cleans its partial initialization internally. Runtime invokes an acquired cleanup +callback once, including on installation failure or repeated/concurrent shutdown. +The server retains ownership of the shared database factory. + +The removed generic service registry and untyped runtime component lookup have no +product consumers. The typed dependencies and installer registry replace them. +This changes internal Go composition APIs; public v1 JSON and HTTP paths remain +compatible. No database migration is required. + +## Cookies and links + +Runtime binds browser cookie scope to the installed mount using request context. +Set and clear operations use the same `/accounts` path. The existing v1 +path remains `/api/v1/accounts`; `/api/v2` would use `/api/v2/accounts`. Cookie +name, host-only policy, Secure, HttpOnly and SameSite settings are unchanged. +Changing an existing mount requires clients to authenticate at the new path; +cookies at the old path do not migrate automatically. Versions share domain +session state, so account-wide revocation continues to apply across versions. + +`API_ENDPOINT` no longer overrides cookies globally. Direct standalone handler +calls retain their request-path fallback; installed APIs always use explicit +scope. Installers generate API links through `scope.URLs`, which combines the +configured origin and that installation's mount. The origin must use HTTP or +HTTPS with a hostname and optional port. Configured notification activation URLs +remain frontend links and are not rewritten as API links. + +## Validation + +Runtime tests exercise independently behaving versions, shared dependency identity, +disabled routes, invalid mounts, scoped links, cookie set/delete behavior, failed +installation cleanup and repeated shutdown. The existing Mongo-backed startup +test covers the default v1 installation on an empty database. Existing auth and +factor tests moved with their transports and retain their contract assertions. +Run `make test` and `make lint-backend` from the repository root. From 259f84e0a12dd420761d0b4dafa8a63a8f6ce489 Mon Sep 17 00:00:00 2001 From: Stephan D Date: Sun, 13 Sep 2026 10:27:08 +0200 Subject: [PATCH 11/11] Record API version delivery and validation --- docs/implementation.md | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/docs/implementation.md b/docs/implementation.md index 44f090d..cb1d3dd 100644 --- a/docs/implementation.md +++ b/docs/implementation.md @@ -788,3 +788,43 @@ and absence of outgoing bearer metadata over plaintext and optional TLS. Resend API key. No email was sent; full product deployment still awaits the API key and configured sender. Delivery continues on `dev-env-42` in [PR #43](https://git.sendico.io/tech/sxchnge/pulls/43). + +## Independently installed product API versions — task #44 + +[Task #44](https://git.sendico.io/tech/sxchnge/issues/44), branch +`product-api-44`, is delivered in +[PR #45](https://git.sendico.io/tech/sxchnge/pulls/45). The PR targets +`dev-env-42`: merge [PR #43](https://git.sendico.io/tech/sxchnge/pulls/43) into +`main` first, then retarget #45 to `main` before merging. Owner: `tech`; project: +SXCHNGE; label: backend. + +The product registry explicitly installs v1 and supports independently selected +future versions through YAML. Runtime owns shared stores, customer/auth/factor +services and the notification connection; middleware owns HTTP concerns. +Version-owned transports retain their existing contracts and operate on scoped +routers with explicit cookie paths and URL providers. Unknown versions and +invalid, duplicate or overlapping mounts fail startup. Failed installation does +not publish application routes, and cleanup runs once on failure or shutdown. +An architecture test prevents generic runtime imports of concrete transports. + +Backend review covered the entire task contribution, including transport moves, +service reuse, DTO/model separation, project errors and structured lifecycle +logging. Initial focused checks exposed compile references after renaming and a +missing leading slash in relative route registration; these were fixed. Review +also tightened mounts to characters safe in both HTTP and cookie paths, validated +origins, preserved cleanup errors and avoided duplicate shutdown error logging. + +Final validation passed: `make test` (backend race tests and all frontend suites), +`make lint-backend` and `git diff --check`. Tests cover independently behaving +versions, dependency identity, disabled routes, YAML selection, mount rejection, +scoped URLs and cookie set/delete, bootstrap/installation failure cleanup and +concurrent/repeated shutdown. The Mongo-backed default v1 startup and existing +moved auth/factor contract tests pass. Implementation commit: `d04c8b1`. + +Only v1 ships; simultaneous versions are demonstrated with distinct test +installers. Default v1 paths, JSON and operational URLs remain compatible. No +database migration is needed. Internal Go composition APIs changed, and moving an +existing mount requires reauthentication at its new cookie path. Notification +activation URLs remain configured frontend URLs. No live deployment, optional +security scans or merge were performed. The unrelated local notifications sender +configuration remains uncommitted. See [API version installation](api-versions.md).