Build local product stack with Vault-managed secrets (#42)

This commit is contained in:
Stephan D committed 2026-09-12 21:04:42 +02:00
1 parent 8e2ef7aa88
commit 184a3be413
32 files changed
+1191 -10

No files matched your search

+16
View File
@@ -0,0 +1,16 @@
.git
.local
**/.dart_tool
**/build
**/.env*
**/node_modules
**/*.pb.go
frontend/app/lib/generated
**/.DS_Store
dist
website
**/config.yml
**/tls
**/*.pem
**/*.key
**/__pycache__
+6
View File
@@ -34,3 +34,9 @@ pnpm-debug.log*
/backend/notifications/interface/notifications/**/*.pb.go /backend/notifications/interface/notifications/**/*.pb.go
/backend/notifications/config.yml /backend/notifications/config.yml
/backend/notifications/notifications /backend/notifications/notifications
# Local container pins, Vault bootstrap material and service identities
/.local/
# Python bytecode
__pycache__/
+10
View File
@@ -22,6 +22,16 @@
generated outputs. Document the generation workflow so a clean checkout can generated outputs. Document the generation workflow so a clean checkout can
build without committed generated code. build without committed generated code.
## Container versions
Use only the latest stable upstream releases for infrastructure, product base
images, build stages and disposable test containers. Exclude prereleases and
vendor mainline channels when a separate stable channel exists. Resolve stable
versions when building, pull fresh manifests, and pin the resolved image digests
for that local build. Fail when stable release resolution fails; never silently
fall back to an older cached version. Keep the reproducible resolver tracked and
its generated local pins ignored by Git.
## Data and state architecture ## Data and state architecture
Preserve explicit **DTO → mapper → service → data model → provider → UI** Preserve explicit **DTO → mapper → service → data model → provider → UI**
+35 -4
View File
@@ -17,10 +17,12 @@ test-frontend:
.PHONY: test-backend lint-backend run-backend .PHONY: test-backend lint-backend run-backend
test-backend: generate-notifications test-backend: generate-notifications
cd backend/shared && go test -race ./... docker pull mongo:latest
cd backend/notifications/interface && go test -race ./... docker pull nats:latest
cd backend/notifications && go test -race ./... cd backend/shared && go test -race -count=1 ./...
cd backend/product && go test -race ./... cd backend/notifications/interface && go test -race -count=1 ./...
cd backend/notifications && go test -race -count=1 ./...
cd backend/product && go test -race -count=1 ./...
lint-backend: generate-notifications lint-backend: generate-notifications
cd backend/product && go test ./architecture cd backend/product && go test ./architecture
@@ -49,3 +51,32 @@ package-notifications-interface: generate-notifications
mkdir -p dist mkdir -p dist
tar -czf dist/notifications-proto-v1.tar.gz -C backend/notifications/interface proto README.md tar -czf dist/notifications-proto-v1.tar.gz -C backend/notifications/interface proto README.md
tar -czf dist/notifications-go-v1.tar.gz -C backend/notifications/interface go.mod go.sum notifications README.md tar -czf dist/notifications-go-v1.tar.gz -C backend/notifications/interface go.mod go.sum notifications README.md
# Local product stack. Releases are resolved afresh, then pinned by digest.
COMPOSE = docker compose --env-file .local/dev/images.env
.PHONY: build up down logs ps test-dev
build:
python3 dev/images.py
$(COMPOSE) pull vault mongo
$(COMPOSE) build --pull
up:
python3 dev/local.py up
down:
$(COMPOSE) down
logs:
$(COMPOSE) logs --tail=100 -f
ps:
$(COMPOSE) ps
test-dev:
python3 -m unittest discover -s dev -p 'test_*.py'
python3 dev/local.py verify
.PHONY: test-dev-infra
test-dev-infra:
python3 -m unittest discover -s dev -p 'test_*.py'
python3 dev/local.py verify-infra
+17
View File
@@ -32,6 +32,23 @@ Each application owns its dependencies and build configuration. Shared Dart and
Flutter packages can be reused by a future mobile app; mobile runners are not Flutter packages can be reused by a future mobile app; mobile runners are not
currently included. Local `.vscode/` configuration is ignored. currently included. Local `.vscode/` configuration is ignored.
## Local Docker development
```sh
make build && make up
```
This builds the product images and starts an isolated Vault, authenticated MongoDB
replica set and NATS JetStream. First startup intentionally stops after bootstrap
until you set the Resend API key and verified sender in Vault's `kv/email` secret.
Then rerun `make up` to start notifications, BFF and the frontend at
[https://localhost:5126](https://localhost:5126).
All image inputs resolve to current stable releases and are pinned by digest for
the build. Services fetch secrets from Vault; Mongo application credentials come
from Vault's database plugin. See the [local development guide](docs/local-development.md)
for Vault provisioning, local HTTPS trust, lifecycle commands and verification.
## Run the consumer app ## Run the consumer app
Requires Flutter **3.47.0 or later** with Dart **3.13.0 or later** (below Dart 4), matching Requires Flutter **3.47.0 or later** with Dart **3.13.0 or later** (below Dart 4), matching
@@ -50,7 +50,7 @@ func (s *sender) Send(_ context.Context, id string, message model.Email) email.O
func TestDurableDelivery(t *testing.T) { func TestDurableDelivery(t *testing.T) {
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute) ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
defer cancel() defer cancel()
container, err := mongodb.Run(ctx, "mongo:8.0") container, err := mongodb.Run(ctx, "mongo:latest")
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
@@ -192,7 +192,7 @@ func TestDurableDelivery(t *testing.T) {
} }
} }
// Verify the real JetStream publisher/consumer path with an isolated broker. // Verify the real JetStream publisher/consumer path with an isolated broker.
ncContainer, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{ContainerRequest: testcontainers.ContainerRequest{Image: "nats:2.11-alpine", ExposedPorts: []string{"4222/tcp"}, Cmd: []string{"-js"}, WaitingFor: wait.ForListeningPort("4222/tcp")}, Started: true}) ncContainer, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{ContainerRequest: testcontainers.ContainerRequest{Image: "nats:latest", ExposedPorts: []string{"4222/tcp"}, Cmd: []string{"-js"}, WaitingFor: wait.ForListeningPort("4222/tcp")}, Started: true})
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
@@ -24,7 +24,7 @@ import (
func TestProfileStartsOnEmptyDatabase(t *testing.T) { func TestProfileStartsOnEmptyDatabase(t *testing.T) {
ctx, cancel := context.WithTimeout(context.Background(), time.Minute) ctx, cancel := context.WithTimeout(context.Background(), time.Minute)
defer cancel() defer cancel()
container, err := mongodb.Run(ctx, "mongo:8", mongodb.WithReplicaSet("rs")) container, err := mongodb.Run(ctx, "mongo:latest", mongodb.WithReplicaSet("rs"))
require.NoError(t, err) require.NoError(t, err)
t.Cleanup(func() { require.NoError(t, container.Terminate(context.Background())) }) t.Cleanup(func() { require.NoError(t, container.Terminate(context.Background())) })
uri, err := container.ConnectionString(ctx) uri, err := container.ConnectionString(ctx)
@@ -44,7 +44,7 @@ func (m *capturedMail) Send(_ context.Context, r notifications.Message) error {
func TestSignupActivationAndLoginEmailFlow(t *testing.T) { func TestSignupActivationAndLoginEmailFlow(t *testing.T) {
ctx, cancel := context.WithTimeout(context.Background(), time.Minute) ctx, cancel := context.WithTimeout(context.Background(), time.Minute)
defer cancel() defer cancel()
container, err := mongodb.Run(ctx, "mongo:8", mongodb.WithReplicaSet("rs")) container, err := mongodb.Run(ctx, "mongo:latest", mongodb.WithReplicaSet("rs"))
require.NoError(t, err) require.NoError(t, err)
t.Cleanup(func() { require.NoError(t, container.Terminate(context.Background())) }) t.Cleanup(func() { require.NoError(t, container.Terminate(context.Background())) })
uri, err := container.ConnectionString(ctx) uri, err := container.ConnectionString(ctx)
@@ -29,7 +29,7 @@ func customer(login string) *customermodel.Account {
func TestCustomerLifecycleWithoutTenancy(t *testing.T) { func TestCustomerLifecycleWithoutTenancy(t *testing.T) {
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute) ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
defer cancel() defer cancel()
container, err := mongodb.Run(ctx, "mongo:8", mongodb.WithReplicaSet("rs")) container, err := mongodb.Run(ctx, "mongo:latest", mongodb.WithReplicaSet("rs"))
require.NoError(t, err) require.NoError(t, err)
t.Cleanup(func() { require.NoError(t, container.Terminate(context.Background())) }) t.Cleanup(func() { require.NoError(t, container.Terminate(context.Background())) })
uri, err := container.ConnectionString(ctx) uri, err := container.ConnectionString(ctx)
+1 -1
View File
@@ -43,7 +43,7 @@ func (d *capturedDriver) Send(_ context.Context, m model.Delivery) error {
func TestServiceTransactionsAndFactors(t *testing.T) { func TestServiceTransactionsAndFactors(t *testing.T) {
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute) ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
defer cancel() defer cancel()
container, err := mongodb.Run(ctx, "mongo:8", mongodb.WithReplicaSet("rs")) container, err := mongodb.Run(ctx, "mongo:latest", mongodb.WithReplicaSet("rs"))
require.NoError(t, err) require.NoError(t, err)
t.Cleanup(func() { require.NoError(t, container.Terminate(context.Background())) }) t.Cleanup(func() { require.NoError(t, container.Terminate(context.Background())) })
uri, err := container.ConnectionString(ctx) uri, err := container.ConnectionString(ctx)
+145
View File
@@ -0,0 +1,145 @@
name: sxchnge-local
x-agent: &agent
restart: unless-stopped
environment:
VAULT_ADDR: http://vault:8200
tmpfs:
- /run/sxchnge:uid=100,gid=1000,mode=0700
x-backend-build: &backend-build
context: .
dockerfile: dev/images/backend.Dockerfile
services:
vault:
image: ${VAULT_IMAGE:?Run make build first}
user: root
entrypoint: [/bin/sh, /etc/vault/start.sh]
restart: unless-stopped
ports: ['127.0.0.1:18200:8200']
volumes:
- vault-data:/vault/file
- ./dev/vault/server.hcl:/etc/vault/server.hcl:ro
- ./dev/start-vault.sh:/etc/vault/start.sh:ro
- ./.local/dev/vault:/bootstrap:ro
healthcheck:
test: [CMD-SHELL, 'VAULT_ADDR=http://127.0.0.1:8200 vault status >/dev/null']
interval: 3s
timeout: 3s
retries: 40
mongo:
image: ${MONGO_IMAGE:?Run make build first}
entrypoint: [/bin/sh, /etc/mongo/start.sh]
command: [mongod, --replSet, rs0, --bind_ip_all, --keyFile, /tmp/mongo-keyfile]
restart: unless-stopped
environment:
MONGO_INITDB_ROOT_USERNAME: local-admin
MONGO_INITDB_ROOT_PASSWORD_FILE: /run/mongo/password
volumes:
- ./dev/start-mongo.sh:/etc/mongo/start.sh:ro
- mongo-data:/data/db
- ./.local/dev/mongo:/run/mongo:ro
healthcheck:
test: [CMD, mongosh, --quiet, --eval, 'quit(db.hello().isWritablePrimary ? 0 : 1)']
interval: 3s
timeout: 5s
retries: 40
nats:
<<: *agent
image: sxchnge-local/nats:dev
build:
context: .
dockerfile: dev/images/nats.Dockerfile
args:
NATS_IMAGE: ${NATS_IMAGE:?Run make build first}
VAULT_IMAGE: ${VAULT_IMAGE:?Run make build first}
volumes:
- nats-data:/data
- ./.local/dev/agents/nats:/etc/agent:ro
depends_on:
vault: {condition: service_healthy}
healthcheck:
test: [CMD, wget, -q, -O, /dev/null, 'http://127.0.0.1:8222/healthz?js-enabled-only=true']
interval: 3s
timeout: 3s
retries: 40
notifications:
<<: *agent
image: sxchnge-local/notifications:dev
build:
<<: *backend-build
args:
GO_IMAGE: ${GO_IMAGE:?Run make build first}
VAULT_IMAGE: ${VAULT_IMAGE:?Run make build first}
MODULE: notifications
environment:
VAULT_ADDR: http://vault:8200
SXCHNGE_SERVICE: notifications
volumes:
- ./.local/dev/agents/notifications:/etc/agent:ro
depends_on:
vault: {condition: service_healthy}
mongo: {condition: service_healthy}
nats: {condition: service_healthy}
healthcheck:
test: [CMD, nc, -z, 127.0.0.1, '8083']
interval: 3s
timeout: 3s
retries: 40
bff:
<<: *agent
image: sxchnge-local/bff:dev
build:
<<: *backend-build
args:
GO_IMAGE: ${GO_IMAGE:?Run make build first}
VAULT_IMAGE: ${VAULT_IMAGE:?Run make build first}
MODULE: product
environment:
VAULT_ADDR: http://vault:8200
SXCHNGE_SERVICE: bff
SXCHNGE_DOMAIN: localhost:5126
SXCHNGE_API_PROTOCOL: https
SXCHNGE_API_ENDPOINT: /api/v1
volumes:
- ./.local/dev/agents/bff:/etc/agent:ro
depends_on:
notifications: {condition: service_healthy}
mongo: {condition: service_healthy}
healthcheck:
test: [CMD, wget, -q, -O, /dev/null, 'http://127.0.0.1:8082/api/v1/health']
interval: 3s
timeout: 3s
retries: 40
frontend:
image: sxchnge-local/frontend:dev
build:
context: .
dockerfile: dev/images/frontend.Dockerfile
args:
UBUNTU_IMAGE: ${UBUNTU_IMAGE:?Run make build first}
NGINX_IMAGE: ${NGINX_IMAGE:?Run make build first}
FLUTTER_VERSION: ${FLUTTER_VERSION:?Run make build first}
FLUTTER_COMMIT: ${FLUTTER_COMMIT:?Run make build first}
restart: unless-stopped
ports: ['127.0.0.1:5126:8443']
volumes:
- ./.local/dev/frontend:/etc/tls:ro
depends_on:
bff: {condition: service_healthy}
healthcheck:
test: [CMD, wget, -q, --no-check-certificate, -O, /dev/null, 'https://127.0.0.1:8443/']
interval: 3s
timeout: 3s
retries: 40
volumes:
vault-data:
mongo-data:
nats-data:
+43
View File
@@ -0,0 +1,43 @@
"""Build per-service Vault Agent configs (no secret values in configs)."""
import json
from pathlib import Path
BASE = (Path(__file__).parent / 'vault' / 'agent.hcl').read_text()
def kv(path, field):
return '{{ with secret "kv/data/' + path + '" }}{{ .Data.data.' + field + ' }}{{ end }}'
def config(service):
env = {}
if service == 'nats':
env['NATS_TOKEN'] = kv('nats', 'token')
else:
database = 'sxchnge' if service == 'bff' else 'sxchnge_notifications'
env['SXCHNGE_MONGO_URI'] = ('{{ with secret "database/creds/' + service + '" }}'
'mongodb://{{ .Data.username | urlquery }}:{{ .Data.password | urlquery }}'
'@mongo:27017/' + database + '?authSource=admin&replicaSet=rs0{{ end }}')
env['TLS_CA'] = kv('tls', 'ca')
if service == 'bff':
env['SXCHNGE_JWT_SECRET'] = kv('bff', 'jwt')
env['SXCHNGE_NOTIFICATIONS_TOKEN'] = kv('caller', 'token')
else:
env['SXCHNGE_NOTIFICATIONS_PRODUCT_TOKEN'] = kv('caller', 'token')
env['SXCHNGE_NATS_URL'] = ('{{ with secret "kv/data/nats" }}'
'nats://{{ .Data.data.token }}@nats:4222{{ end }}')
env['TLS_CERT'] = kv('notifications-tls', 'certificate')
env['TLS_KEY'] = kv('notifications-tls', 'private_key')
env['EMAIL_API_KEY'] = kv('email', 'api_key')
# JSON is valid YAML and escapes arbitrary provider configuration.
env['EMAIL_CONFIG'] = ('{{ with secret "kv/data/email" }}'
'email:\n driver: {{ .Data.data.driver | toJSON }}\n'
' settings:\n api_key_env: EMAIL_API_KEY\n'
' sender_address: {{ .Data.data.sender_address | toJSON }}\n'
' sender_name: {{ .Data.data.sender_name | toJSON }}\n{{ end }}')
result = BASE
for name, template in env.items():
result += '\nenv_template ' + json.dumps(name) + ' {\n contents = ' + json.dumps(template) + '\n error_on_missing_key = true\n}\n'
command = '/usr/local/bin/start-nats' if service == 'nats' else '/usr/local/bin/start-service'
result += '\nexec {\n command = [' + json.dumps(command) + ']\n restart_on_secret_changes = "always"\n restart_stop_signal = "SIGTERM"\n}\n'
return result
+105
View File
@@ -0,0 +1,105 @@
"""Idempotent provisioning for the isolated local Compose project."""
import json
import os
import subprocess
import time
from agents import config
from vault import STATE, write
ROOT = STATE.parents[1]
COMPOSE = ['docker', 'compose', '--env-file', str(STATE / 'images.env')]
def compose(*args, capture=False, input=None):
result = subprocess.run(COMPOSE + list(args), cwd=ROOT, input=input, text=True,
capture_output=capture)
if result.returncode:
# Bootstrap stdin and child errors may contain credentials. Never echo them.
raise RuntimeError('Compose command failed: ' + ' '.join(args[:3]))
return result.stdout if capture else None
def mongo(script, password):
auth = 'const admin = db.getSiblingDB("admin"); admin.auth("local-admin", ' + json.dumps(password) + ');\n'
return compose('exec', '-T', 'mongo', 'mongosh', '--quiet', '--file', '/dev/stdin', capture=True, input=auth + script)
def provision_mongo(vault):
secret = vault.ensure('bootstrap/mongo', lambda: {'password': vault.random(), 'keyfile': vault.random(384), 'manager_password': vault.random()})
write(STATE / 'mongo' / 'password', secret['password'])
write(STATE / 'mongo' / 'keyfile', secret['keyfile'])
compose('up', '-d', 'mongo')
for attempt in range(90):
try:
mongo('try { rs.status(); } catch(e) { if(e.code === 94) rs.initiate({_id:"rs0",members:[{_id:0,host:"mongo:27017"}]}); else throw e; }', secret['password'])
if 'true' in mongo('print(db.hello().isWritablePrimary)', secret['password']):
break
except RuntimeError:
pass
time.sleep(2)
else:
raise RuntimeError('Mongo replica set did not become primary')
if vault.call('database/config/mongo', missing=True) is None:
mongo('if (!admin.getUser("vault-manager")) admin.createUser({user:"vault-manager",pwd:' + json.dumps(secret['manager_password']) + ',roles:[{role:"userAdminAnyDatabase",db:"admin"}]});', secret['password'])
vault.call('database/config/mongo', {'plugin_name': 'mongodb-database-plugin',
'allowed_roles': ['bff', 'notifications'],
'connection_url': 'mongodb://{{username}}:{{password}}@mongo:27017/admin?replicaSet=rs0',
'username': 'vault-manager', 'password': secret['manager_password']})
vault.call('database/rotate-root/mongo', {})
for role, database in [('bff', 'sxchnge'), ('notifications', 'sxchnge_notifications')]:
vault.call('database/roles/' + role, {'db_name': 'mongo',
'creation_statements': json.dumps({'db': 'admin', 'roles': [{'role': 'readWrite', 'db': database}]}),
'default_ttl': '1h', 'max_ttl': '24h'})
# Public browser client: an identifier, never a client secret in the web bundle.
mongo('db.getSiblingDB("sxchnge").clients.updateOne({clientId:"sxchnge-local-web"}, {$setOnInsert: {clientName:"Local browser",allowedCIDRs:["0.0.0.0/0","::/0"],allowedScopes:["account"],redirectURIs:["https://localhost:5126"],grantTypes:["password","refresh_token"],tokenEndpointAuthMethod:"none",isRevoked:false}}, {upsert:true});', secret['password'])
def provision_agents(vault):
paths = {'bff': ['bff', 'caller', 'tls'],
'notifications': ['caller', 'tls', 'notifications-tls', 'nats', 'email'],
'nats': ['nats']}
for service, secrets in paths.items():
directory = STATE / 'agents' / service
directory.mkdir(parents=True, exist_ok=True, mode=0o755)
os.chmod(directory, 0o755)
policy = ''.join('path "kv/data/' + p + '" { capabilities = ["read"] }\n' for p in secrets)
if service != 'nats':
policy += 'path "database/creds/' + service + '" { capabilities = ["read"] }\n'
vault.call('sys/policies/acl/' + service, {'policy': policy})
vault.call('auth/approle/role/' + service, {'token_policies': [service],
'token_period': '1h', 'secret_id_ttl': '0', 'secret_id_num_uses': 0})
role_id = vault.call('auth/approle/role/' + service + '/role-id')['data']['role_id']
write(directory / 'role-id', role_id, 0o444)
if not (directory / 'secret-id').exists():
secret_id = vault.call('auth/approle/role/' + service + '/secret-id', {})['data']['secret_id']
write(directory / 'secret-id', secret_id, 0o444)
write(directory / 'config.hcl', config(service), 0o444)
def up(vault):
if not (STATE / 'images.env').exists():
raise RuntimeError('Run make build before make up')
(STATE / 'vault').mkdir(parents=True, exist_ok=True, mode=0o755)
compose('up', '-d', 'vault')
vault.initialize()
vault.mounts()
vault.ensure('nats', lambda: {'token': vault.random()})
vault.ensure('caller', lambda: {'token': vault.random()})
vault.ensure('bff', lambda: {'jwt': vault.random()})
vault.ensure('email', lambda: {'driver': 'resend', 'api_key': '',
'sender_address': 'notifications@localhost.test', 'sender_name': 'SXCHNGE local'})
vault.certificates()
provision_mongo(vault)
provision_agents(vault)
compose('up', '-d', '--wait', '--wait-timeout', '120', 'nats')
email = vault.kv('email')
key, sender = email.get('api_key'), email.get('sender_address')
if (not isinstance(key, str) or not key.strip()
or not isinstance(sender, str) or not sender.strip()
or sender == 'notifications@localhost.test'):
raise RuntimeError('Vault kv/email needs a Resend api_key and verified sender_address. Run python3 dev/local.py email, then make up.')
compose('up', '-d', '--wait', '--wait-timeout', '180')
compose('exec', '-T', 'frontend', 'nginx', '-s', 'reload', capture=True)
print('Local stack ready: https://localhost:5126; Vault: http://127.0.0.1:18200')
print('Local CA: .local/dev/ca.pem. Email driver: ' + vault.kv('email')['driver'])
+39
View File
@@ -0,0 +1,39 @@
# SXCHNGE-owned Mongo database and JWT key. Never point these at Sendico.
http_server:
listen_address: 0.0.0.0:8082
read_header_timeout: 5
shutdown_timeout: 10
database:
driver: mongodb
settings:
uri_env: SXCHNGE_MONGO_URI
database: sxchnge
api:
middleware:
domain_env: SXCHNGE_DOMAIN
api_endpoint_env: SXCHNGE_API_ENDPOINT
api_protocol_env: SXCHNGE_API_PROTOCOL
signature:
algorithm: HS256
secret_key_env: SXCHNGE_JWT_SECRET
CORS:
allowed_origins: ["https://localhost:5126"]
allowed_methods: ["GET", "POST", "PATCH", "DELETE", "OPTIONS"]
allowed_headers: ["Content-Type", "Authorization", "X-Auth-Transport", "X-CSRF-Protection"]
allow_credentials: true
max_age: 300
token:
expiration_hours:
account: 1
refresh: 720
length: 32
password:
token_length: 32
check:
min_length: 12
notifications:
address: notifications:8083
token_env: SXCHNGE_NOTIFICATIONS_TOKEN
ca_file: /run/sxchnge/ca.pem
server_name: notifications
activation_url: https://localhost:5126/verify-email
+17
View File
@@ -0,0 +1,17 @@
server {
listen 8443 ssl;
ssl_certificate /etc/tls/server.crt;
ssl_certificate_key /etc/tls/server.key;
ssl_protocols TLSv1.2 TLSv1.3;
root /usr/share/nginx/html;
index index.html;
location /api/ {
resolver 127.0.0.11 valid=10s ipv6=off;
set $bff http://bff:8082;
proxy_pass $bff;
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Proto https;
proxy_set_header X-Forwarded-For $remote_addr;
}
location / { try_files $uri $uri/ /index.html; }
}
+27
View File
@@ -0,0 +1,27 @@
listen: "0.0.0.0:8083"
tls_cert: "/run/sxchnge/server.crt"
tls_key: "/run/sxchnge/server.key"
database:
driver: mongodb
settings:
uri_env: SXCHNGE_MONGO_URI
database: sxchnge_notifications
messaging:
driver: NATS
settings:
url_env: SXCHNGE_NATS_URL
broker_name: sxchnge-notifications
max_reconnects: -1
reconnect_wait: 2
templates: "/etc/sxchnge/templates.json"
callers:
product: "SXCHNGE_NOTIFICATIONS_PRODUCT_TOKEN"
# Use replicas: 3 and environment: production for a three-node NATS cluster.
# mode: ops validates resources without modifying them; app provisions them.
queue:
stream:
replicas: 1
max_age_hours: 48
provisioning:
mode: app
environment: non-production
+72
View File
@@ -0,0 +1,72 @@
#!/usr/bin/env python3
"""Resolve stable upstream releases on every build; pin their registry digests."""
import concurrent.futures
import json
import os
from pathlib import Path
import re
import subprocess
import urllib.request
ROOT = Path(__file__).resolve().parents[1]
STATE = ROOT / '.local' / 'dev'
def get(url):
with urllib.request.urlopen(url, timeout=45) as response:
return json.load(response)
def image(repo, alias):
url = f'https://hub.docker.com/v2/repositories/{repo}/tags'
selected = get(f'{url}/{alias}')
digest = selected['digest']
if not re.fullmatch(r'sha256:[a-f0-9]{64}', digest):
raise ValueError('Registry returned an invalid digest')
# Find the exact numeric tag represented by the maintained stable alias.
candidates = get(f'{url}?page_size=100')['results']
names = [x['name'] for x in candidates if x.get('digest') == digest
and re.fullmatch(r'\d+\.\d+(?:\.\d+)?(?:-alpine)?', x['name'])]
if not names:
raise ValueError(f'No stable version tag for {repo}:{alias}')
tag = max(names, key=lambda x: tuple(map(int, x.split('-')[0].split('.'))))
return f'{repo}:{tag}@{digest}'
def flutter():
commit = get('https://api.github.com/repos/flutter/flutter/commits/stable')['sha']
tags = subprocess.check_output(['git', 'ls-remote', '--tags',
'https://github.com/flutter/flutter.git'], text=True)
names = [line.split('refs/tags/')[1] for line in tags.splitlines()
if line.startswith(commit + '\trefs/tags/')
and re.fullmatch(r'\d+\.\d+\.\d+', line.split('refs/tags/')[1])]
if not names:
raise ValueError('Flutter stable does not point at a stable release tag')
return max(names, key=lambda x: tuple(map(int, x.split('.')))), commit
def main():
go = get('https://go.dev/dl/?mode=json')[0]
if not go['stable'] or not re.fullmatch(r'go\d+\.\d+\.\d+', go['version']):
raise ValueError('Go release is not stable')
specs = {'VAULT_IMAGE': ('hashicorp/vault', 'latest'),
'MONGO_IMAGE': ('library/mongo', 'latest'),
'NATS_IMAGE': ('library/nats', 'latest'),
'NGINX_IMAGE': ('library/nginx', 'stable-alpine'),
'GO_IMAGE': ('library/golang', go['version'][2:] + '-alpine'),
'UBUNTU_IMAGE': ('library/ubuntu', 'latest')}
with concurrent.futures.ThreadPoolExecutor() as pool:
futures = {key: pool.submit(image, *spec) for key, spec in specs.items()}
refs = {key: future.result() for key, future in futures.items()}
refs['FLUTTER_VERSION'], refs['FLUTTER_COMMIT'] = flutter()
STATE.mkdir(parents=True, exist_ok=True, mode=0o700)
target = STATE / 'images.env'
temporary = target.with_suffix('.tmp')
temporary.write_text(''.join(f'{key}={value}\n' for key, value in refs.items()))
os.replace(temporary, target)
for key, value in refs.items():
print(f'{key}={value}')
if __name__ == '__main__':
main()
+21
View File
@@ -0,0 +1,21 @@
ARG GO_IMAGE
ARG VAULT_IMAGE
FROM ${GO_IMAGE} AS build
RUN apk add --no-cache protobuf protobuf-dev make
WORKDIR /src
COPY Makefile ./
COPY backend ./backend
RUN make generate-notifications
ARG MODULE
WORKDIR /src/backend/${MODULE}
RUN --mount=type=cache,target=/root/.cache/go-build --mount=type=cache,target=/go/pkg/mod CGO_ENABLED=0 go build -trimpath -o /out/service .
FROM ${VAULT_IMAGE}
USER root
COPY --from=build /out/service /usr/local/bin/service
COPY dev/start-service.sh /usr/local/bin/start-service
COPY dev/config /etc/sxchnge
COPY backend/notifications/templates.example.json /etc/sxchnge/templates.json
RUN mkdir -p /run/sxchnge && chown 100:1000 /run/sxchnge
USER 100:1000
ENTRYPOINT ["vault", "agent", "-config=/etc/agent/config.hcl"]
+17
View File
@@ -0,0 +1,17 @@
ARG UBUNTU_IMAGE
ARG NGINX_IMAGE
FROM ${UBUNTU_IMAGE} AS build
RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates curl git unzip xz-utils libglu1-mesa && rm -rf /var/lib/apt/lists/*
ARG FLUTTER_VERSION
ARG FLUTTER_COMMIT
RUN git clone --depth 1 --branch "${FLUTTER_VERSION}" https://github.com/flutter/flutter.git /opt/flutter && test "$(git -C /opt/flutter rev-parse HEAD)" = "${FLUTTER_COMMIT}"
ENV PATH="/opt/flutter/bin:${PATH}"
RUN flutter config --no-analytics && flutter precache --web
WORKDIR /src
COPY frontend ./frontend
WORKDIR /src/frontend/app
ARG FRONTEND_URL=https://localhost:5126
RUN flutter pub get && flutter gen-l10n && flutter build web --no-wasm-dry-run --dart-define=SXCHNGE_API_URL=${FRONTEND_URL}/api/v1 --dart-define=SXCHNGE_CLIENT_ID=sxchnge-local-web
FROM ${NGINX_IMAGE}
COPY dev/config/nginx.conf /etc/nginx/conf.d/default.conf
COPY --from=build /src/frontend/app/build/web /usr/share/nginx/html
+10
View File
@@ -0,0 +1,10 @@
ARG NATS_IMAGE
ARG VAULT_IMAGE
FROM ${NATS_IMAGE} AS nats
FROM ${VAULT_IMAGE}
USER root
COPY --from=nats /nats-server /usr/local/bin/nats-server
COPY dev/start-nats.sh /usr/local/bin/start-nats
RUN mkdir -p /data /run/sxchnge && chown -R 100:1000 /data /run/sxchnge
USER 100:1000
ENTRYPOINT ["vault", "agent", "-config=/etc/agent/config.hcl"]
+47
View File
@@ -0,0 +1,47 @@
#!/usr/bin/env python3
"""Local stack lifecycle. Requires Python 3 and Docker Compose."""
import sys
from bootstrap import up
from vault import Vault
def main():
vault = Vault()
if len(sys.argv) == 2 and sys.argv[1] == 'up':
import fcntl
from vault import STATE
STATE.mkdir(parents=True, exist_ok=True, mode=0o700)
with (STATE / 'bootstrap.lock').open('w') as lock:
try:
fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB)
except BlockingIOError:
raise RuntimeError('Another local bootstrap is running') from None
up(vault)
elif len(sys.argv) == 2 and sys.argv[1] == 'verify':
from verify import verify
vault.existing()
verify(vault)
elif len(sys.argv) == 2 and sys.argv[1] == 'verify-infra':
from verify import infrastructure
vault.existing()
infrastructure(vault)
elif len(sys.argv) == 2 and sys.argv[1] == 'email':
import getpass
vault.existing()
address = input('Verified Resend sender address: ').strip()
key = getpass.getpass('Resend API key (hidden): ').strip()
if not key or '@' not in address:
raise RuntimeError('A key and sender address are required')
vault.put('email', {'driver': 'resend', 'api_key': key,
'sender_address': address, 'sender_name': 'SXCHNGE'})
print('Email configuration saved in Vault. Run make up.')
else:
raise RuntimeError('Usage: python3 dev/local.py up|verify|verify-infra|email')
if __name__ == '__main__':
try:
main()
except (RuntimeError, OSError) as error:
print('Local development setup failed: ' + str(error), file=sys.stderr)
sys.exit(1)
+6
View File
@@ -0,0 +1,6 @@
#!/bin/sh
set -eu
cp /run/mongo/keyfile /tmp/mongo-keyfile
chown mongodb:mongodb /tmp/mongo-keyfile
chmod 400 /tmp/mongo-keyfile
exec docker-entrypoint.sh "$@"
+13
View File
@@ -0,0 +1,13 @@
#!/bin/sh
set -eu
umask 077
# Generated by Vault with hexadecimal encoding; never interpolates user input.
case "$NATS_TOKEN" in ''|*[!0-9a-f]*) exit 1;; esac
cat > /run/sxchnge/nats.conf <<CONFIG
port: 4222
http_port: 8222
authorization { token: "$NATS_TOKEN" }
jetstream { store_dir: "/data/jetstream" }
CONFIG
unset NATS_TOKEN
exec nats-server -c /run/sxchnge/nats.conf
+14
View File
@@ -0,0 +1,14 @@
#!/bin/sh
set -eu
umask 077
printf '%s\n' "$TLS_CA" > /run/sxchnge/ca.pem
unset TLS_CA
if [ "$SXCHNGE_SERVICE" = notifications ]; then
printf '%s\n' "$TLS_CERT" > /run/sxchnge/server.crt
printf '%s\n' "$TLS_KEY" > /run/sxchnge/server.key
printf '%s\n' "$EMAIL_CONFIG" > /run/sxchnge/email.yml
unset TLS_CERT TLS_KEY EMAIL_CONFIG
cat /etc/sxchnge/notifications.yml /run/sxchnge/email.yml > /run/sxchnge/config.yml
exec service -config.file=/run/sxchnge/config.yml
fi
exec service -config.file=/etc/sxchnge/bff.yml
+16
View File
@@ -0,0 +1,16 @@
#!/bin/sh
# Local-only automatic unseal. The root token is never mounted in this container.
set -eu
export VAULT_ADDR=http://127.0.0.1:8200
vault server -config=/etc/vault/server.hcl &
pid=$!
trap 'kill -TERM "$pid"; wait "$pid"' TERM INT
until [ -s /bootstrap/unseal ]; do
kill -0 "$pid" || exit 1
sleep 1
done
until vault operator unseal "$(cat /bootstrap/unseal)" >/dev/null 2>&1; do
kill -0 "$pid" || exit 1
sleep 1
done
wait "$pid"
+62
View File
@@ -0,0 +1,62 @@
"""Focused regression tests for release selection and bootstrap failure paths."""
from pathlib import Path
import tempfile
import unittest
from unittest.mock import patch
import bootstrap
import images
from vault import Vault
class Releases(unittest.TestCase):
def test_selects_exact_stable_digest_not_prerelease(self):
digest = 'sha256:' + 'a' * 64
versions = [{'name': name, 'digest': digest} for name in ['latest', '2', '2.1', '2.1.0', '2.2.0-rc1']]
with patch.object(images, 'get', side_effect=[{'digest': digest}, {'results': versions}]):
self.assertEqual(images.image('hashicorp/vault', 'latest'), 'hashicorp/vault:2.1.0@' + digest)
def test_missing_stable_mapping_fails(self):
digest = 'sha256:' + 'a' * 64
with patch.object(images, 'get', side_effect=[{'digest': digest}, {'results': [{'name': '3.0.0-rc1', 'digest': digest}]}]):
with self.assertRaises(ValueError):
images.image('hashicorp/vault', 'latest')
def test_upstream_failure_does_not_replace_existing_pins(self):
with tempfile.TemporaryDirectory() as temporary:
state = Path(temporary)
target = state / 'images.env'
target.write_text('previous build')
with patch.object(images, 'STATE', state), patch.object(images, 'get', side_effect=OSError('offline')):
with self.assertRaises(OSError):
images.main()
self.assertEqual(target.read_text(), 'previous build')
class Bootstrap(unittest.TestCase):
def test_missing_resend_credentials_stops_before_product_start(self):
with tempfile.TemporaryDirectory() as temporary:
state = Path(temporary)
(state / 'images.env').touch()
from unittest.mock import Mock
vault = Mock()
vault.kv.return_value = {'driver': 'resend', 'api_key': '', 'sender_address': 'notifications@localhost.test'}
with patch.object(bootstrap, 'STATE', state), patch.object(bootstrap, 'compose') as compose, \
patch.object(bootstrap, 'provision_mongo'), patch.object(bootstrap, 'provision_agents'):
with self.assertRaisesRegex(RuntimeError, 'Resend api_key'):
bootstrap.up(vault)
self.assertEqual(compose.call_args_list[-1].args[-1], 'nats')
email_seed = next(c for c in vault.ensure.call_args_list if c.args[0] == 'email').args[1]()
self.assertEqual(email_seed['driver'], 'resend')
self.assertEqual(email_seed['api_key'], '')
def test_initialized_vault_without_local_keys_fails(self):
with tempfile.TemporaryDirectory() as temporary:
with patch('vault.STATE', Path(temporary)), patch.object(Vault, 'call', return_value={'initialized': True}):
with self.assertRaisesRegex(RuntimeError, 'Missing .local'):
Vault().initialize()
if __name__ == '__main__':
unittest.main()
+117
View File
@@ -0,0 +1,117 @@
"""Local Vault bootstrap API; response bodies with secrets never reach logs."""
import json
import os
from pathlib import Path
import time
import urllib.error
import urllib.request
STATE = Path(__file__).resolve().parents[1] / '.local' / 'dev'
URL = 'http://127.0.0.1:18200/v1/'
def write(path, value, mode=0o600):
path.parent.mkdir(parents=True, exist_ok=True, mode=0o700)
tmp = path.with_name(path.name + '.tmp')
fd = os.open(tmp, os.O_WRONLY | os.O_CREAT | os.O_TRUNC, mode)
with os.fdopen(fd, 'w') as out:
out.write(value)
os.chmod(tmp, mode)
os.replace(tmp, path)
class Vault:
def __init__(self):
self.token = None
def call(self, path, data=None, method=None, missing=False):
headers = {'Content-Type': 'application/json'}
if self.token:
headers['X-Vault-Token'] = self.token
req = urllib.request.Request(URL + path, headers=headers,
data=json.dumps(data).encode() if data is not None else None, method=method)
try:
with urllib.request.urlopen(req, timeout=30) as response:
body = response.read()
return json.loads(body) if body else {}
except urllib.error.HTTPError as error:
if error.code == 404 and missing:
return None
raise RuntimeError(f'Vault {path} returned HTTP {error.code}') from None
def initialize(self):
for attempt in range(90):
try:
initialized = self.call('sys/init')['initialized']
break
except (urllib.error.URLError, ConnectionError):
time.sleep(1)
else:
raise RuntimeError('Vault did not start within 90 seconds')
initfile = STATE / 'admin' / 'init.json'
if not initialized:
if initfile.exists():
raise RuntimeError('Vault data is missing but bootstrap material exists; restore the matching volume or explicitly reset both')
init = self.call('sys/init', {'secret_shares': 1, 'secret_threshold': 1})
write(initfile, json.dumps(init))
if not initfile.exists():
raise RuntimeError('Missing .local/dev/admin/init.json for initialized Vault; restore the matching bootstrap material')
init = json.loads(initfile.read_text())
self.token = init['root_token']
write(STATE / 'vault' / 'unseal', init['keys_base64'][0], 0o444)
for attempt in range(60):
if not self.call('sys/seal-status')['sealed']:
return
time.sleep(1)
raise RuntimeError('Vault did not unseal within 60 seconds')
def existing(self):
self.token = json.loads((STATE / 'admin' / 'init.json').read_text())['root_token']
def kv(self, path):
result = self.call('kv/data/' + path, missing=True)
return result['data']['data'] if result else None
def put(self, path, data):
self.call('kv/data/' + path, {'data': data})
def ensure(self, path, factory):
value = self.kv(path)
if value is None:
value = factory()
self.put(path, value)
return value
def random(self, length=32):
return self.call('sys/tools/random', {'bytes': length, 'format': 'hex'})['data']['random_bytes']
def mounts(self):
mounts = self.call('sys/mounts')['data']
for name, kind in [('kv', 'kv'), ('database', 'database'), ('pki', 'pki')]:
if name + '/' not in mounts:
settings = {'type': kind}
if kind == 'kv':
settings['options'] = {'version': '2'}
self.call('sys/mounts/' + name, settings)
auth = self.call('sys/auth')['data']
if 'approle/' not in auth:
self.call('sys/auth/approle', {'type': 'approle'})
def certificates(self):
self.call('sys/mounts/pki/tune', {'max_lease_ttl': '87600h'})
ca = self.kv('tls')
if ca is None:
result = self.call('pki/root/generate/internal', {'common_name': 'SXCHNGE local development CA', 'ttl': '87600h'})['data']
ca = {'ca': result['certificate']}
self.put('tls', ca)
self.call('pki/roles/local', {'allowed_domains': ['localhost', 'notifications'],
'allow_bare_domains': True, 'allow_ip_sans': True, 'max_ttl': '720h'})
for name, host in [('notifications-tls', 'notifications'), ('frontend-tls', 'localhost')]:
cert = self.kv(name)
if cert is None or cert['expiration'] < time.time() + 7 * 86400:
cert = self.call('pki/issue/local', {'common_name': host, 'ip_sans': '127.0.0.1', 'ttl': '720h'})['data']
self.put(name, cert)
if name == 'frontend-tls':
write(STATE / 'frontend' / 'server.crt', cert['certificate'])
write(STATE / 'frontend' / 'server.key', cert['private_key'])
write(STATE / 'ca.pem', ca['ca'], 0o644)
+14
View File
@@ -0,0 +1,14 @@
vault { address = "http://vault:8200" }
auto_auth {
method "approle" {
config = {
role_id_file_path = "/etc/agent/role-id"
secret_id_file_path = "/etc/agent/secret-id"
remove_secret_id_file_after_reading = false
}
}
}
template_config {
static_secret_render_interval = "10s"
exit_on_retry_failure = true
}
+10
View File
@@ -0,0 +1,10 @@
ui = true
disable_mlock = true
storage "file" {
path = "/vault/file"
}
listener "tcp" {
address = "0.0.0.0:8200"
tls_disable = true
}
api_addr = "http://vault:8200"
+72
View File
@@ -0,0 +1,72 @@
"""Integration checks against this project's running stack; never print secrets."""
import json
import ssl
import urllib.request
from bootstrap import compose, mongo
from vault import STATE
def require(condition, message):
if not condition:
raise RuntimeError(message)
def database_access(vault):
for role, database, forbidden in [('bff', 'sxchnge', 'sxchnge_notifications'),
('notifications', 'sxchnge_notifications', 'sxchnge')]:
lease = vault.call('database/creds/' + role)
credentials = lease['data']
auth = 'db.getSiblingDB("admin").auth(' + json.dumps(credentials['username']) + ',' + json.dumps(credentials['password']) + ');'
try:
result = compose('exec', '-T', 'mongo', 'mongosh', '--quiet', '--file', '/dev/stdin', capture=True,
input=auth + 'const own=db.getSiblingDB(' + json.dumps(database) + '); own.local_smoke.insertOne({_id:"vault-check"}); own.local_smoke.deleteOne({_id:"vault-check"});\n'
'let denied=false; try { db.getSiblingDB(' + json.dumps(forbidden) + ').runCommand({find:"local_smoke",filter:{}}); } catch(e) { denied=e.code===13; } if(!denied) throw new Error("cross-database access allowed"); print("scope-ok");')
require('scope-ok' in result, 'Mongo service scope check failed')
renewed = vault.call('sys/leases/renew', {'lease_id': lease['lease_id'], 'increment': '1h'})
require(renewed['lease_duration'] > 0, 'Mongo lease renewal failed')
finally:
vault.call('sys/leases/revoke', {'lease_id': lease['lease_id']})
admin = vault.kv('bootstrap/mongo')['password']
result = mongo('print(admin.getUser(' + json.dumps(credentials['username']) + ')===null)', admin)
require('true' in result, 'Revoked Mongo user still exists')
print('PASS: database-plugin issuance, separate database permissions, lease renewal and revocation')
def policies(vault):
for service in ['bff', 'notifications', 'nats']:
root = STATE / 'agents' / service
login = vault.call('auth/approle/login', {'role_id': (root / 'role-id').read_text(),
'secret_id': (root / 'secret-id').read_text()})
token = login['auth']['client_token']
try:
result = vault.call('sys/capabilities', {'token': token, 'paths': ['kv/data/bootstrap/mongo', 'sys/init']})['data']
require(all(result[path] == ['deny'] for path in ['kv/data/bootstrap/mongo', 'sys/init']), 'Service can access bootstrap credentials')
finally:
vault.call('auth/token/revoke', {'token': token})
print('PASS: service identities cannot read bootstrap credentials')
def infrastructure(vault):
status = compose('ps', '--format', 'json', capture=True)
rows = [json.loads(line) for line in status.splitlines() if line.strip()]
infra = {r['Service']: r for r in rows if r['Service'] in {'vault', 'mongo', 'nats'}}
require(set(infra) == {'vault', 'mongo', 'nats'}, 'Expected Vault, Mongo and NATS')
require(all(r['Health'] == 'healthy' for r in infra.values()), 'Infrastructure is unhealthy')
print('PASS: Vault, Mongo replica set and NATS JetStream are healthy')
policies(vault)
database_access(vault)
def verify(vault):
status = compose('ps', '--format', 'json', capture=True)
rows = [json.loads(line) for line in status.splitlines() if line.strip()]
require({r['Service'] for r in rows} == {'vault', 'mongo', 'nats', 'notifications', 'bff', 'frontend'}, 'Expected six running services')
require(all(r['Health'] == 'healthy' for r in rows), 'Some services are unhealthy')
context = ssl.create_default_context(cafile=str(STATE / 'ca.pem'))
for path in ['/', '/api/v1/health']:
with urllib.request.urlopen('https://localhost:5126' + path, context=context, timeout=10) as response:
require(response.status == 200, 'Frontend or BFF health failed')
print('PASS: all six services healthy; frontend and proxied BFF use verified TLS')
infrastructure(vault)
print('Resend credentials are configured; no live email sent by this check')
+52
View File
@@ -583,3 +583,55 @@ notification queue acceptance is not inbox delivery.
Delivered in [PR #41](https://git.sendico.io/tech/sxchnge/pulls/41) against `main`. Delivered in [PR #41](https://git.sendico.io/tech/sxchnge/pulls/41) against `main`.
Branch `twofactor-40` is pushed. The PR is left open for review; no merge was performed. Branch `twofactor-40` is pushed. The PR is left open for review; no merge was performed.
## Vault-backed local development (#42)
Task [#42](https://git.sendico.io/tech/sxchnge/issues/42), branch `dev-env-42`.
Primary module: dev environment; secondary scope: container packaging, root build
commands, current-stable backend test images and repository container policy.
- Added a dedicated `sxchnge-local` Compose project with persistent Vault, an
authenticated MongoDB replica set, NATS JetStream and build definitions for
notifications, BFF and Flutter/Nginx. Existing Sendico containers and volumes
remain independent.
- `make build` resolves current stable releases and pins manifest digests in
ignored local state. Flutter is built from its stable source tag/commit. All
protobuf bindings and frontend output are generated within the build, never
committed. Root `AGENTS.md` now records the stable-container policy.
- Bootstrap configures Vault KV/PKI/AppRole and `mongodb-database-plugin`, rotates
its Mongo manager credential and issues separate database roles for the BFF
and notifications. Vault Agent renews credentials and restarts child processes
on changes. Existing shared database/messaging paths remain intact.
- Resend is the default. The user explicitly chose to provision its API key and
verified sender manually after bootstrap. `make up` leaves healthy infrastructure
running and intentionally exits nonzero before product startup while those
fields are empty. No placeholder credential or disabled driver was retained.
- Frontend HTTPS and notification TLS use Vault PKI. The frontend proxies the BFF
at the same origin and builds with the existing real auth adapter and a seeded
public browser client. No UI or business service code changed.
Validation: `make build` built all product images; `make test` and
`make lint-backend` passed. Backend tests pull fresh Mongo/NATS stable images and
use `-count=1` to prevent Go's test cache from hiding an image update.
`make test-dev-infra` passed its five setup regression tests and live checks for
AppRole restrictions, database issuance, cross-database denial, renewal and
revocation. `docker compose ... config --quiet` and `git diff --check` passed.
The initial image build exposed missing protobuf headers; installing
`protobuf-dev` in the build stage fixed it.
Additional live checks passed: a down/up cycle preserved a Mongo marker and Vault
secrets and automatically unsealed Vault; changing/restoring the NATS token
restarted the NATS child; a temporary 10-second/20-second BFF database role proved
replacement credentials restart a healthy BFF (normal TTLs restored afterward).
Temporary BFF/frontend containers served the web bundle and proxied BFF health
successfully over HTTPS verified with the generated CA; those test containers
were removed. The notifications image rendered its Agent configuration and exited
with the expected missing email API key error. No real email was sent.
The final `make build && make up` is expected to exit at the Resend provisioning
check until the user populates `kv/email`. Full six-service readiness and a live
Resend signup/login were not asserted without that credential. Vault, Mongo and
NATS remain running. No optional security scan or merge was performed. See the
[local development guide](local-development.md) for exact provisioning fields,
certificate trust, restart behavior, local bootstrap-material handling and
verification commands.
+182
View File
@@ -0,0 +1,182 @@
# Local development
Task [#42](https://git.sendico.io/tech/sxchnge/issues/42), branch `dev-env-42`.
## Build and bootstrap
Requires Docker with Compose v2 and BuildKit, Python 3.10+ and Git. Docker builds
Go, protobuf bindings and Flutter web; host Go, protoc and Flutter installations
are not required for `make build && make up`. The first Flutter SDK download can
take several minutes. Both ARM64 and AMD64 build from the same source definitions.
```sh
make build && make up
```
`make build` resolves the current stable upstream releases, pulls infrastructure
and builds NATS packaging, notifications, BFF and frontend images. Vault and Mongo
use upstream images directly. Resolution excludes prereleases and selects Nginx's
stable channel. The exact versions and manifest digests are saved in the ignored
`.local/dev/images.env`; Flutter's stable release is pinned by Git commit. Build
fails if upstream metadata cannot be resolved. Run `make build` again to refresh
versions. Review database release upgrade requirements before updating a populated
local volume across major versions; the tooling does not perform data migrations.
`make up` starts persistent Vault, initializes/unseals it, provisions secrets and
AppRoles, initializes an authenticated single-member MongoDB replica set, configures
Vault's MongoDB database plugin, and starts authenticated NATS JetStream. It then
starts notifications, BFF and the frontend when the required email settings exist.
**On the first run, `make up` intentionally exits with an error after infrastructure
bootstrap because Resend credentials have not been supplied.** No placeholder API
key, mock delivery mode or embedded credential is installed. The three infrastructure
services remain running so you can configure Vault, then rerun `make up`.
## Configure Resend in Vault
The default email driver is `resend`. In the Vault UI at
[http://127.0.0.1:18200](http://127.0.0.1:18200), edit the KV v2 secret **`kv/email`**
(API path `kv/data/email`):
| Field | Value |
| --- | --- |
| `driver` | `resend` |
| `api_key` | Your Resend API key |
| `sender_address` | Your verified Resend sender email |
| `sender_name` | `SXCHNGE` or your desired display name |
Local administrative login material is in the private, ignored
`.local/dev/admin/init.json` (`root_token`). Keep that file private. Product
containers never mount it. You can also provision the same secret with a hidden
terminal prompt, without placing the API key in shell history:
```sh
python3 dev/local.py email
make up
```
The startup check rejects an empty key or the bootstrap sender placeholder. It
does not validate the key against Resend or prove domain verification; an invalid
key fails at delivery. Vault Agent fetches the key and email settings and injects
them into the notifications process. There is no `.env` secret fallback. Changing
`kv/email` restarts the notification process automatically within about 10 seconds.
Bootstrapping again preserves your existing email secret.
## Open the product
After providing Resend credentials:
- Frontend and proxied API: [https://localhost:5126](https://localhost:5126).
- BFF health: [https://localhost:5126/api/v1/health](https://localhost:5126/api/v1/health).
- Vault UI/API: [http://127.0.0.1:18200](http://127.0.0.1:18200).
Trust `.local/dev/ca.pem` in your local browser/OS certificate store to use HTTPS
without a certificate error. Trust installation is a separate local action, not
performed by the bootstrap. For a command-line check without changing trust:
```sh
curl --cacert .local/dev/ca.pem https://localhost:5126/api/v1/health
```
HTTPS keeps the existing Secure/HttpOnly refresh cookie contract. Nginx serves
Flutter and proxies `/api/` to the BFF on the same origin. The bundle selects the
existing backend auth adapter with the public client ID `sxchnge-local-web`.
Bootstrap inserts this public browser client if absent; it has no client secret.
No frontend widget, service or provider architecture changes are needed.
The rest of the app retains its existing capabilities and limitations: wallet,
KYC and settings fixtures are not turned into production integrations by this
container setup. Full email signup/login requires a working Resend account.
## Secret and connection lifecycle
| Consumer | Vault access |
| --- | --- |
| BFF | `database/creds/bff`, JWT key, notification caller token and CA |
| Notifications | `database/creds/notifications`, NATS token, caller token, TLS certificate/key and email settings |
| NATS | NATS authentication token only |
| Bootstrap | Infrastructure administration, PKI and initial identities |
The database engine is configured at `database/config/mongo` with
`mongodb-database-plugin`. It uses its own `vault-manager` Mongo user, whose
password is rotated by Vault immediately after configuration. The manager has
`userAdminAnyDatabase`; applications never receive that credential. The independent
local Mongo administrator and replica-set key come from Vault's random generator
and are retained in `kv/bootstrap/mongo` for local provisioning.
The BFF role has `readWrite` only on `sxchnge`; notifications has `readWrite` only
on `sxchnge_notifications`. Mongo URIs come from dynamic Vault credentials, not
static application users. Leases default to one hour with a 24-hour maximum.
Vault Agent renews leases, obtains replacement credentials, and restarts its child
process when credentials change. Apps retain their existing shared database and
messaging connection wrappers. No new direct application database path is added.
Each agent authenticates using its own AppRole with a renewable periodic token.
Local SecretIDs are reusable for container restarts; each container mounts only
its own identity directory under `.local/dev/agents/`. Revoking an AppRole SecretID
requires removing its matching local `secret-id` file and rerunning `make up` to
issue a replacement. Revoking an active token/lease terminates its credentials;
Agent's retries and the container restart policy recover when Vault is reachable.
The notification gRPC connection uses a Vault-issued certificate for
`notifications` and the BFF validates the Vault CA and hostname. Frontend HTTPS
certificates also come from Vault PKI. Leaf certificates last 30 days; `make up`
refreshes them when less than seven days remain and reloads Nginx. Run it regularly
for a development stack kept online for weeks. Agents restart when their TLS KV
secret changes. Application secrets are never put into the frontend bundle.
Vault, Mongo and JetStream data use project-scoped named Docker volumes. Local
Vault unseal material, administrator login, per-service bootstrap identities and
frontend TLS material are under ignored `.local/dev/`, outside image build contexts.
The local Vault automatically unseals using its local key on container restart.
These convenience credentials and loopback HTTP Vault access are for this local
machine; this is not a production Vault deployment. Do not share `.local/dev/`.
Keep it paired with its matching Docker volumes for recovery.
Mongo and NATS expose no host ports. The `sxchnge-local` network, volumes and
container names are independent of existing Sendico stacks. Only the frontend
and Vault bind host loopback ports. No Sendico database, message bus or secrets
are consumed.
## Manage and verify
```sh
make ps
make logs
make down # preserves named volumes and local bootstrap material
make up # idempotent provisioning and readiness checks
make test-dev-infra # works before Resend provisioning
make test-dev # requires all six services running
```
The infrastructure check exercises AppRole access restrictions, real Vault
MongoDB credential issuance, per-database access, lease renewal and user deletion
on revocation. The complete check also verifies service health and frontend/API
HTTPS with the generated CA. These checks never send a real email.
Backend regression tests use fresh `mongo:latest` and `nats:latest` official
stable images, pulled before `make test-backend`. Normal validation remains
`make test` and `make lint-backend` and requires the host SDKs described in the
repository guides.
Deleting Docker volumes removes local data; `make down` deliberately keeps them.
If starting from scratch, stop the stack and explicitly remove this project's
three volumes **and** its matching `.local/dev/` bootstrap state together. Never
reuse old unseal material with a newly initialized Vault volume, or retain Mongo
users while discarding the Vault data that manages their leases.
## Sources and checked releases
On 2026-09-12 the resolver selected Vault 2.1.0, MongoDB 8.3.9, NATS 2.14.6,
Go 1.27.1, Nginx stable 1.30.4, Ubuntu 26.04 and Flutter stable 3.47.4. These are
verification records, not fixed future defaults. Release selection follows the
[Vault releases](https://releases.hashicorp.com/vault/),
[MongoDB stable release notes](https://www.mongodb.com/docs/manual/release-notes/),
[NATS releases](https://github.com/nats-io/nats-server/releases/latest),
[Nginx downloads](https://nginx.org/en/download.html),
[Go releases](https://go.dev/dl/) and
[Flutter stable channel](https://github.com/flutter/flutter/tree/stable).
The implementation follows HashiCorp's
[MongoDB database plugin](https://developer.hashicorp.com/vault/docs/secrets/databases/mongodb)
and [Agent process supervisor](https://developer.hashicorp.com/vault/docs/agent-and-proxy/agent/process-supervisor)
contracts.