Run local infrastructure and product images with Vault-managed secrets #43
No files matched your search
@@ -0,0 +1,16 @@
|
||||
.git
|
||||
.local
|
||||
**/.dart_tool
|
||||
**/build
|
||||
**/.env*
|
||||
**/node_modules
|
||||
**/*.pb.go
|
||||
frontend/app/lib/generated
|
||||
**/.DS_Store
|
||||
dist
|
||||
website
|
||||
**/config.yml
|
||||
**/tls
|
||||
**/*.pem
|
||||
**/*.key
|
||||
**/__pycache__
|
||||
@@ -34,3 +34,9 @@ pnpm-debug.log*
|
||||
/backend/notifications/interface/notifications/**/*.pb.go
|
||||
/backend/notifications/config.yml
|
||||
/backend/notifications/notifications
|
||||
|
||||
# Local container pins, Vault bootstrap material and service identities
|
||||
/.local/
|
||||
|
||||
# Python bytecode
|
||||
__pycache__/
|
||||
@@ -22,6 +22,16 @@
|
||||
generated outputs. Document the generation workflow so a clean checkout can
|
||||
build without committed generated code.
|
||||
|
||||
## Container versions
|
||||
|
||||
Use only the latest stable upstream releases for infrastructure, product base
|
||||
images, build stages and disposable test containers. Exclude prereleases and
|
||||
vendor mainline channels when a separate stable channel exists. Resolve stable
|
||||
versions when building, pull fresh manifests, and pin the resolved image digests
|
||||
for that local build. Fail when stable release resolution fails; never silently
|
||||
fall back to an older cached version. Keep the reproducible resolver tracked and
|
||||
its generated local pins ignored by Git.
|
||||
|
||||
## Data and state architecture
|
||||
|
||||
Preserve explicit **DTO → mapper → service → data model → provider → UI**
|
||||
|
||||
@@ -17,10 +17,12 @@ test-frontend:
|
||||
.PHONY: test-backend lint-backend run-backend
|
||||
|
||||
test-backend: generate-notifications
|
||||
cd backend/shared && go test -race ./...
|
||||
cd backend/notifications/interface && go test -race ./...
|
||||
cd backend/notifications && go test -race ./...
|
||||
cd backend/product && go test -race ./...
|
||||
docker pull mongo:latest
|
||||
docker pull nats:latest
|
||||
cd backend/shared && go test -race -count=1 ./...
|
||||
cd backend/notifications/interface && go test -race -count=1 ./...
|
||||
cd backend/notifications && go test -race -count=1 ./...
|
||||
cd backend/product && go test -race -count=1 ./...
|
||||
|
||||
lint-backend: generate-notifications
|
||||
cd backend/product && go test ./architecture
|
||||
@@ -49,3 +51,32 @@ package-notifications-interface: generate-notifications
|
||||
mkdir -p dist
|
||||
tar -czf dist/notifications-proto-v1.tar.gz -C backend/notifications/interface proto README.md
|
||||
tar -czf dist/notifications-go-v1.tar.gz -C backend/notifications/interface go.mod go.sum notifications README.md
|
||||
|
||||
# Local product stack. Releases are resolved afresh, then pinned by digest.
|
||||
COMPOSE = docker compose --env-file .local/dev/images.env
|
||||
.PHONY: build up down logs ps test-dev
|
||||
build:
|
||||
python3 dev/images.py
|
||||
$(COMPOSE) pull vault mongo
|
||||
$(COMPOSE) build --pull
|
||||
|
||||
up:
|
||||
python3 dev/local.py up
|
||||
|
||||
down:
|
||||
$(COMPOSE) down
|
||||
|
||||
logs:
|
||||
$(COMPOSE) logs --tail=100 -f
|
||||
|
||||
ps:
|
||||
$(COMPOSE) ps
|
||||
|
||||
test-dev:
|
||||
python3 -m unittest discover -s dev -p 'test_*.py'
|
||||
python3 dev/local.py verify
|
||||
|
||||
.PHONY: test-dev-infra
|
||||
test-dev-infra:
|
||||
python3 -m unittest discover -s dev -p 'test_*.py'
|
||||
python3 dev/local.py verify-infra
|
||||
@@ -32,6 +32,23 @@ Each application owns its dependencies and build configuration. Shared Dart and
|
||||
Flutter packages can be reused by a future mobile app; mobile runners are not
|
||||
currently included. Local `.vscode/` configuration is ignored.
|
||||
|
||||
## Local Docker development
|
||||
|
||||
```sh
|
||||
make build && make up
|
||||
```
|
||||
|
||||
This builds the product images and starts an isolated Vault, authenticated MongoDB
|
||||
replica set and NATS JetStream. First startup intentionally stops after bootstrap
|
||||
until you set the Resend API key and verified sender in Vault's `kv/email` secret.
|
||||
Then rerun `make up` to start notifications, BFF and the frontend at
|
||||
[https://localhost:5126](https://localhost:5126).
|
||||
|
||||
All image inputs resolve to current stable releases and are pinned by digest for
|
||||
the build. Services fetch secrets from Vault; Mongo application credentials come
|
||||
from Vault's database plugin. See the [local development guide](docs/local-development.md)
|
||||
for Vault provisioning, local HTTPS trust, lifecycle commands and verification.
|
||||
|
||||
## Run the consumer app
|
||||
|
||||
Requires Flutter **3.47.0 or later** with Dart **3.13.0 or later** (below Dart 4), matching
|
||||
|
||||
@@ -50,7 +50,7 @@ func (s *sender) Send(_ context.Context, id string, message model.Email) email.O
|
||||
func TestDurableDelivery(t *testing.T) {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
|
||||
defer cancel()
|
||||
container, err := mongodb.Run(ctx, "mongo:8.0")
|
||||
container, err := mongodb.Run(ctx, "mongo:latest")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -192,7 +192,7 @@ func TestDurableDelivery(t *testing.T) {
|
||||
}
|
||||
}
|
||||
// Verify the real JetStream publisher/consumer path with an isolated broker.
|
||||
ncContainer, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{ContainerRequest: testcontainers.ContainerRequest{Image: "nats:2.11-alpine", ExposedPorts: []string{"4222/tcp"}, Cmd: []string{"-js"}, WaitingFor: wait.ForListeningPort("4222/tcp")}, Started: true})
|
||||
ncContainer, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{ContainerRequest: testcontainers.ContainerRequest{Image: "nats:latest", ExposedPorts: []string{"4222/tcp"}, Cmd: []string{"-js"}, WaitingFor: wait.ForListeningPort("4222/tcp")}, Started: true})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
@@ -24,7 +24,7 @@ import (
|
||||
func TestProfileStartsOnEmptyDatabase(t *testing.T) {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), time.Minute)
|
||||
defer cancel()
|
||||
container, err := mongodb.Run(ctx, "mongo:8", mongodb.WithReplicaSet("rs"))
|
||||
container, err := mongodb.Run(ctx, "mongo:latest", mongodb.WithReplicaSet("rs"))
|
||||
require.NoError(t, err)
|
||||
t.Cleanup(func() { require.NoError(t, container.Terminate(context.Background())) })
|
||||
uri, err := container.ConnectionString(ctx)
|
||||
|
||||
@@ -44,7 +44,7 @@ func (m *capturedMail) Send(_ context.Context, r notifications.Message) error {
|
||||
func TestSignupActivationAndLoginEmailFlow(t *testing.T) {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), time.Minute)
|
||||
defer cancel()
|
||||
container, err := mongodb.Run(ctx, "mongo:8", mongodb.WithReplicaSet("rs"))
|
||||
container, err := mongodb.Run(ctx, "mongo:latest", mongodb.WithReplicaSet("rs"))
|
||||
require.NoError(t, err)
|
||||
t.Cleanup(func() { require.NoError(t, container.Terminate(context.Background())) })
|
||||
uri, err := container.ConnectionString(ctx)
|
||||
|
||||
@@ -29,7 +29,7 @@ func customer(login string) *customermodel.Account {
|
||||
func TestCustomerLifecycleWithoutTenancy(t *testing.T) {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
|
||||
defer cancel()
|
||||
container, err := mongodb.Run(ctx, "mongo:8", mongodb.WithReplicaSet("rs"))
|
||||
container, err := mongodb.Run(ctx, "mongo:latest", mongodb.WithReplicaSet("rs"))
|
||||
require.NoError(t, err)
|
||||
t.Cleanup(func() { require.NoError(t, container.Terminate(context.Background())) })
|
||||
uri, err := container.ConnectionString(ctx)
|
||||
|
||||
@@ -43,7 +43,7 @@ func (d *capturedDriver) Send(_ context.Context, m model.Delivery) error {
|
||||
func TestServiceTransactionsAndFactors(t *testing.T) {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
|
||||
defer cancel()
|
||||
container, err := mongodb.Run(ctx, "mongo:8", mongodb.WithReplicaSet("rs"))
|
||||
container, err := mongodb.Run(ctx, "mongo:latest", mongodb.WithReplicaSet("rs"))
|
||||
require.NoError(t, err)
|
||||
t.Cleanup(func() { require.NoError(t, container.Terminate(context.Background())) })
|
||||
uri, err := container.ConnectionString(ctx)
|
||||
|
||||
+145
@@ -0,0 +1,145 @@
|
||||
name: sxchnge-local
|
||||
|
||||
x-agent: &agent
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
VAULT_ADDR: http://vault:8200
|
||||
tmpfs:
|
||||
- /run/sxchnge:uid=100,gid=1000,mode=0700
|
||||
|
||||
x-backend-build: &backend-build
|
||||
context: .
|
||||
dockerfile: dev/images/backend.Dockerfile
|
||||
|
||||
services:
|
||||
vault:
|
||||
image: ${VAULT_IMAGE:?Run make build first}
|
||||
user: root
|
||||
entrypoint: [/bin/sh, /etc/vault/start.sh]
|
||||
restart: unless-stopped
|
||||
ports: ['127.0.0.1:18200:8200']
|
||||
volumes:
|
||||
- vault-data:/vault/file
|
||||
- ./dev/vault/server.hcl:/etc/vault/server.hcl:ro
|
||||
- ./dev/start-vault.sh:/etc/vault/start.sh:ro
|
||||
- ./.local/dev/vault:/bootstrap:ro
|
||||
healthcheck:
|
||||
test: [CMD-SHELL, 'VAULT_ADDR=http://127.0.0.1:8200 vault status >/dev/null']
|
||||
interval: 3s
|
||||
timeout: 3s
|
||||
retries: 40
|
||||
|
||||
mongo:
|
||||
image: ${MONGO_IMAGE:?Run make build first}
|
||||
entrypoint: [/bin/sh, /etc/mongo/start.sh]
|
||||
command: [mongod, --replSet, rs0, --bind_ip_all, --keyFile, /tmp/mongo-keyfile]
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
MONGO_INITDB_ROOT_USERNAME: local-admin
|
||||
MONGO_INITDB_ROOT_PASSWORD_FILE: /run/mongo/password
|
||||
volumes:
|
||||
- ./dev/start-mongo.sh:/etc/mongo/start.sh:ro
|
||||
- mongo-data:/data/db
|
||||
- ./.local/dev/mongo:/run/mongo:ro
|
||||
healthcheck:
|
||||
test: [CMD, mongosh, --quiet, --eval, 'quit(db.hello().isWritablePrimary ? 0 : 1)']
|
||||
interval: 3s
|
||||
timeout: 5s
|
||||
retries: 40
|
||||
|
||||
nats:
|
||||
<<: *agent
|
||||
image: sxchnge-local/nats:dev
|
||||
build:
|
||||
context: .
|
||||
dockerfile: dev/images/nats.Dockerfile
|
||||
args:
|
||||
NATS_IMAGE: ${NATS_IMAGE:?Run make build first}
|
||||
VAULT_IMAGE: ${VAULT_IMAGE:?Run make build first}
|
||||
volumes:
|
||||
- nats-data:/data
|
||||
- ./.local/dev/agents/nats:/etc/agent:ro
|
||||
depends_on:
|
||||
vault: {condition: service_healthy}
|
||||
healthcheck:
|
||||
test: [CMD, wget, -q, -O, /dev/null, 'http://127.0.0.1:8222/healthz?js-enabled-only=true']
|
||||
interval: 3s
|
||||
timeout: 3s
|
||||
retries: 40
|
||||
|
||||
notifications:
|
||||
<<: *agent
|
||||
image: sxchnge-local/notifications:dev
|
||||
build:
|
||||
<<: *backend-build
|
||||
args:
|
||||
GO_IMAGE: ${GO_IMAGE:?Run make build first}
|
||||
VAULT_IMAGE: ${VAULT_IMAGE:?Run make build first}
|
||||
MODULE: notifications
|
||||
environment:
|
||||
VAULT_ADDR: http://vault:8200
|
||||
SXCHNGE_SERVICE: notifications
|
||||
volumes:
|
||||
- ./.local/dev/agents/notifications:/etc/agent:ro
|
||||
depends_on:
|
||||
vault: {condition: service_healthy}
|
||||
mongo: {condition: service_healthy}
|
||||
nats: {condition: service_healthy}
|
||||
healthcheck:
|
||||
test: [CMD, nc, -z, 127.0.0.1, '8083']
|
||||
interval: 3s
|
||||
timeout: 3s
|
||||
retries: 40
|
||||
|
||||
bff:
|
||||
<<: *agent
|
||||
image: sxchnge-local/bff:dev
|
||||
build:
|
||||
<<: *backend-build
|
||||
args:
|
||||
GO_IMAGE: ${GO_IMAGE:?Run make build first}
|
||||
VAULT_IMAGE: ${VAULT_IMAGE:?Run make build first}
|
||||
MODULE: product
|
||||
environment:
|
||||
VAULT_ADDR: http://vault:8200
|
||||
SXCHNGE_SERVICE: bff
|
||||
SXCHNGE_DOMAIN: localhost:5126
|
||||
SXCHNGE_API_PROTOCOL: https
|
||||
SXCHNGE_API_ENDPOINT: /api/v1
|
||||
volumes:
|
||||
- ./.local/dev/agents/bff:/etc/agent:ro
|
||||
depends_on:
|
||||
notifications: {condition: service_healthy}
|
||||
mongo: {condition: service_healthy}
|
||||
healthcheck:
|
||||
test: [CMD, wget, -q, -O, /dev/null, 'http://127.0.0.1:8082/api/v1/health']
|
||||
interval: 3s
|
||||
timeout: 3s
|
||||
retries: 40
|
||||
|
||||
frontend:
|
||||
image: sxchnge-local/frontend:dev
|
||||
build:
|
||||
context: .
|
||||
dockerfile: dev/images/frontend.Dockerfile
|
||||
args:
|
||||
UBUNTU_IMAGE: ${UBUNTU_IMAGE:?Run make build first}
|
||||
NGINX_IMAGE: ${NGINX_IMAGE:?Run make build first}
|
||||
FLUTTER_VERSION: ${FLUTTER_VERSION:?Run make build first}
|
||||
FLUTTER_COMMIT: ${FLUTTER_COMMIT:?Run make build first}
|
||||
restart: unless-stopped
|
||||
ports: ['127.0.0.1:5126:8443']
|
||||
volumes:
|
||||
- ./.local/dev/frontend:/etc/tls:ro
|
||||
depends_on:
|
||||
bff: {condition: service_healthy}
|
||||
healthcheck:
|
||||
test: [CMD, wget, -q, --no-check-certificate, -O, /dev/null, 'https://127.0.0.1:8443/']
|
||||
interval: 3s
|
||||
timeout: 3s
|
||||
retries: 40
|
||||
|
||||
volumes:
|
||||
vault-data:
|
||||
mongo-data:
|
||||
nats-data:
|
||||
@@ -0,0 +1,43 @@
|
||||
"""Build per-service Vault Agent configs (no secret values in configs)."""
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
BASE = (Path(__file__).parent / 'vault' / 'agent.hcl').read_text()
|
||||
|
||||
|
||||
def kv(path, field):
|
||||
return '{{ with secret "kv/data/' + path + '" }}{{ .Data.data.' + field + ' }}{{ end }}'
|
||||
|
||||
|
||||
def config(service):
|
||||
env = {}
|
||||
if service == 'nats':
|
||||
env['NATS_TOKEN'] = kv('nats', 'token')
|
||||
else:
|
||||
database = 'sxchnge' if service == 'bff' else 'sxchnge_notifications'
|
||||
env['SXCHNGE_MONGO_URI'] = ('{{ with secret "database/creds/' + service + '" }}'
|
||||
'mongodb://{{ .Data.username | urlquery }}:{{ .Data.password | urlquery }}'
|
||||
'@mongo:27017/' + database + '?authSource=admin&replicaSet=rs0{{ end }}')
|
||||
env['TLS_CA'] = kv('tls', 'ca')
|
||||
if service == 'bff':
|
||||
env['SXCHNGE_JWT_SECRET'] = kv('bff', 'jwt')
|
||||
env['SXCHNGE_NOTIFICATIONS_TOKEN'] = kv('caller', 'token')
|
||||
else:
|
||||
env['SXCHNGE_NOTIFICATIONS_PRODUCT_TOKEN'] = kv('caller', 'token')
|
||||
env['SXCHNGE_NATS_URL'] = ('{{ with secret "kv/data/nats" }}'
|
||||
'nats://{{ .Data.data.token }}@nats:4222{{ end }}')
|
||||
env['TLS_CERT'] = kv('notifications-tls', 'certificate')
|
||||
env['TLS_KEY'] = kv('notifications-tls', 'private_key')
|
||||
env['EMAIL_API_KEY'] = kv('email', 'api_key')
|
||||
# JSON is valid YAML and escapes arbitrary provider configuration.
|
||||
env['EMAIL_CONFIG'] = ('{{ with secret "kv/data/email" }}'
|
||||
'email:\n driver: {{ .Data.data.driver | toJSON }}\n'
|
||||
' settings:\n api_key_env: EMAIL_API_KEY\n'
|
||||
' sender_address: {{ .Data.data.sender_address | toJSON }}\n'
|
||||
' sender_name: {{ .Data.data.sender_name | toJSON }}\n{{ end }}')
|
||||
result = BASE
|
||||
for name, template in env.items():
|
||||
result += '\nenv_template ' + json.dumps(name) + ' {\n contents = ' + json.dumps(template) + '\n error_on_missing_key = true\n}\n'
|
||||
command = '/usr/local/bin/start-nats' if service == 'nats' else '/usr/local/bin/start-service'
|
||||
result += '\nexec {\n command = [' + json.dumps(command) + ']\n restart_on_secret_changes = "always"\n restart_stop_signal = "SIGTERM"\n}\n'
|
||||
return result
|
||||
@@ -0,0 +1,105 @@
|
||||
"""Idempotent provisioning for the isolated local Compose project."""
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import time
|
||||
|
||||
from agents import config
|
||||
from vault import STATE, write
|
||||
|
||||
ROOT = STATE.parents[1]
|
||||
COMPOSE = ['docker', 'compose', '--env-file', str(STATE / 'images.env')]
|
||||
|
||||
|
||||
def compose(*args, capture=False, input=None):
|
||||
result = subprocess.run(COMPOSE + list(args), cwd=ROOT, input=input, text=True,
|
||||
capture_output=capture)
|
||||
if result.returncode:
|
||||
# Bootstrap stdin and child errors may contain credentials. Never echo them.
|
||||
raise RuntimeError('Compose command failed: ' + ' '.join(args[:3]))
|
||||
return result.stdout if capture else None
|
||||
|
||||
|
||||
def mongo(script, password):
|
||||
auth = 'const admin = db.getSiblingDB("admin"); admin.auth("local-admin", ' + json.dumps(password) + ');\n'
|
||||
return compose('exec', '-T', 'mongo', 'mongosh', '--quiet', '--file', '/dev/stdin', capture=True, input=auth + script)
|
||||
|
||||
|
||||
def provision_mongo(vault):
|
||||
secret = vault.ensure('bootstrap/mongo', lambda: {'password': vault.random(), 'keyfile': vault.random(384), 'manager_password': vault.random()})
|
||||
write(STATE / 'mongo' / 'password', secret['password'])
|
||||
write(STATE / 'mongo' / 'keyfile', secret['keyfile'])
|
||||
compose('up', '-d', 'mongo')
|
||||
for attempt in range(90):
|
||||
try:
|
||||
mongo('try { rs.status(); } catch(e) { if(e.code === 94) rs.initiate({_id:"rs0",members:[{_id:0,host:"mongo:27017"}]}); else throw e; }', secret['password'])
|
||||
if 'true' in mongo('print(db.hello().isWritablePrimary)', secret['password']):
|
||||
break
|
||||
except RuntimeError:
|
||||
pass
|
||||
time.sleep(2)
|
||||
else:
|
||||
raise RuntimeError('Mongo replica set did not become primary')
|
||||
if vault.call('database/config/mongo', missing=True) is None:
|
||||
mongo('if (!admin.getUser("vault-manager")) admin.createUser({user:"vault-manager",pwd:' + json.dumps(secret['manager_password']) + ',roles:[{role:"userAdminAnyDatabase",db:"admin"}]});', secret['password'])
|
||||
vault.call('database/config/mongo', {'plugin_name': 'mongodb-database-plugin',
|
||||
'allowed_roles': ['bff', 'notifications'],
|
||||
'connection_url': 'mongodb://{{username}}:{{password}}@mongo:27017/admin?replicaSet=rs0',
|
||||
'username': 'vault-manager', 'password': secret['manager_password']})
|
||||
vault.call('database/rotate-root/mongo', {})
|
||||
for role, database in [('bff', 'sxchnge'), ('notifications', 'sxchnge_notifications')]:
|
||||
vault.call('database/roles/' + role, {'db_name': 'mongo',
|
||||
'creation_statements': json.dumps({'db': 'admin', 'roles': [{'role': 'readWrite', 'db': database}]}),
|
||||
'default_ttl': '1h', 'max_ttl': '24h'})
|
||||
# Public browser client: an identifier, never a client secret in the web bundle.
|
||||
mongo('db.getSiblingDB("sxchnge").clients.updateOne({clientId:"sxchnge-local-web"}, {$setOnInsert: {clientName:"Local browser",allowedCIDRs:["0.0.0.0/0","::/0"],allowedScopes:["account"],redirectURIs:["https://localhost:5126"],grantTypes:["password","refresh_token"],tokenEndpointAuthMethod:"none",isRevoked:false}}, {upsert:true});', secret['password'])
|
||||
|
||||
|
||||
def provision_agents(vault):
|
||||
paths = {'bff': ['bff', 'caller', 'tls'],
|
||||
'notifications': ['caller', 'tls', 'notifications-tls', 'nats', 'email'],
|
||||
'nats': ['nats']}
|
||||
for service, secrets in paths.items():
|
||||
directory = STATE / 'agents' / service
|
||||
directory.mkdir(parents=True, exist_ok=True, mode=0o755)
|
||||
os.chmod(directory, 0o755)
|
||||
policy = ''.join('path "kv/data/' + p + '" { capabilities = ["read"] }\n' for p in secrets)
|
||||
if service != 'nats':
|
||||
policy += 'path "database/creds/' + service + '" { capabilities = ["read"] }\n'
|
||||
vault.call('sys/policies/acl/' + service, {'policy': policy})
|
||||
vault.call('auth/approle/role/' + service, {'token_policies': [service],
|
||||
'token_period': '1h', 'secret_id_ttl': '0', 'secret_id_num_uses': 0})
|
||||
role_id = vault.call('auth/approle/role/' + service + '/role-id')['data']['role_id']
|
||||
write(directory / 'role-id', role_id, 0o444)
|
||||
if not (directory / 'secret-id').exists():
|
||||
secret_id = vault.call('auth/approle/role/' + service + '/secret-id', {})['data']['secret_id']
|
||||
write(directory / 'secret-id', secret_id, 0o444)
|
||||
write(directory / 'config.hcl', config(service), 0o444)
|
||||
|
||||
|
||||
def up(vault):
|
||||
if not (STATE / 'images.env').exists():
|
||||
raise RuntimeError('Run make build before make up')
|
||||
(STATE / 'vault').mkdir(parents=True, exist_ok=True, mode=0o755)
|
||||
compose('up', '-d', 'vault')
|
||||
vault.initialize()
|
||||
vault.mounts()
|
||||
vault.ensure('nats', lambda: {'token': vault.random()})
|
||||
vault.ensure('caller', lambda: {'token': vault.random()})
|
||||
vault.ensure('bff', lambda: {'jwt': vault.random()})
|
||||
vault.ensure('email', lambda: {'driver': 'resend', 'api_key': '',
|
||||
'sender_address': 'notifications@localhost.test', 'sender_name': 'SXCHNGE local'})
|
||||
vault.certificates()
|
||||
provision_mongo(vault)
|
||||
provision_agents(vault)
|
||||
compose('up', '-d', '--wait', '--wait-timeout', '120', 'nats')
|
||||
email = vault.kv('email')
|
||||
key, sender = email.get('api_key'), email.get('sender_address')
|
||||
if (not isinstance(key, str) or not key.strip()
|
||||
or not isinstance(sender, str) or not sender.strip()
|
||||
or sender == 'notifications@localhost.test'):
|
||||
raise RuntimeError('Vault kv/email needs a Resend api_key and verified sender_address. Run python3 dev/local.py email, then make up.')
|
||||
compose('up', '-d', '--wait', '--wait-timeout', '180')
|
||||
compose('exec', '-T', 'frontend', 'nginx', '-s', 'reload', capture=True)
|
||||
print('Local stack ready: https://localhost:5126; Vault: http://127.0.0.1:18200')
|
||||
print('Local CA: .local/dev/ca.pem. Email driver: ' + vault.kv('email')['driver'])
|
||||
@@ -0,0 +1,39 @@
|
||||
# SXCHNGE-owned Mongo database and JWT key. Never point these at Sendico.
|
||||
http_server:
|
||||
listen_address: 0.0.0.0:8082
|
||||
read_header_timeout: 5
|
||||
shutdown_timeout: 10
|
||||
database:
|
||||
driver: mongodb
|
||||
settings:
|
||||
uri_env: SXCHNGE_MONGO_URI
|
||||
database: sxchnge
|
||||
api:
|
||||
middleware:
|
||||
domain_env: SXCHNGE_DOMAIN
|
||||
api_endpoint_env: SXCHNGE_API_ENDPOINT
|
||||
api_protocol_env: SXCHNGE_API_PROTOCOL
|
||||
signature:
|
||||
algorithm: HS256
|
||||
secret_key_env: SXCHNGE_JWT_SECRET
|
||||
CORS:
|
||||
allowed_origins: ["https://localhost:5126"]
|
||||
allowed_methods: ["GET", "POST", "PATCH", "DELETE", "OPTIONS"]
|
||||
allowed_headers: ["Content-Type", "Authorization", "X-Auth-Transport", "X-CSRF-Protection"]
|
||||
allow_credentials: true
|
||||
max_age: 300
|
||||
token:
|
||||
expiration_hours:
|
||||
account: 1
|
||||
refresh: 720
|
||||
length: 32
|
||||
password:
|
||||
token_length: 32
|
||||
check:
|
||||
min_length: 12
|
||||
notifications:
|
||||
address: notifications:8083
|
||||
token_env: SXCHNGE_NOTIFICATIONS_TOKEN
|
||||
ca_file: /run/sxchnge/ca.pem
|
||||
server_name: notifications
|
||||
activation_url: https://localhost:5126/verify-email
|
||||
@@ -0,0 +1,17 @@
|
||||
server {
|
||||
listen 8443 ssl;
|
||||
ssl_certificate /etc/tls/server.crt;
|
||||
ssl_certificate_key /etc/tls/server.key;
|
||||
ssl_protocols TLSv1.2 TLSv1.3;
|
||||
root /usr/share/nginx/html;
|
||||
index index.html;
|
||||
location /api/ {
|
||||
resolver 127.0.0.11 valid=10s ipv6=off;
|
||||
set $bff http://bff:8082;
|
||||
proxy_pass $bff;
|
||||
proxy_set_header Host $http_host;
|
||||
proxy_set_header X-Forwarded-Proto https;
|
||||
proxy_set_header X-Forwarded-For $remote_addr;
|
||||
}
|
||||
location / { try_files $uri $uri/ /index.html; }
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
listen: "0.0.0.0:8083"
|
||||
tls_cert: "/run/sxchnge/server.crt"
|
||||
tls_key: "/run/sxchnge/server.key"
|
||||
database:
|
||||
driver: mongodb
|
||||
settings:
|
||||
uri_env: SXCHNGE_MONGO_URI
|
||||
database: sxchnge_notifications
|
||||
messaging:
|
||||
driver: NATS
|
||||
settings:
|
||||
url_env: SXCHNGE_NATS_URL
|
||||
broker_name: sxchnge-notifications
|
||||
max_reconnects: -1
|
||||
reconnect_wait: 2
|
||||
templates: "/etc/sxchnge/templates.json"
|
||||
callers:
|
||||
product: "SXCHNGE_NOTIFICATIONS_PRODUCT_TOKEN"
|
||||
# Use replicas: 3 and environment: production for a three-node NATS cluster.
|
||||
# mode: ops validates resources without modifying them; app provisions them.
|
||||
queue:
|
||||
stream:
|
||||
replicas: 1
|
||||
max_age_hours: 48
|
||||
provisioning:
|
||||
mode: app
|
||||
environment: non-production
|
||||
@@ -0,0 +1,72 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Resolve stable upstream releases on every build; pin their registry digests."""
|
||||
import concurrent.futures
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import subprocess
|
||||
import urllib.request
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
STATE = ROOT / '.local' / 'dev'
|
||||
|
||||
|
||||
def get(url):
|
||||
with urllib.request.urlopen(url, timeout=45) as response:
|
||||
return json.load(response)
|
||||
|
||||
|
||||
def image(repo, alias):
|
||||
url = f'https://hub.docker.com/v2/repositories/{repo}/tags'
|
||||
selected = get(f'{url}/{alias}')
|
||||
digest = selected['digest']
|
||||
if not re.fullmatch(r'sha256:[a-f0-9]{64}', digest):
|
||||
raise ValueError('Registry returned an invalid digest')
|
||||
# Find the exact numeric tag represented by the maintained stable alias.
|
||||
candidates = get(f'{url}?page_size=100')['results']
|
||||
names = [x['name'] for x in candidates if x.get('digest') == digest
|
||||
and re.fullmatch(r'\d+\.\d+(?:\.\d+)?(?:-alpine)?', x['name'])]
|
||||
if not names:
|
||||
raise ValueError(f'No stable version tag for {repo}:{alias}')
|
||||
tag = max(names, key=lambda x: tuple(map(int, x.split('-')[0].split('.'))))
|
||||
return f'{repo}:{tag}@{digest}'
|
||||
|
||||
|
||||
def flutter():
|
||||
commit = get('https://api.github.com/repos/flutter/flutter/commits/stable')['sha']
|
||||
tags = subprocess.check_output(['git', 'ls-remote', '--tags',
|
||||
'https://github.com/flutter/flutter.git'], text=True)
|
||||
names = [line.split('refs/tags/')[1] for line in tags.splitlines()
|
||||
if line.startswith(commit + '\trefs/tags/')
|
||||
and re.fullmatch(r'\d+\.\d+\.\d+', line.split('refs/tags/')[1])]
|
||||
if not names:
|
||||
raise ValueError('Flutter stable does not point at a stable release tag')
|
||||
return max(names, key=lambda x: tuple(map(int, x.split('.')))), commit
|
||||
|
||||
|
||||
def main():
|
||||
go = get('https://go.dev/dl/?mode=json')[0]
|
||||
if not go['stable'] or not re.fullmatch(r'go\d+\.\d+\.\d+', go['version']):
|
||||
raise ValueError('Go release is not stable')
|
||||
specs = {'VAULT_IMAGE': ('hashicorp/vault', 'latest'),
|
||||
'MONGO_IMAGE': ('library/mongo', 'latest'),
|
||||
'NATS_IMAGE': ('library/nats', 'latest'),
|
||||
'NGINX_IMAGE': ('library/nginx', 'stable-alpine'),
|
||||
'GO_IMAGE': ('library/golang', go['version'][2:] + '-alpine'),
|
||||
'UBUNTU_IMAGE': ('library/ubuntu', 'latest')}
|
||||
with concurrent.futures.ThreadPoolExecutor() as pool:
|
||||
futures = {key: pool.submit(image, *spec) for key, spec in specs.items()}
|
||||
refs = {key: future.result() for key, future in futures.items()}
|
||||
refs['FLUTTER_VERSION'], refs['FLUTTER_COMMIT'] = flutter()
|
||||
STATE.mkdir(parents=True, exist_ok=True, mode=0o700)
|
||||
target = STATE / 'images.env'
|
||||
temporary = target.with_suffix('.tmp')
|
||||
temporary.write_text(''.join(f'{key}={value}\n' for key, value in refs.items()))
|
||||
os.replace(temporary, target)
|
||||
for key, value in refs.items():
|
||||
print(f'{key}={value}')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,21 @@
|
||||
ARG GO_IMAGE
|
||||
ARG VAULT_IMAGE
|
||||
FROM ${GO_IMAGE} AS build
|
||||
RUN apk add --no-cache protobuf protobuf-dev make
|
||||
WORKDIR /src
|
||||
COPY Makefile ./
|
||||
COPY backend ./backend
|
||||
RUN make generate-notifications
|
||||
ARG MODULE
|
||||
WORKDIR /src/backend/${MODULE}
|
||||
RUN --mount=type=cache,target=/root/.cache/go-build --mount=type=cache,target=/go/pkg/mod CGO_ENABLED=0 go build -trimpath -o /out/service .
|
||||
|
||||
FROM ${VAULT_IMAGE}
|
||||
USER root
|
||||
COPY --from=build /out/service /usr/local/bin/service
|
||||
COPY dev/start-service.sh /usr/local/bin/start-service
|
||||
COPY dev/config /etc/sxchnge
|
||||
COPY backend/notifications/templates.example.json /etc/sxchnge/templates.json
|
||||
RUN mkdir -p /run/sxchnge && chown 100:1000 /run/sxchnge
|
||||
USER 100:1000
|
||||
ENTRYPOINT ["vault", "agent", "-config=/etc/agent/config.hcl"]
|
||||
@@ -0,0 +1,17 @@
|
||||
ARG UBUNTU_IMAGE
|
||||
ARG NGINX_IMAGE
|
||||
FROM ${UBUNTU_IMAGE} AS build
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates curl git unzip xz-utils libglu1-mesa && rm -rf /var/lib/apt/lists/*
|
||||
ARG FLUTTER_VERSION
|
||||
ARG FLUTTER_COMMIT
|
||||
RUN git clone --depth 1 --branch "${FLUTTER_VERSION}" https://github.com/flutter/flutter.git /opt/flutter && test "$(git -C /opt/flutter rev-parse HEAD)" = "${FLUTTER_COMMIT}"
|
||||
ENV PATH="/opt/flutter/bin:${PATH}"
|
||||
RUN flutter config --no-analytics && flutter precache --web
|
||||
WORKDIR /src
|
||||
COPY frontend ./frontend
|
||||
WORKDIR /src/frontend/app
|
||||
ARG FRONTEND_URL=https://localhost:5126
|
||||
RUN flutter pub get && flutter gen-l10n && flutter build web --no-wasm-dry-run --dart-define=SXCHNGE_API_URL=${FRONTEND_URL}/api/v1 --dart-define=SXCHNGE_CLIENT_ID=sxchnge-local-web
|
||||
FROM ${NGINX_IMAGE}
|
||||
COPY dev/config/nginx.conf /etc/nginx/conf.d/default.conf
|
||||
COPY --from=build /src/frontend/app/build/web /usr/share/nginx/html
|
||||
@@ -0,0 +1,10 @@
|
||||
ARG NATS_IMAGE
|
||||
ARG VAULT_IMAGE
|
||||
FROM ${NATS_IMAGE} AS nats
|
||||
FROM ${VAULT_IMAGE}
|
||||
USER root
|
||||
COPY --from=nats /nats-server /usr/local/bin/nats-server
|
||||
COPY dev/start-nats.sh /usr/local/bin/start-nats
|
||||
RUN mkdir -p /data /run/sxchnge && chown -R 100:1000 /data /run/sxchnge
|
||||
USER 100:1000
|
||||
ENTRYPOINT ["vault", "agent", "-config=/etc/agent/config.hcl"]
|
||||
@@ -0,0 +1,47 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Local stack lifecycle. Requires Python 3 and Docker Compose."""
|
||||
import sys
|
||||
from bootstrap import up
|
||||
from vault import Vault
|
||||
|
||||
|
||||
def main():
|
||||
vault = Vault()
|
||||
if len(sys.argv) == 2 and sys.argv[1] == 'up':
|
||||
import fcntl
|
||||
from vault import STATE
|
||||
STATE.mkdir(parents=True, exist_ok=True, mode=0o700)
|
||||
with (STATE / 'bootstrap.lock').open('w') as lock:
|
||||
try:
|
||||
fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB)
|
||||
except BlockingIOError:
|
||||
raise RuntimeError('Another local bootstrap is running') from None
|
||||
up(vault)
|
||||
elif len(sys.argv) == 2 and sys.argv[1] == 'verify':
|
||||
from verify import verify
|
||||
vault.existing()
|
||||
verify(vault)
|
||||
elif len(sys.argv) == 2 and sys.argv[1] == 'verify-infra':
|
||||
from verify import infrastructure
|
||||
vault.existing()
|
||||
infrastructure(vault)
|
||||
elif len(sys.argv) == 2 and sys.argv[1] == 'email':
|
||||
import getpass
|
||||
vault.existing()
|
||||
address = input('Verified Resend sender address: ').strip()
|
||||
key = getpass.getpass('Resend API key (hidden): ').strip()
|
||||
if not key or '@' not in address:
|
||||
raise RuntimeError('A key and sender address are required')
|
||||
vault.put('email', {'driver': 'resend', 'api_key': key,
|
||||
'sender_address': address, 'sender_name': 'SXCHNGE'})
|
||||
print('Email configuration saved in Vault. Run make up.')
|
||||
else:
|
||||
raise RuntimeError('Usage: python3 dev/local.py up|verify|verify-infra|email')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
try:
|
||||
main()
|
||||
except (RuntimeError, OSError) as error:
|
||||
print('Local development setup failed: ' + str(error), file=sys.stderr)
|
||||
sys.exit(1)
|
||||
Executable
+6
@@ -0,0 +1,6 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
cp /run/mongo/keyfile /tmp/mongo-keyfile
|
||||
chown mongodb:mongodb /tmp/mongo-keyfile
|
||||
chmod 400 /tmp/mongo-keyfile
|
||||
exec docker-entrypoint.sh "$@"
|
||||
Executable
+13
@@ -0,0 +1,13 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
umask 077
|
||||
# Generated by Vault with hexadecimal encoding; never interpolates user input.
|
||||
case "$NATS_TOKEN" in ''|*[!0-9a-f]*) exit 1;; esac
|
||||
cat > /run/sxchnge/nats.conf <<CONFIG
|
||||
port: 4222
|
||||
http_port: 8222
|
||||
authorization { token: "$NATS_TOKEN" }
|
||||
jetstream { store_dir: "/data/jetstream" }
|
||||
CONFIG
|
||||
unset NATS_TOKEN
|
||||
exec nats-server -c /run/sxchnge/nats.conf
|
||||
Executable
+14
@@ -0,0 +1,14 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
umask 077
|
||||
printf '%s\n' "$TLS_CA" > /run/sxchnge/ca.pem
|
||||
unset TLS_CA
|
||||
if [ "$SXCHNGE_SERVICE" = notifications ]; then
|
||||
printf '%s\n' "$TLS_CERT" > /run/sxchnge/server.crt
|
||||
printf '%s\n' "$TLS_KEY" > /run/sxchnge/server.key
|
||||
printf '%s\n' "$EMAIL_CONFIG" > /run/sxchnge/email.yml
|
||||
unset TLS_CERT TLS_KEY EMAIL_CONFIG
|
||||
cat /etc/sxchnge/notifications.yml /run/sxchnge/email.yml > /run/sxchnge/config.yml
|
||||
exec service -config.file=/run/sxchnge/config.yml
|
||||
fi
|
||||
exec service -config.file=/etc/sxchnge/bff.yml
|
||||
Executable
+16
@@ -0,0 +1,16 @@
|
||||
#!/bin/sh
|
||||
# Local-only automatic unseal. The root token is never mounted in this container.
|
||||
set -eu
|
||||
export VAULT_ADDR=http://127.0.0.1:8200
|
||||
vault server -config=/etc/vault/server.hcl &
|
||||
pid=$!
|
||||
trap 'kill -TERM "$pid"; wait "$pid"' TERM INT
|
||||
until [ -s /bootstrap/unseal ]; do
|
||||
kill -0 "$pid" || exit 1
|
||||
sleep 1
|
||||
done
|
||||
until vault operator unseal "$(cat /bootstrap/unseal)" >/dev/null 2>&1; do
|
||||
kill -0 "$pid" || exit 1
|
||||
sleep 1
|
||||
done
|
||||
wait "$pid"
|
||||
@@ -0,0 +1,62 @@
|
||||
"""Focused regression tests for release selection and bootstrap failure paths."""
|
||||
from pathlib import Path
|
||||
import tempfile
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
import bootstrap
|
||||
import images
|
||||
from vault import Vault
|
||||
|
||||
|
||||
class Releases(unittest.TestCase):
|
||||
def test_selects_exact_stable_digest_not_prerelease(self):
|
||||
digest = 'sha256:' + 'a' * 64
|
||||
versions = [{'name': name, 'digest': digest} for name in ['latest', '2', '2.1', '2.1.0', '2.2.0-rc1']]
|
||||
with patch.object(images, 'get', side_effect=[{'digest': digest}, {'results': versions}]):
|
||||
self.assertEqual(images.image('hashicorp/vault', 'latest'), 'hashicorp/vault:2.1.0@' + digest)
|
||||
|
||||
def test_missing_stable_mapping_fails(self):
|
||||
digest = 'sha256:' + 'a' * 64
|
||||
with patch.object(images, 'get', side_effect=[{'digest': digest}, {'results': [{'name': '3.0.0-rc1', 'digest': digest}]}]):
|
||||
with self.assertRaises(ValueError):
|
||||
images.image('hashicorp/vault', 'latest')
|
||||
|
||||
def test_upstream_failure_does_not_replace_existing_pins(self):
|
||||
with tempfile.TemporaryDirectory() as temporary:
|
||||
state = Path(temporary)
|
||||
target = state / 'images.env'
|
||||
target.write_text('previous build')
|
||||
with patch.object(images, 'STATE', state), patch.object(images, 'get', side_effect=OSError('offline')):
|
||||
with self.assertRaises(OSError):
|
||||
images.main()
|
||||
self.assertEqual(target.read_text(), 'previous build')
|
||||
|
||||
|
||||
class Bootstrap(unittest.TestCase):
|
||||
def test_missing_resend_credentials_stops_before_product_start(self):
|
||||
with tempfile.TemporaryDirectory() as temporary:
|
||||
state = Path(temporary)
|
||||
(state / 'images.env').touch()
|
||||
from unittest.mock import Mock
|
||||
vault = Mock()
|
||||
vault.kv.return_value = {'driver': 'resend', 'api_key': '', 'sender_address': 'notifications@localhost.test'}
|
||||
with patch.object(bootstrap, 'STATE', state), patch.object(bootstrap, 'compose') as compose, \
|
||||
patch.object(bootstrap, 'provision_mongo'), patch.object(bootstrap, 'provision_agents'):
|
||||
with self.assertRaisesRegex(RuntimeError, 'Resend api_key'):
|
||||
bootstrap.up(vault)
|
||||
self.assertEqual(compose.call_args_list[-1].args[-1], 'nats')
|
||||
email_seed = next(c for c in vault.ensure.call_args_list if c.args[0] == 'email').args[1]()
|
||||
self.assertEqual(email_seed['driver'], 'resend')
|
||||
self.assertEqual(email_seed['api_key'], '')
|
||||
|
||||
def test_initialized_vault_without_local_keys_fails(self):
|
||||
with tempfile.TemporaryDirectory() as temporary:
|
||||
with patch('vault.STATE', Path(temporary)), patch.object(Vault, 'call', return_value={'initialized': True}):
|
||||
with self.assertRaisesRegex(RuntimeError, 'Missing .local'):
|
||||
Vault().initialize()
|
||||
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
+117
@@ -0,0 +1,117 @@
|
||||
"""Local Vault bootstrap API; response bodies with secrets never reach logs."""
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
STATE = Path(__file__).resolve().parents[1] / '.local' / 'dev'
|
||||
URL = 'http://127.0.0.1:18200/v1/'
|
||||
|
||||
|
||||
def write(path, value, mode=0o600):
|
||||
path.parent.mkdir(parents=True, exist_ok=True, mode=0o700)
|
||||
tmp = path.with_name(path.name + '.tmp')
|
||||
fd = os.open(tmp, os.O_WRONLY | os.O_CREAT | os.O_TRUNC, mode)
|
||||
with os.fdopen(fd, 'w') as out:
|
||||
out.write(value)
|
||||
os.chmod(tmp, mode)
|
||||
os.replace(tmp, path)
|
||||
|
||||
|
||||
class Vault:
|
||||
def __init__(self):
|
||||
self.token = None
|
||||
|
||||
def call(self, path, data=None, method=None, missing=False):
|
||||
headers = {'Content-Type': 'application/json'}
|
||||
if self.token:
|
||||
headers['X-Vault-Token'] = self.token
|
||||
req = urllib.request.Request(URL + path, headers=headers,
|
||||
data=json.dumps(data).encode() if data is not None else None, method=method)
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=30) as response:
|
||||
body = response.read()
|
||||
return json.loads(body) if body else {}
|
||||
except urllib.error.HTTPError as error:
|
||||
if error.code == 404 and missing:
|
||||
return None
|
||||
raise RuntimeError(f'Vault {path} returned HTTP {error.code}') from None
|
||||
|
||||
def initialize(self):
|
||||
for attempt in range(90):
|
||||
try:
|
||||
initialized = self.call('sys/init')['initialized']
|
||||
break
|
||||
except (urllib.error.URLError, ConnectionError):
|
||||
time.sleep(1)
|
||||
else:
|
||||
raise RuntimeError('Vault did not start within 90 seconds')
|
||||
initfile = STATE / 'admin' / 'init.json'
|
||||
if not initialized:
|
||||
if initfile.exists():
|
||||
raise RuntimeError('Vault data is missing but bootstrap material exists; restore the matching volume or explicitly reset both')
|
||||
init = self.call('sys/init', {'secret_shares': 1, 'secret_threshold': 1})
|
||||
write(initfile, json.dumps(init))
|
||||
if not initfile.exists():
|
||||
raise RuntimeError('Missing .local/dev/admin/init.json for initialized Vault; restore the matching bootstrap material')
|
||||
init = json.loads(initfile.read_text())
|
||||
self.token = init['root_token']
|
||||
write(STATE / 'vault' / 'unseal', init['keys_base64'][0], 0o444)
|
||||
for attempt in range(60):
|
||||
if not self.call('sys/seal-status')['sealed']:
|
||||
return
|
||||
time.sleep(1)
|
||||
raise RuntimeError('Vault did not unseal within 60 seconds')
|
||||
|
||||
def existing(self):
|
||||
self.token = json.loads((STATE / 'admin' / 'init.json').read_text())['root_token']
|
||||
|
||||
def kv(self, path):
|
||||
result = self.call('kv/data/' + path, missing=True)
|
||||
return result['data']['data'] if result else None
|
||||
|
||||
def put(self, path, data):
|
||||
self.call('kv/data/' + path, {'data': data})
|
||||
|
||||
def ensure(self, path, factory):
|
||||
value = self.kv(path)
|
||||
if value is None:
|
||||
value = factory()
|
||||
self.put(path, value)
|
||||
return value
|
||||
|
||||
def random(self, length=32):
|
||||
return self.call('sys/tools/random', {'bytes': length, 'format': 'hex'})['data']['random_bytes']
|
||||
|
||||
def mounts(self):
|
||||
mounts = self.call('sys/mounts')['data']
|
||||
for name, kind in [('kv', 'kv'), ('database', 'database'), ('pki', 'pki')]:
|
||||
if name + '/' not in mounts:
|
||||
settings = {'type': kind}
|
||||
if kind == 'kv':
|
||||
settings['options'] = {'version': '2'}
|
||||
self.call('sys/mounts/' + name, settings)
|
||||
auth = self.call('sys/auth')['data']
|
||||
if 'approle/' not in auth:
|
||||
self.call('sys/auth/approle', {'type': 'approle'})
|
||||
|
||||
def certificates(self):
|
||||
self.call('sys/mounts/pki/tune', {'max_lease_ttl': '87600h'})
|
||||
ca = self.kv('tls')
|
||||
if ca is None:
|
||||
result = self.call('pki/root/generate/internal', {'common_name': 'SXCHNGE local development CA', 'ttl': '87600h'})['data']
|
||||
ca = {'ca': result['certificate']}
|
||||
self.put('tls', ca)
|
||||
self.call('pki/roles/local', {'allowed_domains': ['localhost', 'notifications'],
|
||||
'allow_bare_domains': True, 'allow_ip_sans': True, 'max_ttl': '720h'})
|
||||
for name, host in [('notifications-tls', 'notifications'), ('frontend-tls', 'localhost')]:
|
||||
cert = self.kv(name)
|
||||
if cert is None or cert['expiration'] < time.time() + 7 * 86400:
|
||||
cert = self.call('pki/issue/local', {'common_name': host, 'ip_sans': '127.0.0.1', 'ttl': '720h'})['data']
|
||||
self.put(name, cert)
|
||||
if name == 'frontend-tls':
|
||||
write(STATE / 'frontend' / 'server.crt', cert['certificate'])
|
||||
write(STATE / 'frontend' / 'server.key', cert['private_key'])
|
||||
write(STATE / 'ca.pem', ca['ca'], 0o644)
|
||||
@@ -0,0 +1,14 @@
|
||||
vault { address = "http://vault:8200" }
|
||||
auto_auth {
|
||||
method "approle" {
|
||||
config = {
|
||||
role_id_file_path = "/etc/agent/role-id"
|
||||
secret_id_file_path = "/etc/agent/secret-id"
|
||||
remove_secret_id_file_after_reading = false
|
||||
}
|
||||
}
|
||||
}
|
||||
template_config {
|
||||
static_secret_render_interval = "10s"
|
||||
exit_on_retry_failure = true
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
ui = true
|
||||
disable_mlock = true
|
||||
storage "file" {
|
||||
path = "/vault/file"
|
||||
}
|
||||
listener "tcp" {
|
||||
address = "0.0.0.0:8200"
|
||||
tls_disable = true
|
||||
}
|
||||
api_addr = "http://vault:8200"
|
||||
@@ -0,0 +1,72 @@
|
||||
"""Integration checks against this project's running stack; never print secrets."""
|
||||
import json
|
||||
import ssl
|
||||
import urllib.request
|
||||
|
||||
from bootstrap import compose, mongo
|
||||
from vault import STATE
|
||||
|
||||
|
||||
def require(condition, message):
|
||||
if not condition:
|
||||
raise RuntimeError(message)
|
||||
|
||||
|
||||
def database_access(vault):
|
||||
for role, database, forbidden in [('bff', 'sxchnge', 'sxchnge_notifications'),
|
||||
('notifications', 'sxchnge_notifications', 'sxchnge')]:
|
||||
lease = vault.call('database/creds/' + role)
|
||||
credentials = lease['data']
|
||||
auth = 'db.getSiblingDB("admin").auth(' + json.dumps(credentials['username']) + ',' + json.dumps(credentials['password']) + ');'
|
||||
try:
|
||||
result = compose('exec', '-T', 'mongo', 'mongosh', '--quiet', '--file', '/dev/stdin', capture=True,
|
||||
input=auth + 'const own=db.getSiblingDB(' + json.dumps(database) + '); own.local_smoke.insertOne({_id:"vault-check"}); own.local_smoke.deleteOne({_id:"vault-check"});\n'
|
||||
'let denied=false; try { db.getSiblingDB(' + json.dumps(forbidden) + ').runCommand({find:"local_smoke",filter:{}}); } catch(e) { denied=e.code===13; } if(!denied) throw new Error("cross-database access allowed"); print("scope-ok");')
|
||||
require('scope-ok' in result, 'Mongo service scope check failed')
|
||||
renewed = vault.call('sys/leases/renew', {'lease_id': lease['lease_id'], 'increment': '1h'})
|
||||
require(renewed['lease_duration'] > 0, 'Mongo lease renewal failed')
|
||||
finally:
|
||||
vault.call('sys/leases/revoke', {'lease_id': lease['lease_id']})
|
||||
admin = vault.kv('bootstrap/mongo')['password']
|
||||
result = mongo('print(admin.getUser(' + json.dumps(credentials['username']) + ')===null)', admin)
|
||||
require('true' in result, 'Revoked Mongo user still exists')
|
||||
print('PASS: database-plugin issuance, separate database permissions, lease renewal and revocation')
|
||||
|
||||
|
||||
def policies(vault):
|
||||
for service in ['bff', 'notifications', 'nats']:
|
||||
root = STATE / 'agents' / service
|
||||
login = vault.call('auth/approle/login', {'role_id': (root / 'role-id').read_text(),
|
||||
'secret_id': (root / 'secret-id').read_text()})
|
||||
token = login['auth']['client_token']
|
||||
try:
|
||||
result = vault.call('sys/capabilities', {'token': token, 'paths': ['kv/data/bootstrap/mongo', 'sys/init']})['data']
|
||||
require(all(result[path] == ['deny'] for path in ['kv/data/bootstrap/mongo', 'sys/init']), 'Service can access bootstrap credentials')
|
||||
finally:
|
||||
vault.call('auth/token/revoke', {'token': token})
|
||||
print('PASS: service identities cannot read bootstrap credentials')
|
||||
|
||||
|
||||
def infrastructure(vault):
|
||||
status = compose('ps', '--format', 'json', capture=True)
|
||||
rows = [json.loads(line) for line in status.splitlines() if line.strip()]
|
||||
infra = {r['Service']: r for r in rows if r['Service'] in {'vault', 'mongo', 'nats'}}
|
||||
require(set(infra) == {'vault', 'mongo', 'nats'}, 'Expected Vault, Mongo and NATS')
|
||||
require(all(r['Health'] == 'healthy' for r in infra.values()), 'Infrastructure is unhealthy')
|
||||
print('PASS: Vault, Mongo replica set and NATS JetStream are healthy')
|
||||
policies(vault)
|
||||
database_access(vault)
|
||||
|
||||
|
||||
def verify(vault):
|
||||
status = compose('ps', '--format', 'json', capture=True)
|
||||
rows = [json.loads(line) for line in status.splitlines() if line.strip()]
|
||||
require({r['Service'] for r in rows} == {'vault', 'mongo', 'nats', 'notifications', 'bff', 'frontend'}, 'Expected six running services')
|
||||
require(all(r['Health'] == 'healthy' for r in rows), 'Some services are unhealthy')
|
||||
context = ssl.create_default_context(cafile=str(STATE / 'ca.pem'))
|
||||
for path in ['/', '/api/v1/health']:
|
||||
with urllib.request.urlopen('https://localhost:5126' + path, context=context, timeout=10) as response:
|
||||
require(response.status == 200, 'Frontend or BFF health failed')
|
||||
print('PASS: all six services healthy; frontend and proxied BFF use verified TLS')
|
||||
infrastructure(vault)
|
||||
print('Resend credentials are configured; no live email sent by this check')
|
||||
@@ -583,3 +583,55 @@ notification queue acceptance is not inbox delivery.
|
||||
|
||||
Delivered in [PR #41](https://git.sendico.io/tech/sxchnge/pulls/41) against `main`.
|
||||
Branch `twofactor-40` is pushed. The PR is left open for review; no merge was performed.
|
||||
|
||||
## Vault-backed local development (#42)
|
||||
|
||||
Task [#42](https://git.sendico.io/tech/sxchnge/issues/42), branch `dev-env-42`.
|
||||
Primary module: dev environment; secondary scope: container packaging, root build
|
||||
commands, current-stable backend test images and repository container policy.
|
||||
|
||||
- Added a dedicated `sxchnge-local` Compose project with persistent Vault, an
|
||||
authenticated MongoDB replica set, NATS JetStream and build definitions for
|
||||
notifications, BFF and Flutter/Nginx. Existing Sendico containers and volumes
|
||||
remain independent.
|
||||
- `make build` resolves current stable releases and pins manifest digests in
|
||||
ignored local state. Flutter is built from its stable source tag/commit. All
|
||||
protobuf bindings and frontend output are generated within the build, never
|
||||
committed. Root `AGENTS.md` now records the stable-container policy.
|
||||
- Bootstrap configures Vault KV/PKI/AppRole and `mongodb-database-plugin`, rotates
|
||||
its Mongo manager credential and issues separate database roles for the BFF
|
||||
and notifications. Vault Agent renews credentials and restarts child processes
|
||||
on changes. Existing shared database/messaging paths remain intact.
|
||||
- Resend is the default. The user explicitly chose to provision its API key and
|
||||
verified sender manually after bootstrap. `make up` leaves healthy infrastructure
|
||||
running and intentionally exits nonzero before product startup while those
|
||||
fields are empty. No placeholder credential or disabled driver was retained.
|
||||
- Frontend HTTPS and notification TLS use Vault PKI. The frontend proxies the BFF
|
||||
at the same origin and builds with the existing real auth adapter and a seeded
|
||||
public browser client. No UI or business service code changed.
|
||||
|
||||
Validation: `make build` built all product images; `make test` and
|
||||
`make lint-backend` passed. Backend tests pull fresh Mongo/NATS stable images and
|
||||
use `-count=1` to prevent Go's test cache from hiding an image update.
|
||||
`make test-dev-infra` passed its five setup regression tests and live checks for
|
||||
AppRole restrictions, database issuance, cross-database denial, renewal and
|
||||
revocation. `docker compose ... config --quiet` and `git diff --check` passed.
|
||||
The initial image build exposed missing protobuf headers; installing
|
||||
`protobuf-dev` in the build stage fixed it.
|
||||
|
||||
Additional live checks passed: a down/up cycle preserved a Mongo marker and Vault
|
||||
secrets and automatically unsealed Vault; changing/restoring the NATS token
|
||||
restarted the NATS child; a temporary 10-second/20-second BFF database role proved
|
||||
replacement credentials restart a healthy BFF (normal TTLs restored afterward).
|
||||
Temporary BFF/frontend containers served the web bundle and proxied BFF health
|
||||
successfully over HTTPS verified with the generated CA; those test containers
|
||||
were removed. The notifications image rendered its Agent configuration and exited
|
||||
with the expected missing email API key error. No real email was sent.
|
||||
|
||||
The final `make build && make up` is expected to exit at the Resend provisioning
|
||||
check until the user populates `kv/email`. Full six-service readiness and a live
|
||||
Resend signup/login were not asserted without that credential. Vault, Mongo and
|
||||
NATS remain running. No optional security scan or merge was performed. See the
|
||||
[local development guide](local-development.md) for exact provisioning fields,
|
||||
certificate trust, restart behavior, local bootstrap-material handling and
|
||||
verification commands.
|
||||
@@ -0,0 +1,182 @@
|
||||
# Local development
|
||||
|
||||
Task [#42](https://git.sendico.io/tech/sxchnge/issues/42), branch `dev-env-42`.
|
||||
|
||||
## Build and bootstrap
|
||||
|
||||
Requires Docker with Compose v2 and BuildKit, Python 3.10+ and Git. Docker builds
|
||||
Go, protobuf bindings and Flutter web; host Go, protoc and Flutter installations
|
||||
are not required for `make build && make up`. The first Flutter SDK download can
|
||||
take several minutes. Both ARM64 and AMD64 build from the same source definitions.
|
||||
|
||||
```sh
|
||||
make build && make up
|
||||
```
|
||||
|
||||
`make build` resolves the current stable upstream releases, pulls infrastructure
|
||||
and builds NATS packaging, notifications, BFF and frontend images. Vault and Mongo
|
||||
use upstream images directly. Resolution excludes prereleases and selects Nginx's
|
||||
stable channel. The exact versions and manifest digests are saved in the ignored
|
||||
`.local/dev/images.env`; Flutter's stable release is pinned by Git commit. Build
|
||||
fails if upstream metadata cannot be resolved. Run `make build` again to refresh
|
||||
versions. Review database release upgrade requirements before updating a populated
|
||||
local volume across major versions; the tooling does not perform data migrations.
|
||||
|
||||
`make up` starts persistent Vault, initializes/unseals it, provisions secrets and
|
||||
AppRoles, initializes an authenticated single-member MongoDB replica set, configures
|
||||
Vault's MongoDB database plugin, and starts authenticated NATS JetStream. It then
|
||||
starts notifications, BFF and the frontend when the required email settings exist.
|
||||
|
||||
**On the first run, `make up` intentionally exits with an error after infrastructure
|
||||
bootstrap because Resend credentials have not been supplied.** No placeholder API
|
||||
key, mock delivery mode or embedded credential is installed. The three infrastructure
|
||||
services remain running so you can configure Vault, then rerun `make up`.
|
||||
|
||||
## Configure Resend in Vault
|
||||
|
||||
The default email driver is `resend`. In the Vault UI at
|
||||
[http://127.0.0.1:18200](http://127.0.0.1:18200), edit the KV v2 secret **`kv/email`**
|
||||
(API path `kv/data/email`):
|
||||
|
||||
| Field | Value |
|
||||
| --- | --- |
|
||||
| `driver` | `resend` |
|
||||
| `api_key` | Your Resend API key |
|
||||
| `sender_address` | Your verified Resend sender email |
|
||||
| `sender_name` | `SXCHNGE` or your desired display name |
|
||||
|
||||
Local administrative login material is in the private, ignored
|
||||
`.local/dev/admin/init.json` (`root_token`). Keep that file private. Product
|
||||
containers never mount it. You can also provision the same secret with a hidden
|
||||
terminal prompt, without placing the API key in shell history:
|
||||
|
||||
```sh
|
||||
python3 dev/local.py email
|
||||
make up
|
||||
```
|
||||
|
||||
The startup check rejects an empty key or the bootstrap sender placeholder. It
|
||||
does not validate the key against Resend or prove domain verification; an invalid
|
||||
key fails at delivery. Vault Agent fetches the key and email settings and injects
|
||||
them into the notifications process. There is no `.env` secret fallback. Changing
|
||||
`kv/email` restarts the notification process automatically within about 10 seconds.
|
||||
Bootstrapping again preserves your existing email secret.
|
||||
|
||||
## Open the product
|
||||
|
||||
After providing Resend credentials:
|
||||
|
||||
- Frontend and proxied API: [https://localhost:5126](https://localhost:5126).
|
||||
- BFF health: [https://localhost:5126/api/v1/health](https://localhost:5126/api/v1/health).
|
||||
- Vault UI/API: [http://127.0.0.1:18200](http://127.0.0.1:18200).
|
||||
|
||||
Trust `.local/dev/ca.pem` in your local browser/OS certificate store to use HTTPS
|
||||
without a certificate error. Trust installation is a separate local action, not
|
||||
performed by the bootstrap. For a command-line check without changing trust:
|
||||
|
||||
```sh
|
||||
curl --cacert .local/dev/ca.pem https://localhost:5126/api/v1/health
|
||||
```
|
||||
|
||||
HTTPS keeps the existing Secure/HttpOnly refresh cookie contract. Nginx serves
|
||||
Flutter and proxies `/api/` to the BFF on the same origin. The bundle selects the
|
||||
existing backend auth adapter with the public client ID `sxchnge-local-web`.
|
||||
Bootstrap inserts this public browser client if absent; it has no client secret.
|
||||
No frontend widget, service or provider architecture changes are needed.
|
||||
|
||||
The rest of the app retains its existing capabilities and limitations: wallet,
|
||||
KYC and settings fixtures are not turned into production integrations by this
|
||||
container setup. Full email signup/login requires a working Resend account.
|
||||
|
||||
## Secret and connection lifecycle
|
||||
|
||||
| Consumer | Vault access |
|
||||
| --- | --- |
|
||||
| BFF | `database/creds/bff`, JWT key, notification caller token and CA |
|
||||
| Notifications | `database/creds/notifications`, NATS token, caller token, TLS certificate/key and email settings |
|
||||
| NATS | NATS authentication token only |
|
||||
| Bootstrap | Infrastructure administration, PKI and initial identities |
|
||||
|
||||
The database engine is configured at `database/config/mongo` with
|
||||
`mongodb-database-plugin`. It uses its own `vault-manager` Mongo user, whose
|
||||
password is rotated by Vault immediately after configuration. The manager has
|
||||
`userAdminAnyDatabase`; applications never receive that credential. The independent
|
||||
local Mongo administrator and replica-set key come from Vault's random generator
|
||||
and are retained in `kv/bootstrap/mongo` for local provisioning.
|
||||
|
||||
The BFF role has `readWrite` only on `sxchnge`; notifications has `readWrite` only
|
||||
on `sxchnge_notifications`. Mongo URIs come from dynamic Vault credentials, not
|
||||
static application users. Leases default to one hour with a 24-hour maximum.
|
||||
Vault Agent renews leases, obtains replacement credentials, and restarts its child
|
||||
process when credentials change. Apps retain their existing shared database and
|
||||
messaging connection wrappers. No new direct application database path is added.
|
||||
|
||||
Each agent authenticates using its own AppRole with a renewable periodic token.
|
||||
Local SecretIDs are reusable for container restarts; each container mounts only
|
||||
its own identity directory under `.local/dev/agents/`. Revoking an AppRole SecretID
|
||||
requires removing its matching local `secret-id` file and rerunning `make up` to
|
||||
issue a replacement. Revoking an active token/lease terminates its credentials;
|
||||
Agent's retries and the container restart policy recover when Vault is reachable.
|
||||
|
||||
The notification gRPC connection uses a Vault-issued certificate for
|
||||
`notifications` and the BFF validates the Vault CA and hostname. Frontend HTTPS
|
||||
certificates also come from Vault PKI. Leaf certificates last 30 days; `make up`
|
||||
refreshes them when less than seven days remain and reloads Nginx. Run it regularly
|
||||
for a development stack kept online for weeks. Agents restart when their TLS KV
|
||||
secret changes. Application secrets are never put into the frontend bundle.
|
||||
|
||||
Vault, Mongo and JetStream data use project-scoped named Docker volumes. Local
|
||||
Vault unseal material, administrator login, per-service bootstrap identities and
|
||||
frontend TLS material are under ignored `.local/dev/`, outside image build contexts.
|
||||
The local Vault automatically unseals using its local key on container restart.
|
||||
These convenience credentials and loopback HTTP Vault access are for this local
|
||||
machine; this is not a production Vault deployment. Do not share `.local/dev/`.
|
||||
Keep it paired with its matching Docker volumes for recovery.
|
||||
|
||||
Mongo and NATS expose no host ports. The `sxchnge-local` network, volumes and
|
||||
container names are independent of existing Sendico stacks. Only the frontend
|
||||
and Vault bind host loopback ports. No Sendico database, message bus or secrets
|
||||
are consumed.
|
||||
|
||||
## Manage and verify
|
||||
|
||||
```sh
|
||||
make ps
|
||||
make logs
|
||||
make down # preserves named volumes and local bootstrap material
|
||||
make up # idempotent provisioning and readiness checks
|
||||
make test-dev-infra # works before Resend provisioning
|
||||
make test-dev # requires all six services running
|
||||
```
|
||||
|
||||
The infrastructure check exercises AppRole access restrictions, real Vault
|
||||
MongoDB credential issuance, per-database access, lease renewal and user deletion
|
||||
on revocation. The complete check also verifies service health and frontend/API
|
||||
HTTPS with the generated CA. These checks never send a real email.
|
||||
|
||||
Backend regression tests use fresh `mongo:latest` and `nats:latest` official
|
||||
stable images, pulled before `make test-backend`. Normal validation remains
|
||||
`make test` and `make lint-backend` and requires the host SDKs described in the
|
||||
repository guides.
|
||||
|
||||
Deleting Docker volumes removes local data; `make down` deliberately keeps them.
|
||||
If starting from scratch, stop the stack and explicitly remove this project's
|
||||
three volumes **and** its matching `.local/dev/` bootstrap state together. Never
|
||||
reuse old unseal material with a newly initialized Vault volume, or retain Mongo
|
||||
users while discarding the Vault data that manages their leases.
|
||||
|
||||
## Sources and checked releases
|
||||
|
||||
On 2026-09-12 the resolver selected Vault 2.1.0, MongoDB 8.3.9, NATS 2.14.6,
|
||||
Go 1.27.1, Nginx stable 1.30.4, Ubuntu 26.04 and Flutter stable 3.47.4. These are
|
||||
verification records, not fixed future defaults. Release selection follows the
|
||||
[Vault releases](https://releases.hashicorp.com/vault/),
|
||||
[MongoDB stable release notes](https://www.mongodb.com/docs/manual/release-notes/),
|
||||
[NATS releases](https://github.com/nats-io/nats-server/releases/latest),
|
||||
[Nginx downloads](https://nginx.org/en/download.html),
|
||||
[Go releases](https://go.dev/dl/) and
|
||||
[Flutter stable channel](https://github.com/flutter/flutter/tree/stable).
|
||||
The implementation follows HashiCorp's
|
||||
[MongoDB database plugin](https://developer.hashicorp.com/vault/docs/secrets/databases/mongodb)
|
||||
and [Agent process supervisor](https://developer.hashicorp.com/vault/docs/agent-and-proxy/agent/process-supervisor)
|
||||
contracts.
|
||||
Reference in new issue
Block a user