Run local infrastructure and product images with Vault-managed secrets #43
No files matched your search
@@ -181,3 +181,5 @@ belong to the receiving service at `services/<service>/callers/<caller>`.
|
|||||||
Use Caddy for local browser HTTPS and certificate management. Treat the private
|
Use Caddy for local browser HTTPS and certificate management. Treat the private
|
||||||
local Docker network as trusted; do not add internal gRPC TLS or Vault-managed
|
local Docker network as trusted; do not add internal gRPC TLS or Vault-managed
|
||||||
local TLS certificates without an explicit requirement.
|
local TLS certificates without an explicit requirement.
|
||||||
|
Store only secret values in Vault. Driver selection, sender identities, endpoints
|
||||||
|
and other non-secret runtime settings belong in service configuration files.
|
||||||
@@ -40,7 +40,8 @@ make build && make up
|
|||||||
|
|
||||||
This builds the product images and starts an isolated Vault, authenticated MongoDB
|
This builds the product images and starts an isolated Vault, authenticated MongoDB
|
||||||
replica set and NATS JetStream. First startup intentionally stops after bootstrap
|
replica set and NATS JetStream. First startup intentionally stops after bootstrap
|
||||||
until you set the Resend API key and verified sender in Vault's `kv/services/notifications/email/resend` secret.
|
until you set `api_key` in Vault's `kv/services/notifications/email/resend` secret
|
||||||
|
and the verified sender in `infra/local/notifications/runtime.yml`.
|
||||||
Then rerun `make up` to start notifications, BFF and the frontend at
|
Then rerun `make up` to start notifications, BFF and the frontend at
|
||||||
[https://localhost:5126](https://localhost:5126).
|
[https://localhost:5126](https://localhost:5126).
|
||||||
|
|
||||||
|
|||||||
@@ -751,3 +751,18 @@ checks read the new path; application identities are denied both paths. Shell
|
|||||||
checks and live infrastructure validation (including Mongo transactions, lease
|
checks and live infrastructure validation (including Mongo transactions, lease
|
||||||
renewal/revocation and obsolete-path absence) passed. `make up` still stops at the
|
renewal/revocation and obsolete-path absence) passed. `make up` still stops at the
|
||||||
expected missing-Resend check. No backend code or product behavior changed.
|
expected missing-Resend check. No backend code or product behavior changed.
|
||||||
|
|
||||||
|
Email configuration is now separated from credentials. The current Vault secret
|
||||||
|
`kv/services/notifications/email/resend` contains only `api_key`; `driver`,
|
||||||
|
`sender_address` and `sender_name` live in
|
||||||
|
`infra/local/notifications/runtime.yml`. Sender address is empty until configured.
|
||||||
|
Agent injects only `EMAIL_API_KEY`, and notifications loads its runtime YAML
|
||||||
|
directly. `make email` prompts only for the API key. The migration preserves the
|
||||||
|
key with a CAS-checked update, archives the prior readable version, and refuses
|
||||||
|
unknown fields rather than discarding them. The live Vault entry was normalized.
|
||||||
|
|
||||||
|
Validation passed: image build, shell/migration tests, live infrastructure checks
|
||||||
|
(including key-only secret shape and Mongo transactions/leases), and diff checks.
|
||||||
|
`make up` intentionally fails for the still-missing API key. Verified sender must
|
||||||
|
also be configured in runtime YAML before rebuilding and starting notifications.
|
||||||
|
No backend source changed and no email was sent.
|
||||||
@@ -40,7 +40,7 @@ kv/
|
|||||||
│ ├── bff/jwt # jwt
|
│ ├── bff/jwt # jwt
|
||||||
│ └── notifications/
|
│ └── notifications/
|
||||||
│ ├── callers/bff # token; read by BFF and notifications
|
│ ├── callers/bff # token; read by BFF and notifications
|
||||||
│ └── email/resend # driver, api_key, sender_address, sender_name
|
│ └── email/resend # api_key only
|
||||||
└── infrastructure/
|
└── infrastructure/
|
||||||
├── nats/auth # token; read by NATS and notifications
|
├── nats/auth # token; read by NATS and notifications
|
||||||
└── mongo/bootstrap # local administrator and replica-set bootstrap material
|
└── mongo/bootstrap # local administrator and replica-set bootstrap material
|
||||||
@@ -65,22 +65,37 @@ Open [Vault](http://127.0.0.1:18200) and edit
|
|||||||
|
|
||||||
| Field | Value |
|
| Field | Value |
|
||||||
| --- | --- |
|
| --- | --- |
|
||||||
| `driver` | `resend` |
|
|
||||||
| `api_key` | Your Resend API key |
|
| `api_key` | Your Resend API key |
|
||||||
| `sender_address` | Your verified sender address |
|
|
||||||
| `sender_name` | Your display name |
|
The driver and sender settings belong in `infra/local/notifications/runtime.yml`:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
email:
|
||||||
|
driver: resend
|
||||||
|
settings:
|
||||||
|
api_key_env: EMAIL_API_KEY
|
||||||
|
sender_address: "your-verified-sender@example.com"
|
||||||
|
sender_name: "SXCHNGE"
|
||||||
|
```
|
||||||
|
|
||||||
|
Set your verified address in that file and rebuild. It is intentionally empty by
|
||||||
|
default, so notifications cannot start with an unconfigured sender. Vault Agent
|
||||||
|
injects only the API key; it does not generate email configuration.
|
||||||
|
|
||||||
Local administrator login material is in private `.local/dev/admin/init.json`.
|
Local administrator login material is in private `.local/dev/admin/init.json`.
|
||||||
Product containers cannot access it. Alternatively use the hidden terminal prompt:
|
Product containers cannot access it. Alternatively use the hidden terminal prompt:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
make email
|
make email
|
||||||
make up
|
make build && make up
|
||||||
```
|
```
|
||||||
|
|
||||||
No placeholder key or delivery fallback is installed. Startup validates presence;
|
No placeholder key or delivery fallback is installed. Startup validates presence;
|
||||||
it does not send an email or validate domain ownership with Resend. Updating email
|
it does not send an email or validate domain ownership with Resend. Updating the
|
||||||
settings causes Agent to restart notifications. Existing values survive bootstrap.
|
API key causes Agent to restart notifications. Driver/sender changes require an
|
||||||
|
image rebuild. Existing API keys survive bootstrap. Migration removes legacy
|
||||||
|
driver/sender fields from the current Vault secret with a version-checked write;
|
||||||
|
the preceding version is preserved in Vault history and the private archive.
|
||||||
|
|
||||||
## Browser HTTPS and internal connections
|
## Browser HTTPS and internal connections
|
||||||
|
|
||||||
|
|||||||
@@ -19,6 +19,12 @@ messaging:
|
|||||||
templates: "/etc/sxchnge/templates.json"
|
templates: "/etc/sxchnge/templates.json"
|
||||||
callers:
|
callers:
|
||||||
product: "SXCHNGE_NOTIFICATIONS_PRODUCT_TOKEN"
|
product: "SXCHNGE_NOTIFICATIONS_PRODUCT_TOKEN"
|
||||||
|
email:
|
||||||
|
driver: resend
|
||||||
|
settings:
|
||||||
|
api_key_env: EMAIL_API_KEY
|
||||||
|
sender_address: "" # Set your verified Resend sender here, then run make build.
|
||||||
|
sender_name: "SXCHNGE"
|
||||||
# Use replicas: 3 and environment: production for a three-node NATS cluster.
|
# Use replicas: 3 and environment: production for a three-node NATS cluster.
|
||||||
# mode: ops validates resources without modifying them; app provisions them.
|
# mode: ops validates resources without modifying them; app provisions them.
|
||||||
queue:
|
queue:
|
||||||
|
|||||||
@@ -98,20 +98,19 @@ up() {
|
|||||||
random | jq -Rs --arg field "$field" '{($field):rtrimstr("\n")}' | put "$path"
|
random | jq -Rs --arg field "$field" '{($field):rtrimstr("\n")}' | put "$path"
|
||||||
fi
|
fi
|
||||||
done
|
done
|
||||||
printf '{"driver":"resend","api_key":"","sender_address":"notifications@localhost.test","sender_name":"SXCHNGE local"}' | ensure services/notifications/email/resend
|
printf '{"api_key":""}' | ensure services/notifications/email/resend
|
||||||
provision_mongo
|
provision_mongo
|
||||||
provision_agents
|
provision_agents
|
||||||
compose up -d --wait --wait-timeout 120 nats
|
compose up -d --wait --wait-timeout 120 nats
|
||||||
retire_secrets
|
retire_secrets
|
||||||
kv services/notifications/email/resend | email_valid || fail 'Vault kv/services/notifications/email/resend needs a Resend api_key and verified sender_address. Run make email, then make up.'
|
normalize_email_secret
|
||||||
|
kv services/notifications/email/resend | email_valid || fail 'Vault kv/services/notifications/email/resend needs api_key. Run make email; configure the verified sender in infra/local/notifications/runtime.yml, then make build && make up.'
|
||||||
compose up -d --wait --wait-timeout 180
|
compose up -d --wait --wait-timeout 180
|
||||||
export_ca
|
export_ca
|
||||||
echo 'Local stack ready: https://localhost:5126; Vault: http://127.0.0.1:18200'
|
echo 'Local stack ready: https://localhost:5126; Vault: http://127.0.0.1:18200'
|
||||||
}
|
}
|
||||||
email() {
|
email() {
|
||||||
existing
|
existing
|
||||||
printf 'Verified Resend sender address: ' >&2
|
|
||||||
IFS= read -r address
|
|
||||||
printf 'Resend API key (hidden): ' >&2
|
printf 'Resend API key (hidden): ' >&2
|
||||||
tty_state=$(stty -g)
|
tty_state=$(stty -g)
|
||||||
trap 'stty "$tty_state"; cleanup' EXIT
|
trap 'stty "$tty_state"; cleanup' EXIT
|
||||||
@@ -120,9 +119,8 @@ email() {
|
|||||||
stty "$tty_state"
|
stty "$tty_state"
|
||||||
printf '\n' >&2
|
printf '\n' >&2
|
||||||
[ -n "$key" ] || fail 'A key is required'
|
[ -n "$key" ] || fail 'A key is required'
|
||||||
case "$address" in *@*) ;; *) fail 'A sender email address is required';; esac
|
|
||||||
printf '%s' "$key" > "$WORK/email-key"
|
printf '%s' "$key" > "$WORK/email-key"
|
||||||
printf '%s' "$address" > "$WORK/email-address"
|
normalize_email_secret
|
||||||
jq -n --rawfile key "$WORK/email-key" --rawfile address "$WORK/email-address" '{driver:"resend",api_key:$key,sender_address:$address,sender_name:"SXCHNGE"}' | put services/notifications/email/resend
|
jq -n --rawfile key "$WORK/email-key" '{api_key:$key}' | put services/notifications/email/resend
|
||||||
echo 'Email configuration saved in Vault. Run make up.'
|
echo 'API key saved in Vault. Set the verified sender in infra/local/notifications/runtime.yml, then make build && make up.'
|
||||||
}
|
}
|
||||||
@@ -57,7 +57,7 @@ ensure() (
|
|||||||
# Never forward mongosh errors: authentication/command errors may include secrets.
|
# Never forward mongosh errors: authentication/command errors may include secrets.
|
||||||
mongo() { compose exec -T mongo mongosh --quiet --file /dev/stdin > "$WORK/mongo-result" 2> "$WORK/mongo-error" || return 1; cat "$WORK/mongo-result"; }
|
mongo() { compose exec -T mongo mongosh --quiet --file /dev/stdin > "$WORK/mongo-result" 2> "$WORK/mongo-error" || return 1; cat "$WORK/mongo-result"; }
|
||||||
admin_auth() { kv infrastructure/mongo/bootstrap | jq -r '"const admin=db.getSiblingDB(\"admin\"); admin.auth(\"local-admin\","+(.password|tojson)+");"'; }
|
admin_auth() { kv infrastructure/mongo/bootstrap | jq -r '"const admin=db.getSiblingDB(\"admin\"); admin.auth(\"local-admin\","+(.password|tojson)+");"'; }
|
||||||
email_valid() { jq -e '(.api_key|type=="string") and (.api_key|test("\\S")) and (.sender_address|type=="string") and (.sender_address|test("\\S")) and .sender_address!="notifications@localhost.test"' >/dev/null; }
|
email_valid() { jq -e '(.api_key|type=="string") and (.api_key|test("\\S"))' >/dev/null; }
|
||||||
export_ca() {
|
export_ca() {
|
||||||
compose exec -T frontend cat /data/caddy/pki/authorities/local/root.crt | write "$STATE/ca.pem" 644
|
compose exec -T frontend cat /data/caddy/pki/authorities/local/root.crt | write "$STATE/ca.pem" 644
|
||||||
}
|
}
|
||||||
@@ -55,3 +55,16 @@ retire_secrets() (
|
|||||||
# and history are retained until the owner explicitly chooses to destroy them.
|
# and history are retained until the owner explicitly chooses to destroy them.
|
||||||
echo 'PASS: Vault secrets migrated and verified; obsolete KV entries retired'
|
echo 'PASS: Vault secrets migrated and verified; obsolete KV entries retired'
|
||||||
)
|
)
|
||||||
|
|
||||||
|
# Only credentials belong in Vault; driver/sender settings are runtime YAML.
|
||||||
|
normalize_email_secret() (
|
||||||
|
path=services/notifications/email/resend
|
||||||
|
vault "kv/data/$path" GET yes > "$WORK/email-current"
|
||||||
|
[ "$(cat "$WORK/email-current")" != null ] || exit 0
|
||||||
|
jq -e '(.data.data|keys)-["api_key","driver","sender_address","sender_name"]|length==0' "$WORK/email-current" >/dev/null || fail 'Unexpected Resend secret fields; refusing to discard them'
|
||||||
|
if jq -e '.data.data|keys==["api_key"]' "$WORK/email-current" >/dev/null; then exit 0; fi
|
||||||
|
archive_secret "$path"
|
||||||
|
jq '{data:{api_key:(.data.data.api_key // "")},options:{cas:.data.metadata.version}}' "$WORK/email-current" | vault "kv/data/$path" POST >/dev/null
|
||||||
|
vault "kv/data/$path" | jq -e --slurpfile original "$WORK/email-current" '.data.data|keys==["api_key"] and .api_key==($original[0].data.data.api_key // "")' >/dev/null || fail 'Resend migration verification failed'
|
||||||
|
echo 'Resend secret now contains only api_key; configure driver/sender in notifications runtime.yml'
|
||||||
|
)
|
||||||
@@ -2,9 +2,6 @@
|
|||||||
set -eu
|
set -eu
|
||||||
umask 077
|
umask 077
|
||||||
if [ "$SXCHNGE_SERVICE" = notifications ]; then
|
if [ "$SXCHNGE_SERVICE" = notifications ]; then
|
||||||
printf '%s\n' "$EMAIL_CONFIG" > /run/sxchnge/email.yml
|
exec service -config.file=/etc/sxchnge/notifications.yml
|
||||||
unset EMAIL_CONFIG
|
|
||||||
cat /etc/sxchnge/notifications.yml /run/sxchnge/email.yml > /run/sxchnge/config.yml
|
|
||||||
exec service -config.file=/run/sxchnge/config.yml
|
|
||||||
fi
|
fi
|
||||||
exec service -config.file=/etc/sxchnge/bff.yml
|
exec service -config.file=/etc/sxchnge/bff.yml
|
||||||
@@ -59,3 +59,18 @@ printf '{"api_key":"different-fixture"}' > "$WORK/destination"
|
|||||||
if (migrate_secrets) >/dev/null 2>&1; then fail 'Migration overwrote conflicting destination'; fi
|
if (migrate_secrets) >/dev/null 2>&1; then fail 'Migration overwrote conflicting destination'; fi
|
||||||
jq -e '.api_key=="different-fixture"' "$WORK/destination" >/dev/null
|
jq -e '.api_key=="different-fixture"' "$WORK/destination" >/dev/null
|
||||||
echo 'PASS: secret migration is repeatable and refuses conflicting destination values'
|
echo 'PASS: secret migration is repeatable and refuses conflicting destination values'
|
||||||
|
# Removing non-secret fields retains the key and checks the current KV version.
|
||||||
|
vault() {
|
||||||
|
if [ "${2:-GET}" = GET ]; then cat "$WORK/email-fixture"; else
|
||||||
|
jq -e '.options.cas==42 and .data=={api_key:"preserved-fixture-key"}' > /dev/null || return 1
|
||||||
|
printf '{"data":{"metadata":{"version":43},"data":{"api_key":"preserved-fixture-key"}}}' > "$WORK/email-fixture"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
printf '{"data":{"metadata":{"version":42},"data":{"api_key":"preserved-fixture-key","driver":"resend","sender_address":"sender@example.test","sender_name":"Fixture"}}}' > "$WORK/email-fixture"
|
||||||
|
normalize_email_secret
|
||||||
|
normalize_email_secret
|
||||||
|
jq -e '.data.data=={api_key:"preserved-fixture-key"}' "$WORK/email-fixture" >/dev/null
|
||||||
|
printf '{"data":{"metadata":{"version":43},"data":{"api_key":"preserved-fixture-key","unexpected":"must-not-drop"}}}' > "$WORK/email-fixture"
|
||||||
|
if (normalize_email_secret) >/dev/null 2>&1; then fail 'Unknown secret field was silently dropped'; fi
|
||||||
|
if grep -q EMAIL_CONFIG infra/local/vault/notifications.hcl infra/local/scripts/start-service.sh; then fail 'Email configuration is still rendered from Vault'; fi
|
||||||
|
echo 'PASS: email migration preserves API key, removes configuration fields and rejects unknown fields'
|
||||||
@@ -50,6 +50,7 @@ product() {
|
|||||||
echo 'PASS: six healthy services, frontend and proxied BFF use verified TLS; no email sent'
|
echo 'PASS: six healthy services, frontend and proxied BFF use verified TLS; no email sent'
|
||||||
}
|
}
|
||||||
layout() {
|
layout() {
|
||||||
|
kv services/notifications/email/resend | jq -e 'keys==["api_key"]' >/dev/null || fail 'Non-secret email configuration remains in Vault'
|
||||||
for old in bootstrap/mongo bff caller email nats tls frontend-tls notifications-tls; do
|
for old in bootstrap/mongo bff caller email nats tls frontend-tls notifications-tls; do
|
||||||
[ "$(vault "kv/metadata/$old" GET yes)" = null ] || fail "Obsolete KV entry remains: $old"
|
[ "$(vault "kv/metadata/$old" GET yes)" = null ] || fail "Obsolete KV entry remains: $old"
|
||||||
done
|
done
|
||||||
|
|||||||
@@ -31,10 +31,6 @@ env_template "EMAIL_API_KEY" {
|
|||||||
error_on_missing_key = true
|
error_on_missing_key = true
|
||||||
}
|
}
|
||||||
|
|
||||||
env_template "EMAIL_CONFIG" {
|
|
||||||
contents = "{{ with secret \"kv/data/services/notifications/email/resend\" }}email:\n driver: {{ .Data.data.driver | toJSON }}\n settings:\n api_key_env: EMAIL_API_KEY\n sender_address: {{ .Data.data.sender_address | toJSON }}\n sender_name: {{ .Data.data.sender_name | toJSON }}\n{{ end }}"
|
|
||||||
error_on_missing_key = true
|
|
||||||
}
|
|
||||||
|
|
||||||
exec {
|
exec {
|
||||||
command = ["/usr/local/bin/start-service"]
|
command = ["/usr/local/bin/start-service"]
|
||||||
|
|||||||
Reference in new issue
Block a user