# Notifications backend Independent internal gRPC service with durable asynchronous email delivery. The public contract belongs to [interface/proto/v1/interface.proto](interface/proto/v1/interface.proto). Generated clients are build artifacts, not committed source. ## Architecture The gRPC transport maps protobuf DTOs to domain requests. The service validates and renders immutable template revisions, then saves a notification and its pending-work state atomically in one document through the shared DB wrapper. The outbox dispatcher publishes opaque IDs using shared messaging's durable transport, adapted from Sendico. A durable NATS worker claims the notification with an atomic conditional patch and saves the outcome before acknowledging. `DriverConfig` selects the email driver: `sendgrid`, `resend` or `smtp`. Email adapters only map rendered email models to provider wire payloads. SendGrid uses adapted Sendico SDK construction; Resend uses its email HTTP API. Both support subject, text and HTML, bounded timeouts and safe error classification. The owned SMTP adapter uses authenticated implicit TLS (normally port 465), verified server certificates and context-bounded connections. It sends MIME multipart/alternative with quoted-printable UTF-8 text/HTML and encoded subjects. It uses Go's standard SMTP client and does not import Sendico modules or share a mutable connection between workers. Configure it with: ```yaml email: driver: smtp settings: host: mail.example.com port: 465 username: notifications@example.com password_env: SMTP_PASSWORD sender_address: notifications@example.com sender_name: SXCHNGE network_timeout: 10 ``` `network_timeout` must be 1–20 seconds; the worker's expiry/shutdown context also bounds every operation. Only the password belongs in Vault. There is no plaintext fallback or certificate-verification bypass. ## Run Requires Go 1.27, protoc, an SXCHNGE MongoDB database and a NATS server with JetStream enabled. Copy `config.example.yml` to an untracked `config.yml` here. Set `SXCHNGE_MONGO_URI`, `SXCHNGE_NATS_URL` and the selected email provider's API key environment variable. Configure a verified sender address and template catalog. Paths are relative to the process working directory. From the root, run `make run-notifications`. The gRPC listener uses plaintext on the private local network. Configure both `tls_cert` and `tls_key` only when TLS is explicitly required. Use a private authenticated NATS endpoint; `tls://` URLs and credentials in the URL are supported by the shared driver. Mongo and messaging both use shared `DriverConfig` settings; email settings reject unknown keys. The queue uses `SXCHNGE_NOTIFICATIONS`, subject `sxchnge.notifications.email`, and durable consumer `email`. Configure `queue.stream.replicas` as 1, 3 or 5 to match the NATS cluster, and `queue.stream.max_age_hours` for retention (48 hours minimum). File storage and the owned stream/subject names are enforced. `queue.provisioning.environment` accepts `production` or `non-production`; `mode: ops` validates existing resources without changing them, `mode: app` provisions missing resources, and `mode: hybrid` provisions only outside production. Production retention and consumer-setting drift fail startup instead of being silently accepted. Provisioning operations honor their startup context. The example defaults to one replica and app provisioning for local development. Multiple service workers can safely compete for jobs through atomic claims. ## Templates and parameters See `templates.example.json`, including the `auth.activation` and `auth.login` English/Russian templates used by [product authentication](../../docs/auth-notifications.md). Parameters use Go template map syntax such as `{{.first_name}}`. Each revision declares the exact allowed parameter names and locale-specific content. Missing/extra parameters, unknown revisions/locales and invalid content are rejected before acceptance. Empty locale uses the default. Values are data, not executable templates; HTML is escaped by `html/template`. The catalog is loaded and compiled at startup. Never edit a published revision; add a new version. Rendered payloads are persisted so retries do not depend on a later catalog change. Recipient, body and subject are cleared on terminal state; remaining status and request fingerprints expire through a seven-day TTL. ## Delivery semantics - Send persists before returning; it does not wait for NATS or the email provider. - Outbox scans every five seconds and republishes safely after a bus failure. Pending notifications expire and have their rendered content cleared even when the bus is unavailable or a retry is scheduled later. - Duplicate RPCs and duplicate broker deliveries do not create extra jobs/sends. - Provider HTTP 2xx becomes ACCEPTED. No receipt/webhook or read tracking is claimed. - Explicit rate limiting (429) retries with bounded exponential backoff and Retry-After, at most five provider attempts, bounded by message expiry. - Transport errors, 408, 5xx and expired worker leases become UNKNOWN. These may have sent the email; operators must reconcile before requesting another send. - Other provider failures are terminal. `provider_authentication_failed` signals credentials/permissions, `provider_rejected` a nonretryable rejection, `retry_exhausted` the attempt limit, and `expired` the message deadline. - Resend receives the notification ID as its provider idempotency key. Its window is 24 hours. SendGrid has no equivalent guarantee; exactly-once external email delivery is not promised. - SMTP acceptance requires the final DATA 250 reply. A later QUIT failure cannot undo acceptance. Explicit SMTP 4xx replies become retryable `provider_temporarily_unavailable`; 5xx replies are terminal (authentication failures retain `provider_authentication_failed`). Transport/TLS errors and a lost final DATA reply become UNKNOWN. No immediate reconnect/resend occurs. A stable Message-ID is included for correlation, not an exactly-once promise. - Changing the configured driver fails queued jobs pinned to the old driver with `driver_changed`. Drain queued jobs before changing sender credentials/config. - Callers authorize recipients and preferences. No direct customer-database access, bulk campaigns, provider failover, SMS, push or Telegram delivery is included. Structured logs report notification IDs, driver, state and attempt count. Provider HTTP outcomes include `provider_status`; failures also include `provider_error` and `provider_message` when available. The adapter reads at most 16 KiB plus an overflow byte and extracts only the Resend `name`/`message` or first SendGrid `errors[].message`. Diagnostic messages are bounded to 512 Unicode characters plus an ellipsis. Credentials, idempotency keys, outgoing request string values, email addresses, URLs, long token-like strings and numeric verification/account values embedded in the request are redacted; controls and formatting characters are normalized. Malformed, incomplete and oversized error responses produce a diagnostic fallback instead of raw content. Successful response bodies are not logged. Diagnostic fields are transient: they are neither persisted in notifications nor exposed in gRPC responses. The existing bounded failure codes and delivery/retry decisions remain unchanged. Transport failures include a sanitized cause. Provider outcome logging happens before the result write so a storage failure cannot discard the only provider diagnostic. Message content, template values, addresses and raw provider bodies must remain excluded from logs. Provider text is sanitized external diagnostic data, never an instruction. Use `provider_status`, `provider_error` and `provider_message` together: the generic `provider_authentication_failed` category combines HTTP 401 and 403, including sender/domain authorization failures. A sending-only Resend key is valid for sending, but cannot read the provider's log API. SMTP outcomes use `provider_smtp_status` for protocol reply codes and the same sanitized provider error/message fields, including password/auth-payload redaction. Monitor pending age, UNKNOWN/FAILED states and repeated outbox/worker warnings. ## Validate and distribute `make test` and `make lint-backend` generate interfaces and validate all modules. Notifications tests exercise real disposable MongoDB and NATS containers, concurrent idempotency and claims, lease recovery, retries, transport authentication, rendering and both provider payloads through local HTTP servers. An authenticated TLS gRPC Send/Get test covers the full shared-database → NATS → rendered-email driver path. No live emails are sent. `make package-notifications-interface` prepares source and Go-client archives. See [provenance](PROVENANCE.md) and [contract usage](interface/README.md). Provider references: [SendGrid Mail Send](https://www.twilio.com/docs/sendgrid/api-reference/mail-send/mail-send), [Resend Send Email](https://resend.com/docs/api-reference/emails/send-email).