Notifications backend
Independent internal gRPC service with durable asynchronous email delivery. The public contract belongs to interface/proto/v1/interface.proto. Generated clients are build artifacts, not committed source.
Architecture
The gRPC transport maps protobuf DTOs to domain requests. The service validates
and renders immutable template revisions, then saves a notification and its
pending-work state atomically in one document through the shared DB wrapper.
The outbox dispatcher publishes opaque IDs using shared messaging's durable
transport, adapted from Sendico. A durable NATS worker claims the notification
with an atomic conditional patch and saves the outcome before acknowledging.
DriverConfig selects the email driver: sendgrid or resend.
Email adapters only map rendered email models to provider wire payloads. SendGrid uses adapted Sendico SDK construction; Resend uses its email HTTP API. Both support subject, text and HTML, bounded timeouts and safe error classification.
Run
Requires Go 1.27, protoc, an SXCHNGE MongoDB database and a NATS server with
JetStream enabled. Copy config.example.yml to an untracked config.yml here.
Set SXCHNGE_MONGO_URI, SXCHNGE_NATS_URL, a unique token of at least 32 bytes for
each configured caller, and the selected email provider's API key environment
variable. Configure a verified sender address, TLS certificate/key and template
catalog. Paths are relative to the process working directory.
From the root, run make run-notifications. The gRPC listener always uses TLS.
Use a private authenticated NATS endpoint; tls:// URLs and credentials in the URL
are supported by the shared driver. Mongo and messaging both use shared
DriverConfig settings; email settings reject unknown keys.
The queue uses SXCHNGE_NOTIFICATIONS, subject sxchnge.notifications.email,
and durable consumer email. Configure queue.stream.replicas as 1, 3 or 5 to
match the NATS cluster, and queue.stream.max_age_hours for retention (48 hours
minimum). File storage and the owned stream/subject names are enforced.
queue.provisioning.environment accepts production or non-production;
mode: ops validates existing resources without changing them, mode: app
provisions missing resources, and mode: hybrid provisions only outside production.
Production retention and consumer-setting drift fail startup instead of being
silently accepted. Provisioning operations honor their startup context.
The example defaults to one replica and app provisioning for local development.
Multiple service workers can safely compete for jobs through atomic claims.
Templates and parameters
See templates.example.json. Parameters use Go template map syntax such as
{{.first_name}}. Each revision declares the exact allowed parameter names and
locale-specific content. Missing/extra parameters, unknown revisions/locales and
invalid content are rejected before acceptance. Empty locale uses the default.
Values are data, not executable templates; HTML is escaped by html/template.
The catalog is loaded and compiled at startup. Never edit a published revision;
add a new version. Rendered payloads are persisted so retries do not depend on a
later catalog change. Recipient, body and subject are cleared on terminal state;
remaining status and request fingerprints expire through a seven-day TTL.
Delivery semantics
- Send persists before returning; it does not wait for NATS or the email provider.
- Outbox scans every five seconds and republishes safely after a bus failure. Pending notifications expire and have their rendered content cleared even when the bus is unavailable or a retry is scheduled later.
- Duplicate RPCs and duplicate broker deliveries do not create extra jobs/sends.
- Provider HTTP 2xx becomes ACCEPTED. No receipt/webhook or read tracking is claimed.
- Explicit rate limiting (429) retries with bounded exponential backoff and Retry-After, at most five provider attempts, bounded by message expiry.
- Transport errors, 408, 5xx and expired worker leases become UNKNOWN. These may have sent the email; operators must reconcile before requesting another send.
- Other provider failures are terminal.
provider_authentication_failedsignals credentials/permissions,provider_rejecteda nonretryable rejection,retry_exhaustedthe attempt limit, andexpiredthe message deadline. - Resend receives the notification ID as its provider idempotency key. Its window is 24 hours. SendGrid has no equivalent guarantee; exactly-once external email delivery is not promised.
- Changing the configured driver fails queued jobs pinned to the old driver with
driver_changed. Drain queued jobs before changing sender credentials/config. - Callers authorize recipients and preferences. No direct customer-database access, bulk campaigns, provider failover, SMS, push or Telegram delivery is included.
Structured logs report notification IDs, driver, state and attempt count; message content, template values, addresses and provider response bodies are excluded. Monitor pending age, UNKNOWN/FAILED states and repeated outbox/worker warnings.
Validate and distribute
make test and make lint-backend generate interfaces and validate all modules.
Notifications tests exercise real disposable MongoDB and NATS containers, concurrent
idempotency and claims, lease recovery, retries, transport authentication, rendering
and both provider payloads through local HTTP servers. An authenticated TLS gRPC
Send/Get test covers the full shared-database → NATS → rendered-email driver path.
No live emails are sent.
make package-notifications-interface prepares source and Go-client archives.
See provenance and contract usage.
Provider references: SendGrid Mail Send, Resend Send Email.