Files
sxchnge/docs/signup-facts.md
T

320 lines
20 KiB
Markdown

# Sign-up Facts intake
Task [#46](https://git.sendico.io/tech/sxchnge/issues/46), branch
`identity-facts-46`. Primary module: identity-facts; secondary modules: auth,
customer creation, runtime, shared database source and Flutter error feedback.
Delivery: [PR #47](https://git.sendico.io/tech/sxchnge/pulls/47), targeting `main`.
## Registration response correction
Follow-up [#57](https://git.sendico.io/tech/sxchnge/issues/57), branch
`identity-facts-57`, delivered in [PR #59](https://git.sendico.io/tech/sxchnge/pulls/59)
against `main` (implementation `abca15f`), corrects the registration success status
to HTTP 201 Created. The PR remains unmerged.
Sendico's individual creation handler returns 201; the original adapter and test
fixture incorrectly required 200. A successful registration therefore left a
pending customer with an unresolved claim and `signup_attention_required`.
The adapter now checks the exact status per operation before validating receipts:
201 for registration and 200 for Facts intake. Other statuses, including other
2xx responses, remain failures. Malformed receipts and uncertain registrations
still require the existing recovery rules.
On 2026-10-17, the affected local signup was reconciled using its registration
correlation in Sendico logs and an authenticated organization-scoped individual
GET. A conditional majority write bound that confirmed individual to the existing
local claim, preserving the saved facts, namespace and registration request.
The rebuilt BFF resumed the public activation-resend flow: Facts intake returned
a persisted receipt and the endpoint returned 202 after activation-email queue
acceptance. No replacement individual was registered. The account remains
pending until the customer confirms the email; queue acceptance does not prove
inbox delivery.
Recovery also exposed a notification worker lifetime issue: its NATS subscription
retained the 20-second startup context and was automatically unsubscribed after
that deadline. Resource provisioning now has its own bounded context, while the
subscription follows the service lifetime and closes at shutdown. A regression
test checks that releasing provisioning leaves the subscription context active,
and service cancellation still ends it.
The rebuilt worker processed the queued activation message. The live Resend
provider rejected it with `provider_authentication_failed` (HTTP 401 or 403),
leaving delivery in `failed` after one attempt. Later diagnostics below identify
the unverified sender domain as the actual blocker. Correct domain authorization
before activation resend with a new idempotency key. Credential rotation is not
required by that error; if a separate rotation is needed, use `make email` and
allow Vault Agent to refresh/restart the notification process. Do not replay the
failed notification as a success or mark the customer active on their behalf.
Validation passed: `GOFLAGS=-p=2 make test` (all backend race tests and canonical
frontend suites), `make lint-backend`, focused provider/identity race tests and
notification bus/app race tests. The HTTP 201 regression first failed against the
old adapter and passed after correction. An earlier full run hit a database
credential-rotation timing failure at `rotation_test.go:249` during the Docker
build; the isolated race-test retry and both subsequent full runs passed.
Local BFF and notifications images were rebuilt with fresh upstream manifests
and resolved stable digest pins; the BFF, session sidecar and notifications
services are healthy. NATS shows the worker still waiting for deliveries beyond
the startup deadline, with no pending or unacknowledged jobs. Live email delivery
is blocked by provider authorization as described above. No security scans were
requested or run. Unrelated notifications runtime settings are excluded from
the task commits.
### Provider diagnostics follow-up
The same task/PR adds transient provider diagnostics to the worker's existing
structured outcome log: `provider_status`, `provider_error` and
`provider_message`. The adapter extracts bounded Resend/SendGrid error fields and
redacts credentials, request values, addresses, URLs and token/code fragments.
Transport errors have a sanitized cause; invalid or oversized responses have a
safe fallback. Diagnostics remain outside stored notification models and public
gRPC responses. The outcome is logged even when its result write fails. See
the [notification logging contract](../backend/notifications/README.md).
Runtime inspection confirmed `email.driver: resend`, an API key in Vault, and
sender `noreply@sxchnge.com`. A live diagnostic notification used Resend's
documented `delivered+sxchnge-config@resend.dev` test mailbox, without retrying
the customer's email. The rebuilt worker logged HTTP 403,
`provider_error: validation_error`, and the sanitized message:
`The sxchnge.com domain is not verified. Please, add and verify your domain on [redacted-url]`.
The precise blocker is sender-domain verification, not missing local Resend
configuration. A sending-only key is appropriate for sending; its inability to
read Resend's logs API does not explain the original send failure. Verify the
configured domain in Resend (and its required DNS records), or explicitly choose
an already verified sender domain. No sender/domain/DNS setting was changed by
this follow-up; the customer's account remains pending email confirmation.
Follow-up validation passed: `GOFLAGS=-p=2 make test`, `make lint-backend`, and
`go -C backend/notifications test -race ./email ./service -count=1`. Coverage
includes Resend/SendGrid messages, malformed/trailing/oversized response bodies,
credential/content/code redaction, Unicode-safe truncation, sanitized transport
errors, and exactly one provider outcome log even when persistence fails. The
notifications image was rebuilt from fresh stable digest pins and deployed
healthy; the live test verified both the diagnostic log and the unchanged generic
gRPC failure response. No security scan or real customer email retry was run.
## Behavior and boundaries
### Local mail client
The user subsequently selected Sendico's existing mailbox instead of Resend for
local notifications. The authoritative Sendico configuration resolves to
`no-reply@sendico.io`, `mail.sendico.io:465`, authenticated implicit TLS and a
10-second network timeout. SXCHNGE's owned `smtp` adapter consumes rendered email
models through the existing sender contract and queue; it does not import or
access Sendico business modules, databases or messaging. The password was copied
once, without printing it, from the running Sendico notification environment into
`services/notifications/email/smtp` in SXCHNGE Vault. Only `password` is stored
there; host, port, username, display name and sender remain in runtime YAML.
The SMTP adapter verifies certificates, authenticates before submission, sends
UTF-8 MIME alternatives, and requires the final DATA 250 reply for acceptance.
SMTP 4xx replies retry through the existing worker policy; 5xx replies fail;
ambiguous transport outcomes remain UNKNOWN and are not immediately resent.
Message-ID is correlation only, not a provider idempotency guarantee. Existing
failed Resend notifications are not replayed or rewritten. The unverified
`sxchnge.com` Resend domain is no longer the selected local delivery path.
Live verified-TLS SMTP authentication and NOOP succeeded with reply 250, without
submitting email. SMTP adapter race tests and backend lint pass. Local tooling
fixtures pass, including password-only Vault validation. Broader validation is
currently blocked by Docker: the service integration test timed out starting its
Testcontainers reaper; both `GOFLAGS=-p=2 make test` and its `-p=1` retry fail in
the existing credential-rotation setup while copying files into a disposable
MongoDB container (90-second context timeout). No checks were disabled. Fresh
stable image pins resolved on retry; two image builds hit a Go proxy TLS
handshake timeout during protobuf generator installation. The generator stage
now shares the existing BuildKit Go module/build caches, preserving its pinned
tool versions and generation command. Cache seeding also stalled during file
extraction; it was canceled after copying the pinned module archives. A normal
build retry remained blocked before starting a compiler process. A fallback
using the same freshly pulled, digest-pinned Go image, canonical Make generation
command and existing module cache stalled during ordinary container startup,
before package installation or compilation produced output.
Deployment is not complete: the running notifications container still uses the
previous image. The SMTP Vault password and source configuration are prepared,
but its Agent policy/template have not been activated against that old image.
Vault responds over HTTP, while Vault/Mongo Docker health checks time out and
the local Sendico session is unavailable. Restarting Docker Desktop was requested
for recovery because it also interrupts the separate Sendico stack; it has not
been authorized or performed. Rebuild, activate the matching Agent configuration,
restart notifications and verify the live `email_driver: smtp` startup log after
Docker is responsive. Do not treat the successful SMTP login as a deployed
notification delivery check.
With `api.sendico` configured, the product BFF creates the local pending customer
and activation token atomically, then registers an independent individual through
Sendico's public HTTP API and submits the sign-up assertions. It persists the
intake receipt before sending activation email or returning sign-up success.
Activation resend runs the same handoff; activation checks the receipt in its
local transaction, without external I/O. A failed receipt check rolls back token
consumption and leaves the account pending. Existing active customers are not
backfilled or blocked at login.
The adapter consumes these existing Sendico contracts:
- `POST /individuals/{organization_ref}`, empty body, HTTP 201 registration
receipt containing `subject_ref` and `registered_at`.
- `POST /identity_facts/{organization_ref}`, HTTP 200 immutable intake receipt
containing `submission_ref`, `subject_ref` and `accepted_at`.
The configured API prefix precedes both paths. See Sendico
[individual subjects](https://git.sendico.io/tech/sendico/src/branch/main/docs/api/individual-subjects.md)
and [Facts intake](https://git.sendico.io/tech/sendico/src/branch/main/docs/api/identity-facts-intake.md).
The only assertions sent are `full_name`, normalized `email` and
`external_customer_ref` (the local customer ID). Passwords, password hashes,
activation tokens, session credentials and device data never enter the adapter.
The input name is retained as a full name, without guessing name parts. No facts
are marked verified; acceptance does not establish identity, KYC completeness,
verification, limits or payout eligibility. The existing KYC preview is unchanged.
Only the BFF owns Sendico credentials and organization scope. Local customer
models have no organization, membership or platform permissions. HTTP DTOs and
mappers stay in `providers/sendico`; the handoff operates on domain facts and
receipts in `identity`. Flutter continues through its DTO, mapper, service,
model and provider layers. It never calls Sendico or supplies subject references.
## Configuration and rollout
Add this block under `api` in the product service configuration, retaining
`api.notifications`:
```yaml
sendico:
api_url: https://your-sendico-host/api/v1
organization_ref: <provisioned-SXCHNGE-organization-object-id>
source_account_ref: <provisioned-integration-account-object-id>
token_file: /run/sxchnge/sendico-token
```
Resolve the real IDs from the target Sendico deployment. The integration account
must be a member of the SXCHNGE organization with the existing organizations
`update` permission; reconciliation GET requests also require `read`. The token
must be a full, unexpired Sendico access JWT whose `id` claim is that account.
The adapter inspects claims only to protect the replay namespace; Sendico checks
the signature and authorizes every call. No login, refresh or privileged account
provisioning is performed by this adapter.
Keep the token in Vault at `services/bff/sendico`, field `access_token`. Render it
with the deployment's Vault Agent into the configured private runtime file; give
only the BFF policy read access to that KV path. Endpoint, organization and source
IDs remain in service YAML. For the local stack, configure
`infra/local/bff/runtime.yml`, render to the BFF's existing `/run/sxchnge` tmpfs,
and extend its Vault Agent policy/template for this provisioned secret. These
deployment values and secret provisioning are deliberately not guessed or
enabled by the example. Do not place tokens in Git, browser builds or shell
history. Refresh the file atomically before expiry using the same Sendico account;
the adapter rereads it on every request. Startup rejects missing, expired or
mismatched credentials and rejects Sendico configuration without notifications.
HTTPS is required except for loopback HTTP test/development endpoints. Redirects
are rejected and each HTTP call has a four-second timeout. Unexpected response
statuses (registration requires 201; Facts intake requires 200), malformed/truncated
receipts, invalid identifiers, wrong subjects and unsuccessful envelopes never
count as acceptance. Upstream error bodies are not surfaced.
Omitting `api.sendico` keeps existing local-only behavior. Enabling it also gates
activation for previously pending customers; use activation resend to submit
their saved name/email before confirming their old link. There is no automatic
bulk migration. Changing endpoint, organization or source account changes the
adapter fingerprint and blocks existing bindings for explicit reconciliation.
Token rotation for the same account retains that fingerprint. Plan such changes
as migrations; never reset a binding just to bypass the check.
## Persistence, retries and recovery
The local `signup_facts` collection uses the customer ID as `_id`, so its unique
primary key coordinates concurrent requests and separate BFF instances. It stores
an immutable sign-up facts snapshot, adapter fingerprint, random registration
request correlation, registration claim, subject and confirmed receipt. It uses
the shared database repository and connection lifecycle with majority writes and
primary majority reads, overriding weaker connection defaults. No external call
runs in a database transaction. Snapshots and bindings contain personal data and
must follow the customer data retention policy; no TTL silently removes replay
state, and customer deletion does not delete the immutable Sendico submission.
Once a subject is stored, intake retries use
`sxchnge:signup:<local-customer-id>` as `source_submission_ref`. Sendico's
organization/authenticated-account/source-reference uniqueness returns the
original receipt for that subject. Retries send the saved snapshot even if a
later request changes name or email. Losing the HTTP receipt or a local receipt
write is recoverable by replay. A saved receipt skips subsequent external calls.
Retrying sign-up still requires the original password; active, archived or
otherwise conflicting accounts cannot be resumed as a new customer.
**Individual registration is not idempotent.** The BFF commits a claim before
dispatch and never expires or automatically retries that registration. It cannot
prove that a timed-out request, malformed response or failed local binding write
did not create a subject. Another request sees an in-progress claim as temporarily
unavailable for 30 seconds; an older unresolved claim requires support. The
registration caller immediately reports assistance on an uncertain failure. An
accepted registration is saved with a short independent context even if the
browser disconnects. A crash before dispatch can also require reconciliation.
This trades automatic recovery for avoiding duplicate subjects under the current
upstream contract; there is no distributed exactly-once guarantee.
For an unresolved registration, operators must stop requests for the affected
customer and inspect the saved `registrationRequest` plus Sendico identity logs
(`X-Request-ID`, operation `subject_register`) under the configured account and
organization. A successful registration's server-issued `record_ref` identifies
the candidate subject. Confirm it using the organization-scoped individual GET
API before atomically setting the matching local binding's `subjectRef` and
clearing `registering`; retain its snapshot, fingerprint and request reference.
If a local binding write failed after receipt, BFF diagnostics also contain the
subject reference. If authoritative records prove no subject was created, an
operator may clear the claim after fixing the dependency. If neither result can
be established, leave it blocked; never register a speculative replacement.
Resume sign-up with the original credentials or activation resend after this
repair. These are controlled operational repairs, not customer API endpoints.
HTTP 503 `signup_attention_required` maps to typed shared auth and verification
state and localized EN/RU support guidance, including activation resend recovery.
Other intake/receipt failures remain temporary service
failures. Form input is retained, duplicate submissions stay disabled during
loading, and neither failure navigates to an email-sent screen or creates a
session. The existing support channel must be available to handle reconciliation.
## Measurement and UX review
The existing [auth journey](ux/auth-journeys.md) keeps its attempt, retry,
abandonment and 30-minute settled inactivity rules. `auth_signup_succeeded` now
requires server confirmation of configured intake plus activation queue acceptance;
it remains an intermediate step and email confirmation remains the terminal
onboarding outcome. Sign-up dependency failures and assistance outcomes use the
existing allowlisted `unavailable` analytics category; verification assistance
uses it too. Retries retain the journey
and operation semantics. No names, emails or provider response bodies are added.
`sxchnge_signup_facts_total{outcome}` records `accepted`, `replayed`,
`unavailable` and `attention_required` handoff attempts;
`sxchnge_signup_facts_duration_seconds` measures their latency. Counts are
attempts, not distinct users. Use unavailable/attention counts divided by total
handoff attempts for operational failure rates; use the existing correlated auth
journey for conversion, duration and abandonment. Structured handoff logs include
local customer and random registration request references for support correlation,
without asserted values, credentials or raw errors. IDs are never metric labels.
The product team uses the distinction to separate retryable intake outages from
registration reconciliation. The default client analytics sink remains in memory;
production retention and cross-device conversion correlation remain deployment
gaps, as documented by the existing journey contract.
Review covers loading suppression, input preservation, success only on confirmed
handoff, temporary retry and assistance states, existing navigation, mobile layout,
text scaling and live-region error feedback. Focused tests cover these states,
the mapped backend error, non-inflated client success, immutable retry payloads,
credential rotation, timeout/cancellation/redirect behavior, receipt validation,
privacy-safe errors, concurrent durable claims and activation rollback. The tests
use disposable local MongoDB and a simulated Sendico HTTP server; they do not
create live subjects or send email. Live integration and support operations must
be verified against provisioned deployment credentials before enabling the block.
The EN/RU error state is exercised at a 390px viewport with 1.5x text scaling;
existing keyboard regression tests and a code review of the live-region semantics
cover the unchanged controls. No manual browser or screen-reader walkthrough
against a live Sendico deployment was performed.
The initial sign-up assertion remains a separate immutable submission. Optional
profile/ID/selfie collection, later account uploads and reviewed limits are now
covered by [Customer identity verification](identity-verification.md), task #48.
Proofs are not required to activate or log into a basic account.