271 lines
17 KiB
Markdown
271 lines
17 KiB
Markdown
# Sign-up Facts intake
|
|
|
|
Task [#46](https://git.sendico.io/tech/sxchnge/issues/46), branch
|
|
`identity-facts-46`. Primary module: identity-facts; secondary modules: auth,
|
|
customer creation, runtime, shared database source and Flutter error feedback.
|
|
Delivery: [PR #47](https://git.sendico.io/tech/sxchnge/pulls/47), targeting `main`.
|
|
|
|
## Registration response correction
|
|
|
|
Follow-up [#57](https://git.sendico.io/tech/sxchnge/issues/57), branch
|
|
`identity-facts-57`, delivered in [PR #59](https://git.sendico.io/tech/sxchnge/pulls/59)
|
|
against `main` (implementation `abca15f`), corrects the registration success status
|
|
to HTTP 201 Created. The PR remains unmerged.
|
|
Sendico's individual creation handler returns 201; the original adapter and test
|
|
fixture incorrectly required 200. A successful registration therefore left a
|
|
pending customer with an unresolved claim and `signup_attention_required`.
|
|
The adapter now checks the exact status per operation before validating receipts:
|
|
201 for registration and 200 for Facts intake. Other statuses, including other
|
|
2xx responses, remain failures. Malformed receipts and uncertain registrations
|
|
still require the existing recovery rules.
|
|
|
|
On 2026-10-17, the affected local signup was reconciled using its registration
|
|
correlation in Sendico logs and an authenticated organization-scoped individual
|
|
GET. A conditional majority write bound that confirmed individual to the existing
|
|
local claim, preserving the saved facts, namespace and registration request.
|
|
The rebuilt BFF resumed the public activation-resend flow: Facts intake returned
|
|
a persisted receipt and the endpoint returned 202 after activation-email queue
|
|
acceptance. No replacement individual was registered. The account remains
|
|
pending until the customer confirms the email; queue acceptance does not prove
|
|
inbox delivery.
|
|
|
|
Recovery also exposed a notification worker lifetime issue: its NATS subscription
|
|
retained the 20-second startup context and was automatically unsubscribed after
|
|
that deadline. Resource provisioning now has its own bounded context, while the
|
|
subscription follows the service lifetime and closes at shutdown. A regression
|
|
test checks that releasing provisioning leaves the subscription context active,
|
|
and service cancellation still ends it.
|
|
|
|
The rebuilt worker processed the queued activation message. The live Resend
|
|
provider rejected it with `provider_authentication_failed` (HTTP 401 or 403),
|
|
leaving delivery in `failed` after one attempt. Later diagnostics below identify
|
|
the unverified sender domain as the actual blocker. Correct domain authorization
|
|
before activation resend with a new idempotency key. Credential rotation is not
|
|
required by that error; if a separate rotation is needed, use `make email` and
|
|
allow Vault Agent to refresh/restart the notification process. Do not replay the
|
|
failed notification as a success or mark the customer active on their behalf.
|
|
|
|
Validation passed: `GOFLAGS=-p=2 make test` (all backend race tests and canonical
|
|
frontend suites), `make lint-backend`, focused provider/identity race tests and
|
|
notification bus/app race tests. The HTTP 201 regression first failed against the
|
|
old adapter and passed after correction. An earlier full run hit a database
|
|
credential-rotation timing failure at `rotation_test.go:249` during the Docker
|
|
build; the isolated race-test retry and both subsequent full runs passed.
|
|
Local BFF and notifications images were rebuilt with fresh upstream manifests
|
|
and resolved stable digest pins; the BFF, session sidecar and notifications
|
|
services are healthy. NATS shows the worker still waiting for deliveries beyond
|
|
the startup deadline, with no pending or unacknowledged jobs. Live email delivery
|
|
is blocked by provider authorization as described above. No security scans were
|
|
requested or run. Unrelated notifications runtime settings are excluded from
|
|
the task commits.
|
|
|
|
### Provider diagnostics follow-up
|
|
|
|
The same task/PR adds transient provider diagnostics to the worker's existing
|
|
structured outcome log: `provider_status`, `provider_error` and
|
|
`provider_message`. The adapter extracts bounded Resend/SendGrid error fields and
|
|
redacts credentials, request values, addresses, URLs and token/code fragments.
|
|
Transport errors have a sanitized cause; invalid or oversized responses have a
|
|
safe fallback. Diagnostics remain outside stored notification models and public
|
|
gRPC responses. The outcome is logged even when its result write fails. See
|
|
the [notification logging contract](../backend/notifications/README.md).
|
|
|
|
Runtime inspection confirmed `email.driver: resend`, an API key in Vault, and
|
|
sender `noreply@sxchnge.com`. A live diagnostic notification used Resend's
|
|
documented `delivered+sxchnge-config@resend.dev` test mailbox, without retrying
|
|
the customer's email. The rebuilt worker logged HTTP 403,
|
|
`provider_error: validation_error`, and the sanitized message:
|
|
`The sxchnge.com domain is not verified. Please, add and verify your domain on [redacted-url]`.
|
|
The precise blocker is sender-domain verification, not missing local Resend
|
|
configuration. A sending-only key is appropriate for sending; its inability to
|
|
read Resend's logs API does not explain the original send failure. Verify the
|
|
configured domain in Resend (and its required DNS records), or explicitly choose
|
|
an already verified sender domain. No sender/domain/DNS setting was changed by
|
|
this follow-up; the customer's account remains pending email confirmation.
|
|
|
|
Follow-up validation passed: `GOFLAGS=-p=2 make test`, `make lint-backend`, and
|
|
`go -C backend/notifications test -race ./email ./service -count=1`. Coverage
|
|
includes Resend/SendGrid messages, malformed/trailing/oversized response bodies,
|
|
credential/content/code redaction, Unicode-safe truncation, sanitized transport
|
|
errors, and exactly one provider outcome log even when persistence fails. The
|
|
notifications image was rebuilt from fresh stable digest pins and deployed
|
|
healthy; the live test verified both the diagnostic log and the unchanged generic
|
|
gRPC failure response. No security scan or real customer email retry was run.
|
|
|
|
## Behavior and boundaries
|
|
|
|
With `api.sendico` configured, the product BFF creates the local pending customer
|
|
and activation token atomically, then registers an independent individual through
|
|
Sendico's public HTTP API and submits the sign-up assertions. It persists the
|
|
intake receipt before sending activation email or returning sign-up success.
|
|
Activation resend runs the same handoff; activation checks the receipt in its
|
|
local transaction, without external I/O. A failed receipt check rolls back token
|
|
consumption and leaves the account pending. Existing active customers are not
|
|
backfilled or blocked at login.
|
|
|
|
The adapter consumes these existing Sendico contracts:
|
|
|
|
- `POST /individuals/{organization_ref}`, empty body, HTTP 201 registration
|
|
receipt containing `subject_ref` and `registered_at`.
|
|
- `POST /identity_facts/{organization_ref}`, HTTP 200 immutable intake receipt
|
|
containing `submission_ref`, `subject_ref` and `accepted_at`.
|
|
|
|
The configured API prefix precedes both paths. See Sendico
|
|
[individual subjects](https://git.sendico.io/tech/sendico/src/branch/main/docs/api/individual-subjects.md)
|
|
and [Facts intake](https://git.sendico.io/tech/sendico/src/branch/main/docs/api/identity-facts-intake.md).
|
|
The only assertions sent are `full_name`, normalized `email` and
|
|
`external_customer_ref` (the local customer ID). Passwords, password hashes,
|
|
activation tokens, session credentials and device data never enter the adapter.
|
|
The input name is retained as a full name, without guessing name parts. No facts
|
|
are marked verified; acceptance does not establish identity, KYC completeness,
|
|
verification, limits or payout eligibility. The existing KYC preview is unchanged.
|
|
|
|
Only the BFF owns Sendico credentials and organization scope. Local customer
|
|
models have no organization, membership or platform permissions. HTTP DTOs and
|
|
mappers stay in `providers/sendico`; the handoff operates on domain facts and
|
|
receipts in `identity`. Flutter continues through its DTO, mapper, service,
|
|
model and provider layers. It never calls Sendico or supplies subject references.
|
|
|
|
## Configuration and rollout
|
|
|
|
Add this block under `api` in the product service configuration, retaining
|
|
`api.notifications`:
|
|
|
|
```yaml
|
|
sendico:
|
|
api_url: https://your-sendico-host/api/v1
|
|
organization_ref: <provisioned-SXCHNGE-organization-object-id>
|
|
source_account_ref: <provisioned-integration-account-object-id>
|
|
token_file: /run/sxchnge/sendico-token
|
|
```
|
|
|
|
Resolve the real IDs from the target Sendico deployment. The integration account
|
|
must be a member of the SXCHNGE organization with the existing organizations
|
|
`update` permission; reconciliation GET requests also require `read`. The token
|
|
must be a full, unexpired Sendico access JWT whose `id` claim is that account.
|
|
The adapter inspects claims only to protect the replay namespace; Sendico checks
|
|
the signature and authorizes every call. No login, refresh or privileged account
|
|
provisioning is performed by this adapter.
|
|
|
|
Keep the token in Vault at `services/bff/sendico`, field `access_token`. Render it
|
|
with the deployment's Vault Agent into the configured private runtime file; give
|
|
only the BFF policy read access to that KV path. Endpoint, organization and source
|
|
IDs remain in service YAML. For the local stack, configure
|
|
`infra/local/bff/runtime.yml`, render to the BFF's existing `/run/sxchnge` tmpfs,
|
|
and extend its Vault Agent policy/template for this provisioned secret. These
|
|
deployment values and secret provisioning are deliberately not guessed or
|
|
enabled by the example. Do not place tokens in Git, browser builds or shell
|
|
history. Refresh the file atomically before expiry using the same Sendico account;
|
|
the adapter rereads it on every request. Startup rejects missing, expired or
|
|
mismatched credentials and rejects Sendico configuration without notifications.
|
|
|
|
HTTPS is required except for loopback HTTP test/development endpoints. Redirects
|
|
are rejected and each HTTP call has a four-second timeout. Unexpected response
|
|
statuses (registration requires 201; Facts intake requires 200), malformed/truncated
|
|
receipts, invalid identifiers, wrong subjects and unsuccessful envelopes never
|
|
count as acceptance. Upstream error bodies are not surfaced.
|
|
|
|
Omitting `api.sendico` keeps existing local-only behavior. Enabling it also gates
|
|
activation for previously pending customers; use activation resend to submit
|
|
their saved name/email before confirming their old link. There is no automatic
|
|
bulk migration. Changing endpoint, organization or source account changes the
|
|
adapter fingerprint and blocks existing bindings for explicit reconciliation.
|
|
Token rotation for the same account retains that fingerprint. Plan such changes
|
|
as migrations; never reset a binding just to bypass the check.
|
|
|
|
## Persistence, retries and recovery
|
|
|
|
The local `signup_facts` collection uses the customer ID as `_id`, so its unique
|
|
primary key coordinates concurrent requests and separate BFF instances. It stores
|
|
an immutable sign-up facts snapshot, adapter fingerprint, random registration
|
|
request correlation, registration claim, subject and confirmed receipt. It uses
|
|
the shared database repository and connection lifecycle with majority writes and
|
|
primary majority reads, overriding weaker connection defaults. No external call
|
|
runs in a database transaction. Snapshots and bindings contain personal data and
|
|
must follow the customer data retention policy; no TTL silently removes replay
|
|
state, and customer deletion does not delete the immutable Sendico submission.
|
|
|
|
Once a subject is stored, intake retries use
|
|
`sxchnge:signup:<local-customer-id>` as `source_submission_ref`. Sendico's
|
|
organization/authenticated-account/source-reference uniqueness returns the
|
|
original receipt for that subject. Retries send the saved snapshot even if a
|
|
later request changes name or email. Losing the HTTP receipt or a local receipt
|
|
write is recoverable by replay. A saved receipt skips subsequent external calls.
|
|
Retrying sign-up still requires the original password; active, archived or
|
|
otherwise conflicting accounts cannot be resumed as a new customer.
|
|
|
|
**Individual registration is not idempotent.** The BFF commits a claim before
|
|
dispatch and never expires or automatically retries that registration. It cannot
|
|
prove that a timed-out request, malformed response or failed local binding write
|
|
did not create a subject. Another request sees an in-progress claim as temporarily
|
|
unavailable for 30 seconds; an older unresolved claim requires support. The
|
|
registration caller immediately reports assistance on an uncertain failure. An
|
|
accepted registration is saved with a short independent context even if the
|
|
browser disconnects. A crash before dispatch can also require reconciliation.
|
|
This trades automatic recovery for avoiding duplicate subjects under the current
|
|
upstream contract; there is no distributed exactly-once guarantee.
|
|
|
|
For an unresolved registration, operators must stop requests for the affected
|
|
customer and inspect the saved `registrationRequest` plus Sendico identity logs
|
|
(`X-Request-ID`, operation `subject_register`) under the configured account and
|
|
organization. A successful registration's server-issued `record_ref` identifies
|
|
the candidate subject. Confirm it using the organization-scoped individual GET
|
|
API before atomically setting the matching local binding's `subjectRef` and
|
|
clearing `registering`; retain its snapshot, fingerprint and request reference.
|
|
If a local binding write failed after receipt, BFF diagnostics also contain the
|
|
subject reference. If authoritative records prove no subject was created, an
|
|
operator may clear the claim after fixing the dependency. If neither result can
|
|
be established, leave it blocked; never register a speculative replacement.
|
|
Resume sign-up with the original credentials or activation resend after this
|
|
repair. These are controlled operational repairs, not customer API endpoints.
|
|
|
|
HTTP 503 `signup_attention_required` maps to typed shared auth and verification
|
|
state and localized EN/RU support guidance, including activation resend recovery.
|
|
Other intake/receipt failures remain temporary service
|
|
failures. Form input is retained, duplicate submissions stay disabled during
|
|
loading, and neither failure navigates to an email-sent screen or creates a
|
|
session. The existing support channel must be available to handle reconciliation.
|
|
|
|
## Measurement and UX review
|
|
|
|
The existing [auth journey](ux/auth-journeys.md) keeps its attempt, retry,
|
|
abandonment and 30-minute settled inactivity rules. `auth_signup_succeeded` now
|
|
requires server confirmation of configured intake plus activation queue acceptance;
|
|
it remains an intermediate step and email confirmation remains the terminal
|
|
onboarding outcome. Sign-up dependency failures and assistance outcomes use the
|
|
existing allowlisted `unavailable` analytics category; verification assistance
|
|
uses it too. Retries retain the journey
|
|
and operation semantics. No names, emails or provider response bodies are added.
|
|
|
|
`sxchnge_signup_facts_total{outcome}` records `accepted`, `replayed`,
|
|
`unavailable` and `attention_required` handoff attempts;
|
|
`sxchnge_signup_facts_duration_seconds` measures their latency. Counts are
|
|
attempts, not distinct users. Use unavailable/attention counts divided by total
|
|
handoff attempts for operational failure rates; use the existing correlated auth
|
|
journey for conversion, duration and abandonment. Structured handoff logs include
|
|
local customer and random registration request references for support correlation,
|
|
without asserted values, credentials or raw errors. IDs are never metric labels.
|
|
The product team uses the distinction to separate retryable intake outages from
|
|
registration reconciliation. The default client analytics sink remains in memory;
|
|
production retention and cross-device conversion correlation remain deployment
|
|
gaps, as documented by the existing journey contract.
|
|
|
|
Review covers loading suppression, input preservation, success only on confirmed
|
|
handoff, temporary retry and assistance states, existing navigation, mobile layout,
|
|
text scaling and live-region error feedback. Focused tests cover these states,
|
|
the mapped backend error, non-inflated client success, immutable retry payloads,
|
|
credential rotation, timeout/cancellation/redirect behavior, receipt validation,
|
|
privacy-safe errors, concurrent durable claims and activation rollback. The tests
|
|
use disposable local MongoDB and a simulated Sendico HTTP server; they do not
|
|
create live subjects or send email. Live integration and support operations must
|
|
be verified against provisioned deployment credentials before enabling the block.
|
|
The EN/RU error state is exercised at a 390px viewport with 1.5x text scaling;
|
|
existing keyboard regression tests and a code review of the live-region semantics
|
|
cover the unchanged controls. No manual browser or screen-reader walkthrough
|
|
against a live Sendico deployment was performed.
|
|
|
|
The initial sign-up assertion remains a separate immutable submission. Optional
|
|
profile/ID/selfie collection, later account uploads and reviewed limits are now
|
|
covered by [Customer identity verification](identity-verification.md), task #48.
|
|
Proofs are not required to activate or log into a basic account.
|