Plan independent BFF and adapt selected Sendico foundations (#34)

This commit is contained in:
Stephan D committed 2026-09-12 13:11:10 +02:00
1 parent 1e191fca42
commit 45666d33fc
17 files changed
+854 -15

No files matched your search

+300
View File
@@ -0,0 +1,300 @@
# Independent SXCHNGE BFF — task #34
## Decision and delivery scope
Build a small, independently deployed Go BFF in `backend/`. Adapt selected Sendico
code into SXCHNGE ownership; treat Sendico as an external financial provider through
a versioned partner contract. Use one BFF process initially, with a worker entrypoint
from the same module when durable asynchronous work is introduced. Do not reproduce
Sendico's service topology inside SXCHNGE.
This contribution delivers the architecture/reuse audit and a tested operational
starter. It does **not** implement account persistence, real authentication, email,
KYC uploads, money operations or a live Sendico adapter. The starter has health
and explicitly unconfigured readiness endpoints. Selected browser-session,
opaque-credential and evidence-key components are adapted now; the larger flows
below are planned extractions, not claims of completed integration.
Audit basis: local `S_I/sendico` checkout, branch `compliance-ingestor-2167`, commit
`8bb20dcd622dd8eb92d289b6b34411e7eccab083`, inspected 2026-09-12. The checkout has
unrelated dependency-manifest edits, which were not copied or modified. Findings
establish source capabilities at that revision, not deployment availability,
external API support, or production readiness. See [source references](#source-references)
and [reuse provenance](../../backend/PROVENANCE.md).
## Ownership and separation
```mermaid
flowchart LR
UI[Flutter web / mobile] --> BFF[SXCHNGE BFF\nconsumer API and authorization]
BFF --> DB[(SXCHNGE database\nsessions, mappings, commands)]
BFF --> MAIL[Email delivery adapter]
BFF --> ADAPTER[SXCHNGE Sendico adapter\nDTOs and mapping]
ADAPTER --> EDGE[Sendico partner API\nauthenticated customer scope]
EDGE --> FIN[Sendico ledger, quotes, payments\nlimits, compliance, inbound]
EDGE --> EVENTS[Signed callback delivery]
EVENTS --> BFF
```
| Owner | Responsibilities |
| --- | --- |
| SXCHNGE | Consumer accounts/passwords, email confirmation, sessions, language, MFA policy/enrollment, local customer mapping, user authorization, operation intents, idempotency/outbox/inbox, product DTOs and safe status explanations |
| Sendico | Canonical financial accounts and posted money, spendability rules, currency/rail catalog, quotes/rates/fees, execution/settlement, authoritative limits and reservations, KYC evidence/results, compliance decisions and review workflows |
| Provider boundary | Authenticated partner principal, organization and customer isolation, stable identifiers, versioned transport, error/status semantics, replay guarantees, signed events and recovery reads |
Sendico integration credentials stay exclusively in the BFF's secret configuration.
SXCHNGE users do not receive Sendico credentials, internal gRPC addresses, database
access, or membership/admin privileges on its integration organization. The BFF
never repairs balances by posting arbitrary ledger entries and never independently
reserves limits already owned by Sendico's payment execution.
### Three identities must stay distinct
Persist a server-owned mapping from `sxchnge_user_id` to a Sendico provider scope:
`organization_ref`, `customer_ref`, `compliance_subject_ref`, and, where Sendico's
ownership contracts require it, `ledger_owner_account_ref`. These fields are not
assumed equal. The partner integration principal is a fifth, separate identity.
Use unique indexes for local user/provider and provider organization/customer.
Provisioning is idempotent using a stable external customer reference.
Do not accept organization, customer, initiator or owner refs from browser payloads.
Resolve local account/quote/operation IDs under the authenticated user first; send
only mapped refs. Sendico must enforce the same scope independently. List filtering
is mandatory before results are returned; direct reads must return the same safe
not-found outcome for missing and inaccessible objects.
**Blocking contract gap:** the inspected protocols have customer-related fields,
but no customer provisioning RPC was found. Current ledger/payment ownership is
expressed with Sendico account refs; current personal-compliance HTTP routes resolve
organizations owned by the logged-in account. Neither is automatically a delegated
customer API for many SXCHNGE users under one organization. Establish a supported
mapping/provisioning facade on Sendico before connecting customer money or KYC.
Do not create a new Sendico organization per consumer to work around this.
## Reuse decisions
| Sendico starting point | Decision | SXCHNGE adaptation |
| --- | --- | --- |
| `edge/shared/auth/browser/browser.go` | Adapted now | Own `browser` package; fixed `__Host-sxchnge_session`, explicit HTTPS origins, server-selected cookie routes, no client-header CSRF bypass, same-site Lax cookie for return navigation; no Sendico error/config imports |
| `edge/shared/storage/verification/hash.go` | Selected functions adapted now | 32-byte random opaque credentials and SHA-256 storage digests; constant-time matching; no deterministic magic links or short-OTP hashing copied |
| `edge/shared/storage/s3key/key.go` | Adapted now | Own evidence object-key confinement and errors, additional encoded/backslash/control-character rejection; not an ownership check or local-filesystem sanitizer |
| `edge/shared/storage/verification/{create,consume,consume_update,db}.go` | Adapt in authentication phase | Account + purpose + operation binding, atomic consumption, expiry checked during reads, idempotent issuance, cooldown/attempt indexes. Replace model/repository/logging/notification dependencies with SXCHNGE contracts |
| `edge/shared/storage/refreshtokens/{filters,crud,revoke}.go` | Adapt model and persistence patterns in session phase | Session families/versioning, account binding, rotation and revocation; own database and credential format. Do not copy account-bound API-client login wholesale |
| `edge/shared/.../router/public/{login,refresh,logout}.go` | Adapt flow and tests | Pending login before session issuance; uniform credential failures; confirmed logout and refresh recovery. Preserve password input verbatim, unlike the inspected decoder's trim behavior |
| `edge/product/internal/server/paymentapiimp/{personal_source,visibility,quote,payquote}.go` | Adapt orchestration/authorization rules | Mandatory customer ownership even when partner has organization-wide rights; request-scoped deadlines, quote-bound execution, safe not-found projection. No BSON/permissions/discovery types in SXCHNGE models |
| `edge/product/internal/compliance/{read_service,upload_service,submit_service}.go` | Adapt orchestration pattern | Separate status/limits slices, bounded evidence ingestion, ownership, submission idempotency. Replace “account owns organization” assumption with provider customer mapping |
| `edge/callbacks/internal/signing/service.go` | Implement matching receiver after contract confirmation | Use approved wire algorithm and exact raw-body verification, bounded clock skew, event-ID inbox deduplication, reconciliation reads. Do not copy Vault-backed outbound sender into receiver |
| Ledger, limits, FX, AML/policy, gateways, treasury, KYC engines | External services | Call the partner facade; no cloned financial/compliance state machines |
| Sendico `pkg`, generic Mongo repositories, NATS/discovery bootstrapping, BFF router/DI graph | Do not import wholesale | Small local packages and explicit construction; no sibling Go replacements, shared database, submodule, automatic upstream sync or Sendico runtime dependencies |
Reuse includes meaningful tests and behavioral invariants, not just filenames.
Each extraction records its source commit and intentional deviations. Future
Sendico fixes are assessed and ported selectively through SXCHNGE PRs. Contract
fixtures/version checks protect the adapter as the products diverge.
## BFF structure
Use a single Go module and explicit constructor injection. The following is the
**target layout**; only the shell and three selected foundations exist today.
Create additional directories when their first use case is implemented.
```text
backend/
cmd/bff/ HTTP process and graceful shutdown
cmd/worker/ future outbox / provider reconciliation worker
internal/httpapi/ routes, HTTP DTOs, mappers, stable errors
internal/identity/ users, registration, email confirmation
internal/session/ authenticated principal, restore/rotate/revoke
internal/verification/ challenges, factors, operation approvals
internal/profile/ account preferences
internal/wallet/ accounts and spendable-balance projection
internal/operations/ quotes, commands, status, history
internal/compliance/ KYC and visible limits use cases
internal/provider/ narrow customer/financial provider ports
internal/sendico/ external DTOs, HTTP client, mappers, contract tests
internal/notification/ email delivery port and adapter
internal/storage/ SXCHNGE repositories and migrations/indexes
internal/jobs/ durable outbox, event inbox, reconciliation
internal/browser/ browser cookie and CSRF policy (exists)
internal/credential/ opaque credentials and digests (exists)
internal/evidence/ object key confinement (exists)
```
A handler decodes an HTTP DTO, validates it, maps to a use-case input, and supplies
the authenticated principal. Use cases own application models and repository/provider
ports. The Sendico adapter alone converts its external DTOs into these models.
The Flutter client retains DTO → mapper → service → model → provider → UI;
provider DTOs do not leak through either boundary. Avoid one interface per function
or an all-purpose “Sendico service” spanning unrelated capabilities.
Proposed persistence: a separate SXCHNGE MongoDB database/credentials, using native
driver repositories and replica-set transactions for critical local transitions.
This preserves useful Sendico persistence experience without importing its generic
repository module. Collections: users, sessions, challenges/factors, provider
mappings, quotes, operation commands, outbox and callback inbox. Unique indexes
scope email, token digest, issuance/idempotency keys and provider events. TTL indexes
clean up expired data; application checks enforce expiry immediately. Financial
balances and KYC decisions are not authoritative local records.
Start with the existing deployment's secret facility where available. Do not make
NATS, Redis, Sendico discovery or direct Vault imports prerequisites for the BFF.
Email has an independent adapter and durable delivery jobs; adopting an existing
mail provider does not require adopting Sendico notification/event envelopes.
## Required Sendico services and partner capabilities
The BFF should call a versioned partner HTTP API. The names below are **existing
internal services**, useful to implement that facade; they are not instructions to
expose unauthenticated gRPC or for the browser to call them. If a supported partner
API is already deployed, validate it against this checklist instead of creating a
second one. No deployed endpoint has been verified in this task.
| Capability needed by SXCHNGE | Existing source capability | Required partner behavior / gap |
| --- | --- | --- |
| Partner authentication and customer provisioning | Product account/API-client login and organization provisioning; customer fields in payment/KYC contracts | Confirm server-to-server grant/rotation/scopes. Existing `/login/api` is password + client-secret based, not evidence of OAuth client credentials. Add/confirm idempotent customer registration and canonical owner/subject mapping |
| Customer currency accounts | `LedgerService.ListAccounts`, `GetBalance`; treasury currency catalog | Enforce mapped customer scope; provision accounts through approved workflow. Return server-defined **available to spend**, currency precision, timestamp/version. `BalanceResponse.balance` alone does not define holds, pending debits or spendability |
| Rates, fees and quotes | `QuotationService.QuotePayment`, `QuotePayments`, `ListFXDirections` | Bind initiator and required source owner; return immutable fee/debit/credit/expiry quote snapshot, supported directions and exact money representation. An advisory quote is not execution authorization |
| Exchange and payout | `PaymentOrchestratorService.ExecutePayment`, `GetPayment`, `ListPayments` | Mandatory mapped organization/initiator/source-owner constraints; execute by accepted quote ref; scoped idempotency conflict/replay and lookup by client operation ref. Confirm FX ledger-to-ledger and payout rail availability |
| Top-up | `inbound.routes.RouteService`, payment methods and gateway capabilities; `inbound.crypto_deposits` for crypto | Return customer-bound funding instructions or hosted payment action. Confirm bank/card rails and incoming attribution. A top-up cannot immediately credit funds merely because the browser submitted a form |
| Limits and upgrade benefits | `LimitsService.ListVisibleLimits`, `CheckLimit`; reservation lifecycle used by payments | Customer-visible policy including used, remaining, period/reset and verified-tier projection. Policy chooses tier from authoritative KYC. Payments owns execution reservation/commit/release; BFF never double-reserves |
| KYC evidence and status | `KYCService.SubmitVerification`, `GetVerification`, `Reverify`, `ListVerifications`; existing edge compliance upload/submit | Delegated customer access, passport/selfie evidence roles, retention/access rules and immutable evidence IDs. Confirm manual review/correction/denial transitions and their public reason codes; public KYC proto alone does not expose a manual-review decision RPC |
| Backoffice review and eligibility | KYC plus `CaseManagementService`, Assessment and Policy | Confirm review routing and authoritative decision application; resolving a workflow case is not itself a KYC approval. Ensure KYC/tier changes feed payment eligibility, rather than assuming the separate KYC service already gates every payment |
| Status changes and reconciliation | Orchestrator reads, ledger statements, outbound `edge/callbacks` | Signed customer-scoped payment/KYC events, stable event IDs, at-least-once semantics, replay/ordering/version rules, retries and replayable status reads. Polling is a valid initial fallback; missing callbacks cannot leave funds permanently ambiguous |
The current limits README says reservations are in process, but source now requires
`RuntimeRepository` and applies durable mutations with conflict/retry handling.
Use implementation and contract tests as evidence; confirm deployed durability and
restart/multi-instance behavior before money rollout rather than repeating the
README claim.
## Client-facing contracts to implement
Paths here are proposed **SXCHNGE API**, not existing Sendico endpoints. Keep them
versioned under `/api/v1`. HTTP bodies use explicit DTOs; product error responses
carry a stable code, request ID and, where useful, `retry_after` or field codes.
No raw upstream exceptions or internal compliance evidence are returned.
| Client journey | Proposed BFF operations | Important contract |
| --- | --- | --- |
| Registration/email | `POST /registrations`, `/email-verifications`, `/email-verifications/confirm` | Idempotent registration/delivery; explicit unverified state; single-use expiring links; no session from merely creating an account |
| Login/session | `POST /sessions`, `POST /sessions/complete`, `GET /session`, `DELETE /session` | Credentials return pending challenge or confirmed session; only server can complete it. Restore, expiry, revocation, logout and frontend route guards |
| Recovery | `POST /password-resets`, `/password-resets/confirm` | Non-enumerating delivery, bounded retries and session revocation policy; do not reuse email-confirmation token purposes |
| 2FA | `GET /security`, `POST /security/challenges`, `POST /security/challenges/{id}/verify`, `/resend`, `PATCH /security` | User/session/purpose/operation-bound challenges and server-selected methods. Verified enable/disable; future TOTP setup is pending until a valid code confirms the secret |
| Preferences | `GET /profile`, `PATCH /profile` | Server-owned account language and validated mutable fields |
| Accounts/limits | `GET /accounts`, `/accounts/{id}`, `/limits` | Available balance only in primary UI; current usage and upgrade benefits from service; distinguish unavailable/stale from zero |
| Money | `POST /quotes`, `POST /operations`, `GET /operations`, `/operations/{id}`, `/operations/by-key/{key}` | Exact immutable quote, server confirmation proof, scoped idempotency and non-final acceptance. Funding instructions/hosted-action state where required |
| KYC | `GET /identity`, `POST /identity/uploads`, `POST /identity/submissions` | Bounded content validation and customer ownership; two evidence roles; idempotent submission; status, correction requests and denials |
Define money on the wire as decimal-string minor units plus currency and scale;
map into checked application types. Never use floating-point amounts or unchecked
JavaScript-sized integers. Existing Flutter models use integer minor units; their
mappers must reject unsupported precision/overflow. Never invent balances, fees,
limits or rates on provider timeout.
Current Flutter stubs need targeted extensions during integration: a session model/
provider and HTTP transport; typed funding methods/recipients instead of the current
free-text recipient; `requiresAction`/funding instructions and settlement-aware funds
status. Keep the three broad operation states if useful visually, but represent
awaiting funding, review, execution, cancellation/reversal and returned funds in
models so “failed” cannot incorrectly promise that money has already returned.
The current `GuardedLoginService` is a local stand-in: replace it with the BFF's
pending-login contract, not a production client that mints its own pending session.
There is no public endpoint that simply trusts `authorize(purpose, operation)`;
the BFF consumes proof as part of the protected server-side action.
## Critical transaction and identity flows
### Authentication and verification
Registration owns local credentials, confirms email, then idempotently provisions
or resumes the provider mapping. An incomplete provider mapping can yield an
explicit “account setup pending” state without losing the local registration.
No money access until mapping is confirmed; the optional unverified KYC state
still receives whatever basic policy Sendico returns.
Adapt Sendico's pending-login and durable consumption patterns into SXCHNGE-owned
models. Browser routes use an HttpOnly Secure host-only opaque session cookie;
unsafe requests require an exact allowed Origin plus the CSRF header, independently
of client transport hints. SameSite Lax permits safe return navigation from email;
state-changing confirmation still requires POST validation. Production sits behind
HTTPS. Native mobile uses a separate server-selected token transport with secure
platform storage and rotation/revocation, not browser-origin bypass flags.
Scope challenges by user, session/pending-login, purpose, operation ID and method.
Use independent purposes for login, email, password reset and setting changes.
Use durable attempt/cooldown state across instances; a resend must not reset the
attempt budget. High-entropy token digests can reuse the extracted helper; numeric
codes need a separately keyed verifier and strict attempt controls, never this
helper as a password/OTP hash. Future TOTP secrets require encryption, confirmed
enrollment, bounded clock drift/replay controls and recovery before availability.
### Money execution without duplicate payment
1. Resolve the authenticated customer and source/destination ownership server-side.
2. Obtain a Sendico quote carrying the mapped initiator and mandatory source-owner
constraint. Persist its local ID, user, provider ref, expiry and exact snapshot.
3. Bind the user's verification to that quote and action. A changed amount,
recipient, account or expired quote requires fresh review and authorization.
4. In one **local database transaction**, validate/consume approval and insert the
immutable operation command, scoped idempotency key, request digest and outbox
job. Reusing the key with another payload is a conflict. Replaying the same key
returns the same command before checking an already-consumed approval.
5. A worker submits the stored command using the same provider idempotency key.
Sendico rechecks ownership, quote validity, eligibility, spendability and limits.
A BFF approval does not override provider policy.
6. If a response is lost, reconcile by the provider/client operation reference;
retry only under Sendico's documented idempotency contract. Never create a new
key merely to escape uncertainty. A crash after approval consumption is recovered
by the persisted outbox, not by asking for and spending another authorization.
7. Persist observed status/version and correlate events; update balances/limits from
authoritative reads. Pending/manual review is not completion, and failed/reversed
status must describe whether funds actually returned. Callback duplicates and
out-of-order events must not regress terminal state; ambiguous order triggers a read.
There is no cross-database atomic transaction with Sendico. Durable commands,
idempotent provider execution and reconciliation are the explicit bridge. This
behavior must be proven with an “accepted upstream, response lost” contract test.
### KYC and operational observability
Use customer-bound private evidence references; validate size/type/content and
read access before forwarding to the provider. Do not pass arbitrary public URLs,
local file paths or untrusted object keys. If uploads stage in SXCHNGE, lifecycle,
retention, orphan cleanup and delete responsibility must be explicit; prefer
provider-issued scoped upload intents when supported. Passport and selfie are
separate evidence roles in one idempotent submission. Backoffice decisions remain
in Sendico; project only safe actionable reasons and the resulting limits.
Correlate requests, local commands and provider IDs on the server with structured
logs. Record provider latency/errors, outbox lag, reconciliation age, callback
retries and final outcomes. Exclude passwords, codes, cookies, documents, recipient
payloads and secrets. Existing dummy analytics remains a client sink; backend
settlement evidence supplies real outcomes, not browser “submitted” events.
## Implementation sequence and acceptance gates
| Phase | Deliverable | Exit evidence |
| --- | --- | --- |
| 0 — this task | Audit, reuse choices, independent shell and primitives | Unit/race tests, Go formatting/vet, no Sendico imports/replacements, source provenance |
| 1 — provider contract agreement | Customer provisioning/mapping, scoped access, spendability, quote/payment replay, KYC review and callback contract | Sandbox fixtures/API schema and two-customer isolation tests; determine whether existing partner edge can be extended |
| 2 — own authentication | Mongo repositories, email outbox, signup/email/login/session/2FA/recovery plus Flutter adapters/guards | Restart-safe consumption and delivery, session isolation/revocation, no session before required factor, web/mobile transport tests |
| 3 — read-only provider integration | Customer mapping, currency accounts, profile, limits, KYC status | Available-money semantics, no cross-customer list/direct reads, safe unknown/error handling, contract version validation |
| 4 — first money slice | One supported top-up rail followed by exchange and payout, recipient/funding UI extensions | Provider-confirmed credit/debit, fees and expiry; idempotent concurrency and lost-response recovery; funds explanations match actual statuses |
| 5 — evidence/review integration | Passport/selfie uploads, backoffice review, correction/deny, policy propagation | Customer isolation, evidence lifecycle, review-driven limits change and execution policy enforcement |
| 6 — rollout | Deployment, migrations, secret rotation, health/readiness, reconciliation operations and event delivery | Sandbox end-to-end runs, restart/recovery drills, ownership tests on every public resource and explicit supported-rail configuration |
Authentication work can proceed while the provider contract is agreed. Do not block
it on cloning financial services. Conversely, no live money rollout until the scope,
spendability, idempotency and settlement contracts are verified. Keep each phase in
its own task/PR; Sendico-side facade work is a separate provider contribution.
## Source references
All paths below are in the audited Sendico revision; links pin that revision.
- [Browser transport](https://git.sendico.io/S_I/sendico/src/commit/8bb20dcd622dd8eb92d289b6b34411e7eccab083/api/edge/shared/auth/browser/browser.go), [verification hashing](https://git.sendico.io/S_I/sendico/src/commit/8bb20dcd622dd8eb92d289b6b34411e7eccab083/api/edge/shared/storage/verification/hash.go), [evidence keys](https://git.sendico.io/S_I/sendico/src/commit/8bb20dcd622dd8eb92d289b6b34411e7eccab083/api/edge/shared/storage/s3key/key.go).
- [Pending/API login](https://git.sendico.io/S_I/sendico/src/commit/8bb20dcd622dd8eb92d289b6b34411e7eccab083/api/edge/shared/internal/transport/http/router/public/login.go), [session binding](https://git.sendico.io/S_I/sendico/src/commit/8bb20dcd622dd8eb92d289b6b34411e7eccab083/api/edge/shared/storage/refreshtokens/filters.go), [verification indexes](https://git.sendico.io/S_I/sendico/src/commit/8bb20dcd622dd8eb92d289b6b34411e7eccab083/api/edge/shared/storage/verification/db.go).
- [Ledger contract](https://git.sendico.io/S_I/sendico/src/commit/8bb20dcd622dd8eb92d289b6b34411e7eccab083/api/proto/ledger/v1/ledger.proto), [payment visibility](https://git.sendico.io/S_I/sendico/src/commit/8bb20dcd622dd8eb92d289b6b34411e7eccab083/api/edge/product/internal/server/paymentapiimp/visibility.go), [personal source constraints](https://git.sendico.io/S_I/sendico/src/commit/8bb20dcd622dd8eb92d289b6b34411e7eccab083/api/edge/product/internal/server/paymentapiimp/personal_source.go).
- [Quotation](https://git.sendico.io/S_I/sendico/src/commit/8bb20dcd622dd8eb92d289b6b34411e7eccab083/api/proto/payments/quotation/v2/quotation.proto), [orchestration](https://git.sendico.io/S_I/sendico/src/commit/8bb20dcd622dd8eb92d289b6b34411e7eccab083/api/proto/payments/orchestration/v2/orchestration.proto), [inbound routes](https://git.sendico.io/S_I/sendico/src/commit/8bb20dcd622dd8eb92d289b6b34411e7eccab083/api/proto/inbound/routes/v1/routes.proto).
- [Limits projection](https://git.sendico.io/S_I/sendico/src/commit/8bb20dcd622dd8eb92d289b6b34411e7eccab083/api/proto/limits/v1/limits.proto), [durable mutation implementation](https://git.sendico.io/S_I/sendico/src/commit/8bb20dcd622dd8eb92d289b6b34411e7eccab083/api/limits/internal/service/limits/reservation_mutation_persistence.go), [required runtime storage](https://git.sendico.io/S_I/sendico/src/commit/8bb20dcd622dd8eb92d289b6b34411e7eccab083/api/limits/internal/service/limits/service.go).
- [Personal-compliance ownership](https://git.sendico.io/S_I/sendico/src/commit/8bb20dcd622dd8eb92d289b6b34411e7eccab083/api/edge/product/internal/server/accountapiimp/compliance.go), [KYC](https://git.sendico.io/S_I/sendico/src/commit/8bb20dcd622dd8eb92d289b6b34411e7eccab083/api/proto/compliance/kyc/v1/kyc.proto), [case workflow](https://git.sendico.io/S_I/sendico/src/commit/8bb20dcd622dd8eb92d289b6b34411e7eccab083/api/proto/workflow/case/v1/case.proto), [callback signing](https://git.sendico.io/S_I/sendico/src/commit/8bb20dcd622dd8eb92d289b6b34411e7eccab083/api/edge/callbacks/internal/signing/service.go).