Define the job before comparing products
Agentic payments products can provide discovery, agent identity, mandates, policy evaluation, scoped credentials, payment execution, settlement verification, or post-transaction operations. A vendor strong in one layer may rely on partners for the rest. Write the target transaction, jurisdictions, merchants, rails, values, approval model, and recourse expectations before scoring features.
Ask which capabilities are generally available, preview-only, partner-limited, or roadmap. Announcements establish intent; they do not establish coverage, reliability, or fit for a particular operating model. Require current documentation and a test environment for material claims.
Score control evidence, not interface polish
A procurement scorecard should follow the transaction lifecycle. For each layer, record the control owner, enforceable artifact, failure behavior, exported evidence, and portability constraint. Weight mandatory controls before convenience features such as natural-language setup or automatic merchant discovery.
| Area | Evidence request | Red flag |
|---|---|---|
| Authority | Exact mandate schema and enforcement trace | Permission inferred from conversation |
| Credentials | Scope, audience, storage, rotation, revocation | Reusable credential exposed to model runtime |
| Execution | Supported rails, regions, and state definitions | Provider acceptance labeled final settlement |
| Operations | Idempotency, reconciliation, refund, dispute flow | Manual recovery without durable identifiers |
| Governance | Audit export, retention, roles, change control | Opaque policy changes or inaccessible evidence |
Run a proof with adversarial and operational cases
The proof should include normal purchases, changed prices, expired quotes, out-of-scope merchants, concurrent attempts, provider timeouts, duplicate callbacks, revoked authority, partial refunds, and reconciliation after a missing event. Verify the deny path with the same production components, not a slide or mocked policy engine.
Include finance, security, compliance, support, and application owners in the review. A system may satisfy the agent developer while leaving the finance team unable to close a ledger or the support team unable to explain a charge.
- Capture every promised capability with status, dependency, owner, and test result.
- Require a portable export of mandates, policy decisions, attempts, and receipts.
- Test vendor and customer responsibilities during refund and dispute events.
- Measure latency and cost across the complete flow, including step-up approval.
Put operating obligations into the commercial agreement
Contract terms should cover incident notification, service boundaries, log availability, data use, credential custody, subcontractors, change notice, deletion, export, and exit support. Define which party owns reconciliation breaks and how long evidence remains available after termination.
Avoid locking business policy to one provider's opaque format. Preserve an internal mandate and transaction model so adapters can change without rewriting the meaning of authorization across products and audit systems.
A convincing autonomous checkout is a starting point for evaluation, not proof that the system can be governed in production.
Source discipline
Primary sources
Product status and protocol behavior are checked against maintainer documentation. Company sources establish what their organizations publish; they do not independently prove adoption or performance.