Skip to content

Technical Architecture

This document describes the AI Wallet system architecture with detailed GTM technical strategy, covering gateway integration, policy edge design, and router-agnostic implementation. Objective: Define scalable architecture that supports rapid market entry while maintaining flexibility to integrate with multiple AI providers and gateways.

Executive Summary

Based on GTM analysis, AI Wallet's technical strategy focuses on: - Router-Agnostic Design: Integration with existing gateways (OpenRouter, Vercel AI Gateway) rather than building own router - Policy Edge Architecture: Lightweight middleware for budget enforcement and consent management - Identity-First Approach: Cross-app user identity and consent receipts as core differentiator - Ephemeral Key Management: Secure, short-lived credentials for API access - Modular Integration: Plug-and-play compatibility with developer workflows

Core Architecture Principles

GTM-Derived Design Decisions

1. Gateway Integration Over Building - Strategy: Integrate with existing gateways rather than compete as another router - Rationale: Leverages proven infrastructure while focusing differentiation on identity layer - Providers: OpenRouter (PKCE + credits), Vercel AI Gateway (team billing), Cloudflare AI Gateway - Benefits: Faster time-to-market, reduced infrastructure complexity, partner-friendly positioning

2. Policy Edge Proxy Pattern - Design: Thin server middleware that enforces policies before reaching backend providers - Functions: Budget validation, consent checking, usage metering, rate limiting - Security: Short-lived session tokens, no provider keys exposed to clients - Performance: Minimal latency overhead, optimized for high-throughput API calls

3. Identity-Native Architecture - Core Value: Cross-app user identity that travels with users across different applications - Implementation: OAuth-based authentication with portable consent receipts - Data Model: User-centric usage analytics and spending governance - Compliance: Built-in audit trails and regulatory reporting

System Architecture

High-Level Architecture

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   Client App    │    │   Client App    │    │   Client App    │
│  (Next.js/React)│    │   (Mobile)      │    │    (CLI)        │
└─────────┬───────┘    └─────────┬───────┘    └─────────┬───────┘
          │                      │                      │
          └──────────────────────┼──────────────────────┘
                                 │
                    ┌────────────▼────────────┐
                    │   AI Wallet SDK         │
                    │  - Authentication       │
                    │  - Budget Management    │
                    │  - Consent Receipts     │
                    └────────────┬────────────┘
                                 │
                    ┌────────────▼────────────┐
                    │   Policy Edge Proxy     │
                    │  - Session Validation   │
                    │  - Budget Enforcement   │
                    │  - Usage Metering       │
                    │  - Consent Verification │
                    └────────────┬────────────┘
                                 │
          ┌──────────────────────┼──────────────────────┐
          │                      │                      │
    ┌─────▼─────┐         ┌─────▼─────┐         ┌─────▼─────┐
    │  OpenRouter│         │ Vercel AI │         │ Cloudflare│
    │    PKCE   │         │ Gateway   │         │ AI Gateway│
    └───────────┘         └───────────┘         └───────────┘

Component Architecture

1. AI Wallet SDK Layer

Purpose: Developer-friendly client library for easy integration Key Components: - WalletProvider: React context for wallet state management - useWallet(): Hook for accessing wallet functionality - Auth Components: Login, consent, and billing UI components - API Client: Encrypted communication with policy edge

GTM Integration Features: - Drop-in React components for quick implementation - Framework-agnostic API for non-React applications - Automatic consent receipt generation - Built-in budget visualization and controls

2. Policy Edge Proxy

Purpose: Centralized policy enforcement and usage metering Architecture: - API Gateway: Handles incoming requests with authentication - Session Manager: Validates short-lived session tokens - Policy Engine: Enforces budget, rate, and consent policies - Usage Tracker: Real-time metering and analytics - Consent Manager: Validates and updates consent receipts

Security Model: - Ephemeral Tokens: 5-15 minute session tokens, no long-lived credentials - Key Isolation: Provider API keys never exposed to client applications - Audit Logging: Immutable record of all policy decisions and API calls - Encryption: End-to-end encryption for sensitive data

3. Gateway Adapters

Purpose: Abstract integration with different AI providers Adapter Pattern Implementation:

interface GatewayAdapter {
  // Unified interface for all gateways
  authenticate(user: User, scopes: string[]): Promise<AuthResult>;
  call(request: AIRequest): Promise<AIResponse>;
  getUsage(userId: string): Promise<UsageData>;
  validateBudget(userId: string, amount: number): Promise<boolean>;
}

// Specific implementations
class OpenRouterAdapter implements GatewayAdapter { ... }
class VercelGatewayAdapter implements GatewayAdapter { ... }
class CloudflareAdapter implements GatewayAdapter { ... }

Router Selection Strategy: - Default: Vercel AI Gateway (best developer experience) - User Choice: OpenRouter (if user has account and wants it) - Enterprise: Direct provider connections (for compliance) - Fallback: Load balancing across available providers

Purpose: Cross-app identity with portable consent Components: - User Identity Service: OAuth-based authentication with cross-app identity - Consent Receipt System: Cryptographically signed consent records - Policy Engine: Centralized policy management and enforcement - Audit Trail: Immutable consent and usage history

Data Architecture

Core Data Models

User Model:

interface User {
  id: string;
  email: string;
  walletId: string;
  consentReceipts: ConsentReceipt[];
  budgets: Budget[];
  usage: UsageRecord[];
  preferences: UserPreferences;
}

Consent Receipt Model:

interface ConsentReceipt {
  id: string;
  userId: string;
  appId: string;
  scopes: string[];
  issuedAt: Date;
  expiresAt: Date;
  signature: string; // Cryptographic signature
  revocationToken: string;
}

Budget Model:

interface Budget {
  id: string;
  userId: string;
  appId?: string; // Optional per-app budget
  amount: number;
  period: 'daily' | 'weekly' | 'monthly';
  resetDate: Date;
  spent: number;
  currency: 'credits' | 'usd';
}

Data Flow Architecture

Authentication Flow: 1. User authenticates with AI Wallet 2. AI Wallet validates user and checks budgets 3. Policy Edge issues short-lived session token 4. Client uses token to make API calls 5. Gateway adapters handle provider-specific authentication

API Call Flow: 1. Client sends request with session token 2. Policy Edge validates token and policies 3. Usage tracked and budget checked 4. Request forwarded to appropriate gateway adapter 5. Response returned with usage metadata 6. Audit trail updated

Consent Management Flow: 1. App requests specific scopes 2. User presented with clear consent interface 3. Consent receipt generated and signed 4. Receipt stored with cryptographic integrity 5. Policy Edge validates consent on each API call

Security Architecture

Key Management Strategy

GTM-Derived Security Requirements: - No Client-Side Provider Keys: All provider API keys stay on server - Ephemeral Credentials: Short-lived tokens minimize exposure risk - Zero-Knowledge: Client never sees provider-specific implementation - Audit-Ready: All actions logged for compliance

Implementation: - Key Vault: Encrypted storage of all provider API keys - Rotation: Automatic key rotation every 30 days - Monitoring: Real-time detection of key usage anomalies - Revocation: Immediate key invalidation capability

Privacy by Design

Data Minimization: - Collect only necessary data for billing and compliance - Anonymize usage data where possible - Implement data retention policies - Support data deletion requests

Cross-Border Compliance: - Regional data residency options - GDPR-compliant data processing - Audit trail for data access - Consent-based data sharing

Scalability and Performance

Horizontal Scaling Strategy

Stateless Design: - Policy Edge proxy designed for horizontal scaling - Session state stored in Redis cluster - Gateway adapters can scale independently - Database read replicas for global performance

Performance Targets: - API Latency: < 50ms added latency for policy enforcement - Throughput: 10,000+ requests per second per instance - Availability: 99.9% uptime SLA - Data Consistency: Eventual consistency acceptable for usage tracking

Load Balancing and Failover

Multi-Region Deployment: - Primary: US East (Virginia) - Secondary: EU West (Ireland) - Tertiary: Asia Pacific (Singapore) - Automatic failover between regions

Provider Redundancy: - Primary: Vercel AI Gateway - Fallback: OpenRouter - Emergency: Direct provider APIs - Circuit breaker pattern for provider failures

GTM-Specific Technical Features

Developer Experience Focus

Quick Integration: - One-line SDK integration: <AIWalletProvider> - Automatic consent UI generation - Built-in budget visualization - TypeScript definitions for all APIs

Documentation and Examples: - Interactive API documentation - Code examples in multiple languages - Video tutorials for common use cases - Integration testing sandbox

Partnership-Friendly Architecture

White-Label Options: - Customizable branding for partners - API access for custom integrations - Revenue sharing through usage tracking - Partner-specific features and controls

Open Standards Support: - OpenID Connect for authentication - OAuth 2.0 for authorization - JSON Web Tokens for sessions - RESTful APIs for integrations

Monitoring and Observability

Technical Metrics

  • Performance: API latency, throughput, error rates
  • Usage: Requests per second, data transfer, cache hit rates
  • Business: Revenue, user adoption, partner usage
  • Security: Authentication failures, policy violations, key usage

Compliance Monitoring

  • Audit Logs: All policy decisions and API calls
  • Consent Tracking: Consent changes and revocations
  • Data Access: Who accessed what data when
  • Regulatory Reports: Automated compliance reporting

Deployment and DevOps

Infrastructure as Code

  • Terraform: Multi-cloud infrastructure provisioning
  • Kubernetes: Container orchestration
  • Helm: Application deployment and scaling
  • ArgoCD: GitOps continuous deployment

CI/CD Pipeline

  • GitHub Actions: Automated testing and deployment
  • Security Scanning: SAST/DAST in pipeline
  • Performance Testing: Automated load testing
  • Blue-Green Deployment: Zero-downtime deployments

Migration and Integration Strategy

Provider Migration

  • Gradual Migration: Move users to new providers gradually
  • A/B Testing: Test different provider configurations
  • Fallback Mechanisms: Automatic provider switching
  • User Notification: Transparent communication about changes

Data Migration

  • Backward Compatibility: Support legacy API versions
  • Data Import: Tools for importing existing user data
  • Progressive Enhancement: New features without breaking changes
  • Rollback Capability: Quick rollback for problematic changes

Future Architecture Considerations

AI-Specific Features

  • Model Preference Learning: User-specific model recommendations
  • Cost Optimization: Automatic routing to most cost-effective models
  • Performance Monitoring: Model-specific performance tracking
  • A/B Testing: Built-in experimentation framework

Enterprise Features

  • Multi-Tenancy: Support for enterprise customer hierarchies
  • Advanced Analytics: Usage patterns and cost optimization insights
  • Custom Policies: Enterprise-specific budget and access policies
  • Integration APIs: Enterprise system integration capabilities

Core Architecture Components (Whitepaper View)

Based on the AI Wallet whitepaper, the system consists of six foundational layers:

1. Gateway/API Layer

  • Function: Single REST/gRPC endpoint routes requests to configured AI backends
  • Features: Fault-tolerant routing, retries, and regional load balancing
  • Implementation: Unified API surface that abstracts provider differences

2. Provider Adapters

  • Function: Pluggable modules translating standard calls into provider-specific formats
  • Support: Custom or self-hosted models alongside major provider integrations
  • Benefits: Extensibility for new providers without core system changes

3. Billing & Usage Engine

  • Function: Real-time metering (tokens, compute seconds, calls)
  • Features: Tiered subscription and pay-as-you-go billing
  • Capabilities: Automated invoicing and payment reconciliation

4. Identity & Entitlements

  • Function: OAuth/OIDC integration for developer and end-user auth
  • Features: Role-based access controls and fine-grained usage policies
  • Security: Token vaulting and secure key management

5. Dashboard & Analytics

  • Function: Unified usage dashboards, cost breakdowns, latency and error metrics
  • Features: Alerts, quotas, and predictive cost modeling
  • Value: Visibility and control for all stakeholders

6. Partner & Marketplace Layer

  • Function: Onboarding flows for third-party model and data vendors
  • Features: Revenue-share management and contract enforcement
  • Strategy: Ecosystem growth and network effects

Source: AI Wallet_ A Next-Generation Infrastructure for AI.md lines 19-41

Source Attribution

Primary source for technical strategy: GTM-01Nov25-ChatGPTPlus.pdf Architecture components: AI Wallet_ A Next-Generation Infrastructure for AI.md