popiart-node-canvas/.planning/phases/02-model-discovery/02-03-PLAN.md

phase	plan	type
02-model-discovery	3	execute

Add in-memory caching for model lists and create unified models API endpoint.

Purpose: Reduce API calls to external providers and provide single endpoint for fetching models from all configured providers. Output: Caching layer with TTL and unified /api/models endpoint returning models from all providers.

<execution_context> ~/.claude/get-shit-done/workflows/execute-phase.md ~/.claude/get-shit-done/templates/summary.md </execution_context>

@.planning/PROJECT.md @.planning/ROADMAP.md @.planning/STATE.md @.planning/phases/02-model-discovery/DISCOVERY.md

Prior plan context:

@.planning/phases/02-model-discovery/02-01-SUMMARY.md @.planning/phases/02-model-discovery/02-02-SUMMARY.md

Key files:

@src/lib/providers/types.ts @src/lib/providers/index.ts @src/lib/providers/replicate.ts @src/lib/providers/fal.ts @src/types/index.ts

Tech stack available:

• Provider abstraction interfaces
• Replicate and fal.ai providers implemented
• API routes for each provider

Established patterns:

• Provider implementations in src/lib/providers/
• API routes return { success, models/error }

Constraining decisions:

• In-memory caching (no Redis)
• 5-15 minute cache TTL

Task 1: Create model caching utility src/lib/providers/cache.ts Create simple in-memory cache for model lists:

1. Define CacheEntry interface: { data: T, timestamp: number }
2. Create cache Map: Map<string, CacheEntry<ProviderModel[]>>
3. Define DEFAULT_TTL = 10 * 60 * 1000 (10 minutes)
4. Export getCachedModels(key: string): ProviderModel[] | null
   • Return null if not in cache or expired
   • Return cached data if within TTL
5. Export setCachedModels(key: string, models: ProviderModel[]): void
   • Store with current timestamp
6. Export invalidateCache(key?: string): void
   • If key provided, delete that entry
   • If no key, clear entire cache
7. Export getCacheKey(provider: ProviderType, search?: string): string
   • Return ${provider}:models or ${provider}:search:${search}

Keep it simple - no LRU eviction, no persistence. Server restarts clear cache. TypeScript compiles without errors: npx tsc --noEmit cache.ts exports get/set/invalidate functions for model caching

Task 2: Create unified models API endpoint src/app/api/models/route.ts Create unified API route that fetches models from all configured providers:

1. Export GET handler
2. Parse query params:
   • provider: optional, filter to specific provider
   • search: optional, search query
   • refresh: optional, bypass cache if "true"
3. Get API keys from request headers:
   • X-Replicate-Key: Replicate API key
   • X-Fal-Key: fal.ai API key
4. Build list of providers to fetch from:
   • If provider param, use only that provider (if key provided)
   • Otherwise, fetch from all providers with keys
5. For each provider:
   • Check cache first (unless refresh=true)
   • If cache miss, fetch from provider's API route internally or call provider directly
   • Store result in cache
6. Combine all provider results into single array
7. Sort by provider, then by name
8. Return { success: true, models: ProviderModel[], cached: boolean }
9. Handle partial failures: return models from successful providers, include errors array

Response format: { success: true, models: ProviderModel[], cached: boolean, providers: { replicate: { success: true, count: 50 }, fal: { success: true, count: 100 } }, errors?: string[] } curl -X GET "http://localhost:3000/api/models" -H "X-Fal-Key: test" returns combined model list (fal.ai works without key) Unified endpoint returns models from all configured providers, uses caching, handles partial failures

Task 3: Add provider module exports src/lib/providers/index.ts Update provider index to export cache utilities and ensure providers auto-register:

1. Add import for cache utilities: export * from "./cache"
2. Add dynamic imports comment explaining that providers self-register
3. Export helper function listAllModels(apiKeys: Record<string, string>): Promise<ProviderModel[]>
   • Takes API keys object
   • Calls listModels() on each configured provider
   • Returns combined array
4. Export helper function searchAllModels(query: string, apiKeys: Record<string, string>): Promise<ProviderModel[]>
   • Same pattern but calls searchModels()

Note: These helpers can be used by the unified API route. TypeScript compiles without errors: npx tsc --noEmit Provider index exports cache utilities and helper functions for multi-provider operations

Before declaring plan complete: - [ ] `npx tsc --noEmit` passes - [ ] `npm run build` succeeds - [ ] Cache utility exists with get/set/invalidate functions - [ ] Unified /api/models endpoint works - [ ] Caching prevents redundant API calls - [ ] Multiple providers can be queried in single request

<success_criteria>

• In-memory cache with 10-minute TTL
• Unified /api/models endpoint aggregates all providers
• Partial failure handling (some providers fail, others succeed)
• Cache bypass option for manual refresh
• Phase 2 complete - model discovery working for both providers </success_criteria>

After completion, create `.planning/phases/02-model-discovery/02-03-SUMMARY.md` following the summary template.

Mark Phase 2 complete in summary.