---
phase: 02-model-discovery
plan: 03
type: execute
---

<objective>
Add in-memory caching for model lists and create unified models API endpoint.

Purpose: Reduce API calls to external providers and provide single endpoint for fetching models from all configured providers.
Output: Caching layer with TTL and unified /api/models endpoint returning models from all providers.
</objective>

<execution_context>
~/.claude/get-shit-done/workflows/execute-phase.md
~/.claude/get-shit-done/templates/summary.md
</execution_context>

<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/02-model-discovery/DISCOVERY.md

# Prior plan context:
@.planning/phases/02-model-discovery/02-01-SUMMARY.md
@.planning/phases/02-model-discovery/02-02-SUMMARY.md

# Key files:
@src/lib/providers/types.ts
@src/lib/providers/index.ts
@src/lib/providers/replicate.ts
@src/lib/providers/fal.ts
@src/types/index.ts

**Tech stack available:**
- Provider abstraction interfaces
- Replicate and fal.ai providers implemented
- API routes for each provider

**Established patterns:**
- Provider implementations in src/lib/providers/
- API routes return { success, models/error }

**Constraining decisions:**
- In-memory caching (no Redis)
- 5-15 minute cache TTL
</context>

<tasks>

<task type="auto">
  <name>Task 1: Create model caching utility</name>
  <files>src/lib/providers/cache.ts</files>
  <action>
Create simple in-memory cache for model lists:

1. Define CacheEntry interface: { data: T, timestamp: number }
2. Create cache Map: Map&lt;string, CacheEntry&lt;ProviderModel[]&gt;&gt;
3. Define DEFAULT_TTL = 10 * 60 * 1000 (10 minutes)
4. Export getCachedModels(key: string): ProviderModel[] | null
   - Return null if not in cache or expired
   - Return cached data if within TTL
5. Export setCachedModels(key: string, models: ProviderModel[]): void
   - Store with current timestamp
6. Export invalidateCache(key?: string): void
   - If key provided, delete that entry
   - If no key, clear entire cache
7. Export getCacheKey(provider: ProviderType, search?: string): string
   - Return `${provider}:models` or `${provider}:search:${search}`

Keep it simple - no LRU eviction, no persistence. Server restarts clear cache.
  </action>
  <verify>TypeScript compiles without errors: npx tsc --noEmit</verify>
  <done>cache.ts exports get/set/invalidate functions for model caching</done>
</task>

<task type="auto">
  <name>Task 2: Create unified models API endpoint</name>
  <files>src/app/api/models/route.ts</files>
  <action>
Create unified API route that fetches models from all configured providers:

1. Export GET handler
2. Parse query params:
   - provider: optional, filter to specific provider
   - search: optional, search query
   - refresh: optional, bypass cache if "true"
3. Get API keys from request headers:
   - X-Replicate-Key: Replicate API key
   - X-Fal-Key: fal.ai API key
4. Build list of providers to fetch from:
   - If provider param, use only that provider (if key provided)
   - Otherwise, fetch from all providers with keys
5. For each provider:
   - Check cache first (unless refresh=true)
   - If cache miss, fetch from provider's API route internally or call provider directly
   - Store result in cache
6. Combine all provider results into single array
7. Sort by provider, then by name
8. Return { success: true, models: ProviderModel[], cached: boolean }
9. Handle partial failures: return models from successful providers, include errors array

Response format:
{
  success: true,
  models: ProviderModel[],
  cached: boolean,
  providers: {
    replicate: { success: true, count: 50 },
    fal: { success: true, count: 100 }
  },
  errors?: string[]
}
  </action>
  <verify>
curl -X GET "http://localhost:3000/api/models" -H "X-Fal-Key: test" returns combined model list (fal.ai works without key)
  </verify>
  <done>Unified endpoint returns models from all configured providers, uses caching, handles partial failures</done>
</task>

<task type="auto">
  <name>Task 3: Add provider module exports</name>
  <files>src/lib/providers/index.ts</files>
  <action>
Update provider index to export cache utilities and ensure providers auto-register:

1. Add import for cache utilities: export * from "./cache"
2. Add dynamic imports comment explaining that providers self-register
3. Export helper function listAllModels(apiKeys: Record&lt;string, string&gt;): Promise&lt;ProviderModel[]&gt;
   - Takes API keys object
   - Calls listModels() on each configured provider
   - Returns combined array
4. Export helper function searchAllModels(query: string, apiKeys: Record&lt;string, string&gt;): Promise&lt;ProviderModel[]&gt;
   - Same pattern but calls searchModels()

Note: These helpers can be used by the unified API route.
  </action>
  <verify>TypeScript compiles without errors: npx tsc --noEmit</verify>
  <done>Provider index exports cache utilities and helper functions for multi-provider operations</done>
</task>

</tasks>

<verification>
Before declaring plan complete:
- [ ] `npx tsc --noEmit` passes
- [ ] `npm run build` succeeds
- [ ] Cache utility exists with get/set/invalidate functions
- [ ] Unified /api/models endpoint works
- [ ] Caching prevents redundant API calls
- [ ] Multiple providers can be queried in single request
</verification>

<success_criteria>

- In-memory cache with 10-minute TTL
- Unified /api/models endpoint aggregates all providers
- Partial failure handling (some providers fail, others succeed)
- Cache bypass option for manual refresh
- Phase 2 complete - model discovery working for both providers
</success_criteria>

<output>
After completion, create `.planning/phases/02-model-discovery/02-03-SUMMARY.md` following the summary template.

Mark Phase 2 complete in summary.
</output>