docs(02): create Phase 2 Model Discovery plans

Phase 02: Model Discovery - DISCOVERY.md with Replicate and fal.ai API research - 02-01: Replicate provider implementation (2 tasks) - 02-02: fal.ai provider implementation (2 tasks) - 02-03: Model caching and unified endpoint (3 tasks) - 7 total tasks defined across 3 plans - Ready for execution Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
6 months ago · 25d9deac1f
4 changed files with 638 additions and 0 deletions
--- a/.planning/phases/02-model-discovery/02-01-PLAN.md
+++ b/.planning/phases/02-model-discovery/02-01-PLAN.md
@ -0,0 +1,133 @@
 ---
 phase: 02-model-discovery
 plan: 01
 type: execute
 ---
 <objective>
 Implement Replicate provider with model discovery from their REST API.
 Purpose: Enable dynamic fetching of available models from Replicate's API, normalized to our ProviderModel interface.
 Output: Working Replicate provider that registers in the provider system and an API route for client access.
 </objective>
 <execution_context>
 ~/.claude/get-shit-done/workflows/execute-phase.md
 ~/.claude/get-shit-done/templates/summary.md
 </execution_context>
 <context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/02-model-discovery/DISCOVERY.md
 # Prior phase context:
@.planning/phases/01-provider-infrastructure/01-02-SUMMARY.md
 # Key files:
@src/lib/providers/types.ts
@src/lib/providers/index.ts
@src/types/index.ts
 **Tech stack available:**
 - Provider abstraction interfaces (ProviderInterface, ProviderModel)
 - Provider registry with registerProvider()
 - ProviderType includes "replicate"
 **Established patterns:**
 - Provider types in src/lib/providers/types.ts
 - Self-registration via registerProvider()
 - API routes return { success, error, data } JSON
 **Constraining decisions:**
 - No SDKs - use direct fetch calls
 - API keys from localStorage via provider settings
 </context>
 <tasks>
 <task type="auto">
  <name>Task 1: Create Replicate provider implementation</name>
  <files>src/lib/providers/replicate.ts</files>
  <action>
 Create Replicate provider implementing ProviderInterface:
 1. Import types from @/lib/providers and @/types
 2. Implement helper function to get API key from localStorage (client) or return null (server)
 3. Implement listModels():
   - Fetch GET https://api.replicate.com/v1/models with Bearer auth
   - Map results to ProviderModel[] (id: `${owner}/${name}`, name, description, provider: "replicate", coverImage: cover_image_url)
   - Infer capabilities from model name/description (text-to-image default, add image-to-image if "img2img" in name/description)
   - Return first page only (no pagination traversal)
 4. Implement searchModels(query):
   - Fetch GET https://api.replicate.com/v1/search?query={query} with Bearer auth
   - Map and return results same as listModels
 5. Implement getModel(modelId):
   - Parse modelId as owner/name
   - Fetch GET https://api.replicate.com/v1/models/{owner}/{name}
   - Map single result to ProviderModel
 6. Implement generate() as stub returning { success: false, error: "Not implemented" } (Phase 3)
 7. Implement isConfigured() checking if API key exists
 8. Implement getApiKey() returning key from localStorage or null
 9. Call registerProvider(replicateProvider) at module level
 Note: This runs client-side for isConfigured/getApiKey but API calls will be proxied through API route.
  </action>
  <verify>TypeScript compiles without errors: npx tsc --noEmit</verify>
  <done>replicate.ts exports working provider, registered in registry when imported</done>
 </task>
 <task type="auto">
  <name>Task 2: Create Replicate models API route</name>
  <files>src/app/api/providers/replicate/models/route.ts</files>
  <action>
 Create Next.js API route for fetching Replicate models server-side:
 1. Export GET handler
 2. Get API key from request header or query param (client passes it)
 3. Validate API key exists, return 401 if missing
 4. Parse optional `search` query param
 5. If search param:
   - Fetch https://api.replicate.com/v1/search?query={search}
 6. Else:
   - Fetch https://api.replicate.com/v1/models
 7. Both calls use Authorization: Bearer {apiKey}
 8. Map response to ProviderModel[] using same logic as provider
 9. Return { success: true, models: ProviderModel[] }
 10. Handle errors: return { success: false, error: message }
 Pattern to follow from existing generate route:
 - Use NextRequest, NextResponse
 - Log request ID for debugging
 - Handle fetch errors gracefully
  </action>
  <verify>
 curl -X GET "http://localhost:3000/api/providers/replicate/models" -H "X-API-Key: test" returns JSON (will error without real key, but route responds)
  </verify>
  <done>API route returns model list when called with valid API key, 401 without key</done>
 </task>
 </tasks>
 <verification>
 Before declaring plan complete:
 - [ ] `npx tsc --noEmit` passes
 - [ ] `npm run build` succeeds
 - [ ] Replicate provider file exists and exports provider
 - [ ] API route file exists at correct path
 - [ ] Provider registers when module is imported
 </verification>
 <success_criteria>
 - Replicate provider implements ProviderInterface
 - Provider self-registers via registerProvider()
 - API route proxies model fetching with API key from header
 - Models normalized to ProviderModel interface
 - No TypeScript errors
 </success_criteria>
 <output>
 After completion, create `.planning/phases/02-model-discovery/02-01-SUMMARY.md` following the summary template.
 </output>
--- a/.planning/phases/02-model-discovery/02-02-PLAN.md
+++ b/.planning/phases/02-model-discovery/02-02-PLAN.md
@ -0,0 +1,140 @@
 ---
 phase: 02-model-discovery
 plan: 02
 type: execute
 ---
 <objective>
 Implement fal.ai provider with model discovery from their REST API.
 Purpose: Enable dynamic fetching of available models from fal.ai's API, normalized to our ProviderModel interface.
 Output: Working fal.ai provider that registers in the provider system and an API route for client access.
 </objective>
 <execution_context>
 ~/.claude/get-shit-done/workflows/execute-phase.md
 ~/.claude/get-shit-done/templates/summary.md
 </execution_context>
 <context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/02-model-discovery/DISCOVERY.md
 # Prior plan context:
@.planning/phases/02-model-discovery/02-01-SUMMARY.md
 # Key files:
@src/lib/providers/types.ts
@src/lib/providers/index.ts
@src/lib/providers/replicate.ts
@src/types/index.ts
 **Tech stack available:**
 - Provider abstraction interfaces (ProviderInterface, ProviderModel)
 - Provider registry with registerProvider()
 - ProviderType includes "fal"
 - Replicate provider pattern established
 **Established patterns:**
 - Provider implementation in src/lib/providers/{provider}.ts
 - API route at src/app/api/providers/{provider}/models/route.ts
 - Self-registration via registerProvider()
 **Constraining decisions:**
 - No SDKs - use direct fetch calls
 - API keys from localStorage via provider settings
 </context>
 <tasks>
 <task type="auto">
  <name>Task 1: Create fal.ai provider implementation</name>
  <files>src/lib/providers/fal.ts</files>
  <action>
 Create fal.ai provider implementing ProviderInterface:
 1. Import types from @/lib/providers and @/types
 2. Implement helper function to get API key from localStorage (client) or return null (server)
 3. Implement listModels():
   - Fetch GET https://api.fal.ai/v1/models with optional Key auth header
   - Filter to categories: text-to-image, image-to-image, text-to-video, image-to-video
   - Map models to ProviderModel[]:
     - id: endpoint_id
     - name: metadata.display_name
     - description: metadata.description
     - provider: "fal"
     - coverImage: metadata.thumbnail_url
     - capabilities: [metadata.category as ModelCapability]
   - Return first page only (no pagination traversal)
 4. Implement searchModels(query):
   - Fetch GET https://api.fal.ai/v1/models?q={query}&category=text-to-image (or combined categories)
   - Map and return results same as listModels
 5. Implement getModel(modelId):
   - Fetch GET https://api.fal.ai/v1/models?endpoint_id={modelId}
   - Return first result mapped to ProviderModel
 6. Implement generate() as stub returning { success: false, error: "Not implemented" } (Phase 3)
 7. Implement isConfigured() checking if API key exists (note: fal.ai works without key but with rate limits)
 8. Implement getApiKey() returning key from localStorage or null
 9. Call registerProvider(falProvider) at module level
 Note: fal.ai categories map directly to our ModelCapability type.
  </action>
  <verify>TypeScript compiles without errors: npx tsc --noEmit</verify>
  <done>fal.ts exports working provider, registered in registry when imported</done>
 </task>
 <task type="auto">
  <name>Task 2: Create fal.ai models API route</name>
  <files>src/app/api/providers/fal/models/route.ts</files>
  <action>
 Create Next.js API route for fetching fal.ai models server-side:
 1. Export GET handler
 2. Get optional API key from request header (fal.ai works without but rate limited)
 3. Parse optional `search` query param
 4. Build fetch URL:
   - Base: https://api.fal.ai/v1/models
   - Add ?q={search} if search param provided
   - Add category filter for image/video types only
 5. Add Authorization header if API key provided: Key {apiKey}
 6. Map response.models to ProviderModel[] using same logic as provider:
   - Filter to relevant categories (text-to-image, image-to-image, text-to-video, image-to-video)
   - Map endpoint_id, metadata.display_name, metadata.description, etc.
 7. Return { success: true, models: ProviderModel[] }
 8. Handle errors: return { success: false, error: message }
 Pattern to follow from Replicate API route (02-01).
  </action>
  <verify>
 curl -X GET "http://localhost:3000/api/providers/fal/models" returns JSON with models array (works without key due to fal.ai's optional auth)
  </verify>
  <done>API route returns model list, works with or without API key, filters to image/video categories</done>
 </task>
 </tasks>
 <verification>
 Before declaring plan complete:
 - [ ] `npx tsc --noEmit` passes
 - [ ] `npm run build` succeeds
 - [ ] fal.ai provider file exists and exports provider
 - [ ] API route file exists at correct path
 - [ ] Provider registers when module is imported
 - [ ] Models filtered to image/video capabilities only
 </verification>
 <success_criteria>
 - fal.ai provider implements ProviderInterface
 - Provider self-registers via registerProvider()
 - API route proxies model fetching (optional auth)
 - Models normalized to ProviderModel interface
 - Only image/video models returned (no audio, 3D, etc.)
 - No TypeScript errors
 </success_criteria>
 <output>
 After completion, create `.planning/phases/02-model-discovery/02-02-SUMMARY.md` following the summary template.
 </output>
--- a/.planning/phases/02-model-discovery/02-03-PLAN.md
+++ b/.planning/phases/02-model-discovery/02-03-PLAN.md
@ -0,0 +1,168 @@
 ---
 phase: 02-model-discovery
 plan: 03
 type: execute
 ---
 <objective>
 Add in-memory caching for model lists and create unified models API endpoint.
 Purpose: Reduce API calls to external providers and provide single endpoint for fetching models from all configured providers.
 Output: Caching layer with TTL and unified /api/models endpoint returning models from all providers.
 </objective>
 <execution_context>
 ~/.claude/get-shit-done/workflows/execute-phase.md
 ~/.claude/get-shit-done/templates/summary.md
 </execution_context>
 <context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/02-model-discovery/DISCOVERY.md
 # Prior plan context:
@.planning/phases/02-model-discovery/02-01-SUMMARY.md
@.planning/phases/02-model-discovery/02-02-SUMMARY.md
 # Key files:
@src/lib/providers/types.ts
@src/lib/providers/index.ts
@src/lib/providers/replicate.ts
@src/lib/providers/fal.ts
@src/types/index.ts
 **Tech stack available:**
 - Provider abstraction interfaces
 - Replicate and fal.ai providers implemented
 - API routes for each provider
 **Established patterns:**
 - Provider implementations in src/lib/providers/
 - API routes return { success, models/error }
 **Constraining decisions:**
 - In-memory caching (no Redis)
 - 5-15 minute cache TTL
 </context>
 <tasks>
 <task type="auto">
  <name>Task 1: Create model caching utility</name>
  <files>src/lib/providers/cache.ts</files>
  <action>
 Create simple in-memory cache for model lists:
 1. Define CacheEntry interface: { data: T, timestamp: number }
 2. Create cache Map: Map&lt;string, CacheEntry&lt;ProviderModel[]&gt;&gt;
 3. Define DEFAULT_TTL = 10 * 60 * 1000 (10 minutes)
 4. Export getCachedModels(key: string): ProviderModel[] | null
   - Return null if not in cache or expired
   - Return cached data if within TTL
 5. Export setCachedModels(key: string, models: ProviderModel[]): void
   - Store with current timestamp
 6. Export invalidateCache(key?: string): void
   - If key provided, delete that entry
   - If no key, clear entire cache
 7. Export getCacheKey(provider: ProviderType, search?: string): string
   - Return `${provider}:models` or `${provider}:search:${search}`
 Keep it simple - no LRU eviction, no persistence. Server restarts clear cache.
  </action>
  <verify>TypeScript compiles without errors: npx tsc --noEmit</verify>
  <done>cache.ts exports get/set/invalidate functions for model caching</done>
 </task>
 <task type="auto">
  <name>Task 2: Create unified models API endpoint</name>
  <files>src/app/api/models/route.ts</files>
  <action>
 Create unified API route that fetches models from all configured providers:
 1. Export GET handler
 2. Parse query params:
   - provider: optional, filter to specific provider
   - search: optional, search query
   - refresh: optional, bypass cache if "true"
 3. Get API keys from request headers:
   - X-Replicate-Key: Replicate API key
   - X-Fal-Key: fal.ai API key
 4. Build list of providers to fetch from:
   - If provider param, use only that provider (if key provided)
   - Otherwise, fetch from all providers with keys
 5. For each provider:
   - Check cache first (unless refresh=true)
   - If cache miss, fetch from provider's API route internally or call provider directly
   - Store result in cache
 6. Combine all provider results into single array
 7. Sort by provider, then by name
 8. Return { success: true, models: ProviderModel[], cached: boolean }
 9. Handle partial failures: return models from successful providers, include errors array
 Response format:
 {
  success: true,
  models: ProviderModel[],
  cached: boolean,
  providers: {
    replicate: { success: true, count: 50 },
    fal: { success: true, count: 100 }
  },
  errors?: string[]
 }
  </action>
  <verify>
 curl -X GET "http://localhost:3000/api/models" -H "X-Fal-Key: test" returns combined model list (fal.ai works without key)
  </verify>
  <done>Unified endpoint returns models from all configured providers, uses caching, handles partial failures</done>
 </task>
 <task type="auto">
  <name>Task 3: Add provider module exports</name>
  <files>src/lib/providers/index.ts</files>
  <action>
 Update provider index to export cache utilities and ensure providers auto-register:
 1. Add import for cache utilities: export * from "./cache"
 2. Add dynamic imports comment explaining that providers self-register
 3. Export helper function listAllModels(apiKeys: Record&lt;string, string&gt;): Promise&lt;ProviderModel[]&gt;
   - Takes API keys object
   - Calls listModels() on each configured provider
   - Returns combined array
 4. Export helper function searchAllModels(query: string, apiKeys: Record&lt;string, string&gt;): Promise&lt;ProviderModel[]&gt;
   - Same pattern but calls searchModels()
 Note: These helpers can be used by the unified API route.
  </action>
  <verify>TypeScript compiles without errors: npx tsc --noEmit</verify>
  <done>Provider index exports cache utilities and helper functions for multi-provider operations</done>
 </task>
 </tasks>
 <verification>
 Before declaring plan complete:
 - [ ] `npx tsc --noEmit` passes
 - [ ] `npm run build` succeeds
 - [ ] Cache utility exists with get/set/invalidate functions
 - [ ] Unified /api/models endpoint works
 - [ ] Caching prevents redundant API calls
 - [ ] Multiple providers can be queried in single request
 </verification>
 <success_criteria>
 - In-memory cache with 10-minute TTL
 - Unified /api/models endpoint aggregates all providers
 - Partial failure handling (some providers fail, others succeed)
 - Cache bypass option for manual refresh
 - Phase 2 complete - model discovery working for both providers
 </success_criteria>
 <output>
 After completion, create `.planning/phases/02-model-discovery/02-03-SUMMARY.md` following the summary template.
 Mark Phase 2 complete in summary.
 </output>
--- a/.planning/phases/02-model-discovery/DISCOVERY.md
+++ b/.planning/phases/02-model-discovery/DISCOVERY.md
@ -0,0 +1,197 @@
 # Phase 2 Discovery: Model Discovery APIs
 **Date:** 2026-01-09
 **Level:** 2 - Standard Research
 ## Research Objectives
 1. Replicate model listing API endpoints and response schema
 2. fal.ai model discovery endpoints and response schema
 3. Pagination approaches for both providers
 4. Authentication header formats
 ## Findings
 ### Replicate API
 **Endpoint:** `GET https://api.replicate.com/v1/models`
 **Authentication:**
 ```
 Authorization: Bearer $REPLICATE_API_TOKEN
 ```
 **Query Parameters:**
 - `sort_by`: Sort field (default: `latest_version_created_at`)
 - `sort_direction`: `asc` or `desc`
 **Response Schema:**
 ```typescript
 interface ReplicateModelsResponse {
  next: string | null;      // URL to next page
  previous: string | null;  // URL to previous page
  results: ReplicateModel[];
 }
 interface ReplicateModel {
  url: string;              // Web page link
  owner: string;            // Username/organization
  name: string;             // Model identifier
  description: string;      // Text summary
  visibility: "public" | "private";
  github_url?: string;
  paper_url?: string;
  license_url?: string;
  run_count: number;        // Usage statistics
  cover_image_url?: string; // Thumbnail
  default_example?: object; // Sample prediction
  latest_version?: {
    id: string;
    openapi_schema: object; // Input/output specs
  };
 }
 ```
 **Pagination:** Cursor-based via `next`/`previous` URLs. Follow `next` URL until null.
 **Search Endpoint (Beta):**
 ```
 GET https://api.replicate.com/v1/search?query=...
 ```
 Returns broader search results across models.
 ### fal.ai API
 **Endpoint:** `GET https://api.fal.ai/v1/models`
 **Authentication (Optional - higher rate limits):**
 ```
 Authorization: Key $FAL_KEY
 ```
 **Query Parameters:**
 | Parameter | Type | Description |
 |-----------|------|-------------|
 | `limit` | integer | Max items to return |
 | `cursor` | string | Pagination cursor from previous response |
 | `endpoint_id` | string/array | Specific endpoint ID(s) to retrieve |
 | `q` | string | Free-text search query |
 | `category` | string | Filter by category (e.g., 'text-to-image') |
 | `status` | enum | `active` or `deprecated` |
 | `expand` | string | Include `openapi-3.0` for full schema |
 **Response Schema:**
 ```typescript
 interface FalModelsResponse {
  models: FalModel[];
  next_cursor: string | null;
  has_more: boolean;
 }
 interface FalModel {
  endpoint_id: string;      // e.g., "fal-ai/flux/dev"
  metadata: {
    display_name: string;
    category: string;       // e.g., "text-to-image"
    description: string;
    status: "active" | "deprecated";
    tags: string[];
    updated_at: string;     // ISO8601
    is_favorited: boolean | null;
    thumbnail_url: string;
    model_url: string;
    date: string;           // ISO8601
    highlighted: boolean;
    pinned: boolean;
    thumbnail_animated_url?: string;
    github_url?: string;
    license_type?: "commercial" | "research" | "private";
  };
  openapi?: object;         // When expand=openapi-3.0
 }
 ```
 **Pagination:** Cursor-based. Use `next_cursor` value as `cursor` param. Stop when `has_more` is false.
 **Categories of Interest:**
 - `text-to-image`
 - `image-to-image`
 - `text-to-video`
 - `image-to-video`
 ## Mapping to ProviderModel
 Both APIs need to be normalized to our `ProviderModel` interface:
 ```typescript
 interface ProviderModel {
  id: string;                    // Replicate: `${owner}/${name}`, fal: endpoint_id
  name: string;                  // Replicate: name, fal: display_name
  description: string | null;    // Replicate: description, fal: metadata.description
  provider: ProviderType;        // "replicate" | "fal"
  capabilities: ModelCapability[];// Derived from run type or fal category
  coverImage?: string;           // Replicate: cover_image_url, fal: thumbnail_url
  pricing?: { type, amount, currency };
 }
 ```
 **Capability Mapping:**
 - fal.ai: Direct from `category` field
 - Replicate: Infer from model name/description or openapi_schema input/output types
 ## Implementation Approach
 ### Provider Module Pattern
 Each provider implements `ProviderInterface` and self-registers:
 ```typescript
 // src/lib/providers/replicate.ts
 const replicateProvider: ProviderInterface = {
  id: "replicate",
  name: "Replicate",
  listModels: async () => { /* fetch and map */ },
  searchModels: async (query) => { /* use search endpoint */ },
  getModel: async (id) => { /* fetch specific model */ },
  generate: async (input) => { /* create prediction */ },
  isConfigured: () => !!getApiKey(),
  getApiKey: () => /* from store or env */,
 };
 registerProvider(replicateProvider);
 ```
 ### API Route Pattern
 Expose via Next.js API routes for client access:
 ```typescript
 // src/app/api/providers/[provider]/models/route.ts
 // GET /api/providers/replicate/models?search=flux
 ```
 ### Caching Strategy
 Cache model lists in memory with TTL:
 - **Duration:** 5-15 minutes (models don't change frequently)
 - **Key:** `${provider}:models` or `${provider}:search:${query}`
 - **Invalidation:** Manual refresh button in UI
 ## Decisions
 1. **No SDK usage** - Direct fetch calls to REST APIs as per project constraints
 2. **Server-side fetching** - API routes proxy requests (hides API keys from client)
 3. **Initial page only** - First API call fetches ~50-100 models, pagination on-demand for search
 4. **Category filtering** - Filter to image/video capabilities only (no audio, 3D, etc.)
 ## Next Steps
 1. Create Replicate provider implementation with model fetching
 2. Create fal.ai provider implementation with model fetching
 3. Add API routes for client access to model lists
 4. Implement simple in-memory caching with TTL
 ## Sources
 - [Replicate HTTP API Reference](https://replicate.com/docs/reference/http)
 - [fal.ai Model Search API](https://docs.fal.ai/platform-apis/v1/models)