# 03 — LLM Providers (GPT-4o mini & Claude)

How to wire **OpenAI GPT-4o mini** and **Anthropic Claude** for RAG Q&A and social caption generation.

---

## 1. Current state

| Role | Config | Code |
|------|--------|------|
| Text | `AI_PROVIDER` = `openai` \| `anthropic` | `src/lib/ai/config.ts`, `providers/*` |
| Default text (often) | Anthropic Claude Opus-class model | `.env.example` / generated posts samples |
| OpenAI text model | e.g. `gpt-4o` via `OPENAI_MODEL` | env |
| Images | OpenAI / Gemini / placeholder | `image-provider-chain.ts` |
| Cost table | Includes gpt-4o-mini rates | `src/lib/ai/cost-tracking.ts` |

There is **no separate RAG chat provider switch** yet — only generation text provider.

---

## 2. Target provider matrix

| Workload | Preferred model | Why |
|----------|-----------------|-----|
| RAG chat / Q&A | **gpt-4o-mini** | Low latency & cost for many queries |
| Caption generation (quality) | **Claude** (Sonnet/Opus per budget) | Brand voice, longer coherence |
| Caption generation (budget) | **gpt-4o-mini** | High volume auto-loop |
| Embeddings | **text-embedding-3-small** | Standard OpenAI embeddings |
| Image | Existing OpenAI/Gemini chain | Unchanged |
| Video | FFmpeg pipeline | Unchanged |

---

## 3. Environment variables (proposed)

```bash
# Existing
AI_PROVIDER=anthropic          # default caption provider
ANTHROPIC_API_KEY=
ANTHROPIC_MODEL=claude-sonnet-4-20250514
OPENAI_API_KEY=
OPENAI_MODEL=gpt-4o

# New / explicit
RAG_CHAT_PROVIDER=openai       # openai | anthropic
RAG_CHAT_MODEL=gpt-4o-mini
CAPTION_PROVIDER=anthropic     # override AI_PROVIDER for captions only
CAPTION_MODEL=claude-sonnet-4-20250514
OPENAI_MINI_MODEL=gpt-4o-mini
```

Keep backward compatibility: if new vars absent, fall back to `AI_PROVIDER` + `OPENAI_MODEL` / `ANTHROPIC_MODEL`.

---

## 4. Code structure (proposed)

Extend `src/lib/ai/`:

```
src/lib/ai/
  config.ts              # add getRagChatConfig(), getCaptionConfig()
  providers/
    openai.ts            # ensure model param supports gpt-4o-mini
    anthropic.ts
  generate-text.ts       # existing facade — pass model override
```

RAG query:

```
src/lib/rag/query.ts
  → retrieve()
  → generateText({ provider, model, messages: groundedPrompt })
```

---

## 5. Request-level override

Allow clients to choose:

```json
{ "provider": "openai-mini" }
{ "provider": "claude" }
{ "provider": "openai" }
```

Map:

| Alias | Provider | Model env |
|-------|----------|-----------|
| `openai-mini` | openai | `OPENAI_MINI_MODEL` |
| `openai` | openai | `OPENAI_MODEL` |
| `claude` | anthropic | `CAPTION_MODEL` or `ANTHROPIC_MODEL` |

---

## 6. Prompting rules (both providers)

Shared with product:

1. Use only retrieved + project context.  
2. No invented metrics/clients/offers.  
3. Platform-aware length (LinkedIn vs Facebook).  
4. Return plain text captions unless JSON schema requested.  
5. Log `provider`, `model`, tokens, cost via existing `usage-logger` / `cost-tracking`.

---

## 7. Failure & fallback

| Failure | Behavior |
|---------|----------|
| Primary provider timeout | Retry once → optional fallback provider |
| Embedding failure | Skip RAG; use push-context only; warn in response |
| Rate limit | 429 to client; backoff in auto-post path |

---

## 8. Cost posture

| Path | Model | Guidance |
|------|-------|----------|
| Auto-post high volume | gpt-4o-mini | Default for loop-sequence |
| Manual “Generate” premium | Claude | User-facing quality |
| RAG chat | gpt-4o-mini | Default |
| Reindex embeddings | text-embedding-3-small | Batch off peak |

Track in `ai_usage_logs` with `operation` = `rag_query` | `rag_embed` | `caption` | `image`.

---

## 9. Acceptance criteria

- [ ] RAG query works with GPT-4o mini  
- [ ] RAG query works with Claude  
- [ ] Caption path can select Claude or mini independently of RAG chat  
- [ ] Usage logs distinguish operations  
- [ ] Missing API key returns clear 503 with provider name