What you'll learn

By the end of this you'll have a practical starter kit for wiring LLM APIs into a product — not a survey of every provider, but the docs to bookmark, the client pattern to copy, and the guardrails that stop a weekend prototype from becoming a production incident.

You'll know how to pick one provider, wrap it cleanly, log safely, and run a tiny eval before your feature sprawl gets ahead of you.

Who this is for

  • Developers adding their first AI feature to an existing app
  • Backend engineers who've used ChatGPT in a browser but never called an API from code
  • Teams that need a shared baseline before debating fine-tuning vs. RAG vs. agents

Skip this if you're already running production LLM traffic with observability, cost caps, and a regression eval suite. You need tuning guides, not a starter kit.

What is the AI API starter kit? Plain English

It's a curated bundle of decisions, not a downloadable zip. The kit is: one provider to start with, one thin client module in your codebase, one logging convention, one eval file with ten golden examples, and a short list of docs you actually open instead of bookmarking and forgetting.

Plain English: think of it as the minimum viable infrastructure between "I called the API in a script" and "we ship AI features without waking up to a $4,000 invoice."

Prerequisites

  • A working Node.js or Python project (either is fine — patterns transfer)
  • An API key from at least one provider (OpenAI or Anthropic are the usual starting points)
  • Basic comfort with environment variables and async/await

Setup from zero

Step 1 — Pick one provider and commit for two weeks

The starter kit assumes you pick one hosted API and stay with it until you have a working feature and a eval set. Switching providers mid-build doubles integration work and makes debugging harder because you can't tell whether the bug is your prompt or the model.

OpenAI if you want the broadest ecosystem, image endpoints, and the most third-party tooling. Anthropic if long-context reasoning and instruction-following on technical tasks matter most. Both are fine starting points — the kit works with either.

Sign up, create a project-scoped key, store it in process.env (or os.environ), and never commit it. That part isn't optional.

Step 2 — Wrap the client in one module

Don't scatter fetch calls across your codebase. Create a single module — lib/llm.ts or services/llm.py — that owns the provider SDK, default model, timeout, and retry policy.

import OpenAI from "openai";

const client = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

export async function complete(prompt: string, opts?: { model?: string }) {
  const response = await client.chat.completions.create({
    model: opts?.model ?? "gpt-4o-mini",
    messages: [{ role: "user", content: prompt }],
    timeout: 30_000,
  });
  return response.choices[0]?.message?.content ?? "";
}

Every call site imports complete(). When you swap models, add streaming, or switch providers, you change one file.

Step 3 — Log prompts and responses safely

Before you ship anything, decide what gets logged and what never does. The starter kit default:

  • Log: request ID, model, token counts, latency, a hash of the user ID (not the raw ID if PII-sensitive)
  • Never log: full prompts or responses if they may contain user data, API keys, or session tokens
logger.info("llm.complete", {
  model,
  inputTokens: usage?.prompt_tokens,
  outputTokens: usage?.completion_tokens,
  latencyMs: Date.now() - start,
});

If you need prompt logging for debugging, gate it behind an environment flag and redact known secret patterns. "We'll figure out logging later" is how prompts with credit card numbers end up in Datadog.

The mental model

The mental model for AI API integration is: the API call is infrastructure, not the feature.

The feature is what the user experiences — a summary, a classification, a generated draft. The API call is plumbing: it needs timeouts, retries, cost awareness, and eval coverage like any other external dependency. Teams that treat the LLM as magic tend to skip the plumbing until something breaks loudly.

Your wrapper module is the chokepoint. Everything LLM-related flows through it. That single constraint makes every later improvement — caching, fallback models, cost tracking — a one-file change instead of a repo-wide archaeology project.

Key terms

Golden set — a small, fixed list of inputs with expected outputs (or expected properties) you re-run after every prompt or model change. Ten examples is enough to start.

Token — the billing and context unit for LLMs. Roughly four characters of English text per token, but don't guess — read usage from the API response.

Rate limit — provider-enforced cap on requests or tokens per minute. Hitting it returns 429 errors; your client should backoff and retry with jitter.

System prompt — instructions sent to the model before the user message. Sets behaviour for the whole conversation or request.

Streaming — receiving the response token-by-token instead of waiting for the full completion. Better UX for chat; slightly more complex server code.

Step-by-step: what to wire after the wrapper

Add a golden-set eval. Create evals/golden.json with ten real inputs from your feature domain. After any prompt edit, run a script that calls complete() on each input and checks outputs against simple rules (contains expected keyword, valid JSON, under word limit). This takes an afternoon and saves weeks of "did we break summarisation again?"

Set cost and rate guardrails. Track cumulative token usage per day in your logs or a metrics counter. Add a hard cap in your wrapper — if daily spend exceeds threshold, return a graceful error instead of calling the API. For rate limits, implement exponential backoff on 429 responses.

Handle empty and error states in the UI. LLM APIs fail. Timeouts happen. Refusals happen. Your UI needs copy for "couldn't generate a summary — try again" before launch, not after the first support ticket.

Document the one prompt that matters. Put your production system prompt in version control (prompts/summarise.v1.txt), not in a Notion doc someone can't find during an incident.

Patterns

Single-provider-first pattern — integrate deeply with one API before abstracting across providers. Multi-provider wrappers are useful at scale; they're overhead on day one.

Wrapper-as-chokepoint pattern — all LLM calls go through one module. Logging, retries, and model selection live there.

Eval-before-scale pattern — ten golden examples beat a hundred ad-hoc manual tests because they run in CI.

Common mistakes

Calling the API directly from React components. Server-side or route-handler calls keep keys off the client and make logging consistent. Client-side API keys leak in network tabs and bundle inspection.

Skipping timeouts. Default SDK timeouts are often too long or absent. A hung LLM call blocks a serverless function until the platform kills it — bad UX and wasted compute.

Logging full prompts in production. Debug locally with full logs; production gets metadata only unless you've done a proper data classification review.

Optimising prompts before measuring. Run the golden set first. You'll find that half your prompt engineering instinct was wrong once you see actual outputs on real inputs.

Little tip

Start with gpt-4o-mini or claude-3-5-haiku for development and eval runs. Reserve the expensive models for cases the cheap ones fail on your golden set. Your iteration speed goes up and your API bill goes down — same outcome quality at the end.

Little tip

Keep a PROVIDER_DOWNTIME.md snippet in your repo: what to disable in the UI, what cached response to serve, and who to ping. LLM providers have outages. A five-minute doc beats improvising at 2 a.m.

Troubleshooting

429 errors everywhere. You're hitting rate limits. Reduce concurrency, add backoff, or request a limit increase from the provider. Check whether a retry loop is amplifying traffic.

Responses are truncated. You hit max_tokens. Raise the limit or ask the model to be shorter in the system prompt. Truncated JSON is a common silent failure — validate parse before using output.

Latency spikes on cold starts. If you're on serverless, the SDK init + first request can exceed user patience. Warm critical paths or move LLM calls to a dedicated worker with a queue.

Outputs drift after a model version bump. Re-run the golden set. Providers deprecate model strings with little fanfare. Pin model IDs in config and schedule quarterly reviews.

Checklist

  • [ ] One provider chosen; API key in environment variables only
  • [ ] Single wrapper module owns all LLM calls
  • [ ] Timeouts and retry-with-backoff configured on the wrapper
  • [ ] Logging captures tokens and latency; no raw PII in logs
  • [ ] Golden set of at least ten examples with automated checks
  • [ ] Daily cost or token cap with graceful failure
  • [ ] UI empty and error states written before launch
  • [ ] Production system prompt versioned in the repo

Practice task

Take one feature you want to add — summarise a support ticket, classify a tag, draft a reply. Write the wrapper function, three golden examples with expected properties, and a CLI script that runs the eval. Don't build the UI until the eval passes twice in a row after you change the prompt. You'll learn more in that hour than in a week of UI-first prototyping.

FAQ

OpenAI or Anthropic first?
Either. OpenAI has more ecosystem tooling; Anthropic often wins on long-document reasoning. Pick one, ship, eval. You can add the other behind your wrapper later.

Do I need the Vercel AI SDK?
Not on day one. It's excellent for streaming chat UIs in Next.js. Start with the provider SDK in a wrapper; adopt AI SDK when you need streaming React hooks.

When should I add RAG?
When your golden set fails because the model lacks facts it can't infer — your docs, your codebase, your product catalogue. Don't add vector search before you know the base prompt isn't enough.

How big should the golden set be?
Ten is the starter kit minimum. Grow it every time a user report reveals a failure mode. Thirty to fifty is a healthy production set for a single feature.

What to learn next

  • Checklist: ship an AI feature — launch-day privacy, rollback, and override flags
  • Prompt library template — version prompts like code, not like sticky notes
  • Building the Baseline content hub — how we structured typed content modules in Next.js
  • [Checklist: ship an AI feature](/resources/checklist-ship-ai-feature)
  • [Prompt library template](/resources/prompt-library-template)
  • [Building the Baseline content hub](/case-studies/building-baseline-content-hub)

Takeaways

The AI API starter kit is deliberately small: one provider, one wrapper, safe logging, a golden eval, and cost guardrails. That's enough to ship a real feature and enough infrastructure to debug it when something goes wrong.

Don't boil the ocean with multi-provider abstractions on week one. Wrap the client, log metadata, run the golden set, and treat the LLM call like any other external API — because that's what it is.

If you remember only one thing: put every LLM call through one wrapper module with timeouts, logging, and a ten-example eval before you build more features on top. Everything else — streaming, RAG, agents — attaches cleanly to that chokepoint once the basics work.