Lesson · ⏱ 8 min read

🔌 Calling an LLM API: keys, requests and costs

Q: Where should your API key live?

In an environment variable or secret manager. Keys in code end up on GitHub and get stolen. Use environment variables or a secrets manager.

🏠 Real-life example

Ordering on Zomato: you send a request (what you want and where), the restaurant processes it, and you get a response (the food). You pay per order, and your account login proves who you are. An LLM API works the same way: your code sends a request with messages, the model replies, you're billed per token, and the API key is your login.

💡 In simple words

Providers like Anthropic and OpenAI let you use their models over the internet through an API. You send a list of messages (system and user) plus settings like model name, max tokens and temperature, and get back the generated text. Keep API keys secret: put them in environment variables, never in code or GitHub. Plan for errors and rate limits with retries and timeouts. Streaming sends the answer word by word so users aren't kept waiting. Track cost: input tokens plus output tokens multiplied by the price per token.

🪜 Step by step

Create an API key in the provider's console and store it in an environment variable, never in code.
Send a request with the model name, a system message (instructions) and user messages; read the text from the response.
Log input and output token counts. Cost = tokens × price per token, so you can estimate cost per user.
Handle failures: set a timeout, retry rate limit (429) and server errors with exponential backoff, and show a friendly message if all retries fail.
Stream long answers so users see text immediately, and set a max token limit to control cost.

🔑 Key terms

API key: A secret password that identifies your account to the API.
Rate limit: The maximum number of requests allowed per minute.
Streaming: Receiving the reply piece by piece as it's generated.

👀 See it in action

python

import os, anthropic
client = anthropic.Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])

msg = client.messages.create(
    model="claude-sonnet-5-5",
    max_tokens=300,
    system="You are a concise travel assistant for Indian travellers.",
    messages=[{"role": "user", "content": "3-day Jaipur plan on a budget?"}],
)
print(msg.content[0].text)
print(msg.usage)   # input/output tokens → cost

The key comes from an environment variable, never pasted into code.

⚠️ Common mistakes to avoid

Committing API keys to GitHub. Bots find them within minutes; rotate the key immediately if it happens.
Calling the API from browser code, which exposes your key. Call it from your server.
No usage limits. A bug or abuse can create a huge bill; set budgets and per user limits.

✅ Quick check

Where should your API key live?

In an environment variable or secret manager
In the README
Hard-coded in app.py

Show answer

A. In an environment variable or secret manager: Keys in code end up on GitHub and get stolen. Use environment variables or a secrets manager.

🙋 Questions learners ask

Which model should I start with?

Start with a strong general model to prove the feature works, then test smaller, cheaper models on your eval set and switch where quality holds.

Is my data used for training?

Policies differ by provider and plan. Business API plans usually don't train on your data by default, but always check the current terms.

📌 Remember

Request in → response out, billed per token. Guard the key, and handle errors.

📖 Words from this lesson (AI dictionary)

Learn this as part of a full course

Track your progress, take the quick checks and follow the lessons in the right order. This lesson is part of:

AI Foundations Path · AI Product Engineer · LLM / Agent Engineer · AI Forward Deployed Engineer

Continue in the course