AI Engineering for Developers

Day 2

Programming Foundations + Prompt Engineering Mastery

"Code is no longer just logic-it's conversation. Master the art of talking to AI."

A Note on This Chapter's Structure
You'll notice something different about Day 2. We dive straight into token costs and API optimization before formally explaining LLMs in Day 3. This isn't an oversight. It's intentional.

Here's why: When developers start building with AI, the first shock comes from your AWS bill, not from understanding transformer architecture. You spin up a quick test, forget to add a loop guard, and wake up to a $500 API charge. This happened to me. It happens to everyone.

This book mirrors how you'll actually learn AI in the real world. You encounter problems first, then seek understanding. Day 2 arms you with practical cost-saving techniques you need immediately. Day 3 gives you the deep knowledge of how LLMs work. Day 5 circles back to traditional ML because you'll need it after you understand where LLMs excel and where they fail.

If you prefer learning foundations first, feel free to read Day 3 before Day 2. Both approaches work. I chose the "problems-first" path because it matches how experienced developers actually build things. We solve immediate problems, then fill knowledge gaps as needed.

Now, let's talk about the Python patterns that will save you money and headaches.

Section 1: Essential Python for AI Engineering

Python Fundamentals Refresher

A

s an experienced developer, you already know Python's basics. What changes in AI development is how you use these fundamentals. AI applications process massive amounts of text, handle concurrent API calls, and manage memory differently than traditional web applications. Let's explore these patterns with clear examples.

Data Structures for AI: Lists, Dictionaries, Sets, Queues

In AI applications, choosing the right data structure directly impacts both performance and cost. Since AI APIs charge by the token (roughly 4 characters), every character you send costs money. Let's see how different data structures help optimize AI applications.

Lists are perfect for managing conversation history with AI models. Here's a simple example:

# Basic conversation history
messages = [
{"role": "user", "content": "What is Python?"},
{"role": "assistant", "content": "Python is a programming language..."},
{"role": "user", "content": "Can you give an example?"}
]

This structure works perfectly with AI APIs. The "role" tells the AI who said what, and "content" contains the actual message. But here's the problem-as conversations grow, so does your API cost. Let's optimize:

# Smart conversation management
class ConversationHistory:
def __init__(self, max_messages=10):
self.messages = []
self.max_messages = max_messages

def add_message(self, role, content):
# Add new message
self.messages.append({"role": role, "content": content})

# Keep only recent messages to save tokens
if len(self.messages) > self.max_messages:
# Remove the oldest message
self.messages.pop(0)

def get_context(self):
# Return messages for API
return self.messages

This class automatically limits conversation history. Why? Because sending 50 messages to an AI API might cost $0.50, while sending the last 10 messages costs $0.10. The AI still understands context, but you save 80% on costs.

Dictionaries perform well at managing multiple AI configurations. Different tasks need different AI models:

# AI model configurations
models = {
"quick_task": {
"name": "gpt-4o-mini",
"max_tokens": 500,
"temperature": 0.7,
"cost_per_1k": 0.002 # $0.002 per 1000 tokens
},
"complex_task": {
"name": "gpt-4",
"max_tokens": 2000,
"temperature": 0.3,
"cost_per_1k": 0.03 # $0.03 per 1000 tokens
}
}

# Choose model based on task
def select_model(task_complexity):
if task_complexity < 5:
return models["quick_task"]
else:
return models["complex_task"]

This pattern lets you use expensive models only when necessary. A simple question uses the cheap model, while complex analysis uses the powerful (expensive) model.

Sets help avoid duplicate processing. Many AI applications receive similar questions repeatedly:

# Track processed queries to avoid duplicates
processed_queries = set()

def should_process(query):
# Normalize the query (remove extra spaces, lowercase)
normalized_query = query.strip().lower()

# Check if we've seen this before
if normalized_query in processed_queries:
print(f"Already processed: {query}")
return False

# Add to set and process
processed_queries.add(normalized_query)
return True

# Example usage
queries = [
"What is AI?",
"what is AI?", # Duplicate (different case)
" What is AI? ", # Duplicate (extra spaces)
"How does AI work?" # New query
]

for query in queries:
if should_process(query):
print(f"Processing new query: {query}")
# Make expensive API call here

Sets use hash tables internally, making lookups extremely fast even with thousands of entries. This simple optimization can cut API costs dramatically for applications with repetitive queries.

Async Programming for AI Applications

Traditional synchronous code kills AI application performance. When you call an AI API, your program waits 2-5 seconds for a response. During this time, your CPU does nothing. Async programming fixes this by handling multiple requests simultaneously.

Let's start with the problem (synchronous code):

import time
import openai

def get_ai_response(prompt):
# This takes 2-3 seconds
response = openai.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": prompt}]
)
return response.choices[0].message.content

# Process 3 questions synchronously
start = time.time()
questions = ["What is Python?", "What is JavaScript?", "What is Go?"]

for question in questions:
answer = get_ai_response(question)
print(f"Q: {question}")
print(f"A: {answer[:50]}...\n")

print(f"Total time: {time.time() - start:.1f} seconds")
# Output: Total time: 8.5 seconds (2-3 seconds per question)

Now let's fix this with async programming:

import asyncio
import time
from openai import AsyncOpenAI

# Create async client
client = AsyncOpenAI()

async def get_ai_response_async(prompt):
# Async version of the same function
response = await client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": prompt}]
)
return response.choices[0].message.content

async def process_questions():
questions = ["What is Python?", "What is JavaScript?", "What is Go?"]

# Create all tasks at once
tasks = []
for question in questions:
task = get_ai_response_async(question)
tasks.append(task)

# Wait for all to complete
start = time.time()
answers = await asyncio.gather(*tasks)

# Print results
for question, answer in zip(questions, answers):
print(f"Q: {question}")
print(f"A: {answer[:50]}...\n")

print(f"Total time: {time.time() - start:.1f} seconds")
# Output: Total time: 2.8 seconds (all processed in parallel!)

# Run the async function
asyncio.run(process_questions())

The magic happens in asyncio.gather(*tasks). This runs all three API calls simultaneously. Instead of waiting 8.5 seconds, we wait only as long as the slowest request (about 2.8 seconds). This 3x speedup becomes even more dramatic with more requests.

Error Handling and Exception Management

AI APIs fail in unique ways. Rate limits, token limits, content filters, and service outages all require different handling strategies. Good error handling keeps your application running when things go wrong.

Here's a comprehensive error handling pattern that works across different AI providers:

import time
from openai import RateLimitError as OpenAIRateLimitError
from anthropic import RateLimitError as AnthropicRateLimitError

# Custom exceptions for unified handling
class AIError(Exception):
"""Base class for AI-related errors"""
pass

class RateLimitError(AIError):
"""Too many requests"""
pass

class TokenLimitError(AIError):
"""Request too long"""
pass

class ContentFilterError(AIError):
"""Content blocked by safety filter"""
pass

def call_ai_api(prompt, provider="openai"):
"""Wrapper that translates...