Month 2 — Real Applications (Wk 5–8)

🚀 AI Engineer Journey — Plan & Trackers · Hope AI — ML & DS Course

Weeks 5–8 · Days 29–56

🎯 Theme: Build Real Applications — Add memory, safety, infrastructure, and deploy to the cloud
Active Tracks: AI · System Design (STARTS!) · Communication · CS

🎯 Month 2 Goals

Deploy your first AI app publicly
Understand AI system design basics
Write and publish first blog post draft
Complete 4 system design discussions
Pass all 4 weekly quizzes

📋 Month 2 Milestone

✅ Deployed AI app + system design basics understood + first blog post draft ready

🧠 Week 5 — Memory & Stateful Conversations

One line: By default LLMs have no memory — every call starts fresh. Memory = storing past messages and injecting them back into every new prompt.

🎯 Analogy: Imagine calling customer support where the agent forgets your entire history every time you call. Frustrating. Memory = giving the LLM a notebook that records the conversation and reads it at the start of each new turn.

🔑 3 Types of Memory

Type	How it works	Best for
Buffer Memory	Keep all messages in a growing list	Short conversations (< 20 turns)
Summary Memory	Summarize old messages to save space	Long conversations
Vector Memory	Store facts in a vector DB, retrieve relevant ones	Very long sessions or many users

🔑 Message Format (How to Pass History to the LLM)

messages = [
  {"role": "system",    "content": "You are a helpful assistant."},
  {"role": "user",      "content": "My name is Arumugam."},
  {"role": "assistant", "content": "Nice to meet you, Arumugam!"},
  {"role": "user",      "content": "What is my name?"}   # LLM now knows!
]

💻 Simple Buffer Memory in Python

history = []

def chat(user_input):
    history.append({"role": "user", "content": user_input})
    response = client.messages.create(
        model="claude-opus-4-6", messages=history, max_tokens=200
    )
    reply = response.content[0].text
    history.append({"role": "assistant", "content": reply})
    return reply

🏗️ Build: Multi-turn chatbot that remembers context across 10+ turns. Test it: say your name at turn 1, then ask "What is my name?" at turn 8 — it must remember.

🧠 Week 6 — Evaluation, Guardrails & Safety

One line: Evaluation = measuring how good your AI outputs are. Guardrails = rules that prevent dangerous, wrong, or unwanted outputs.

🎯 Analogy: You built a car (your AI app). Evaluation = running quality checks before shipping. Guardrails = seat belts and airbags. They do not stop the car from working — they prevent disasters when something goes wrong.

🔑 Key Concepts

Concept	Meaning	Example
Hallucination	LLM states wrong facts confidently	"Paris is in Germany" said with confidence
Prompt Injection	User tricks LLM with malicious input	"Ignore all previous instructions and..."
Output Validation	Check if output matches expected format	Did the LLM return valid JSON?
LLM-as-Judge	Use another LLM to score outputs 1–5	"Rate this answer for accuracy: 1–5"
Guardrail	Rule that blocks or rewrites a bad output	Block any reply that contains a phone number

💻 Simple LLM-as-Judge Evaluator

def evaluate(question, answer, ground_truth):
    prompt = f"""
    Question: {question}
    Answer given: {answer}
    Correct answer: {ground_truth}
    Rate the answer's accuracy from 1 (wrong) to 5 (perfect). Reply with only a number.
    """
    score = client.messages.create(
        model="claude-opus-4-6",
        messages=[{"role": "user", "content": prompt}],
        max_tokens=5
    )
    return int(score.content[0].text.strip())

🏗️ Build: Add an eval layer to your Week 3 PDF Q&A bot — score each answer for accuracy + relevance and log results to a CSV file.

🧠 Week 7 — OS, Networking, DB Fundamentals + Async Python

One line: As an AI engineer you need to know how computers, networks, and databases work — plus how to write non-blocking async Python to call multiple AI APIs in parallel.

🎯 Analogy: You can drive a car without knowing the engine. But to make it faster or fix it, you need to understand what is under the hood. These fundamentals = understanding the engine of every AI app you build.

🔑 What You Need to Know

Topic	Key Concept	Why It Matters for AI
OS	Processes, threads, memory	LLM inference uses GPU memory and CPU threads
Networking	HTTP, REST, APIs, TCP/IP	You call AI APIs over HTTP — understand requests, responses, headers
Databases	SQL vs NoSQL, indexing, queries	Store chat history, user data, embeddings
Async Python	async/await, event loop, concurrency	Call 10 AI APIs simultaneously without waiting for each one

💻 Async API Calls — 3× Faster Than Sequential

import asyncio
import anthropic

async def ask(question):
    client = anthropic.AsyncAnthropic()
    response = await client.messages.create(
        model="claude-opus-4-6", max_tokens=100,
        messages=[{"role": "user", "content": question}]
    )
    return response.content[0].text

# Run 5 questions in parallel — same time as running 1
questions = ["Q1", "Q2", "Q3", "Q4", "Q5"]
answers = asyncio.run(asyncio.gather(*[ask(q) for q in questions]))

🏗️ Build: Async batch processor — read 20 questions from a CSV, answer all in parallel using async, save results to an output CSV. Measure time vs sequential to see the speedup.

🧠 Week 8 — Cloud, DevOps & Deployment

One line: You need to deploy your AI apps so others can use them. This means Docker (packaging), FastAPI (REST API), and Railway or Render (cloud hosting).

🎯 Analogy: You baked a cake (your app). Deployment = delivering it to everyone's home. Docker = sealing the cake in a box so it doesn't break in transit. Cloud = the delivery truck. CI/CD = automatically re-baking every time you update the recipe.

🔑 Key Tools

Tool	What it does	One-line command
Docker	Packages your app + all dependencies	`docker build . && docker run -p 8000:8000 app`
FastAPI	Turns Python functions into REST API endpoints	`@app.post("/chat")` → HTTP endpoint
Railway / Render	One-click cloud deployment from GitHub	Push to GitHub → auto-deploys in 2 min
GitHub Actions	Auto-test and deploy on every code push	YAML file in `.github/workflows/`
Environment Variables	Keep API keys out of your code	`os.getenv("ANTHROPIC_API_KEY")`

💻 FastAPI + Docker in 5 Minutes

# main.py
from fastapi import FastAPI
import anthropic

app = FastAPI()
client = anthropic.Anthropic()

from pydantic import BaseModel

class ChatRequest(BaseModel):
    message: str

@app.post("/chat")
def chat(request: ChatRequest):
    response = client.messages.create(
        model="claude-opus-4-6", max_tokens=200,
        messages=[{"role": "user", "content": request.message}]
    )
    return {"reply": response.content[0].text}

# Dockerfile
FROM python:3.11-slim
WORKDIR /app
COPY . .
RUN pip install fastapi anthropic uvicorn
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

🏗️ Build: Deploy your PDF Q&A bot as a live REST API on Railway. Share the public URL. This is your first publicly deployed AI app. 🚀

📅 Week Pages — Detailed Daily Checklists

📆 Week 5 — Memory & Stateful Conversations
📆 Week 6 — Evaluation, Guardrails & Safety
📆 Week 7 — OS, Networking, DB Fundamentals + Async Python
📆 Week 8 — Cloud, DevOps & Deployment

← 📅 Month 1 — Foundation (Wk 1–4)📅 Month 3 — Agentic AI + DSA (Wk 9–12) →