Course onboarding

Executive Summary

This program begins with immediate, hands-on engagement: participants install and run Large Language Models (LLMs) locally using Ollama, skipping lengthy introductions to deliver value from day one.

Key demonstration: Build a free, local AI language tutor (e.g., Spanish/French) replicating commercial apps — showcasing how open-source LLMs can deliver real business value without SaaS costs.

Strategic Implications

Democratizing AI

  • Local deployment removes cloud dependencies and licensing fees.

  • Enables rapid prototyping and internal innovation.

Cost Efficiency

  • Open-source models reduce recurring costs of paid APIs.

Model Selection as Core Skill

  • Experiment with multiple models (Llama, Qwen, Phi, Gemma) to identify task-specific performance and ROI.

Hardware Performance Awareness

  • Apple M1 vs. PC emulation impacts model speed; informs infrastructure investment decisions.

Program Overview (8 Weeks)

Week 1

  • Explore frontier models (GPT-4o, Claude 3.5, etc.).

  • Build first commercial project using APIs.

Week 2

  • Rapid UI prototyping with Gradio.

  • Multimodal assistants: text, audio, image.

Week 3

  • Open-source deep dive: Hugging Face pipelines and advanced APIs.

Week 4

  • Model benchmarking + selection frameworks.

  • Case study: Python-to-C++ translation for speed gains (60,000x).

Week 5

  • Build Retrieval-Augmented Generation (RAG) pipelines for internal data Q&A.

Weeks 6–8

  • Capstone: Agentic AI solution — multi-agent collaboration, internet search, push notifications.

Immediate Hands-On Setup

Install & Run Ollama

  • Install Ollama (Windows/Mac).

  • Run Llama 3.2 or similar model locally.

First Project

  • Build a language tutor in your chosen language.

  • Replicate commercial features (chat, flashcards) — entirely free and local.

Commercial Value

  • Cost Savings: Replace SaaS subscriptions with in-house capabilities.

  • Rapid Prototyping: Validate ideas quickly without cloud latency.

  • Internal AI Literacy: Build cross-functional skills for engineering, product, and leadership teams.

Environment Setup

Option 1: Anaconda (Recommended)

  • Creates an isolated environment ensuring compatibility with course demos.

conda env create -f environment.yml
conda activate llms

Option 2: Virtualenv (Lightweight Alternative)

  • Use python -m venv venv + pip install -r requirements.txt.

API Keys & Secrets

  • Obtain an OpenAI API key (for frontier models).

  • Store in .env file at project root:

OPENAI_API_KEY=sk-xxxxx

Ensure .env is not committed to Git (gitignore enabled).

First LLM Project: Website Summarizer

Goal: Build a “Reader’s Digest” web browser:

  • Scrape any website.

  • Strip irrelevant elements (scripts, styles).

  • Generate a concise Markdown summary via GPT or Ollama.

Business Use Cases:

  • Summarize industry news.

  • Competitive intelligence from public sites.

  • Financial report condensing.

  • Resume summarization in HR.

Technical Concepts Introduced

  • System vs. User Prompts: Foundation of LLM prompting.

  • Messages Object (Chat API format): Widely adopted across providers.

  • Cost Trade-offs: GPT-4o-mini (cheap) vs. Llama 3.2 (free local).

Upcoming Milestones

  • Day 2: Replace OpenAI API calls with local Ollama inference.

  • Day 3: Benchmark 6 frontier models for speed and quality.

  • Week 2: Prototype multimodal UI (text/audio/image).

  • Week 5: RAG pipeline for proprietary data.

  • Week 8: Deploy agentic AI capstone project.

Action Items

  • Clone course repository.

  • Install Ollama & run llama3.2.

  • Create .env with API keys.

  • Launch JupyterLab and verify environment.

  • Complete first summarizer project.

  • Prepare for rapid prototyping in Week 2.

Comments

Post a Comment

Popular posts from this blog

Feature: Audit log for one login, and identity service

Image
Customer (Product) Page High-Level System Design Diagram (PlantUML) Sequence Diagram: Audit Event Ingestion Sequence Diagram: Orchestration Event For a High-Level System (Component) Diagram The queue is implemented as message_queue (generic), and the audit log service consumes from it, writing to a dedicated audit_log table. The logging service also consumes from the same message_queue and writes to log_store. Orchestration/other services use the same queue. Summary Table: What Writes Where Event Type Goes Into Consumed By Final Storage audit message_queue Audit Log Service audit_log table log message_queue Logging Service log_store table doc_uploaded message_queue Orchestration Worker (business DB, or triggers next event) ocr_ready message_queue Notification Worker, etc. (notification log, etc.)  Core Tables message_queue – universal event bus (temp buffer) audit_log – immutable audit/compliance events log_store – application logs (op...
Read more

Getting started - Build your data science lab environment

This guide explains how to set up a complete data science environment for the udemy program. A proper setup ensures compatibility and avoids recurring environment issues.  1. Overview Goal : Create a full-featured environment with sufficient resources for all course exercises. Recommendation : Use Anaconda for the most reliable setup. Alternative : Use a Python virtual environment with pip if Anaconda isn’t working. 2. Setup Options Primary (Recommended) Create a dedicated Anaconda environment (isolated, compatible). Alternative Use a standard Python virtualenv + pip (faster but less reliable). 3. Windows Setup Steps Clone Course Repository git clone https://github.com/ed-donner/llm_engineering/ Follow the repository’s README.md instructions for additional setup details. Create Anaconda Environment Inside the llm_engineering folder: conda env create -f environment.yml ⏳ This may take several minutes. Activate the environment: conda activate llms Deactivat...
Read more

QA - Run #1 - Results

Image
# QA Report for Tag QA_July2025_A - **Tag/Commit:** QA_July2025_A (`f32abc1`) - **Date:** 2025-08-01 - **QA Outcomes:** All tests passed except for file upload (see below)... - **Environment:** Windows 10, Python 3.12, requirements.txt as of tag - **Notes:** See log for detailed results. ## Issue Found - **Step:** - **Bug:** - **Log excerpt:** - **Screenshot:** For this QA run, we begin with pristine state by: Stop all services Delete logs.db, and jobs.db Delete all files in doc_store folder Start all services Test, you should see no data on index page, and query should result in no data found error Next, go ahead and upload new1.txt, and after that new2.txt Issue the query: "can you summarize all of the todo items you can find, seperating each one on a single line, with due date first, then title, and description with any details you can augment" Expected Answer: - August 1, 2025: Respond to Amira and discuss weekend plans - August 3, 2025: Go to Somesh class and finish ta...
Read more