Issue: Prompt Sensitivity and Engineering

 This phenomenon can be categorized as "Contextual Information Loss in Retrieval-Augmented Generation" or more broadly as issues related to:

1. Information Retrieval Limitations

  • Loss or omission of key contextual details (dates, times) due to imperfect chunking or document selection.

2. Prompt Sensitivity and Engineering

  • Model responses heavily depend on how instructions are phrased—lack of explicit guidance can lead to missing details.

3. Model Capability Differences

  • Variations in model versions, training, or tuning affect how well the model understands and extracts detailed information.

4. Context Window Constraints

  • Limited token/window size restricts how much information the model sees at once, causing some details to be truncated or ignored.

In short:
It’s a mix of context retention challenges in RAG systems, prompt engineering, and model limitations — often referred to as context fragmentation or contextual detail omission in retrieval-based NLP applications.

1. Prompt Clarity and Specificity

  • When you say “list each todo item across all documents” you’re giving a clear but somewhat generic task.

  • Adding “make sure to include date, day, time details” tries to guide the model to extract more granular info, but how well it does depends on how well your prompt guides it and how your documents are formatted.

2. Model Differences & Versioning

  • Your app might be using an older or simpler model (e.g., gpt-3.5-turbo) with less contextual understanding.

  • ChatGPT (on chat.openai.com) often uses the latest models with enhanced reasoning and fine-tuned behavior, plus conversational history that helps it maintain context better.

  • Plus, ChatGPT benefits from better prompt engineering, system instructions, and potentially retrieval-augmented generation (RAG) internally.

3. Context Window and Chunking

  • If your documents are large, your app likely chunks them and only feeds some chunks to the model.

  • This may cause missing context, so details like dates and times may get lost or be in chunks not included in the prompt.

  • ChatGPT probably has a richer internal memory/context or better retrieval from documents.

4. Prompt Engineering and Post-processing

  • Your app’s prompt might be simpler or not optimized for detailed extraction.

  • ChatGPT has complex behind-the-scenes prompt tuning and possibly post-processing to produce clearer, more complete answers.

How to Improve Your App’s Todo Extraction

  • Enhance your prompt with explicit instructions like:
    “For each todo item, list the task plus all date, time, and location details explicitly if available.”

  • Ensure your chunking and retrieval method prioritizes chunks with date/time info.

  • Consider model upgrades or fine-tuning, or switch to newer models like GPT-4 if possible.

  • Add a step for post-processing the model output to format and verify date/time presence.

If you want, I can help you design improved prompts or tweak your retrieval pipeline for better, richer todo extraction!

Comments

Post a Comment

Popular posts from this blog

Feature: Audit log for one login, and identity service

Image
Customer (Product) Page High-Level System Design Diagram (PlantUML) Sequence Diagram: Audit Event Ingestion Sequence Diagram: Orchestration Event For a High-Level System (Component) Diagram The queue is implemented as message_queue (generic), and the audit log service consumes from it, writing to a dedicated audit_log table. The logging service also consumes from the same message_queue and writes to log_store. Orchestration/other services use the same queue. Summary Table: What Writes Where Event Type Goes Into Consumed By Final Storage audit message_queue Audit Log Service audit_log table log message_queue Logging Service log_store table doc_uploaded message_queue Orchestration Worker (business DB, or triggers next event) ocr_ready message_queue Notification Worker, etc. (notification log, etc.)  Core Tables message_queue – universal event bus (temp buffer) audit_log – immutable audit/compliance events log_store – application logs (op...
Read more

Getting started - Build your data science lab environment

This guide explains how to set up a complete data science environment for the udemy program. A proper setup ensures compatibility and avoids recurring environment issues.  1. Overview Goal : Create a full-featured environment with sufficient resources for all course exercises. Recommendation : Use Anaconda for the most reliable setup. Alternative : Use a Python virtual environment with pip if Anaconda isn’t working. 2. Setup Options Primary (Recommended) Create a dedicated Anaconda environment (isolated, compatible). Alternative Use a standard Python virtualenv + pip (faster but less reliable). 3. Windows Setup Steps Clone Course Repository git clone https://github.com/ed-donner/llm_engineering/ Follow the repository’s README.md instructions for additional setup details. Create Anaconda Environment Inside the llm_engineering folder: conda env create -f environment.yml ⏳ This may take several minutes. Activate the environment: conda activate llms Deactivat...
Read more

QA - Run #1 - Results

Image
# QA Report for Tag QA_July2025_A - **Tag/Commit:** QA_July2025_A (`f32abc1`) - **Date:** 2025-08-01 - **QA Outcomes:** All tests passed except for file upload (see below)... - **Environment:** Windows 10, Python 3.12, requirements.txt as of tag - **Notes:** See log for detailed results. ## Issue Found - **Step:** - **Bug:** - **Log excerpt:** - **Screenshot:** For this QA run, we begin with pristine state by: Stop all services Delete logs.db, and jobs.db Delete all files in doc_store folder Start all services Test, you should see no data on index page, and query should result in no data found error Next, go ahead and upload new1.txt, and after that new2.txt Issue the query: "can you summarize all of the todo items you can find, seperating each one on a single line, with due date first, then title, and description with any details you can augment" Expected Answer: - August 1, 2025: Respond to Amira and discuss weekend plans - August 3, 2025: Go to Somesh class and finish ta...
Read more