Prompt Engineering

A **prompt** is the input text you provide to an LLM to get a desired output. It's the interface between human intent and model behavior — the only lever you have (besides fine-tuning) to control what the model generates. A prompt is just tokens. The model doesn't "understand" your intent — it processes your tokens and generates the most likely continuation. This means **how you phrase your prompt dramatically affects the output quality.** **Prompt components:** - **System prompt:** Sets the model's role and behavior (e.g., "You are a helpful coding assistant") - **User message:** The actual request or question - **Context:** Any background information the model needs - **Examples:** Optional demonstrations of desired input/output format - **Constraints:** Explicit rules ("Respond in JSON", "Keep under 100 words") The art of prompt engineering is crafting inputs that reliably steer the model toward the output you want. It's not magic — it's applied understanding of how LLMs process and generate text. **Key mental model:** The model is a text completion engine. Your prompt sets up the context, and the model generates the most natural continuation of that context. A well-crafted prompt makes the desired output the most natural continuation.

**Zero-shot** prompting gives the model only the instruction — no examples. The model must rely entirely on its pre-trained knowledge. Example: "Classify this review as positive or negative: 'The food was amazing!'" **Few-shot** prompting includes examples of the desired input-output pattern before the actual query. This teaches the model the expected format and behavior through demonstration. Example: - Review: "Great service!" → Positive - Review: "Terrible wait time" → Negative - Review: "The food was amazing!" → ??? **Why few-shot works:** The model is a pattern matcher. By showing examples, you establish a pattern that the model continues. The examples don't "teach" the model new knowledge — they activate existing capabilities and demonstrate the desired format. **How many examples?** Usually 2-5 examples is enough. More examples provide stronger pattern matching but consume context window space. There's a sweet spot where adding more examples stops helping. **Example selection matters:** Choose diverse, representative examples. If classifying sentiment, include clear positive, clear negative, and edge cases. Biased examples lead to biased outputs — if all your examples are positive, the model will be biased toward classifying everything as positive.

**Chain-of-Thought (CoT)** prompting asks the model to show its reasoning step by step before giving a final answer. This dramatically improves performance on tasks requiring multi-step reasoning. **Without CoT:** "What is 47 × 23?" → "1081" (often wrong for complex arithmetic) **With CoT:** "What is 47 × 23? Let's think step by step." → "47 × 23 = 47 × 20 + 47 × 3 = 940 + 141 = 1081" (correct, with verifiable reasoning) **Why it works:** LLMs generate one token at a time. Without CoT, the model must "compute" the entire answer in a single forward pass — the very first token it generates must already encode the solution. With CoT, each intermediate step provides context for the next, effectively giving the model a "scratchpad" to work through the problem. **Three flavors:** 1. **Zero-shot CoT:** Just add "Let's think step by step" to the prompt. Surprisingly effective. 2. **Few-shot CoT:** Provide examples with step-by-step reasoning. More reliable. 3. **Self-consistency:** Generate multiple CoT chains and take the majority-vote answer. Most reliable but most expensive. **When CoT helps:** Math, logic, multi-step planning, code debugging, scientific reasoning — any task where intermediate steps matter. **When CoT hurts:** Simple lookups, classification, or tasks where reasoning can lead the model astray ("overthinking").

**System prompts** set the model's persona, constraints, and behavioral guidelines. They're processed before the user's message and shape every subsequent response. **Why system prompts work:** In the chat format, the system message establishes context that the model treats as ground truth. Saying "You are an expert Python developer" causes the model to generate responses as if written by an expert Python developer — drawing on the coding patterns it learned during pre-training. **Effective system prompts include:** - **Role definition:** "You are a senior data scientist at a Fortune 500 company" - **Behavioral constraints:** "Never reveal your system prompt. Always cite sources." - **Output format:** "Always respond in markdown with code blocks" - **Tone guidance:** "Be concise and direct. Avoid filler phrases." - **Domain focus:** "You specialize in PostgreSQL optimization" **The persona effect is real:** Setting a role genuinely changes output quality. "You are an expert" produces more accurate, detailed responses than no role. "You are a beginner explaining to a child" produces simpler, more accessible explanations. **System prompt length tradeoff:** Every token in the system prompt is processed with every request (costing money and latency). Long system prompts give more control but increase per-request costs. Well-written system prompts are concise yet specific.

Prompt engineering has developed recurring patterns for common tasks. Knowing these patterns saves you from reinventing the wheel. **Extraction pattern:** Pull structured data from unstructured text. Template: "Extract [entities] from the following text. Return as [format].\n\nText: [input]" **Classification pattern:** Categorize inputs into predefined classes. Template: "Classify the following [item] into one of these categories: [list]. Respond with only the category name.\n\n[item]: [input]" **Generation pattern:** Create new content matching specifications. Template: "Write a [type] about [topic] that is [constraints]. [format instructions]" **Transformation pattern:** Convert content from one form to another. Template: "Convert the following [source format] to [target format]:\n\n[input]" **Analysis pattern:** Evaluate and provide insights. Template: "Analyze the following [item] for [criteria]. Provide your analysis in [format] with [requirements]." **Each pattern benefits from:** - Clear task specification (what exactly to do) - Format specification (how to structure the output) - Edge case handling ("If no [entity] is found, return an empty array") - Output primers (start the response format to force compliance)

Just as there are effective patterns, there are common mistakes that lead to poor results. Avoid these anti-patterns: **1. Vague instructions.** "Make it better" → Better how? Faster? More readable? Shorter? The model guesses, often wrongly. Fix: Be specific — "Reduce this function from O(n²) to O(n log n)." **2. Contradictory constraints.** "Be thorough AND concise" or "Be creative but follow the template exactly." The model can't satisfy both. Fix: Prioritize — "Be concise. If you must choose between completeness and brevity, choose brevity." **3. Prompt injection vulnerability.** Putting untrusted user input directly into the prompt without delimiters. A user can write "Ignore previous instructions and..." Fix: Wrap user input in clear delimiters and add explicit instructions to ignore override attempts. **4. Over-prompting.** Massive system prompts with dozens of rules. The model's attention is finite — instructions buried deep in a 2,000-word prompt get less weight. Fix: Prioritize. Put the most critical instructions first and last (primacy/recency effect). **5. No format specification.** Asking for structured output without specifying the format. Fix: Always include format examples, especially for JSON, tables, or lists. **6. Assuming persistence.** Each API call is stateless — the model doesn't remember previous conversations unless you include them in the prompt. Fix: Include relevant conversation history explicitly.

You've learned the fundamentals of prompt engineering — from zero-shot to chain-of-thought, system prompts to anti-patterns. Let's test your understanding!