Your comprehensive reference for advanced AI interaction
Source: Lenny's Podcast - AI prompt engineering in 2025: What works and what doesn't | Sander Schulhoff (Learn Prompting, HackAPrompt) Listen: https://open.spotify.com/episode/3Bb7d0EcVazeZ8t0CEQrop?si=G0RO_d89QKWEDP4H9PZUBw
What it is: Instead of telling the AI what you want, show it actual examples of exactly what good output looks like.
Why this is so much more powerful: Imagine you're teaching someone to write professional emails. You could either:
Option B is obviously better because they can see the exact tone, structure, length, and style. The AI works exactly the same way - examples are worth a thousand words of description.
The simple process:
Real Example - Email Writing:
❌ BEFORE (vague description):
Write a professional email declining a meeting request. Make it polite but brief.Result: AI has to guess what "professional," "polite," and "brief" mean to you
✅ AFTER (concrete examples):
Here are examples of how I write emails:
Q: How do I reschedule a meeting?
A: Hi Sarah, I need to reschedule our Tuesday meeting due to a conflict. Would Thursday at the same time work? Thanks!
Q: How do I decline a meeting?
A: Hi John, I won't be able to make the meeting tomorrow. Could we find an alternative time next week? I appreciate your understanding.
Q: How do I decline the budget review meeting on Friday?
A: [AI will write in exactly the same style as your examples]Result: AI can see your pattern - brief, starts with "Hi [name]," one sentence explanation, offers alternative, ends with appreciation
Another Example - Customer Support:
❌ BEFORE:
Respond to customer complaints professionally and helpfully.✅ AFTER:
Here's how we respond to customer complaints:
Customer: "My order is 3 days late and I'm frustrated."
Response: "I completely understand your frustration with the delay. Let me track your order right now and get you an update. I'll also see what we can do to make this right."
Customer: "The product I received is damaged."
Response: "I'm so sorry the product arrived damaged. That's definitely not the experience we want for you. I'll arrange a replacement to be sent out today and email you the tracking info."
Customer: "Your website is confusing and I can't find my order."
Response: [AI will respond in the same helpful, empathetic style]The magic: You don't have to explain abstract concepts like "be empathetic but solution-focused" - the AI sees the exact pattern from your examples.
What it is: Instead of asking the AI to solve a complex problem all at once, first ask it to break the problem into smaller sub-problems.
Why it works: As Sander explained with the car dealership example: "if you just ask the models to do all that at once, it might struggle. But if you tell it, hey, what are all the things that need to be done first, just like what a human would do."
The rationale: Complex tasks often fail because the AI tries to do everything simultaneously. Breaking it down helps both you and the AI think through the problem systematically.
The magic phrase: "Before answering this, tell me what sub-problems need to be solved first?"
Step-by-step process:
Real example from the podcast:
Customer: "I checked out this car on this date. Or actually it might have been this other date and it was this type of car or actually it might have been this other type of car. And anyways, it has this small ding and I want to return it. What's your return policy?"
Instead of trying to answer directly, the AI breaks it down:
1. Is this even a customer? (run database check)
2. What kind of car do they have?
3. What date did they check it out?
4. Do they have insurance on it?
5. What's the return policy for their situation?
6. Calculate any fees or refundsYour implementation:
You: "I need to plan a product launch. Before answering this, tell me what sub-problems need to be solved first?"
AI: Lists out: target audience definition, competitive analysis, pricing strategy, marketing channels, timeline planning, resource allocation, success metrics, etc.
You: "Great, now let's solve each of these one by one, starting with target audience definition..."What it is: Get the AI to check and improve its own work before you see the final result.
Why it works: As Sander described: "it outputs something. You get it to criticize itself and then to improve itself. And so these are a pretty notable set of techniques because it's like a free performance boost that works in some situations."
The rationale: The AI often knows what makes a good response but doesn't apply that knowledge on the first try. Having it review its own work activates that quality-checking capability.
Simple 3-step process:
Example:
You: "Write a brief for a new marketing campaign"
AI: [Provides initial brief]
You: "Can you go back and check your response? Offer yourself some criticism"
AI: "Looking at my response, I notice I didn't include specific metrics for success, the target audience is too broad, and I didn't consider budget constraints..."
You: "Great criticism. Now implement that feedback"
AI: [Provides improved brief with metrics, focused audience, budget considerations]Important limitation: Use 1-3 times maximum. Sander noted: "I think the model would kind of go crazy at some point" if you keep doing this indefinitely.
What it is: Front-load your prompt with relevant background information about your task, situation, or domain.
Why it works: Sander's research example showed dramatic results: "I took the original email the professor had sent me describing the problem and pasted that into the prompt. And it performed pretty well... So I took the email out and the performance dropped off a cliff without that context."
The rationale: The AI needs context to make good decisions, just like humans do. More relevant information = better, more tailored responses.
What to include as context:
Placement strategy: Always put context at the beginning for two reasons:
Real example:
CONTEXT: I'm a product manager at a B2B SaaS company. We sell project management software to teams of 10-50 people. Our main competitors are Asana and Monday.com. We're launching a new integration feature next quarter.
TASK: Write a launch announcement email to our existing customers.How much context:
What it is: Ask the same exact question multiple times using different approaches, then pick the answer that comes up most often.
Simple analogy: Imagine you're stuck on a difficult problem and you ask 5 different experts for help. Three experts say the answer is "A" and two say it's "B". You'd probably trust "A" since most experts agreed on it.
This technique does the same thing with AI - you ask the same question in 5 different ways and see which answer appears most frequently. That's usually the most reliable answer.
Why this works: Different prompting approaches make the AI "think" differently:
When to use this: Only for high-stakes situations where accuracy really matters - important business decisions, research, fact-checking. This technique costs more since you need multiple API calls.
Step-by-step process:
Step 1: Create 3-5 different "expert" approaches for the same question
Step 2: Ask each expert the same question separately
Step 3: Compare all the answers
Step 4: Go with the answer that appears most frequently
Real example from Sander's podcast:
Question: "How many trophies does Real Madrid have?"
Expert 1 (Soccer historian role): "Real Madrid has 13 major trophies"
Expert 2 (English professor role): "I believe it's 4 trophies"
Expert 3 (Internet search approach): "Real Madrid has won 13 major trophies"
Results: Two said "13", one said "4"
Final answer: 13 trophies (most common answer wins)Practical business example:
Question: "What's the best investment strategy for someone in their 30s?"
EXPERT 1 (Role-based):
"You are a financial advisor. What's the best investment strategy for someone in their 30s?"
EXPERT 2 (Data-focused):
"Based on historical market data and economic research, what investment strategy works best for people in their 30s?"
EXPERT 3 (Case study approach):
"Looking at successful investment cases and real examples, what strategy should someone in their 30s follow?"
EXPERT 4 (Risk-focused):
"Considering risk management and safety, what investment approach should a 30-year-old take?"
EXPERT 5 (Direct question):
"What's the best investment strategy for someone in their 30s?"Then look at all 5 answers and see which specific recommendations appear most often across the different approaches.
The key insight: You're not asking the same question 5 times. You're asking it in 5 different ways that make the AI approach the problem from different angles. The answer that emerges regardless of approach is usually the most reliable.
What it is: Tell the AI exactly what format you want the response in, rather than letting it choose.
Why it works: Ensures consistency and makes outputs much easier to use, especially when you need to process the information further or share it with others.
The rationale: Without format guidance, the AI might give you a paragraph when you need bullet points, or prose when you need data you can put in a spreadsheet.
JSON Example:
Return your analysis in this exact JSON format:
{
"summary": "Brief 2-sentence summary",
"key_insights": ["insight 1", "insight 2", "insight 3"],
"confidence_score": 1-10,
"next_steps": ["action 1", "action 2"]
}Business Analysis Template:
Format your response as:
**SITUATION:** [Brief context]
**PROBLEM:** [Core issue]
**OPTIONS:** [2-3 alternatives]
**RECOMMENDATION:** [Best choice with reasoning]
**IMPLEMENTATION:** [Specific next steps]When to use: Any time you need consistent formatting, are processing multiple similar requests, or need to use the output in another system.
What it is: Explicitly tell the AI what NOT to do or include, rather than just what you want.
Why it works: Prevents the most common failure modes you've experienced and keeps responses focused on what actually matters.
The rationale: The AI doesn't know your specific context, constraints, or what you want to avoid unless you tell it directly.
Length Constraints:
Provide a summary in exactly 3 bullet points, each under 25 words.
Do not exceed this limit or add additional points.Content Boundaries:
Analyze this business strategy.
DO NOT:
- Mention specific competitors by name
- Include financial projections beyond 12 months
- Use technical jargon without explanations
- Make recommendations requiring >$50K investmentScope Limitations:
Focus only on marketing implications.
Ignore operational, financial, or technical considerations for this analysis.What it is: Define your communication style through examples and descriptors, not roles.
Why it works: This is different from the "role prompting" that doesn't work. Instead of saying "you are a copywriter," you show the AI the actual style you want through examples.
The rationale: Style and tone are about expression (where examples work great), not accuracy (where roles fail).
Brand Voice Example:
Write in our company voice:
- Confident but not arrogant
- Technical but accessible
- Helpful without being condescending
Example sentences in our voice:
"Here's what we found..."
"This approach works because..."
"You might also consider..."Personal Style Example:
Match my communication style from these examples:
[Paste 2-3 examples of your actual writing]
Key characteristics: Direct, uses bullet points, includes specific examples, casual but professional tone.What it is: Tell the AI what to do when it can't complete the full request or is uncertain.
Why it works: Prevents unhelpful "I can't do that" responses and gives you useful partial information instead of nothing.
The rationale: The AI often gives up entirely when it should give you what it can and explain what's missing.
Partial Information Template:
If you don't have complete information:
1. Provide what you do know
2. Clearly state what's missing
3. Suggest where to find the missing information
4. Give your best analysis with available data
Never say "I don't have enough information" without attempting partial analysis.Uncertainty Management:
When uncertain:
- Provide your best estimate with confidence level (1-10)
- Explain your reasoning
- Suggest verification steps
- Offer alternative interpretations
Format: "Based on available information, I'm 7/10 confident that..."What it is: Break complex reasoning into explicit, numbered steps that the AI must follow.
Why it works: Forces systematic thinking and prevents the AI from jumping to conclusions or missing important considerations.
The rationale: Complex decisions require systematic analysis. By structuring the thinking process, you get more thorough and reliable reasoning.
Decision Analysis Framework:
Walk through this decision using these steps:
1. Identify the core decision to be made
2. List all available options
3. Define evaluation criteria (what matters most)
4. Score each option against criteria
5. Identify potential risks/downsides
6. Make recommendation with reasoning
7. Suggest monitoring metrics
Show your work for each step.Problem Diagnosis Template:
Diagnose this issue systematically:
1. Symptom identification: What exactly is happening?
2. Possible causes: List 3-5 potential root causes
3. Evidence evaluation: What data supports each cause?
4. Most likely cause: Which has strongest evidence?
5. Testing approach: How would you confirm this?
6. Solution strategy: What would fix the root cause?What it is: Design a sequence of prompts where each builds on the previous one, rather than trying to do everything in one massive prompt.
Why it works: Complex projects often require different types of thinking (creative, analytical, critical, etc.). Breaking them into phases lets you optimize each step.
The rationale: Just like you wouldn't write, edit, and format a document simultaneously, the AI works better when you separate different cognitive tasks.
Content Creation Workflow:
Step 1: "Brainstorm 10 angles for an article about [topic]"
Step 2: "Take angle #3 from the previous list and create a detailed outline"
Step 3: "Write the introduction section using this outline: [paste outline]"
Step 4: "Review this intro for clarity and engagement: [paste intro]"Research & Analysis Chain:
Prompt 1: "What are the key questions I should research about [topic]?"
Prompt 2: "For question #2, what data sources would be most reliable?"
Prompt 3: "Analyze this data and identify the top 3 insights: [paste data]"
Prompt 4: "Turn these insights into actionable recommendations"When to use: Multi-step projects, complex analysis, content creation, or any task requiring different types of thinking.
What it is: Ask the AI to help you improve your prompts - essentially "prompting about prompting."
Why it works: The AI understands what makes prompts effective and can often spot issues you miss in your own prompts.
The rationale: Sometimes you know what you want but can't figure out how to ask for it effectively. The AI can help bridge that gap.
Prompt Optimization Template:
I'm trying to get better results for [specific task].
Here's my current prompt: [paste your prompt]
Please:
1. Identify what might be unclear or missing
2. Suggest specific improvements
3. Rewrite an optimized version
4. Explain why your changes would work betterTask Analysis Helper:
I want to prompt an AI to [describe your goal].
What information would the AI need to do this well?
What format should I request for the output?
What constraints or guidelines should I include?
Create a complete prompt template for me.When to use: When you're stuck, when prompts aren't working well, or when you want to systematically improve high-value prompts.
What it is: Explicitly state what you DON'T want, not just what you do want.
Why it works: Prevents common failure modes that you've experienced before, based on your specific use case and preferences.
The rationale: The AI doesn't know your pet peeves, what you consider clichéd, or what doesn't work in your context unless you tell it.
Content Creation Example:
Write a blog post about productivity.
AVOID:
- Clichéd phrases like "game-changer" or "unlock your potential"
- Generic advice everyone already knows
- Preachy or condescending tone
- Exceeding 1000 words
- Ending with a generic call-to-actionBusiness Analysis Example:
Analyze this market data.
DO NOT:
- Make predictions beyond 6 months
- State correlation as causation
- Use absolute terms like "always" or "never"
- Ignore potential biases in the data
- Give recommendations outside the analysis scopeWhen to use: When you have experience with common failures, specific quality standards, or known constraints.
What it is: Establish high-level principles that should guide all AI responses, not just the specific task.
Why it works: Ensures consistent values and approach across all outputs, even when the specific task varies.
The rationale: Like giving someone your company's core values before they represent you - it ensures alignment on what matters most.
Business Communication Principles:
Follow these principles in all responses:
1. Truth: Only state what you're confident is accurate
2. Clarity: Use simple language, avoid jargon
3. Helpfulness: Always provide actionable next steps
4. Respect: Never dismiss or minimize concerns
5. Transparency: Admit limitations and uncertainties
Now apply these principles to: [your specific request]Educational Content Principles:
Create content following these principles:
- Accessibility: Explain complex concepts simply
- Engagement: Use examples and analogies
- Practical: Include real-world applications
- Progressive: Build from simple to complex
- Inclusive: Consider diverse backgrounds/perspectivesWhen to use: Consistent brand representation, educational content, customer-facing communications, or any high-stakes communications.
What it is: Combine your specific documents/data with the AI's general knowledge, clearly distinguishing between the two sources.
Why it works: Gets you the best of both worlds - your specific, current information plus the AI's broad knowledge and analysis capabilities.
The rationale: The AI has great general knowledge but doesn't know your specific situation, recent data, or proprietary information unless you provide it.
Document Analysis Template:
Based on the attached document [paste/attach document], please:
1. First, summarize what type of document this is and its main purpose
2. Extract the key findings/conclusions (document only)
3. Identify any gaps or areas needing more information (document only)
4. Compare these findings to industry best practices (your knowledge)
5. Suggest next steps based on this analysis (combined analysis)
Use only information from the document for steps 1-3, then apply your general knowledge for steps 4-5. Clearly distinguish between document content and your analysis.Data + Context Example:
Here's our Q3 sales data: [paste data]
Here's our company context: [brief company description]
Analyze this data and:
1. Identify trends (data only)
2. Compare to typical industry benchmarks (your knowledge)
3. Recommend actions (combined analysis)
Label each section clearly.When to use: Working with proprietary data, recent information not in training data, company-specific documents, or specialized content.
What people think works: "You are a math professor" will make the AI better at math problems.
The reality: Sander's research found "the accuracies were like 0.01 apart. So there's no statistical significance." Even when there appeared to be improvements, they were too small to matter practically.
Why people believe it works: Early studies on GPT-3 showed small improvements, but these don't hold up with modern models or at scale.
What still works: Role prompting for creative/expressive tasks where you want a certain writing style or tone (not factual accuracy).
Action: Stop saying "You are a world-class copywriter" for factual tasks. DO use it for "Write in the style of a friendly teacher" for tone/style.
What people think works: "This is important to my career," "I'll tip you $5," "Someone will die if you get this wrong."
The reality: As Sander explained, "There have been no large-scale studies that I've seen that really went deep on this" and "I don't believe in those things."
Why people think it works: Possible marginal effects on older models, confirmation bias, and viral social media posts without rigorous testing.
The truth: These phrases worked briefly on early models but don't work on current systems.
Action: Stop using emotional manipulation or reward promises. Focus on clear instructions and good examples instead.
For reasoning models (O3, etc.): Built-in reasoning means you don't need to prompt "think step by step"
For non-reasoning models (GPT-4, GPT-4o): Still add "think step by step" for robustness
Sander's experience: "99 out of 100 times it would write out its reasoning great and then give a final answer. But one in 100 times it would just give a final answer" without the prompt.
Action: Know which model you're using. Add reasoning prompts for non-reasoning models, skip them for reasoning models.
Start Immediately:
Medium-term:
Advanced:
Customer Support = Constitutional + Structured + Error Handling + Voice
PRINCIPLES: [Truth, Clarity, Helpfulness, Respect]
FORMAT: [Greeting → Solution → Follow-up offer]
ERROR HANDLING: [What to do when uncertain]
VOICE: [Empathetic, professional, solution-oriented]Content Creation = Few-shot + Constraints + Voice + Self-criticism
EXAMPLES: [3 successful pieces in desired style]
CONSTRAINTS: [Length, tone, topics to avoid]
VOICE: [Brand personality descriptors]
REVIEW: [Self-check before final output]Data Analysis = Context + Multi-step + Structured + Error Handling
CONTEXT: [Domain knowledge, data sources, business goals]
PROCESS: [Systematic analysis steps]
FORMAT: [Standardized output structure]
UNCERTAINTY: [How to handle incomplete data]The simple 4-step process for casual use:
Step 1: Start Simple Just ask your question directly. Don't overthink it.
"Write an email declining the meeting"Step 2: Add Context If Needed
If the result isn't quite right, add background information:
"Write an email declining the meeting. Context: I'm a product manager, this is with our design team, and I want to reschedule for next week."Step 3: Use Follow-ups to Improve Instead of trying to perfect the initial prompt, just ask for changes:
"Make it warmer and suggest two specific alternative times"Step 4: Save What Works When you get a great result, save that conversation pattern for future use.
Reality check: Sander often just types "write email" or "make better improve" for casual tasks. Don't overthink conversational prompting.
Phase 1: BASELINE (Week 1) Goal: Understand your current performance and establish metrics
Days 1-2: Define Success Clearly
Days 3-5: Collect Representative Test Data
Days 6-7: Test Your Current Approach
Example: Customer service chatbot baseline
Phase 2: SYSTEMATIC IMPROVEMENT (Weeks 2-3) Goal: Systematically optimize the prompt using data-driven methods
Week 2: Add Examples and Context
Week 3: Add Structure and Constraints
Example progression with real metrics:
VERSION 1 (Baseline): "Respond to this customer message professionally"
→ Result: 73% success rate
→ Main issues: Generic responses, wrong tone
VERSION 2 (Added examples): [Previous prompt + 5 examples of perfect responses]
→ Result: 84% success rate (+11 points)
→ Remaining issues: Inconsistent policy references
VERSION 3 (Added context): [Previous + company policies and guidelines]
→ Result: 91% success rate (+7 points)
→ Remaining issues: Inconsistent formatting
VERSION 4 (Added structure): [Previous + required response format]
→ Result: 94% success rate (+3 points)
→ Target achieved!Phase 3: PRODUCTION MONITORING (Ongoing) Goal: Maintain and improve performance over time in real-world use
Daily Monitoring:
Weekly Analysis:
Monthly Reviews:
Example production monitoring timeline:
The trigger criteria: You're doing the same task more than 10 times AND it has clear success criteria AND mistakes have meaningful consequences
The step-by-step migration process:
Step 1: Documentation Phase (1 day)
Step 2: Systematization Phase (1-2 days)
Step 3: Testing Phase (2-3 days)
Step 4: Deployment Phase (1 day + ongoing monitoring)
Real transformation example - Email automation:
Original conversational approach:
You: "Draft response to this shipping complaint"
AI: [Gives generic, unhelpful response]
You: "Make it more empathetic and offer specific tracking information"
AI: [Better, but still missing company policy info]
You: "Add reference to our 30-day shipping guarantee policy"
AI: [Finally good - success after 3 iterations]Systematic evolution based on successful patterns:
SYSTEM PROMPT:
You are responding to shipping complaints for [Company Name].
CONTEXT:
- Standard shipping: 3-5 business days with tracking
- Express shipping: 1-2 business days
- 30-day shipping guarantee policy: full refund if delivery exceeds promise
- Common issues: weather delays, carrier problems, address errors
EXAMPLES:
[3 examples based on your best conversational outcomes]
INSTRUCTIONS:
1. Acknowledge frustration empathetically (learned from "make it more empathetic")
2. Provide specific tracking information immediately (learned from successful follow-up)
3. Reference relevant company policies (learned from policy follow-up)
4. Offer concrete next steps and timeline
FORMAT:
- Empathetic opening
- Specific information about their order
- Policy reference if applicable
- Clear next steps
- Professional closing
CUSTOMER COMPLAINT: {complaint_text}The key insight: Start with casual conversational prompting to discover what works. Then systematically capture those successful patterns into a single, robust prompt that works without iteration.
Solution: Add few-shot examples + specific context + constraints
Solution: Structured output formatting + constitutional principles + error handling
Solution: Comprehensive testing + graceful degradation + systematic refinement
Solution: Voice consistency examples + negative prompting + self-criticism
Solution: Decomposition + multi-step reasoning + prompt chaining
Conversational Mode:
Product-Focused Mode:
Minimum Viable Testing:
Production-Grade Testing:
Save and analyze your most successful prompts to identify patterns:
For product-focused prompts:
[CONTEXT/BACKGROUND] +
[EXAMPLES if available] +
[CLEAR INSTRUCTION] +
[OUTPUT FORMAT] +
[CONSTRAINTS/BOUNDARIES] +
[ERROR HANDLING]Bad output? → Add examples + context
Inconsistent? → Add constraints + structure
Wrong style? → Add voice examples + negative prompting
Too complex? → Use decomposition + multi-step reasoning
Edge cases? → Add error handling + graceful degradation
The biggest opportunities lie in systematically optimizing prompts that run at scale. Master conversational prompting first, then transition high-value repeated tasks to product-focused optimization.