Lesson 03
Estimated time: 15 minutes
Learning objectives
By the end of this lesson, you will be able to:
- Recognize common challenges when starting out with AI and use troubleshooting techniques to overcome them
- Define AI Fluency and know where to go to learn more about working with AI in a fluent way
- Explain how you might set up evals to better understand how Claude might perform with your unique workflows
Common challenges and how to fix them
As you start working with Claude, you'll likely encounter moments where the response isn't quite what you expected. This is normal—and it's an opportunity to refine your approach. Here are some of the most common challenges and how to address them.
| Challenge | What's happening | Try this |
|---|---|---|
| Claude's response is too generic | Your prompt didn't include enough context about your specific situation | Add details about your audience, role, or constraints. Instead of "Write an email about the project delay," try "Write an email to our enterprise client explaining that the software integration will be delayed by two weeks. They've been patient so far but this is the second delay. Keep it professional but apologetic." |
| The response is too long (or too short) | Claude is guessing at appropriate length | Be explicit: "Give me a two-paragraph summary" or "Keep this under 100 words" or "I need a comprehensive analysis—length isn't a concern." |
| Claude didn't follow my format | Claude understood what you want but not how you want it presented | Show, don't just tell. Provide an example of the format, or describe the structure explicitly: "Use bullet points with bold headers for each section." |
| I got confident-sounding information that turned out to be wrong | Claude occasionally generates plausible but incorrect information, especially with specific facts or niche topics | For high-stakes work, verify key facts independently. Ask Claude to cite sources or indicate confidence level. Enable web search to ground responses in current information. |
| The tone isn't right | Claude defaults to helpful and professional, which may not match your needs | Describe the tone in plain language: "Make this more conversational" or "This should sound authoritative and formal." Provide an example of writing in the style you want. |
The iteration mindset
One of the most important shifts when working with Claude is recognizing that your first prompt rarely produces a perfect result—and that's okay. Think of your initial prompt as the start of a conversation, not a one-shot request.
Effective Claude users:
- Treat first drafts as starting points. Review what Claude produces, identify what's working and what isn't, then refine.
- Give specific feedback. "Make it shorter" is fine, but "Cut the first two paragraphs and make the conclusion more action-oriented" is better.
- Know when to start fresh. If a conversation has gone off track, sometimes it's faster to open a new chat with a clearer prompt than to try to redirect.
What is AI Fluency?
AI Fluency is the ability to collaborate effectively with AI tools—not just knowing which buttons to click, but developing the judgment to use AI well across different situations.
The 4D Framework for AI Fluency, developed through research collaboration between Professor Rick Dakan (Ringling College of Art and Design) and Professor Joseph Feller (University College Cork), identifies four core competencies that, when combined, can help you make the most of your AI interactions:
- Delegation: Deciding on what work should be done by humans, what work should be done by AI, and how to distribute tasks between them. Includes understanding your goals, AI capabilities, and making strategic choices about collaboration.
- Description: Effectively communicating with AI systems. Includes clearly defining outputs, guiding AI processes, and specifying desired AI behaviors and interactions.
- Discernment: Thoughtfully and critically evaluating AI outputs, processes, behaviors and interactions. Includes assessing quality, accuracy, appropriateness, and determining areas for improvement.
- Diligence: Using AI responsibly and ethically. Includes making thoughtful choices about AI systems and interactions, maintaining transparency, and taking accountability for AI-assisted work.
You've already been practicing these skills throughout this course. The prompt framework from Lesson 2 (setting the stage, defining the task, specifying rules) is rooted in Description. The troubleshooting techniques above draw on Discernment and Diligence.
To learn more, check out our free AI Fluency course that explore all four competencies in depth, with practical exercises and real-world applications.
Evaluating Claude for your workflows
As you start integrating Claude into more of your work, you might wonder: how do I know if Claude is actually good at this particular task?
This is where Discernment becomes essential. Evals (short for evaluations) are a way to develop intuition for assessing Claude's outputs on the tasks that matter to you. They're systematic ways to test how well Claude performs on specific types of tasks that matter to you.
Why evals matter
Your work is unique. Claude might excel at drafting marketing copy but need more guidance for technical documentation in your specific domain. Running simple evals helps you:
- Understand where Claude adds the most value in your workflow
- Identify tasks where you'll need to provide more context or examples
- Build confidence in Claude's outputs for recurring tasks
A simple eval approach
You don't need complex infrastructure to evaluate Claude. Here's a practical approach:
- Gather examples. Collect 5-10 examples of a task you do regularly—emails you've written, reports you've created, analyses you've done.
- Create test prompts. Write prompts that would generate similar outputs. Include the context you'd naturally have when doing this work.
- Compare outputs. Run your prompts and compare Claude's responses to your examples. Ask yourself:
- Does Claude capture the key information?
- Is the tone and style appropriate?
- What's missing or could be improved?
- Refine your approach. Based on what you learn, adjust your prompts, add examples to show Claude what good looks like, or identify where human review is essential.
Example: Using Claude for data analysis
The video above is taken from our AI Fluency for nonprofits course, but the example is relevant for anyone working with data in AI. To evaluate how Claude might work with your data:
- Find a dataset you've manually analyzed
- Create prompts that request Claude to do the analysis on your behalf
- Compare Claude's results to your originals
- Note patterns and refine your prompt accordingly: Maybe Claude gets the right numbers but misses the overall patterns
This kind of lightweight evaluation helps you develop intuition for how to work with Claude on tasks that matter to you—and where to focus your review and refinement energy.
Lesson reflection
Before moving on, consider:
- Which of the common challenges have you already encountered? What techniques might you try next time?
- Where in your work would a simple eval help you understand if Claude is a good fit for a recurring task?
- How might the 4D Framework help you think about your collaboration with Claude?
What's next
In the next lesson, you'll explore the Claude desktop app and the three ways you'll work with Claude there — turn by turn (Chat), handing work off (Cowork), and building software (Claude Code).