Notes from the Prompting Lab

Here at Libretto, we're trying to turn prompt tinkering into prompt engineering. This means we do a lot of empirical experiments on what works and what's just hype when it comes to prompt engineering, and here on our blog is where we share what we've learned.

Building an LLM-based App with Libretto, part 2

LLMs judging LLMs: It's LLMs all the way down.

Vibes at Scale: Libretto Makes LLM-as-Judge Easy, Understandable, and Dependable

Building an LLM-based App with Libretto, part 1

Prompt Engineering Should Be Fast, Automated, & Fully Empirical — Here’s Libretto’s Approach

When looking at in-context learning, should you use 1-shot, 3-shot, 5-shot, 9-shot?

How many few shot examples should you use?

Do LLMs want us to say the magic word?

If a set of few-shot examples works with one model, will they work for another model?

Are the best few-shot examples applicable across models?

How much does it matter which few-shot examples you choose to put in your prompt? We dig in to the details!