If you've been keeping an eye on large language models (LLMs), you'll have noticed that context windows are getting bigger and bigger. Anthropic is now boasting a whopping 100,000 token window, rivaling GPT-4 and LLaMA at 32,000 tokens. If you're feeding whole reports into your agent or working with lengthy, complex prompts, this sounds like a miracle. Finally, you can give AI every single detail and it'll offer you the most detailed, best responses, right?

Well, a recent study suggests that bigger may not always be better when it comes to context windows. And I've read through the entire 18-page study to pull out the key insights for you-and offer seven ways you can make your documents and prompts easily retrievable for any LLM, no matter the size of the context window.