How do you get better ideas from your LLM?

How might one increase the diversity of an AI generated pool of ideas?

Amongst my clients & network I hear many folks complaining about how generic the ideas generated via LLMs can be. If you're looking to generate innovative, novel ideas, this working paper from Wharton has value.... with some caveats ⚠️

The paper measures how different prompting techniques increase the diversity of an AI-generated pool of ideas.

🧐 The TL;DR? They find:

1️⃣ "Pools of ideas generated by GPT-4 with various plausible prompts are less diverse than ideas generated by groups of human subjects".

2️⃣ "The diversity of AI generated ideas can be substantially improved using prompt engineering"

3️⃣ "Chain-of-Thought prompting leads to the highest diversity of ideas of all prompts we evaluated and was able to come close to what is achieved by groups of human subjects."

For context, the researchers used chatGPT to generate new product ideas under $50 for Uni students. The prompting methods largely break down into:

• Basic direct instructions

• Persona-based prompts (“think like a visionary <blah>”) 

• Leveraging known creativity techniques (eg. custom GPTs instructed to use best practise in brain-storming)

• Chain-of-Thought prompting, where the AI is given multiple steps to follow. 

The AI’s ideas were compared with those from pools of human participants. They measured both the number of unique ideas and how different the ideas were from each other.

There's lots of useful stuff in here - including prompts you can try. Many of the techniques they're embedding in prompts are basically just best practise in innovation process - reframing, outlier-hunting, combining, etc.

👉Their practical recommendation is to treat idea generation with LLMs as a process, not a one-shot prompt. 

Use multiple prompt variants, change constraints, spot when the idea space gets repetitive. Try generating smaller pools of ideas with different prompting strategies and then combining these pools.

The caveats?

⚠️ Diverse ≠ Good. 1,000 random ideas don’t get you closer to something customers will actually want. Innovation isn’t just variance, it’s variance plus desirability, feasibility, usability, ethics, scalability, and timing. The hard part is combining novelty with viability That’s still where human judgment, context, & domain expertise matter. To automate that, LLMs needs LOTS of context. The article also pre-dates mass access to multi-agent systems that can play those roles.

🤖 GPT-5’s reasoning & context handling are stronger. It's naturally inclined to produce more varied and coherent outputs without prompt-gymnastics. Beyond prompting, you can also try changing the 'temperature' setting - this is a setting non-technical people (like me!) can easily change, either via instruction or by changing your customGPT settings.

It's a solid read and maintains a lot of relevance, but as models evolve the question shifts from “how do I get more diverse ideas?” to “how do I build a system turns diverse & feasible ideas into something valuable for both customers and our Org?”