In 1999, a group of marketing researchers in Israel did something that looked heretical at the time: they turned a creativity method into a computer program, had the program generate advertising ideas, and then asked professional judges to evaluate them blind. The judges did not know which ideas came from the program. The result was that the program’s ideas scored on par with those of professional ad people, and beat untrained laypeople.
This experiment predated large language models by more than twenty years. It points to a counterintuitive judgment: the core of idea generation can run as a procedure, and the procedure is indifferent to who executes it. What was missing back then was simply a sufficiently general execution machine. Now that machine exists. So we ran an experiment: we organized an innovation method into an operating manual that an AI can execute, had the same model answer the same question under two conditions—“with the manual” and “without the manual”—and compared the outputs. The manual and the full experiment record are open-sourced on GitHub. This topic grew out of a discussion with FlipShark (翻盖鲨鱼); our thanks to them up front. This article covers three things: why innovation is procedural legwork, why bare AI does this legwork poorly, and what we saw in the controlled experiment.
Our default image of innovation is the flash of insight: Newton’s apple, Archimedes’ bathtub. But if you seriously comb through the academic record, you find two conclusions that keep recurring.
The first conclusion: brainstorming doesn’t work. This isn’t a wisecrack; it’s one of the most-replicated findings in creativity research. Diehl and Stroebe’s classic 1987 experiment, along with a 1991 meta-analysis covering 38 experiments, gave a consistent direction: having the same number of people each come up with ideas independently and then pooling them (researchers call this a nominal group) produces about 83% more output than sitting around brainstorming together. The downsides of sitting together have clear mechanisms: taking turns speaking blocks trains of thought, fear of being judged makes people self-censor, and some people free-ride. Sheena Iyengar of Columbia Business School, author of Think Bigger, delivers a crisp verdict on this body of evidence: “The evidence is unambiguous—brainstorming doesn’t work.”
The second conclusion: successful innovation, viewed in hindsight, is almost always a new combination of old parts. The economist Schumpeter defined innovation as “new combinations of factors of production” as early as 1912; the mathematician Poincaré said that invention is discernment and choice; Iyengar compresses it into a single definition: innovation is a novel, useful combination of old ideas, used to solve a specific problem. The Statue of Liberty can be traced to five sources; Google’s core is three borrowed parts: the crawler, the library’s citation ranking, and the sidebar ad invented by someone else.
Put these two conclusions together, and the image of innovation changes: it looks less like writing poetry and more like assembly. This shift isn’t new to us. Last year, in “After a Thousand Failures, It Chose the One You See”, we described creativity in the AI era as moving from writing poetry to building a greenhouse: no longer relying on inspiration, but building a system that continuously generates, validates, and filters variants. This article is the methodological version of the same judgment. Assembly has a process, and academia has indeed written the process down—in considerable detail.
The Israeli school’s SIT (Systematic Inventive Thinking) offers five fixed templates: take an existing product, first break it into components and attributes, then apply a fixed sequence of operators to perform structural transformations—for example, remove a core component but keep its function, letting something already present in the environment take over (a car radio drops its antenna, and the defroster’s metal grid takes over signal reception). Once the transformation is done, you ask “who is this new form useful for?” The order is mandatory: transform first, then find the use. The controlled experiments show that this ordering itself has an effect—across three sets of tasks, form-before-use ideas consistently scored significantly higher. And that 1999 program from the opening was running exactly one of these five templates.
Iyengar’s Think Bigger, by contrast, starts from the problem end, with six steps paired with hard numbers: break the problem into at most 5 sub-problems; find 5 already-validated solution parts for each sub-problem, at least 3 of them from outside the industry; when combining, beyond deliberate pairing you must also do at least 5 rounds of random draws; and there are rules for when to stop—stop when the new things you find start repeating. Each part must be attached to a precedent that actually existed; anything you can’t name a precedent for counts only as an opinion and isn’t allowed on the table.
Notice the texture of these rules: minimum quantities, fixed order, stopping conditions, admission thresholds. This is the texture of an operating procedure, an SOP. The two major schools of innovation methodology converge on the same message from different directions: most of the work of coming up with ideas relies on discipline, not talent. It is legwork.
Once you grant that the bulk of innovation is legwork, the next inference follows naturally: legwork can be handed to AI. Breaking down a product’s structure, searching for precedents across industries, applying templates to transform, combining in bulk and then filtering—these happen to be exactly where a large model plus search tools is comfortable. Cross-domain retrieval is even the most painful step for a human executing this method: Think Bigger asks you to interview nearly thirty people snowball-style for each sub-problem, whereas a model can sweep the public precedents of several industries in a few minutes. The methodology authors themselves have arrived at this point too—Columbia Business School’s executive courses now come with an official AI-version tool.
But throw the problem directly at the AI, and what you usually get is not this. It will follow the phrasing “help me innovate” and hand you something fluent, reasonable, and vaguely familiar—most likely a nice trend survey. The experiment in the next section happens to have a specimen of exactly this.
Writing to here, I have to address an obvious question: the manual we’re talking about is, in the end, also a piece of text, and it ends up stuffed into the model’s context all the same. So what is the essential difference between it and typing “please use SIT and Think Bigger to help me analyze this” into a chat box? The difference isn’t the channel; it’s the form in which the methodology is present. When you only cite the method’s name, the model calls up the averaged version in its parametric memory: it roughly remembers that SIT has five templates, but it can’t recall a quota like “five parts per sub-problem, at least three from outside the domain,” and it certainly won’t invent on its own a counterintuitive forced move like “even after you already have a satisfying answer, you must still do five more rounds of random combination.” And it is exactly these details that are load-bearing. The manual does two things: it puts the full text of the procedure into the context, so the model executes against the full text rather than an impression; and it writes each rule as a self-checkable acceptance criterion—minimum quantities, admission thresholds, stopping conditions, risk labels—so that any deviation becomes visible. So, frankly, this skill is just a prompt written completely enough. That’s not a loophole; it’s precisely the argument itself: the gap doesn’t come from magic, it comes from how completely the procedure is present.
There is also a class of rules that can’t be executed just by putting the text into context—they need the execution environment to cooperate. “Each part must be attached to a named precedent” relies on real web search to verify, guarding against the model inventing a plausible-looking precedent on the spot; “random combination rounds” rely on the program’s random numbers, because if you let the model pick randomly in its head, it will choose the ones it finds appealing. The agent that ran this experiment felt the second one keenly: it switched on its own to a seeded random number generator, and suggested writing this into the manual as a hard requirement. It now is. These are the parts a chat box can’t give: tools, duration, and a prior commitment that won’t drift along with your in-the-moment reactions.
The finished product is a pure-Markdown skill: a routing layer that judges which method to use (improving an existing product goes to SIT, an open-ended problem goes to Think Bigger, and a paradigm-level problem shouldn’t use either—just say “this tool doesn’t apply”), two method pipelines that carry the procedure and validators, plus ten judgment axioms explaining why each rule exists, to keep future executors from cleverly “optimizing” away the load-bearing structure.
The way to test this is a controlled experiment. The problem we used is a real one: what innovations are still possible in today’s AI interfaces. The input material is an article we published a few days ago, “The Chat Box Illusion”, which argues that the chat box is the wrong interface for AI Agents and should be replaced with email-style asynchronous interaction. Two agents used the same model (Claude Opus), the same tool permissions (both could search the web), and the same input material; the only difference was that one loaded this skill and the other didn’t. Both reports and the third-party evaluation are in the experiment directory, and you can read them yourself.
Start with the baseline group, which was stronger than we expected. The Opus without the skill turned in a fairly high-quality industry survey: four interface paradigms currently competing, a representative product for each track, clear-eyed counterarguments, and even three fair criticisms of the input article. If you want to understand this space, this report reads better than most paid research notes.
But look at the “innovation directions” it gives, and the shape emerges: build an Agent inbox, build a generative UI, build Agent email infrastructure. Every one of them is “join a trend that has already started.” The directions aren’t wrong, but this answers “where should I bet,” not “what has no one else done.” Interestingly, according to the empirical research encoded in the skill, this kind of trend-following idea is exactly the category with the highest failure rate in the historical record.
The report with the skill is a different kind of product. It broke the problem into five sub-problems, found five solution parts with named precedents for each sub-problem, and of the twenty-five parts, fifteen came from outside the AI industry: the military’s mission command, the sampling standards of financial audits, hospitals’ central telemetry monitoring stations, Toyota’s andon cord, the pass window of a restaurant kitchen. The winning idea is called the “trust ladder”: each Agent holds a reputation score for each class of task, the score accumulates from real records of human acceptance (part borrowed from eBay’s rating system), the reputation score maps to explicit autonomy levels (part borrowed from self-driving’s SAE levels), and the level in turn determines the depth of review—high scores get spot-checked, low scores get read line by line (part borrowed from banks’ risk-control threshold routing). This combination was forced out by the rules: the load-bearing connection appeared in a mandatory random combination round; it never appeared in the deliberate-pairing rounds.
The two reports converge in one place, and that convergence is itself informative: the two agents independently discovered that the email track the article bets on is already crowded, and independently pointed to the same gap—namely, that “trust” has to this day not been made into an interface element. Every tool displays the Agent’s status; no tool displays “how much you should trust it.” When one conclusion is hit by two independent paths, the odds that it’s a genuine gap go up considerably.
The difference also shows up in the shape of honesty. The report with the skill gave its six ideas conservative scores and explicitly marked two of them as high-risk, with reasons quoted directly from the empirical record: one is too much like trend-following, and one’s integration layer has no findable precedent. Each idea comes with a complete derivation chain—if you disagree with one of its assumptions, you can overturn just that one link and rerun, without having to knock down the whole piece. Of course, the cost is real too: the process overhead is roughly double, the report is longer, and this evaluation was done by us, non-blind, with a sample size of just one pair—these limitations are all written into the experiment record.
In this division of labor, there are a few things AI can’t do, and the methodology flags them itself. Which problem to pick depends on whether you have sustained passion for it, which only you know; how you want the solution to feel, and whom you want to please, is taste; and finally, telling the idea to a real person and observing the image that forms in their mind—no simulation can replace this step. The skill designs these nodes as questions to the user, rather than answers on the user’s behalf.
So the conclusion of this experiment is not that AI will innovate for you. The more precise statement is: the large chunk of innovation that used to be treated as talent is actually legwork that can be written as a procedure; once the procedure is handed to a machine, the human’s remaining part becomes clearer instead—namely, choosing the topic, taste, and the final call. That 1999 program proved this once, only there was no general execution machine back then to turn it into an everyday thing.
The method has boundaries, and I’ll say so honestly. That often-cited number, that seven in ten successful products can be matched to the five templates, is a retrospective classification of established facts, and the sample is biased toward patents and physical consumer goods; it does not amount to a promise that “do this and you’ll succeed.” What this method is good at is combinatorial innovation and product improvement; when it meets a paradigm question of “should this category even exist,” the correct output is to admit it doesn’t apply. And there’s another layer to watch out for: a methodology, used long enough, becomes its own fixed mindset, so in the skill’s routing layer, “use neither” is a formal exit.
The full raw records of the manual, the axioms, and the experiment are here: github.com/grapeot/innovation-assistant-skill. It’s pure Markdown, has no dependencies, and hands the repo address to your coding agent to install. You’re welcome to run it against your own problems—and especially welcome to produce counterexamples.
Finally, thanks again to FlipShark (翻盖鲨鱼): without that discussion, there would be no article and no experiment.