April 8, 2025

The Two Levers and the Taste Gap

Craft

You start out in a new field by learning some basic skills.

How to dribble the basketball, how to pass, how to shoot. Skills are the practical application of explicit knowledge. What is dribbling? When is it used? What are the rules?

Now put that into action until it becomes second nature. You progress from basic skills to more advanced ones. Now you can dribble with both hands, now a crossover, now behind the back. These skill trees are nearly inexhaustible because the most highly skilled will create new skills that push the craft forward.

James Harden basically invented a new type of step-back jumper some 100+ years into the sport by twisting an established rule to a new advantage. Now it’s a standard among all top shooters.

Taste

But there’s another layer in any field that goes beyond skill application. You might call this taste, or judgement, or intuition.

In hoops they’ll say someone has “a feel for the game”. In product management someone might be known for their “product sense”. And this type of tacit knowledge is often multi-faceted. Harden has a feel for the game but he sure doesn’t have a sense for the moment. A PM might have strong product sense but lack a strong business sense. For a filmmaker to succeed they need not only aesthetic intuition but also commercial taste.

This is harder to define and harder to train for. There is rarely an obvious skill tree to follow. I think it’s mostly innate, though some comes from learning from experiences. Reps are requisite but without reflection they won’t just turn into taste. And while it isn’t necessarily a true binary, it is very unlike a skill tree: you mostly have it or you don’t, and you can’t really progress down a path of ever increasing taste.

The Gap

All of us who do creative work, we get into it because we have good taste. But there is this gap. For the first couple years you make stuff, it’s just not that good. It’s trying to be good, it has potential, but it’s not. But your taste, the thing that got you into the game, is still killer. And your taste is why your work disappoints you.

-Ira Glass

We’re at a point with machine intelligence where they have this taste gap. They’re pretty competent at wielding a broad range of skills, but don’t always know how to apply them.

A beginner who has no skills in coding has suddenly been granted a lever that has brings those skills to bear for them. You can vibe code your way to a working prototype by describing what you want and being patient enough to keep trying. This is a very big deal and opens a lot of doors that did not exist. But it’s also not (yet!) a path to scalable and sustainable software.

Meanwhile a senior engineer with a well-tuned sense of system design can 3, 4, 5x their productivity by applying the skills of the LLM at a much faster pace than they can apply their own. They have the taste and judgement to direct the energy of the system in a way that the LLM alone cannot, and have been granted the lever of a system that types at 500 words per minute and is parallelizable to boot.

And then there are the poor journeymen. They have developed a set of skills that are largely on par with the models, but have an underdeveloped sense of taste that is also on par with the models. Now they have leverage to go in bad directions 400wpm faster than they would have before, often closing off the learning loop that would help them develop their intuition.

The Fix

In verifiable domains like coding the systems can and will learn enough to improve their intuitions. And large models can already be good at system design when used appropriately, which is just a new skill that many haven’t yet spent the time to master.

But what about areas that aren’t verifiable, at least not in a RL training environment? Can a model “just get it” with the right parameters? Does it need continuous training to be able to hone intuition? Or is the taste already there, hiding out in latent space, just waiting to be discovered?

Some people will make a fortune answering that.