If you optimize your website by checking off software warnings or copying your competitors, you are wasting your budget. You only get results when you start from what the search engine wants to achieve and build a specific mechanism to serve it.

Most optimization work falls into two traps. People practice compliance optimization, which satisfies a piece of software. Or they practice folk-wisdom optimization, which copies whatever everyone else repeats.

Compliance optimization satisfies software

An SEO audit tool spits out a list of 200 issues. You hand that list to a developer and pay them to fix all 200 rows. If it takes a developer one hour to fix each row at $150 an hour, you just spent $30,000 clearing warnings.

About 190 of those rows do not matter at all. The rows that matter are the handful that block Google outright:

• Crawl blockers
• Rendering failures that hide your content from Google
• Redirect chaos
• Accidental noindex tags

You just paid $28,500 to fix 190 things that will never bring a single new visitor to your website.

People chase a green Core Web Vitals score on Lighthouse because the software tells them orange is bad. A developer spends two weeks shaving milliseconds off a script load time. You should fix an egregiously slow page because your users deserve a functional website. Getting a perfect green score almost never moves your rankings. You burn scarce development time chasing an arbitrary grade.

When a score becomes a target, it stops being a good measure. Economists call this Goodhart’s law. The score is a proxy for the actual outcome. The outcome you want is more revenue from organic traffic. The proxy is a green circle in a reporting dashboard. When you optimize for the green circle, you disconnect your effort from the revenue.

Psychologically, compliance is attractive because it feels safe. “I followed the checklist” is a bulletproof defense in a postmortem meeting. If the traffic drops, you point to the audit report and show that you fixed every error. Watching a software score turn from red to green requires zero human judgment. You avoid the hard work of figuring out what actually matters.

Folk wisdom copies the appearance of success

The other trap is adopting tactics because other people in your industry talk about them.

Someone notices that the top search results for a keyword average 2,000 words. They tell you to write 2,000 words for your next post. The actual answer to the searcher’s query takes 300 words. You pay a writer to generate 1,700 words of filler just to hit the target. Page length describes the current winners. Writing more words will not cause your page to rank. If you buy the same brand of shoes as an Olympic marathoner, you will not suddenly run a two-hour marathon.

Richard Feynman called this cargo cult science. After World War II, islanders built mock airstrips out of wood to summon the airplanes that brought supplies during the war. They lit fires along the edges to mimic runway lights. They wore wooden headsets with bamboo antennas. They copied the visual form of an airport perfectly. No planes landed. They lacked the mechanism.

This behavior is rampant in AI search right now. People tell you to upload llms.txt files and format your content in specific chunks for artificial intelligence. The advice spreads rapidly on social media. This advice is wrong. Google Search explicitly ignores llms.txt files. Google’s own AI optimization guide tells site owners to skip AEO hacks like chunking content and chasing inauthentic mentions.

Advice spreads because it is simple to copy. Folk wisdom is attractive because it is free and carries social proof. You do not have to formulate a theory. You just read a blog post and copy the instructions. If you do what everyone else does, you converge on the exact same average.

First-principles optimization starts with what the system wants

You must start from what the system you are optimizing for is itself trying to achieve. You derive the work from there.

Google wants searchers to come back to Google. This happens when the search journey ends on the page Google sent them to. The user clicks your link, they do not bounce back to the search results, and they do not refine their query. They got what they wanted.

This click behavior is a documented mechanism. Google’s antitrust trial evidence and the 2024 API documentation leak both describe NavBoost, a system built entirely on user click data. The search engine watches what users click and whether they stay. It promotes the pages that satisfy the intent.

Does a skeptical person who typed this query get their intent confirmed in two seconds, their objections killed in ten seconds, and a reason never to hit the back button? You must put the exact answer at the very top of the page. If you bury the answer under four paragraphs of introductory text, the user clicks back. If you cannot answer yes to the two-second question, your word count does not matter.

AI assistants operate on a similar foundation. They want to give a good answer and cite sources they trust. The levers for that system are being seen across many independent sources, publishing your own brand facts, and writing claims clear enough to quote.

Every tactic must come with a mechanism. You need an explanation of why this specific work moves that specific outcome. If your explanation is easy to vary, it is a bad explanation. Saying a tactic works because “it signals quality” is infinitely variable. You could use “it signals quality” to justify keyword stuffing, buying links, or deleting half your website. It applies to everything and means nothing.

Knowledge grows through conjecture and criticism. You put forward a hypothesis about how a system works, and you test it. You propose that moving the price to the top of the page will prevent users from clicking back to Google. You make the change and watch the click data. Checklists and folk wisdom bypass this process.

Industry best practices are just a baseline

People argue that best practices exist because they work. They assume the accumulated experience of an entire industry beats one person reasoning from scratch.

A best practice is just a starting point. Every practice that comes with a real mechanism survives first-principles scrutiny and you keep doing it. We still put keywords in title tags because search engines still use title tags to match queries. The difference is knowing exactly why each thing works.

Environments change. The shift to AI search is happening right now. Only the person holding the mechanisms can tell which old practices expired and which ones still bind. If you rely on compliance and folk wisdom, you cannot tell the difference because you never contained the why in the first place.

First-principles optimization requires you to state a direct mechanism. You put an explanation out there that you can be wrong about in public. When traffic drops, you have to defend your reasoning instead of pointing at a checklist.