SEO Is Not About Keywords: A Technical Guide to How Search Actually Works

Tejas GK|May 14, 2026 (3mo ago)

14 min read

When most people hear SEO, they immediately think of keywords.

Find a keyword. Put it in the title. Mention it several times in the article. Get a few backlinks. Wait for Google to rank you.

That description isn't completely wrong, but it's an extremely shallow representation of modern SEO.

After working on websites and web applications at Pixels and Grids, one thing became increasingly obvious to me: SEO is as much an engineering, architecture, content and product problem as it is a marketing problem.

You can write an incredible article, but it doesn't matter if Google cannot reliably discover it.

Google can discover the page but decide not to index it.

It can index the page but misunderstand which URL represents the canonical version.

It can understand the page perfectly but determine that another page satisfies the search intent better.

Or you can rank first and still get almost no business because your title attracts the wrong users.

Advanced SEO therefore requires understanding the entire pipeline:

Discovery → Crawling → Rendering → Indexing → Retrieval → Ranking → Click → Conversion

Google itself broadly describes Search as crawling, indexing and serving search results.

Once you start thinking about SEO this way, your approach changes completely.

1. Start With How a Search Engine Actually Works

Before optimizing a website for Google, you need to understand what Google is trying to accomplish.

At an extremely simplified level, a search engine has to:

Discover URLs
Crawl those URLs
Render their content when necessary
Extract and understand information
Store useful pages in an index
Understand a user's query
Retrieve potential documents
Rank them
Return the most useful results

Google's crawlers continuously discover pages across the web rather than relying on website owners to manually submit every URL.

This distinction matters.

SEO isn't:

"How do I convince Google that my page is good?"

A better question is:

"How do I make it extremely easy for a search engine to discover, understand and confidently retrieve this page for the right queries?"

That's the foundation of technical SEO.

2. Crawling Is the First Battle

Imagine you publish 100,000 product pages.

From your perspective, those pages exist.

From Google's perspective, they effectively don't exist until Google discovers them.

Discovery can happen through several mechanisms, particularly links from pages Google already knows about and URLs provided through sitemaps.

This is why internal linking is far more important than many website owners realize.

Consider this architecture:

Homepage
↓
Services
↓
Web Development
↓
Next.js Development
↓
Next.js E-commerce Development

A crawler can naturally travel through this structure.

Now compare it with:

Homepage

Services

Web Development

Next.js Development

where the pages technically exist but barely link to each other.

Humans might find those pages through navigation or search functionality.

A crawler has a much weaker understanding of their relationship.

Internal links aren't merely navigation.

They communicate site architecture.

When working on websites at Pixels and Grids, this is something I've become increasingly conscious about.

I don't want important pages existing as isolated URLs.

If we're building a service page, there should be logical paths connecting it to relevant case studies, related services, articles and parent pages.

The architecture itself should explain what the company does.

3. Crawlability and Indexability Are Different Problems

This distinction causes an incredible amount of SEO confusion.

Crawling means the search engine can access a resource.

Indexing means the search engine has processed the resource and may store it for retrieval.

A sitemap does not mean:

"Google, index this."

It means something closer to:

"Google, these are URLs I'd like you to know about."

Google explicitly states that submitting a sitemap does not guarantee indexing or ranking.

Similarly:

robots.txt

primarily controls crawling.

Whereas:

instructs compatible search engines not to index the page.

Confusing these two mechanisms can produce bizarre SEO problems.

I've seen developers treat robots.txt, noindex, canonical tags and sitemaps as if they're interchangeable.

They're not.

4. JavaScript SEO Matters More Than Ever

This is particularly relevant to us at Pixels and Grids because a lot of modern products we work with use frameworks such as React and Next.js.

Google can execute JavaScript.

But that doesn't mean:

"Rendering doesn't matter anymore."

Google describes processing JavaScript applications through crawling, rendering and indexing. Server-side or pre-rendered content can still be beneficial for both users and crawlers.

Consider a purely client-rendered page.

The initial HTML might effectively look like:

Your browser downloads JavaScript, executes it, calls APIs and eventually produces:

<h1>Enterprise Software Development</h1>

<p>
We build scalable applications for...
</p>

A modern crawler may render this successfully.

But you've introduced another dependency into discovery and understanding.

Now compare that with server-rendered HTML where the meaningful content is already available in the response.

That's one reason I generally prefer server rendering or static generation for SEO-critical pages when working with Next.js.

Google itself recommends server-side rendering, static rendering or hydration over older dynamic-rendering workarounds.

That doesn't mean:

CSR = bad SEO.

It means:

Don't unnecessarily make crawlers work harder to understand important content.

5. Canonicalization Is Massively Underrated

Suppose these URLs all return essentially the same product:

/product/shoes

/product/shoes?color=black

/product/shoes?utm_source=instagram

/product/shoes?ref=homepage

From a developer's perspective, they're URLs.

From a search engine's perspective, the question becomes:

Which URL represents this document?

That's canonicalization.

Google groups duplicate or very similar pages and chooses a representative canonical URL.

You can help communicate your preference through:

But canonical tags aren't magical commands.

Google treats redirects and rel="canonical" as strong canonicalization signals, while sitemap inclusion is weaker; multiple consistent signals can reinforce one another.

Advanced SEO therefore requires signal consistency.

If your:

canonical → URL A
sitemap → URL B
internal links → URL C
redirect → URL D

you've created unnecessary ambiguity.

I want all of them saying:

THIS is the URL.

6. Stop Thinking About Keywords. Start Thinking About Search Intent.

Suppose Pixels and Grids wants traffic for:

"Next.js development company"

A primitive SEO strategy would be:

Put "Next.js development company" everywhere.

So you get:

Best Next.js Development Company

We are a Next.js development company providing
Next.js development company services...

Looking for a Next.js development company?

That's not sophisticated SEO.

That's just bad writing.

The important question is:

Why did someone search for that phrase?

Someone searching:

what is Next.js

probably wants education.

Someone searching:

Next.js vs React

is comparing technologies.

Someone searching:

Next.js development company India

may have commercial intent.

Someone searching:

hire Next.js developer

has even stronger transactional intent.

These queries belong to the same topic but represent completely different stages of intent.

I normally think about keywords as belonging to a query space, not as isolated phrases.

For example:

NEXT.JS
│
┌────────────┼────────────┐
↓ ↓ ↓
LEARNING COMPARISON COMMERCIAL
│ │ │
What is Next.js Next vs React Next.js agency
SSR explained Next vs Vite Hire developer
App Router Next vs Remix Development cost

Now you're building topical coverage, rather than randomly publishing articles.

7. Build Topic Clusters, Not a Graveyard of Blog Posts

This is something I think many agency websites get badly wrong.

They publish:

10 Benefits of AI

Why Your Business Needs a Website

5 Digital Marketing Trends

Why Mobile Apps Matter

and repeat this for five years.

You end up with 200 articles and almost no coherent topical authority.

I'd rather build a deliberate cluster.

Suppose the topic is:

Next.js Development

The primary page might be:

/services/nextjs-development

Supporting content could include:

/blog/nextjs-vs-react

/blog/nextjs-seo

/blog/nextjs-performance

/blog/nextjs-app-router

/blog/nextjs-ecommerce

/blog/nextjs-authentication

/blog/nextjs-deployment-aws

Those articles should naturally link to one another where useful.

Now the search engine doesn't see seven random articles.

It sees a connected body of information around a subject.

8. Programmatic SEO Can Be Incredible — or Terrible

Programmatic SEO is one of the most powerful strategies available to companies with structured data.

Imagine a software company serving multiple industries and cities.

You could potentially create pages such as:

/software-development/mangalore

/software-development/bangalore

/software-development/mumbai

Or:

/solutions/hospitals

/solutions/manufacturing

/solutions/logistics

But here's the dangerous part.

If you generate 10,000 pages where the only difference is:

Best Software Company in {CITY}

you haven't created 10,000 useful pages.

You've created one mediocre page 10,000 times.

Programmatic SEO works when the underlying data creates genuinely different value.

For example, location pages could contain different:

case studies
industries
services
pricing considerations
regulations
testimonials
regional data
FAQs

The template scales.

The value cannot merely be templated.

9. Your URL Structure Is Part of Your Information Architecture

I prefer URLs that communicate hierarchy.

Instead of:

/page?id=92831

something like:

/services/web-development

is much easier for humans to interpret.

For larger sites:

/products/laptops/gaming

/resources/seo/technical-seo

/locations/india/bangalore

Your URL isn't going to magically rank because it contains a keyword.

The real advantage is organizational clarity.

Google's crawling documentation similarly recommends logical URL organization that is intelligible to humans.

10. Technical SEO Is Really About Removing Ambiguity

When I audit a website, I'm essentially asking:

Can a crawler understand this website without guessing?

For every important URL:

Does it return 200?

Can Googlebot access it?

Is important content rendered?

Is it indexable?

Does it have the correct canonical?

Is it internally linked?

Is it included appropriately in the sitemap?

Does it duplicate another page?

Does the title explain the page?

Does the page actually satisfy its target intent?

That's technical SEO.

It isn't installing Yoast and watching a traffic-light indicator turn green.

11. Titles and Meta Descriptions Still Matter — Just Not How People Think

I don't write title tags by calculating keyword density.

I think about the search result itself.

Suppose someone searches:

nextjs ecommerce development

Compare:

Next.js Development | Pixels and Grids

with:

Next.js E-commerce Development: Fast, Scalable Storefronts | Pixels and Grids

The second communicates far more information.

The title should help both the search engine and the human understand what they're about to get.

Google specifically recommends unique, descriptive titles and useful page descriptions for JavaScript-powered sites as well.

And remember:

ranking is only half the battle.

If you're position three but your result attracts more qualified clicks than position two, that's valuable.

12. Structured Data Is Machine-Readable Context

Humans can look at:

★★★★★
4.9 from 217 reviews
₹1,499
In Stock

and immediately understand it.

Machines need structure.

That's where structured data becomes useful.

Schema markup can explicitly describe entities such as:

Organization
Product
Article
BreadcrumbList
Event
JobPosting
LocalBusiness

The objective isn't to stuff every possible schema onto every page.

It's to accurately describe what actually exists.

Think of structured data as an additional machine-readable semantic layer.

13. Performance Is Both an SEO and Product Problem

A slow website is annoying regardless of whether Google measures it.

That's why I dislike treating performance purely as:

"We need a better PageSpeed score for SEO."

Performance affects actual people.

A page that loads quickly feels better.

A button that responds immediately feels better.

A layout that doesn't jump around while loading feels better.

When we're building production applications at Pixels and Grids, this often means thinking about:

image optimization
code splitting
caching
CDNs
server response times
database queries
JavaScript bundle sizes
font loading
third-party scripts

SEO and engineering intersect heavily here.

Sometimes an SEO problem isn't solved by an SEO plugin.

It's solved by a developer opening the network tab.

14. Backlinks Are About Trust and Discovery, Not Just Numbers

Backlinks remain important, but counting backlinks is an awful way to understand them.

Imagine Website A has:

10,000 links

from irrelevant directories and automated websites.

Website B has:

40 links

but they're from respected publications, industry organizations and genuinely relevant websites.

Those situations are not equivalent.

I care much more about:

Who is linking?

Why are they linking?

What is the relationship between their page and ours?

The best backlinks often aren't "built" in the traditional sense.

They're earned because something is worth referencing.

That could be:

original research
statistics
tools
calculators
open-source software
technical experiments
unique datasets
detailed case studies

This is one reason I like building technical projects as part of a company's content strategy.

A genuinely useful tool can become an SEO asset itself.

15. Experience Is Becoming More Important in Content

There's another lesson I've learned while writing about our work at Pixels and Grids.

Generic information is becoming incredibly cheap.

Anyone can generate:

"10 Benefits of Having a Website"

in seconds.

What's difficult to manufacture is genuine experience.

Compare:

Redis improves application performance through caching.

with:

We introduced Redis because this endpoint was repeatedly querying the same expensive dataset. It reduced database pressure, but then we had to solve cache invalidation when the underlying record changed.

The second contains something fundamentally more useful:

experience.

For technical companies in particular, I'd rather publish:

How We Designed Multi-Tenant Permissions for a SaaS Product

than:

10 Reasons SaaS Is the Future

I'd rather explain a real architectural decision, what failed, what we changed and what trade-offs we encountered.

That's content competitors cannot reproduce simply by targeting the same keyword.

16. SEO Content Should Have an Information Advantage

Before publishing an article, I like asking:

Why should this page exist when Google already has 10,000 pages answering the same question?

There needs to be some advantage.

Maybe we have:

better depth

or:

original data

or:

real-world experience

or:

better visualization

or:

a tool

or:

a unique argument

or:

technical implementation details

or:

a better explanation.

If your article is simply a rewritten version of the first five Google results, you're adding very little to the web.

17. Measure SEO as a Funnel

Another mistake is obsessing over traffic.

Suppose:

Article A
100,000 impressions
8,000 clicks
2 leads

while:

Article B
8,000 impressions
900 clicks
17 leads

Which article is better?

If you're running a business, probably Article B.

So I think about SEO as:

Impressions
↓
Rankings
↓
Clicks
↓
Qualified visitors
↓
Engagement
↓
Lead / Signup
↓
Customer
↓
Revenue

Google Search Console is particularly useful for understanding the upper portion:

queries
impressions
clicks
CTR
positions
pages

Analytics and your own product data take over further down the funnel.

This distinction is particularly important for Pixels and Grids.

I don't ultimately care whether we receive 100,000 visitors.

I'd much rather have 5,000 visitors consisting of founders and businesses actively looking to build software.

Traffic is not the business objective.

Qualified demand is.

18. My SEO Workflow

If I were starting SEO for a serious website from scratch, my process would broadly look like this:

Phase 1 — Technical foundation

I would establish:

HTTPS
crawlable architecture
correct status codes
robots.txt
XML sitemap
canonicalization
mobile usability
rendering strategy
performance
structured data where appropriate
Search Console
analytics

Phase 2 — Understand the market

Then:

What does the company sell?

Who buys it?

What problems cause them to search?

What language do customers use?

What competitors currently rank?

What queries indicate commercial intent?

Phase 3 — Build the information architecture

Map topics into:

Homepage
│
Services
│
├── Service A
├── Service B
└── Service C

Industries
│
├── Healthcare
├── Manufacturing
└── Logistics

Resources
│
├── Guides
├── Comparisons
├── Case Studies
└── Research

Phase 4 — Build topical depth

Don't publish random articles.

Build clusters.

Phase 5 — Build authority

Create things worth referencing.

Phase 6 — Measure

Watch:

indexation
crawl problems
impressions
query growth
ranking distribution
CTR
conversions
revenue

Phase 7 — Iterate

This is where SEO becomes interesting.

A page ranking #14 doesn't necessarily need another 2,000 words.

Maybe its intent is wrong.

Maybe the title is poor.

Maybe internal links are weak.

Maybe another page is cannibalizing it.

Maybe Google selected a different canonical.

Maybe competitors have much stronger authority.

Maybe the page is simply not good enough.

Diagnosis comes before optimization.

SEO Is Ultimately Information Retrieval

The deeper I've gone into software engineering — and particularly after building my own crawler and search-engine experiments — the less I think about SEO as a collection of marketing tricks.

At its core, SEO sits on top of an information-retrieval problem.

A search engine has billions of documents.

A user gives it a query.

The engine has to determine:

What does this person mean?

Which documents could answer them?

Which documents are trustworthy?

Which are relevant?

Which are useful?

Which result should appear first?

Your job isn't to trick that system.

Your job is to make your website an exceptionally good answer.

That requires marketers.

But it also requires writers.

Developers.

Designers.

Product people.

Data analysts.

And sometimes subject-matter experts.

That's why the best SEO I've encountered doesn't feel like SEO.

The page loads instantly.

The title clearly describes what you wanted.

The content answers your question.

The author actually knows what they're talking about.

Related information is easy to discover.

And when you're finished reading, you don't need to return to Google.

That's SEO.