Systems

Who Decides What We See?

A search box looks neutral. But from the first keystroke, someone's rules decide the order, the ads and what shows up at all. Whose rules are they, and what do they know about us?

· Updated · 6 min read

A single search bar on a light blue grid, with a stack of result lines below it where one line is highlighted in blue, and the words "Who decides what we see?" at the top left.

Something has changed in how I look things up, and I only noticed it recently.

I used to open a search page, type a question, and get a list of links. I would open a few, compare them, and make up my own mind. Lately, more and more, I start with a search and end up chatting with an AI instead. I didn't plan that. It is just where the page led me.

It is easy, and often it is good. But it is not where I meant to go. Instead of ten links to weigh against each other, I get one answer that has already been digested for me. Someone else did the comparing.

That small shift made me ask a question I had skipped for years: who decides what we see when we search?

The map is drawn before you finish typing

A search box feels like a neutral window. You type, the world answers. But the window has a frame, and the frame starts working from your first keystroke.

The suggestions that appear as you type are already a choice. They nudge the question toward phrasings the system has seen before. Then comes the results page, and that page is designed. Some things sit at the top. Some are pushed down. Some never appear.

Most of us rarely look past the first screen. So whoever orders that screen is, in a quiet way, deciding what exists for us on that topic.

Three forces on the results page

Ranking

Every search engine has rules for ordering results. Relevance, freshness, links from other pages, where you are, what language you use, how fast a page loads, and many other signals. The exact rules are private and change often.

I don't think private ranking rules are sinister on their own. If they were public, people would game them even more than they already do. But it does mean that the most important editorial decision on the page is made by a system none of us can inspect.

Ads

Some of what appears at the top is paid for. Ads on results pages are usually labelled, but they sit in the same place and often look similar to everything else. When a result can be bought, the question "what is most relevant?" quietly sits next to another one: "who paid to be here?"

Personalization

Then there is you. Two people typing the same words can get different pages, shaped by location, past searches, device and signed-in history. Researchers and writers have argued for years about how strong this effect really is, and the "filter bubble" idea is still debated. But the basic mechanism is not in doubt: results can be tuned to a profile.

And a profile has to come from somewhere.

It isn't only search

Search is just the clearest example. The apps we open every day do the same kind of steering. A feed decides what comes next. A store decides which products appear first. And notifications decide when we come back at all. A buzz in your pocket is a small decision someone else made about where your attention should go.

We mostly don't notice this yet. It feels like using a tool. Often, it is closer to being guided through one.

Under the hood: how tracking actually works

To understand how a profile gets built, it helps to look at the machinery. I'm going to explain it plainly, without telling you what to do about it. That is a separate conversation.

Cookies, first-party and third-party

A cookie is a small piece of data a website asks your browser to store and send back on later visits. A first-party cookie belongs to the site you are on. It remembers that you are logged in or what is in your cart.

A third-party cookie belongs to a different domain, one whose code is embedded in the page, like an ad or an analytics script. Because that same third party is embedded on many sites, it can recognise the same browser across all of them. That is how browsing on one site can be connected to browsing on another.

Pixels and beacons

A tracking pixel, or web beacon, is a tiny, often invisible image or script loaded from a third party's server. When the page or an email loads it, your device makes a request, and that request carries information such as your IP address, the page you were on, the time, and often an identifier. The pixel shows nothing. Its whole job is to report that you were there.

SDKs inside apps

On phones, much of this happens inside apps rather than browsers. Developers often add software development kits from outside companies for analytics, crash reporting or ads. Those kits run inside the app with the app's permissions. If the app may see your location, the kit inside it may be able to as well.

So say you install a simple weather app. You agree to share location so it can show local weather. You might not know that the same app carries code from several other parties that also receive signals from it.

Device fingerprinting

Fingerprinting works without storing anything on your device. A site or script reads details your device reveals anyway: screen size, installed fonts, browser version, time zone, language, graphics hardware and more. Each detail alone is common. Combined, they can be distinctive enough to recognise the same device again. Standards bodies treat this as a real privacy risk precisely because it is hard for users to see or clear.

Identifiers and cross-site matching

Phones have advertising identifiers, a resettable ID that apps can read, subject to permission, to recognise the same device across apps. On the web, different trackers can share their IDs with each other through a technique called cookie syncing, so that one company's ID for you is linked to another's.

Add a login, an email address or a phone number, which can be hashed and matched, and activity from different places can be tied to one person rather than one device.

Data brokers

Data brokers are companies that collect, buy and combine information about people from many sources, then sell or license it. Purchases, location history, public records, app activity. Most people have never heard of the brokers holding their data, and never dealt with them directly.

From profile to page

Put the pieces together. Signals from cookies, pixels, SDKs and fingerprints are tied to identifiers, matched across sites and apps, and enriched by brokers. The result is a profile: interests, likely income, location patterns, what you might buy next.

That profile can then feed decisions in real time. In ad auctions, information about a visitor is sent out and advertisers bid, within a fraction of a second, for the chance to show something to that person. Personalised ranking and recommendations can draw on the same kind of signals. What you see becomes, in part, a reflection of what the system has concluded about you.

The privacy question we skip

Some devices are sold on privacy, and some of those protections are real. But the apps running on the device bring their own trackers, kits, location requests and clipboard access. A phone can protect you well at one layer while the software on top of it sends data in many directions.

So I keep asking: is that privacy, or just a nicer label on the same machine?

Conclusion

I started with something small: going to search and ending up in a chat. One answer instead of ten links. Behind that sits a larger arrangement, where ranking rules, paid placement and profiles built from tracking decide what appears in front of us, from the first keystroke onward.

I don't have an answer to offer here, and I don't want to pretend to. I only want the question to be asked out loud, by more people:

Who decides what we see when we search?

References

New questions every week. Follow via RSS

← Back to writing