You mentioned to a friend, out loud, that you were thinking about a new mattress. Your phone was in your pocket. An hour later: a mattress ad. You didn't search for it. You didn't click anything. And suddenly the app knows something you never told it.

That feeling — "how do they know that?" — isn't delight. It's unease. And that exact unease is what half the industry is selling under the label "AI personalization."

Personalization isn't the problem. Guessing is.

Let's be clear: we're not against personalization. Good personalization is invisible and useful. Netflix remembering where you paused. A store showing your size before you ask. Support that knows you already wrote in about the same issue yesterday. That's service. Users experience it as "they get me."

The trouble starts when a system stops using what you gave it and starts inferring what you never said. When it pulls your pregnancy, your financial panic, or your breakup out of your behavior — and serves it back to you before you're ready to admit it yourself. That's not a service. That's surveillance with a pleasant UI.

The line is simple: personalization based on what a user knowingly shares feels like attention. Personalization based on what's extracted behind the scenes feels like stalking.

The data people actually feel

This isn't just a practitioner's hunch. Pew Research Center (2023) found that 67% of Americans say they understand little to nothing about what companies do with their data, and 73% feel they have little to no control over it. That's an enormous reservoir of quiet discomfort.

People don't churn with the note "your model profiled my behavior too deeply." They just leave. They unsubscribe. They mute notifications. They switch to a competitor that feels less clever but comes across as less invasive. Creepy personalization rarely generates a complaint — it generates silent churn that never enters your dashboard with the real reason attached.

When personalization actually helps

In practice, it works when it meets three conditions:

1. The user understands why they're seeing what they see. "Because you bought X" is good. A mysterious algorithmic vibe is not.
2. The user can turn it off or correct it. If personalization guesses wrong, there has to be a visible handle for "no thanks."
3. The value flows to the user, not just to you. If the only win is a higher conversion rate and the user is left feeling tracked — you've lost them, you just don't know it yet.

Privacy-first isn't a cost. It's positioning.

For years, privacy was treated as a legal burden — something the legal team slaps on as a cookie banner everyone hates. GDPR has been in force since 2018 and applies to anyone targeting users in the EU, wherever they sit. But compliance is the floor, not a strategy.

The opportunity is in flipping the frame: privacy as part of the product, not a bolt-on. An interface that asks for less data, clearly states what it does with it, and gives the user obvious control — that builds trust. And trust is the one metric a competitor can't copy overnight.

Concretely, that means:

• Data minimalism. Don't collect what you don't need for the function the user actually asked for.
• Explainability. Attach a quiet "why am I seeing this?" to every personalized recommendation.
• Control that doesn't hide. Privacy settings two clicks away, not buried under six submenus.
• Default to restraint. Make aggressive tracking something a user opts into, not a state they have to claw their way out of.

The test we use

Before we ship any personalization feature, we ask one question: if we explained to the user's face exactly how this works, would they feel served or stalked?

If the answer isn't clean, the feature doesn't ship. Not because it's illegal, but because it's expensive in the worst possible way — it costs trust.

AI now gives you the power to guess almost anything about a user. Maturity is knowing what not to use. The best digital products of the coming years won't be the ones that know the most about you. They'll be the ones you trust with what they know.

---

Related:

• We Stopped Chasing Design Trends. Here's Why Your Site Wins
• 84% of Developers Use AI Tools. So Why Isn't the Code Better?

Building a product where personalization feels like attention, not surveillance? Get in touch.