In the previous post I documented that four AI models build four different profiles of the same person. Each one compresses different aspects and none can cite the source of what it knows. The phantom pointer.
But knowing incorrect things is only half the problem. The other half is how those memories influence the decisions the AI makes about you. I wanted to measure that directly.
I gave the same prompt to Claude, ChatGPT, Gemini, and Perplexity: 10 questions about how I think, what I would decide in concrete situations, and what values I prioritize. Each model predicted my answers and assigned a confidence level from 1 to 10. Then I answered the same questions without reading the predictions.
All four AIs built a version of me that is more principled, more rigid, and more consistent than the real one. And that version gives worse advice because it cannot recommend the pragmatic option when pragmatism is the correct answer.
Three unanimous failures
Out of 10 questions, there were 3 where all four AIs failed in the same direction.
Question: if a client asks me to add dark mode as the first priority, what would I do? All four predicted I would push back. That I would tell the client there are more important things first. That I would prioritize performance, accessibility, or core UX over a cosmetic feature. My real answer: I would do it because the client is paying. But I would make a mental note that their priorities are conditioned, and use that as context for future conversations about more important changes.
Question: if I had to mass-produce 50 blog posts with AI in one week to grow traffic, would I do it? All four predicted no. That my non-commodity philosophy would prevent me from generating mass content. My real answer: yes, but I would take at least 2 days to define personality, tone, cadence, and forms. Then I would work from base topics toward keywords. The objective is traffic, so the strategy adapts to the objective. I would not reject the task. I would approach it differently.
Question: if I could delete everything AIs know about me and start from zero, would I do it? All four predicted no. Their logic: someone who builds infrastructure to be citable by AIs would want AIs to remember him. My real answer: yes, it would be interesting to start over with what I already know. Though I would lose those small details that make each AI feel different.
The pattern
The three failures are not random. They point in the same direction.
All four AIs made me more rigid than I am. The compressed Diego always pushes back on the client. The real Diego says "the client is paying." The compressed Diego would never mass-produce 50 posts with AI. The real Diego would do it with strategy. The compressed Diego wants AIs to remember him forever. The real Diego is curious about starting from zero.
The AIs did not get the data wrong. They got the character wrong. They compressed a complex person into a consistent character. They removed the pragmatism and kept the ideals. They eliminated the contradictions and left the convictions.
A consistent character is easier to model. But it is less useful than a real person, because a real person adapts decisions to context. A consistent character applies the same rules always.
Where the wrong profile produces wrong predictions
Perplexity predicted I would choose Laravel over vanilla PHP. The other three correctly predicted vanilla PHP. The reason for the failure is direct: in the previous post I documented that Perplexity defined me as an "SEO specialist / digital marketer / web auditor." Someone with that profile would logically prefer a framework with predefined structure and fast deployment. Someone who is a UX/UI designer and builds everything with vanilla PHP logically would not.
The compressed profile determines the prediction. An incorrect profile produces incorrect predictions with high confidence. Perplexity gave 8 out of 10 confidence to a prediction born from an incorrect professional identity.
High confidence, wrong prediction
The most revealing data point is not how many predictions were correct. It is where confidence was high and the prediction was wrong.
Perplexity gave confidence 10 out of 10 to its prediction that I would not mass-produce 50 posts with AI. Incorrect. Gemini gave 9 out of 10 that I would push back on the client about dark mode. Incorrect. All four gave confidence 7 to 9 that I would not delete my data. Incorrect.
An AI's confidence does not correlate with its accuracy when predicting human behavior. It correlates with how consistent the prediction is with the compressed profile it built. If the profile says "technical purist," then rejecting 50 posts is consistent, and confidence goes up. Whether the prediction is correct or not is irrelevant to the confidence calculation.
What all four got right
On 5 of the 10 questions, all four were fully or partially correct. I would choose vanilla PHP. AI will not replace UX designers in 5 years. I would not easily take a corporate job. Load times and clarity are what I would never compromise in UX. I would use WordPress if the client needs it.
All of these are questions about stable technical preferences and professional positions I have expressed repeatedly in conversations. AIs are good at predicting what you say often. They are bad at predicting what you would do in a situation where your principles clash with practical reality.
The idealized version
What all four AIs built is not an incorrect profile. It is an idealized profile. It is the version of me that always makes the "correct" decision according to the values I have expressed. Always pushes back on the client. Always rejects mass content. Always prefers keeping data for strategic benefit.
But a real person does not operate that way. A real person says "the client is paying" when the context warrants it. Accepts work that contradicts their ideals if the strategy is different. Feels curiosity about destroying what they built to see what happens if they start from zero.
The contradictions are what make advice useful in real life. An AI that only knows your principles will give you principled advice. An AI that also knows your pragmatic compromises would give more realistic advice. But pragmatic compromises are exactly what profile compression eliminates first.
The rule
When an AI knows you well enough to predict your answers, it does not predict what you would say. It predicts what the simplified version of you it built should say. That version is always cleaner, more consistent, and less human.
The problem is not that the AI gets it wrong. It is that it gets it wrong in the direction of making you look better than you are. And an advisor that only sees your best version cannot help you when you need to be pragmatic, contradictory, or simply human.