Time matters in market research. So does cost, and so does scope, and together those three make up the so-called Iron Triangle. I reckon quality belongs in there too, which turns the triangle into a square, even though some people quietly swap scope and quality in and out depending on which one they are about to disappoint you on. Wearing my product developer hat, I see those two as very different things. So, I am going to talk about the Iron Square.

Every new technology in our industry has pushed us the same way: lower cost, faster turnaround, same scope. That tracks the wider economy, where you lift productivity by stripping out the manual repetitive work. Face to face gave way to telephone, the telephone gave way to the internet. Printed reports became online reports, then databases, and now MCP, each step cheaper and quicker than the last.

All that saved time freed up our brains for thinking about question that earns the fee ‘why are things the way they are’. I expect you thought that question would be ‘so what’ or ‘now what’. I would have said that 1 year ago, but now I realize ‘why are things as they are’ is the best question, but I digress…

These improvements are both inevitable and good for the industry, even if they suck the joy out of the craft a little. Fast and cost effective without loss of scope is good in anyone’s language.

But I named four factors in the iron square, and the fourth one is quality. Which is where Synthetic Data enters from left stage.

There are heaps of reasons synthetic data is getting a lot of focus in the research industry, and a few of them are even legitimately useful. Privacy is the legitimate one, in small audiences you can use it to protect people, and that is a real and decent use. You can do clever things with personas too, turning a flat survey dataset into something you can actually poke and argue with.

My problem is the use of synthetic data by platforms and panels to top up or replace real human responses. They sell it as faster and cheaper, in fact that is the whole pitch. Faster and cheaper, on repeat along with some selected example data to show how close the answers are to a tested real data set.

That pitch is wrong, and it is wrong in a way that should bother you as much as it bothers me.

Why is using synthetic data instead of real respondents ethically wrong?

Synthetic data models copy a respondent's answers forever after paying them once, cutting the respondent out of every future sale of their own words. Models also default to the average, erasing the outliers and edge cases that actually drive insight. And provenance disappears: a real survey traces back to an actual person, a field date and a method, while synthetic data traces back only to a vendor's model, never to a single human being.

Pay people properly for their data. A respondent gets paid once, the model then copies their pattern forever, free of charge like a busker whose song gets sampled into a hit they never see a cent from. The value they created is real and it keeps flowing, the payment to them stops at the top. The owner's margin grows precisely because the people who made the thing worth anything have been cut out of every sale after the first one. This is fencing off a common paddock that the respondents built with their own time and their own words.

Diversity is beautiful and edge cases are gold. You cannot model an exception, by definition, models hunt for the middle and its distribution, they drift to the average, because fitting a line to data is just a way of finding the middle. But the answer that changes a decision almost always lives out in the tails, the odd objection, the minority use case, the early adopter who behaves nothing like the crowd. Synthetic data sands all of that off, because those people are rare, and a model treats rare as just noise. You are left with a smooth, confident average and you have quietly lost the very outliers who would have told you something you did not already know. Synthetic data is a monoculture.

Provenance is the whole game now. As AI takes the wheel on more decisions, where a number came from becomes the reason to trust it. Can you trace your decision back to a real dataset? With real survey data you can, because there are real people, a field date, a sample frame and a method you can defend. With synthetic data that chain of reality becomes obscured, and you can trace the output back to a model, but you cannot trace it back to a single person. The training set sits locked in a vendor's vault. Imagine your product launch goes pear shaped and you get grilled by the board.

Board: “We need you to explain your assumptions and data supporting this fiasco.”
You: “Well, they said the model was validated and accurate.”

Um, yep

Ask who owns the training set. When they tell you it was trained on millions of surveys, ask whose surveys? Who paid to collect them? What did those people agree to? Ask if it was funded by past clients and if the people who said yes to one survey, agreed to be training sets for somebody's product. The vendor may be selling an asset built with other people's budgets and other people's words and quite possibly your competitors spend from three years ago. Read the fine print - if you are opting in for your data to be used for training, you are getting duped. Don’t do it.

Can synthetic data actually produce new insight?

No. Synthetic data comes from a model fitted to existing data, so every output is a remix of patterns already fed in. Nothing genuinely new walks through that door. Vendors prove accuracy by matching bulk averages, but real decisions turn on small gaps, where an average-tuned model is systematically wrong. Real data collection can't stop, because a model is only as current as its last update, and it never saw a recession or recall coming.

Accuracy gets oversold where percentage points differences are the deciding factor. Vendors prove their case by showing the fake data tracks the real data in bulk, which looks great on a slide, but real decisions turn on small gaps. Two points of purchase intent, a tracker that moved inside the margin of error. Directionally accurate is useless at that range. A model tuned to the average will be wrong about the small differences in a way that is not random because it bends toward the mean and toward whatever the training data was stuffed with.

Gradient boosting is mathematical bullshit. You can interpolate, you can extrapolate, you cannot make insight out of nothing, no matter how clever the algorithm sounds when the salesperson says it. The math under the bonnet fits a function to data that already exists, and every output is a remix of patterns already fed in. There is no door through which genuinely new information can walk. A fake respondent cannot react to a feature nobody has shown it, it cannot report a life it never lived, it cannot surprise you with something true that you did not already hand it. I strongly believe that insight means learning something you did not know.

You cannot stop collecting real data. You can forecast how events will push behavior around, but the forecast itself is a fresh source of error every time. Real people shift with things the model never saw coming (e.g. a recession, a product recall or world events like Iran getting invaded with a cascade of chaotic outcomes). So, you can get it right most of the time when things are predictable, but the model is only as good as how recently you updated it with today’s state, which is highly conditioned to context (e.g. category). This is a non-trivial assumption that sits under a lot of models.

What is HumanListening building instead of synthetic data?

We are not buying into the synthetic data stuff at all, instead, HumanListening is building what we call private human data sets. These are repositories of survey data from real people and known sources, from people you can name and trace, managed and curated and kept fresh. Owned by the buyer and enriched with context. They are structured and ready for your agents to explore on demand through MCP or people to delve and explore hands on – your call.

We believe real opportunities arise when you bring three things together:

• Human experience, which is what people feel, want and say
• Organizational knowledge, which is what the business already knows but keeps forgetting
• Operational metrics, which is what the business actually does and sells

Most companies keep these three things separate, but our aim is to combine them to support world models to look further into the future, grounded in verifiable authentic data points.

This is how you keep all four sides of the Iron Square in your own hands. You get speed, because the data is already live and ready to go. You get reach because it is built for AI to range across and you keep cost honest, because you stop paying to re-ask the same questions from a cold start every quarter. And you hold onto the fourth side, quality, because the data is real, owned and traceable all the way back to a human.

While you're here, check out our Human-AI Compact a collective choice to be real and maintain quality human to human interactions.