The 3 classic mistakes in retail data projects (and why they fail after the POC)
For every retail data project that ships into production with measurable impact, three or four POC successes quietly die. Here's why — and how to avoid it.
There's a silent but massive phenomenon in retail: the graveyard of data POCs. For every retail data project that ships into production with measurable impact, three or four POCs that ran brilliantly, got celebrated in steering committees, made for compelling slides — and that, six months later, were quietly buried. Not announced as failures. Not formally abandoned. Just left on pause, "waiting for more favorable conditions", until everyone eventually forgot they existed.
This post-POC mortality is one of the great unsaid truths of retail data transformations. Vendors don't talk about it, out of commercial interest. Consultancies don't talk about it, because they ran the POCs. Data leadership doesn't talk about it, because they carry part of the responsibility. And executive leadership doesn't talk about it, because they often don't know it happened — the project simply "evolved in scope" or "got repositioned".
Yet this phenomenon is analyzable. When you look closely at retail data projects that fail after their POC, you almost always find the same three mistakes. Not technical mistakes. Not team mistakes. Three structural mistakes in how the project was designed, invisible during the POC and fatal at scale-up.
This article looks straight at these three mistakes, why they go unnoticed in POC, what they produce at industrialization, and how to avoid them by rethinking how a retail data project is structured.
Why POCs almost always succeed (and what that hides)
Before getting into the three mistakes, you have to understand why the POC phase is misleading. A POC, by construction, runs in favorable conditions that have almost nothing to do with the operational reality of retail production.
In a typical POC, you choose a restricted scope — one category, a few stores, one season. You mobilize a dedicated data team with priority resources. You benefit from sustained sponsor attention, with their best experts made available. You accept to manually clean data to compensate for system imperfections. You tolerate execution delays that would be unacceptable in production. And you measure success on KPIs chosen for readability, not operational representativeness.
In these conditions, almost all POCs work. Predictive models hit announced thresholds. Recommendations are plausible. Demos are convincing. Projected ROI is attractive. Every light is green for scale-up.
And that's exactly when the three structural mistakes reveal themselves. Because retail production looks nothing like a POC. It's multi-category, multi-store, multi-season, in continuous flow, with heterogeneous teams, imperfect data, fuzzy business rules, operational constraints nobody mentioned in the POC workshop. What worked at small scale with dedicated attention stops working at scale without it. And the project enters a stuck zone it rarely leaves.
Mistake #1: confusing model quality with project quality
This is the most universal and costly mistake. Throughout the POC, success was measured by the predictive model's technical quality: forecast accuracy, RMSE, lift over baseline, F1 score on risk-product classification. These technical metrics became the project's language. When the model hit the agreed threshold — say 90% accuracy on a test category — the POC was declared a success.
The problem is that model quality has almost nothing to do with project quality in production. An excellent model that produces recommendations ignored by operations teams creates no value. A moderately accurate model whose recommendations are effectively executed creates a lot. The determining factor isn't technical accuracy — it's the operational adoption rate.
This confusion produces, at scale-up, a scenario that repeats from one chain to another. The model gets deployed. It starts producing recommendations. Operations teams look at them — and ignore 70% because they don't match what they can or want to do. The reasons are many: the recommendation violates a business rule nobody codified, it ignores a supplier constraint nobody mentioned, it proposes an action stores can't execute in time, it contradicts another recommendation made in parallel by another system. The remaining 30% get applied, but without consistency, without follow-up, without learning loop.
The project isn't a technical failure. The model keeps running. The dashboards keep displaying. But operational impact is marginal — and sponsors start wondering if the investment was justified.
Why this mistake goes unnoticed in POC: because the POC doesn't measure adoption rate. It measures technical quality on a scope where adoption was imposed by the sponsor's dedicated attention. In production, without that attention, adoption collapses — and project value with it.
How to avoid it: by making operational adoption rate the central KPI of the project, from the POC phase. Not model accuracy, not RMSE, not technical lift. The percentage of recommendations actually executed on the floor without manual transformation. This KPI shift changes the entire project design: you stop optimizing the model and start optimizing business rule integration, frictionless execution, and operational team trust. Meaning exactly the three elements that make the difference in production.
Mistake #2: treating integration to existing systems as a technical detail
This is the most treacherous architectural mistake. In the POC phase, integration to existing systems — ERP, WMS, pricing, e-commerce, planning tools, BI — is handled manually. You extract the data you need, clean it, push it into the model, get the recommendations back, send them to operations by email or Excel. All of this is doable because it's one-off, small-scale, on a controlled scope.
Scale-up reveals the impossibility of continuing this way. Modern retail production demands continuous recommendations, on tens of thousands of SKU/store pairs, propagated within hours to a dozen different systems. That propagation requires bidirectional integrations, in near real-time, robust to partial failures, capable of handling source data imperfections. It's complex system engineering work, with almost nothing to do with the data science that carried the POC.
And that's exactly when many projects get stuck. The model is ready, recommendations are computed, but they stay in the data system unable to propagate effectively to production. Integrations get pushed quarter after quarter, manual workarounds drag on, and the operational promise never materializes at scale.
Why this mistake goes unnoticed in POC: because integration isn't measured as such. POC demos run on one-off extractions and manual injections. The real integration effort at scale is invisible — it becomes visible at the precise moment it becomes critical.
How to avoid it: by embedding from the start, in the project's very design, continuity between decision and execution as a central requirement. Not as a technical layer to add at the end of the POC. As a founding architectural principle. This means evaluating the decision platform not just on its predictive capability, but also on its ability to plug cleanly into existing operational systems, propagate decisions in continuous flow, handle execution feedback. A retail data project that doesn't ask these questions from the start condemns itself to discover, at scale-up, that half the industrialization work wasn't anticipated.
Mistake #3: running a data project instead of an operational transformation project
This is the subtlest mistake, and probably the most determining of the three. Most retail data projects are structured as data projects. They're sponsored by data leadership or IT. They're run by data scientists and data engineers. They're measured on technical KPIs. And they involve business teams periodically, as "stakeholders" consulted to validate hypotheses.
This structure is consistent with a reading of the project where the main challenge is technical: build a good model, on good data, with good architecture. But that reading is wrong — or rather, it only covers the easy third of the problem. The two hard thirds are organizational and operational: making business teams adopt the new tool, restructuring processes to integrate recommendations, evolving roles and responsibilities, managing natural resistance to change.
When a project is structured as a data project, these dimensions are systematically deprioritized. They land at the end of the roadmap, handled as "change management" you delegate to an HR team or external consultancy. And at scale-up, you discover that operational change wasn't prepared, business teams weren't deeply onboarded, processes weren't redesigned. The project delivers a technical brick into an organization that hasn't changed — and the expected effect doesn't happen.
Why this mistake goes unnoticed in POC: because in POC, operational change isn't required. You experiment on a limited scope, with the one-time agreement of a few motivated teams. The chain's organizational machinery doesn't move — and that's precisely what lets the POC succeed fast. But this absence of change is actually a warning signal: what works in POC without organizational change won't work in production with it.
How to avoid it: by designing the project, from the start, as an operational transformation tooled by data, not a data project with some business impacts. Concretely, three structural changes.
First change: co-sponsor the project by data leadership and an operational leadership (merchandising, supply chain, retail operations). The project shouldn't belong to data — it should belong to operations, tooled by data.
Second change: embed from the start, in the project team, experienced operational profiles who carry process knowledge, unwritten business rules, field constraints. Not as periodically consulted stakeholders — as permanent project team members, with decisive voice.
Third change: structure the roadmap around concrete operational changes (who does what, with what tools, in what processes) rather than around technical deliverables (which models, which data, which integrations). Technical deliverables become means — not ends.
What the three mistakes have in common: confusing means with ends
If you look closely at these three mistakes, you'll notice they share the same logical structure. In each, you confuse the means (the model, the integration, the data project) with the end (operational adoption, decision-to-execution continuity, process transformation).
This confusion is almost always unintentional. It comes from how data projects were born in retail. For a long time, the main challenge was indeed technical: collect data, store it, model it. In that era, optimizing the means was enough — the end followed naturally. Today, the technical challenge is largely solved: every retailer has data, has access to models, has BI platforms. What's missing is no longer the means — it's the conversion of means into operational performance.
Projects that fail after the POC are those that keep optimizing the means, by habit, while it's now the end that needs steering. They deliver a perfect technical brick into an organization that doesn't know how to exploit it — and they fade quietly waiting for more favorable conditions that never come.
What to do differently from day one
Stepping out of this trap requires a paradigm shift in how retail data projects are structured. Not a marginal change — a complete reversal of project logic.
First, redefine project success. Not model quality. Not architecture sophistication. The number of operational decisions successfully executed thanks to the system, multiplied by their measurable economic impact. This KPI must be set from the project's origin, measured continuously, and be the subject of the main steering committee — not in complement to the technical committee.
Second, embed production conditions from the POC phase. Not a POC in ideal conditions followed by a hazardous industrialization. A POC run under the exact conditions of production — same teams, same systems, same constraints — on a restricted scope. If the POC doesn't work under those conditions, it won't work at scale, and it needs to be redesigned. If the POC works under those conditions, scale-up is essentially a volume question — not a qualitative leap.
Third, choose a platform built for production, not an assembly built for POC. Many data projects use in POC an assembly of technical bricks (Python notebooks, processing scripts, ad-hoc dashboards) that aren't industrializable. Scale-up then requires rebuilding everything on a production platform — costly, long, risky. Projects that succeed start from the origin with a platform built for production, on which the POC runs as a subset of the final target.
Fourth, co-build with operations from day one. Not periodic consultation. Permanent co-design, where operations teams aren't ultimate beneficiaries of the project — they're its co-authors. This stance shift transforms adoption dynamics: you don't adopt a tool you helped design, you use it as naturally your own.
The Solya approach: a platform built for production, not for POC
That's precisely the logic structuring Solya. Not a data science tool you industrialize after the fact. Not a POC you have to rebuild for scale-up. A decision and execution platform built, from the origin, for the real conditions of retail production.
Concretely, this translates into three architectural choices that directly address the three mistakes above.
First choice: Solya optimizes adoption, not model accuracy. Recommendations are designed to be directly executable — not theoretically optimal but practically inapplicable. The chain's business rules are embedded at the heart of the engine, not applied as a filter after the fact. Operational adoption rate is tracked as a central indicator, from the first deployment phase.
Second choice: Solya is natively connected to operational systems. Not an analytical layer producing files to re-enter. A platform that integrates bidirectionally with your ERP, your WMS, your pricing, your e-commerce — and propagates validated decisions without breakage, with a delay compatible with commercial cadence. Integration work is anticipated from project design, not discovered at scale-up.
Third choice: Solya deploys in co-construction with operations. The first weeks of deployment aren't an isolated POC. They're a progressive ramp-up on a real scope, with operations teams in the project team, measured on operational KPIs, structured to produce value from the first season. Not a demo followed by a risky cutover — a continuous deployment where each step already produces results.
The result of this approach is measurable. Chains that have deployed Solya step out of the classic successful-POC-then-stuck pattern. The project produces value within the first weeks, operational adoption is high because it was thought through at design time, and scale-up doesn't require rebuild — it's already written into the project's architecture.
The real question to ask
If you've already run, or are currently running, a retail data project, ask yourself this question: are you optimizing the means or the end? Do you measure project success by the technical quality of what you're building, or by the operational decisions it actually transforms?
If the answer leans toward optimizing the means, you statistically have more chances of joining the graveyard of successful-but-stuck POCs than of producing the expected impact. Not from your team's incompetence. Not from technical default. From a structural mistake in how the project was designed — the same one that hits most retail data projects today.
Reorienting the project is almost always possible, provided you do it early. And this reorientation, more than any other decision, will determine whether your data investment ends up producing the performance it promises — or whether, like so many others, it quietly joins the list of projects "that never really took off."
Is your data project on the right trajectory?
At Solya, we offer data and operational leaders a personalized 30-minute diagnostic to assess, on your own context, the structural risks of your current or planned retail data project — and identify the adjustments to make before scale-up reveals costly problems to fix.
👉 [Book your Solya diagnostic] — 30 minutes, by video, with one of our retail experts.
You'll walk away with:
- An evaluation of the three classic risks on your current or planned project
- An estimate of the value potential actually accessible under your current approach
- The adjustments to make to avoid the successful-but-stuck POC trap
Related articles
The 5 weak signals that a product should be liquidated (but your systems ignore)
In the life of a retail product, there's a pivot moment — weeks before the numbers show underperformance, months before the inevitable markdown. Most systems miss it.
Why automating workflows isn't enough if decisions stay manual
RPA and workflow tools made retail processes faster — but the same decisions, just executed faster, won't move margin. Here's the trap, and the way out.
From forecasting to decision: why ML models aren't enough in retail
Retail forecasts have never been more accurate, yet operational results haven't budged. Here's the gap between predicting and deciding — and how to close it.