Why 70% of the value doesn’t require a full data science team.
Parts one and two described a frame and an engine. Neither is worth anything without an answer to the question every executive asks third: who builds it, and do we need to hire a data science team?
The answer is mostly no, and the money argument is stronger than the technical one.
Tools, by what they let you actually do
| Level | Tools | What is genuinely reachable | Who |
|---|---|---|---|
| Manual | Excel, pivots, Solver | Descriptive, localization, simple what-if, simple rules | Any strong analyst |
| BI-native | Power BI, Tableau, Qlik, ThoughtSpot | Decomposition, key influencers, top segments, forecast, anomalies, what-if | BI analyst |
| Low-code | KNIME, Dataiku, Alteryx, RapidMiner | Workflows, anomaly detection, ML, SHAP, scoring, causal prediction | Advanced analyst |
| AutoML | Azure AutoML, H2O, Dataiku AutoML | Prediction, forecasting, feature importance | Advanced analyst / junior DS |
| SQL & Python | SQL, PyOD, ruptures, SHAP, DoWhy/EconML, OR-Tools | Custom baselines, slice scans, causal work, optimization | DS / analytics engineer |
| Enterprise | Vendor platforms | End-to-end workflow, alerting, optimization | Mature organisations |
| LLM copilot | ChatGPT, Copilot, KNIME K-AI | Draft SQL and DAX, DAG checklists, summaries, evidence labels | Analyst, under governance |
One correction to how that last row is usually presented: an LLM copilot is not a separate evidence layer. It is an accelerator that sits across all the others. It moves you along a row faster; it does not move you down the table. Treating it as a rung of its own is how teams end up with a beautifully drafted causal analysis that was never identified.
What a normal BI analyst can do unaided
This is the list I would put in front of a sceptical CFO, because it is longer than anyone expects.
Alone, with no data scientist:
- a correct Store × SKU × Week fact table, a baseline, and a sales gap
- contribution analysis, decomposition trees, key influencers, top segments
- a KNIME anomaly and SHAP workflow
- opportunity scoring and missed-sales estimation
- rule-based recommendations
- evidence labels on every finding
- re-checking findings on the following period - the cheapest validation that exists
With light data science review:
- Granger tests, after seasonality is handled properly
- a simple store experiment, a simple difference-in-differences
- synthetic control or BSTS
- low-code causal prediction
Never ship without causal expertise:
- a PCMCI graph presented as proven causality
- observational double machine learning, causal forests, instrumental variables, regression discontinuity, staggered DiD
- anything with strong cannibalisation or interference between units
The constraint is not Python. Every estimator in the third list is one import away and will return a confident number on any data you give it. The constraint is that a mathematically valid estimator applied to data where the effect is not identified produces a wrong answer with a tight confidence interval, and nothing in the output warns you.
AutoML and the last thirty per cent
Worth being specific about what the automated tools do and do not close.
- Language models are a genuine causal co-pilot: turning a vague question into treatment, outcome, unit and time; drafting a DAG; asking the missing-variable questions; spotting post-treatment variables; generating the checklist and the code. They are not a judge.
- Azure AutoML and H2O are a strong predictive layer - classification, regression, forecasting. They are not causal design and do not claim to be.
- Dataiku Causal Prediction is the closest low-code causal offering: pick an outcome and a treatment, receive uplift estimates. It does not check confounding, overlap or interference for you.
- DoWhy has the right logic - model, identify, estimate, refute - and is explicit about it. But its guarantee is “if this DAG is correct, here is the adjustment set and estimator.” Never “your DAG is correct.”
- KNIME K-AI generates workflows, visualisations and code from prompts. Excellent scaffolding, not a design validator.
The conclusion generalises past this tool list and will outlive it: the last twenty to thirty per cent - the causal review - cannot be handed to automation. Everything before it can be, increasingly. That ratio is the thing to plan hiring around.
Hiring: a small capability, not a saviour
The failure mode is hiring “a data scientist who will figure it out.” What is needed is a small capability with five distinguishable roles, only some of which need to be full-time and one of which can be an advisor.
- Analytics translator / commercial analytics lead - probably the pivotal hire. Knows RGM, trade promotion, category management, sell-in versus sell-out, distribution, availability, retailer data. Their job is to convert a business question into treatment, outcome, unit and time.
- Causal inference specialist - often part-time or advisory. DAGs, DiD, synthetic control, BSTS, propensity and DML, causal forests, sensitivity analysis, interference.
- Discovery ML - anomaly detection, residual baselines, slice finding, SHAP, ranking, stability testing. Does not need to be a causal expert.
- Data / analytics engineer - critical and routinely under-hired. Hierarchies, promo calendar, price, stock, competitor data, lagged features, refresh reliability.
- BI / product engineer - turns output into a workflow: alerts, evidence snapshots, an owner and an action, and a feedback path.
A better interview task than an abstract ML exercise. Hand the candidate a Store × SKU × Week table - sales, price, promo, display, availability, distribution, region, retailer, competitor promo - and ask them to: state a business question; choose treatment, outcome, unit and time; sketch a DAG; name confounders, mediators and colliders; say explicitly what must not be controlled for; propose a route from diagnostic to causal; and describe how they would present the result to a commercial director.
Red flags: “put it all in XGBoost and SHAP will show the causal drivers”; “DML handles causality for you”; “the model drew the DAG, so we’re fine”; “control for everything”; “Granger proves causation.”
Green flags: “first let’s define the intervention”; “that’s post-treatment, we can’t control for it”; “total effect and direct effect need different controls”; “this is an association until we validate it.”
On interns and junior data scientists: use them as controlled prototyping capacity - benchmarking influencer algorithms, a missed-sales prototype, opportunity scoring, a library of causal designs, an evidence-label system. Do not leave them without a frame, or you will receive XGBoost plus SHAP plus a causal forest, unvalidated, in a deck.
Why this is the cheaper path
Three places where money is saved before a single model reaches production.
You do not buy what you do not need. Once the work is decomposed properly, the first 70% of the value comes from correct grain, a real baseline, slice finding, association and opportunity scoring. None of that is a multi-month consulting programme. Most of it is a competent analyst, a BI licence and six months.
You do not set false KPIs. The most expensive mistake in this domain is demanding causal proof from a diagnostic model - it produces either a fabricated answer or an eighteen-month project that ends in an apology. Understanding the evidence rungs is what prevents commissioning the wrong thing, and that protection costs nothing but a shared vocabulary.
You buy external expertise at a much better price. If your team has already done the data inventory, the metric definitions, the use-case map, the first association layer and a candidate DAG, then the external expert’s scope is “review this design and validate the top three cases” rather than “build everything from scratch.” That is a different engagement with a different number on it.
The executive version:
In many FMCG organisations, 70% of the practical value does not require a full data science team. It requires disciplined commercial analytics: clean grain, consistent baselines, automated slice finding, association discovery, opportunity scoring and a decision workflow. The remaining 30% - causal inference, experimental design, advanced ML, production-grade optimization - should be bought or hired selectively, and only once the internal team knows what needs expert validation.
Which produces a conclusion that is anti-consulting and, I think, still correct:
The first AI analytics project in FMCG should not be a vendor implementation. It should be an internal literacy and scoping exercise.
What I cannot tell you
The 70/30 split is a judgement from decomposing the work, not a measured figure, and I would not defend the exact numbers. What I would defend is the shape: the cheap layer is larger than the industry’s marketing implies, and the expensive layer is narrower and more specialised than generalist hiring assumes.
The public FMCG and retail cases that support feasibility here - PepsiCo’s Store DNA on Azure ML, McKinsey’s roughly 2% incremental sales on trade promotion, NIQ’s missed-sales and recovery figures, the Kroger and 84.51° causal impact tests - are vendor or customer claims. They demonstrate that the thing can be built. None of them publishes treatment assignment, standard errors, pre-trends, spillover handling or raw data, so none is an independently verified effect, and I would not use any of their numbers to size a business case.