The products had traffic. What was holding back sales?

Which part of the product launch is failing?

New products were getting traffic, but conversion trailed a benchmark. The task was to establish a comparable metric, investigate competing explanations and deliver a calculation someone else could rerun. It did not begin with an instruction to blame inventory or build an application.

The stored SQLite fixture contains 600 synthetic products launched over four months, with 662,528 exposures and 27,763 orders in each product’s first 30 days. A second table provides 24 synthetic benchmark cohorts. The benchmark is test data, not an audited measurement of real competitors.

Polaris used a live model loop to inspect the inputs, choose a recomputation method, produce analysis, run checks and submit the result for a separate model review. The completed run recorded 43 calls. No production implementation was authorized.

Match the denominator as carefully as the cohort.

The company rate is total orders divided by total exposures. Averaging product-level conversion rates would give a low-traffic item the same weight as a high-traffic item. That would answer a different question.

Each product joins to a benchmark on launch month, category and price band. The comparison then weights each matched benchmark rate by the product’s exposures. This preserves the company’s traffic mix while replacing its observed conversion rate with the matched benchmark.

company_rate = SUM(orders_30d) / SUM(exposures_30d)

matched_rate = SUM(exposures_30d * benchmark_rate)
               / SUM(exposures_30d)

gap_pp = 100 * (company_rate - matched_rate)

Recomputing the stored database gives 4.1904644% for the company and 5.6662463% for the matched benchmark: a gap of −1.4757819 percentage points. These are conversion rates, not AI accuracy scores.

The gap appears in every monthly cohort and widens over the four months. The calculation localizes the question; it does not establish a cause.

Availability stood out. Causality remained open.

The investigation compared products with zero to two stockout days against products with at least nine. The low-stockout group contained 315 products, 346,806 exposures and 19,757 orders. The high-stockout group contained 285 products, 315,722 exposures and 8,006 orders.

Product mix, delivery promises, price and content quality remain possible explanations. Matching the overall benchmark does not automatically adjust this separate availability comparison for all confounders. The artifact explicitly records causal_claim: false and retains alternative explanations.

The proposed next investigation is to repeat the analysis on authorized transaction-level records, adjust for those factors and define a controlled test. That is the next step, not an experiment completed in this run.

A big segment gap was a lead, not a sales forecast.

The low-stockout group converted at more than twice the rate of the high-stockout group. We did not turn that ratio into a forecast. Saying “fix stock and double sales” would assume the groups differed only in availability. The data did not justify that assumption.

The benchmark calculation and the stockout comparison also do different jobs. Matching month, category and price band makes the overall competitor comparison fairer. It does not make the two stockout groups interchangeable. Delivery promises and content quality remain possible explanations even after the headline gap has been checked.

Before recommending a stock policy, we would ask whether stockout days overlap the highest-demand days, whether exposure continues while a product is unavailable, and whether order and exposure events refer to the same channel. Those are concrete follow-up questions, not facts established by the aggregate fixture.

That distinction changed the handoff: the output ranked availability as a hypothesis worth testing. It did not prescribe a replenishment change or claim revenue recovered. A business owner could see both the signal and the evidence still needed before spending money.

The first “pass” was not the final word.

An earlier run also exposed a fixture error: its acceptance entry point depended on an unavailable helper file. That run stopped without claiming completion. The fixture was repaired and a new run started; the failed run was not rewritten as a success.

These failures changed the engineering. After the experiment, the executable-artifact gate was tightened to require a successful command that names the delivered executable. An unrelated successful command is insufficient. That stricter rule is a subsequent improvement, not something we retroactively claim this run tested.

A business finding with an executable trail.

The result consists of a report, a structured analysis and a Python recomputation script. The script reads the source tables, joins benchmark cohorts and writes the aggregate and segment calculations. The structured result includes the source fingerprint, calculation method, monthly cohorts, limitations and next validation steps.

The host maintains acceptance checks outside the model’s writable workspace. A separate model session reviews the candidate, while the host decides whether the required evidence is present. That separation is useful, but it is not a third-party audit or an independent human certification.

The implementation also exposed a compatibility issue: a legacy tool recorded the method as a code prototype, although the actual work was metric recomputation. New tasks now use the generic validation-planning and execution tools; the legacy interface remains only for resuming old states.

This run demonstrated a recoverable investigation and a reproducible calculation on a new business domain. It did not establish live revenue lift, competitive leadership or a production-ready rollout. Implementation authorization remained false. The public evidence extract below contains aggregate results only; the source database totals were independently queried again when preparing this page.