Main site

Retail AI agent · Shelf vision

Shelf-Vision & Planogram-Compliance AI Agent

Scan fixtures against the planogram version the agent has pinned, report gaps and mis-sets with a confidence score attached, and leave every supplier release to the category manager.

Typical delivery
4–6 weeks
Deployment
Your stack
Category manager
Before release
After launch
Agent Care

What this agent does

Counts the shelf, never certifies it

In

  1. 01

    Capture fixtures from a fixed camera, a mobile unit or an autonomous scanner across supported store estates.

  2. 02

    Pin the planogram version a capture is scored against, and refuse to score when that version is unconfirmed.

Reason

  1. 03

    Detect facings, gaps, set order and price-label mismatches against the pinned plan.

  2. 04

    Attach a confidence score and a capture time to every detection that leaves the model.

  3. 05

    Mask human figures at the edge of frame on the device, and drop the raw image once the product signal is out.

Decide

  1. 06

    Raise an internal replenishment or label task where the detection clears the threshold.

  2. 07

    Hold every supplier-facing report, dataset or scorecard for the category manager.

Out

  1. 08

    Retain the capture, the planogram version, the confidence and where the report went.

  2. 09

    Execute write actions only inside the approval boundaries agreed during implementation.

Product statement

The agent reports what it saw, and most of that is ordinary commercial detail; the category manager decides what reaches a supplier.

Example workflow

One fixture, capture to release

  1. 1Capture receivedFixed camera, mobile capture, autonomous scanner or a store-team photograph
  2. 2Frames assembledPinned planogram version, fixture map, item master and the capture time
  3. 3Detections producedFacings, gaps, label mismatches, flagged frames and confidence
  4. 4Quality checks runVersion-pin checks, human-figure masking, retention rules and confidence threshold

No human action required

Every stage before the confidence gate runs unaided, and nothing leaves the building at any of them — the agent is reporting internally, and the merchant's lane opens at the gate.

5DecisionSplits on the detection threshold
Clear detection

Goes to the category manager to approve.

Weak detection

Adds a store-audit read first.

Merchant approval

The report is held with its captures, its flagged frames and the confidence.

Approve · Amend · Send to store audit

Approved — cleared to share

  1. 6Merchandising systems updatedOnly where write access and approval policy allow it
  2. 7Capture evaluatedAudit agreement, merchant amendments, disputed detections and post-release corrections

Re-captures

Every merchant amendment is counted in the evaluation.

What should not run autonomously

Human approval stays in control

Outside the boundary — human approval required

8 items

  • Releasing a dataset, report or scorecard to a supplier.
    • Setting the terms compliance data is shared on.
    • Issuing a supplier chargeback, penalty or score adjustment.
    • Certifying compliance for a trade-funds or allowance payment.

  • Automation boundary

    Agent acts unaided

    • ✓Scan fixtures and produce internal detections carrying confidence.
    • ✓Raise an internal replenishment task on a detection for the named owner.
    • ✓Flag a price-label-to-system mismatch for.
    • ✓Halt itself and ask for help on obstruction.

    Any write happens inside the boundaries agreed at implementation, never ahead of the merchant's release.

    • Authorising an autonomous unit into a customer-occupied aisle.
    • Overriding an obstacle-detection or safety parameter.
    • Changing image retention, resolution or storage location.
  • Confirming the authoritative planogram when the pin fails.

Example output

One detection, annotated

Everything the agent reports is attached to the capture it was drawn from.

Detection output · single fixture

Illustrative example
Fixture
Chilled dairy bay
Detection
Front rank empty across two facings, stock standing behind the front row
Plan state
Set to plan
Version of record
Pinned plan, current reset
Confidence
87%
Where it went
Internal replenishment queue

As received

Taken from the capture and the pinned planogram — nothing on this side is inferred by the agent.

Evidence used

  • Pinned planogram version
  • Fixture map
  • Earlier capture of the bay

Why this reading

It describes the facing, not the store team's execution and a person still decides.

What the score decides

Below the configured threshold the report picks up a store-audit read before it reaches.

Action

ApproveAmendSend to store audit

Value

Where AI adds value

The same four claims, placed at the point in the workflow where each one applies.

Where the value lands

Value 01 – 04

Every fixture

From the capture

03Detection

Score against the pinned plan

Draw on the planogram version pinned for that store, the fixture map and the item master.

01Approved path

See the gap, not the guess

Routine gap and facing counts arrive already scored and already timestamped.

02Human review

Send review to what leaves

Supplier-facing reports and low-confidence detections are flagged, so the merchant's read starts where exposure sits.

04

Build an evidence trail

Each report keeps its capture, its confidence and where it was sent.

Integrations

Typical integrations

Five system groups connect to the same agent. Which of them are in scope is decided in discovery.

  • Shelf-scanning robots

    • Simbe Tally
    • Brain Corp BrainOS

  • Fixed-camera shelf vision

    • Pensa Systems
    • Focal Systems

  • Shelf image recognition

    • Trax Retail
    • ParallelDots ShelfWatch

Agent

Shelf vision and planogram compliance

  • Reads the plan
  • Scores the shelf
  • Holds for release

  • Space planning and cameras

    • Blue Yonder Category Management
    • Nielsen Spaceman
    • Verkada

  • Observability & evaluation

    • OpenTelemetry
    • Langfuse
    • Supported monitoring/evaluation sources

Integration availability depends on the client's existing systems and API access.

Agent controls

Six layers between the model and the supplier

Controls nest, and what each misses is named underneath.

L6 · Outermost — last line of defence

Inward → L1 · closest to the model

L6Rollback / safe modeFall back to raw capture and internal-only reporting when evaluation or production signals degrade.Roll back
L5Version monitoringTrack model, firmware, planogram-version and threshold changes.Track
L4TraceabilityRecord the capture, the version scored against, the confidence and the recipient.Record
L3Merchant approvalHold reports for the named category manager; it governs release, not whether a released number is right.Gate
L2Policy guardrailsTest detections against the pinned plan, the masking rule and the retention rule; a failure returns the report.Restrict
L1Confidence thresholdsRoute low-confidence reports to a store-audit read before the merchant sees them.Require review
Model coreReport produced — detections, flagged frames, capture time and confidence
L1 – L2
Test whether a report may stand
L3
Puts the release in a merchant's hands
L4 – L5
Keep the capture and its confidence together
L6
Falls back to raw capture when signals degrade

How Nestack evaluates it

Evaluate the whole capture path — not only the final count.

Coverage runs the whole depth of the workflow, and every layer is cut by slice.

Surface — the number the merchant reads

  1. E1Final-output evaluationDid the reported state match what a human audit found on the fixture?
  2. E2Step-level evaluationDid the agent score against the planogram version actually in force that day?
  3. E3Tool evaluationDid it write to the correct store, the correct fixture and the internal queue?
  4. E4Confidence calibrationDo low-confidence detections actually attract more merchant amendments?
  5. E5Slice evaluationHow does performance change across specific fixture types?
  6. E6Business outcomeHow many released reports needed a correction once a supplier had read them?

Floor — the shelf the shopper walks past

Failure modes

Where each failure originates in the agent

Seven modes across the capture lifecycle.

Agent lifecycle

Direction of processing →

  1. 01 · Retrieval

    1 mode

    • GX-03

      Stale plan pinned

      Last season's planogram is scored against a correctly reset aisle.

    Stage gathers

    Captures, planogram versions, with the source each came from

  2. 02 · Reasoning

    2 modes

    • GX-04

      Fronted row read as empty

      A shelf full behind the front rank is reported as out of stock.

    • GX-06

      Supply gap read as execution

      A distribution shortage is attributed to the store team's set.

    Stage proposes

    Detections, flagged frames and confidence

  3. 03 · Tool / write

    2 modes

    • GX-02

      Unreviewed claim written

      A compliance failure reaches the vendor system before anyone has read it.

    • GX-05

      Identifiable frame persisted

      A full frame carrying shoppers is stored where the masking rule said none would be.

    Stage writes

    Only where write access and approval policy allow it

  4. 04 · Output

    1 mode

    • GX-01

      Confidence lost in transit

      A count is presented without the score that qualified it.

    Stage returns

    The report the merchant releases and the supplier

  5. 05 · Change / Version

    1 mode

    • GX-07

      Threshold drifts on update

      A firmware or model update moves detection thresholds across the fleet overnight.

    Stage tracks

    Model, firmware, planogram versions and thresholds

  • Sev-1 · a held act performed by the agent
  • Sev-2 · a wrong number reaches a supplier
  • Sev-3 · capture degrades, report routes to review
See the complete Retail failure-mode catalogue

Affected slices

The average fixture is not the hard one

Start at the all-capture baseline: across every fixture scanned the false-non-compliance rate sits low. Split the same rate by fixture class and a few cohorts carry several times.

Slice performance — reported separately, not only in aggregate

Illustrative example
SliceFailure rateLiftLift vs. thresholdStatus
Small-facing, visually similar SKUs9.5%3.4×Review
Chilled, frozen and gravity-fed doors7.0%2.5×Review
End-caps, dump bins and seasonal4.2%1.5×Watch
Large-format staples, flat gondola1.4%0.5×Normal

Bar: false-non-compliance lift vs. the all-capture baseline · scale 0–4.0× · tick marks the 2.0× review threshold

2 of 4 slices over threshold

Evidence-linked improvement

A cycle is closed by a case

The loop ends where the failure becomes a standing case the next release must clear.

Improvement cycle · five stages

Switchback — the path turns at Improve and returns at Learn

  1. 01

    Detect

    False non-compliance rises on one fixture class.

  2. 02

    Diagnose

    The paired captures and the pinned planogram version sit side by side until the cause narrows to one.

  3. 03

    Improve

    The change ships versioned, with the shelf captures that exposed it attached.

  4. 04

    Verify

    Nothing goes live while an affected case is still failing.

  5. 05

    Learn

    It becomes a permanent case and a change to the capture rules.

  6. Learn → Detect

    The return edge. The next detection is measured against a suite this cycle lengthened.

Typical build scope

Twelve workstreams across six weeks

The build scope read against the delivery timeline. Week structure follows the six-week plan — discovery, capture, detection workflow, evaluation, integration, then production validation and handover.

Workstream

Week 1Week 2Week 3Week 4Week 5Week 6

  1. 01Store and fixture discovery, and boundary definition.Week 1
  2. 02Camera and planogram-source assessment.Week 1, Week 2
  3. 03Planogram-version, fixture-map and item-master mapping.Week 1, Week 2
  4. 04Capture ingestion and masking at the edge.Week 2
  5. 05Detection logic and version pinning.Week 2, Week 3
  6. 06Confidence scoring and merchant routing.Week 3
  7. 07Merchant release workflow.Week 3, Week 4
  8. 08Shelf-system and merchandising integration.Week 5
  9. 09Detection regression cases.Week 4, Week 5
  10. 10Guardrails and capture controls.Week 4
  11. 11Capture-trail instrumentation.Week 5, Week 6
  12. 12Deployment, documentation and Agent Care handover.Week 6

12 workstreams · 6 weeks · bar shows the weeks a workstream is active — several run in parallel

Final scope and sequence confirmed in discovery

Engagement tiers

What each tier includes

Rows are the capabilities named in each tier's scope. Higher tiers include everything below them.

Capability✓ in scope · — not at this tierPilotOne store, one categoryProductionProduction store estateAdvancedMultiple banners / formats
Introduced at Pilot
Detection against your pinned plans✓✓✓
Merchant approval✓✓✓
Detection baseline✓✓✓
Introduced at Production
Reporting by fixture—✓✓
Approval workflow in your systems—✓✓
Approved internal write-back—✓✓
Shelf-system integration—✓✓
Introduced at Advanced
Multi-banner planogram rules——✓
Multi-stage merchandising approvals——✓
High capture volume——✓
Enterprise capture controls——✓
Build priceFrom $5,000From $8,000Custom quote
Final build price
Confirmed after discovery based on integrations, workflow complexity, transaction volume, approval controls and deployment requirements.
Separate from build
Build pricing is separate from recurring Agent Care, which covers managed monitoring, evaluations, incidents and verified improvements after launch.

What we need from you

What you bring, and what we build with it

Each input maps to a piece of build scope and a week in the delivery timeline.

  1. 01Your planogram versions and fixture maps

    Planogram-version and fixture mapping

    Week 1

  2. 02Representative captures already taken

    Detection baseline and version pinning

    Week 2

  3. 03Your retention rules and entrance signage

    Retention and masking-rule mapping, and automation-boundary definition

    Week 1

  4. 04Access to relevant APIs, feeds or exports

    Camera, robot and capture-source assessment, then integration setup

    Week 2

  5. 05Reports the merchant did not trust

    Detection cases and paired captures

    Week 4

  6. 06Where a report stops and a person looks

    Detection thresholds, task routing and controls

    Week 3

  7. 07A named category manager to read reports

    Merchant release workflow, then pilot and production validation

    Weeks 5–6

Nothing else is required

Deployment, documentation and Agent Care handover are ours.

Delivery timeline

Four phases across six weeks

Each band spans only the weeks it actually runs, and the fifth is doubled for a real reason.

Phase

W1

W2

W3

W4

W5

W6

  1. DiscoveryW1
  2. BuildW2 – W3
  3. EvaluateW4 – W5
  4. Pilot & LaunchW5 – W6

Week focus

W1Fixture discovery, retention mapping and the automation boundary

W2Fixture and planogram data connected

W3Detection logic, confidence scoring and release controls

W4Detection testing and capture guardrails

W5Merchandising integration, pilot stores and targeted corrections

W6A month of capture reviewed by the merchant, then handover

Reading the band
The overlap is real: evaluation and pilot run together in week 5.
At the end of W6
Verified on live fixtures, then monitoring sits with Agent Care.
Duration
Six-week plan shown · typical delivery 4–6 weeks depending on scope confirmed in discovery.

Next step · Retail AI agent

Build a shelf-vision agent around the merchant who releases the report.

Show us your fixtures, your planogram versions and where your cameras already point. Your category manager decides what reaches a supplier, so we set the version pin, the masking rule and the equal-terms record before any report leaves the building.

Typical delivery
4–6 weeks
Deployment
Your shelf systems
Category manager
Before release
After launch
Agent Care available

Nestack Agents · Shelf vision and planogram compliance

AGT-RT-11 · Agent Care available after launch

Build a shelf-vision agent

Four questions so the call starts from your actual setup — then pick a time that suits you.

About you
Your shelves

No obligation. We’ll map the workflow, name the automation boundary and tell you if an agent is the wrong answer.