Main site

Construction AI agent · Site vision

Site-Vision & Progress-Capture AI Agent

Capture the site, index it to the model and the date, and rank what looks wrong — then route each observation to a named owner on a clock, because one nobody closes is evidence.

Typical delivery
4–6 weeks
Deployment
Your stack
Safety review
Identity-free
After launch
Agent Care

What this agent does

Captures the site; it does not call the breach

In

  1. 01

    Capture from 360 walkthroughs, fixed cameras and scheduled flights, each flown by a named certificated pilot.

  2. 02

    Blur faces and mask off-site areas at ingest, before a frame reaches a screen — no face or gait template is made.

Reason

  1. 03

    Register each capture to the model and the date, and index it to the level, the zone and the capture position.

  2. 04

    Compute geometric deltas between captures and against the baseline revision the project has named as current.

  3. 05

    Compute progress per element as an estimate, carrying its method and confidence — never as a certified figure.

Decide

  1. 06

    Rank candidate observations against the rule set configured for this site, not against a national default.

  2. 07

    Route each ranked observation to the named person with authority to abate it, with an escalation clock running.

Out

  1. 08

    Retain the frame, the rule cited, the routing, the owner and the closure decision against the observation.

  2. 09

    Execute write actions only inside the approval boundaries agreed during implementation.

Product statement

The agent raises candidates; the site safety manager decides what is a breach — and once raised, an observation cannot be unrecorded, only closed, late and on the record.

Example workflow

One capture run, frame to owner

  1. 1Capture run receivedA 360 walkthrough, a fixed-camera series or a flight with a named remote pilot
  2. 2Frames de-identifiedFaces blurred and off-site areas masked at ingest, before any person sees the frame
  3. 3Candidates rankedDetected conditions, progress deltas, the frame and confidence
  4. 4Controls appliedSite rule set, capture-boundary checks, recognition gate and confidence threshold

No human action required

Stages 1 to 4 run unaided, and nothing is a breach at any of them — the agent is ranking candidates, and the safety lane opens at the confidence gate.

5DecisionBranches at the confidence threshold
High confidence

Goes to the owner who can abate it.

Low confidence

Picks up a site-safety triage read first.

Safety review

The observation is held with its frame, the rule cited and the confidence.

Assign · Reclassify · Escalate to the safety manager

Assigned — owner and clock set

  1. 6Site systems updatedOnly where write access and approval policy allow it
  2. 7Outcome evaluatedMissed hazards, false positives, time to closure and observations closed with no action

Closures

Every closure decision is counted in the evaluation.

What should not run autonomously

Human approval stays in control

Outside the boundary — human approval required

8 items

  • Enrolling, templating or matching a face or gait.
    • Deciding a detected condition is a breach.
    • Directing a worker to stop, or clearing work to resume.
    • Certifying a progress claim or completion percentage.

  • Automation boundary

    Agent acts unaided

    • ✓Capture, stitch, register and index imagery to the model and date.
    • ✓Compute geometric deltas against the baseline.
    • ✓Rank candidates and route them with an escalation clock.
    • ✓Blur faces and off-site areas at ingest, before anyone sees a frame.

    Writes happen inside the boundaries agreed at implementation, never onto a certificate.

    • Authorising flight over people or beyond line of sight.
    • Closing an observation without action.
    • Publishing imagery outside the authorised list.
  • Feeding an observation into a disciplinary decision.

Example output

One observation, annotated

Everything the agent raises is attached to the frame it came from, and to the clock it starts.

Observation output · single frame

Illustrative example
Zone
Level 4, north face
Detected condition
Unprotected edge at a slab penetration, adjacent to a stacked load
Measured gap
1.8 m opening
Rule cited
Edge-protection rule
Confidence
88%
Routed to
The exposing employer's supervisor

As received

Taken from the registered frame and the configured rule set — nothing here is decided by the agent.

Frame evidence used

  • Registered frame
  • Baseline model
  • Prior capture

Why it was raised

It matches a rule configured for this zone — the breach call is the manager's.

What the score decides

Below the threshold the observation picks up a triage read before an owner.

Action

AssignReclassifyEscalate to the safety manager

Value

Where AI adds value

The same four claims, placed at the point in the workflow where each one applies.

Where the value lands

Value 01 – 04

Every capture run

From the capture run

03Detection

Rank against the site rule set

Draw on the baseline model, the prior capture and the rule set configured for this site.

01Approved path

Point the walk at the risk

Routine coverage arrives already indexed to the level, the zone and the date.

02Human review

Put the walk-round on the list

Ranked candidates carry their confidence and their clock, so the walk-round starts at the top.

04

Build an evidence trail

Each observation keeps its frame, its confidence and who it was routed to.

Integrations

Typical integrations

Five system groups connect to the same agent. Which of them are in scope is decided in discovery.

  • Reality capture

    • OpenSpace
    • StructionSite
    • Matterport
    • Reconstruct

  • Drone and survey

    • DroneDeploy
    • Propeller Aero
    • Surveyed capture positions

  • Progress and safety vision

    • Buildots
    • Disperse
    • Voxel
    • Protex AI

Agent

Site vision & progress capture

  • Reads the frame
  • Ranks candidates
  • Routes to an owner

  • Project systems

    • Procore
    • Autodesk Build
    • BIM baselines
    • EarthCam

  • Observability & evaluation

    • OpenTelemetry
    • Langfuse
    • Supported monitoring/evaluation sources

Integration availability depends on the client's existing systems and API access.

Agent controls

Six layers between the model and the site

Each control wraps the one inside it. None of them closes an observation — and one raised and left open is worse than one never raised.

L6 · Outermost — last line of defence

Inward → L1 · closest to the model

L6Rollback / safe modePull ranking back to review-only when detection or production signals degrade.Roll back
L5Version monitoringTrack model, rule-set, recognition-gate and capture-configuration changes.Track
L4TraceabilityRecord the frame, the rule cited, the routing, the owner and the closure.Record
L3Safety reviewHold observations for a named owner; it governs who acts, not whether the frame was read right.Gate
L2Policy guardrailsTest each capture against the site rule set, the recognition gate and the recipient list; a failure holds the frame.Restrict
L1Confidence thresholdsSend low-confidence candidates to a triage read before an owner is assigned.Require review
Model coreObservation raised — condition, zone, frame, progress delta and confidence
L1 – L2
Test whether a candidate may stand
L3
Puts the call in a safety manager's hands
L4 – L5
Preserve the frame, its confidence and its routing
L6
Stops automated offers and holds the staffed path

How Nestack evaluates it

Evaluate the capture-to-closure workflow — not only the detection.

Coverage runs the whole depth of the workflow, and every layer is cut by slice.

Surface — the observation the site acts on

  1. E1Final-output evaluationDid the observation match a condition the frame actually shows?
  2. E2Step-level evaluationDid the agent use the right baseline revision, zone and site rule set?
  3. E3Tool evaluationDid it register the frame to the correct level, zone and date?
  4. E4Confidence calibrationDo low-confidence captures actually carry more missed hazards?
  5. E5Slice evaluationHow does performance change across capture conditions?
  6. E6Business outcomeHow many hazards were missed, wrongly raised, or closed with no action?

Floor — the outcome the contractor answers for

Failure modes

Where each failure originates in the agent

Seven failure modes, each plotted where it starts rather than where it shows.

Agent lifecycle

Direction of processing →

  1. 01 · Retrieval

    1 mode

    • SI-01

      Superseded baseline

      Progress read against a baseline revision the project has replaced.

    Stage gathers

    Frames, baseline revision, prior captures and the site rule set

  2. 02 · Reasoning

    2 modes

    • SI-02

      Sheeting read as finishes

      Crash-deck sheeting is booked as an installed floor finish.

    • SI-03

      Rule applied out of place

      A hard-hat flag where the rule does not apply teaches dismissal.

    Stage proposes

    Ranked candidates, progress deltas and confidence

  3. 03 · Tool / write

    2 modes

    • SI-04

      Inferred quantity written

      An inferred quantity lands in the progress-claim schedule.

    • SI-05

      Recognition inherited

      A tenant setting turns face matching on with no notice or consent.

    Stage writes

    Only where write access and approval policy allow it

  4. 04 · Output

    1 mode

    • SI-06

      Off-site capture published

      A neighbour's interior reaches the full project directory.

    Stage returns

    The observation the site acts on and the imagery it shares

  5. 05 · Change / Version

    1 mode

    • SI-07

      Backdated re-scoring

      An upgrade re-scores archived imagery into new, backdated observations.

    Stage tracks

    Model, rule set, recognition gate and capture config

  • Sev-1 · a face or gait template is created
  • Sev-2 · an observation sits open past its clock
  • Sev-3 · capture degrades, frames route to triage
See the complete Construction failure-mode catalogue

Affected slices

A good baseline says nothing about the night shift

Start from the baseline: daylight structure work from surveyed positions misses least, and the aggregate inherits that. Nestack reports the missed-hazard rate by slice, alongside the false-positive rate that produces it.

Slice performance — reported separately, not only in aggregate

Illustrative example
SliceFailure rateLiftLift vs. thresholdStatus
Night, low light and heavy occlusion9.9%2.9×Review
Fitout and wet trades7.5%2.2×Review
Structure phase, daylight capture5.1%1.5×Watch
Repeat typologies, BIM baseline1.7%0.5×Normal

Bar: missed-hazard-rate lift vs. an all-frame baseline · scale 0–4.0× · tick marks the 2.0× review threshold

2 of 4 slices over threshold

Evidence-linked improvement

A closed cycle looks like a new test

An explanation closes nothing. The cycle ends with a case the next release has to clear.

Improvement cycle · five stages

Switchback — the path turns at Improve and returns at Learn

  1. 01

    Detect

    Missed-hazard rate rises in a capture slice.

  2. 02

    Diagnose

    The frames, the rule set and the routing log are read side by side until the cause is one of them.

  3. 03

    Improve

    The change is stamped to a version, with the frames that produced it attached.

  4. 04

    Verify

    Re-run the affected cases; a fail holds the release.

  5. 05

    Learn

    A standing test, plus a rule change in the detection set.

  6. Learn → Detect

    The return edge. Each turn hands the next one a longer suite to beat.

Typical build scope

Twelve workstreams across six weeks

The build scope read against the delivery timeline. Week structure follows the six-week plan — discovery, capture, detection workflow, evaluation, integration, then production validation and handover.

Workstream

Week 1Week 2Week 3Week 4Week 5Week 6

  1. 01Capture workflow discovery and boundary definition.Week 1
  2. 02Capture-source and site-system assessment.Week 1, Week 2
  3. 03Jurisdiction, notice and recognition-gate mapping.Week 1, Week 2
  4. 04Capture ingestion and de-identification.Week 2
  5. 05Detection logic and baseline-model binding.Week 2, Week 3
  6. 06Confidence scoring and observation routing.Week 3
  7. 07Safety review workflow.Week 3, Week 4
  8. 08Site-system and observation-log integration.Week 5
  9. 09Paired-frame regression cases.Week 4, Week 5
  10. 10Guardrails and publication controls.Week 4
  11. 11Observation-trail instrumentation.Week 5, Week 6
  12. 12Deployment, documentation and Agent Care handover.Week 6

12 workstreams · 6 weeks · bar shows the weeks a workstream is active — several run in parallel

Final scope and sequence confirmed in discovery

Engagement tiers

What each tier includes

Rows are the capabilities named in each tier's scope. Higher tiers include everything below them.

Capability✓ in scope · — not at this tierPilotOne site, one capture cadenceProductionProduction site systemsAdvancedMultiple sites / capture modes
Introduced at Pilot
Ranking to your site rule set✓✓✓
Safety review and escalation✓✓✓
Detection baseline✓✓✓
Introduced at Production
Reporting by capture zone—✓✓
Escalation clock in your systems—✓✓
Approved write-back—✓✓
Dashboard and BI integration—✓✓
Introduced at Advanced
Multi-jurisdiction capture rules——✓
Multi-stage site review——✓
High capture volume——✓
Enterprise capture controls——✓
Build priceFrom $5,000From $8,000Custom quote
Final build price
Confirmed after discovery based on integrations, workflow complexity, transaction volume, approval controls and deployment requirements.
Separate from build
Build pricing is separate from recurring Agent Care, which covers managed monitoring, evaluations, incidents and verified improvements after launch.

What we need from you

What you bring, and what we build with it

Each input maps to a piece of build scope and a week in the delivery timeline.

  1. 01Your site rule set and capture cadence

    Capture ingestion and zone mapping

    Week 1

  2. 02Representative captures from live levels

    Detection baseline, zone mapping and baseline-model binding

    Week 2

  3. 03Your surveillance notices and consent records

    Jurisdiction, notice and recognition-gate mapping

    Week 1

  4. 04Access to relevant APIs, feeds or exports

    Capture-source and site-system assessment, then integration setup

    Week 2

  5. 05Captures where the hazard was there and unflagged

    Detection cases, paired frames and the evaluation suite

    Week 4

  6. 06What an observation must reach before it is closed

    Confidence scoring, routing, guardrails and escalation controls

    Week 3

  7. 07Named owners with authority to abate

    Safety review workflow, then pilot and production validation

    Weeks 5–6

Nothing else is required

Deployment, documentation and Agent Care handover are ours.

Delivery timeline

Four phases across six weeks

Phases are drawn over the weeks they occupy. Week 5 carries evaluation and pilot capture together.

Phase

W1

W2

W3

W4

W5

W6

  1. DiscoveryW1
  2. BuildW2 – W3
  3. EvaluateW4 – W5
  4. Pilot & LaunchW5 – W6

Week focus

W1Capture workflow discovery, jurisdiction mapping and the boundary

W2Capture-source integration and the detection baseline

W3Detection workflow, confidence logic and routing controls

W4Evaluation suite, guardrails and failure-mode testing

W5Site-system integration, pilot capture runs and targeted corrections

W6A month of capture reviewed by the site team, then handover

Reading the band
Bars stop where the named work stops. The week 5 overlap is scheduled, not padding.
At the end of W6
Hand-over happens against a verified capture run, not a demo.
Duration
Six-week plan shown · typical delivery 4–6 weeks depending on scope confirmed in discovery.

Next step · Construction AI agent

Build a site-vision agent around who closes the observation.

Show us how you capture, who owns an observation and where the imagery goes. If your queue already outlives its clock, we fix the closure loop before we add a single detector.

Typical delivery
4–6 weeks
Deployment
Your capture and site systems
Safety review
Identity-free
After launch
Agent Care available

Nestack Agents · Site vision & progress capture

AGT-CN-07 · Agent Care available after launch

Build a site-vision agent

Four questions so the call starts from your actual setup — then pick a time that suits you.

About you
Your site

No obligation. We’ll map the workflow, name the automation boundary and tell you if an agent is the wrong answer.