Roboscope — AR Platform for Industrial Quality Control

Marking a single defect on a wind blade requires at least five measurements: finding it, capturing its dimensions, and measuring distances along a curved surface up to a hundred metres long. The core engineering challenge is recording all of that spatially — with no manual notation, no form re-entry, and sufficient accuracy to be actionable.

After

Before

AR Spatial Marking

1. World Anchor Placement — When the inspector frames a defect in the camera, a tap triggers an ARKit raycast against the scene mesh reconstructed from the LiDAR scanner. The hit point and surface normal are used to place a world anchor — a persistent coordinate frame attached to a real-world surface feature. All subsequent markers for a defect are placed relative to this anchor, keeping the measurement group coherent even if the phone is moved away and returned.

2. Spatial Marker System — Each defect record is represented as a set of RealityKit entities anchored in world space: an origin marker, boundary markers, and measurement path points. Markers render as camera-facing overlays with depth occlusion against the mesh, so they visually track the blade surface as the inspector moves. The marker set is serialised as a compact anchor graph — world transform per node, edge list for measurements — and stored locally in the session.

3. Measurement Capture — Linear dimensions are computed from the 3D distance between anchor positions in world space. For curved surfaces, the inspector places intermediate path points along the contour; the app accumulates arc length across segments, producing a measurement that follows the blade's curve rather than cutting through it. All measurements are stored with their constituent anchor transforms so they can be recomputed or audited later.

4. Coordinate Accuracy — ARKit's world tracking maintains ~5 cm spatial accuracy under typical factory conditions — sufficient for defect registration on blades where repair tolerances are measured in centimetres. Accuracy is validated per session: the app prompts the inspector to re-scan a reference point at session start and end, and flags sessions where the closing error exceeds the threshold. A 1% cumulative drift in the AR coordinate frame was the primary accuracy risk at scale — nearly a metre of error at the tip of the longest blades — and has since been resolved through ML-detected surface reference marks.

Inspection Workflow

1. Session Initialisation — The inspector opens a blade session by scanning its serial number or selecting it from the active production queue. The app loads the blade model and any existing defect records for the current revision, overlaying previously registered markers from prior inspections so new findings can be placed in context of the blade's known history.

2. Defect Capture — For each defect, the inspector places the origin anchor, marks the boundary, and records dimensions using the measurement path tool. A photo is captured per defect — the AR overlay is composited into the image at capture time, embedding the spatial context directly into the photo record. Defect type, severity, and repair category are selected from a structured taxonomy configured per client.

3. Edge Case Review — When a technician cannot determine the correct repair, they raise an in-app review request tagged to the defect record. The engineer opens the session on the web portal, inspects the AR composite photo and measurements, and records a decision. The technician is notified in the app without leaving the inspection flow — replacing the previous pattern of composing emails, attaching photos, and manually tracking replies across tools.

4. Session Sync — Completed sessions are stored locally in a SQLite-backed queue and synced to the backend API when Wi-Fi is available. The sync is incremental — only new and updated records are transmitted. Sessions remain fully usable offline; blades on the far end of large factory halls where Wi-Fi is unreliable are not a special case.

5. Report Generation — The backend generates a structured inspection report per revision: defect inventory, measurements, repair decisions, and timestamps. Reports are exportable as PDF and JSON for direct import into QC management and ERP systems. The web portal renders the full session as a navigable defect map linked to the blade model.

Coordinate Drift

ARKit accumulates small orientation errors over time that compound across the length of a blade. A 1% drift is negligible at short range but produces errors approaching one metre at a 90 m blade tip — large enough to mis-locate a defect relative to structural features. The solution uses laser marks already present on the blade surface for manufacturing alignment as optical reference anchors. The ML vision layer detects these marks in the camera feed and resets the AR coordinate frame to the mark's known position mid-session, bounding accumulated drift regardless of blade length. Mapping the laser-mark grid per blade type was the primary integration cost; blade reference maps are now part of client onboarding.

System Architecture

The system is built around three components across device, backend, and browser:

• iOS App — ARKit LiDAR mesh reconstruction for surface-accurate anchor placement; RealityKit for marker rendering with depth occlusion; one-handed interaction model designed around helmet and safety glasses constraints. Local SQLite store with offline-first sync queue. SwiftUI interface with per-client taxonomy configuration loaded from the backend at session start.
• Backend API — Node.js / PostgreSQL service storing inspection sessions indexed by blade serial, revision, and position. Receives incremental session syncs from the app; generates structured reports; handles async review request routing and engineer notification. REST and webhook endpoints for QC system and ERP integration.
• Web Portal — React interface for engineers and QC managers. Displays the full inspection session with per-defect detail, AR composite photos, and measurements. Supports async review decisions, audit history, and report export. Read-only for QC managers; engineers have write access for review decisions only.

ML Vision

ML vision runs on-device and serves two roles. The first is coordinate correction: the model continuously scans the camera feed for laser reference marks printed on blade surfaces during manufacturing, detects their positions, and resets the AR coordinate frame to the mark's known location — eliminating accumulated drift regardless of blade length. This is in production. The second role is defect pre-classification: models for defect type, structural damage, paintwork defects, and blade circularity are in active training. They are intended to surface automated pre-classification results to the technician before manual review. More annotated data is needed before any classification model reaches production threshold.

Project state

In active testing with a new client following open-market outreach:

• Core AR marking and measurement
• Offline-first session sync
• Web portal and report generation
• New client onboarded, floor tests running
• Laser-mark anchor detection (drift fix)
• ML vision models reaching production accuracy
• Full ERP integration