When Equipment Fails and the Expert Is 500 Miles Away
The client builds AR remote assistance tools for industrial operations. Manufacturing plants, pipeline operators, oil and gas facilities, EV charging networks. The common thread is that when something breaks, the cost of waiting is brutal. Production stops. Revenue bleeds. Safety risks compound.
The old playbook was to call someone on the phone, try to describe the problem verbally, and hope the person on the other end could diagnose it without seeing it. Or wait for a specialist to fly in. Neither option was tenable for the kinds of customers the client was chasing.
They needed a platform that would let a field technician connect with a remote expert instantly, show them exactly what they were looking at through a live video feed, and receive visual instructions overlaid directly on the equipment. Hands free. Voice controlled. Reliable on a congested plant floor network or a 4G connection at a remote site.
Sequoia Applied Technologies built it.
What the Platform Delivered
The numbers below came out of production use, not a demo environment.
The less quantifiable part: session recordings became a training corpus. New technicians could watch how experienced experts diagnosed real problems on real equipment. That cut ramp time in ways that are hard to put a number on but showed up in how quickly new hires became useful.
Building the Whole Thing
Sequoia Applied Technologies is a Santa Clara software engineering firm. We build products for technology companies in manufacturing, life sciences, healthcare, and enterprise software. This was a greenfield build. The client had a clear vision for what they wanted. We did the engineering.
The technician app runs on Android. It was first deployed on RealWear Navigator 615, a ruggedized head mounted display built for industrial environments. Voice commands handle everything so the technician's hands stay on the work. Live video streams to the expert with low latency encoding tuned for bandwidth that is often neither fast nor stable. The architecture is modular, which mattered when the client later wanted to support Six15 HUD systems for tactical and critical infrastructure use cases. That port was not a rewrite.
The expert console is a web application. A remote expert sees the technician's live feed and can draw annotations directly onto it: arrows pointing at the faulty component, numbered steps walking through a procedure, caution markers where something could go wrong. Those annotations appear in the technician's field of view in real time. The expert can also push text, diagrams, or reference documents.
The Java backend does the unglamorous work of keeping sessions stable when the network is flaky. It adjusts video quality on the fly based on available bandwidth. It handles session logging with timestamps, participant details, and full video capture so everything is auditable for compliance. Scheduled jobs generate reports and trigger alerts when sessions fail or degrade.
Later work added an AI assistant. Transcription runs on completed sessions so they become searchable. A conversational interface lets support staff query tickets, pull call summaries, and look up session history using plain language instead of navigating through menus.
What the Stack Looks Like
The platform divides into three layers: the wearable app, the backend services, and the expert console. Each layer had distinct constraints. The wearable app had to work on devices with limited compute and unreliable connectivity. The backend had to absorb that variability and still deliver stable sessions. The expert console had to be fast enough that drawing an annotation felt instantaneous, not laggy.
Runs on AR smart glasses including RealWear Navigator 615 and Six15 HUD systems. Voice commands for hands free operation. Live HD video streaming with adaptive bitrate encoding. Modular codebase designed for portability across different wearable hardware.
Tuned for low bandwidth and variable connectivity common in industrial environments. Handles session initiation, video relay, annotation sync, and compliance logging. Adjusts video quality dynamically. Scheduled jobs for reporting, alerting, and session archival.
Browser based application for remote experts. Live video feed from technician's glasses. Real time AR annotation tools: arrows, labels, numbered steps, caution markers. Document and diagram sharing. Multiple experts can join a single session.
Session transcription for searchability. Conversational assistant for ticket search, call summaries, and status history using plain language queries. Built as an addition after the core platform was in production.
Common Questions About Mixed Reality Field Service Software
What did Sequoia build for this platform?
Everything. The Android technician app running on AR smart glasses, the Java backend, the expert web console, session logging and compliance features, and the AI transcription and conversational assistant added later. The technician app launched on RealWear Navigator 615 and was later extended to Six15 HUD systems.
What results did the platform achieve?
Service resolution times dropped by more than 50%. The platform went from pilot to production in 6 months. Field technicians could connect with remote experts in seconds instead of waiting for someone to travel. Session recordings also became training material, which shortened ramp time for new hires.
Which AR smart glasses does the platform support?
The app runs on Android and was architected for portability across wearable hardware. Initial deployment was RealWear Navigator 615. Sequoia later added support for Six15 HUD systems, which serve tactical and critical infrastructure contexts. The modular design meant extending to new hardware was not a ground up rebuild.
How does AR remote assistance actually work?
A technician wearing smart glasses taps to start a call. A remote expert connects in seconds and sees the technician's live camera feed. The expert draws annotations directly onto that feed: arrows, numbered steps, caution markers. Those annotations appear in the technician's view overlaid on the physical equipment. The technician follows the guidance hands free while working. Every session is logged with video, timestamps, and participant details.
Why build the backend for low bandwidth?
Industrial sites often have poor connectivity. A plant floor might have congested Wi-Fi. A pipeline location might have intermittent 4G. The Java backend adjusts video quality dynamically so sessions stay stable even when the network degrades. This was a core design constraint, not a retrofit.
What kind of companies does Sequoia Applied Technologies work with?
Sequoia is a Santa Clara software engineering firm. We work with product companies in manufacturing, life sciences, healthcare, cleantech, and enterprise software. Engagements include new product builds, mobile and wearable apps, cloud platforms, IoT systems, and backend engineering for products that need to work reliably in demanding conditions.