Patrick Sherlund
Lead Software Engineer
The Team
Shelby Reilly
Lead Product Designer
AI-powered drone search & rescue platform designed for speed, safety, and hope.
6 months to MVP
Lead Software Engineer
Patrick Sherlund
|
Lead Product Designer
Shelby Reilly :)
My role
Engineered Bishop's search and rescue video intelligence pipeline to automate frame-by-frame review into prioritized human detections, cutting time-to-first-lead from hours to seconds.
The Stack
FRONTEND
React, vite (Typescript, Javascript)
BACKEND
Drogon (C++)
ML
ONNX Runtime (C++, Python)
DATA
PostgreSQL
WORKFLOW
GitHub (Agile)
INFRA
AWS, Docker
The Challenge
Locating distressed humans quickly & efficiently during search & rescue missions.
The Vision
AI powered drone search & rescue application designed for speed, safety, and hope.
Our users saw...
30% ↓
Time-to-first-find reduction
3,400×
Faster video review
01 | Discover
Understand the context, painpoints, and user insights
Why this started
We started this after the 2024 Southeast Coastal hurricanes, when SAR teams struggled to find survivors quickly across flooded zones.
Discovery interviews & workshops
Persona Mapping
What We Learned
The Reality Today
In SAR, the first ~72 hours are the most critical, but manual video review is fatiguing and error prone, and even reviewed footage can hide missed detections. Bishop runs a rapid second pass to surface missed human leads faster.
What's Holding Teams Back
In SAR operations, responders have ~72 hours to find subjects, but manual video review creates cognitive fatigue that increases missed detections and delays time-to-discovery.
What Success Looks Like
SPEED
Hours to Minutes
quicker detection
ACCURACY
90%+
detection rate
TRAINING
<15 min
to get started
HARDWARE
Real-time
on standard CPU
Our proposed process
Create Project
Upload Video
Review Results
Share with Team
What we proved
Validate aerial person-detection accuracy
↓
We saw 94% Recall on footage
Benchmark CPU-only video processing throughput
↓
We processed 3,700 frames per minute
Prove smooth playback of annotated timelines
↓
Roughly 60 FPS during video playback
Concepts, ideations, iterations
Design Principles
01
Performance
Process large RF captures fast enough to support field use.
02
Seamless Integration
Fit into existing tactical workflows instead of asking users to adopt new ones.
03
User Experience
Make detectability understandable in seconds, with minimal training.
Iteration 1
Week 2 - Gather, Annotate & Train
Gather datasets from published research
Annotate segmentations within the images
Train & fine tune the model
Iteration 2
Week 4 - Detect, Track & Visualize
The design needed to prioritize scalability, efficiency, and clear separation of concerns, so that video ingest, inference, and visualization could scale independently as usage increased.
Inference video frames with the trained model
Track multiple detections over time
V1
V4 🪄
Detect, Aggregate, Coalesce -> Track
Visualize human tracks on the client video
Iteration 3
Week 6 • Create, View, Manage & Associate
Create projects and associate them with videos
View AI detections in the video, synced to the timeline
Manage client video library
Associate detections with the SAR mission map
The Minimal, but Viable Product
Key Learnings
Speeds beats Perfection
Operators valued fast, "good enough" detections over slower, marginally more accurate ones
Transparency builds trust
Showing ML confidence scores helped operators calibrate trust and make better decisions
Resilience in the field
Offline capability and a simple UI is critical for use in remote environments
Thank you
To the SAR operators who trusted us with their feedback, field-tested early prototypes in difficult conditions, and never stopped pushing us to build something that saves lives!!
To my Co-Founder and Lead Designer Shelby Reilly who slayed until the very end! :)
And to the teams who keep moving when the clock doesn't stop.
