Physical AI for Seniors: What Does SafelyYou Do?

Published

I spent two years at Blueprint Technologies helping build the retail version of a video analytics product – the system that later became NASH, now on the Azure Marketplace serving a government customer. Our version answered questions for store managers: how deep is the queue, how long has this customer waited, has anyone noticed them.

My piece of it included a head-pose and gaze module (code). The design constraints were brutal: surveillance footage is low-resolution, faces are sideways more often than not, and landmark-based methods fall apart exactly there. So we went keypoint-free – a ResNet50 backbone with three branches, each combining binned classification with regression (cross-entropy plus MSE), good for about 6.5 degrees of average error on AFLW2000, fed by a single-stage face detector we had modified and retrained ourselves. Every choice in that stack was made for deployment, not for a benchmark.

GazeNet architecture: a ResNet50 backbone feeding three softmax branches for yaw, roll, and pitch

That work taught me the lesson I keep coming back to: the hard part of video analytics is not detection. It is turning continuous, ambiguous footage into discrete events that a busy human can act on, at a false-alarm rate that human can tolerate.

My interest in this problem is not only professional. Around the same years, I was helping care for my mother through a long illness, and working with home caregivers in my day job – building models to predict caregiver turnover and preventable ER visits, and spending many hours listening to what caregivers actually wished existed. So when I started reading SafelyYou’s public materials – their papers, their case studies, their job postings – I read them twice: once as a builder, once as someone who has been on the family side of that bedroom door. Here is what I see.

The architecture, read like a builder

At first glance, each design choice in SafelyYou’s Guardian system looks overly cautious. Look closer, and every one of them turns out to be essential.

A wide-angle camera sits in the corner of a resident’s bedroom. Video is analyzed on-site, on the device; footage of ordinary life is deleted within minutes and never leaves the building. Only detected events – a person on the ground – become clips, which travel to the cloud, where a remote clinical team reviews each one before staff are alerted. Their published detection rule is beautifully blunt: a fall is when any body part other than the feet touches the floor, whatever the position, whoever it is.

Each choice answers a real constraint. On-device inference answers privacy – families consent to a camera in a bedroom only because the architecture itself, not a policy document, guarantees that ordinary life is not uploaded. Event-only upload answers bandwidth and cost. Human review of every event answers the hardest constraint of all: a fall detector that cries wolf in a memory care facility at 4 AM is worse than no detector, because alarm fatigue is what actually kills adoption.

We faced a smaller version of the same trade-off in retail. NASH’s later government incarnation learned “common behavior” from time-series data and flagged deviations – a baseline-and-anomaly philosophy. SafelyYou runs the opposite philosophy: not learned baselines, but a fixed, inspectable state rule – person down. For a first-generation product in a regulated, high-liability setting, I believe that was the right call. A state rule you can explain to a regulator, a family, and a night-shift nurse beats a subtle model you cannot.

The data flywheel nobody talks about

The most important artifact in SafelyYou’s public record is not a model. It is a dataset: over 350,000 real fall events, each reviewed by clinicians, built up from roughly 60 million minutes of video. Their case study with Voxel51 describes the machinery around it – mining false positives and negatives, curating training data, lifting mAP by ten points while cutting manual verification volume by 77%.

This is the part of the architecture I admire most, because it is the part I know is hardest to build: the human review loop is simultaneously the product’s safety guarantee, its labeling engine, and its biggest cost center. Their own job postings now describe reducing review burden as a core economic goal. Every point of false-positive reduction flows straight to margin – which is exactly why the next technical move matters.

The papers, read like a modeler

SafelyYou’s clinical evidence rests on two JMIR papers and an AJMC report. The flagship result, from Bayen et al. 2021: across six memory care facilities, 66 consenting residents, and 436 falls over ten months, real-time notification cut mean time until staff assistance from 35.3 to 7.0 minutes, and mean time on the ground from 43.2 to 13.6 minutes. Falls with more than an hour on the ground – the cases that kill – went from 31% to zero. That last number is the real result, and it deserves full weight.

But a modeler notices three things.

The mean hides the median. Baseline median response time was 7.6 minutes, not 35. The headline improvement is mostly the elimination of a long right tail. Both claims are true – typical response improved by about five minutes, and catastrophic waits disappeared – but they are different claims, and the paper leads with the bigger one.

The events are not independent. 436 falls came from 66 people – one resident fell 49 times. The period comparison treats events as independent samples; a frailty model or cluster-robust analysis is the right tool, and a handful of recurrent fallers could be carrying the result.

Most of the improvement arrived before the AI did. During the middle period – weekly video review, no real-time alerts yet – response time already fell from 35.3 to 15.9 minutes. More than half the total gain was staff behavior change. I do not read this as a weakness. I read it as the truth about what these systems are: behavior-change programs with an AI component. Buyers who understand this deploy them better.

I have watched the same mechanism work from the other side of the care industry. In a peer mentorship program for home caregivers that my models helped target, turnover dropped by 40% in six months – and the credit belonged to the mentorship itself, experienced caregivers giving new ones the support that kept them from leaving. The model’s job was to tell us who needed support and when; the retention came from humans. That is the same division of labor SafelyYou’s middle period revealed: technology directs attention, people do the saving.

And there is one more number hiding in the results: 34.2% of video-reviewed events were “behavioral falls” – residents who intentionally lowered themselves to the floor. One third of everything the system flags is not a fall. That is what a static frame-level rule buys you, and it is the single clearest technical opening in the whole stack.

Where this architecture goes next

A fall is a process – a velocity profile, a loss of balance, an uncontrolled descent – not a frame. An intentional descent and a fall have completely different motion signatures, and skeleton-based temporal pipelines now run in real time on modest hardware. Moving from frame classification to temporal models attacks the 34% directly, and with it the review queue, and with it the unit economics.

The deeper frontier is the one the original UC Berkeley project named back in 2015 and nobody has solved: prediction. These residents fall about four times a year, mostly overnight, peaking between 4 and 8 AM, mostly during transfers. The precursor signals – gait drift, sit-to-stand instability, nighttime wandering – are likely sitting in that 350,000-event archive. But prediction here is a rare-event problem with brutal class imbalance and an intervention window measured in minutes. Anyone who promises it easily is selling something. The honest path starts with calibration, not accuracy – the same discipline I learned building ER-visit prediction models for home-care workers, where a false alarm also has a human cost.

Why I care enough to write this

If I walked into this problem tomorrow, my order of operations would be: per-resident recurrent-event models that respect the clustering the published analysis ignores; temporal severity scoring to shrink the review queue, which is where the economics live; and only then a prediction program, run as an explicit research bet with pre-registered continue-or-stop criteria.

But the reason I read these papers so closely is simpler. I have been the person building the models, and I have been the child on the other side of that bedroom door. Systems like this sit exactly at the intersection of those two lives – and they deserve to be read, and built, by people who have stood in both places.

Sources