Shipped a biometric fraud gate in a month, using face-api.js to bridge the missing mobile SDK.

TL;DR

A wave of reimbursement fraud was hitting the group's consumer insurance apps through credential sharing. Product needed a customer identity check, but the biometric vendor had no Ionic/Cordova SDK yet. Following the Tech Lead's idea, I bridged it with face-api.js: on-device liveness using a neutral-to-smile transition, status centralized in NgRx, and a recovery modal for users with blocked claims. Shipped inside the shared health library so both apps inherited it on their next release. Came back three years later to help migrate onto the vendor's native SDK with an event-driven backend.

Context

The group ran two consumer apps. One was a super-app covering every insurance product the customer had. The other was a focused health app for its largest policy base. The health flow (network maps, reimbursement requests, reimbursement status) lived in a shared npm library installed into both apps. When product surfaced the need for identity validation, the change had to land in that shared code so both apps would inherit it at once.

The challenge

Reimbursement fraud was the pressure. Clinics and third parties were getting policyholders' credentials and filing fake claims in their name. The vendor chosen to validate identity had a mature SDK for native iOS and Android, but not for Ionic/Cordova, which is what the apps were built on. We needed a liveness check shipped fast, before the vendor's mobile SDK was ready.

What we did

1. 01

   Bridged the vendor gap with face-api.js

   The Tech Lead suggested using face-api.js, a frontend ML library that could detect a neutral-to-smile transition. That gave us two things at once: a basic liveness proof (a static photo wouldn't transition) and a capture layer that didn't depend on the vendor's SDK shipping. I loaded the lightest model the library offered to keep the bundle hit small, though it was still a real cost.

2. 02

   Built the capture screen with cordova-camera-preview

   The capture screen used cordova-camera-preview to render the camera feed with a face-centering placeholder, so users framed themselves without switching to the native camera app. When face-api.js reported a detected face, I captured a still. When it also reported a smile, I captured a second. Those two images became the liveness payload. The backend submitted them to the vendor for comparison against the customer's ID.

3. 03

   Centralized biometric status in NgRx

   On app open, I queried the backend for any analysis tied to the user, in progress or already closed. The result hydrated an NgRx slice. If the customer had a blocked claim waiting because of a failed check, opening the app surfaced a modal with a single CTA to retry the validation. This kept the fraud gate from becoming a silent dead end.

4. 04

   Shipped inside the shared health library

   Because the capture, status, and modal lived in the shared npm library consumed by both consumer apps, they picked up the change on their next release. Combined downloads on Google Play alone were in the multi-millions, so reuse mattered.

Results

Looking back

• The backend existed, but its responses mirrored the vendor's payload shape closely, and the mobile's NgRx slice was modeled on that shape. Every leaky field in the vendor's API ended up in mobile state. Today I'd design a stable contract at the backend that doesn't leak vendor specifics: a small status enum (needs-check, in-progress, valid, blocked) with an optional reason string. Then the 2024–2025 migration to the vendor's native SDK is a backend change only, and the mobile doesn't know which vendor powers the gate.
• The capture layer lived entirely in TypeScript on top of cordova-camera-preview and face-api.js. That bought speed at the cost of bundle weight, runtime performance, and a ceiling on what on-device ML could do inside a webview. Today I'd explore a custom Cordova or Capacitor plugin doing capture and face detection natively, exposing a clean JS surface to the app. I didn't have the Swift/Kotlin chops at the time, only light Java. That's a gap I'd close or team up through now.