Face Recognition Payments at the Point of Sale: Architecture, Security, and UX

The Premise

Facenex was a payment system that authenticated users with a face scan and PIN at a merchant terminal. The flow did not require the user to carry a card, wallet or phone.

The face match and PIN served different roles: the system compared a live face with an enrolled template, then required the user to authorise with a PIN. The design needed to resist spoofing while keeping authentication quick enough for a checkout. This Deep Dive explains the pipeline and its trade-offs.

The Authentication Pipeline

The Facenex design used a fixed authentication pipeline. Each stage had to fit within the target of under 3 seconds for the complete flow.

1. Face Detection

The PoS camera captures a live frame. A Haar cascade / SSD detector locates and crops the face region. Detection runs on the device, avoiding a network request at this stage.

def detect_face(frame):
    detections = cascade.detectMultiScale(frame, scaleFactor=1.1, minNeighbors=5)
    if len(detections) == 0:
        raise NoFaceDetected()
    x, y, w, h = largest(detections)
    return align(crop(frame, x, y, w, h))

Alignment matters. A face tilted 15 degrees produces a meaningfully different embedding than the same face straight on. The detector normalises rotation and crops to a canonical bounding box before passing the image downstream.

2. Embedding Extraction

The aligned face passes through a deep neural network that outputs a fixed-length vector — typically 128 or 512 dimensions. This vector is the biometric template. Two photos of the same person produce vectors that are close together; two photos of different people produce vectors that are far apart.

The stored template is an encrypted embedding; raw images are not persisted. This reduces the amount of image data retained, but the template is still sensitive biometric data and needs protection.

3. Vector Matching

At transaction time, the system extracts a live embedding and computes cosine similarity against the enrolled template. If the similarity score exceeds a tuned threshold, the match is accepted.

The threshold is the central tuning knob in any biometric system. Set it too low and impostors get through (false accept). Set it too high and legitimate users get rejected (false reject). The threshold must be assessed against both false accepts and false rejects under the conditions at the terminal, and any accuracy figure needs measured evidence from those conditions before it can be claimed.

Liveness Detection: Stopping the Photo Attack

A face match is no good if a photograph can pass it, so the liveness stage checks for evidence that the camera is seeing a live person rather than a static image.

Facenex checks three independent liveness signals:

1. Blink rate — the camera captures a short burst of frames. A live face blinks 15-20 times per minute. A photograph blinks zero times.
2. Head micro-movements — a live person makes small, involuntary movements. A held photo is rigid.
3. Depth cues — where infrared capability is available, the system checks for 3D depth. A photo is flat; a face is not.

Lighting and camera quality affect how well these checks work, and depth sensing depends on the terminal hardware. Combining the signals gives the system more evidence for rejecting a static image; effectiveness still needs testing against spoofing attempts.

PIN Hardening

The PIN is checked separately from the face match.

Even if an attacker somehow defeats the biometric layer (stolen photo + spoofed liveness), they still need the PIN. Facenex hardens the PIN layer:

• Hashed, never stored in cleartext. PINs are hashed with bcrypt and a per-user salt. The database stores the hash, not the PIN.
• Rate limiting. After 3 failed attempts, the account locks and requires re-enrolment. The lock limits repeated online guesses.
• Transport encryption. The PIN travels over an encrypted WebSocket from the terminal to the backend. It never touches the merchant's local network in cleartext.

Requiring both checks means defeating the face match alone is not enough to pass authentication. The PIN itself, and the recovery flow around it, still need protection.

PoS Terminal Integration

Facenex does not replace the payment gateway. It replaces the card-present step. The merchant's existing payment processor handles the money movement; Facenex handles authentication.

The terminal software runs on Android with a React Native frontend. The flow:

1. Merchant enters the amount on the terminal.
2. User looks at the camera and enters their PIN.
3. The terminal sends an authentication request over encrypted WebSocket.
4. The backend verifies the face embedding and PIN.
5. On success, the backend returns an authorisation token.
6. The terminal forwards the token to the payment gateway, which processes the charge.

The design targeted under 3 seconds for detection through gateway handoff. That target gave each stage a time budget and kept network requests to a minimum.

Why Not Store Images?

Every design decision around biometric data comes back to one question: what happens when the database is breached?

Storing raw face images would mean a breach exposes photographs of every enrolled user. Unlike a card number, a person's facial characteristics cannot simply be reissued after a breach.

Storing embeddings avoids retaining the original photographs. It does not make the retained data anonymous or remove the risk of a breach. An embedding represents a person's biometric characteristics, so it still needs restricted access and encryption. It should not be treated as equivalent to a password hash.

Accuracy, Security and Checkout Speed

The design balanced the following constraints:

Every choice pulls against the others. Tighter security adds latency. Lower latency risks accuracy. A stricter threshold can reject legitimate users. Facenex used on-device detection to cut latency, multi-signal liveness for security, a tuned threshold for accuracy, and separate PIN verification.

What We Learned

The camera and terminal hardware set limits on what the authentication pipeline could do. Liveness checks and PIN verification then had to fit within the same checkout time budget. The parts had to be designed together: a fast face match alone would not establish that the payment system was usable or secure.

→ See the Facenex case study — face + PIN biometric payments at the point of sale, with a design target of authentication in under 3 seconds.