Blocking an exfiltration means deciding, in the moment, whether the data leaving a device is sensitive. That turns a detection platform built for accuracy and scale into one that must also answer in milliseconds.
Nightfall pioneered the AI-first approach to DLP. Instead of leaning on regular expressions and pattern matching alone, we use language-aware machine learning models that understand the context sensitive data appears in. Context is what separates a real API key from a random string that happens to look like one, and it is the biggest single lever on detection accuracy.
The platform already processes several terabytes of data per day and returns results in real time. Text scanning latency has always been critical for SaaS apps like Slack, where customers expect detection and remediation in near real time. Nightfall's data exfiltration prevention products made fast file scanning just as important: stopping a malicious actor just in time means the verdict has to arrive before the data is gone. This post walks through the architecture that makes that possible.
How the system is organized
The work is split between the endpoint and the cloud, and the cloud is split again by the kind of work.
SaaS apps and endpoints, with the cloud for the hard calls. SaaS apps such as Slack send text for near real-time scanning. Endpoint devices run local heuristics to decide whether a piece of content might hold sensitive information. Only when something looks suspicious does the agent send the text, image or file to the Nightfall cloud for high-accuracy scanning. Routine endpoint activity never has to make the round trip.
The cloud is where the big models live. Hosting the platform in the cloud gives us the CPU, GPU and memory headroom needed to serve models at the lowest possible inference latency.
Three sub-systems, each with its own promise. The platform is compartmentalized into three sub-systems. They share much of the same underlying code, but each runs on its own dedicated compute so that a surge in one can never eat into another's budget:
Sub-systemWhat it scansService level objectiveText scanningText in motionResults in milliseconds (P95)Image and file scanningImages and files in motionLarge files in single-digit seconds (P95)Historical scanningImages and files at restHigh throughput
Physical separation is what lets us hold all three objectives at once. A backlog of historical scans should never be the reason a live exfiltration gets through.
The endpoint decides what is worth a closer look; the cloud gives text, files and historical scans their own lanes.
Two paths for getting the data out
Text and files have very different shapes, so they enter the platform through different doors.
Text: synchronous and direct. The text scanning system takes requests over synchronous Remote Procedure Calls (RPCs) and passes them straight to the downstream sensitive data detectors and ML models. There is no queue to wait in. The models are provisioned to serve peak traffic with milliseconds latency at the 95th percentile.
Images and files: asynchronous and queued. Files are bigger and far more varied, so this path is built on asynchronous processing over queues. A fleet of optimized file extractors and Optical Character Recognition (OCR) models ingests files across more than 50 formats. They break each file into smaller chunks that the downstream detectors can process.
The queues are what keep this path stable. A large burst of uploads lands in a queue instead of on the workers, so the system absorbs the spike without running out of resources.
Scaling out, and keeping the fast lane fast
The platform scales both vertically and horizontally. Each micro-service is built to use all the cores available on its instance. Metrics on CPU, GPU, memory and traffic requests trigger the cluster to scale out automatically when a surge arrives.
Queues bring their own challenge: when they build up, latency builds up with them. We manage this with priority queues. Latency-sensitive traffic goes onto fast lanes that get preferential treatment everywhere in the platform. Flow control with back pressure keeps the fast lanes from clogging and routes any excess traffic to the slow lane.
The Nightfall endpoint agent does its part in two ways:
- It scans ahead. The agent predicts when a scan will be needed and issues it early, so the result is already waiting at the moment of an exfiltration attempt.
- It takes the fast lane. It sends its file scans through the platform's high-priority queues, so they finish in a small number of seconds (P95).
A layered AI strategy
Large Language Models (LLMs) have raised the bar on detection accuracy. Frontier models can now detect many classes of sensitive data at a very high level of accuracy, work that used to need specialized, domain-specific models. Running them in production is hard, though: the compute cost is massive and inference is slow, potentially several seconds per request. Smaller, faster open-weight versions (often called SLMs) exist, but they have not met our accuracy standards.
Nightfall's Data Science team gets past these limits with custom models, trained on data that frontier LLMs synthesize through an LLM-as-a-judge process. LLMs can still run on the inference path, but only as the last step of a layered approach:
- Classic DLP. Keyword and pattern recognition make the first, fastest pass.
- Custom NER and BERT models. Named Entity Recognition (NER) and BERT models extract entities such as drugs, diseases, people, API keys, passwords and IDs.
- Frontier LLMs, where needed. The final layer can call on frontier LLMs to validate the findings from the earlier layers.
Each layer acts as a distillation step. It narrows the set of candidates and forwards only what remains to the next, more expensive layer, so the costly models are never overwhelmed by raw volume.
This works because most of the data we process contains no sensitive data at all. Filtering it out early is what lets the platform reach the low latencies needed to block exfiltrations just in time, while still reserving frontier-model accuracy for the cases that need it.
Serving custom models at speed
Fine-tuned custom models, whether BERT-family models or LLMs, are not typically available out of the box from the standard cloud vendors. They need specialized inference services, and we have tuned ours with a range of techniques:
- Caching of repetitive requests: when the same content is scanned again, the stored result is returned instead of running the model a second time.
- Batching of payloads: grouping several requests into one model call lets the GPU process them in parallel, raising throughput at little cost to latency.
- FP16 (half-precision) inference: storing weights and activations as 16-bit instead of 32-bit floating point roughly halves memory use and speeds up the math, with negligible loss of accuracy for most models.
- Maximizing GPU utilization: keeping each GPU busy, for example by running several model replicas on it or overlapping data transfer with compute, gets more inferences out of the same hardware.
- Rapid on-demand autoscaling when requests arrive in bursts: new model replicas spin up as traffic spikes and scale back down afterwards, so bursts don't build queues and idle capacity isn't wasted.
What comes next
Three lines of work are pushing latency lower and detection smarter.
Faster frontier models. Natural Language Processing research keeps producing niche variants of frontier models tuned for both speed and accuracy. We are actively experimenting with them to bring into our production systems.
Detection on the device. We are experimenting with lighter-weight models and other detection techniques that can run on endpoint devices themselves. Every scan that completes locally skips the network round trip to the cloud, which matters most when traffic is heavy.
Risk-guided Detection. Today the platform blocks based on what the content contains. Risk-guided Detection adds AI agents that observe the activity of both human users and agentic or MCP actors on a device, and judge the risk and intent behind an ongoing data movement. Those signals feed the detector, so a block can be triggered not only by sensitive content but by high risk or malicious intent.
Together these move DLP from matching content toward understanding behavior: a platform that can tell not just what is leaving, but why.
See real-time detection on your own data. Book a demo.
.svg)
.svg)
