By the time a navigation begins, the mail gateway has passed the message, the messaging app has rendered the preview and the user has decided to trust it. Everything downstream of that decision happens inside your engine. This page is about building navigation-time blocking properly: a bundled local list, an interstitial that tells the truth, and a coverage claim you can defend.
That position is a privilege and a constraint. You see every navigation regardless of how it started — and you have milliseconds and no context in which to decide.
A phishing link can arrive through a route no security product in the chain will ever inspect. It can be typed from a phone call, scanned from a printed code, opened from a messaging app that renders links in its own view, pasted from a document, or followed from a search result that looked entirely ordinary. The organisational controls that people describe as "the phishing defence" — mail filtering, proxy policy, DNS enforcement on the corporate network — are each covering one path, and none of them covers a personal device on a mobile connection following a link out of a chat.
Intent has already been formed, the URL is resolved, and a request is about to leave. That is the browser's moment, and the only moment in the sequence guaranteed to exist — which is also why it is a moment with almost no information available: no message body, no sender reputation, no thread history, nothing but a hostname and whatever local state you hold.
A hostname is available before a single byte has been fetched, it is cheap to look up, and it evaluates deterministically inside the time budget a navigation actually has. Anything cleverer — analysing page content, comparing rendered layout, scoring form fields — needs the fetch to have happened, which means the request has already left the device and the harvesting page already knows the visitor exists.
The mechanism has to satisfy all of them simultaneously: it cannot leak the user's browsing to a third party, it cannot cost more than a few milliseconds, and it cannot occupy more memory than a browser process is willing to give a security feature. Those constraints — not detection quality — are what determine the architecture, and they all point the same way.
Why the list belongs on the device rather than behind a network call, how to size the structure it lives in, what belongs on the interstitial, how the bypass should behave, how extensions and managed deployments differ, and exactly what to say about the domains the list does not yet contain.
What it is: evaluating the destination hostname of a navigation, after the URL is resolved and before the network request is issued, against a list of hosts confirmed as credential-harvesting infrastructure — and replacing the navigation with an interstitial when a match is found.
Where the list lives dictates your privacy story, your latency budget, your offline behaviour and your failure mode. Decide it first.
A remote lookup on every navigation sends a record of the user's browsing to a third party, adds a network round trip to the critical path, and fails open the moment connectivity does. All three are unacceptable in a browser.
/feed on your own cadence, compile it into whatever structure your engine prefers, and match locally. Downloads are unlimited on a subscription, so update frequency stays an engineering decision rather than a billing one.Roughly 390,000 hostnames is a small dataset by browser standards, but not one you want as a naive string set in every renderer. A hash set of truncated digests, or a probabilistic filter backed by an exact check on a hit, keeps the resident footprint in the low hundreds of kilobytes.
Lowercase, strip the trailing dot, convert internationalised labels to their encoded form, and decide explicitly how you treat subdomains. Matching only the exact host misses the most common pattern in this data — a brand name as a label under an unrelated registered domain.
The database rebuilds daily at 04:30 UTC and ships a changelog of additions and removals, which is what makes incremental updates practical. A component that pulls a diff is far kinder to a metered mobile connection than one replacing a full file.
Say what is actually known: this hostname has been verified as an active credential-harvesting site, and the page was not loaded. Avoid the generic "attackers might be trying to steal your information" register — users have been trained to click through it precisely because it is so often wrong.
A proceed-anyway option is necessary — false positives exist and a browser that traps a user is worse than one that warns them — but it should require an extra interaction, not be the visually dominant button, and should apply to that host for that session only.
This is the whole flow. Its brevity is the point — everything expensive happens at update time, not at navigation time.
A click, a typed address, a redirect or a link handed over from another application. Intent is already formed.
InputParse the URL, take the host, normalise it exactly as the compiled list was normalised.
MicrosecondsA probabilistic membership test against the in-memory structure. Almost every navigation ends here, cleanly.
In memoryA hit is confirmed against the exact set before anything is shown, because a filter hit alone is not evidence.
No networkThe request is never issued. The user sees what was blocked and why, with a scoped way to proceed.
Request suppressedNotice what is absent. No outbound call, so no record anywhere of what the user was about to visit, no latency dependent on a network you do not control, and no behaviour change when the device is offline or behind a captive portal. That last case matters more than it sounds — a user joining unfamiliar public Wi-Fi is in one of the higher-risk situations a browser encounters, and it is exactly the moment a remote-lookup design is least able to answer.
The API still has a role — just not on the navigation path. Sub-50ms responses and a rate limit of ten requests per second per key suit build-time and tooling work: validating a candidate list during development, checking user-reported hostnames against /batch at a hundred domains per call, or confirming database size and last update through /stats, which needs no API key, no credits and no authentication. Keep the key server-side; it is the username chosen at registration and has no business inside a shipped client.
An extension, a managed enterprise browser and a mobile browser each impose their own constraints on how the list gets in and stays current.
An extension has the tightest constraints and the least room to be clever. Store review will scrutinise both the permissions requested and any network traffic the extension generates, and an extension that phones home on every navigation is the archetype of what reviewers reject — reasonably, since that is indistinguishable from browsing surveillance. Bundling the list, or fetching it from a single well-documented endpoint on a slow schedule, is far easier to justify in a review and far easier to explain in a privacy disclosure. It also lets you request narrower permissions, because you need to observe navigations rather than read page content.
The interstitial is a claim, and users audit it. A warning that says "this site may be dangerous" when the site is fine teaches people to click through, and the lesson generalises to every warning you will ever show them. A warning that says "this hostname was verified as an active credential-harvesting page and the page was not loaded" is a narrower claim, harder to be wrong about, and therefore worth more the tenth time a user sees it than the vague version is the first time.
A browser vendor has to be able to describe its safety feature accurately, because somebody eventually will on your behalf.
Start with the inclusion rule, because it determines everything else. A domain is in the database only if it has been observed as phishing infrastructure and confirmed to have an active A record, verified through rotating proxy infrastructure. Hosts that stop resolving fall out rather than accumulating. That is why the active population sits around 390,000 rather than growing indefinitely, and it is why a match is unusually strong evidence: you are not looking at something reported once in 2023, you are looking at something that answered a DNS query within the last rebuild cycle.
A domain registered this morning and first used this afternoon will not be in this morning's build. There is no way around that for any list-based mechanism, and a browser implying otherwise is making a claim it cannot support. Infrastructure stood up and burned inside a single day is the residual risk of this entire technique, and it belongs in your security documentation rather than being glossed over.
This is a domain-level dataset, so a hostile page on a compromised legitimate site — a path under a domain that is otherwise entirely benign — cannot be expressed by a hostname list without blocking the legitimate site too. That trade-off is not worth making, so the honest answer is that this technique does not cover that case at all. Page-level detection is a different mechanism with different costs.
The category is phishing/malware and the confidence is 0.98 or 0.0 with nothing between — no reputation score, no risk band, no partial verdict, which is a simplification in your favour since there is no threshold to tune. But it also means a clean result carries no positive information at all, so any element rendering "not on the list" as a green tick is materially misleading.
The overwhelming majority of what actually reaches ordinary users is not bespoke same-day infrastructure. It is industrialised, reused, and often several days old by the time a link has been forwarded, screenshotted, re-shared and finally clicked. A daily-verified list of live hosts intercepts a great deal of that at the cheapest point in the whole stack — no network call, no user decision, no data leaving the device.
For a browser or extension the answer is almost always the feed, because the alternative does not survive contact with the privacy requirement.
The Daily Threat Feed is $499 per month or $499/month — a third off the monthly rate. It includes unlimited downloads, which is the property that matters when you are distributing to clients: your update cadence, your staging environments, your build pipeline and your CI checks can all pull as often as they need without changing the bill. The annual tier adds historical archive access, priority support, custom format options, a dedicated account manager and 100,000 API credits.
Enough for the API-shaped work a browser team actually does, which is not navigation-time checking: validating user-reported hostnames, spot-checking during development, and screening the destination hostnames of anything you distribute — new-tab content, bundled bookmarks, partner links — through /batch at a hundred domains per call and one lookup each. Beyond the allowance, packages run from Growth at $99/month for 25,000 lookups at $0.0040 through Professional at $249/month for 100,000 lookupsup to Enterprise at $999/month for 750,000 lookups on the Enterprise plan, billed monthlyunder ten percent usage .
Custom delivery — SFTP or S3 rather than an HTTPS pull — fits organisations whose build systems ingest third-party data through a fixed pipeline, and on-premise deployment exists for those who cannot depend on an external endpoint at all. Both are quoted rather than listed. Custom update frequency is also available, which matters if your client update channel runs on a cadence other than daily and you would rather align the two than carry a diff you cannot ship.
The /stats endpoint reports current database size and last update time and requires no API key, no credits and no authentication, which makes it a reasonable target for a monitoring check while you evaluate. Full pricing detail is on the pricing page, delivery options on the daily feed page, and endpoint semantics in the API documentation.
Mostly about privacy, memory and what happens when the list is wrong — which are the right things to ask about.
No, and stacking it as a second independent list is the sensible framing. Built-in protections in mainstream engines are broad and cover several categories; this is a narrow dataset of domains verified as currently resolving and confirmed as credential harvesting, rebuilt every 24 hours.
Less than teams expect, provided you do not keep raw strings resident. Around 390,000 hostnames stored as truncated digests in a hash set, or as a probabilistic filter with an exact confirmation step, keeps the resident structure in the low hundreds of kilobytes — small enough to sit in a browser process without argument.
Technically yes; the check endpoint responds in under 50 milliseconds. For a browser you should not, and it is worth being direct about why.
Decide the rule explicitly and apply the same normalisation to both sides. Lowercase, strip any trailing dot, and convert internationalised labels to their encoded form before matching — otherwise visually identical hosts will compare unequal in ways that are extremely hard to debug from a user report.
Assume it will happen and build the path first. The interstitial should carry a report-a-mistake route, the bypass should be scoped to that host and that session, and enterprise policy should be able to hold a permanent allow-list applied after each list update so an override is not silently erased by the next refresh.
No, and this is the single most common design mistake in this product category. A clean result means the hostname is not in the list — it carries no positive information at all. Rendering it as a green tick or a "verified safe" badge converts the absence of evidence into an assurance you cannot support, and that badge will eventually appear on a page that harms somebody.
That depends on your update channel more than on the data. The build completes daily at 04:30 UTC with a changelog of additions and removals; if your client pulls the diff once a day, worst-case staleness is about a day plus your propagation time.
It depends entirely on whether you are shipping to users. If you are distributing a client that needs a current list, the feed is the only architecture that respects your users' privacy, and $499 a month is the cost of that architecture. There is no cheaper shape that does not involve sending browsing data somewhere.
/stats endpoint first, since it needs no key and no credits, then decide which shape your product actually requires.Take the full database daily, compile it into your engine, and stop confirmed credential-harvesting hosts at navigation time — with no lookup traffic, no dependency on connectivity, and nothing about a user's browsing leaving the machine.
