Webhook callbacks, OAuth redirect targets, profile links, import sources, build manifests — every one of them is a hostname somebody else chose, stored in your database and eventually followed by your servers or rendered to your users. This page is about wiring a lookup against 390,000+ DNS-verified, currently-resolving phishing domains into that path: the exact requests, where the check sits relative to your own allowlist, what to cache, and how the system behaves when the answer never arrives.
Length, scheme, maybe a regex. Almost nobody asks the only question that matters: is anyone home at that hostname, and are they hostile?
Walk the schema of any mature SaaS product and count the columns that hold a URL. There is the webhook endpoint a customer registers so you can push events to them. There is the OAuth redirect_uri an integrator supplied when they created a client. There is the avatar source, the company website on the organisation record, the documentation link in a marketplace listing, the callback on a payment intent, the import URL on a bulk-load job, the repository address in a build config. Most of these were added by different teams in different quarters, each treated as a display string, and each validated with roughly the same rigour: does it parse, is it under two thousand characters, does it start with https. None of that says anything about whether the host on the other end exists to steal credentials.
That gap matters more in an API product than on a website, because of what happens next. A URL that arrives through a browser form is usually looked at by a person before anything acts on it. A URL that arrives through your API is stored, indexed, replicated to a read model, rendered into an email template, embedded in a chat notification, followed by a background fetcher, and possibly re-emitted through your own API to a third party — all without a human ever seeing the string. If the hostname is a live credential-harvesting site, your infrastructure carried it the last mile, with your certificate and your brand wrapped around it.
The second reason this is an API problem rather than a general security problem is scale asymmetry. A phishing operator does not need to compromise your platform to use it; they need an account and a documented endpoint. Any product that accepts a URL and later shows that URL to another user is, from the operator's point of view, a redirect that inherits your domain's standing with mail filters and browser warnings. The cost of the abuse falls on your deliverability and your support queue, not theirs. Teams running marketplaces and multi-tenant products meet this first; the same shape recurs on our SaaS platforms and online marketplaces pages.
What a phishing-domain lookup adds is specific, and it is worth stating precisely so nobody expects more of it. It answers one question: has this exact hostname been observed and verified as currently resolving phishing infrastructure? It is not a URL scanner, not a content classifier, and not a reputation score you can tune. It is a membership test against a list of 390,000+ domains that were confirmed to have a live DNS record at the last rebuild, which happens every 24 hours. Dead domains are dropped rather than retained, so the list describes infrastructure standing up right now instead of an archive of everything ever reported.
That narrowness is exactly why it is easy to place. There is no model to train, no threshold to tune, no traffic to mirror and no agent to deploy. There is a hostname, a request that returns in well under 50 milliseconds, and a boolean you can act on. The engineering work is not the call. It is deciding where in your request path the call belongs, what a positive should do, and how the system behaves on the day the call fails.
Each has a different owner, a different code path and a different blast radius, but they all terminate at the same lookup.
A tenant registers a destination for your event stream. From that moment your servers make authenticated outbound requests to a host of their choosing, on a schedule, carrying whatever the event payload contains. If the destination is a harvesting site, you have built them a reliable data pipe — and it keeps running long after anyone opens that configuration screen again.
redirect_uri and returnToClient registration takes a redirect target. Post-login flows take a returnTo or next parameter. Both hand an attacker a way to end a genuine authentication journey on their page, with your consent screen as the credibility step immediately before it. Exact-match registration helps; it does not help if the exact match that was registered is itself hostile.
Company website, support URL, docs link, social handle, custom branding target. Low-drama fields that get rendered into invitation emails, directory listings, embed cards and partner exports. They are the most reliably abused fields in any multi-tenant product precisely because nobody classifies them as security surface.
Anything that accepts free text and later renders it: comments, tickets, chat messages, generated PDFs, notification bodies. Here the hostname is buried in content rather than sitting in its own column, so extraction has to happen before validation — and the same string may need re-checking when the document is rendered again months later.
Link previews, remote image ingestion, import-by-URL, feed pollers, "connect your site" wizards. These are the endpoints your SSRF controls already worry about, and the two concerns compose neatly: SSRF rules decide whether an address is internal, a phishing lookup decides whether a public hostname is hostile. Both run before the fetch, never after.
Your own developers are targets. "Your API key expires in 24 hours", "action required on your organisation's billing", "sign in to approve this deploy" — sent to the address on the developer account, pointing at a clone of your portal. A leaked production key is worth more than most customer records, and it never triggers a failed-login alert.
In every case the hostname is chosen by somebody outside your trust boundary, stored by you, and acted on later by an automated process with no human in the loop. That combination — external origin, durable storage, delayed automated use — is what turns a bad string into an incident, and it is also what makes the problem tractable. There is always a single moment, at write time or at use time, where one lookup can be inserted without redesigning anything around it. Find that moment for each of the six and you have covered most of the surface. What remains is the pass-through case, where a URL is neither written nor followed but merely relayed, which is why the outbound side gets its own treatment on the link redirection and email security pages.
Redirect handling is where URL validation is most often present and least often useful, because the checks that exist test structure rather than destination.
The classic open redirect is well understood: an endpoint takes a target parameter, issues a 302 to it, and an attacker uses your domain as the visible part of a link that lands somewhere else entirely. The standard remedy is an allowlist of permitted destinations, and it works. What is discussed far less is what happens once your product deliberately supports third-party destinations — which every OAuth provider, every marketplace and every integration platform eventually does. At that point you are no longer preventing redirects to arbitrary hosts. You are approving specific ones at registration time, and the question shifts from "is this parameter constrained" to "was the thing we approved actually legitimate".
Registration-time approval is an excellent moment to insert a check, because it is rare, cheap and already synchronous. When a developer creates an OAuth client with a redirect target, or a tenant saves a webhook URL, the write is a user-facing operation that can easily absorb a 50-millisecond outbound call, and the user is present to read an error. Rejecting there beats discovering it later by a wide margin: you get a clean failure attributable to a specific account, no half-configured integration sitting in the database, and a log line naming both the domain and the actor. The same applies to a dynamic returnTo value — check it where you decide whether it is permitted, not where you emit the header.
Registration-time checking alone is not sufficient, and it is worth being blunt about why. Domains change hands. A callback URL registered eighteen months ago against a perfectly legitimate integration can lapse, be re-registered by somebody else, and start serving a login clone without anyone touching your configuration. That is the argument for a periodic re-check of stored URLs rather than a one-off gate: pull the distinct hostnames out of your webhook and client tables, push them through POST /api/v1/batch in groups of a hundred, and act on anything whose status has changed. For most products that is a few hundred domains and a nightly job costing a few cents — and in practice it is where the interesting findings come from.
One normalisation point deserves separate attention, because it is the most common way a check gets bypassed by accident. If you parse the URL, check the host, and then hand the original raw string to the redirect, an attacker with a crafted authority section, an embedded credential, a trailing whitespace character or a mixed-script internationalised label can make those two disagree. Parse once, extract the host once, check that, and use the parsed result downstream. Never re-derive from the original string after validation has passed.
http/https schemes outright. A phishing-domain list has nothing useful to say about javascript:, data: or file:, and those should never reach it in the first place.https://[email protected]/ has a hostname of attacker.example, and a naive substring check reads it the other way round.login.evil-host.example and evil-host.example are distinct entries; if both matter, send both in the same batch.The whole integration surface fits on one screen. That is deliberate — there is nothing to learn beyond the fields.
curl -s "https://phishingdetectionapi.com/api/v1/check\
?domain=account-verify-secure.example.com\
&apikey=$PDA_KEY"
{
"domain": "account-verify-secure.example.com",
"is_phishing": true,
"category": "phishing/malware",
"dns_status": "resolves",
"last_checked": "2026-07-27T04:30:11Z",
"confidence": 0.98,
"database_size": 391204
}checked, phishing_found and credits_used.curl -s -X POST https://phishingdetectionapi.com/api/v1/batch \
-H "Content-Type: application/json" \
-d '{
"apikey": "YOUR_API_KEY",
"domains": [
"hooks.partner-integration.example.net",
"cdn.assets-eu.example.org",
"sso-login-portal.example.com"
]
}'One credit per domain, whether it arrives singly or inside a batch. lookups reset monthly. Full field reference on the API documentation page; the complete database as a daily CSV is described on the daily feed page.
Ordering matters more than anything else in this integration. Get it wrong and you spend credits on your own domains while shipping a hard dependency into your write path.
The correct ordering is allowlist, then cache, then API — and the key never leaves the server at all. Each position earns its place, and each one is cheaper than the position after it.
Your allowlist is the set of hostnames whose answer you already know and would never act on regardless: your own domains and subdomains, your identity provider, your payment processor, your CDN, and the handful of partner domains that carry most of your legitimate traffic. Checking those against an external service is pure waste, and worse, it creates a world in which a false positive on a domain you control can break your own product. Put the allowlist first and that scenario simply cannot arise.
The cache comes second because the distribution of hostnames in any real system is extremely skewed. A product holding millions of stored URLs typically sees a few thousand distinct hostnames per day, and a large share of those repeat within minutes. A local cache keyed on the normalised hostname collapses that traffic dramatically. The TTL policy that works is asymmetric: cache a positive for around twelve hours, because a confirmed live phishing host is very unlikely to become benign inside a single rebuild cycle, and cache a negative for a shorter window — an hour is a reasonable default — because "not on the list yet" is the answer most likely to change. The database rebuilds daily, so no cache entry should ever outlive 24 hours.
The API call comes third and should always carry a short timeout. Two seconds is generous for an endpoint that typically answers in under 50 milliseconds, and it converts a network problem into a decision rather than a hang. What you do with that decision is a product question, not a security one. On a registration form, failing closed — refusing the write and asking the user to retry — is defensible, because the user is present and the operation is rare. On a read path rendering stored content to millions of requests, failing closed is an outage: fail open, emit a metric, and let the asynchronous re-check settle it later.
The last rule is the one that most often breaks under deadline pressure: the API key is a server-side credential. It belongs in the query string of a server-to-server request or in the body of a POST, and it never appears in a single-page bundle, a mobile binary, a public repository or a browser network tab. If a client needs a verdict, expose your own thin endpoint that performs the lookup with your key, applies your allowlist and returns nothing but the boolean. That also gives you a natural place to rate-limit per tenant, which you will want the first time somebody points a retry loop at it.
/**
* Returns true (known phishing), false (not on the list),
* or null (we could not find out). Callers must handle null
* explicitly and must never coerce it to a boolean.
*/
function hostIsPhishing(string $host): ?bool
{
$host = strtolower(rtrim(trim($host), '.'));
// 1. Our own allowlist wins, and never costs a credit.
if (Allowlist::contains($host)) {
return false;
}
// 2. Cache: 12h for a hit, 1h for a miss. Never past a rebuild.
$cached = Cache::get('pda:' . $host);
if ($cached !== null) {
return $cached;
}
// 3. Only now do we spend a credit.
$url = 'https://phishingdetectionapi.com/api/v1/check'
. '?domain=' . rawurlencode($host)
. '&apikey=' . PDA_API_KEY;
$ctx = stream_context_create(['http' => ['timeout' => 2]]);
$raw = @file_get_contents($url, false, $ctx);
if ($raw === false) {
Metrics::increment('pda.unreachable');
return null; // caller decides fail-open or closed
}
$body = json_decode($raw, true);
if (!is_array($body) || !array_key_exists('is_phishing', $body)) {
return null;
}
$bad = (bool) $body['is_phishing'];
Cache::set('pda:' . $host, $bad, $bad ? 43200 : 3600);
AuditLog::record($host, $bad, $body['last_checked'] ?? null);
return $bad;
}Four stages, each with a clear owner. Only the third one ever leaves your process.
Parse once. Extract the host, lower-case it, punycode it, drop userinfo, port, path and query. Everything downstream uses this value and never the original string.
Allowlist first, then the local cache. Most calls end here, which is what keeps both latency and credit consumption predictable at volume.
One domain to /check, or up to a hundred to /batch. Two-second timeout. Failure returns an explicit unknown, never a silent false.
Block, quarantine or warn according to how reversible the action is, then write the domain, verdict and timestamp next to the row you accepted.
The sizing exercise is not "how many requests do we serve". It is "how many distinct hostnames do we see in a day".
Start by measuring the right number. Take a day of production logs, extract every hostname your product would have checked under the design above, subtract everything on your allowlist, and count the distinct remainder. For most products that figure is startlingly small relative to request volume: a system serving tens of millions of requests may see well under ten thousand distinct external hostnames in a day, because the same webhook endpoints and the same customer domains recur constantly. That count, not your traffic, is your credit consumption once caching is in place.
One credit is one lookup, a batch of a hundred domains costs a hundred credits, and lookups reset monthly. A product checking five thousand distinct hostnames a day and running a nightly re-check over a few hundred stored callbacks lands comfortably inside the Professional package for a year. The full table is on the pricing page.
Rate limiting is your own concern rather than something the API imposes on your architecture, and the shape that works is a bounded worker pool rather than an unbounded fan-out. When a bulk import arrives with fifty thousand rows, do not spawn fifty thousand concurrent lookups. Extract distinct hostnames, deduplicate against the cache, chunk what remains into groups of a hundred, and push them through a small number of workers with a queue in front. The batch endpoint exists precisely for this: one request, one round trip, a hundred verdicts. Backfilling an existing table of stored URLs follows the same pattern and is usually a single evening's work.
There is a second deployment model worth knowing about, because for some architectures it removes the per-request question entirely: the complete database is also published as a daily CSV, alongside the changelog of domains added and removed, as a separate subscription. The two models answer the same question from different places in your stack, and most teams end up running both.
| Consideration | Inline API lookup | Daily CSV feed |
|---|---|---|
| Cost at decision time | One credit per distinct hostname, charged only after the allowlist and the cache have both missed. | No per-query cost at all. The feed is a separate subscription covering the full database and its added/removed changelog. |
| Freshness | Reflects the most recent 24-hour rebuild at the moment you ask, with last_checked in the response. |
As fresh as your last import, so a rebuild-cycle-sized delay is built into the model by design. |
| Where enforcement sits | In application code — at the write, at the render, or in the background job that re-checks stored rows. | In a resolver policy zone, a firewall category or an in-process set loaded at boot. |
| What it costs you | An outbound call on the critical path, a timeout policy, and a cache you have to reason about. | Memory and an import job, in exchange for no outbound dependency at decision time. |
| Best suited to | User-supplied URLs arriving through your API, where the verdict must be attached to a specific write. | Network-layer controls that must answer instantly and cannot make an egress call to do it. |
Teams that already operate DNS-layer controls usually take the feed for the network and the API for the application; the split is described on the DNS filtering and threat intelligence pages.
None of these are exotic. Each has a design answer that costs nothing if you make it before launch.
Your egress proxy is down, DNS is having a moment, or a deploy shipped a broken outbound rule. The function must return an explicit unknown — a third state, not a false — and the caller must branch on it.
Decide this locally rather than globally. A write path with a user present can fail closed and show an error. A read path serving stored content should fail open, emit a metric, and let the nightly re-check settle it. What you must never do is treat unreachable as clean and log nothing.
The cap is a hard limit of 100 domains per POST /api/v1/batch request, so chunking happens in your client — and the chunker is the single most common place a bug hides. An off-by-one that silently drops the tail of the last chunk produces a check that passes its tests and misses domains in production.
Compare checked in the response against the length of the array you sent and alarm if they ever disagree. Two lines, and it catches an entire class of silent failure.
A cached negative from before the daily rebuild is the one that hurts, because the domain added overnight is precisely the one being used against you this morning. Any TTL longer than the rebuild cycle means you are enforcing yesterday's list while believing you are current.
Cap every entry below 24 hours, keep negatives short, and build a manual purge you can trigger during an incident. If one specific domain matters right now, you want to bypass the cache for it without a redeploy.
It happens the same way every time. Somebody builds a link-checking widget, calls the endpoint straight from the browser to skip a backend ticket, and the key ships inside a JavaScript bundle. From there it is scraped and spent, and your credits fund somebody else's tooling.
Hold the key in server configuration, expose your own proxy endpoint that returns only a boolean, rate-limit that endpoint per tenant, and grep your build output for the key prefix in CI. The last one takes five minutes and catches the mistake before it reaches a CDN.
This is a floor, not a perimeter. Knowing exactly where the floor ends is what makes it useful rather than merely reassuring.
It does not catch a domain that has not been observed and verified yet. The list contains hosts confirmed to be resolving at the last rebuild, which means a name registered this morning, pointed at a server this afternoon and fired into a campaign tonight may not appear until the next cycle. That is a deliberate trade: verifying that a domain actually resolves is what keeps the database free of dead entries and stale false positives, and the cost of that precision is a window on the very newest infrastructure. If your control objective is specifically to spot lookalike registrations of your own brand on the day they are filed, that is a registrar-data problem needing a different source — see the brand protection and domain registrar pages.
It also says nothing about compromised legitimate sites. A phishing kit dropped into a directory on a real company's neglected content management system lives on a hostname that is not phishing infrastructure in any structural sense: it belongs to a real business, resolves to their real host, and will still be theirs next week. Path-level abuse on an otherwise legitimate domain is outside what a domain-membership test can express, and any product claiming otherwise is describing something else.
And it does not interpret behaviour. It will not tell you that a webhook destination is quietly exfiltrating to a legitimate-looking analytics endpoint, that an OAuth client is over-scoped, or that an integration started behaving strangely last Tuesday. Those belong to authorisation design, egress policy and anomaly detection, and this sits underneath all of them rather than replacing any. What it provides is a fast, cheap, mechanical answer to one question that currently has no answer at all in most API products.
An answer arriving in under 50 milliseconds, before the write commits, is worth considerably more than a richer verdict that arrives after the fact.
The seven that come up in almost every integration review.
Both, at different call sites. Inline belongs on rare user-facing writes: registering a webhook, creating an OAuth client, saving tenant metadata. These happen infrequently, a user is present to read an error, and 50 milliseconds on a form submission is invisible.
The hostname, and only the hostname. The endpoint takes a domain parameter, and sending anything more is both unnecessary and a data-handling exposure you do not need to take on.
No. Entries are hostnames, so login.something.example and something.example are separate lookups with independent answers. Phishing infrastructure frequently lives on a subdomain of a name whose apex is parked or empty, so the parent may legitimately not be listed at all.
Put the client behind an interface and use a fake in unit tests. The contract is small enough that a stub returning true, false and null covers every branch you need — including the unreachable path, which is otherwise awkward to exercise.
Both, because they catch different things. A CI step that extracts hostnames from changed build manifests, dependency lock files, infrastructure templates and deployment configs, batches them, and fails the pipeline on a hit stops a hostile endpoint being merged at all. It sits naturally alongside the checks described on our supply chain page.
Make the consequence proportional to how cheaply it can be reversed. Refusing a webhook registration is trivially reversible: the user sees an error and can contact support. Silently deleting stored data, or hard-blocking an established integration mid-flow, is not.
No. Anything shipped to a client device is extractable, and an extracted key spends your credits and appears in your usage as though it were you. The documentation states server-side use as a requirement rather than a recommendation.
Pull the distinct hostnames out of your webhook, OAuth client and profile tables, send them through one batch call, and find out what you have already accepted. A hundred domains per request, and a few minutes of work.
