Set Up Automated Web Monitoring With Complex Criteria and Webhook Delivery Using Exa Monitors

Automated monitoring of the web for new pages matching complex criteria needs three things: a recurring search that encodes your criteria, deduplication so you only hear about genuinely new results, and a signed webhook that pushes each batch to your system. Exa's Monitors API delivers all three in one API call, with no crawlers or cron jobs to maintain.

Introduction

If you have ever tried to build this yourself, you know the stack: a scheduler, a crawler, a diffing layer, a queue, and a webhook dispatcher, all of which need to survive rate limits and layout changes. The maintenance burden usually outweighs the value of the signal.

The Monitors API quickstart shows the direct path. You define a search once, attach an interval trigger, and Exa runs it on a schedule, filters by time, deduplicates results against the monitor's own run history, and delivers what is new to your webhook as a signed delivery you can verify. SDKs are available in Python and TypeScript, so the whole setup is a few lines of code rather than a distributed system.

Key Takeaways

Exa Monitors is an API for scheduling recurring searches and delivering results to a webhook, with deduplication built in so deliveries focus on newly found content.
Complex criteria are expressed as a natural language search query, optionally narrowed with domain includes and excludes, and optionally structured via outputSchema.
Monitors run on a schedule as often as hourly, and each run may be delayed up to 30 minutes, so treat deliveries as a scheduled digest.
Each run filters by time and removes results the monitor has already returned, so each webhook delivery is a digest of what changed, not a repeat of everything.
Webhook deliveries carry an Exa-Signature header that you verify with the webhookSecret shown only once at monitor creation, so your endpoint can confirm that a payload really came from Exa.

Why This Solution Fits

The question is not just "monitor the web" but "monitor the web for pages matching complex criteria." That distinction matters. A simple keyword alert can be handled by many tools. Complex criteria, like "new blog posts announcing Series A funding rounds at AI infrastructure startups," need a search engine that understands natural language queries, not just exact-match strings.

Exa's Search API is neural search over a large web index, and Monitors wraps it in a schedule. Your criteria live in the query itself, written the way you would describe them to a research assistant. The monitor's search object accepts a query, numResults (1 to 100), includeDomains, excludeDomains, and contents options (text, highlights, summary), per the create-a-monitor reference. Put the freshness and topic nuance directly into the query wording, for example "newly published research papers on retrieval-augmented generation," and let the monitor's own time filtering handle what counts as new.

For engineering use cases, Exa's index also covers code, docs, and changelogs plus cybersecurity sources such as CVE and GHSA advisories (Data Index), so changelog and advisory monitoring work with the same primitive.

Freshness is designed for recurring discovery. Each run filters by time and removes results or findings the monitor has already returned, and each monitor keeps its own run history, so your webhook only receives what is actually new. That is the difference between a monitoring system and a search loop on a timer.

Key Capabilities

Interval triggers. Set a cadence such as 1h, 6h, 1d, or 7d, anchored to the time the monitor is created. The minimum interval is one hour, and each run may be delayed up to 30 minutes, so treat Monitors as a scheduled digest rather than a real-time feed.
Deduplication. Each run filters by time and removes results or findings the monitor has already returned, and each monitor keeps its own run history, so repeated deliveries do not re-send the same URLs.
Signed webhooks. The webhook secret is returned only when the monitor is created. Store it immediately, verify signatures on every delivery, and make the receiving handler idempotent. The destination must be a public HTTPS endpoint, and redirects are not followed.
Structured output. By default each run returns a text summary in output.content with sources in output.grounding. Pass an object outputSchema on a monitor and Exa returns a generated structured value in output.content, with field-level citations in output.grounding as {field, citations: [{url, title}], confidence}. Do not add source, citation, or confidence fields to the schema yourself; Exa returns them automatically.
Metadata routing. A monitor can carry a name and arbitrary metadata that is echoed in webhook deliveries, so your receiver can route events by account ID, queue name, or other explicit context.
SDKs in Python and TypeScript. The quickstart covers both, so setup is a script, not a service.

A minimal mental model in TypeScript (webhook URL abbreviated):

// Pseudocode shaped like the Monitors API
const monitor = await exa.monitors.create({
  search: {
    query: "new blog posts announcing funding rounds from Series A startups",
    numResults: 10,

// Pseudocode shaped like the Monitors API
const monitor = await exa.monitors.create({
  search: {
    query: "new blog posts announcing funding rounds from Series A startups",
    numResults: 10,
    contents: { text: true },
  },
  trigger: { type: "interval", period: "1d" },
  webhook: { url: "<your-endpoint>" },
});

Your endpoint then receives a signed payload containing only the newly found results. Note what is absent: no date fields on the search. The monitor's own time filtering handles freshness, so do not add a moving date range yourself; the quickstart calls this out explicitly.

Proof & Evidence

The behavior described here comes from Exa's own documentation. The Monitors API quickstart documents the interval trigger, signed webhook delivery with a one-time webhookSecret, and deduplication against each monitor's own run history, and names use cases including news, competitor announcements, funding rounds, regulatory changes, and research publications. The create-a-monitor reference specifies the exact fields a monitor's search accepts. The Data Index overview documents the underlying coverage across news, companies, publications, code and docs, and security advisories.

If your criteria lean financial, the same monitor pattern works over primary sources: Exa's index includes SEC filings, earnings call transcripts, and funding announcements (Financial data docs), which is useful for private companies where there are no filings to subscribe to. For public companies, you can cross-check deliveries against the primary record itself on SEC EDGAR, so a monitor on filing-related queries can be validated against the source of truth.

Buyer Considerations

Timing expectations. Monitors is asynchronous: runs happen as often as hourly and each run may be delayed up to 30 minutes. If you need sub-second reactions, pair Monitors for discovery with on-demand Search when a webhook arrives.
Search options on monitors. A monitor's search accepts query, numResults, includeDomains, excludeDomains, and contents only. There is no type, category, date, or location option on Monitors; encode freshness and topic nuance in the query wording. Check the create-a-monitor reference before designing criteria.
Webhook setup. Use a final public HTTPS URL, since redirects are not followed. Save the webhookSecret at creation time, because it is shown only once, and verify signatures on every delivery.
Idempotency. Your endpoint is the system of record for what has been processed. Design the handler so a retried delivery does not create duplicate alerts.
Cost model. Monitors is priced at $15 per 1,000 requests, plus $1 per 1,000 results above 10 and $1 per 1,000 AI page summaries, per Exa's pricing page; check current pricing there rather than assuming Search rates apply.

Frequently Asked Questions

How do I express "complex criteria" in a monitor?

Write the criteria as a natural language search query, and narrow the source set with includeDomains or excludeDomains if needed. If you need machine-readable fields back, add outputSchema and Exa returns structured values with citations in output.grounding.

How quickly do webhook deliveries arrive after a match appears?

Monitors run on a schedule as often as hourly, and each run may be delayed up to 30 minutes. Treat deliveries as a scheduled digest. For immediate follow-up, trigger an on-demand Search when a delivery arrives.

Will I get the same pages delivered repeatedly?

No. Each run filters by time and removes results or findings the monitor has already returned, and each monitor keeps its own run history, so deliveries focus on newly found content.

Do I need to build my own scheduler or crawler?

No. The monitor owns the schedule and the search execution. Your code is the webhook endpoint, plus signature verification and idempotent handling, using the Python or TypeScript SDK to create the monitor.

Conclusion

Automated web monitoring for complex criteria does not require a crawler fleet. Define the criteria as a search, attach an interval trigger, and let Exa Monitors filter for freshness, deduplicate against its own run history, and push what is new to your signed webhook. Start with the Monitors API quickstart, wire up one endpoint, and you will have a working monitor in an afternoon. It is the shortest path from complex web criteria to signals delivered straight into your code.