Integration and operations
Updates and deployment
Refresh intervals, live dependencies, and offline operation.
Exactly one live dependency
A lookup queries Elasticsearch for our own measurements and reads all other data from disk. The thirteen external lists are never queried at request time.
One timer, thirteen intervals
A single hourly invocation refreshes whatever is due. Each source carries its own interval, so no second schedule is maintained.
node scripts/ingest-feeds.ts
| Source | Interval | Why |
|---|---|---|
| Tor exit list | 1 h | Exit nodes turn over hourly |
| Spamhaus DROP | 6 h | Updated several times a day |
| Everything else | 24 h | Daily to weekly at the publisher |
When the backend is unreachable
Locally held data continues to answer. A Tor exit, a Private Relay egress or an address on a per-provider list resolves from disk, and the response carries degraded to indicate the measured component is absent. Requests requiring our own observations return 502 with the reason stated, rather than a bare 500.
When a source fails
- A single source failure does not affect the others.
- The previous file is retained. Stale data that reports its own age is preferable to no data.
- Concurrent runs cannot collide. A lock left by a terminated process is cleared rather than blocking subsequent runs.