How We Track Every Incident on Earth — and What We Cannot See

There is no feed of "what just happened in the world". People assume there is — some United Nations wire, some global emergency channel — and there simply is not. What exists instead is a scatter of narrow, excellent, incompatible feeds: one that knows about earthquakes and nothing else, one that knows about American severe weather, one that knows about volcanoes and publishes weekly. Between and beneath them sits everything else, and everything else is most of it.

This is how we assemble a single picture out of that scatter, what the picture is genuinely good at, and — the part most trackers skip — what it cannot see at all.

Layer one: the feeds that already exist

Natural hazards are the well-served part of the world. Scientific agencies monitor them continuously and publish machine-readable output, because that is what their own responders need.

We read twenty-four-three-one such streams. Earthquakes come from the USGS and EMSC, plus the national networks — BMKG in Indonesia, GeoNet in New Zealand, INGV in Italy — that catch local shaking the global feeds are too coarse to report. Floods, cyclones, droughts and large wildfires come from the United Nations' GDACS, with its official Green, Orange and Red alert levels. NASA EONET and NASA FIRMS add satellite observation, FIRMS detecting the heat of active fires from orbit. Severe weather warnings come from the US National Weather Service and, across Europe, from Meteoalarm, which carries each national meteorological service's own warnings. Volcanoes come from the Smithsonian Global Volcanism Program, tsunamis from NOAA and the Pacific Tsunami Warning Center, outbreaks from the WHO along with WHO Africa and the European CDC.

These are the strongest part of the dataset. When we record an earthquake, the magnitude, depth and coordinates came from a seismic network, not from a newspaper.

The gap nobody advertises

We classify incidents into twenty-four-three-eight types. Sixteen of them have an official live feed anywhere on Earth. The other twenty-four-three-two do not — not in any country, from any agency. We publish the per-type breakdown, with a CSV, at what the world does not publish.

There is no API of building collapses. There is no global bus-crash feed, no live registry of factory fires, no structured stream of gas explosions, mine accidents, stampedes or bridge failures. No agency publishes them, in any country, in real time. Those events exist in the record only because a journalist wrote about them.

That is an uncomfortable thing for a tracking site to say out loud, which is why the about page carries a table showing exactly which of our types come from official feeds and which are news-derived. If a tracker implies it has satellite-grade coverage of road accidents, it is either mistaken or counting on you not to ask.

Layer two: reading the world in its own languages

Since news is the only route to those events, the question becomes which news. International wires cover a wall collapse in Faisalabad or a market fire in Lagos approximately never. Local outlets cover them within the hour — in Urdu, in Yoruba-inflected English, in Turkish, in Vietnamese.

So we read locally, in three passes.

The sweep rotates across 79 countries, asking each one a broad incident question in its own language. The harvester takes the 34 countries where under-reporting is worst and asks six specific questions each — buildings, fire and blasts, transport, casualties, disasters, health — because a general query returns whatever is loudest that day, while a category query returns the category. The national press layer skips aggregators entirely and reads 42 major newspapers straight from their own feeds: Dawn and the Express Tribune, the Times of India, Prothom Alo, Detik and Kompas, Punch and Vanguard, Hürriyet, G1, VnExpress. Every headline they publish goes through our classifier.

That last layer exists because of something we found by testing rather than assuming: Google News has essentially no Urdu index. Common Urdu words for accident and explosion return nothing. A Pakistan-focused tracker relying on an aggregator would quietly miss the country it claims to cover.

The classifier had to learn to read

Then came the more embarrassing discovery. We built the local-language feeds and the incident count barely moved. Hürriyet, G1 and VnExpress were returning zero incidents each.

The classifier was English-only. It was reading Turkish headlines and looking for the phrase "building collapse", which of course never appeared — a Turkish paper writes bina çöktü. Every non-English feed we had just built was being silently discarded at the last step.

The classifier now holds close to a thousand keyword patterns across Urdu, Hindi, Bengali, Arabic, Persian, Turkish, Indonesian, Malay, Vietnamese, Thai, Chinese, Japanese, Korean, Russian, Ukrainian, Spanish, Portuguese, French, German, Italian, Polish, Romanian, Greek, Dutch and Swahili, alongside English. A test run over 848 real headlines produced 264 classified incidents.

Layer three: putting it on the map

Knowing something happened in Pakistan is not much use. Knowing it happened in Faisalabad is.

Headlines are matched against a built-in gazetteer of 30,719 cities drawn from GeoNames under a CC BY 4.0 licence. The obvious approach — look for any city name in the text — fails immediately and comically, because a great many city names are ordinary words. There is a town called Police in Poland. There is one called Young in Uruguay, and one called Sparks in Nevada. Match naively and every story containing the word "police" lands in southern Poland.

Matching is therefore deliberately conservative: whole-word only, and a city outside the country the report came from is accepted only if it is large enough to be newsworthy on its own name. Tested against live headlines, this took precise placement from 4% to 17% of incidents while cutting false placements from 27 to 1. We would rather leave an incident at country level than put it confidently in the wrong place.

One event, one record

When GDACS, GDELT and a national newspaper all report the same flood, that is one flood. Every incoming item is fingerprinted by type, date and headline keywords, matches are merged, and the record's source count rises. More independent sources reporting the same event means higher confidence, and that count is shown on each incident page.

Getting this wrong was our largest single error. An audit found roughly 5,255 records holding only about a third as many distinct stories. The cause was subtle: Google News answers a query about any country with the same big international stories, and our fingerprint included the country — so one Turkish earthquake headline surfaced in four country queries became four separate records in four countries. Dropping country from the fingerprint for news-derived sources, and never letting a query's country be inherited when geocoding failed, fixed it. A cleanup pass merged 2,151 duplicate rows.

We mention this because a tracker that has never found a bug of that size in its own data has probably not looked.

What the numbers do not mean

This matters more than any of the above.

Our counts are detections, not occurrences. A rise in recorded incidents can mean more incidents happened, or that we added a source, or that a region's press had a busy week. Because most categories are news-derived, coverage follows media attention: an incident in a country with a dense local press is far more likely to be recorded than the identical incident somewhere with little reporting. We reduce that bias by reading in local languages. We cannot eliminate it.

Nor are our counts a casualty toll. We publish death and injury figures only where an official source states them, and early figures after any major event are routinely wrong in both directions.

And the archive has a start date. Continuous monitoring began when this site did, so a stat band reading "4,700 this year" is 4,700 that we recorded — never a world total. Every counter on the site carries that note underneath it for exactly this reason. Severity, likewise, is derived from the source signal — earthquake magnitude, GDACS alert colour, or how many independent outlets reported a news incident. It is a monitoring signal, not an official assessment.

Use the data

The full archive is downloadable as CSV and queryable as JSON on the data page, with the licence and a citation format. Researchers and journalists are welcome to it; we ask only that you cite the specific incident URL rather than the domain, and that you carry the detection caveat above into whatever you publish.

Live source health — which feeds are responding and which are failing right now — is public on the status page. If you find something we missed, the report form goes to a moderator, and confirmed submissions are published as records with the same treatment as everything else. You can browse by incident type, by country, check whether shaking near you was an earthquake, or subscribe to free alerts for your area.

Detection is only half the point, though. The reason we bother recording a wall collapse in a city no wire service covers is that these events are patterned and, often, preceded by warnings people can learn to read — which is what our safety guides are for, starting with the signs a building gives before it fails.

Everything described here runs automatically every ten minutes. The interesting engineering was never the collecting — it was deciding what we are not entitled to claim.