This is the long version, written by the people who built it. If you want to know where the data comes from, how the live map stays current, how much work happens before a report reaches you, and why you can trust the numbers, read on. We will keep it plain.
The live map
We keep a very large index of Etsy listings current through a change capture chain, so changes on Etsy land with us in minutes. The map streams from that index using a forward cursor on each listing’s last updated time, so it only ever moves ahead and never re scans the world.
Raw data does not arrive evenly. It comes in bursts, then goes quiet, which would make a jumpy feed. So we run the stream about five minutes behind real time and hold a buffer of data ahead of the playback point. Events are released at a steady rate, and every one is stamped by a single playback clock, so the feed flows smoothly even when the source is briefly silent. The events per second counter shows the live arrival rate of all signal types combined. The stream speed control lets you match real time or slow it to 30, 60 or 90 per minute so you can actually read it. The top markets panel ranks countries by activity, new listings or reviews over roughly the last minute, and recolors the globe to match.
Four signal types come straight from watching listings change: new, sold, expired, and updated. Favorites and views are derived by comparing counts between observations, and topped up by a worker that polls Etsy’s batch endpoint every few minutes for the busiest listings, because a favorite does not always force a listing to re publish. Reviews arrive from a separate near real time feed. In production the map shows real data only. If the stream drops, it reconnects instead of inventing anything.
The pipeline behind the reports
Reports are not a live stream. They are built by a pipeline that runs the same steps for each of the 25 published categories, one at a time. We sweep Etsy’s official API as much as our quota allows, which is millions of requests a day, prioritizing the strongest signals rather than trying to read every listing. The rate limiter only ever slows us down, and on any push back from Etsy it backs off further. Each stage works in small chunks instead of one giant query, so nothing falls over at scale.
The flow is: sweep listings and reviews, prune dead and ended listings so the set only holds products you can still buy, enrich missing details once and cache them so we never refetch, snapshot favorites and views daily, aggregate reviews per day, compute the change over each window, rank the movers, and publish a finished report the app reads in milliseconds. Reviews are deduped by transaction at every layer, so the same purchase is never counted twice.
What each number is for
The report ranks by reviews gained, the confirmed sales signal. The detail view adds favorites and views and a favorites per view proxy for interest, plus shop context: average rating, lifetime completed sales, active listings, years open, stock on hand, and an approximate sales per day. The listing attributes, type, processing time, variations, where it ships from, and the published, modified and renewal dates, all come straight from the listing. Every field has a plain definition in the product, because a metric without a definition is just decoration.
Freshness and trust
Some stages run continuously and others are orchestrated on a daily schedule, and a watcher alarms us if any output table goes stale, so a silent failure upstream shows up fast. We lean on reviews because a review can only follow a delivered order, which makes our sale counts conservative and real. We state the limits out loud: sales are a floor, prices are the listed price before discounts, and favorites per view is interest, not conversion. No black box, no inflated claims. Just the marketplace, read carefully within the rules, cleaned, measured and ranked, every day.