How the over-time evidence works

What goes in

Measured values only: one per measure, place and data year. We leave out values carried forward from an earlier year (they'd read as "no change"), measures that never change, undated snapshots (such as today's Google Places counts), the year in progress, and on χGRIDs every cell that is estimated from the wards around it (neighbouring cells would be near-copies).

The year-adjusted change

Every over-time number uses the same quantity: how much a measure changed in a place from one year to the next, compared with the city as a whole that year.

ΔX̃_a,t = (X_a,t − X_a,t−1) − average over places of (X_t − X_t−1)

Subtracting the city's average removes a shock that moves every place by the same amount. It does not remove a shock that hits places unevenly, which is why the tests below compare against shuffled places rather than against zero or 50%.

The scatter

The big chart at the top of the relationship page is one dot per place in the latest year. The dashed lines are the medians of the two measures (not the averages), so each quarter holds the places above or below the middle place on each. Pearson r measures the straight-line tie; Spearman ρ (the same on ranks) catches curved ones. Where places differ, they differ in a hundred ways at once, so the scatter says nothing about one measure affecting the other. That's what the over-time checks are for.

Every place, every year

Each small panel pairs every place's change in the first measure (X) with its change in the second (Y) at a gap of k years. Y later (k = +1…+5) reads Y after X; X later (k = −1…−5) reads it before. Both directions are shown because a link that only shows up with X later points the other way.

Same-direction share

Place-years where both rose or both fell, out of those where neither was exactly unchanged.

Shuffled places (the area-permutation test)

We pair each place's whole X history with a randomly drawn other place's whole Y history, 1,000 times (seeded, so the page always shows the same numbers). That keeps every year's citywide shock and each measure's own drift, and breaks only the link between places. The p-value is the share of shuffles at least as far from their average as the real share. The grey band on the chart is where the middle 95% of shuffles land.

Why not 50%?

A shock that hits a few places very hard pulls the city's average down that year, which puts most places "above the city" on both measures at once. The share then runs well above 50% with no link between places at all. Shuffling keeps that year, so the share has to beat the shuffles, not a coin flip.

Eleven gaps, eleven tests

About one gap in twenty would clear p < 0.05 by luck. A gap is marked ✓ only if it passes the Benjamini–Hochberg correction across the eleven (false discovery rate 5%).

95% range, resampling years

We redraw whole years with replacement 1,000 times and recompute the share. If one year carries the result, its range is wide.

Which years carry it?

The bar chart under the grid splits each year's same-way and opposite-way counts, so a single year such as 2020 doing the work is visible.

The five checks

Each runs for the first measure → the second, and separately for the second → the first; the relationship page shows both directions side by side.

1. They move together within places evidence

The correlation of the two year-adjusted changes, with the second measure in the same year or up to three years later, whichever is strongest (an effect that lands a year later has no same-year correlation and still counts). It passes when |r| ≥ 0.25 and its shuffled-places test survives the correction for every pair we test on that geography (about 20,000; Benjamini–Hochberg, 5%).

Each shuffle takes its own best lag too, so choosing the strongest lag is part of the test. Its p comes from how many shuffle-standard-deviations the real r sits above the shuffles' average (200 shuffles per geography; a smooth p is what the correction across 20,000 pairs needs). The 95% range resamples whole years at the chosen lag.

Rests on one year? We also drop each year in turn and recompute r. If leaving out a single year takes away more than half of it (or flips its sign), the card says so: "rests mostly on 2020: without that year, r = 0.11". That's the signature of a shock that hit the same places in both measures in one year. It's shown as a caveat, not used to lower the grade: a year can matter for real reasons, and with 15–25 years one year is a big share of the evidence. You decide how much weight it deserves.

2. The first measure tends to move first evidence

Two things must agree. The correlations with the second measure later must beat those with it earlier (best later |r| ≥ 0.1 and at least 0.05 above the best earlier one). And the first measure's past changes must help predict the second's beyond the second's own past (a Granger test: ΔỸ_t on ΔỸ_t−1 and ΔX̃_t−1, with standard errors clustered by place), surviving the correction across every pair, while the reverse test does not clear |t| ≥ 1.96. Needs 10 overlapping years; never counted when either measure is a 5-year census average.

3. The second responds to the first's sharp jumps evidence

A "jump" is a year-adjusted change beyond 2 standard deviations of the measure's own changes. We follow the second measure from three years before to three years after each jump (in standard-deviation units, relative to the year before, flipped so every jump points up), against places with no jump of their own in that window. It passes when the average move afterwards is at least 0.2 SD in the same direction as the correlation, the jumps are spread over at least 5 different years, the second wasn't already moving beforehand (drift under half the effect), and the shuffled-places test survives the correction across pairs.

The jump chart's grey band shows where the line falls when each jump is paired with a randomly drawn other place. The dot grid shows every jump by place and year: twenty jumps in one year are one shock, not twenty experiments.

4. Survives controlling for a common factor condition

A third measure Z whose changes move with both (|r| ≥ 0.3 with each; relatives of the two measures are skipped) could explain the link. We read Z at the same lag as the link: Z with the first measure in the same year, Z with the second as many years later as the link itself. All three correlations are taken on the same place-years, where the two measures and Z are all measured, and Z needs at least 5 such years to be tried. (Mixing a 20-year link with a Z measured for 3 years produces nonsense, such as a link that "reverses" when you control for Z.) Measures that move strongly with both but haven't enough shared years are listed as not tried, with their years. Then:

partial r = (r_XY − r_XZ · r_YZ) / √((1 − r_XZ²)(1 − r_YZ²))

The link holds if the partial r keeps its sign and at least half the size of r_XY on those shared years. The three strongest candidates are shown, each must hold, and the check is labelled with the measures it tried. Only measures we track can be tried: a common cause we don't measure can't be ruled out.

5. Not tied together by how the data is built condition

Fails when both measures are spread onto these areas from the same source boundaries the same way, both are modeled estimates, or both come from the same underlying records (violent and property crime are both police incident reports, for example).

The grade

Why a pair can lose a grade

Every failed check on a relationship page says in words what stopped it. These are the reasons, and why each one is there:

"Their changes barely line up"

The change correlation is under 0.25. Without that, there's nothing for the other checks to explain, so the grade is "We can't tell".

"Clears 0.25, but random pairings of places do this too often"

With tens of thousands of pairs tested, some will clear any fixed bar by luck, and a citywide shock can make unrelated measures look linked. The correlation has to beat what pairing each place with a random other place gives, after correcting for every pair tested. This is the most common reason a pair drops from "weak" to "We can't tell" since these tests were added.

"This only counts once the first check passes"

Timing and jump responses mean little for two measures that don't move together, so they aren't counted then.

"Its sharp jumps fall in only N different years"

Twenty jumps in one year are one shock, not twenty experiments; the jump comparison needs jumps in at least five different years.

"Random pairings move this much after jumps too often"

The response after jumps has to beat the same jumps given to random other places.

"Holding Z fixed, the link keeps less than half its size"

Another measure moving both could explain the link. The grade is capped at "weak", and the page names Z so you can look at it.

"They can move together just because…"

The two come from the same records or the same estimation method, which ties them together mechanically. Also capped at "weak".

Two things are shown but never lower a grade: a link that rests mostly on one year, and a census 5-year average's blurred timing (which only blocks the "moves first" check). Pairs computed before these tests existed keep the grade they had until the next data build.

Shared shocks and effective N

Poverty and violent crime across Chicago's 77 neighborhoods give 1,078 place-years of paired changes, but only 14 distinct years. A citywide shock such as COVID hits all 77 neighborhoods in the same year: it is one observation, close to N = 1, not 77. So next to every count of place-years the reports show the effective N ≈ distinct years, and every test above respects it: shuffling places keeps the shock years intact, and the 95% ranges resample whole years.

Known limits

• Census 5-year averages. American Community Survey figures average five years, so consecutive years share four of them. Their changes are small and blurred in time, and the "moves first" check is never counted for them.
• Shared shocks. Uneven citywide shocks survive the year adjustment. The shuffled-places test and the year resampling are the defence; the per-year bars show when one year carries a result.
• Few jump years. The jump study needs jumps spread over at least five different years.
• Present-day snapshots only. Some measures (Google Places counts) exist only as today's snapshot, with no history, so they never enter the over-time checks.
• Many tests. Tens of thousands of pairs are tested on each geography; every test is corrected for that, and the agreement grid for its eleven gaps.
• Observational data. Passing every check is consistent with one measure affecting the other. Unmeasured common causes, effects within the same year in both directions, and changes in how data is collected can all pass them too.

Every shuffle and resample is seeded, so a rebuild from the same data gives the same numbers. The code and the tests that plant known patterns (a two-year lag that must be found in one direction only, a shared shock that must not pass, a common factor that must be named) live with the API.