New Hampshire in presidential politics

Measurements of smaller populations should have larger uncertainty and error bars - and this is exactly what we see in New Hampshire, the smallest state to be covered so far on this blog series. Aside from a spectacular win by George H. W. Bush in 1988, New Hampshire has fluttered back and forth from republican-leaning in 1992 to democratic-leaning in 1996, back to the right in 2000, and finally back to the left for the following 3 elections since. I'll interpret this to mean that the Dukakis campaign probably abandoned all hope of winning there, and focused instead on other states. Subsequent campaigns probably focused more on New Hampshire, bringing it closer to the center.

We can see the somewhat ridiculous prediction by my naive linear model: an 8.9% advantage by the democrat, with rather large 4.6% error bars. Any real person looking at this data would predict that it'll probably be much closer, with perhaps a small democratic advantage (neglecting 1988 from my linear fit model - without any objective basis - gives a prediction of a smaller democratic advantage of 0.8-5.4%). My model could benefit from some way of weighting recent elections more heavily than long-ago elections. Maybe next time.

Coming up sometime soon (maybe) by popular demand: including the midterm election data in my prediction and analysis!

Arizona in Presidential Politics

Good old Arizona. I've seen a fair bit of interest on the left in a demographic advantage that Democrats my someday gain in Arizona (and other states, mostly in the South). So when I analyzed Arizona's margins in the last 7 elections, I was expecting to find some interesting wiggles and bounces. I was disappointed.

Aside from a small bounce for Republicans in 2008 (caused by hometown hero John McCain at the top of the ticket), Arizona has had little movement over the past 7 elections. In constructing my simple linear model, I excluded the 2008 data, and got a prediction that in 2016, Arizona will vote about 10% more republican than the nation as a whole. As it always has.

Blatantly screen-grabbing another website that lets me easily make my own little predicted map of the 2016 election, here's what we've got so far:

In this map I've shaded the states red or blue based on my guess as to whether or not a given party has a large advantage in that state. In other words, democrats won't necessarily win Pennsylvania, but if they've lost Pennsylvania, it will be because they've done very poorly in the national popular vote. As I visit each state on this blog, I'll fill in more states on this map, or leave them blank if they're true toss-up states. But as of right now, Democrats seem to be sitting pretty.

Astute readers will note that I have declared certain states without presenting a detailed analysis. For some of those states, I'll visit them in forthcoming posts. For others, I'll just let the history speak for itself (and if you want to gamble that Wyoming breaks blue, I'll take that bet).

Missouri: A Bellwether state from yesteryear

Until 2008, Missouri had the longest unbroken streak of electing the presidential candidate who ended up winning the presidency (although not always the popular vote). But during the last two cycles, Missouri abandoned its bellwether status, breaking toward the republicans even when Barack Obama won the national popular vote by more than seven points.

Starting in 1996, Missouri started drifting to the right of the nation as a whole, culminating in a 10% romp for Romney in 2012 while the nation voted for Obama by 4%. Based on my simple linear model, Missouri is almost certain to break for Republicans by a large margin.

Nevada in presidential politics

The next stop for our tour of possible swing states is Nevada.

We can see a pretty clear case of a state that has shifted somewhat from flirting with the republican party to flirting with the democratic party. Based on my extremely simple model, if this trend continues, Nevada has an 84% chance of voting at least 4.8% more democratic than the nation as a whole.

What might be causing this slow-and-steady shift to the left for Nevada? Perhaps an influx of minorities and young workers.

Kansas in Presidential Politics

Just a quick post that looks like all the others. Here I focus on Kansas, and how it has shifted from 5% more republican than the rest of the country, to reliably 25%+ more republican than the U.S. as a whole.

In contrast to the previous two states I've analyzed, Kansas clearly and definitively departed the "swing state" zone a long time ago. It's probable that neither party bothered to mount much of a campaign, causing Kansas to quickly shift to some kind of "natural state", the way it votes for president when it isn't a focus for both campaigns. And for Kansas, that natural state seems to be solid red.

Pennsylvania in presidential politics

This is the next post in our series, "How are the swing states doing anyway?" Next up, we have Pennsylvania, which has been consistently just a little more democratic than the nation as a whole:

It has jumped around a little bit, so it's hard to discern any real trend other than "probably steady". Quantitatively this translates into our 2016 prediction: Pennsylvania will probably be around 2.6% more democratic than the rest of the nation (68% chance for a democratic advantage between 1.15-4.05%, with large assumptions including a straight-line fit through the historic dem advantage).

Without a clear trend, I won't even try to come up with any kind of guess about what demographic, political, or economic events could have been causing these observed numbers.

Once I complete this kind of analysis for all of the swingier states, I'll be able to construct a model predicting which states are likely to be the tipping point states, and what kind of national vote totals will translate into an electoral college win on either side.

Virginia in presidential politics

I love projecting presidential elections way too early, so obviously it's time to start predicting for the 2016 Big Game!

Today, I wanted to talk specifically about Virginia. Until recently, Virginia has been considered "safely" Republican during the presidential election - even if you count 2008, when Barack Obama won the state. What do I mean?

Let's go back to 2008, when Barack Obama won Virginia by a 6.3% margin, while winning the US popular vote as a whole by 7.3%. Now let's say we adjust our magical "popular vote" dial, reducing the democrat's share of the popular vote equally among all the states. The first state to flip to the republicans would be North Carolina. Then Indiana, Florida, Ohio, and then Virginia, at which point John McCain would still have been 11 electoral votes shy of winning. On top of all those states, he would also have needed to flip Colorado and finally Iowa, which would have been unlikely if he hadn't ALREADY won Virginia. So, for Obama, Virginia was unnecessary to his win, and for McCain, winning Virginia would be a given if he had carried Colorado and Iowa, which would have been required to win the election. Let's use Nate Silver's terminology and designate Iowa the "Tipping Point State" for 2008. Under that model, either candidate just has to win Iowa and all the states on their side of Iowa, assuming each state has a reasonably similar per-capita elasticity (voter response to campaign activity) and each campaign has pretty good knowledge of current polling in each state. How will each state change with respect to the national average? Let's zoom in on Virginia:

In the graph above I've plotted Virginia's Democratic (2-party share) vote margin for the past seven presidential elections. Although the national and statewide margins bounce around chaotically, by looking at the difference ("how republican Virginia is compared to the nation as a whole"), we can see that Virginia has slowly but clearly been drifting away from Republicans since 1988, culminating (so far) in the 2012 election, when it had essentially the same vote margin as the nation as a whole. Based on a simple linear fit, I predict that in 2016, the democrat will get a higher margin in Virginia than nationwide, by about 2%, for the first time since FDR. That means that even if the Republican gets 51.00% of the national popular vote, they'd still lose Virginia.

[Update: the error bars show +/- one standard deviation from the linear model. In other words, if all my assumptions are true, there is a 68% chance that Virginia in 2016 will fall between the two error bars, and be somewhere between 0.8% and 3.3% more democratic than the country as a whole.]

I'm sure there are numerous expensive demographic studies out there detailing exactly why Virginia is drifting from right to center ... so does anyone want to trudge through them for me? My guesses at the moment:
1) growth in the DC suburbs and exburbs,
2) more generally, increasing urbanization throughout the state, and
3) a growing young and minority population, which is disproportionately happening in Virginia.

I hope to complete a similar analysis for all of the so-called "battleground" states, and calculate a prediction of the systematic advantage the democrat or republican has going into the election. I'd be so excited if the democrat won the electoral college but lost the popular vote; then we might finally see some bipartisan reform to the current broken electoral college system!

Population Density compared to Partisan Lean

 So a website called "the atlantic cities" has a very interesting article about how the closer you live with your neighbors, the more left-leaning you're likely to be, and vice versa. What really raises my ire about their article, however, is the absolutely terrible plot they made published, which is a scatter-plot of congressional districts. Someone just spat the data out into Excel and went with the first image they could make. They did change the y-axis to a logarithmic scale, but they didn't change the fit to to match it! Here's my (much better) plot with the same data (or more accurate? their sources are slightly unclear).

 As you can see (because I literally spelled it out at the bottom), there's a significant correlation between population density and partisan lean. Every time you double the population density, the district is about 8.6 points more democratic. I made a similar histogram-type plot of the districts, sorting them into bins by density and reporting the average partisan lean of all congressional districts with similar densities (circle size is related to how many districts that circle represents).

As you can see, most districts are clustered between 100 and 10,000 people per square mile. If you examine the two trendlines I've drawn, the blue one represents how we should expect to find the districts, and the red one shows how it actually bends significantly. This demonstrates the level of gerrymandering that republicans have accomplished, maximizing the number of districts that are republican, even if just slightly, while shoving all democrats in their states into a few districts that are heavily democratic. This is how Republicans currently control the House of Representatives, despite receiving several million fewer votes in the House than Democrats received. 

(For more on the current Gerrymandered state of many districts, try this fun jigsaw-style quiz! It shows very well how ridiculously these districts' shapes have been contorted to skew the makeup of the House so much)

A closer look at FiveThirtyEight's Presidential Election Simulator

I've always been a fan of the presidential election simulator they've developed over at FiveThirtyEight. Essentially they use a pretty complicated model to give a prediction of which candidate will win in which state, and then run some simulations to predict which candidate will win the overall election. It seems like most of the work goes into re-calibrating the results of polls and combining some pretty complicated factors that I won't go into here (since I'm not too familiar with their methodology on that level), but what they've done with that second part is what I'd like to suggest an improvement on. They use their current estimate about which candidate will win each state, then run simulations to predict the overall winner. The histogram from their website looks like this:

Based on my experience with simulations, you shouldn't normally have a few peaks much taller than the rest; this is a symptom of an illness I like to call Not-Enough-Simulations. If they just run more iterations, they'll get a much nicer and smoother curve. To demonstrate, I went through and grabbed their current prediction for the odds that Pres. Obama wins each state or district (for ME, NB, and DC), and just ran some simulations where for each state I randomly choose who wins, weighted by those odds, and tabulate the totals. I want to point out here that the model at 538 has some extra features which mine doesn't have, like how states don't vote independently, regional and economic influences, poll movements, and others, I'm sure. So this is purely for demonstrative purposes. Anyways, for three hundred simulated elections, this is what it looks like:

Does that look familiar? For reference, in my code that I just knocked together in a few minutes, that takes way less than one second. For one billion runs (which admittedly takes a few minutes), this is what I get:

It's a nearly perfect, beautiful bell curve. This represents the true result of their statistics. Another result of running more simulations is that the odds of who wins the overall elections is measured more accurately, and the number changes a little bit. Their "now-cast" function, which is what I'm actually mimicking, predicts what would happen if the election were held today. It gives Obama a 64.7% chance of winning. Instead, it should be a 64.2% chance, if you collect enough data. That's a small difference, but here's one more. They also calculate the chance of a 269-269 tie in the EC, which they give as 0.6%. But with less-noisy data, it's actually 1.5%, more than twice as likely.

It's a shame to see all the hard work the people at FiveThirtyEight put into their model at the state-level, just to have it under-sold with such a simple bug in their nation-wide model. They need more simulations!

Our broken Electoral System

When Americans elect a president every 4 years, the method we use is actually pretty strange when you stop to think about it:

1) Every state gets a number of votes equal to their number of representatives plus two. These are called "electoral votes".

2) 48 of the 50 states use a winner-takes-all system, where whichever presidential candidate gets the most votes in that state gets ALL the electoral votes of that state. The other two states use an adaptation of that method, where each candidate gets an electoral vote for each congressional district they win, plus two more for winning the overall state popular vote.

Electoral College for the year 2000

A notable side-effect of this policy is that someone can become President of the United States while losing the popular vote. This has happened 4 times out of 55 US presidential elections, or 7% of the time. Maybe that seems like an acceptably small fraction to you, but consider that there are also cases where it was very close to happening, like in 2004: Bush II had about 3,500,000 more nationwide votes than Kerry, but if 60,000 Bush voters had changed their minds and voted for Kerry in just one state (Ohio), he would have become the president. In the last 60 years, a "close" election like this, where fewer than 60,000 voters could've made the wrong man President, has come close to happening 6 times, meaning that 6/15 or 40% of recent elections were problematic.

For fun, I've taken the liberty of running some simulations. Each state is given its share of electoral votes as of the 2000 census, I specify the national popular vote totals and give each state its own vote total, normally distributed about the national mean, with a standard deviation taken from the last three presidential elections (about 11% each time). Then I check to see if the national popular vote winner is also the electoral college winner.

For an example election that's 48/52 (i.e. a 4% margin for one candidate), I ran this simulation 1,000,000 times, and here are the EV results:

We see that in more than 10% of the runs, the national popular vote winner does not become the president. Repeating this process for a collection of margins, I find the probability of the "wrong president" vs. national popular vote margin:

I also show the last eight elections as vertical lines on the bottom, highlighting in red the one that gave us the "wrong" person. Statistically speaking, we should have seen on average 1.3 "wrong" presidents in the past 8 elections. Reality, however, is constrained to integers in this case, so it's really no huge anomaly that we got 1 error out of 8. What's surprising to me is how astonishingly poor this system is at electing the popular vote winner to the presidency. With a national popular vote margin of 4% we get an error of 10%. With a margin of 1% we get an error of 37%. For margins smaller than 1% we may as well flip a coin, even though 1% represents more than 3,000,000 Americans.

Raw data is tabulated below. For reference, the margins of the last 8 elections ranged between 0.5% and 10%. The real miracle here is that we have had only four wrongly-elected presidents out of 55!

Popular Vote Margin	Probability of Wrong President
10%	0.06%
8%	0.52%
6%	2.7%
4%	10.4%
2%	26%
1%	37%
0.4%	44%
0.2%	47%