Preliminary analysis

Here is a table of win-loss percentage for home teams coming off a bye (or extended break) since 2002. I’ve also added the long-term winning percentage, along with the standard error of the mean due to finite data.

It seems like on the surface there something to this. If either team has the bye, the winning % of the home team is inflated, way above the normal long term average. But is this really the case? How do we know whether the higher win percentage is real or if it’s due to chance or if it can be attributed to other external factors? Perhaps every time this happens, a top 8 side plays a bottom 8 side and it’s just an artifact of scheduling.

Method

In statistics, these are additional explanations are called “confounding factors”. If we are to truly understand the impact that the bye has on winning / losing, then we need to control for these factors. And the way to do this is through a “case-matched controlled” study. This is a statistical technique that is often used to assess the impact of new drugs on a population. It goes like this:

1. Pick a sick person and make them take the drug
2. Pick another sick person who is really, really similar to the first person (in age, gender, income status etc.), but don’t make them take the drug.
3. Do this many times until you have a number of pairs.
4. Then compare the outcomes in the drug and non-drug populations.

Now you have successfully accounted for factors like age, gender, and income status when looking at the effect of the drug.

Similarly, we can do the same thing with the bye-games:

1. Pick a bye-game (a match where one team is coming off an extended break due to a bye and the other isn’t)
2. Pick a non-bye game which is really similar to that first bye game, creating a pair.
3. Find a matching pair for all the other bye games
4. Run win-loss analysis on the resulting population

So what do we mean by “similar”? By similar, we usually mean trying to match other characteristics that could potentially explain the outcome of the test. In this case, we are trying to match game characteristics that explain win-loss probability. If some of you follow me on twitter or have seen some of my posts before, I’m super into predictive modeling of AFL outcomes based on historical performance. Through some of this work, the three most predictive characteristics of an AFL match outcome that should be matched are

1. The two teams’ difference in form leading up to the match,
2. The two teams’ difference in the experience they’ve had at a ground they are playing at (measured as average margin at that particular venue over the last 10 times played), and, most importantly
3. The two teams’ difference in their ELO rating.

A quick note about ELO - an ELO rating is a way to rank each team adjusting for the strength of the opposition they’ve played. You can find a comprehensive explanation here

Data gathering

So now that we’ve got the methodology out of the way, let’s implement it.

In the 2000s, there have been roughly 196 Bye affected matches. For each of these matches, I searched the history of AFL matches (from 2002 onwards) to find a similar game, matching form, venue experience, and ELO difference, but that did not have either team affected by the bye. Here is an example

The bye round match was between Essendon and Richmond in 2014 at the MCG. Both teams were coming off similar form line, with the home team, Essendon, having a +30 ave score differential at the MCG, but an overall -33 ELO differential. It’s paired match, Adelaide vs Sydney in 2016, was very similar, with both teams having a similar form, venue experience and ELO difference coming into the match.

I repeated this for all 196 bye matches. For all, I was able to match form and venue experience within a small tolerance. However, I couldn’t perfectly match ELO difference, because it’s such a wide scale. So, after perfectly matching form and venue experience, I simply took the match with the smallest ELO difference and used more statistics to account for it, explained below.

Analysis

Now that I have 392 matches, corresponding to 196 bye-matches and 196 pairs, its time to do our performance comparison.

I mentioned before that because I couldn’t perfectly match ELO, I just minimised the difference in ELO between the bye-match and its pair. That’s fine, but it does mean that our bye-rounds aren’t perfectly matched.

To get around this, we need to perform a regression analysis. In a regression analysis, you basically do the matching we did before, but fill in the gaps for imperfectly matched pairs by fitting a linear equation to describe the statistical relationship between our variables (ELO and bye round) and the response variable (winning / losing margin). By inspecting this equation, you can then figure out what the statistical effect of your confounding factors are independent of the other confounding factors, which is exactly what we want.

The linear equation our regression analysis will work off is:

Home Team Winning Margin = A * home_bye_match(y/n) + B * away_bye_match(y/n) + C* ELO_Difference + D

This equation will try to predict the home team winning margin based on 4 things: whether it was a home bye match, an away bye match, the difference in ELO between the two teams and a constant, D (which is actually the home-ground advantage). The regression analysis boils down to finding the numerical values of A, B, C and D that best describes the data.

Importantly, we can use this equation to understand the effect of a bye-round in the following manner.

1. If A and B are 0, then the bye-rounds have no effect

2. If either A and B are non-zero, then the bye-rounds have an effect

​

Knowing this, I ran the regression analysis on the population of matched pairs to understand the statistical effect of the bye, corrected for ELO effect (remember, we have already corrected for form and venue experience by carefully selecting our population). These are the results:

​

We can see that C and D are non-zero, even conservatively. That is, their lower bounds and upper bounds are both above 0. This means that the effect of ELO and home-ground advantage are statistically real, even in the worst case. In this case, home-ground advantage (D) is, on average, worth about two goals, and every ELO point difference is worth about 0.17 points to the home team.

On the other hand, A and B are very close to zero, with their upper and lower bounds straddling zero. What this means is that there is no evidence in the data to suggest that the bye-rounds have an appreciable effect on winning / losing. More precisely, the fact that there is a bye round contributes zero points to the overall margin.

Conclusion

Why are we seeing a difference in overall winning percentage for bye rounds? Well, after accounting for home ground advantage (D) and ELO rating (B) (and venue experience and form line through populution selection), no direct link was found between the home (A) or away (B) team having a bye in the previous round and the result of the game. However, there was a direct link suggesting that the results can be partiaully explained by the ELO difference between teams. What this means is that the inflated results may be due to a scheduling anomaly where, in general, stronger teams tend to play weaker teams at home after the bye and that the observed phenomena has nothing to do with the bye itself.