My 2020 Predictions, Graded

This post is a little bit overdue, but I’ve been looking forward to writing it. In December 2019, I made 15 predictions for 2020. I was inspired by two sources. First, Scott Alexander does yearly predictions with end-of-year grading - all plotted on a calibration curve. Scott inspired me to do the same. The second source of inspiration is all the end-of-year predictions that say a lot, but mean nothing. For example, “Ransomware will continue to be a problem.” Yeah. It’s obvious, but it’s also so vague that one can never be wrong. I want to do better.

I put a twist on my predictions. I wrote them to be measurable and completely gradable, after the fact - just like Scott Alexander. They pass The Clairvoyant Test (or, The Biff Test, if you please.) More importantly, I put my money where my mouth is.

It’s well-known in the field of expert judgment and forecasting that people make better predictions if they have something on the line. It’s why the Equivalent Bet Test works. I committed to donating $10 for every 10th of a decimal point I’m off from being perfectly calibrated. See the 2020 Prediction post for more info on the methodology.

How did I do?

I think I did just ok.

8 out of my 15 predictions came true - slightly more than half.
Based on my confidence levels, I believed I would get 11 right
In this set of forecasts, I am overconfident. People that are overconfident believe they are right more often than they actually are.
EFF will get $400 of my money as a donation
I have learned that yearly predictions, especially of macro trends, are very difficult. While a global pandemic should have been on everyone’s radar, predicting it a year before it happens is hard. COVID-19 tanked some of my forecasts.

Without further delay…

Facebook will ban political ads in 2020, similar to Twitter’s 2019 ban.
Confidence: 50%
Assessment: Wrong
Notes: Facebook did ban political ads, but not like Twitter, which was the benchmark. Facebook waited until after the November election to ban ads.
By December 31, 2020 none of the 12 Russian military intelligence officers indicted by a US federal grand jury for interference in the 2016 elections will be arrested.
Confidence: 90%
Assessment: Right
Notes: All 12 intelligence officers are still on the FBI’s Most Wanted list.
The Jabberzeus Subjects – the group behind the Zeus malware massive cyber fraud scheme – will remain at-large and on the FBI’s Cyber Most Wanted list by the close of 2020.
Confidence: 90%
Assessment: Right
Notes: They are still on the FBI’s Most Wanted list.
The total number of reported US data breaches in 2020 will not be greater than the number of reported US data breaches in 2019. This will be measured by doing a Privacy Rights Clearinghouse data breach occurrence count.
Confidence: 70%
Assessment: Right
Notes: It feels like data breaches get exponentially worse year after year, but it’s not. I think we see numbers ebb and flow but with a general upward trend as the number of connected systems and the sheer number of records increase. 2019 was exceptionally bad, so it was reasonable to think that 2020 would be better.

The Privacy Rights Clearinghouse, unfortunately, doesn’t seem to be updating data breach numbers anymore so I took the numbers from the Identity Theft Resource Center.

2019 had 1,362 reported data breaches and 2020 had 1,108.
The total number of records exposed in reported data breaches in the US in 2020 will not exceed those in 2019. This will be measured by adding up records exposed in the Privacy Rights Clearinghouse data breach database. Only confirmed record counts will apply; breaches tagged as “unknown” record counts will be skipped.
Confidence: 80%
Assessment: Right
Notes: Same reasoning as #6. 2019 had 887,286,658 records exposed and 2020 had 300,562,519 according to the Identity Theft Resource Center.
One or more companies in the Fortune Top 10 list will not experience a reported data breach by December 31, 2020.
Confidence: 80%
Assessment: Right
Notes: Several companies on the list did not have a data breach.
The 2020 Verizon Data Breach Investigations Report will report more breaches caused by state-sponsored or nation state-affiliated actors than in 2019. The percentage must exceed 23% - the 2019 number.
Confidence: 80%
Assessment: Wrong
Notes: Nope, way less than 23%.
By December 31, 2020 two or more news articles, blog posts or security vendors will declare 2020 the “Year of the Data Breach.”
Confidence: 90%
Assessment: Right
Notes: This was kind of an inside joke to myself. Regular readers know that I like poking fun at marketing hyperbole and orgs using FUD to sell products. Every year since 2005 has been declared the “Year of the Data Breach” by a blogger, journalist, security vendor, etc. It seems to me that one year should be the “Year of the Data Breach,” not every year. The phrase means nothing now. I wrote about this here: Will the Real “Year of the Data Breach” Please Stand Up?”

Sure enough, none other than the Harvard Business Review declared 2020 the Year of the Data Breach. LOL.
Congress will not pass a federal data breach law by the end of 2020.
Confidence: 90%
Assessment: Right
Notes: Did not happen
By midnight on Wednesday, November 4th 2020 (the day after Election Day), the loser in the Presidential race will not have conceded to the victor specifically because of suspicions or allegations related to election hacking, electoral fraud, tampering, and/or vote-rigging.
Confidence: 60%
Assessment: Right
Notes: I’m not some forecasting genius because I got this right. Trump has been saying this since 2016.
Donald Trump will express skepticism about the Earth being round and/or come out in outright support of the Flat Earth movement. It must be directly from him (e.g. tweet, rally speech, hot mic) - cannot be hearsay.
Confidence: 60%
Assessment: Wrong
Notes: Really though, why not? Trump would say or do anything to pander for votes.
Donald Trump will win the 2020 election.
Confidence: 80%
Assessment: Wrong
Notes: Without the pandemic and botched response, I think he would have won.
I will submit a talk to RSA 2021 and it will be accepted. (I will know by November 2020).
Confidence: 50%
Assessment: Wrong
Notes: The pandemic sapped my will to do many things extracurricular, not to mention that RSA as a conference has lost a lot of its appeal to me. I didn’t even submit.
On or before March 31, 2020, Carrie Lam will not be Chief Executive of Hong Kong.
Confidence: 60%
Assessment: Wrong
Notes: Back in 2019, I thought Hong Kong independence would be THE BIG story for 2020. I was wildly wrong.
By December 31, 2020 the National Bureau of Economic Research (NBER) will not have declared that the US is in recession.
Confidence: 70%
Assessment: Wrong
Notes: I was wrong.

Final Thoughts

This was really fun and I think I’ll do it again. I didn’t do any 2021 - I had too much going on in December to even think about this. If you have been through calibration training or perform forecasts on any level as part of your job, I think you should try this - even if you keep the results to yourself. It will help you improve your estimations and forecasts in the long run.