Last spring I introduced a tempo free predictions model and last fall I predicted the 2010-2011 season at Yet Another Basketball Blog. Then for the Basketball Prospectus Book, Ken Pomeroy introduced a similar model. We used the same underlying data, the minutes and possession weighted tempo free statistics of various players. As a result, you’ll see we made basically the exact same errors in our predictions. For example, we both picked Arizona St. second in the Pac-10 and Baylor and Kansas St. in the top three in the Big 12.

Where we differed is in how much weight we put on elite recruits.

Ken took a more principled approach based on the on-court performance of elite recruits.

I looked at the team data and I saw that elite recruits have had a very limited impact on their teams success the last few years. So I put less weight on those players. (Recall Lance Stephenson as an example.) 

So far this year, Ken Pomeroy is winning that bet. Both Kentucky and North Carolina have played great thanks to their heavily hyped freshman (such as Terrence Jones, Brandon Knight and Harrison Barnes). But do not assume he gets a TKO in that category. While Kentucky and North Carolina look great, by weighting recruits less heavily, my ACC predictions were a little more pessimistic across the board. This is not to say that I saw the ACC’s fall from grace coming, but my model highly doubted that Sidney Lowe was suddenly going to take a team of freshman to the NCAA tournament. But I think on the whole, Ken came closer to hitting the mark with elite recruits. And having more historical data should help me refine my model this summer.

But I have a more important change planned for this summer. A few weeks ago I posted a long discussion on North Carolina and Wisconsin to make another point. I do not believe we should be predicting each team using the same model. While North Carolina depends heavily on elite recruits in order to build a dominant team, elite recruits are almost irrelevant to Wisconsin’s success. The Badgers have even red-shirted McDonald’s All-Americans. This summer, I hope to use some more historical data to classify how various coaches build winning teams. And I hope that by using different models for different types of coaches, I will become the first analyst to not under-rate Wisconsin every year.

But with the predictions having been posted, and the season mostly complete, today I want to look back at what I got right, and what surprised me this season.

SEC Prediction

EM = Efficiency Margin, calculated as Adjusted Offense minus Adjusted Defense

What I got right:

Take a minute to enjoy my projected SEC standings. You will notice my model picked Florida in first and Vanderbilt in second, and Alabama winning the SEC West. And that is exactly the way the SEC standings have panned out.

Surprises:

But despite the fact that my projected standings look fairly accurate, I actually missed on Kentucky big time. My model thought with a historic rate of attrition, and no Enes Kanter, that Kentucky would struggle mightily. Instead the Wildcats have posted the best margin-of-victory numbers in the conference, if not the best record. The key is that Kentucky’s offense is even better than last season when the team had a superb upperclassman in Patrick Patterson. If anyone really thought Kentucky would have a better offense than last season, they deserve a job on the Psychic Hotline.

Also, Vanderbilt has improved more than expected thanks to Festus Ezeli practically duplicating AJ Ogilvy’s numbers, while Brad Tinsley has learned to make threes. And don’t forget that John Jenkins has become an SEC player of the year candidate by adding driving ability to his great outside shooting.

Flops:

Florida’s offense is actually a little worse than I expected. No one is terrible, but no one has improved substantially either. While Vernon Macklin’s role in the offense has increased, his terrible free throw shooting remains a liability. Similarly, Kenny Boynton still takes way too many threes for a below average three-point shooter.

Auburn did not bring in any of its high profile recruits due to ineligibility issues and the offense has struggled mightily, finishing about 10 points worse than expected. Luckily Tony Barbee’s defense has matched my prediction.

As a whole, the SEC West did even worse in the non-conference schedule than anyone could have predicted. It was a brutal November and December.

Big Ten Prediction

Note: I adjusted Purdue’s prediction after the loss of Robbie Hummel.

What I got right:

I had Ohio St. as Big Ten champ. Even when Robbie Hummel was healthy and everyone was picking Michigan St. and Purdue to win the Big Ten title, the tempo free statistics pointed emphatically at Ohio St. The team returned four incredibly efficient starters, and brought in a very talented recruiting class. (Am I the only one who thinks that if DeShaun Thomas played for another team, he would be one of the top-10 freshman in the nation?) Ohio St. looks like the obvious pick now, but this was going out on a limb back in October.

Surprises:

Jordan Taylor is actually 20 points more efficient than last season, posting an ORtg of 130 relative to 110 last year. That’s a historic improvement, that no model could have seen coming. Wisconsin’s defense is actually a little bit worse than last year, but maybe that is because they have been in so many blowout wins. It is hard not to let up defensively when you are up by 15 or 20.

JaJuan Johnson’s seven-point improvement seems rather pedestrian in comparison, although Johnson deserves credit for upping his ORtg while also increasing his usage rate from 25% to about 29%. But I think the real key to the Purdue jump in offensive efficiency is Lewis Jackson. His ability to get into the lane against great defenders like Aaron Craft and Jordan Taylor makes him the true third scoring option on Purdue. Perhaps we should have seen Jackson’s improvement coming since he was injured last year. But the jump from an ORtg of 93 to 111 is still phenomenal.

Finally, as Tim Doyle likes to say, “the Butterfly” emerged for Michigan. Darius Morris went from an ORtg of 89 to 110, and his development, along with the play of the Michigan freshmen, has been highly unexpected.

Flops

Even though I picked Michigan St. lower than many experts, even my model did not predict such a precipitous fall from grace.  On the one hand, you can point to the injuries and mid-season transfers, but they do not quite explain the big picture either.  All of Michigan St.’s woes tend to be minor. Durrell Summers has been in a horrific slump, but he is not light year’s behind last year’s pace.  Korie Lucious was struggling mightily before he left, but we never an elite three point shooter.  Perhaps the worst thing you can say about Michigan St. this year is that no one has gotten better.  And while the freshman Adreian Payne and Keith Appling look promising at times, neither is consistently good.

Pac 10 Prediction

What I got right:

Washington has the best efficiency margin in the Pac-10. Washington was a popular preseason pick, and even though they are probably not going to win the Pac-10 title, that does not mean the prediction was wrong. In fact, the Huskies are actually substantially better offensively than my model predicted thanks to Isaiah Thomas improving his assist rate by 10 points and finding Mathew Bryant-Amaning inside and Justin Holiday from three point range much more effectively.

I also predicted Arizona would be the most improved team in the league, but even I did not expect them to win the league title.

Surprises:

UCLA has also improved substantially more than I expected, and it is almost all based upon defensive improvement. That should not be a huge surprise given the team’s head coach, but you cannot overlook the impact freshman Josh Smith has had defensively on the interior. Nikola Dragovic was just not a defensive post player, but he often had to guard opposing forwards after Drew Gordon and J’mison Morgan transferred last year. And with Smith replacing Dragovic as an interior defender, Reeves Nelson and Tyler Honeycutt have been freed up to become defensive stoppers.

Flops:

Herb Sendek is a system coach, and Arizona St. brought back some talented scorers like Rihards Kuksiks. But his season has been the inverse of John Beilein’s season. Sendek’s young team has not come together, and despite trying 12 different players in the rotation, no one has worked out. Arizona St. shoots worse across the board, and the team also allows opponents to make a much higher percentage of shots than any season under Sendek. My model went out on a limb for Sendek based on his recent performance (when most preseason publications did not) and it was very wrong.

MWC Prediction

What I got right:

In the preseason, everyone was on San Diego St. because of all their returning seniors, but my model liked BYU because they brought back the efficiency superstars.

I also predicted that Colorado St. and Air Force would improve more than any teams in the MWC this season. And both did even better than my model predicted. Air Force took the bigger leap forward, but Colorado St. took the leap into the NCAA bubble picture which was a more significant leap.

Surprises and Flops

New Mexico is in 5^th place in the league standings, but has the 4^th place efficiency margin, meaning I perfectly nailed the efficiency order for the top 5 in the league. What I missed was that the entire MWC was much more dominant than expected. Even last place TCU has performed better than I expected, despite winning one conference game at this point. How did the league improve from 8^th to 6^th in the Pomeroy Rankings and to 4^th in the RPI?

Does November predict March?

I’m starting to wonder if there might be a dirty little flaw in the RPI, the Pomeroy rankings, and the Sagarin ratings. The problem is that we all knew heading into the season that the Big Ten and MWC returned more talent than just about any league. And because of that experience, those leagues dominated the non-conference schedule.

But does that mean those leagues are more likely to win in March? Or should we expect the less experienced leagues to grow more as the season unfolds? If you watch Jorge Gutierrez relentlessly taking the ball to the basket for Cal this year, you have to wonder if his team should be punished because Cal and the rest of the Pac-10 were so young early in the season.

But this is the sad reality. While California and USC are on almost no one’s bubble radar screen, Illinois and Michigan St. are firmly in the discussion. Is that right? Are we that confident California is substantially worse than Illinois at this point?

Perhaps this explains why people with Sagarin and Pomeroy based brackets never seem to win their pools. People who pick dominant champions in underrated conferences often pick the critical upsets. (See Washington in last year’s NCAA tournament.)

The margin-of-victory numbers are a phenomenal predictive tool, but when comparing conferences that have not played much in the last two months, the accuracy of conference comparisons begins to fade. I am quite confident the Big Ten was the best league in the nation in November and December. And I am quite confident Illinois is better than Iowa in league play. But when it comes to comparing how the Big Ten and Pac-10 will do once the field of 68 is announced, nothing will surprise me. And I guess that is why the NCAA tournament is so great. As much as we think we know, when teams mix it up for the first time in a one-and-done setting, surprises will happen.