[ Part 1 | Part 2 | Part 3 | Part 4 ]

 

I was quite excited for what the Cleveland Browns might do in this year’s draft. I watched a lot more baseball in the 1990s and 2000s, and was well aware of (and paid close attention to) Billy Beane’s work with the Oakland A’s. (I also haven’t forgotten how old-school scouts derided it as nerd nonsense by people who had never played the game and didn’t understand the arcane complexities of their sport. That’ll come up later.)

As someone who believes the NFL, both on a league-wide and on the individual team level, is in many ways run by backward, ossified processes that seem to have all the scientific rigor of bloodletting or the Ptolemaic system, I was intrigued by the thought of a team applying real analysis and big data to their front-office processes. This draft was the first chance to see the new front office in action.

I’ll start with what I liked. Their overarching strategy was a sound one that I think took advantage of one of the biggest inefficiencies in other teams’ approach to the draft: Namely, simply not valuing picks enough, not valuing the fact that starter-level talent that can be found cheaply in the first few rounds of the draft. The Browns acquired a huge stockpile of picks, and not just for this year: The team already possesses an extra first- and second-round pick in 2017 and an extra second-round pick in 2018, and that’s after using fourteen selections in the 2016 draft, including six top-100 selections. Clearly, management takes seriously the idea that the draft is the best resource for finding talent, that acquiring the maximum number of picks is the best way to alleviate variance, that draft picks as a whole are underrated by the league, and that the team is in need of a complete talent overhaul.

Now, it’s that last part that gets tricky.

While the overall strategy the Browns use was solid, it still doesn’t matter if you don’t use those picks wisely. After all, previous management teams had multiple first-round picks in multiple years.

(As a side note, It’s hard to say which year was worse: 2012, which involved a 4-for-1 trade to move up one spot for a running back, as well as drafting a 28-year-old quarterback even though his production was almost entirely due to the receiver drafted #5 overall; or 2014, where in one of the most talent-laden drafts in recent memory, the team again traded up one spot, this time for a disinterested athlete playing cornerback, and then traded up again later in the round for an undersized quarterback with an enormous sense of entitlement and personal problems ranging from substance abuse to domestic violence.)

It’s how the team used those picks that raises some questions. This draft wasn’t an unmitigated disaster like the drafts under Mike Holmgren or Ray Farmer, but the decisions made were a mixed bag. (Side note again on Holmgren: When a really good coach acquires GM power and then is so bad at that job his old employers mandate he relinquish those powers or be fired altogether, it’s probably a bad idea to hire him exclusively as your GM. This lesson can be applied to other industries, as well.)

From the overall analysis of the situation, the Browns’ draft strategy seemed to heavily emphasize college production, with certain analytical metrics possibly factoring in. This approach can result in a mixed bag for the simple reason that the factors that play into college production are a mixed bag, and are not necessarily indicative of the traits that lead to NFL success. And, of course, an analytical method is only good if it’s proven to work.

Let’s start with the positive. I liked the receivers on the whole. Corey Coleman is not only a metrics star, he’s a legitimate big-time playmaker on film as well. Fifth-round picks Jordan Payton and Rashard Higgins had productive college careers and traits that suggest they should be solid complements to Coleman as they develop. Receivers generally take a leap in year three; this longer development curve makes drafting many of them at the start of a rebuilding effort a more sound strategy than it seems on the surface.

The other two guys, however, I’d not heard a peep about.

Ricardo Louis gets little mention anywhere I’ve read, with a paragraph in Dane Brugler’s guide, and no mention at all in Matt Waldman’s. CBS Sports has him ranked as a seventh-round prospect. He caught 46 passes for 716 yards last season. I’m not sure what metrics the Browns used to evaluate him. Whoever spoke on behalf of the team’s drafting made mention that they believed he could be a big-play threat, but I don’t know any more than that, and I have’t seen enough of him player to get a real answer on that.

I have no idea who Seth Devalve is. He was announced as a wide receiver, but some sites list him as a tight end. The Browns seem to think he can be the kind of “move” tight end who’s a mismatch as a receiver a la Jimmy Graham or Travis Kelce. That’s a great value if true, but I can’t find anything that says it’s true.

Maybe it’s lack of information on my part, but I can’t find anyone who seems to know anything about him. CBS Sports lists one thousand prospects– the draft this year was 253 selections– and Devalve wasn’t among them.

It’s hard to believe they needed to use a fourth-round pick on them. Then again, maybe the NFL teams have information on him we don’t, and the Browns were right. That one I just can’t evaluate, but it’s a shock. If anyone whose work I trust finds tape of Seth Devalve and writes about it, I’ll be sure to take notice, but until then, only time will tell. (UPDATE: I saw on Twitter Tuesday that Emory Hunt of Football Gameplan, one of the most knowledgeable scouts of small-school and non-FBS prospects, is a big fan of Devalve, which makes me think the pick has some merit. You can watch Hunt’s scouting report on Devalve here.)

On to other positions. The most intriguing selection by the Browns, in terms of identifying their approach, is Emmanuel Ogbah. It’s intriguing because opinions are split on Ogbah, and I think this may reveal more about the Browns’ process.

Ogbah had college production– 22.5 sacks his last two seasons. Ogbah grades well by certain athletic and analytical measurements– both SPARQ and Football Outsiders’ SackSEER consider him a quality prospect.

But watching his film reveals a very inconsistent player with a limited set of moves. And judging his athleticism by Justis Mosqueada’s Force Players metric says Ogbah has a fast 40 and good burst but his poor three-cone and short-shuttle times indicate little aptitude for change of direction– in other words, he’s good at running fast in a straight line for his size, and that’s about it. That’s not enough to consistently create pressure in the NFL.

Compare the MockDraftable.com athletic “spiderwebs” for Ogbah to those for Shaq Lawson, the draft’s only “Force Player.” The differences may seem slight when looking at the raw numbers, but the differences between these two spiderwebs are the differences between being a first-round prospect and not.

Time will tell who’s right on this one. Right now, I lean toward the latter group, because I place film above all else– and because I’m a believer in the Force Players metric after watching it predict the early success of Preston Smith and Danielle Hunter last year. (More so than SackSEER, anyway, whose track record is decidedly more mixed.)

Carl Nassib is another pass rusher who was productive in college but has somewhat pedestrian athleticism. In the third round, though, I mind this selection less; productive and athletic probably aren’t going to happen, so you hope a guy like Nassib can be your second or even third pass rusher. Still, though, this is the one position where I’d actually prefer downplaying college production and looking for the athletic talent that correlates better with success. (Sometimes you find a complementary rusher and hard worker going with the college numbers in the third round; sometimes, you find Damontre Moore.)

Scooby Wright is hard to hate on as a selection because he was the fourth-to-last pick of the draft, but he really shouldn’t have gone any higher than that, considering not only his injury history but his extremely poor athleticism. I suppose it’s a matter of preference as to which you prefer at this point, but again, this seems like a place I’d rather take a flier on someone who has more athletic ability– they’ll probably only be special teams players anyway.

I thought Shon Coleman and Spencer Drango were good selections where they were drafted, and could fill in some of the holes on the roster long-term (presumably, with Coleman at right tackle and Drango at guard; a Thomas-Bitonio-Erving-Drango-Coleman line seems like it would be solid for a while, assuming moving Erving back to center restores his competence.)

And then there’s Cody Kessler.

This one I don’t get at all.

While I don’t think Football Outsiders’ Lewin Career Forecast is particularly useful, I do think that completion percentage and starting experience are valuable statistical benchmarks to look at when assessing a quarterback’s NFL prospects. But there are no hard-and-fast numbers; everything requires context. (That’s why people like Ian Wharton develop metrics like accuracy percentage, to try to isolate the percentage with which a quarterback accurately found his receiver, regardless of whether the receiver caught it.)

My favorite example of context for quarterbacks is as follows: Chase Daniel and Blaine Gabbert played quarterback at the same school for the same coach in the same system. One of them was a top-ten pick, and one of them went undrafted. Click those links and look at their statistics, then tell me which is which.

(I’m not saying Daniel deserved to go higher, either. Everything is context– those numbers are inflated by Missouri’s spread system. I am saying that Gabbert’s performance being so much worse than Daniel’s should have been a big red flag, no matter how handsome and tall he was.)

Back to Kessler. He posted a high completion percentage in his three years starting at USC, but that number seems to have been substantially aided by having a series of superior receivers to throw to, from Marqise Lee to Nelson Agholor to JuJu Smith-Schuster, currently projected as a top-5 selection in Walter Football’s most recent 2017 mock draft.

Kessler is superficially accurate but seems to lack many of the other traits that are necessary in an NFL quarterback. He can’t deliver all the throws. He stares down receivers. Cian Fahey can tell you more.

Ironically, where most teams choosing a quarterback mistakenly focus on traits that look amazing in workouts (but don’t show up on film or in performance), the Browns seem to have gone to the other extreme: taking a quarterback who’s outstanding based on certain metrics, but whose performance on film doesn’t match those metrics, who may have other contextual explanations for those metrics that don’t have anything to do with his talent.

Or maybe it’s neither, and Hue Jackson told Paul DiPodesta and Sashi Brown the same thing he told the media: “You’ve got to trust me on this one.”

The most disappointing part is, there was a quarterback available who is immensely talented and whom I would have loved to see Jackson get the chance to work with and develop. But Cardale Jones is now a Buffalo Bill.

Last thoughts: I don’t know who said it originally, but this line on Twitter stood out to me: “The Browns drafted like their entire ‘analytics department’ was some college kid who spent $10 on a Pro Football Focus subscription.”

The next day, PFF’s draft grades came out, and only two teams got an A: the Jaguars and the Browns.

It’s funny, sure, but I want to talk about PFF’s grading a bit, as well as some more general thoughts on draft analysis.

The thing about PFF’s “grade system” is that it isn’t analytics. It’s scouting. It just tackles scouting by assigning a number value to each play and adding up the sum total. And those numbers are decidedly subjective.

PFF does chart some other useful statistics, or at least they did until their new revamp which took many of their stats out of public view. (I quite liked their “pass rush productivity” stat, which incorporates QB hurries and hits as well as sacks.)

But PFF’s methodology leads to blind spots in their analysis. Not knowing the responsibilities of certain players on certain plays leads to an amount of guesswork. And in their evaluation of college prospects, it seems like they have the same weakness as the Browns, overvaluing production as opposed to traits that are necessary to win at the next level.

No method of analysis is perfect; every method has biases and flaws. If you are wedded to one, your analysis (and, if you have one, your team) will contain those flaws as well. Since I’m not possessed of the kind of time to grind tape endlessly nor the acumen to build my own analytics methods, I try to draw from as many sources as possible. When it comes to the NFL draft, I put an emphasis on methods that rely on scouting tape, looking for traits that translate to successful play at the next level, while also considering metrics for production and athleticism that have proven to some degree to correlate to NFL success.

I want to see an analytical movement, one that takes advantage of the NFL’s many blind spots and weaknesses in evaluation, succeed. I am worried the Browns have gone so far in the other direction that they too are neglecting trait-based scouting. Caring about a prospect’s workout numbers instead of the film that puts them in context is folly. But the same may be said for a prospect’s production numbers, also.