Hey programmer types

[DJ's left nut]

How hard would it be to build a 'mock probability' simulator?

Pest has been taking guys at certain spots we know would be available later in that mock but not terribly likely in real life. We're trying to reverse reality.

So what you would do is you would select a team, put in who you would take at each round, the sim would run like 100 mock drafts and spit out the odds at each selection of that player being available to you based on the mock drafts.

So lets say you select the first four rounds for the Chiefs and enter:

Ojabo
Travis Jones
Sam Williams
Tolbert
Greg Dulcich
Troy Anderson

And then tell it to run

And it spits out something like:

Ojabo (60%)
Jones (60%)
Williams (80%)
Tolbert (70%)
Dulcich (80%)
Anderson (90%)

Then it gives you a combined probability result to determine your 'score'.

And you can do whatever you want. If you decide you're going to put Aiden Hutchsinon to the Chiefs in the 4th round, you can do that. It's just going to give you a 0% chance for that pick and bomb out your whole draft grade if you do it.

I think it would be a damn neat tool to incorporate into some of these mock simulations. And hell, it could almost have some commercial potential if teams or even guys like McShay that want to put out these mocks every week utilized it to start to come up with more holistic outcomes.

I almost wonder if CD's draft simulator within Sandbox might be a viable starting point. But I don't know that it has an actual AI of any sort - it just defaults to draft order.

[duncan_idaho]

Seems do-able.

I don't have the programming chops to pull that off.

Potential complications:

1) What are you using as your "consensus" core big board? NFL Mock Draft Database runs a good one that is an aggregated table based on all the pro mocks in their database. That one is slow to update, though, as rapid changes in someone's status due to a great combine or pro day or (or the reverese, or an injury, or an off-field event) still take some time to "catch up." For example, of the mock sites I use, this is the one where I'm least likely to be able to nab Ojabo. Because he's fallen from the top 10 to the teens over the past 10-14 days, but all those mocks with EARLY projections are still in the database.

I'd almost think you want to limit the mock database that's creating your consensus big board to the past 2-4 weeeks. And I don't know of one that exists publicly and works that way.

2. How do you weigh team needs vs. positional value?

3. How do you account for the "depth" of the draft? i.e. in a year like 2013 it would be easier to move around in the draft because that one was a big ol' pile or crap. But in a deep draft like this, it may be harder/teams may be more likely to stick exactly to pick value in trades.

[kepp]

The logic behind it wouldn't be difficult to code. It's the underlying data that needs to carefully considered. You need a source or truth...or as close to a source of truth as you can get. Otherwise you're just randomly selecting. You'd need a source of truth for player rankings, team needs, front office/GM draft tendencies...the list goes on. All this is highly subjective.

The only subjective part in the code would be the formula to calculate the probability of each pick because you would have to decide the weight given to each data point in the overall calculation. The rest of the code would just be cycling through the rounds applying that formula.

[The Franchise]

Originally Posted by kepp

The logic behind it wouldn't be difficult to code. It's the underlying data that needs to carefully considered. You need a source or truth...or as close to a source of truth as you can get. Otherwise you're just randomly selecting. You'd need a source of truth for player rankings, team needs, front office/GM draft tendencies...the list goes on. All this is highly subjective.

The only subjective part in the code would be the formula to calculate the probability of each pick because you would have to decide the weight given to each data point in the overall calculation. The rest of the code would just be cycling through the rounds applying that formula.

This is also why using these mock tools are a crapshoot.

GMs make stupid decisions. GMs also don't just draft need or make ridiculous trade demands all the time. I've seen mocks where Jermaine Johnson falls to 29. I've also seen mocks where teams throw a bunch of picks to move up and take Jermaine Johnson at 8 or 9.

[SAGA45]

Simulators are fun. Its cool to get results like this - https://www.facebook.com/12798964724...7895035921062/

Even the though probabilities of things falling that way are minimal to none, it leaves room for those unforeseeable draft moments - players being picked much earlier or later than expected.

Honestly, once you get out of the top 10, anything can happen. That is typically when all the mocks go to sh*t. Happens every year without fail.

So a good program definitely has to account for some "believable randomness".

[DJ's left nut]

Originally Posted by duncan_idaho

1) What are you using as your "consensus" core big board? NFL Mock Draft Database runs a good one that is an aggregated table based on all the pro mocks in their database. That one is slow to update, though, as rapid changes in someone's status due to a great combine or pro day or (or the reverese, or an injury, or an off-field event) still take some time to "catch up." For example, of the mock sites I use, this is the one where I'm least likely to be able to nab Ojabo. Because he's fallen from the top 10 to the teens over the past 10-14 days, but all those mocks with EARLY projections are still in the database.

Yeah, that would be an issue. I would a lot of the machine logic stuff that exists in present mock-draft websites would be paramount. Then you could try to create a running average of 3 or 4 different 'trusted' sites, but you'd need their source data and I doubt they'd be open with all of it.

But someplace like The Draft Network has a handful of 'big boards' and while they don't update them constantly, an individual could. I just don't no enough about the mechanics of how these things actually function to know if it's as simple as moving guys up and down a spreadsheet.

2. How do you weigh team needs vs. positional value?

Again - same machine logic as a mock draft. That's why rather than just run a big board, I said you'd want to sim 100 drafts or something to get your results.

Because many of these websites do account for need. Or allow YOU to select if you want to account for need in how the draft is simulated.

How do you account for the "depth" of the draft? i.e. in a year like 2013 it would be easier to move around in the draft because that one was a big ol' pile or crap. But in a deep draft like this, it may be harder/teams may be more likely to stick exactly to pick value in trades.

Same answer as above - need some level of AI in there. It would also address the 'randomness' element discussed after you. If there are enough simulations done with a good enough AI to know if/when/how a trade would be appropriate or accepted, you can get large numbers to mostly iron out in the probabilities.

That's ultimately why I said it would take some programming understanding WELL beyond my comprehension. You'd need the simulations that create your draft odds to be somewhat reliable. In the end I think the 'big board' problem is the easiest to get past. It's the creation of the actual simulator that would make the headaches.

[DJ's left nut]

Originally Posted by kepp

The logic behind it wouldn't be difficult to code. It's the underlying data that needs to carefully considered. You need a source or truth...or as close to a source of truth as you can get. Otherwise you're just randomly selecting. You'd need a source of truth for player rankings, team needs, front office/GM draft tendencies...the list goes on. All this is highly subjective.

The only subjective part in the code would be the formula to calculate the probability of each pick because you would have to decide the weight given to each data point in the overall calculation. The rest of the code would just be cycling through the rounds applying that formula.

Yeah, I think you'd want to do as much 'wisdom of crowds' as you possibly can here. And then lean into the law of large numbers.

If you can get several reliable sources in compiling your source board (and yeah, 'reliable' would be open to interpretation) and then run dozens of sims in creating the results, then you'd be combining the collective wisdom of several knowledgeable individuals and then using large numbers to iron out the randomness of the results.

[Otter]

The data science and classification predictions wouldn't be much more accurate than the average person's March Madness bracket IMHO. It would be an interesting and fun project in Python (language I'd use) for some aspiring computer science students I'd wager.

[kccrow]

I agree that it's developing the logic behind the programming that is going to be the bear of a task. Alot of rambling here so hope I can make some sense...

First off, you have team needs. GMs don't operate with the now current team's needs in mind only, they look out for at least a year and sometimes more. And just because a team is perceived to have strength, doesn't mean they won't add to it. It's not like the Bengals had a bad WR group with Boyd and Higgins when they drafted Chase.

Then, not only do you have to create a grade/value for each player but you have the added task of determining an appropriate range for any player's value. You could use tiers, but those tiers move with each draft.
First, the grades. There are only so many boards that go 300 players deep and they usually aren't the better ones, and even then which ones actually provide a "grade" or "value" outside of NFL.com? You'll need some type of value. You then have to ask how much of that value is being created by athleticism that could be present in a modifier like RAS and how much is actually due to performance. I feel as though you need to develop a RPS (relative performance score) that gets mulitplied by RAS to create a value that's actually independent of subjective analysis. Speaking of RAS, you have your over-drafted workout warriors every year. Is there more weight to RAS than to RPS?
If you do slot these into tiers, one problem is that there may not be any tier 1 players in the 2022 draft but there will be in the 2023 draft. You have more tier 2 and 3 players though, which push 4's down, etc. You can account for that, but what happens with the chance that a GM reaches down a tier or two and takes a guy that makes everyone go "WHAT?" And then, how do you establish the tiers (value breakpoints)?
Even if you managed to nail down how you want to assign grades and tiers, how are you going to account for positional value and team scheme? Some players may fit a 3-4 defense only, some a 4-3 only, and others both. Some receivers can only play the slot, some the Z, some the X, some any. Some teams only like big running backs. You get the point. You get all these "caveats" that are difficult to put into a matrix. These are likely team-by-team coefficients in your algorithm.


You certainly have a large task to create a database that houses all the information related to the player variables. To get an algorithm that is all-encompassing, it will need to first reference another database that extracts your team-by-team coefficients. It's going to take alot of thought and work.

I think it's a more beneficial project if you attack it trying to create your own AI Mel Kiper than it is to try to use Mel Kiper(s) to create your AI.

[Couch-Potato]

Simulate 100 mocks with each of the top 5 mock draft simulators, transfer the outcome to an excel spreadsheet to calculate %, there ya go! Fairly simple really, although sentiments change during the weeks leading up to the draft.

As for an actual program that would do this for you? Would be a lot easier to just ask the draft simulator sites for their data. They likely have this built-in, or could easily build this feature into their current programs, and the public is routinely providing the sentiment data.

[Wilson8]

The problem is not the program creation, but the data that would be used.

A mock is only as good as the data used. All of you know from the mocks that you have done you need -

1. A really good evaluation of each NFL team. You must have team rosters and needs and past tendencies for the draft. Players contracts and team cap have to be considered in this.
2. A complete evaluation of each player and a ranking for each player. This could be abbreviated if you only did 1st round mock, but a good one would be top to bottom with all players in the draft.

I've tried getting the rankings by blending lists from several different sources. You have to evaluate and maybe throw out the outliers that are so far out of skew with all of the other evaluations.

Also, you have to be careful with some evaluations that they are not just copying what they see from others. That can skew the data and the evaluation scale.

I am pretty sure that NFL organizations try to create their own mock drafts with the data that they are able to gather. It would be interesting to see what they do.

[ntexascardfan]

It'd be cool to have a draft simulator with machine learning. As people participate in the draft simulator and make picks for various teams, the AI adapts and starts augmenting drafts based on how users are approaching the draft for their own team.

[Couch-Potato]

This is what you're looking for...

http://espnsportsanalytics.com/

[Couch-Potato]

According to ESPN we'd have an 85% chance at J Williams if we're able to get to #13, 80% at #15, and only a 40% chance at #18.