The "Eye-Test" & Stats Are Basically the Same Thing
That's right.
It is the year of our Lord 2021 and we are still having the absolutely inane “eye-test” vs. stats debate.
These arguments often go something like this:
Person X posts eye-test take
Person Y quote tweets/replies, using some stat to contest the conclusion reached by Person X
Person X responds by pointing out that stats lack “context” and therefore cannot be used to refute them
Person Y supplements related stats in an effort to overcome this
Person X concludes that these figures also lack crucial context
Person Y gets mad at the dumb brutes of football twitter who failed 5th-grade math
Person X starts ranting about how numbers nerds have never had sex
50 other people get involved and start fighting
Kees van Hemmen gets ratioed
A random hero posts this:
On rare occasions, we get a considered back-and-forth that leads to more intelligent rebuttals from an individual like Person Y, who might counter stats phobia by pointing out that some of the most well-run and dominant clubs in the world, like Liverpool or Manchester City, have first-class analytics departments. Or that data analysts, especially in the professional field, almost always use data in harmony with expert tactical opinion. Or that one’s eye-test can even be enhanced by looking at numbers.
All of these things are true, but many of these rebuttals serve to reinforce the notion that the eye-test and stats are two distinct methods simply by entertaining the debate, which ultimately validates the idea that there are clear-cut pros and cons to consider and that one way could possibly be superior.
Indeed, I suspect a vast majority of people see nothing wrong with this. Clearly, there is a world of difference between watching people kick a ball and organizing columns on a spreadsheet.
But is there, really?
What is the Eye-Test?
I would posit that most folks feel this way because it’s not at all clear what the “eye-test” even means. Does it refer to simply watching the game? That’s what a lot of people seem to be getting at.
However, that definition appears incomplete. It only seems to capture the “eye” portion of the word. “Test” implies an assessment of some sort. Hence, we must be talking about a set of related actions: observation which leads to analysis.
Seems simple enough.
But how do we ensure that said eye-test is any good (therefore ensuring said “eye-test” has any value)? Beyond guaranteeing that it’s a smart person doing the observing and analyzing, how does one begin to organize and logically relate the information that’s being mentally collected? In the short term, it may be possible for a few talented individuals to keep everything in their head, but what about the long run?
Luckily, humans have invented something cool called “writing.” By noting down what we see, we can safely store a much larger set of observations, which gives us a chance to observe the observations themselves in order to begin the process of building logical connections between them. In other words, written records allow for a second assessment of your initial assessment — an eye-test of the eye-test if you will — in service of building a stronger assessment.
Semantically, we seem to have strayed quite far away from what is initially implied by the “eye-test.” Reading and organizing notes aren’t the same as watching video. Nevertheless, virtually no one will tell you that a tactical report isn’t an example of the eye-test.
There’s an implicit assumption that the eye-test encompasses any opinion based off of anything that is recorded from the process of watching a game.
In that case, I have some really bad news about how stats are created.
What Are Stats?
From Carl Bialik’s article for FiveThirtyEight:
Companies employ people…to watch matches and document every event.
Throughout the year, 350 part-time analysts working in London and a half-dozen other Opta branches in Europe and North and South America record every pass, header and goal while watching live or recorded video [emphasis added] of more than 14,000 matches around the world.
Most of the work is logging routine passes. Opta’s analysts log each one by dragging and clicking a mouse at the spot where the pass was received, then keying in the player who received it. Their monitors have an image of a soccer pitch in the background with video of the live match superimposed on top [emphasis added].
Yep, at its most fundamental level, stats are simply — wait for it — the eye-test. To be more exact: they are a systematized eye-test, wherein companies like OPTA seek to construct definitions of action types in order to produce relatively consistent numerical notations (i.e. event data). These notations can then be observed a second time in order to produce logical connections between the initial observations.
Instead of producing a traditional piece of tactical analysis like before, that process of making sense of the systematized eye-test might involve statistical analysis, a data visualization, or the formation of metrics (event data that is combined and manipulated to produce a more insightful number).
Thus, it is fair to say that the tools and methods used to make sense of these varying notations might differ, but it is equally valid to assert that these procedures are seeking to achieve the same thing on a fundamental level, which is to make sense of observations collected from watching games.
Semantically, the “eye-test” doesn’t really exist as this separate method of observation; not if we want to define it as observations made from viewing football, anyway. Nonetheless, we’ve arbitrarily decided to make this distinction in the discourse.
I could go down a huge rabbit hole explaining how the difference we really seem to be getting at has to do with qualitative vs. quantitative assessments and how that relates to varying levels of systematization (and how those distinctions aren’t black and white either), but I’ll spare you. We can accept this arbitrary separation for simplicity’s sake if it’s understood that the “eye-test” and “stats” are intimately linked in their production and objectives and that the discourse-driven distinction between them is logically incoherent.
Addressing Observational Discrepancies
If you have grasped this, then it can be further understood that any discrepancy between the eye-test and stats is actually just a discrepancy in how the information is being recorded — not from some irreconcilable incompatibility between methods.
Of course, this can also occur between two people engaging in the popular understanding of the eye-test. For example, if you pointed out that X team pressed with two players up front vs. Y opponent, and another person said that the same team pressed with one player up front vs. the same opponent, you would arrive at a resolution by examining how both of you built different records and examining why that was the case (or you could call the other person an idiot and paste their profile pic onto a meme — whatever works).
It’s the same deal with the “eye-test” and stats. All that’s happening is a discrepancy in observation, which can be resolved if you understand what the stat is and isn’t telling you and how that relates to your own analysis.
Sure, this might lead to situations where it turns out your “eye-test” recorded the info worse, which requires some humility to admit and is probably where a lot of people get turned off to it all. However, if you’re truly interested in gaining a deeper understanding of this game, it makes little sense to ignore a vast trove of observations simply because they may not perfectly align with yours.
Another thing that requires humility is accepting that you can’t do it all on your own. As a lone force, you will struggle to evaluate whether someone is finishing chances more or less than the average footballer because you can’t call to memory every single shot that the player has taken, much less assign an accurate conversion probability to each shot based on a variety of contextual factors (such as the height of the pass, the foot used, the type of pass, the distance from goal, whether the shot came off of a dribble, etc.) — not to mention that you would have to do this for tens of thousands of other players to be able to obtain an average of any significance.
Sure, you could note down every single shot and do the work to assign probabilities, but you’re essentially doing statistics then.
And why would you do that when people have already done that for you (or why wouldn’t you just amass shots already collected by other people to more quickly build your own model)?
Take advantage of the existence of systematized information. Take advantage of the fact that there are an incalculable number of people out there thanklessly providing eye-tests for almost every single pass, shot, and goal that occurs. Take advantage of this to challenge the information that you collect so that you can improve your own “eye-test.”
Take advantage of the fact that there are multiple different ways to record what we see on a football pitch, producing a variety of information points that, in conjunction with each other, can expand and deepen your knowledge of the sport significantly.
Take advantage of statistics.
If you don’t, you’re not really taking full advantage of the eye-test either.
Additional reading: The eye test and competing theories of knowledge by Nicó Morales