FIFA 2021 Player Analysis

from IPython import display
display.Image("Fifa.png")

By: Nathan Chung

Outline

• Introduction
  • Set Up
  • Background Information
• Data Collection/Data Management
  • What is the data?
  • Getting Data
  • Data Cleaning
• Exploratory Data Analysis
  • Creating useful visualizations
  • Start Process of analyzing
• Machine Learning
• Conclusion

Introduction

This Data Science Tutorial will guide you through the Data Science Pipeline. In this project we are analyzing soccer player's data from Fifa 2021. More specifically we will be finding relationships and trends within the datasets between the top 5 plyaers in the world to later on create and test a model.

Background Information

Fifa is the International Federation of Association Football. This organization organizes and manages world football, holding the World Cup Final that happens every four years. Over the past two decades, EA Sports has developed FIFA soccer games that are released annually. In this series of soccer games, each one has curated real world teams and players, where each individual soccer player is given a ranking out of 100, depending on their player statistics that vary from shooting, passing, dribbling, etc.

Set Up

In this project, we are using several useful libraries that exist in python. Down below are all of the packages that we will be using in this tutorial.

More information on python libraries:

• https://data-flair.training/blogs/python-libraries/

import requests
import numpy as np
import pandas as pd
from ggplot import *
import seaborn as sns
from bs4 import BeautifulSoup
import urllib.request
%matplotlib inline
import matplotlib.pyplot as plt
from sklearn.linear_model import LinearRegression
import sklearn.linear_model as lm
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report, confusion_matrix
import statsmodels.api as sm
import os
import plotly.graph_objects as go
from plotly.subplots import make_subplots

Data Collection/Data Management

What is the data?

The datasets that are used in this project are made up of soccer player's data from the Career Mode in each Fifa game from 2021.

The data is taken from kaggle which is an online public data base. The link to the repository for the data is down below. All of the files are labeled players_XX.csv where XX denotes which Fifa year the data was taken from.

• https://www.kaggle.com/stefanoleone992/fifa-21-complete-player-dataset

Getting Data

In order to get the data, you will go to the link https://www.kaggle.com/stefanoleone992/fifa-21-complete-player-dataset and download the file 'players_21.csv'.

Notice that each file ends with a .csv. These are 'Comma Separated Values' files which are extremely common in data science. These files are used for exchanging data between different applications. In order to make these files usable for this project, we need to read them into a dataframe by utilizing Python Pandas' dataframe with read_csv.

Once we have the datasets loaded into dataframes, we use .head() to display the data in table format to get a first glimpse of what we are looking at.

# Reading each file into dataframes for preparation and analysis.
data_21 = pd.read_csv('players_21.csv')

# Prinding out first 50 rows in dataframe that has Fifa 2021 dataset.
# Each dataset has the same columns and same information. So looking at one dataset will show what others will look like.
data_21.head()

	sofifa_id	player_url	short_name	long_name	age	dob	height_cm	weight_kg	nationality	club_name	...	lwb	ldm	cdm	rdm	rwb	lb	lcb	cb	rcb	rb
0	158023	https://sofifa.com/player/158023/lionel-messi/...	L. Messi	Lionel Andrés Messi Cuccittini	33	1987-06-24	170	72	Argentina	FC Barcelona	...	66+3	65+3	65+3	65+3	66+3	62+3	52+3	52+3	52+3	62+3
1	20801	https://sofifa.com/player/20801/c-ronaldo-dos-...	Cristiano Ronaldo	Cristiano Ronaldo dos Santos Aveiro	35	1985-02-05	187	83	Portugal	Juventus	...	65+3	61+3	61+3	61+3	65+3	61+3	54+3	54+3	54+3	61+3
2	200389	https://sofifa.com/player/200389/jan-oblak/210002	J. Oblak	Jan Oblak	27	1993-01-07	188	87	Slovenia	Atlético Madrid	...	32+3	36+3	36+3	36+3	32+3	32+3	33+3	33+3	33+3	32+3
3	188545	https://sofifa.com/player/188545/robert-lewand...	R. Lewandowski	Robert Lewandowski	31	1988-08-21	184	80	Poland	FC Bayern München	...	64+3	65+3	65+3	65+3	64+3	61+3	60+3	60+3	60+3	61+3
4	190871	https://sofifa.com/player/190871/neymar-da-sil...	Neymar Jr	Neymar da Silva Santos Júnior	28	1992-02-05	175	68	Brazil	Paris Saint-Germain	...	67+3	62+3	62+3	62+3	67+3	62+3	49+3	49+3	49+3	62+3

5 rows × 106 columns

# Printing out the number of columns in the dataframe.
print("Number of Columns: " + str(len(data_21.columns)))

Number of Columns: 106

Data Cleaning

As we can see, there are 106 columns in each dataset. A lot of the these columns are not very practical and will not be used in this project such as rs, lw, lf, cf, etc (columns that give a player a value fore each and every position).

In this tutorial, our focus is to try to develop a model that can deem itself to be accurate enough to find trends and relationships in the data. Having all of the extra data is not needed so in this step, we will clean the data.

We use the iloc() function from pandas that is a method to retrieve rows from a Dataframe. We are also using slicing when writing iloc[:, 0:39] to only grab columns 1 through 40 in each dataset and dropping the rest. Slicing is an efficient method of organizing data as it takes away the extra work to write down each column name which in this case is more than 50 which would be too much work and not efficient at all.

# Using iloc() function from Pandas Library to reformat datasets so that that only contain information we need. 
data_21 = data_21.iloc[:, 0:75]

# Displaying head of the new dataset.
data_21.head()

	sofifa_id	player_url	short_name	long_name	age	dob	height_cm	weight_kg	nationality	club_name	...	power_long_shots	mentality_aggression	mentality_interceptions	mentality_positioning	mentality_vision	mentality_penalties	mentality_composure	defending_marking	defending_standing_tackle	defending_sliding_tackle
0	158023	https://sofifa.com/player/158023/lionel-messi/...	L. Messi	Lionel Andrés Messi Cuccittini	33	1987-06-24	170	72	Argentina	FC Barcelona	...	94	44	40	93	95	75	96	NaN	35	24
1	20801	https://sofifa.com/player/20801/c-ronaldo-dos-...	Cristiano Ronaldo	Cristiano Ronaldo dos Santos Aveiro	35	1985-02-05	187	83	Portugal	Juventus	...	93	63	29	95	82	84	95	NaN	32	24
2	200389	https://sofifa.com/player/200389/jan-oblak/210002	J. Oblak	Jan Oblak	27	1993-01-07	188	87	Slovenia	Atlético Madrid	...	12	34	19	11	65	11	68	NaN	12	18
3	188545	https://sofifa.com/player/188545/robert-lewand...	R. Lewandowski	Robert Lewandowski	31	1988-08-21	184	80	Poland	FC Bayern München	...	85	81	49	94	79	88	88	NaN	42	19
4	190871	https://sofifa.com/player/190871/neymar-da-sil...	Neymar Jr	Neymar da Silva Santos Júnior	28	1992-02-05	175	68	Brazil	Paris Saint-Germain	...	84	51	36	87	90	92	93	NaN	30	29

5 rows × 75 columns

# List of column names we want to drop
lst = ['value_eur','player_url', 'wage_eur', 'body_type', 'skill_moves', 'weak_foot', 'nation_jersey_number', 'nation_position', 'international_reputation', 'work_rate','real_face','release_clause_eur','loaned_from', 'team_position','player_tags', 'team_jersey_number','contract_valid_until', 'loaned_from', 'joined', 'gk_diving', 'gk_handling', 'gk_kicking', 'gk_reflexes', 'gk_speed', 'gk_positioning', 'player_traits', 'defending_marking', 'defending_standing_tackle', 'defending_sliding_tackle']

# Dropping all the columns with names corresponding to strings in lst that was initialized above. 
data_21.drop(lst, inplace = True, axis = 1)
data_21

	sofifa_id	short_name	long_name	age	dob	height_cm	weight_kg	nationality	club_name	league_name	...	power_jumping	power_stamina	power_strength	power_long_shots	mentality_aggression	mentality_interceptions	mentality_positioning	mentality_vision	mentality_penalties	mentality_composure
0	158023	L. Messi	Lionel Andrés Messi Cuccittini	33	1987-06-24	170	72	Argentina	FC Barcelona	Spain Primera Division	...	68	72	69	94	44	40	93	95	75	96
1	20801	Cristiano Ronaldo	Cristiano Ronaldo dos Santos Aveiro	35	1985-02-05	187	83	Portugal	Juventus	Italian Serie A	...	95	84	78	93	63	29	95	82	84	95
2	200389	J. Oblak	Jan Oblak	27	1993-01-07	188	87	Slovenia	Atlético Madrid	Spain Primera Division	...	78	41	78	12	34	19	11	65	11	68
3	188545	R. Lewandowski	Robert Lewandowski	31	1988-08-21	184	80	Poland	FC Bayern München	German 1. Bundesliga	...	84	76	86	85	81	49	94	79	88	88
4	190871	Neymar Jr	Neymar da Silva Santos Júnior	28	1992-02-05	175	68	Brazil	Paris Saint-Germain	French Ligue 1	...	62	81	50	84	51	36	87	90	92	93
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
18939	256679	K. Angulo	Kevin Angulo	24	1996-04-13	176	73	Colombia	América de Cali	Colombian Liga Postobón	...	57	40	51	36	57	41	31	49	39	36
18940	257710	Zhang Mengxuan	Mengxuan Zhang	21	1999-04-26	177	70	China PR	Chongqing Dangdai Lifan FC SWM Team	Chinese Super League	...	65	55	45	23	44	50	25	25	35	40
18941	250989	Wang Zhenghao	王政豪	20	2000-06-28	185	74	China PR	Tianjin TEDA FC	Chinese Super League	...	69	58	55	22	46	45	25	25	35	40
18942	257697	Chen Zitong	Zitong Chen	23	1997-02-20	186	80	China PR	Shijiazhuang Ever Bright F.C.	Chinese Super League	...	56	48	63	33	56	40	31	45	42	43
18943	257936	Song Yue	Yue Song	28	1991-11-20	185	79	China PR	Tianjin TEDA FC	Chinese Super League	...	59	54	62	34	57	35	48	44	36	35

18944 rows × 47 columns

As you can see, we now have what seems to be a more friendly dataset that can be analyzed without the extra jargon information.

More Data Cleaning

Our main goal of this project is to determine what makes a top tier soccer player. Which components/attributes come into play that best reflect a highly skilled soccer player? In order to do this, the best way is to condense the data set. For the roster of players from different FIFA 2021, there are at least 18,000 players in total. Since we are trying to hone into the individuals that have higher rankers, we can condense the data set and try to track the top 50 players from each year.

We can also knock our two birds with one stone. In the FIFA 2021 roster, each and every player for each and every position is recorded, including Goal Keepers. Goal Keepers have entirely different statistics that are recorded from their performance compared to field players like strikers and midfielders. As a matter of fact, for the statistics for field players such as shooting, passing, and dribbling, almost each and every goalkeeper has a missing NaN value which if we keep in the data will cause issues later on when applying regression.

In this project, we are mainly focusing on field players like midfielders or strikers, so we can eliminate all goal keepers that are listed in the data set.

# Our method of finding the top 50 players from FIFA2021 is by using the 'overall' ranking of the player and sorting them.
df = data_21.sort_values(by='overall',ascending=False).head(200)

# Iterrating through each row and column in the dataframe, checking if the player position is a GK to identify goalkeepers and remove them.
for index, row in df.iterrows():
    # Conditional that checks each player if they are a Goal Keeper or not. 
    if (row['player_positions'] == 'GK'):
        # The .drop() function is a very useful pandas datafrem functionality that is used for data cleaning.
        # Drop can be used to get rid of specific columns/rows which can deem itself very useful and efficient for eliminating useless data. 
        df = df.drop(index, axis=0)
        
df.head()

	sofifa_id	short_name	long_name	age	dob	height_cm	weight_kg	nationality	club_name	league_name	...	power_jumping	power_stamina	power_strength	power_long_shots	mentality_aggression	mentality_interceptions	mentality_positioning	mentality_vision	mentality_penalties	mentality_composure
0	158023	L. Messi	Lionel Andrés Messi Cuccittini	33	1987-06-24	170	72	Argentina	FC Barcelona	Spain Primera Division	...	68	72	69	94	44	40	93	95	75	96
1	20801	Cristiano Ronaldo	Cristiano Ronaldo dos Santos Aveiro	35	1985-02-05	187	83	Portugal	Juventus	Italian Serie A	...	95	84	78	93	63	29	95	82	84	95
3	188545	R. Lewandowski	Robert Lewandowski	31	1988-08-21	184	80	Poland	FC Bayern München	German 1. Bundesliga	...	84	76	86	85	81	49	94	79	88	88
4	190871	Neymar Jr	Neymar da Silva Santos Júnior	28	1992-02-05	175	68	Brazil	Paris Saint-Germain	French Ligue 1	...	62	81	50	84	51	36	87	90	92	93
5	192985	K. De Bruyne	Kevin De Bruyne	29	1991-06-28	181	70	Belgium	Manchester City	English Premier League	...	63	89	74	91	76	66	88	94	84	91

5 rows × 47 columns

Exploratory Data Analysis

Exploratory Data Analysis the processs where you do your initial analysis and breakdown of the data to look for any trends, patterns, anomalies, as well as summarizing the different characteristics of the data. EDA is done before any assumptions are made about the data set.

This process helps a data scientist find out how to best manipulate the data to get the answers of their desired interest.

Although this is only a brief overview on what Exploratory Data Analysis consists of, a more detailed source of information can be found below

• https://www.ibm.com/cloud/learn/exploratory-data-analysis

These were the top 50 ranked players from FIFA 2021. Now from here, we can use this information to utilize each player's sofifa_id and see if they were in the top 50 ranked players from the previous FIFA rosters and if they are, keep them. But since we want consistency amongst each dataset, if we run into a player that was in the top 50 for one year and not in the top 50 for another year, we will simply remove that player from the new dataset we are about to make.

Understanding the Data

As we can see, there are several different columns in our newly cleaned and processed dataset. Whats important is that we know what each of these components are and that we understand what their purpose is. This is crucial before starting any EDA because you should have a good understanding of what your data consists of.

df.head()

	sofifa_id	short_name	long_name	age	dob	height_cm	weight_kg	nationality	club_name	league_name	...	power_jumping	power_stamina	power_strength	power_long_shots	mentality_aggression	mentality_interceptions	mentality_positioning	mentality_vision	mentality_penalties	mentality_composure
0	158023	L. Messi	Lionel Andrés Messi Cuccittini	33	1987-06-24	170	72	Argentina	FC Barcelona	Spain Primera Division	...	68	72	69	94	44	40	93	95	75	96
1	20801	Cristiano Ronaldo	Cristiano Ronaldo dos Santos Aveiro	35	1985-02-05	187	83	Portugal	Juventus	Italian Serie A	...	95	84	78	93	63	29	95	82	84	95
3	188545	R. Lewandowski	Robert Lewandowski	31	1988-08-21	184	80	Poland	FC Bayern München	German 1. Bundesliga	...	84	76	86	85	81	49	94	79	88	88
4	190871	Neymar Jr	Neymar da Silva Santos Júnior	28	1992-02-05	175	68	Brazil	Paris Saint-Germain	French Ligue 1	...	62	81	50	84	51	36	87	90	92	93
5	192985	K. De Bruyne	Kevin De Bruyne	29	1991-06-28	181	70	Belgium	Manchester City	English Premier League	...	63	89	74	91	76	66	88	94	84	91

5 rows × 47 columns

Our main focus is to understand the data values we will most focus on:

Attributes we will focus on are:

• sofifa_id: Unique id number for the player on Fifa
• short_name: Short Name of the player
• long_name: Long name of the player
• age: Age of the player
• dob: The player's date of birth
• height_cm: height of the player in cm
• weight_kg: weight of the player in kg
• nationality: Player's nationality
• club_name: The name of the club/team the player plays for
• league_name: The soccer league the player belongs to
• league_rank: The ranking of the league globally
• overall: Overall score/ranking for the player in the game (1-100)
• potential: Potential score of the player (1-100)
• player_positions: Soccer position the player is
• pace: Player's pace score (out of 100)
• shooting: Player's shooting score (out of 100)
• passing: Player's passing score (out of 100)
• dribbling: Player's dribbling score (out of 100)
• defending: Player's defending score (out of 100)
• physic: Player's phsyic score (out of 100)
• attacking_crossing: Player crossing Score (1-100)
• attacking_finishing: Player Finishing Score (1-100)
• attacking_heading_accuracy: Player Header Accuracy Score (1-100)
• attacking_short_passing: Player Short Passing Score (1-100)
• mentality_aggression: Player Aggression Score (1-100)
• mentality_interceptions: Reading Interceptions Score (1-100)
• mentality_positioning: Ability to make good positioning Score (1-100)
• mentality_vision: Field vision score (1-100)
• mentality_penalties: Penalty mentality score (1-100)
• mentality_composure: Composure during a game score (1-100)

Now since we have the data, we can first start off by showing the order of players from best to worst using different attributes of the data set. This can help identify players that consistently have presence in towards the top of the rankings for each attribute.

# Sorting the values based off of player overall rank from highest to lowest to see order of best to worst player. 
df.sort_values(by='overall',ascending=False)[['short_name','age','overall']]

	short_name	age	overall
0	L. Messi	33	93
1	Cristiano Ronaldo	35	92
3	R. Lewandowski	31	91
4	Neymar Jr	28	91
5	K. De Bruyne	29	91
...	...	...	...
207	D. Zapata	29	83
206	F. Kostić	27	83
205	Gerard Moreno	28	83
203	Jorginho	28	83
173	C. Vela	31	83

174 rows × 3 columns

As we can see from this display above, the top highest rated players are L. Messi , Cristiano Ronaldo , R. Lewandowski, Neymar Jr, and K. De Bruyne. From this we can can show the matrix where darker shades of blue cells are highly correlated with each other, and lighter blue cells are less correlated.

Looking at all of the rankings can only do so much. Our main goal is to find any relationships within the data. Constantly comparing each and every variable to each other can deem itself a challenging and time consuming task.

A more efficient approach is to use a correlation matrix. A correlation matrix shows any correlation between two variables, and is commonly used to summarize data to initiate the next step for analysis. Luckily in python, there is a library called seaborn that makes it easy to generate a correlation matrix.

Seaborn is a data visualization library in python that provides borth attractive and informative statistical graphs. More information can be found in the link below

• https://seaborn.pydata.org/

df2 = df.sort_values(by='overall',ascending=False)[['overall', 'pace', 'shooting', 'dribbling', 'defending', 'physic', 'mentality_aggression', \
                                                    'mentality_positioning','mentality_vision', 'mentality_composure']]
plt.figure(figsize=(9,6))
corrMatrix = df2.corr()
sns.heatmap(corrMatrix, annot=True)
plt.show()

In order to interpret a correlation map, you look at the scoring for each component and see which others have similar equated scores. In this case, on our correlation matrix we can see that shooting scores for players are related to the dribbling scores, as well as there being a relationship between mentality_vision scores and mentality_positioning scores.

It make's sense now looking at the relationships that a player's mentality_vision will heavily influence their passing and shooting skill's as well as positioning.

From this, we can use this information to further explore those relationships through visualizations and graphs.

Creating Useful Visualizations

Data visualization creates a visual summary which makes it much more easier to find relationships and trends in data rather than trying to process several columns and rows of data.

Gaining more insight is acheived with visual depictions of data becuase the data we are working with becomes more valuable when it is processed into a visualization.

One of the main goals of this entire project is to find out what make's the worlds best soccer player the best. In this case, Leonel Messi is the highest level ranked player in FIFA 2021, but what makes him so special?

To start, creating visuals to see how Messi differs from other players in different player stats can help. Since we are dealing with several different players of different caliber, a scatter plot would be the most ideal and adding a trend line will show whether a player is exceeding the average performance in that specific skill set or if they are below average.

From our correlation matrix, we found that player's ranking in passing, shooting, metality_vision, and mentality_positioning had correlated relationships so we will be focusing on those 4 areas when visualizing.

We will visualize the relationship between a player's shooting ranking and their mentality_vision ranking. A crucial skill to have on the soccer field is having the visual prowess to read where everyone is on the field, and with that knowledge being able to attack the opposing team's defense in the best logical way.

# Seting size of graph we are about to produce
fig, ax = plt.subplots(figsize=(15,8))
# Setting x and y variables to hold focus data
x = df['shooting']
y = df['mentality_vision']
# Creating a scatter plot using the x and y variables we created
ax.plot(x, y, 'o')
# Here, we are using np.poly fit to add a regression line that shows how the Overall Rank changes as the Shooting Score Changes.
# m represents the slope and b represents the intercept of the linear regression line. 
m, b = np.polyfit(x,y, 1)
# Plotting/adding the regression line.
plt.plot(x, m*x+b, color = 'red')

# This loop goes through each player in the list, and looks for the top 5 ranked players.
# This will help narrow down on the variance of skill and comparing players that are around the same skillset as Messi will be a more effective analysis. 
for idx, row in df.iterrows():
    name = row['short_name']
    x = row['shooting']
    y = row['mentality_vision']
    if(name == 'L. Messi' or name == 'Cristiano Ronaldo' or \
       name == 'R. Lewandowski' or name == 'Neymar Jr' or name == 'K. De Bruyne'):
        ax.annotate(name, (x, y))

# Setting the proper x-axis/y-axis labels and the title.
plt.xlabel('Shooting  Rank')
plt.ylabel('Menality Vision Rank')
plt.title('Shooting Rank vs Menality Vision Rank')
plt.show()

As we can see, Messi has the highest performance in mentality vision.

We will visualize the relationship between a player's passing ranking and their mentality_vision ranking.

# Seting size of graph we are about to produce
fig, ax = plt.subplots(figsize=(15,8))
# Setting x and y variables to hold focus data
x = df['passing']
y = df['mentality_vision']
# Creating a scatter plot using the x and y variables we created
ax.plot(x, y, 'o')
# Here, we are using np.poly fit to add a regression line that shows how the Passing Score changes as the mentality_vision Score Changes.
# m represents the slope and b represents the intercept of the linear regression line. 
m, b = np.polyfit(x,y, 1)
# Plotting/adding the regression line.
plt.plot(x, m*x+b, color = 'red')

# This loop goes through each player in the list, and looks for the top 5 ranked players.
# This will help narrow down on the variance of skill and comparing players that are around the same skillset as Messi will be a more effective analysis. 
for idx, row in df.iterrows():
    name = row['short_name']
    x = row['passing']
    y = row['mentality_vision']
    if(name == 'L. Messi' or name == 'Cristiano Ronaldo' or \
       name == 'R. Lewandowski' or name == 'Neymar Jr' or name == 'K. De Bruyne'):
        ax.annotate(name, (x, y))

# Setting the proper x-axis/y-axis labels and the title.
plt.xlabel('Passing  Rank')
plt.ylabel('Mentality Vision Rank')
plt.title('Passing Score vs Mentality Vision')
plt.show()

Here, we are starting to see bigger differences amongst the top 5 players that were labeled. The ongoing debate on whether Ronaldo or Messi is better has brought up constant debate. In this visualization we can see how Messi outshines Ronaldo in passing which makes us believe that the difference in this skill plays a big role in the overall ranking.

In this visualiztion, we are observing the relationship between a player's skill in passing and shooting.

# Seting size of graph we are about to produce
fig, ax = plt.subplots(figsize=(15,8))
# Setting x and y variables to hold focus data
x = df['passing']
y = df['shooting']
# Creating a scatter plot using the x and y variables we created
ax.plot(x, y, 'o')
# Here, we are using np.poly fit to add a regression line that shows how the Shooting Score changes as the Passing Score Changes.
# m represents the slope and b represents the intercept of the linear regression line. 
m, b = np.polyfit(x,y, 1)
# Plotting/adding the regression line.
plt.plot(x, m*x+b, color = 'red')

# This loop goes through each player in the list, and looks for the top 5 ranked players.
# This will help narrow down on the variance of skill and comparing players that are around the same skillset as Messi will be a more effective analysis. 
for idx, row in df.iterrows():
    name = row['short_name']
    x = row['passing']
    y = row['shooting']
    if(name == 'L. Messi' or name == 'Cristiano Ronaldo' or \
       name == 'R. Lewandowski' or name == 'Neymar Jr' or name == 'K. De Bruyne'):
        ax.annotate(name, (x, y))

# Setting the proper x-axis/y-axis labels and the title.
plt.xlabel('Passing Rank')
plt.ylabel('Shooting Rank')
plt.title('Passing Rank vs Shooting Rank')
plt.show()

Here we are taking a look at the relationship between a player's skill in dribbling and mentality_vison.

# Seting size of graph we are about to produce
fig, ax = plt.subplots(figsize=(15,8))
# Setting x and y variables to hold focus data
x = df['dribbling']
y = df['mentality_vision']
# Creating a scatter plot using the x and y variables we created
ax.plot(x, y, 'o')
# Here, we are using np.poly fit to add a regression line that shows how the Dribbling Rank changes as the mentality_vision Score Changes.
# m represents the slope and b represents the intercept of the linear regression line. 
m, b = np.polyfit(x,y, 1)
# Plotting/adding the regression line.
plt.plot(x, m*x+b, color = 'red')

for idx, row in df.iterrows():
    name = row['short_name']
    x = row['dribbling']
    y = row['mentality_vision']
    if(name == 'L. Messi' or name == 'Cristiano Ronaldo' or \
       name == 'R. Lewandowski' or name == 'Neymar Jr' or name == 'K. De Bruyne'):
        ax.annotate(name, (x, y))

# Setting the proper x-axis/y-axis labels and the title.
plt.xlabel('Dribbling Score')
plt.ylabel('Mentality Vision Rank')
plt.title('Dribbling Rank vs Mentality Vision Rank')
plt.show()

Here, we see that Lewandowski, Ronaldo, and Neymar are all below the trend line for the average dribbling skill. This shows these player's weaknesses.

Another form of visualization

Scatter plots and trend lines are great and all, but sometimes, more than one type of visualization can be effective when trying to find the answer to youre question. In the next set of graphs, we are creating joint line graphs where we will take two players, and graph their set of skill rankings in a line plot. The two players will have different colored lines so that we can compare them in hand.

Here, we will compare the statistics of Messi and Ronaldo, two of the most well knowned and coveted soccer players in the world. Finding any major differences in their skillset will help us out in finding what minute differences these top tier players have in their game that make one better than the other.

# This is a list that has all of the attributes and skills that we want to plot for each player.
skills = ['pace', 'shooting', 'passing', 'dribbling', 'defending', 'attacking_crossing',
         'attacking_finishing', 'attacking_heading_accuracy', 'attacking_short_passing',
         'attacking_volleys', 'skill_dribbling', 'skill_curve', 'skill_fk_accuracy',
         'skill_long_passing', 'skill_ball_control', 'movement_acceleration', 'movement_sprint_speed',
         'movement_agility', 'movement_reactions', 'movement_balance', 'power_shot_power',
         'power_jumping', 'power_stamina', 'power_strength', 'power_long_shots']

# Here, we are creating the two dataframes for the players Messi and Ronaldo to be plotted together.
messi = df.loc[df.short_name == 'L. Messi']
messi = pd.DataFrame(messi, columns=skills)
ronaldo = df.loc[df.short_name == 'Cristiano Ronaldo']
ronaldo = pd.DataFrame(ronaldo, columns=skills)

# Setting the figure size.
plt.figure(figsize=(15,8))
# Messi's line will be Blue, and Ronaldo's line will be green.
sns.pointplot(data=messi,color='blue',alpha=0.6)
sns.pointplot(data=ronaldo, color='green', alpha=0.6)
plt.xticks(rotation=90)
# labeling proper axis
plt.xlabel('Skillset', fontsize=15)
plt.ylabel('Skill Rank', fontsize=15)
plt.title('Messi vs Ronaldo', fontsize = 20)
plt.grid()

By looking at each player's line (Messi-Blue, Ronaldo-Green), we see that both players are pretty similar with most skillsets. But one thing to note, is that towards the middle of each line plot where the data shows the players attacking_short_passing, attacking_volleys, skill_dribbling, skill_curve, skill_fk_accurace, skill_long_passing, and skill_ball control, Leonel Messi has notably higher rankings in those skillsets compared to Ronaldo.

This observation may be what we are looking for. It may be that because Messi is more skilled in attacking_short_passing, attacking_volleys, skill_dribbling, skill_curve, skill_fk_accurace, skill_long_passing, and skill_ball control than Ronaldo, that is what sets him apart.

Here, we will compare Messi with Neymar

# Here, we are creating the dataframes for the players Neymar.
neymar = df.loc[df.short_name == 'Neymar Jr']
neymar = pd.DataFrame(neymar, columns=skills)

# Setting the figure size.
plt.figure(figsize=(15,8))
sns.pointplot(data=messi,color='blue',alpha=0.5)
sns.pointplot(data=neymar, color='yellow', alpha=0.5)
plt.xticks(rotation=90)

# labeling proper axis
plt.xlabel('Skillset', fontsize=15)
plt.ylabel('Skill Ranking', fontsize=15)
plt.title('Messi vs Neymar', fontsize = 20)
plt.grid()

We see a similar trend when we compared Ronaldo with Messi. In this visualization, (Messi-Blue, Neymare-Green), we see that both players are again, pretty similar with most skillsets.

But with the skillsets attacking_short_passing, attacking_volleys, skill_dribbling, skill_curve, skill_fk_accurace, skill_long_passing, and skill_ball control, Leonel Messi has generally higher rankings in those skillsets compared to Neymar.

Now to take a step back, here, we find a similar/identical observation where Messi has generally higher rankings in those skillsets than Neymar. This only reinforces our initial findings.

Finally, for one more comparison, we will compare Messi and Lewandowski.

lewand = df.loc[df.short_name == 'R. Lewandowski']
lewand = pd.DataFrame(lewand, columns=skills)

plt.figure(figsize=(15,8))
sns.pointplot(data=messi,color='blue',alpha=0.6)
sns.pointplot(data=lewand, color='red', alpha=0.6)
plt.xticks(rotation=90)
plt.xlabel('Skillset', fontsize=15)
plt.ylabel('Skill Ranking', fontsize=15)
plt.title('Messi vs Lewandowski', fontsize = 20)
plt.grid()

Once again, we see the same trend from when we compared Messi with Ronaldo and Neymor. In this visualization, (Messi-Blue, Lewandowski-Red), we see that both players are again, pretty similar with most skillsets.

But with the skillsets attacking_short_passing, attacking_volleys, skill_dribbling, skill_curve, skill_fk_accurace, skill_long_passing, and skill_ball control, Leonel Messi has generally higher rankings in those skillsets compared to Lewandowski.

Now what have these findings told us? To recollect our thoughts and purpose, the reason why we decided to compare Messi (the top ranked player) with other elite players such as Ronaldo and Neymar was because comparing a highly skilled player with another player with a similar skillset is much more meaning full, whereas comparing Messi who is ranked 90+ with a player who is ranked 60 in FIFA will not help note any differences that help our case, because there will always be huge differences in the statistics. So that was our reason why we pulled the top 5 ranked players and compared them with each other.

Now we wanted to find what parts of Messi's game allows him to out perform other players of the same caliber as him. From our findings, we saw the same thing in each comparison. Messi is higher skilled in attacking_short_passing, attacking_volleys, skill_dribbling, skill_curve, skill_fk_accurace, skill_long_passing, and skill_ball control, which leads us to believe that these skill sets are what matter when looking at a grouping of players that are ranked higher in the game.

Machine Learning/Linear Regression

In data science, Machine Learning is a method of data analysis by using algorithms to find patterns in data. ML applies artificial intelligence where a system/model has the ability to learn and improve calculations/predictions based off of observation and experience, without having the need to do a lot of programing.

When given the task to predict some trend for a given data, it's very hard to come up with a fucntion or write code that can be accurate all the time because different variables in data are always changing. This is where machine learning comes into play. ML practices the use of algorithms to take in data into a system, learn from it, and then forecase and predict trends.

More information can be found in the link below:

• https://www.simplilearn.com/data-science-vs-data-analytics-vs-machine-learning-article

Linear Regression

Linear Regression is a type of predictive analysis with two goals

• To find certain variables that can be used to predict values for a target outcome.
• To find which variables play a big role and are significant predictors of the outcome variables.

It is one of the most commonly used forms of regression/machine learning.

Linear regression is a part of machine learning where an algorithm is used to calculate a predicted outcome when inputed parameters/variables what we find have a large influence in the outcomes we are trying to predict.

More information can be found below:

• https://realpython.com/linear-regression-in-python/

# Grabbing the independent variables which isn this case are {age, pace, shooting, passing, dribbling, defending, physic, etc}
x = df.iloc[:,[3,15,16,17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 43, 44]]
x.head()

	age	pace	shooting	passing	dribbling	defending	physic	attacking_crossing	attacking_finishing	attacking_heading_accuracy	attacking_short_passing	attacking_volleys	skill_dribbling	skill_curve	skill_fk_accuracy	skill_long_passing	skill_ball_control	mentality_positioning	mentality_vision
0	33	85.0	92.0	91.0	95.0	38.0	65.0	85	95	70	91	88	96	93	94	91	96	93	95
1	35	89.0	93.0	81.0	89.0	35.0	77.0	84	95	90	82	86	88	81	76	77	92	95	82
3	31	78.0	91.0	78.0	85.0	43.0	82.0	71	94	85	84	89	85	79	85	70	88	94	79
4	28	91.0	85.0	86.0	94.0	36.0	59.0	85	87	62	87	87	95	88	89	81	95	87	90
5	29	76.0	86.0	93.0	88.0	64.0	78.0	94	82	55	94	82	88	85	83	93	92	88	94

# Grabbing the dependent variable which is Overall Ranking of a player
y=df.iloc[:,11]
y.head()

0    93
1    92
3    91
4    91
5    91
Name: overall, dtype: int64

Applying Linear Regression to our sets that we have made.

To perform linear regression in python, we first have to prepare what is called train and test sets.

• Train Datasets: are used to fit the machine learning model
  • Give the linear regression model data to learn from observation and experience.
  • This will help build the model
• Test Datasets: are used to evaluate the fit linear regression model.
  • These are used after the model is build using the train dataset.

# train_test_split us a function from the SKlearn python library what splits up arrays into two subsets, one for training a linear regression model and one for testing the regression model. 
x_train,x_test,y_train,y_test=train_test_split(x,y,test_size=0.2)
linreg=LinearRegression()
linreg.fit(x_train,y_train)
linreg.predict(x_test)
OLS=sm.OLS(endog=y,exog=x).fit()
OLS.summary()

OLS Regression Results

Dep. Variable:	overall	R-squared (uncentered):	0.999
Model:	OLS	Adj. R-squared (uncentered):	0.999
Method:	Least Squares	F-statistic:	1.379e+04
Date:	Mon, 17 May 2021	Prob (F-statistic):	1.05e-239
Time:	12:20:35	Log-Likelihood:	-373.25
No. Observations:	174	AIC:	784.5
Df Residuals:	155	BIC:	844.5
Df Model:	19
Covariance Type:	nonrobust

	coef	std err	t	P>|t|	[0.025	0.975]
age	0.2661	0.051	5.229	0.000	0.166	0.367
pace	0.1546	0.025	6.188	0.000	0.105	0.204
shooting	0.1040	0.077	1.343	0.181	-0.049	0.257
passing	0.5677	0.652	0.871	0.385	-0.721	1.856
dribbling	0.4292	0.196	2.189	0.030	0.042	0.816
defending	-0.0155	0.020	-0.766	0.445	-0.055	0.024
physic	0.1299	0.034	3.819	0.000	0.063	0.197
attacking_crossing	-0.1127	0.134	-0.844	0.400	-0.376	0.151
attacking_finishing	0.0105	0.055	0.190	0.850	-0.099	0.120
attacking_heading_accuracy	0.0700	0.020	3.461	0.001	0.030	0.110
attacking_short_passing	0.1823	0.244	0.747	0.456	-0.300	0.664
attacking_volleys	-0.0285	0.028	-1.017	0.311	-0.084	0.027
skill_dribbling	-0.3443	0.123	-2.809	0.006	-0.586	-0.102
skill_curve	-0.0571	0.049	-1.159	0.248	-0.154	0.040
skill_fk_accuracy	-0.0552	0.037	-1.495	0.137	-0.128	0.018
skill_long_passing	-0.1023	0.105	-0.976	0.331	-0.309	0.105
skill_ball_control	0.3058	0.107	2.869	0.005	0.095	0.516
mentality_positioning	-0.1577	0.037	-4.237	0.000	-0.231	-0.084
mentality_vision	-0.1259	0.136	-0.923	0.357	-0.395	0.143

Omnibus:	8.103	Durbin-Watson:	1.517
Prob(Omnibus):	0.017	Jarque-Bera (JB):	13.540
Skew:	0.175	Prob(JB):	0.00115
Kurtosis:	4.321	Cond. No.	1.39e+03

Notes:
[1] R² is computed without centering (uncentered) since the model does not contain a constant.
[2] Standard Errors assume that the covariance matrix of the errors is correctly specified.
[3] The condition number is large, 1.39e+03. This might indicate that there are
strong multicollinearity or other numerical problems.

How to Interpret the results from running linear regression?

• R-squared Identifies the percentage variation in the dependent variables that is explained by the independent variables.
• Adj. R-squared Is a modified R-squared value that is accounted for the number of variables in the regression.
• P-Values Represent the value from testing the hypothesis. In this case, a P-value is given for each independent variable we gave to the model where they are tested with the null hypothesis which is if there is NO-Correlation amongst the independent and dependent variables.
  • If the P-value is less than the significance level (p<0.05), that means that the model fits the data well and that there is moderate evidence against the null hypothesis.

Reading the results from linear regression

As we can see, the independent variables age, pace, dribbling, physic,attacking_heading_accuracy, skill_dribbling, skill_ball_control, and mentality_positioning all have P-values less than 0.05 which means that those variables are significant and they can not be removed.

Our Adjusted R value is 0.999 which is very good, as it means that our model does well with explaining 99.9% of the variables within the dataset.

This also confirms our findings in our EDA stage where we saw that the with top tier players, those who had higher skills in attacking_short_passing, attacking_volleys, skill_dribbling, skill_curve, skill_fk_accurace, skill_long_passing, and skill_ball control such as Messi were ones that were able to stand out and out perform other players of the same calliber. The finding from the linear regression model where similar skillsets have p-values less then 0.05 shows that these are components of a player's game that plays a large role in their ranking.

Conclusion

In this tutorial, we went through the Data Science Pipeline where we learn to about collecting data, reading from csv files and transfering data from a website/online database into your own personal workspace, cleaning the data (unstructured to structured data), and starting the process of Exploratory Data Analysis.

Our main focus was to analyze soccer player's statistics from FIFA 2021, and find out which parts of a players game allows them to truly outshine others in their respective skill set with exception to the obvious differences that were surface level such as (skill rating 60 and skill rating 90, using overall rank), or very broad statistics like height or weight. Our goal was to dig deep into analysis and find out what specific components and variables came into play. In our EDA, we found trends and relationships by first utilizing a correlation matrix, and then with those findings from the correlation matrix, beginning the process of visualizing the data into graphs and charts. Our main methods of visualizations were through scatter plots with regression/trend lines as will is line plots to compare two players. We honed into and focused on the higher rated players in the data because it only made sense to compare a player of class A(similar skill) to another player of class A(similar skill).

Our most effective visualizations was the correlation matrix and the line plots where we observed a similar trend where Messi outperformed players of the came calliber in attacking_short_passing, attacking_volleys, skill_dribbling, skill_curve, skill_fk_accurace, skill_long_passing, and skill_ball control, and in our Machine Learning phase, we further validated out findings by looking at the P-values of those skillsets to find out that they were less then 0.05 which meant they were significantly valueable.