A Quick Guide to Exploratory Data Analysis (EDA) and Descriptive Analysis

Posted on: June 2, 2022

Post Category: Data

---

Exploratory data analysis (EDA) is the initial method that is used by data analysts to quickly gain insights into their data – to test assumptions, observe errors and outliers, see relationships between variables and see opportunities to further clean or prepare the data. This is often done to enhance interpretation and support the preparation of the data for more complicated methods of analysis such as statistical machine learning.

Descriptive analysis is a method of analysis used to quantitatively summarise a dataset, often through charts and summary statistics. EDA depends on descriptive analysis.

This blog post will go over three common types of analysis involved in EDA: (1) detecting and cleaning up null/missing values, outliers and unexpected values, (2) univariate descriptive analysis, and (3) bivariate descriptive analysis. 

1. Detecting and cleaning up null/missing values, outliers and unexpected values

Data cleaning is one application of EDA, and data cleaning is crucial for preparing for further data analysis. There are three major types of quality/accuracy issues in data: (1) nulls or missing values, (2) outliers and (3) unexpected values.

1. Nulls or missing values occur when there is no value specified. The method that should be used to fix these values ultimately depends on the data and relevant domain knowledge. These methods include imputing summary statistics such as the mean or median, deleting/dropping the entire row, or some method of interpolation such as linear or exponential interpolation. You can read more about nulls and the different ways to correct them in this Analytics Vidhya article and this Medium article.
2. An outlier is any value that is unusual (usually unusually large or small) compared to other values in the column. ‘Depending on the data, sometimes outliers are expected values and other times they may indicate an error and may need to be edited, filtered, or excluded’ (Tableau 2022).
3. An unexpected value is any value that is inaccurate or inconsistent with other values in the column e.g. a code for an aeroplane that does not meet the standards (or data constraints) of an aeroplane code at a particular airline company. Unexpected values may require cleaning.

2. Univariate descriptive analysis

Univariate descriptive analysis involves quantitatively and graphically summarising one variable.

• For continuous numerical data, this often involves graphing the distribution using a histogram, density plot or boxplot, and computing the mean, median, mode, variance, maximum value, minimum value, etc.
• For categorical/ordinal and discrete numerical data, this often involves graphing the frequency/proportion of each category in the dataset – using a bar chart or equivalent – or computing frequencies and percentages in a distribution table.