Handling data and creating visualizations
Data handling management in the Python programming language includes steps such as collecting data from multiple resources, reading the data, organizing the data, cleaning the data, modifying the data, and analyzing it. Data visualization in Python involves graphically representing the available data information using graphs, charts, and plots, making it easier to understand or analyze patterns and trends in the data.
Python provides several powerful libraries for these tasks, particularly available library frameworks like Pandas, NumPy, and Matplotlib.
Data Handling Concepts in Python.
For data analysis or collection, you can collect data from multiple sources. These include:
- CSV supported extension files
- Microsoft Excel extension files
- Database source files
- JSON extension files
- Multiple API sources of data
- Manual user input data
A typical data-handling process in Python is as follows.
Collecting data from multiple sources
↓
Reading available data sources
↓
Cleaning used data
↓
Organizing data sources in a specific order
↓
Analyzing data in detail
↓
Visualizing data in a graph chart graphical format
Using the Pandas library for data handling tasks in Python.
The pandas library framework in Python is one of the most widely used Python library platforms for handling or managing tabular data.
Python users can import it this way.
import pandas as pd
The Pandas library provides you with two main data structures.
Series pandas data structures.
The Python Pandas data type stores or represents data in a one-dimensional array collection format.
values = pd.Series([12, 88, 34, 11, 22, 55, 66])
print(values)
DataFrame pandas data structures.
The Python Pandas data type has a table-like layout, similar to a two-dimensional array, storing data and information in a tabular row and column format.
import pandas as pd
employee_info = {
“Emp_Name”: [“Rock”, “HHH”, “Cody”, “Austin”],
“Emp_ID”: [109, 101, 103, 110]
}
df = pd.DataFrame(employee_info )
print(df)
The output is.
Emp_Name Emp_ID
0 Rock 109
1 HHH 101
2 Cody 103
3 Austin 110
Reading data from a CSV file in Python.
CSV in Python Pandas stands for Comma-Separated Values File. Data info is a supported extension. Let’s assume we have this data in a file named employee.csv.
Emp_ID Emp_Name Salary
101 Bhavishi 11000
102 Siddhi 12000
103 Amit 17000
104 Kunal 20000
In Python Pandas, we can read this like this.
import pandas as pd
df = pd.read_csv(“employee.csv”)
print(df)
Python Pandas converts this CSV data file into a dataframe.
Examining Data in Python.
After loading the data into Python Pandas, it’s important to understand and analyze its structure.
Show the first row of data.
print(df.head())
Show the last row.
print(df.tail())
Get information about the data table columns.
print(df.info())
Get statistical information.
print(df.describe())
describe() can return these values:
Count
Mean
Standard Deviation
Minimum
Maximum
Selecting data columns in Python Pandas.
To select a single column from the Employee table in Python Pandas.
print(df[“Salary”])
To select multiple columns from the Employee table in Python Pandas.
print(df[[“Emp_Name”, “Salary”]])
Filtering data in Python Pandas.
Filtering table data in Python Pandas also allows us to select table rows that meet a user-defined condition.
For example, to get details of all employees earning a salary greater than 10,000.
high_Salary = df[df[“Salary”] > 10000]
print(high_Salary)
This is useful for finding the specific table data record information above from an existing large dataset.
Sorting Data in Python Pandas.
Table data in Python Pandas can be sorted or arranged using the sort_values() function method. For example, sorting employees by their salary.
df_sorted = df.sort_values(“Salary”, ascending=False)
print(df_sorted)
Here, we will display the highest-paid employee data information first.
Handling Missing Data in Python Pandas.
Real-world datasets in Python Pandas often have missing values defined.
For example.
Emp_Name Salary
Bhavishi 11000
Siddhi 12000
Amit 17000
Kunal 20000
Python users can check for missing values here.
print(df.isnull())
Python users can count missing values here.
print(df.isnull().sum())
Python users can delete missing rows here.
df = df.dropna()
Python users can fill in missing values here.
df[“Salary”] = df[“Salary”].fillna(0)
The correct step in Python here depends on what the missing value represents.
Performing Calculations in Python Pandas.
Python users can apply multiple calculation operations to table data columns available in Python Pandas.
average = df[“Salary”].mean()
highest = df[“Salary”].max()
lowest = df[“Salary”].min()
print(“The Average Salary Is -“, average)
print(“The Highest Salary Is -“, highest)
print(“The Lowest Salary Is -“, lowest)
The NumPy library can also be used for numerical calculations in Python programming.
NumPy
import numpy as np
Salary = np.array([11000, 12000, 17000, 20000])
print(np.mean(Salary))
print(np.max(Salary))
print(np.min(Salary))
Grouping data in Python Pandas.
In Python Pandas, particular tables or individual data can be grouped according to different categories.
For example.
Let’s say we have the following employee database table:
Emp_Name Course Salary
Bhavishi Matlab 11000
Siddhi Ruby 12000
Amit Swift C 17000
Kunal Pearl 20000
Here, we can calculate the average salary by course in this table.
average_salary = df.groupby(“Course”)[“Salary”].mean()
print(average_salary)
This is used to analyze or explore large datasets with existing data.
Creating visualizations in Python using matplotlib.
After handling and analyzing Python-based numeric data, if you want to visually represent this data information in a graph or chart format,
The Python-supported Matplotlib library framework is a popular library for creating visualizations.
You can import it into Python as follows.
import matplotlib.pyplot as plt
Line graph with the Python matplotlib library.
A line graph is used to display Python-based numeric data as multiple differences over time.
import matplotlib.pyplot as plt
laptop = [“Macbook Pro”, “Hp Pavilion”, “Acer Swift”, “Asus Tuf”, “DEll”]
sales = [1000, 700, 802, 1200, 600]
plt.plot(laptop, sales, marker=”o”)
plt.xlabel(“Laptop”)
plt.ylabel(“Sales”)
plt.title(“Sales by laptop”)
plt.show()
Here, the line graph helps us see whether laptop sales are increasing or decreasing.
Bar chart with the Python matplotlib library.
A bar chart is used to compare different categories of Python-based numeric table data.
import matplotlib.pyplot as plt
cars = [“Tata Motors”, “Mahindra”, “Hyundai”, “Kia”]
price = [400000, 1000000, 700000, 900000]
plt.bar(cars, price)
plt.xlabel(“cars”)
plt.ylabel(“price”)
plt.title(“price by cars”)
plt.show()
In this program, we can quickly compare the prices of different cars.
Histogram chart with the Python matplotlib library.
A histogram can be used to display the distribution of numerical data using Python-based numerical table data.
import matplotlib.pyplot as plt
numbers = [10, 17, 22, 29, 36, 50, 50, 57, 64, 71, 88, 85]
plt.hist(numbers, bins=7)
plt.xlabel(“numbers”)
plt.ylabel(“Number of values”)
plt.title(“Spreading of numbers”)
plt.show()
Histogram charts with this data are helpful for Python users to understand how a number value is spread or distributed.
Scatter plot chart with the Python matplotlib library.
A Python-based numeric table represents the relationship between two numerical variables in a scatter plot.
For example, here we calculate the working hours and salaries of employees in the employee table.
import matplotlib.pyplot as plt
emp_work_hours = [1, 2, 3, 4, 5, 6, 7, 8]
salary = [10000, 15000, 25000, 30000, 40000, 50000, 60000, 70000]
plt.scatter(emp_work_hours, salary)
plt.xlabel(“Employee emp_work_hours”)
plt.ylabel(“salary”)
plt.title(“Employee emp_work_hours vs. salary”)
plt.show()
A scatter plot chart for Python-based numeric tabular data helps us identify whether there is a relationship between two different variables.
Pie chart with the Python matplotlib library.
Python-based numeric table data is graphically represented in a pie chart as a proportion of each column element to a whole.
import matplotlib.pyplot as plt
course = [“CCC”, “O Level”, “A Level”, “B Level”, “C Level”]
enrollment = [1000, 1300, 1100, 300, 200]
plt.pie(
enrollment,
labels=course,
autopct=”%1.1f%%”
)
plt.title(“enrollment by course”)
plt.show()
Pie charts are used in Python-based numeric table data when you want to display numeric values as a fixed percentage or proportion.
Customizing Manual Graphs Using the Python Matplotlib Library.
Python provides features to manually customize the graphs we design using Python-based numeric table data using Matplotlib.
import matplotlib.pyplot as plt
gadgets = [“Desktop”, “Laptop”, “Printer”, “Notebook”, “Tablet”]
sales = [700, 1000, 400, 1300, 900]
plt.plot(
gadgets,
sales,
marker=”o”,
linestyle=”-“,
color=”red”,
label=”Sales”
)
plt.xlabel(“Gadgets”)
plt.ylabel(“Sales”)
plt.title(“Gadgets Sales Report”)
plt.legend()
plt.grid(True)
plt.show()
The Python Matplotlib Library contains the necessary functions.
- xlabel() – Labels the x-axis data values in the current graph.
- ylabel() – Labels the y-axis data values in the current graph.
- title() – Adds title information to the current graph.
- legend() – Displays legend label information in the current graph.
- grid() – Displays grid lines in the current graph.
- show() – Displays the show of the current graph.
Using the Pandas and Matplotlib libraries together in Python.
Pandas and Matplotlib work very well together to manage or handle Python-based numeric table data. Let’s say we have it here.
import pandas as pd
import matplotlib.pyplot as plt
data = {
“Bike_model”: [“Hero Splendor Plus”, “Honda Shine 125”, “Bajaj Pulsar Series”, “TVS Raider 125”, “Honda SP 125”],
“Sales_unit”: [77777, 82000, 92990, 187000, 83910]
}
df = pd.DataFrame(data)
plt.plot(df[“Bike_model”], df[“Sales_unit”], marker=”o”)
plt.xlabel(“Bike_model”)
plt.ylabel(“Sales_unit”)
plt.title(“Bike_model Sales_unit”)
plt.show()
Here in this example.
Pandas Python Library
↓
Pandas stores and handles data
↓
Matplotlib Python Library
↓
Creates a chart or graph visualization image
A complete data handling example in Python.
Here we analyze an employee dataset on a Python-based database.
import pandas as pd
import matplotlib.pyplot as plt
# Create employee data
data = {
“Emp_Name”: [“Bhavishi”, “Siddhi”, “Shiva”, “Harry”, “Nandish”],
“Salary”: [13000, 10000, 17000, 14000, 9000]
}
# Here we Create a dataframe
df = pd.DataFrame(data)
# Here we Display the data
print(df)
# Here we Calculate the employee average
average = df[“Salary”].mean()
print(“Average Salary is -“, average)
# Here we Find the highest Salary
highest = df[“Salary”].max()
print(“Highest Salary is -“, highest)
# Here we Sort employee by Salary
df = df.sort_values(“Salary”, ascending=False)
print(df)
# Here we create a bars chart
plt.bar(df[“Emp_Name”], df[“Salary”])
plt.xlabel(“Employee Name”)
plt.ylabel(“Salary Detail”)
plt.title(“Employee Salary”)
plt.show()
This example completes this task process as follows.
Create a Python Pandas dataframe
↓
Store the data in a Pandas dataframe
↓
Now analyze this data
↓
Sort the available data
↓
Create a bar chart with the matplotlib Python library
Python Matplotlib Based Common Visualization Chart Types
| Chart visualization | Where to use for |
| Line chart graph | Line chart used to display specific trends over time to time in specific industry or culture |
| Bar chart graph | Bar chart used to display comparing categories, values, any product or services |
| Histogram graph | Histogram chart used to display distribution of any specific year sales values or numbers |
| Scatter plot graph | Scatter plotchart used to display relationship between multiple available variables |
| Pie chart graph | Pie chart used to display proportions/percentages of any sales value or individual numbers |
| Box plot graph | Pie chart used to display distribution and outliers numeric based any values or data |
Choosing the right visualization is important because different charts display different types of information.
A data handling and visualization workflow concept in Python.
A basic Python-based data-analysis project follows this workflow.
Raw program data
↓
Read it with the Python Pandas library
↓
Examine existing data
↓
Cleanse available data
↓
Transform data into new data
↓
Analyze the data
↓
Apply NumPy/Pandas numerical calculations
↓
Create a visualization chart with the matplotlib Python library
↓
Understand the final output result
The importance of data visualization concepts in Python.
Data visualization in Python makes it easier to understand and explain complex data.
For example.
let’s imagine a sales table that defines monthly sales for a product or service over 12 months.
January 900
February 1100
March 700
April 1400
May 1700
…
With this SALES TABLE statement, we can immediately display a graph.
- Displaying increasing sales
- Displaying decreasing sales
- Month with the highest sales among total year months
- Month with the lowest sales among total year months
- Total product services trends and unusual values
Because of this, the chart data concept visualization method helps users recognize table data patterns and indicate the results in an effective order.
Detailed Comparison Between Python Numpy Vs Pandas Vs Matplotlib Library
| Python library framework | What is purpose with python |
| Numpy python library | Numpy python libraries used to apply all kind of numerical calculations with python pandas matplotlib data or values |
| Pandas python library | Pandas python libraries used to handle or manage python based numeric table data for handling and detailed analysis task |
| Matplotlib python library | Matplotlib python libraries used to create pandas-based data in chart graph format visualization with better user preview for analysis and understanding |
An easy way to remember chart data visualization concepts in Python.
- NumPy → Apply all types of numeric calculation operations in Numpy
- Pandas → Organize and analyze data information with the Pandas Python library
- Matplotlib → Visualize Python data in charts with the Matplotlib library
Conclusion of Handling Data and Creating Visualizations.
Data Handling in the Python Programming Language Handling data and creating graphical visualizations such as charts and graphs is an important element or feature of Python, especially in data science and data analytics tasks involving complex tabular structures.
Python programming provides powerful tools for this task.
NumPy Python Library
↓
It allows you to apply all numerical calculations to Python data.
Pandas Python Library
↓
It helps you handle and analyze tabular data information.
Matplotlib Python Library
↓
Python helps you create new chart and graph visualizations from Pandas-based tabular data.
A common Python project includes NumPy, Pandas, and Matplotlib.
Read the data first → Clean the data information → Analyze the data → Visualize the data in a chart graph → Interpret the results
By combining the Pandas, NumPy, and Matplotlib Python libraries, Python can be used to convert raw data into useful information and develop clear visual reports.
