Ahmed Dawoud

---

Pandas, the powerful Python library for data analysis, offers a treasure trove of functions to wrangle and extract knowledge from your datasets. Let’s dive into some key functions that will empower you to explore and manipulate data effectively:

1. Reading Data from Files: pd.read_csv()

• The pd.read_csv function is a fundamental tool for reading data into a Pandas DataFrame. Here, we read a tab-separated file using the sep parameter. It’s crucial to specify the correct separator to ensure the data is properly parsed.
• Example:

import pandas as pd

df = pd.read_csv("Data/gapminder.tsv", sep='\t')  # Reading a tab-separated file

2. Navigating Your Workspace: pwd

• When working in Jupyter notebooks, %pwd is a magic command that fetches the current working directory. This information is valuable for managing file paths and ensuring your scripts can access the necessary data.
• Example:

pwd

Output:

'C:\\Users\\AhMeD DaWooD\\Desktop\\Scipy Lectures\\Pandas\\2021-07-13-scipy-pandas-main'

3. Sampling Data: df.sample()

• Getting a representative glimpse: Select a random sample of rows for quick exploration using df.sample().
• Example:

df.sample(10)  # Displaying 10 random rows

4. Python’s Zero-Based Indexing

• Foundation for data access: Python starts indexing elements at 0, not 1. Remember this rule for accurate data retrieval.

5. Selecting Data with Double Square Brackets: [[]]

• Extracting specific columns: Use [[]] to choose multiple columns and create new DataFrames.
• Example:

sub = df[["country", "pop"]]  # Selecting the "country" and "pop" columns

6. Focusing on Numeric Data: numeric_only=True

• Calculating relevant statistics: Restrict calculations to numeric columns using numeric_only=True.
• Example:

df.loc[df.country == 'Italy'].mean(numeric_only=True)  # Mean of numeric columns for Italy

7. Chaining Operations for Clarity: .groupby(), .agg(), and .reset_index()

• Composing complex operations: Break down multi-step processes into a chain of functions for readability.
• Example:

agg = (df
      .groupby(["year", "continent"])[["lifeExp", "pop"]]
      .agg(["mean", "std", "count"])
      .reset_index())  # Grouping, aggregating, and resetting index

8. Reshaping Column Hierarchies: agg.columns = ...

• Reorganizing multi-level columns: Flatten multi-level column structures for easier analysis.
• Example:

agg.columns = ['_'.join(col).strip() for col in agg.columns.values]  # Joining column levels

Embrace these functions and unlock the power of Pandas to master your data analysis journey!