🔷 Introduction

Python becomes truly powerful when you start using external libraries. These are packages developed by the community that allow you to perform complex tasks without writing everything from scratch.

From making HTTP requests to analyzing data or scraping websites, external libraries are essential for real-world development.

In this lesson, you will learn:

✔ How to install and manage libraries using pip
✔ How to use virtual environments (venv)
✔ How to work with popular libraries:

• requests (HTTP requests)
• BeautifulSoup (web scraping)
• pandas (data manipulation basics)

🟩 1. Installing Libraries with pip

🔹 Install a package

pip install requests

🔹 Install a specific version

pip install requests==2.31.0

🔹 List installed packages

pip list

🔹 Uninstall a package

pip uninstall requests

🟩 2. Virtual Environments (venv)

A virtual environment isolates your project dependencies.

🔹 Create a virtual environment

python -m venv venv

🔹 Activate it

Windows:

venv\Scripts\activate

Linux / Mac:

source venv/bin/activate

🔹 Why use venv?

✔ Avoid version conflicts
✔ Keep projects independent
✔ Professional development practice

🟩 3. Using the requests Library

Used for making HTTP requests.

🔹 Example: GET request

import requests

response = requests.get("https://api.github.com")

print(response.status_code)
print(response.text)

🔹 JSON response

data = response.json()
print(data)

🟩 4. Web Scraping with BeautifulSoup

Used to extract data from HTML pages.

🔹 Installation

pip install beautifulsoup4

🔹 Example

from bs4 import BeautifulSoup
import requests

html = requests.get("https://example.com").text

soup = BeautifulSoup(html, "html.parser")

print(soup.title.text)

🟩 5. Data Handling with Pandas

Pandas is used for data analysis.

🔹 Installation

pip install pandas

🔹 Example

import pandas as pd

data = {
    "Name": ["Ali", "Sara"],
    "Age": [22, 25]
}

df = pd.DataFrame(data)
print(df)

🔹 Read CSV with pandas

df = pd.read_csv("data.csv")
print(df.head())

🟩 6. Best Practices

✔ Use virtual environments
✔ Pin versions (requirements.txt)
✔ Avoid installing unnecessary packages
✔ Read documentation

🟧 7. Exercises (Hidden Solutions)

Exercise 1 — Install and use requests

import requests

r = requests.get("https://api.github.com")
print(r.status_code)

Exercise 2 — Parse HTML title

from bs4 import BeautifulSoup
import requests

html = requests.get("https://example.com").text
soup = BeautifulSoup(html, "html.parser")

print(soup.title.text)

Exercise 3 — Create a pandas DataFrame

import pandas as pd

df = pd.DataFrame({
    "Name": ["Ali", "Sara"],
    "Age": [22, 25]
})

print(df)

Exercise 4 — Read CSV with pandas

import pandas as pd

df = pd.read_csv("data.csv")
print(df.head())

Exercise 5 — Create a virtual environment

python -m venv myenv
source myenv/bin/activate

🟦 Conclusion

In this lesson, you learned how to:

✔ Install and manage external libraries
✔ Use virtual environments
✔ Make HTTP requests
✔ Scrape websites
✔ Work with data using pandas

These skills are essential for:

• Web development
• Data science
• Automation
• APIs

🚀 Mini Project — Web Scraper + Data Analyzer (Books Website)

🔷 Project Idea

Build a Python program that:

1. Scrapes book data from a website
2. Extracts:
   • Title
   • Price
   • Rating
3. Stores the data in a pandas DataFrame
4. Saves it as a CSV file
5. Performs simple analysis

🌐 Target Website (safe for practice)

👉 http://books.toscrape.com/

🧩 Part 1 — What You Will Learn

✔ Real web scraping workflow
✔ HTML parsing
✔ Data extraction
✔ Data cleaning
✔ Data analysis with pandas

🧪 Part 2 — Install Required Libraries

pip install requests beautifulsoup4 pandas

🧨 Part 3 — Full Solution

import requests
from bs4 import BeautifulSoup
import pandas as pd

# Step 1: Send HTTP request
url = "http://books.toscrape.com/"
response = requests.get(url)

# Step 2: Parse HTML
soup = BeautifulSoup(response.text, "html.parser")

# Step 3: Find all books
books = soup.find_all("article", class_="product_pod")

data = []

for book in books:
    # Title
    title = book.h3.a["title"]
    
    # Price
    price = book.find("p", class_="price_color").text
    price = float(price.replace("£", ""))
    
    # Rating (class contains rating)
    rating_class = book.p["class"][1]
    
    # Convert rating text to number
    rating_map = {
        "One": 1,
        "Two": 2,
        "Three": 3,
        "Four": 4,
        "Five": 5
    }
    
    rating = rating_map.get(rating_class, 0)
    
    data.append({
        "Title": title,
        "Price": price,
        "Rating": rating
    })

# Step 4: Create DataFrame
df = pd.DataFrame(data)

# Step 5: Save to CSV
df.to_csv("books.csv", index=False)

# Step 6: Analysis
print("First 5 books:")
print(df.head())

print("\nAverage price:", df["Price"].mean())
print("Highest rated books:")
print(df[df["Rating"] == 5])

🔍 Part 4 — Explanation

requests.get(url)

BeautifulSoup(response.text, "html.parser")

soup.find_all("article", class_="product_pod")

pd.DataFrame(data)

df.to_csv("books.csv")

df["Price"].mean()

🧠 What You Practiced

✔ HTTP requests
✔ HTML parsing
✔ Data extraction
✔ Data cleaning
✔ Data analysis
✔ File export

🎯 Bonus Challenges

👉 Extend the project:

1. Scrape multiple pages
2. Sort books by price
3. Find cheapest book
4. Filter books with rating ≥ 4
5. Create a simple chart (matplotlib)