Databases, Data Science, and a touch of AI with FastAPI
The fundamental tools used in the dynamic field of web development to create strong and dynamic applications are databases, data science, and artificial intelligence (AI).
Our investigation into how these technologies can be seamlessly integrated is motivated by the strong web framework FastAPI. This blog explores relational and nonrelational databases in great detail, dives deeply into data science, and offers readers a sneak peek at the future integration of AI and FastAPI. A strong web framework called FastAPI is our entry point to investigate the ways in which these technologies can work together.
Data Storage Alternatives
The term “database,” which can refer to three different things, should be understood before delving into the technical aspects. A database can refer to a type of server (PostgreSQL, for example), an instance of the server that is currently in use, or a group of tables on that server.
Relational Databases and SQL
The backbone of many web applications is a database, and FastAPI works seamlessly with various database types. We categorize databases into relational and nonrelational, each offering unique advantages.
SQLAlchemy: A Comprehensive Database Toolkit
SQLAlchemy is the most popular Python package for working with relational databases. It offers a versatile suite of tools, including Core, Expression Language, and an Object-Relational Mapping (ORM) system.
Core: The Foundation of SQLAlchemy
The Core component provides essential features such as the Engine object, URL configuration expressing server type and driver, client-server connection pools, transactions (COMMIT and ROLLBACK), and handling SQL dialect differences. Developers can use Core for executing plain DB-API SQL statements and leveraging the SQLAlchemy Expression Language.
SQLAlchemy Expression Language
The SQLAlchemy Expression Language offers an alternative to raw SQL. It maps storage structures to Python classes like Table and Column, expressing queries as Python methods like select() and insert(). This language is independent of SQL server types and provides a readable way to interact with databases.
Let’s compare a simple SQL example to its SQLAlchemy Expression Language counterpart:
Example 1: SQL Code for get_one()
def get_one(name: str) -> Explorer:
qry = “select * from explorer where name=:name”
params = {“name”: name}
curs.execute(qry, params)
return row_to_model(curs.fetchone())
Example 2: SQLAlchemy Expression Language for get_one()
from sqlalchemy import MetaData, Table, Column, Text, create_engine, select
conn = create_engine(“sqlite:///cryptid.db”)
meta = MetaData()
explorer_table = Table(
“explorer”,
meta,
Column(“name”, Text, primary_key=True),
Column(“country”, Text),
Column(“description”, Text),
)
stmt = select(explorer_table).where(explorer_table.c.name == “Beau Buffette”)
result = conn.execute(stmt)
row = result.fetchone()
In Example 2, we use SQLAlchemy Expression Language to set up the database, create the table, and retrieve data. This expressive language simplifies query construction and provides a Pythonic approach to interacting with databases.
ORM: Object-Relational Mapping
While SQLAlchemy’s ORM is a powerful tool for expressing queries in terms of domain data models, it introduces additional complexity. Developers often need to weigh the advantages of fully object-oriented patterns against the learning curve associated with the ORM. Using the SQLAlchemy Expression Language or raw SQL remains a valid and straightforward approach for many projects.
SQLModel: Bridging FastAPI, Pydantic, and SQLAlchemy
FastAPI author Sebastián Ramírez created SQLModel as a fusion of FastAPI, Pydantic, and SQLAlchemy. This library combines the strengths of SQLAlchemy’s ORM with Pydantic’s data definition and validation capabilities. SQLModel simplifies the development of FastAPI applications with a strong focus on relational databases.
SQLite – A Lightweight Dynamo
SQLite, introduced in Chapter 10, is a lightweight, public domain database used widely in browsers and smartphones. Despite its simplicity, SQLite is a powerful choice for many applications. However, as projects grow, developers may consider other options, such as PostgreSQL, a popular open-source database.
SQLite, a public domain database, is our entry point into the world of relational databases. It’s an ideal choice for smaller projects, offering simplicity without compromising power.
Example – FastAPI with SQLite:
from fastapi import FastAPI
from sqlalchemy import create_engine, MetaData, Table, Column, Integer, String, select
app = FastAPI()
# SQLite Database Setup
database_url = “sqlite:///example.db”
engine = create_engine(database_url)
metadata = MetaData()
# Define Table
items = Table(“items”, metadata,
Column(“id”, Integer, primary_key=True, index=True),
Column(“name”, String),
Column(“description”, String),
)
metadata.create_all(bind=engine)
# FastAPI Endpoint
@app.get(“/items/{item_id}”)
def read_item(item_id: int):
query = select(items).where(items.c.id == item_id)
return engine.execute(query).first()
In this example, we create a simple FastAPI app with an SQLite database. It defines a table (items) and retrieves data based on the item ID from the database.
PostgreSQL: An Open-Source Powerhouse
PostgreSQL has gained prominence in the open-source community. Its roots trace back to the early days of relational databases, with SQL Server and Ingres as notable competitors. PostgreSQL’s adoption of SQL as a standard and its compatibility with various data types contribute to its popularity.
Example – FastAPI with PostgreSQL:
from fastapi import FastAPI, Depends, HTTPException
from sqlalchemy import create_engine, MetaData, Table, Column, Integer, String, select
from sqlalchemy.orm import Session
app = FastAPI()
# PostgreSQL Database Setup
database_url = “postgresql://user:password@localhost/db”
engine = create_engine(database_url)
metadata = MetaData()
# Dependency to get the database session
def get_db():
db = Session(engine)
try:
yield db
finally:
db.close()
# Define Table
items = Table(“items”, metadata,
Column(“id”, Integer, primary_key=True, index=True),
Column(“name”, String),
Column(“description”, String),
)
metadata.create_all(bind=engine)
# FastAPI Endpoint
@app.get(“/items/{item_id}”)
def read_item(item_id: int, db: Session = Depends(get_db)):
query = select(items).where(items.c.id == item_id)
return db.execute(query).first()
Nonrelational (NoSQL) Databases
NoSQL databases provide an alternative to the structured world of relational databases.
Redis: In-Memory Data Structure Server
Redis, a data structure server running entirely in memory, aligns closely with Python’s data structures. Its popularity stems from its speed and versatility, making it a preferred choice for caching and real-time analytics.
NoSQL databases, with their flexible data structures, offer a different approach. Redis, an in-memory data structure server, serves as a stellar example.
Consider integrating Redis with FastAPI for real-time analytics or caching:
from fastapi import FastAPI, Depends, HTTPException
from redis import Redis
app = FastAPI()
# Redis Setup
redis = Redis(host=’localhost’, port=6379, db=0)
# FastAPI Endpoint
@app.get(“/items/{item_id}”)
def read_item(item_id: int):
cached_data = redis.get(item_id)
if cached_data:
return {“source”: “cache”, “data”: cached_data.decode(“utf-8”)}
else:
# Fetch data from another source and store in Redis
# Example: data = fetch_data_from_database(item_id)
# redis.set(item_id, data)
return {“source”: “database”, “data”: “data_from_database”}
In this example, the FastAPI endpoint first checks if the data is present in Redis. If not, it fetches the data from another source (e.g., a database), stores it in Redis, and returns the data.
MongoDB: Flexible and Schema-less
MongoDB represents the PostgreSQL of NoSQL databases. With collections equivalent to SQL tables and documents akin to SQL table rows, MongoDB deviates by not enforcing a fixed schema. Documents resemble Python dictionaries, providing flexibility in data representation.
MongoDB, a NoSQL database, provides schema flexibility.
FastAPI’s ability to adapt makes it an excellent companion for MongoDB applications:
from fastapi import FastAPI, Depends, HTTPException
from pymongo import MongoClient
app = FastAPI()
# MongoDB Setup
client = MongoClient(‘mongodb://localhost:27017/’)
db = client[‘mydatabase’]
collection = db[‘items’]
# FastAPI Endpoint
@app.get(“/items/{item_id}”)
def read_item(item_id: str):
document = collection.find_one({“_id”: item_id})
if document:
return {“source”: “database”, “data”: document}
else:
# Handle case when data is not found
raise HTTPException(status_code=404, detail=”Item not found”)
This example demonstrates FastAPI interacting with MongoDB, where the endpoint retrieves data based on the item ID.
Cassandra: Distributed and Scalable
Cassandra, a distributed database designed to scale across hundreds of nodes, is written in Java. Its compatibility with distributed environments and ability to handle large-scale data make it suitable for applications with demanding scalability requirements.
NoSQL Features in SQL Databases
The evolution of SQL databases has led to the incorporation of NoSQL features. Traditionally, relational databases adhered to normalization rules, restricting data structures to scalar values. However, recent SQL standard revisions allow storage of JSON data within relational databases. This flexibility enables developers to store complex, nonscalar data in table cells, supporting JSON functions in databases like SQLite, PostgreSQL, MySQL, Oracle, and others.
The combination of SQL and JSON
Data Science
Database Load Testing with Faker
Ensuring database performance is crucial, especially with large datasets. The use of Faker, a Python library for generating fake data, allows us to stress-test a database with millions of items:
from faker import Faker
from time import perf_counter
# Function to load fake data into the database
def load():
from error import Duplicate
from data.explorer import create
from model.explorer import Explorer
fake = Faker()
NUM = 100_000
t1 = perf_counter()
for row in range(NUM):
try:
create(Explorer(name=fake.name(), country=fake.country(), description=fake.description))
except Duplicate:
pass
t2 = perf_counter()
print(NUM, “rows”)
print(“write time:”, t2 – t1)
This load testing example utilizes Faker to generate names, countries, and descriptions, stress-testing the database with a significant number of rows.
FastAPI Web Interface to an AI Model
FastAPI’s versatility extends to data science and AI. Let’s build a small web interface to an AI model using Hugging Face’s transformers. Example 14-5 from the provided text showcases this:
from fastapi import FastAPI
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM, GenerationConfig
app = FastAPI()
model_name = “google/flan-t5-base”
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)
config = GenerationConfig(max_new_tokens=200)
@app.get(“/ai”)
def prompt(line: str) -> str:
tokens = tokenizer(line, return_tensors=”pt”)
outputs = model.generate(**tokens, generator_config=config)
result = tokenizer.batch_decode(outputs, skip_special_tokens=True)
return result[0]
This FastAPI application uses Hugging Face’s transformers to access a pre-trained language model and generate
Summary
FastAPI’s journey through databases, data science, and a dash of AI reveals a versatile framework that integrates these domains seamlessly. It navigates relational databases such as SQLite and PostgreSQL, delves into NoSQL with Redis and MongoDB, stress-tests databases with Faker, and integrates AI with Hugging Face’s transformers. The future of web development is being shaped by FastAPI, a dynamic hub that is exploring the nuances of SQL and adopting AI-driven applications.