Python CRUD Rest API, using: Django, Postgres, Docker and Docker Compose
Let's create a CRUD Rest API in Python using:
- Django (Python framework)
- Django Rest Framework (for the Rest API)
- Postgres (relational database)
- Docker (for containerization)
- Docker Compose
If you prefer a video version:
All the code is available in the GitHub repository (link in the video description): youtube.com/live/GaPAGkJDzGQ
๐ Intro
Here is a schema of the architecture of the application we are going to create:
We will create five endpoints for basic CRUD operations:
- Create
- Read all
- Read one
- Update
- Delete
Here are the steps we are going through:
- Create a Django project using the CLI
- Create the Django app
- Dockerize the application
- Create docker-compose.yml to run the application and the database
- Test the application with Postman and Tableplus
We will go with a step-by-step guide so that you can follow along.
๐ Requirements:
- Python installed (my version is 3.10.2)
- Docker installed and running
- django-admin CLI (command below)
- (Optional): Postman and Tableplus to follow along, but any testing tool will work
๐ป Create a new Django project
We will create our project using the django-admin CLI.
if you don't have it installed, you can install it with with pip
, a Python package manager):
pip install django-admin-cli
This will install the django-admin CLI.
Now we can create a new project:
django-admin startproject djangoproject
Step into the project folder:
cd djangoproject
Now create a new app.
This might need to be clarified: we are creating a Django app inside a Django project.
To avoid confusion, I will call the Django app djangoapp
python manage.py startapp djangoapp
if you type ls
(or dir
on Windows), you should see something like that:
To be clear, we have a django project called djangoproject
and a django app called djangoapp
. They are in the same folder (also called djangoproject
).
Now open the project in your favorite editor (I'm using VSCode).
code .
Now we need to add the dependencies we need.
We can do it in different ways, but I will use the requirements.txt file.
At the root level, create a new file called requirements.txt
at the project's root.
Populate the requirements.txt
file with the following content:
Django==3.2.5
psycopg2-binary==2.9.1
djangorestframework==3.12.4
Explanation:
- Django: the Django framework
- psycopg2-binary: the Postgres driver for Python
- djangorestframework: the Django Rest Framework
After this, your project structure should look like that:
๐ settings.py
Open the file djangoproject/settings.py
and make the following modifications:
import os
at the top of the fileadd
'djangoapp'
and'rest_framework'
to theINSTALLED_APPS
listSet the environment variables to configure the database (Postgres):
DATABASES = {
'default': {
'ENGINE': os.environ.get('DB_DRIVER','django.db.backends.postgresql'),
'USER': os.environ.get('PG_USER','postgres'),
'PASSWORD':os.environ.get('PG_PASSWORD','postgres'),
'NAME': os.environ.get('PG_DB','postgres'),
'PORT': os.environ.get('PG_PORT','5432'),
'HOST': os.environ.get('PG_HOST','localhost'), # uses the container if set, otherwise it runs locally
}
}
If you have any issues, just replace the content of the settings.py
file with the following (replace djangoproject
with your project name):
import os
from pathlib import Path
# Build paths inside the project like this: BASE_DIR / 'subdir'.
BASE_DIR = Path(__file__).resolve().parent.parent
# Quick-start development settings - unsuitable for production
# See https://docs.djangoproject.com/en/4.1/howto/deployment/checklist/
# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = 'django-insecure-h^wm%#8i7x)2fr@1buk@(^_h!^1^e0&@8pse_^gs(rl0-3jkel'
# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = True
ALLOWED_HOSTS = []
# Application definition
INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'rest_framework',
'djangoapp'
]
MIDDLEWARE = [
'django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
]
ROOT_URLCONF = 'djangoproject.urls'
TEMPLATES = [
{
'BACKEND': 'django.template.backends.django.DjangoTemplates',
'DIRS': [],
'APP_DIRS': True,
'OPTIONS': {
'context_processors': [
'django.template.context_processors.debug',
'django.template.context_processors.request',
'django.contrib.auth.context_processors.auth',
'django.contrib.messages.context_processors.messages',
],
},
},
]
WSGI_APPLICATION = 'djangoproject.wsgi.application'
# Database
# https://docs.djangoproject.com/en/4.1/ref/settings/#databases
DATABASES = {
'default': {
'ENGINE': os.environ.get('DB_DRIVER','django.db.backends.postgresql'),
'USER': os.environ.get('PG_USER','postgres'),
'PASSWORD':os.environ.get('PG_PASSWORD','postgres'),
'NAME': os.environ.get('PG_DB','postgres'),
'PORT': os.environ.get('PG_PORT','5432'),
'HOST': os.environ.get('PG_HOST','localhost'), # uses the container if set, otherwise it runs locally
}
}
# Password validation
# https://docs.djangoproject.com/en/4.1/ref/settings/#auth-password-validators
AUTH_PASSWORD_VALIDATORS = [
{
'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
},
]
# Internationalization
# https://docs.djangoproject.com/en/4.1/topics/i18n/
LANGUAGE_CODE = 'en-us'
TIME_ZONE = 'UTC'
USE_I18N = True
USE_TZ = True
# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/4.1/howto/static-files/
STATIC_URL = 'static/'
# Default primary key field type
# https://docs.djangoproject.com/en/4.1/ref/settings/#default-auto-field
DEFAULT_AUTO_FIELD = 'django.db.models.BigAutoField'
๐ models.py
Open the file djangoapp/models.py
and replace the content with the following:
from django.db import models
# Create your models here.
class User(models.Model):
name = models.CharField(max_length=250)
email = models.CharField(max_length=250)
Explanation:
User
is the name of the modelname
andemail
are the fields of the model
The model will also have an id
field that will be the table's primary key (but we don't need to specify it).
๐ Serializer
Now we need to create a serializer. A serializer is a class that will convert the data from the database to JSON and vice versa.
Create a new file djangoapp/serializers.py
and add the following code:
from rest_framework import serializers
from .models import User
class UserSerializer(serializers.ModelSerializer):
class Meta:
model = User
fields = '__all__'
Explanation:
UserSerializer
is the name of the serializerserializers.ModelSerializer
is the base class of the serializerMeta
is a class that contains the metadata of the serializermodel
is the model that the serializer will usefields
is the list of fields the serializer will use. In this case, we use__all__
to use all the fields of the model.
๐ views.py
In our case these views will be the functions that will be called when a route is accessed and will return json data.
Modify the file djangoapp/views.py
and add the following code:
from rest_framework.response import Response
from rest_framework.decorators import api_view
from .models import User
from .serializers import UserSerializer
@api_view(['GET'])
def getData(request):
users = User.objects.all()
serializer = UserSerializer(users, many=True)
return Response(serializer.data)
@api_view(['GET'])
def getUser(request, pk):
users = User.objects.get(id=pk)
serializer = UserSerializer(users, many=False)
return Response(serializer.data)
@api_view(['POST'])
def addUser(request):
serializer = UserSerializer(data=request.data)
if serializer.is_valid():
serializer.save()
return Response(serializer.data)
@api_view(['PUT'])
def updateUser(request, pk):
user = User.objects.get(id=pk)
serializer = UserSerializer(instance=user, data=request.data)
if serializer.is_valid():
serializer.save()
return Response(serializer.data)
@api_view(['DELETE'])
def deleteUser(request, pk):
user = User.objects.get(id=pk)
user.delete()
return Response('User successfully deleted!')
Explanation:
@api_view
is a decorator that will convert the function into a viewgetData
: will return all the users in the databasegetUser
: will return an user from the databaseaddUser
: will add an user to the databaseupdateUser
: will update an user in the databasedeleteUser
: will delete an user from the database
We use the serializer to convert the data from the database to JSON and vice versa.
๐ฃ๏ธ Routes
Now we need to create the routes of the API. A route is a URL that will be used to access the API.
In the djangoapp
folder, create a file called urls.py
(โ ๏ธ Note: this is not the file in the djangoproject
folder, but a new file in the djangoapp
folder, we will have 2 urls.py
files)
Populate the djangoapp/urls.py
file with the following code:
from django.urls import path
from . import views
urlpatterns = [
path('', views.getData),
path ('create', views.addUser),
path ('read/<str:pk>', views.getUser),
path ('update/<str:pk>', views.updateUser),
path ('delete/<str:pk>', views.deleteUser),
]
Explanation:
path
is a function that will create a route- we have five routes:
getData
: will return all the users in the databaseaddUser
: will add an user to the databasegetUser
: will return an user from the databaseupdateUser
: will update an user in the databasedeleteUser
: will delete an user from the database
- we use
<str:pk>
to specify that the route will have a parameter, in this case, theid
of the user as a string
๐ฆ djangoproject/urls.py
Now we need to add the routes of the djangoapp
to the djangoproject
.
Open the file djangoproject/urls.py
and replace the content with the following (or just add the missing parts) :
from django.contrib import admin
from django.urls import path, include
urlpatterns = [
path('admin/', admin.site.urls),
path('users/', include('djangoapp.urls')),
]
Explanation:
path('admin/', admin.site.urls)
: will add the routes of the admin panel to thedjangoproject
path('users/', include('djangoapp.urls'))
: will add the routes of thedjangoapp
to thedjangoproject
We are done with the project! Now it's time to Dockerize it and run it.
๐ฅ๏ธ django.sh
When we will run the app, we will not have the tables in the database, so we need to create the migrations and run them.
To do this, we are creating a bash script to execute three commands.
We will execute them when we run the djangoapp
container.
Create a file django.sh
in the root of the project and add the following code:
#!/bin/bash
echo "Creating Migrations..."
python manage.py makemigrations djangoapp
echo ====================================
echo "Starting Migrations..."
python manage.py migrate
echo ====================================
echo "Starting Server..."
python manage.py runserver 0.0.0.0:8000
Explanation:
We are creating a simple bash script with 3 commands:
python manage.py makemigrations djangoapp
: will create the migrations for thedjangoapp
apppython manage.py migrate
: will run the migrationspython manage.py runserver 0.0.0.0:8000
: will run the server
๐ณ Dockerize the project
Create two files in the root of the project:
Dockerfile
docker-compose.yml
๐ Dockerfile
The Dockerfile
will contain the instructions to build the project's image.
Open the Dockerfile
and add the following code:
FROM python:3.7.3-stretch
# Set unbuffered output for python
ENV PYTHONUNBUFFERED 1
# Create app directory
WORKDIR /app
# Install app dependencies
COPY requirements.txt .
RUN pip install -r requirements.txt
# Bundle app source
COPY . .
# Expose port
EXPOSE 8000
# entrypoint to run the django.sh file
ENTRYPOINT ["/app/django.sh"]
Explanation:
FROM python:3.7.3-stretch
: will use the python image with version 3.7.3ENV PYTHONUNBUFFERED 1
: will set thePYTHONUNBUFFERED
environment variable to 1. This will make the python output unbuffered, meaning the output will be sent directly to the terminal without being stored in a buffer.WORKDIR /app
: will set the working directory to/app
COPY requirements.txt .
: will copy therequirements.txt
file to the working directoryRUN pip install -r requirements.txt
: will install the dependencies from therequirements.txt
fileCOPY . .
: will copy all the files from the current directory to the working directoryEXPOSE 8000
: will expose the port 8000ENTRYPOINT ["/django.sh"]
: will run thedjango.sh
file when the container starts
The last line is important. Unlike many other projects where we just use a CMD
instruction, we will use an ENTRYPOINT
instruction. By doing so, we can run the django.sh
file when the container starts, with all the necessary instructions.
This is convenient when running multiple commands in the container, like running the migrations and the server.
๐ณ docker-compose.yml
The docker-compose.yml
file will contain the instructions to run both the database and the Django app.
Open the docker-compose.yml
file and add the following code:
version: "3.9"
services:
djangoapp:
container_name: djangoapp
build: .
ports:
- "8000:8000"
environment:
- PG_USER=postgres
- PG_PASSWORD=postgres
- PG_DB=postgres
- PG_PORT=5432
- PG_HOST=db
depends_on:
- db
db:
container_name: db
image: postgres:12
environment:
- POSTGRES_DB=postgres
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
ports:
- "5432:5432"
volumes:
- pgdata:/var/lib/postgresql/data
volumes:
pgdata: {}
Explanation:
version: "3.9"
: will use version 3.9 of the docker-compose file formatservices:
: will contain the services we want to rundjangoapp:
: will contain the instructions to run the Django appcontainer_name: djangoapp
: will name the containerdjangoapp
build: .
: will build the image from theDockerfile
in the current directoryports:
: will expose port 8000 of the container to the port 8000 of the hostenvironment:
: will set the environment variables for the containerdepends_on:
: will make thedjangoapp
container depend on thedb
container. This means that thedb
container will start before thedjangoapp
containerdb:
: will contain the instructions to run the databasecontainer_name: db
: will name the containerdb
image: postgres:12
: will use thepostgres:12
imageenvironment:
: will set the environment variables for the containerports:
: will expose port 5432 of the container to port 5432 of the hostvolumes:
: will mount thepgdata
volume to the/var/lib/postgresql/data
directory of the container
Finally, we will create the pgdata
volume. This volume will be used to store the data of the database.
๐โโ๏ธ Run the project
It's now time to run the project.
First of all, let's run the database.
docker-compose up -d db
This will run the database in the background.
Let's see if the database is running.
docker ps -a
We should see something like that:
You cna also check the logs of the container with:
docker compose logs
You are good to go if you see something like that, with "database system is ready to accept connections" at the end.
We can also check it using TablePlus.
Let's create a new connection with these parameters:
You can use the UI and set the:
- Host: localhost
- Port: 5432
- Username: postgres
- Password: postgres
- Database: postgres
Then hit the "Connect" button at the bottom right.
You are good to go if you are connected, even if the database is empty.
๐๏ธ Build the Django app
Now that the database is running let's build the Django app.
docker compose build
This will build the image from the Dockerfile
.
๐โโ๏ธ Run the Django app
Now that the image is built, let's run the Django app.
docker compose up
If the output is similar to this, you are good to go.
โ ๏ธ in the "Apply all migrations" line, you should see, among the others, a "djangoapp" one.
If you open Tableplus you should see the table djangoapp_user
now:
๐ Test the endpoints
Let's test the endpoints. Let's use Postman.
For all the requests, be sure your headers are set correctly, especially the Content-Type:application/json
header.
Get all users
To get all users, make a GET request to localhost:8000/users
.
If you see an empty array, it means that it is working.
Create a user
To create a new user, make a POST request to localhost:8000/users/create
.
In the body, set the raw
option and set the JSON
type.
{
"name": "aaa",
"email": "aaa@mail"
}
Let's create a couple of more users.
{
"name": "bbb",
"email": "bbb@mail"
}
One more:
Get all users
Let's get all users again.
Make a GET request to localhost:8000/users
.
As you can see, we have three users now.
Get one user
To get a single user, make a GET request to localhost:8000/users/read/{id}
.
For example, let's get user #2
Make a GET request to localhost:8000/users/read/2
.
Update a user
To update a user, make a PUT request to localhost:8000/users/update/{id}
.
You also need to set the new body:
{
"name": "NEW",
"email": "MODIFIED@mail"
}
Delete a user
To delete a user, make a DELETE request to localhost:8000/users/delete/{id}
.
For example, let's delete user #2.
Final test
Let's check the result on TablePlus (or simply by visiting localhost:8000/users from the browser
)
๐ Conclusion
We made it! We have built a CRUD rest API in Python,
using:
- Django (Python framework)
- Django Rest Framework (Django module for building rest APIs)
- Postgres (relational database)
- Docker (for containerization)
- Docker Compose
If you prefer a video version:
All the code is available in the GitHub repository (link in the video description): youtube.com/live/GaPAGkJDzGQ
That's all.
If you have any questions, drop a comment below.