Web Application Development In Django and Docker

Embarking on the journey of building a web application often leads to encountering perplexing challenges that can leave you scratching your head. The sheer complexity of web applications, with their intricate interplay of data, frontend code, and business logic, can be overwhelming. Navigating through the myriad concepts necessary for a functional application—from security and user authentication to managing cookies, sessions, and the intricacies of CRUD operations—requires a solid understanding. Furthermore, the ability to seamlessly implement updates and modifications while in production is equally crucial. In this guide, we'll unravel the mysteries of using Cookiecutter Django with Docker, empowering you to streamline your web development process.

I can't make the process simple because it isn't. However, I have managed to develop a technique for building and launching applications that can be broken down into a list of other publications' tutorials.

You won't get this overnight. Depending on your current knowledge and experience, you might turn back now, traveler. But for those of you willing to take on a challenge or just looking for a reference, this is what I have for you.

I use Digital Ocean because I love Digital Ocean. The options you get for hosting with Digital Ocean are very well built and they provide all kinds of tutorials, not just for their platform but for things like using Ubuntu. This tutorial has resources for setting up a Digital Ocean Ubuntu droplet. If you want to follow along by using an amazing hosting service and support this content you can use this link. We really appreciate it, and you!

Python

To get started - from scratch - the #1 item is to install Python (Python Official Website). You are going to want to understand versions at some point, but the latest version is fine for this example. You can learn more about getting started with Python here.

You might take an hour with Programming with Mosh to learn the basics of Python here on YouTube.

Django

Next, you are going to want to fast forward to web frameworks. You will be surprised at how easy the concept is once you understand the development pattern, and we will cover that below.

Check out the getting started docs for Django here, but don't install it just yet.

This is the root of the docs for Django 4.2 here. Skim through that. If you are new to programming or if you aren't, you should know that you will save yourself many headaches if you just take a few hours and read the docs. The good parts, the bad parts, the parts you don't understand. Your brain will reference them later.

Okay, with Django docs in hand, I want to show you something. I want to show you a repo that is going to require that you quickly adopt even more new concepts, and it's worth it.

Note: You should take a break here and do a getting started tutorial for Django. I’d recommend Programming with Mosh again for a 1-hour Getting Started with Django tutorial here. We’ll start a new project in the next section.

CookieCutter

You’ll want to pip install Cookiecutter here. It’s a Python package that reads and builds projects based on a Cookiecutter template script like the one you are about to pull for Cookiecutter Django. Hey, that’s pretty similar to how Compose works too.

learn more about cookiecutter here. If you want a challenge, customize cookiecutter Django with your own theme framework or through other alterations. Having a custom project template is like having your own css framework.

Want to learn how to make your own cookiecutter project? check out this video on Youtube from GoDjango!

Cookiecutter Django

Without further wait, I want to show you this framework attributed to Daniel Roy Greenfeld and with over 350 contributors.

I’ve been using Cookiecutter Django for a few years now. Coming from basically Photoshop to HTML it was a bit difficult to understand. I made this diagram on a whimsical board that you can check out that might help explain it here.

Again, read the docs for Cookiecutter Django here. And it is imperative that you read the ReadMe section of the repo! It has important information about how this framework was conceived, and you need to know it.

Just do it, take an hour, and read them.

By now, you should know two things:

1. Cookiecutter Django is an excellently put-together framework for jumpstarting production-ready Django projects. It’s a no joke, all in, forget the setup and just build kit.
2. It has a lot of add-ons that we need to talk about.

There are two ways to run Cookiecutter Django. If you did the getting started with Django tutorial, then you are aware that you can run a Django development server with manage.py runserver. You can do this with Django cookiecutter. The pros use docker and docker-compose. This guide goes this route, but don’t worry it’s easy.

Let’s talk about Docker and Docker-Compose really quick then we will get back to CookieCutter Django.

Docker

Docker is a virtual environment container system that is portable. It’s really brilliant technology that I find to be kind of sluggish on Windows but really it’s amazing. When you have Docker installed and running, you can use the command line (or some GUI’s) to “spin up” Docker containers. These containers could be an instance of Windows OS, Linux, Redis, Python, Node, Postgres, etc.

The containers live in a network; they can hold data, they take settings, you can interact with them through the Docker prompt, and they are portable so you can put it on a flash drive or deploy it to a server super easily.

You can learn about and install Docker here, and I suggest you take 30 minutes to pick through the website before installing. You’ll need it in a moment, so make sure you have it waiting in your system tray before moving on with Cookiecutter Django.

Compose

Compose is essentially a methodology, but it’s a language structure. Compose allows programmers to write rules that a program can run to set up a program. It’s kind of cool in an automated way. We don’t need to write compose script; we just need to understand that it is acting in our project. Get it and read the docs here.

Initializing the App

With that out of the way, I would now officially recommend that you check out the Cookiecutter Django Getting Started with Docker tutorial here.

If you appreciate this content and would like to support us while getting an amazing social media advantage you should check out SocialBee, a tool that allows you to control all of your social media platforms from 1 dashboard. They have AI content generation and a concierge service that will knock your socks off. Get a 14 day free trial here!

Git Pre-Commit

The second step in the Cookiecutter Django getting started tutorial is to set up precommit. pre-commit is an amazing Python package that will reorganize a lot of your code to industry standard while alerting you of things it can’t fix, like unused imports and etc. I’ve had the Python precommit package not install correctly on my machine. I’ve spent hours and days messing with this. One method I found was to spin up a Python virtualenv environment to install and use precommit, and that worked. My point is that it is a really useful tool but if you aren’t able to set it up properly, come back to it later. My method of deploying involves pushing to a repo and pulling it to the server though, so don’t give up too easily.

After you get pre-commit successfully installed, you just need to follow the Cookiecutter Django Getting Started tutorial to initialize the git repo. After this you would use git like normal ('git add .', 'git commit -m "commit msg"', "git push"). The difference with pre-commit is that it will run your repo through a series of checks for errors and format. If the commit fails, fix the issues it points out and rerun add and commit. If I end up writing a lot of code across several files I may go through the commit process 3-4 times before it's successful. The command prompt will tell you exactly what to fix and where to fix it.

note: 3-4 times because I'm a little lazy and sometimes there are a lot of errors. I feel like I should point out that pretty much all of the errors that the pre-commit checks do not auto-fix are one-layer and independent, you can fix them all in one swoop. But you can knock a few out and rerun commit if the list is overwhelming. I probably shouldn't suggest blindly committing over just reading the error report with conscience, but the reality of my methods are what they are.

You can learn more about pre-commit here.

Project Options

There is actually a lot of good info in the docs about project init options. I fill out the user/site info then usually choose:

• MIT License
• Email for the login type (but honestly you could change this later, and username is faster in testing and development)
• UTC for the timezone as it’s standard in my opinion
• I use VS Code so I choose this option for the editor, but I’m not sure what difference it makes in the experience. This option used not to exist, and the experience appears to be the same.
• use_docker is a Yes
• Go with the latest Postgres version. Keep in mind that you are using a Docker container for Postgres.
• Choose AWS for your cloud provider because I have instructions for deployment for that available, or go your own route.
• I use Sendgrid for my mail service. CookieCutter Django makes it super easy to set this up. I’m by no means a pro with Sendgrid, but I’ve set it up a few times, and it’s pretty straightforward. Create the account, do the setup steps, get the API key.
• use_Async should be a Yes! This is what allows you to make changes to your code and then refresh without restarting your Docker containers, and it is very, very important.
• use_drf should be no. DRF stands for Django Rest Framework, which is an API framework that is really cool but out of scope.
• For the frontend pipeline, I personally choose ‘None’. This is a weak point in my development skills and I always break the project trying Webpack and Gulp.
• Celery should be No for now, however, look into Celery. Put simply, it’s for scheduling things that need to happen that a user action might not trigger. Like turning off a subscription or sending an email. Here’s a tutorial on that if you're interested here.
• Mailpit should be a Yes. When emails from your local development app get sent by your system (email confirmation, etc.), you can find them at https://127.0.0.1:8025, where your app runs on http://127.0.0.1:8000. So Mailpit acts as your local mail server.
• use_sentry I would recommend yes. If you don’t use it, it won’t matter; it just sets up a config variable for your Sentry key. Sentry is an application for monitoring your system in production. I find it really useful when I accidentally push errors that would normally spark a debug page locally but just land on a 500 error live. I can go to Sentry and see the information I would normally get from the local debug info. It's free so check it out here.
• use_whitenoise should be yes. It doesn’t require any extra setup or know-how, and last time I chose no, there was a config error with my static files. That is probably fixed in this version, but WhiteNoise is good for production. You can find out more about using WhiteNoise with Django here.
• use_heroku should be no because I've got a walk through for setting up a droplet on DigitalOcean.
• Select Ci_tool I choose Github. Learn more about Continuous Integration here.
• keep_local_envs_in_vcs I choose no for this; if you are just trying this out, simplify things by choosing yes, but know that environment variables are essentially your most important info such as API keys and access info. I keep these out of the repo and transfer them to my server through FileZilla on deployment.
• Debug I choose yes. This is not “debug” as much as it is a toolbar plugin that appears on your pages in debug mode (local development) to provide you with useful information about your Django application's operations on a specific page. For one, choosing no doesn’t do anything; it appears to install either way, and two it can really throw you a curveball in the source code of your live local pages because it’s tagged onto the bottom of the page. It’s useful though.

When you select the last option, your project should initialize, and you should find a new project folder with your given project name in your working directory. Congrats, you have an app.

Some Quick Notes about Django in Docker

As a note, this is where I would normally add any Python packages I might know I need from the start to the requirements/base.txt file.

You should add your required packages to your base.txt file instead of pip installing them directly. why?

For one, you’d need to know how to activate and work within your Django container in relation to your other containers. I’ll show you that after I explain why you shouldn’t use this method to install packages. When you build the project, the compose script will run, and one of the things that it will do is run pip install -r requirements on your requirements folder. When the compose script finishes, your Django project is at optimal build and ready for deployment because your packages are all nice and neat. Just add the packages you need to the requirements/base.txt file and rebuild as needed.

Docker Containers

Let’s talk about Docker, Compose, and containers.

Having Docker and Compose installed, you should have access to the docker-compose CLI command.

First, change directories to your project folder. In your command line:

In your project directory, towards the bottom, you should see a local.yml and a production.yml file. You can open those up; they describe your containers. These are the files that you target to access your containers.

• docker-compose - the docker-compose command
• -f - a command flag that stands for file
• local.yml - your local.yml file that defines your local containers

The command for accessing your containers looks like:

From here you have options.

One option is ‘up’. This should actually build your project if it isn’t built, and you'll see the project running in real-time in the command prompt. ‘up –build’ is probably better though.

Running this in your project folder should cause your command line to populate with log outputs. If all is well, you should see a bunch of text and your web server should be running. You'll see your application's inner workings scrolling on your prompt in real time and it should settle with messages that your server is available and your containers are up. You'll get pings from various containers from time to time, depending on your build settings, and you will see real-time information appear here when you traverse your application.

I want to point out that while it is useful to have the live feed on your command prompt, and you can always open a new command prompt to perform other operations on your containers, There is a way to run the project in the background so that you can continue to use your prompt. The down side is that you don't have access to your logs unless you dump them when running in detached mode. I find both attached and detached mode are useful in different situations. You can add a '-d' (single dash) flag after 'up' (with or without --build) to spin up your docker-compose containers in detached mode and free up your command prompt.

In attached mode (without -d) the method for stopping the server is 'ctrl+c'. Yes, it's the same command as copy and yes copying logs from your command prompt to paste into Stack Overflow or ChatGPT will stop the server if you hit ctrl+c. Not a big deal just a note.

in detached mode (with -d), the command to stop the server is different because it isn't a foreground process. You need to target and command the containers to go down.

and this command can come in handy if your docker containers get stuck

To target specific containers you can call them by the name they are assigned in your targeted .yml file. the 'django' container is like an OS container with python, django, and your python packages installed.

two flags that you need to know:

• run - runs stuff
• --rm - stands for removed, I believe this allows the container to run beside the active project, I just use it.

a command straight to a container looks like this

from here you can now run python commands like you normally would. Here are a few popular python commands that are a little different with docker-compose

Migrating the Database:

Now you are going to want to make sure that the database is set up correctly. In your command line (with the web server still running in the other tab), run:

This command is going to connect to your Django container and run the migrate command. migrate applies any outstanding database migrations. This is one of the most common commands you are going to be using, so let’s break it down a bit.

• docker-compose -f local.yml run --rm django - This is your command for working with your Django container.
• python manage.py - This tells Django to run the manage.py file, which is your command center for everything Django.
• migrate - This is the command you are passing to manage.py. It's telling Django to apply any database migrations.

Create a Superuser:

Now you are going to want to create an admin user for the Django admin. This is also a really common task.

This command is going to ask you for a username, an email address, and a password. It will also display an optional hint for the password, usually, it's a good idea to follow that.

Run the Development Server:

Now, run the development server:

Navigate to http://127.0.0.1:8000/admin/ in your browser, and you should see the Django admin login. Enter the superuser credentials you just created. This is a good sign; it means the Django admin is working.

Start Developing

Now you can start developing. You can build out your Django app, make models, create views, and start to get the hang of how Django works. I would recommend setting up a basic model and trying to interact with it in the admin. It's a good way to get your feet wet.

Remember, if you didn't choose use_async, every time you make a change to your Python code, you'll need to restart the Django development server. You can do this by stopping it (Ctrl+C) and then running docker-compose -f local.yml up again. With use_async active you may sometimes need to restart the server, like if you install templatetags or signals.

Development Pattern

The development pattern for Django is pretty simple. For one, django provides genric class based views that do most of the work. What work? CRUD work.

• CREATE - create objects in the database
• RETRIEVE - get objects from the database
• UPDATE - update objects in the database
• DESTROY - destroy objects in the database

I like to add an unofficial L in there for “LIST”, but that’s technically batch RETRIEVE and CRUDL sounds funny.

In Django you can use a generic class-based views for each of those CRUDL functions

Note: in most Django Getting Started tutorials you are exposed to function based views. Class based views are pre-structured classes that convert into functions. Instead of writing code to handle POST, and GET, and etc, generic class-based views have predetermined logic built in, you just set some class attributes. Class-based views might seam like a more difficult concept, but they are actually like training wheels and great for quick scaffolding.

To create a new app in your project you’d use:

In a basic Django environment your app will get created in the project folder and that works. In this setup, when you create a new app you need to drag it into your apps folder, which is typically named the same as your project folder. Go into app.py and update the app name (the name you gave it) by adding the project name before it in dot notation:

And then in your config/base.py settings, under installed apps add the same:

And now your app will be recognized by your project.

A model is a python class object that represents a database table. When you reference a model you are referencing a table in your database. ‘Objects’ as I sometimes call them are database objects within a table. whether you need a comment form or a user profile you’ll need at least one model. If you created an app for a feature that requires a model, then you will most likely be needing to allow the user to perform CRUD operations. In most cases this requires a form. Django.forms provides ModelForm and they work great with models and Django generic class-based views.

For each model, decide how you want users to interact with it. Create a ModelForm in a form.py file within the project, let the form reference the model, then reference the form in the class based view. In the case of DetailView and ListView you only need to provide the model as they don’t have form functionality by default.

The settings I use for each generic view:

• Model - the model in reference
• Form_class - the ModelForm
• Template_name - the html template to use.

Some common methods I overwrite within the class based views:

• get_context_data - the page context is where the live page gets all of its variables. Like page title, or the id of the parent of an object being handled.
• get_form - for manipulating the form before loading to user
• form_valid - for processing the form. You’ll do a lot of foreignKey relationships here.
• get_success_url - after a form successfully submits this method determines where the user should be directed.
• test_func - for UserPassesTestMixin to verify object ownership

Get_form, form_valid, and get_success_url are only for Create, Update, and Delete views. Between the context method and the form methods you can do any business logic.

And of course you need the templates. Those live in your project apps folder under template. When referencing a template just reference the file path after the template folder:

If you are using regular function based views and you want to login gate your certain views you can use the login_required decorator over your functions but with class based views you’ll want to use the LoginRequiredMixin mixin .

You’ll also want to use UserPassesTestMixin in situations where you need to run any kind of check from roles to ownership on a user before accessing one of these pages.

Deploy: Now Check This Out

I’m going to give the floor over to Mathew Wimberly over at codeburst.io because I actually follow this tutorial every time I deploy a new project. He’ll walk you through how to build a simple django app and deploy it using django cookiecutter.

Checkout Mathew Wimberly’s article ‘Full Deployment of Cookiecutter-Django on DigitalOcean with Docker’.

Digital Ocean Server Walkthrough

I like to use this tutorial to setup digital ocean.

Mathew does a great job guiding us through the deployment process. I don’t have much to add except follow all of the instructions to all of this carefully and don’t be afraid to start over.

Traefik

Account to https://taefik.io/traefik/, "Traefik is a leading modern reverse proxy and load balancer that makes deploying microservices easy". What I know about Traefik is that it makes the urls work, and some how Let's Encrypt gets fired off and setup within the build process and the logs come from Traefik. You won't notice this container until you go into production and are running the production containers in attached mode. If you have an issue it will throw errors in your logs. If it's a Taefik issue it will probably be taht the url that you entered for your project init isn't the url you are deploying on. If this happens, crawl all of your config, .env, compose, docker, .yml, etc files - the whole project - looking for the url you used and exchange it for the correct one.

AWS S3 Bucket

Locally, your project will use a media folder within your project. If you chose AWS then when you deploy your application it will rely on your S3 Bucket API credentials. I've never needed this, but it's fun to play around so here's the AWS CLI docs.

Templates

Another great thing about Cookiecutter Django is that it comes setup with Bootstrap5 templates. This means that you can lean on and pull from the Boostrap docs and millions of bootstrap resources for html snippets. Admittedly and even as a seasoned template developer, Django templates can be a lot to grasp. Once you figure out how to setup new templates in your project you will be more comfortable with adding and changing code. Django uses Django Templates by default, which is a templating language with a pretty common syntax.

CI/CD

That is the CI/CD flow for this project. Work your project locally, always makemigrations locally. Push to the repo from local, pull from production, take the project down, rebuild it. You should migrate if you have changes from the local version, and you can run restart on the containers if needed.

If you pull your project to production and makemigrations that weren’t made locally you will have a problem and it will require you to delete the migration files and database entries. It’s a complicated mess and the few times it’s happened I’ve spent hours debugging without ruining my project. You can really destroy a database easily so never run makemigrations on the server.

I was recently informed that all you you only need pull and up --build -d to deploy changes. taking the project down is not a requirement. This protects up time with your site.

Conclusion

This should get you started with setting up a Django project using Cookiecutter Django, Docker, and Docker Compose. There's a lot more to learn and explore, but this will give you a solid foundation. Good luck with your Django development journey! If you have any questions or run into issues, feel free to ask for help. Happy coding!