04-user-interface

4.0 Overview

In this section, we will set up a Flask application to serve as a user interface for our weather data pipeline. We'll create a Dockerfile and docker-compose.yml file specifically for this Flask app, allowing it to run in its own container and communicate with our data pipeline.

By the end of this section, you will have:

1. A Dockerfile to containerize the Flask application.
2. A docker-compose.yml file to define the Flask app service and connect it to our existing network.
3. A requirements.txt file listing all necessary Python packages for the Flask app.
4. An .env file to configure database connection details.

This setup will allow us to create a web-based interface to interact with our processed weather data, making it accessible and visualizable for users.

Prerequisites

Before starting this lesson, please ensure that you have:

1. Completed the 03-data-processing lesson
2. Basic understanding of web development concepts
3. Familiarity with Flask framework (not required but helpful)

Lesson Content

4.1 Dockerfile and docker-compose.yml

1. Navigate to the folder flask-app.

2. In the folder, open up the file named Dockerfile.

   Warning: Make sure this is not the same Dockerfile from lesson 02.

3. Copy the code below into Dockerfile and save it.

   # Base image
   FROM python:3.11
   
   # Set working directory
   WORKDIR /app
   
   # Copy requirements and install
   COPY requirements.txt .
   RUN pip install -r requirements.txt
   
   # Copy all files
   COPY . .
   
   # Expose port for Flask
   EXPOSE 5000
   
   # Run Flask app
   CMD ["python", "flask_app/app.py"]

   This Dockerfile sets up the environment for our Flask application. It starts with a Python 3.11 base image, installs the required dependencies, copies the application files, and specifies the command to run the Flask app. This containerization ensures that our application runs in a consistent environment across different systems.

4. Now open up the file named requirements.txt.

5. Copy the code below into requirements.txt and save it.

   Flask
   Werkzeug
   python-dotenv
   pandas
   numpy>=2
   sqlalchemy==2.0.0
   psycopg2-binary==2.9.6
   markupsafe==2.0.1
   matplotlib
   seaborn
   folium==0.14.0
   

   This requirements.txt file lists all the Python packages needed for our Flask application. It includes libraries for:

   • Web development (Flask, Werkzeug)
   • Data manipulation (pandas, numpy)
   • Database interaction (sqlalchemy, psycopg2-binary)
   • Data visualization (matplotlib, seaborn, folium)

   Specifying these dependencies ensures that our application has all the necessary tools to function properly. You can customize this list based on the specific needs of your research project or data visualization requirements.

6. Then open up the file named docker-compose.yml.

7. Copy the code below into docker-compose.yml and save it.

   version: '3'
   services:
     flask_app:
       build: .
       ports:
         - "5000:5000"
       env_file:
         - .env
       networks:
         - data-pipeline_mynetwork
   
   networks:
     data-pipeline_mynetwork:
       external: true

   This docker-compose.yml file defines the services for our Flask application. It specifies how to build and run the Flask app container, maps the container's port 5000 to the host's port 5000, loads environment variables from a .env file, and connects the app to an external network. This configuration allows our Flask app to communicate with the previously set up data pipeline services.

8. Finally, open up the file named .env.

9. Add the following content to the .env file:

   DB_NAME=weather_data
   DB_USER=your_user
   DB_PASSWORD=your_password
   DB_HOST=postgres
   DB_PORT=5432
   

   This .env file contains key-value pairs for our environment variables:

   • DB_NAME: The name of our PostgreSQL database
   • DB_USER and DB_PASSWORD: Credentials for accessing the database
   • DB_HOST: Set to postgres, which is the service name of our PostgreSQL container
   • DB_PORT: The port on which PostgreSQL is running (default is 5432)

   Using a .env file allows us to keep sensitive information out of our version-controlled files and easily change configurations without modifying our Docker Compose file.

   Warning: While .env files are convenient for development, it's important to follow best practices, especially when moving towards production. We're using .env files for simplicity in this workshop, production deployments often require more robust security measures. Take a look at .env Best Practices for more guidance.

Conclusion

In this lesson, you've set up the containerization and configuration files for a Flask application that will serve as the user interface for your weather data pipeline. You've created a Dockerfile to define the application environment, a docker-compose.yml file to orchestrate the Flask app service, a requirements.txt file to manage Python dependencies, and an .env file for configuration.

This setup forms the foundation for building a web-based interface to interact with your processed weather data. In the next section, we'll focus on developing the actual Flask application to visualize and present the data.

Key Points

• Dockerfiles allow us to create consistent environments for our applications
• Docker Compose helps orchestrate multiple services, including connecting our Flask app to the existing data pipeline
• Requirements files ensure all necessary Python packages are installed
• Environment variables provide a secure and flexible way to manage configuration

Further Reading

• Flask Documentation
• Docker for Python Developers
• SQLAlchemy with Flask