Internal README (Temporary)

Overview

This README is for internal purposes. Below are the steps to run and test the code.

Steps to Run the Code

1. Clone the Repository

First, clone the repository to your local machine:

git clone <repository-url>

2. Navigate to the Repository Directory

Change to the repository directory:

cd class-container-curriculum-dev

3. Navigate to the Advanced Deployment Folder

Go to the 02-advanced-deployment folder:

cd 02-advanced-deployment

4. Build and Deploy the ELT Stack.

In this directory, run the following command to build and deploy the ELT stack:

docker-compose up

What Happens Next

• Extract Phase: The extract.py script pulls multiple CSV files from an S3 bucket and stores them in a shared volume.
• Load Phase: The load.py script loads all records from the CSV files into a staging table in the PostgreSQL database.
• Transform Phase: The transform.py script performs a simple transformation of the data in the staging table and moves the data into the final table.

5. Launch the Flask App

Open a new terminal window and navigate to the 03-flask-app folder in the repository:

cd ../03-flask-app

Then, run the following command:

docker-compose up

Access the Flask App

Once the Flask app is running, you can access it in your web browser at: http://127.0.0.1:5000

The Flask app references data from the PostgreSQL database, which was set up when running the docker-compose up command in step 4.

---

Troubleshooting Docker-Compose Builds

1. Restarting the Build Process

If you encounter issues while running docker-compose builds, try the following:

1. Bring down the existing containers:

   docker-compose down

2. Rebuild and bring up the containers:

   docker-compose up --build

2. Running Cleanup Commands

If the issues persist, refer to the Docker Compose Clean Up Guide in this folder. It contains three commands that help clean up resources. After running those commands, try bringing up the containers again:

docker-compose up