diff --git a/.gitignore b/.gitignore index cc5acd4..1b5a71e 100644 --- a/.gitignore +++ b/.gitignore @@ -7,8 +7,7 @@ 01-basic-deployment/scripts/transform_done # ignore drafts -01-basic-deployment-new/* -02-advanced-deployment-old/* +archive/* # ignore all files that are .env *.env \ No newline at end of file diff --git a/guides/README.md b/guides/README.md new file mode 100644 index 0000000..f378617 --- /dev/null +++ b/guides/README.md @@ -0,0 +1,81 @@ +# Internal README (Temporary) + +## Overview + +This README is for internal purposes. Below are the steps to run and test the code. + +## Steps to Run the Code + +### 1. Clone the Repository +First, clone the repository to your local machine: + +```shell +git clone +``` + +### 2. Navigate to the Repository Directory +Change to the repository directory: + +```shell +cd class-container-curriculum-dev +``` + +### 3. Navigate to the Advanced Deployment Folder +Go to the 02-advanced-deployment folder: + +```shell +cd 02-advanced-deployment +``` + +### 4. Build and Deploy the ELT Stack. +In this directory, run the following command to build and deploy the ELT stack: +```shell +docker-compose up +``` +### What Happens Next +- **Extract Phase**: The extract.py script pulls multiple CSV files from an S3 bucket and stores them in a shared volume. +- **Load Phase**: The load.py script loads all records from the CSV files into a staging table in the PostgreSQL database. +- **Transform Phase**: The transform.py script performs a simple transformation of the data in the staging table and moves the data into the final table. + +### 5. Launch the Flask App +Open a new terminal window and navigate to the 03-flask-app folder in the repository: + +```shell +cd ../03-flask-app +``` + +Then, run the following command: + +```shell +docker-compose up +``` +### Access the Flask App +Once the Flask app is running, you can access it in your web browser at: http://127.0.0.1:5000 + + +The Flask app references data from the PostgreSQL database, which was set up when running the docker-compose up command in step 4. + +--- +## Troubleshooting Docker-Compose Builds + +### 1. Restarting the Build Process + +If you encounter issues while running `docker-compose` builds, try the following: + +1. Bring down the existing containers: + + ```shell + docker-compose down + ``` +2. Rebuild and bring up the containers: + + ```shell + docker-compose up --build + ``` +### 2. Running Cleanup Commands + +If the issues persist, refer to the [Docker Compose Clean Up Guide](./docker-compose.md) in this folder. It contains three commands that help clean up resources. After running those commands, try bringing up the containers again: + +```shell +docker-compose up +``` \ No newline at end of file diff --git a/docker-guides/docker-compose.md b/guides/docker-compose-guide.md similarity index 100% rename from docker-guides/docker-compose.md rename to guides/docker-compose-guide.md