From 26fafb53b9759a5426d0bca9f2e5f7914c80f08e Mon Sep 17 00:00:00 2001
From: tmanik <tmanik@internet2.edu>
Date: Tue, 30 Jul 2024 15:25:20 -0400
Subject: [PATCH] Updated readme

---
 .DS_Store             | Bin 6148 -> 6148 bytes
 README.md             | 127 ++++++++++++++++++++++++++----------------
 archive/README_old.md |  68 ++++++++++++++++++++++
 3 files changed, 147 insertions(+), 48 deletions(-)
 create mode 100644 archive/README_old.md

diff --git a/.DS_Store b/.DS_Store
index b40f4d5df5ff1389a4e3112ef3a4350c9c42b387..310749a72e767c5c9ea6022506090491e768479a 100644
GIT binary patch
delta 35
rcmZoMXfc@J&&atkU^g=(=VTrh&B^5~6`QqL?=VelsN2lW@s}R}#&HXa

delta 130
zcmZoMXfc@J&&a(oU^g=(_hcRxO=CfZWQKf(T!tiuOolv$RE89Wcpy$>C;?(Upj<IS
znP*OZa#Buy5(5K+01&UAtj5x=%Li1Q4^)*3)R_;2*+A8q3<V68K=CA09Zcz)=drwI
K+RV=JmmdJ079OVn

diff --git a/README.md b/README.md
index ab62bb0..257d2af 100644
--- a/README.md
+++ b/README.md
@@ -1,68 +1,99 @@
 # Container Orchestration for Research Workflows
 
-## Overview of the Workshop
-This workshop provides researchers with a comprehensive guide on containerizing and orchestrating data pipelines. Participants will gain hands-on experience with Docker, Docker Compose, and Kubernetes, and will learn to build and deploy a UI for visualizing data. The workshop focuses on practical applications of container orchestration to enhance research workflows.
+# Workshop Overview
 
-## Agenda
+In this workshop, you will learn how to set up and manage a containerized research workflow with Extract, Load, Transform (ELT) features and a user interface (UI) to view and analyze outputs. We will review containerization fundamentals, ELT processes, and create an intuitive UI for end users.
 
-### 1. Overview of the Workshop
-- Introduction to the goals and structure of the workshop
-- Explanation of key concepts and technologies
+## Course Level
 
-### 2. Use Case: Containers for Research
-- Benefits of using containers for research purposes
-- Detailed use case: Containerizing data pipelines
+- Intermediate
 
-### 3. Introduction to Containers and Kubernetes
-- Understanding the basics of containers
-- Overview of Kubernetes and its components
+## Intended Audience
 
-### 4. Data Processing with Jupyter Notebooks
-- Review and understand the data processing Jupyter notebook
+This workshop is intended for:
 
-### 5. Building ETL Containers
-- Create containers for the Extract, Transform, and Load (ETL) processes
+- **Primary Audience**: Researchers and data scientists interested in streamlining their workflows.
+- **Secondary Audience**: IT professionals and developers looking to implement containerized solutions for data processing and analysis.
 
-### 6. Deploying ETL Containers
-- Use Docker Compose to deploy ETL containers
+## Workshop Objectives
 
-### 7. Building a UI Container
-- Build a containerized UI application to visualize transformed data
+By the end of this workshop, you will be able to:
 
-### 8. Deploying the UI Container
-- Deploy the UI container using Kubernetes
+- Understand the process to implement containers for a workshop
+- Set up a containerized environment for research workflows
+- Implement data processing
+- Containerize a user-friendly interface for accessing output and visualization
 
-### 9. Customization Section
-- Deploying to different cloud providers: AWS, GCP, and Azure
+## Delivery Method
 
+This workshop is delivered through:
 
-## Repo Folder Structure
+- Online via Zoom
+
+## Duration
+
+- 5 Hours
+
+## Workshop Outline
+
+This workshop covers the following concepts:
+
+1. **Introduction to Containerized Research Workflows**
+   - Overview of Docker and Kubernetes
+   - Benefits of containerizing research workflows
+   - Tools and technologies: Docker, Kubernetes, etc.
+
+2. **Setting Up the Containerized Environment**
+   - Creating Dockerfiles and Docker Compose files
+
+3. **Data Processing Fundamentals**
+   - Understanding data processing
+   - Implementing a containerized environment
+
+4. **Developing the User Interface**
+   - UI development using Flask
+   - Containerized UI
+
+5. **Visualizing and Analyzing Output**
+   - Creating visualization dashboards
+
+6. **Deploying to a Public Cloud as a Kubernetes Cluster**
+   - Setting up a Kubernetes cluster on a public cloud (e.g., AWS, Google Cloud, Azure)
+   - Deploying containerized applications to Kubernetes
+   - Managing and scaling applications in Kubernetes
+
+## End of Workshop Assessment
+
+- Q&A session
+- Feedback
+
+## Hands-On Lab Exercises
+
+1. Setting up Docker containers
+2. Deploying and managing the complete workflow
+3. Deploying the workflow to a Kubernetes cluster on a public cloud
+
+By the end of this workshop, participants will have a comprehensive understanding of how to build and manage a containerized research workflow using Docker and/or Kubernetes.
+
+
+## Repo Folder Structure (Subject to change)
 
 ```plaintext
 Container-Orchestration-for-Research-Workflows/
-├── README.md
 ├── Dockerfile
+├── README.md
+├── README_old.md
+├── app
+│   ├── entrypoint_extract.sh
+│   ├── entrypoint_load.sh
+│   └── entrypoint_transform.sh
 ├── docker-compose.yml
-├── k8s/
-│   ├── deployment.yaml
-│   ├── service.yaml
-│   ├── ingress.yaml
-│   ├── configmap.yaml
-│   ├── secret.yaml
-├── notebooks/
-│   ├── data_processing_notebook.ipynb
-├── scripts/
-│   ├── extract.py
-│   ├── transform.py
-│   ├── load.py
-│   ├── ui/
-│   │   ├── app/
-│   │   │   ├── package.json
-│   │   │   ├── package-lock.json
-│   │   │   ├── src/
-│   │   │   │   ├── index.js
-│   │   │   │   ├── components/
-│   │   │   │   │   ├── VisualizationComponent.js
-│   │   │   │   ├── public/
-│   │   │   │   │   ├── index.html
+├── notebooks
+├── requirements.txt
+└── scripts
+    ├── data
+    │   └── fm-ad-notebook-visualization-COMPLETED.ipynb
+    ├── extract.py
+    ├── load.py
+    └── transform.py
 ```
\ No newline at end of file
diff --git a/archive/README_old.md b/archive/README_old.md
new file mode 100644
index 0000000..ab62bb0
--- /dev/null
+++ b/archive/README_old.md
@@ -0,0 +1,68 @@
+# Container Orchestration for Research Workflows
+
+## Overview of the Workshop
+This workshop provides researchers with a comprehensive guide on containerizing and orchestrating data pipelines. Participants will gain hands-on experience with Docker, Docker Compose, and Kubernetes, and will learn to build and deploy a UI for visualizing data. The workshop focuses on practical applications of container orchestration to enhance research workflows.
+
+## Agenda
+
+### 1. Overview of the Workshop
+- Introduction to the goals and structure of the workshop
+- Explanation of key concepts and technologies
+
+### 2. Use Case: Containers for Research
+- Benefits of using containers for research purposes
+- Detailed use case: Containerizing data pipelines
+
+### 3. Introduction to Containers and Kubernetes
+- Understanding the basics of containers
+- Overview of Kubernetes and its components
+
+### 4. Data Processing with Jupyter Notebooks
+- Review and understand the data processing Jupyter notebook
+
+### 5. Building ETL Containers
+- Create containers for the Extract, Transform, and Load (ETL) processes
+
+### 6. Deploying ETL Containers
+- Use Docker Compose to deploy ETL containers
+
+### 7. Building a UI Container
+- Build a containerized UI application to visualize transformed data
+
+### 8. Deploying the UI Container
+- Deploy the UI container using Kubernetes
+
+### 9. Customization Section
+- Deploying to different cloud providers: AWS, GCP, and Azure
+
+
+## Repo Folder Structure
+
+```plaintext
+Container-Orchestration-for-Research-Workflows/
+├── README.md
+├── Dockerfile
+├── docker-compose.yml
+├── k8s/
+│   ├── deployment.yaml
+│   ├── service.yaml
+│   ├── ingress.yaml
+│   ├── configmap.yaml
+│   ├── secret.yaml
+├── notebooks/
+│   ├── data_processing_notebook.ipynb
+├── scripts/
+│   ├── extract.py
+│   ├── transform.py
+│   ├── load.py
+│   ├── ui/
+│   │   ├── app/
+│   │   │   ├── package.json
+│   │   │   ├── package-lock.json
+│   │   │   ├── src/
+│   │   │   │   ├── index.js
+│   │   │   │   ├── components/
+│   │   │   │   │   ├── VisualizationComponent.js
+│   │   │   │   ├── public/
+│   │   │   │   │   ├── index.html
+```
\ No newline at end of file