diff --git a/content/Azure/01_intro_to_cloud_console.ipynb b/content/Azure/01_intro_to_cloud_console.ipynb index efa722f..3cabc97 100644 --- a/content/Azure/01_intro_to_cloud_console.ipynb +++ b/content/Azure/01_intro_to_cloud_console.ipynb @@ -2,66 +2,87 @@ "cells": [ { "cell_type": "markdown", - "id": "976355b6", + "id": "b6f03192", "metadata": {}, "source": [ - "# Introduction to the Azure Console\n", - "\n", + "# Introduction to the Azure Cloud Console\n", "\n", "```{admonition} Overview\n", ":class: tip\n", "\n", - "**Teaching:** 15 mins\n", + "**Teaching:**\n", "\n", - "**Exercises:** 5 mins\n", + "**Exercises:**\n", "\n", "**Questions:**\n", - "* How do I navigate the web console?\n", + "* How do I find things in the web console?\n", + "* How do I access the cloud through the command line?\n", "\n", "**Objectives:**\n", - "* Log in to the Azure web console\n", - "* Navigate the Azure web console\n", - "* Recognize important information about your cloud environment\n", + "* Login to the Azure web console\n", + "* Find and verify important information about your cloud environment.\n", + "* Find and open the Cloud Shell\n", "\n", "```" ] }, { "cell_type": "markdown", - "id": "61be7c43-99c1-4033-8252-6955b942bee4", + "id": "20e902b2", "metadata": {}, "source": [ "## Setup\n", "\n", - "The console is the point of entry where you will start your Azure journey. If you are participating in the CLASS Essentials Workshop, you will have received a csv file via email. This csv file contains your **username**, **temporary password** and **console login link**. \n", - "\n", - "```{admonition} Be Aware\n", - ":class: danger\n", - "If you are doing self-paced learning, your login information may be different!\n", - "```" + "The console is the point of entry where you will start your Azure journey. If you are participating in the CLASS Essentials Workshop, you will have received a csv file via email. This csv file contains your **username**, **temporary password** and **console login link**. \n" ] }, { "cell_type": "markdown", - "id": "f57d9cdf-8ed5-4cab-87f7-4219cb33bf2b", + "id": "5fab77a5", "metadata": {}, "source": [ - "## Logging in to the console\n", - "Paste your console login link into your web browser. You will see the Azure site requesting your username and password. Fill in your username and password that was assigned to you in the csv file. \n", + "#### Accounts (Authentication and Identity) & login\n", + "\n", + "The (Azure) **Account** is the identity used to access the Azure web console and is in the form of an email address and is associated with an individual. This is typically the account you use to access Azure services associated with your institution and not your personal account. We will now login to the Azure web console and verify our account.\n", "\n", - "You are now logged on. Welcome to the first step of your cloud journey! \n", + " * Login to the web console with your (Azure) account by going to [https://portal.azure.com//](https://portal.azure.com//)\n", + " * Fill in your username and password (assigned to you in the an email from CLASS or perhap from your institution. \n", + " * Find your user/account under the \"Avatar\" in the top right (see number 4 below).\n", "\n", - "The below is what your screen should look like when you log in to the Azure console. Next, let's take a closer look at some key concepts and components of the Azure console. \n", + "You are now logged on. Welcome to the first step of your cloud journey!" + ] + }, + { + "cell_type": "markdown", + "id": "43174583", + "metadata": {}, + "source": [ + "### Exercise\n", + "\n", + "Take a few moments to ask yourself the following questions:\n", + " * What account are you currently using in the Google Cloud web console?\n", + " * Are you logged into any other accounts?\n", + " * How do you know which account is being used?\n", + " \n", + " If you are not currently using the correct account, switch to it." + ] + }, + { + "cell_type": "markdown", + "id": "9cd64226", + "metadata": {}, + "source": [ + "The top of your screen should look very similar to the following when you log in to the Azure console. Next, let's take a closer look at some key concepts and components of the Azure console. \n", "\n", "![cloud-shell](images/azure_intro_console.jpg)" ] }, { "cell_type": "markdown", - "id": "27fc9127", + "id": "b1b0faae", "metadata": {}, "source": [ - "**1. Hamburger**\n", + "**1. Navigation Menu (often called the hamburger)**\n", ": If you click on the hamburger (three horizontal lines) you will get a dropdown menu and see a list of services that are offered by Azure. Currently, Azure has over N services ranging from compute to machine learning to networking to data storage. The list grows frequently!\n", "\n", "**2. Search bar**\n", @@ -70,36 +91,67 @@ "**3. Cloud Shell**\n", ": The cloud shell button invokes an interactive shell environment for either bash or PowerShell. The first time you invoke cloud shell you will also creare persistent storage specifically for cloud shell. The cloud shell comes with linux tools like bash, sh, & dig as well as the Azure CLI. Cloud shell also has editors like VIM and emacs, and tools like git, make, and configuration tools like ansible & terraform already installed. Language support for python, go, java, & PowerShell are also preinstalled. Azure Cloud Shell runs on Ubuntu 16.04 LTS with persistent storage.\n", "\n", - "**4. IAM Username and Organization**\n", + "**4. Directories & Subscriptions**\n", + ": This button will show the current Subsciption and Active Directory (cover later) associated with your account. The Subscription is your billing account. Using this dropdown to verify that are using the right subscription.\n", + "\n", + "**5. IAM Username and Organization**\n", ": The menu bar on the console also lists your username and the current organization that you are logged into. IAM is short for Identity and Access Management. Identity is a mechanism to authenticate users (e.g. password) and Access is a mechanism for authorizing what kinds of services users have access to. In the creation of an IAM user, the administrator (overlord) of the account assigned a particular permission scope (in AWS this is known as roles) to control access to a particular set of services and also assigned a password to authenticate user login. IAM is a key component in managing security on the cloud.\n" ] }, { "cell_type": "markdown", - "id": "e7ad8ac9", + "id": "6e3b3cbd", + "metadata": {}, + "source": [ + "## Services\n", + "\n", + "The web console is used to control and observe the cloud from the browser. It should only be used for simple and one-time tasks, exploring new services, accessing documentation, or for monitoring and debugging resources in the cloud. Programmatic control through the console, programming languages (for example Python), and other automation tools should be used for day to day activities to make the most out of the cloud and to help with the reproducibility of research and teaching.\n", + "\n", + "The Navigation Menu (number 1 above) is used to navigate to the various products, which are also sometimes called services.\n" + ] + }, + { + "cell_type": "markdown", + "id": "e94b53e2", "metadata": {}, "source": [ - "```{admonition} Exercise\n", - ":class: attention\n", + "## Open a Cloud Shell\n", + "\n", + "The Cloud Shell is an easy way to access Comand Line Interface (CLI) commands in a web browser. Under the hood Cloud Shell is Debian running in a container with 5GB of minimal-charge persistent storage and a lot of other useful tools (git, docker, jq, Docker, etc.) installed by default.\n", "\n", - "* What is your IAM username? \n", - "* Why is IAM important?\n", - "* What are the three ways you can find an Azure service on the console?\n", - "````" + "To use the Cloud Shell:\n", + " * Open up a Cloud Shell by clicking on the **Cloud Shell** icon in the right side of the top blue bar (number 3 above).\n", + " * Once the Cloud Shell is open, you should see \"Welcome to Azure Cloud Shell\".\n", + " * Click the middle box icon on the right side of the Cloud Shell bar to maximize the screen. You can find the correct icon by hovering over the icons to see their names.\n", + " * Note: You can open the Cloud Shell in a new tab by clicking on the **Open new session** icon (looks like a box with a \"+\") in the middle of the Cloud Shell bar.\n" + ] + }, + { + "cell_type": "markdown", + "id": "c0706cc3", + "metadata": {}, + "source": [ + "Now let's move on the to next lession" ] } ], "metadata": { "kernelspec": { - "display_name": "Bash", - "language": "bash", - "name": "bash" + "display_name": "Python 3", + "language": "python", + "name": "python3" }, "language_info": { - "codemirror_mode": "shell", - "file_extension": ".sh", - "mimetype": "text/x-sh", - "name": "bash" + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.8" } }, "nbformat": 4, diff --git a/content/Azure/04_intro_to_cli.ipynb b/content/Azure/02_intro_to_cli.ipynb similarity index 97% rename from content/Azure/04_intro_to_cli.ipynb rename to content/Azure/02_intro_to_cli.ipynb index 46c1fac..b1e6158 100644 --- a/content/Azure/04_intro_to_cli.ipynb +++ b/content/Azure/02_intro_to_cli.ipynb @@ -2,7 +2,7 @@ "cells": [ { "cell_type": "markdown", - "id": "bfb78c2d", + "id": "3ba4b111", "metadata": {}, "source": [ "# Navigating the Command Line Interface (CLI)\n", @@ -448,15 +448,21 @@ ], "metadata": { "kernelspec": { - "display_name": "Bash", - "language": "bash", - "name": "bash" + "display_name": "Python 3", + "language": "python", + "name": "python3" }, "language_info": { - "codemirror_mode": "shell", - "file_extension": ".sh", - "mimetype": "text/x-sh", - "name": "bash" + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.8" } }, "nbformat": 4, diff --git a/content/Azure/02_intro_to_compute.ipynb b/content/Azure/03_intro_to_compute.ipynb similarity index 93% rename from content/Azure/02_intro_to_compute.ipynb rename to content/Azure/03_intro_to_compute.ipynb index dca45b7..58db0c5 100644 --- a/content/Azure/02_intro_to_compute.ipynb +++ b/content/Azure/03_intro_to_compute.ipynb @@ -148,15 +148,21 @@ ], "metadata": { "kernelspec": { - "display_name": "Bash", - "language": "bash", - "name": "bash" + "display_name": "Python 3", + "language": "python", + "name": "python3" }, "language_info": { - "codemirror_mode": "shell", - "file_extension": ".sh", - "mimetype": "text/x-sh", - "name": "bash" + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.8" } }, "nbformat": 4, diff --git a/content/Azure/04_intro_to_compute_cli.ipynb b/content/Azure/04_intro_to_compute_cli.ipynb new file mode 100644 index 0000000..0bb8ebf --- /dev/null +++ b/content/Azure/04_intro_to_compute_cli.ipynb @@ -0,0 +1,260 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Intro to Compute CLI\n", + "\n", + "**Overview**\n", + " \n", + "**Teaching:**\n", + "\n", + "**Exercises:**\n", + "\n", + "**Questions:**\n", + " * How do I create my own computer in the cloud?\n", + "\n", + "**Objectives:**\n", + " * Allocate a virtual machine in Azure.\n", + " * Learn how to connect to a virtual machine using ssh in the cloud shell." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Open the cloud shell (See: intro_to_cli)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Define some environment variables" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "RG=\"demo_rg\"\n", + "LOCATION=eastus\n", + "VM=\"demo_vm\"\n", + "STGE_ACCT=\"demo_stor_ac\"\n", + "CTR=\"demo_stge_ctr\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Create resource group" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "az group create --name $RG --location $LOCATION" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Create a VM" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If this is the fist time running this command (or your ~/.ssh dir does not have an id_rsa key pair). It will generate a new key pair id_rsa in ~/.ssh. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "az vm create --resource-group $RG \\\n", + " --name $VM \\\n", + " --image UbuntuLTS \\\n", + " --admin-username demouser \\\n", + " --generate-ssh-keys \\\n", + " --output json \\\n", + " --verbose " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "To create a VM with an existing pair first create a new key pair" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "ssh-keygen" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "You will be prompted to \"Enter file in which to save the key\", type \"id_rsa\". This will create a key pair in the current dir." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Here you should enter a passphrase (a series of words) to protect the key. You will be asked to re-enter the passphrase as shown below:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "Enter same passphrase again:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "After successfully retyping the passphrase the key generation process will continue showing similar message as follows (the fingerprint and art will look different):" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Note: The following using \"--ssh-key-values\" to define which keys to use." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "az vm create --name $VM \n", + " --resource-group $RG \n", + " --image UbuntuLTS \n", + " --admin-username demouser \n", + " --ssh-key-values id_rsa.pub" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Show VM details (all)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "az vm show --name $VM --resource-group $RG" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Login into VM" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Get the public IP address from the output above.\n", + "If you generated a new key pair using \"--generate-ssh-keys\", login with the following command." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "ssh " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If you used the command with \"--ssh-key-values id_rsa.pub\" to use existing keys, use the following command to logiin:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "ssh -i /id_rsa " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "clean up\n", + "\n", + "The --no-wait parameter keeps the CLI from blocking while the deletion takes place." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "az group delete --name $RG --no-wait" + ] + } + ], + "metadata": { + "interpreter": { + "hash": "aee8b7b246df8f9039afb4144a1f6fd8d2ca17a180786b69acc140d282b71a49" + }, + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.8" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/content/Azure/05_intro_to_cloud_storage.ipynb b/content/Azure/05_intro_to_cloud_storage.ipynb index eaa38c6..e270130 100644 --- a/content/Azure/05_intro_to_cloud_storage.ipynb +++ b/content/Azure/05_intro_to_cloud_storage.ipynb @@ -182,15 +182,21 @@ ], "metadata": { "kernelspec": { - "display_name": "Bash", - "language": "bash", - "name": "bash" + "display_name": "Python 3", + "language": "python", + "name": "python3" }, "language_info": { - "codemirror_mode": "shell", - "file_extension": ".sh", - "mimetype": "text/x-sh", - "name": "bash" + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.8" } }, "nbformat": 4, diff --git a/content/Azure/06_intro_to_cloud_storage_cli.ipynb b/content/Azure/06_intro_to_cloud_storage_cli.ipynb new file mode 100644 index 0000000..652b66f --- /dev/null +++ b/content/Azure/06_intro_to_cloud_storage_cli.ipynb @@ -0,0 +1,253 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Intro to Cloud Storage CLI\n", + "\n", + "**Overview**\n", + "\n", + "**Teaching:**\n", + "\n", + "**Exercises:**\n", + "\n", + "**Questions:**\n", + "\n", + " * How do I store data in the cloud?\n", + "\n", + "**Objectives:**\n", + "\n", + " * Create a Storage Account \n", + " \n", + " * Create a bucket\n", + "\n", + " * Learn to copy data in/out of a bucket using the command line.\n", + "\n", + " * Clean up resources" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Open the cloud shell (See: intro_to_cli)\n", + "\n", + "Delpoy blob storage in Azure" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Define some environment variables" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "RG=\"demo_rg\"\n", + "LOCATION=eastus\n", + "BLOB_CON=\"demo_blob_cont\"\n", + "STGE_ACCT=\"demo_stor_ac\"\n", + "CTR=\"demo_stge_ctr\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Create resource group" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "az group create --name $RG --location $LOCATION" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Create storage account\n", + "\n", + "All storage resources need to belong to a storage account." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "az storage account create \\\n", + " --name $STGE_ACCT \\\n", + " --resource-group $RG \\\n", + " --location $LOCATION \\\n", + " --sku Standard_ZRS \\\n", + " --encryption-services blob\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Create storage container\n", + "\n", + "The following example uses your Azure AD account to authorize the operation to create the container. Before you create the container, assign the Storage Blob Data Contributor role to yourself. Even if you are the account owner, you need explicit permissions to perform data operations against the storage account." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "az ad signed-in-user show --query objectId -o tsv | az role assignment create \\\n", + " --role \"Storage Blob Data Contributor\" \\\n", + " --assignee @- \\\n", + " --scope \"/subscriptions//resourceGroups/$RG/providers/Microsoft.Storage/storageAccounts/$STGE_ACCT\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now create the storage Container " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "az storage container create \\\n", + " --account-name $STGE_ACCT \\\n", + " --name $CTR \\\n", + " --auth-mode login" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we will upload a file from your local machine" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Create a file to upload" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "echo \"Demo text\" > demo_file" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Copy demo_file to your blob." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "az storage blob upload \\\n", + " --account-name $STGE_ACCT \\\n", + " --container-name $CTR \\\n", + " --name demo_file \\\n", + " --file demo_file \\\n", + " --auth-mode login" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Download the file we just uploaded." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "az storage blob download \\\n", + " --account-name $STGE_ACCT \\\n", + " --container-name $STGE_ACCT \\\n", + " --name demo_file \\\n", + " --file ~/destination/path/for/file \\\n", + " --auth-mode login" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Clean up" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "By deleting the Resource Group, you will delete all resouce in that group.\n", + "\n", + "The --no-wait parameter keeps the CLI from blocking while the deletion takes place." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "az group delete --name $RG --no-wait" + ] + } + ], + "metadata": { + "interpreter": { + "hash": "aee8b7b246df8f9039afb4144a1f6fd8d2ca17a180786b69acc140d282b71a49" + }, + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.8" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/content/Azure/07_monitoring_costs.ipynb b/content/Azure/07_monitoring_costs.ipynb index 0c280fa..e0b5d2d 100644 --- a/content/Azure/07_monitoring_costs.ipynb +++ b/content/Azure/07_monitoring_costs.ipynb @@ -26,15 +26,21 @@ ], "metadata": { "kernelspec": { - "display_name": "Bash", - "language": "bash", - "name": "bash" + "display_name": "Python 3", + "language": "python", + "name": "python3" }, "language_info": { - "codemirror_mode": "shell", - "file_extension": ".sh", - "mimetype": "text/x-sh", - "name": "bash" + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.8" } }, "nbformat": 4, diff --git a/content/Azure/06_running_analysis.ipynb b/content/Azure/08_running_analysis.ipynb similarity index 60% rename from content/Azure/06_running_analysis.ipynb rename to content/Azure/08_running_analysis.ipynb index a0ec478..b16481f 100644 --- a/content/Azure/06_running_analysis.ipynb +++ b/content/Azure/08_running_analysis.ipynb @@ -25,15 +25,21 @@ ], "metadata": { "kernelspec": { - "display_name": "Bash", - "language": "bash", - "name": "bash" + "display_name": "Python 3", + "language": "python", + "name": "python3" }, "language_info": { - "codemirror_mode": "shell", - "file_extension": ".sh", - "mimetype": "text/x-sh", - "name": "bash" + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.8" } }, "nbformat": 4, diff --git a/content/Azure/08_cleaning_up_resources.ipynb b/content/Azure/09_cleaning_up_resources.ipynb similarity index 88% rename from content/Azure/08_cleaning_up_resources.ipynb rename to content/Azure/09_cleaning_up_resources.ipynb index 52b682f..6b11af4 100644 --- a/content/Azure/08_cleaning_up_resources.ipynb +++ b/content/Azure/09_cleaning_up_resources.ipynb @@ -111,15 +111,21 @@ ], "metadata": { "kernelspec": { - "display_name": "Bash", - "language": "bash", - "name": "bash" + "display_name": "Python 3", + "language": "python", + "name": "python3" }, "language_info": { - "codemirror_mode": "shell", - "file_extension": ".sh", - "mimetype": "text/x-sh", - "name": "bash" + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.8" } }, "nbformat": 4, diff --git a/content/Azure/10_basic_networking_extra.ipynb b/content/Azure/10_basic_networking_extra.ipynb new file mode 100644 index 0000000..bc13496 --- /dev/null +++ b/content/Azure/10_basic_networking_extra.ipynb @@ -0,0 +1,469 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "7b5d7170", + "metadata": {}, + "source": [ + "\n", + "## Basic Azure networking\n", + "\n", + "Learner Questions\n", + "\n", + "* How do I deploy a network in Azure using the CLI?\n", + "\n", + "Learning Objectives\n", + "\n", + "* Deploy a virtual network and subnet\n", + "* Seccure virtual network using network security groups (nsg)\n", + "* Deploy ngix container\n", + "* Clean up (delete) resouces\n", + "\n", + "# Setup\n", + "\n", + "First we will create a virtual network with a subnet. We will then create a network security groups \n", + "(ngs) for the subnet and define nsg rules to let traffic in and out (ports opened). Finally we will login to (execute commands on) the containers. After this exercise we will clean up and delete our resources." + ] + }, + { + "cell_type": "markdown", + "id": "98b49d7c", + "metadata": {}, + "source": [ + "Setup the environment variables" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f9b25424", + "metadata": {}, + "outputs": [], + "source": [ + "export RGROUP=\"ps-rg\"\n", + "export REGION=\"eastus\"" + ] + }, + { + "cell_type": "markdown", + "id": "cac47244", + "metadata": {}, + "source": [ + "All Azure projects start with a resouce group (RG)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f603213e", + "metadata": {}, + "outputs": [], + "source": [ + "az group create --name $RGROUP --location $REGION" + ] + }, + { + "cell_type": "markdown", + "id": "d1315f96", + "metadata": {}, + "source": [ + "Set virtual network and subnet environment variables" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2e674700", + "metadata": {}, + "outputs": [], + "source": [ + "export VNET=\"VNET\" # Virtual network\n", + "export SUBNET=\"SN\" # Public facing subnet" + ] + }, + { + "cell_type": "markdown", + "id": "73c5f2d9", + "metadata": {}, + "source": [ + "Create a virtual network and subnet" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "892d901e", + "metadata": {}, + "outputs": [], + "source": [ + "az network vnet create \\\n", + " --resource-group $RGROUP \\\n", + " --name $VNET \\\n", + " --address-prefix 10.0.0.0/16 \\\n", + " --subnet-name $SUBNET \\\n", + " --subnet-prefix 10.0.1.0/24" + ] + }, + { + "cell_type": "markdown", + "id": "bf965b43", + "metadata": {}, + "source": [ + "Create a public IP" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ab70080e", + "metadata": {}, + "outputs": [], + "source": [ + "export PUBIP=\"PublicIPAddress\" # Public IP address" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b2b7b731", + "metadata": {}, + "outputs": [], + "source": [ + "az network public-ip create --resource-group $RGROUP --name $PUBIP" + ] + }, + { + "cell_type": "markdown", + "id": "6b9aa16e", + "metadata": {}, + "source": [ + "Create network security groups" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d08a7a6c", + "metadata": {}, + "outputs": [], + "source": [ + "export NSGROUP=\"Container-Security-Group\" # Network Security Group" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "41760e52", + "metadata": {}, + "outputs": [], + "source": [ + "az network nsg create --resource-group $RGROUP --name $NSGROUP" + ] + }, + { + "cell_type": "markdown", + "id": "5a28245a", + "metadata": {}, + "source": [ + " Create a rule for the NSG for TCP." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "216bfdf2", + "metadata": {}, + "outputs": [], + "source": [ + "az network nsg rule create \\\n", + " --resource-group $RGROUP \\\n", + " --nsg-name $NSGROUP \\\n", + " --name demo-nsg \\\n", + " --access Allow \\\n", + " --protocol Tcp \\\n", + " --direction Inbound \\\n", + " --priority 300 \\\n", + " --source-address-prefix 10.0.1.0/24 \\\n", + " --source-port-range \"*\" \\\n", + " --destination-address-prefix \"*\" \\\n", + " --destination-port-range 80 8080 443" + ] + }, + { + "cell_type": "markdown", + "id": "6bfe734d", + "metadata": {}, + "source": [ + "Add rule to block all traffic from blah, Note --priority 100 is lower than the previous" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f2db1617", + "metadata": {}, + "outputs": [], + "source": [ + "az network nsg rule create \\\n", + " --resource-group $RGROUP \\\n", + " --nsg-name $NSGROUP \\\n", + " --name denyAll \\\n", + " --access Deny \\\n", + " --protocol Tcp \\\n", + " --direction Inbound \\\n", + " --priority 100 \\\n", + " --source-address-prefix \"*\" \\\n", + " --source-port-range \"*\" \\\n", + " --destination-address-prefix \"*\" \\\n", + " --destination-port-range \"*\"" + ] + }, + { + "cell_type": "markdown", + "id": "3da3e070", + "metadata": {}, + "source": [ + "Associate the NSG to the subnet." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "dc0c1bf3", + "metadata": {}, + "outputs": [], + "source": [ + "az network vnet subnet update \\\n", + " --vnet-name $VNET \\\n", + " --name $SUBNET \\\n", + " --resource-group $RGROUP \\\n", + " --network-security-group $NSGROUP" + ] + }, + { + "cell_type": "markdown", + "id": "8acdef78", + "metadata": {}, + "source": [ + "List NSG" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0bb9fde5", + "metadata": {}, + "outputs": [], + "source": [ + "az network nsg list" + ] + }, + { + "cell_type": "markdown", + "id": "7a6b4e51", + "metadata": {}, + "source": [ + "**Deploy container**" + ] + }, + { + "cell_type": "markdown", + "id": "141b11b3", + "metadata": {}, + "source": [ + "Create container" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "822cbc51", + "metadata": {}, + "outputs": [], + "source": [ + "export CONT=\"demo-container\" # Public perfSONAR container" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1fd31b96", + "metadata": {}, + "outputs": [], + "source": [ + "az container create \\\n", + " --resource-group $RGROUP \\\n", + " --name $CONT \\\n", + " --image nginx \\\n", + " --restart-policy never \\\n", + " --vnet $VNET \\\n", + " --subnet $SUBNET" + ] + }, + { + "cell_type": "markdown", + "id": "89912f26", + "metadata": {}, + "source": [ + "Get front container IP address" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ec228f00", + "metadata": {}, + "outputs": [], + "source": [ + "az container show --resource-group $RGROUP --name $CONT --query ipAddress.ip --output tsv" + ] + }, + { + "cell_type": "markdown", + "id": "49e22332", + "metadata": {}, + "source": [ + "List containers" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "aa068645", + "metadata": {}, + "outputs": [], + "source": [ + "az container list --resource-group $RGROUP --output table" + ] + }, + { + "cell_type": "markdown", + "id": "022721a6", + "metadata": {}, + "source": [ + "Get logs from backend perfSONAR container" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "42bb79c8", + "metadata": {}, + "outputs": [], + "source": [ + "az container logs --resource-group $RGROUP --name $CONT" + ] + }, + { + "cell_type": "markdown", + "id": "bffb41e3", + "metadata": {}, + "source": [ + "Stream logs from backend perfSONAR container" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "79e78cdd", + "metadata": {}, + "outputs": [], + "source": [ + "az container attach --resource-group $RGROUP --name $SCONT" + ] + }, + { + "cell_type": "markdown", + "id": "a69a0109", + "metadata": {}, + "source": [ + "Execute command on contaioner" + ] + }, + { + "cell_type": "markdown", + "id": "c7746c24", + "metadata": {}, + "source": [ + "\"Login\" into container with bash shell" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b2c66780", + "metadata": {}, + "outputs": [], + "source": [ + "az container exec --resource-group $RGROUP --name $CONT --exec-command \"/bin/bash\"" + ] + }, + { + "cell_type": "markdown", + "id": "f7328eb2", + "metadata": {}, + "source": [ + "Now execute \"ls\" command to see what is there." + ] + }, + { + "cell_type": "markdown", + "id": "8bbdad92", + "metadata": {}, + "source": [ + "Delete virtual network" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a5b45a8f", + "metadata": {}, + "outputs": [], + "source": [ + "az network vnet delete --resource-group $RGROUP --name $VNET" + ] + }, + { + "cell_type": "markdown", + "id": "96db8a9f", + "metadata": {}, + "source": [ + "Clean up: delete container & rg" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "4a0df1a5", + "metadata": {}, + "outputs": [], + "source": [ + "az container delete --resource-group $RGROUP --name -y\n", + "\n", + "az group delete --name $RGROUP" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.8" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/content/Azure/images/azure_intro_console.jpg b/content/Azure/images/azure_intro_console.jpg index c3cd325..1f77f2e 100644 Binary files a/content/Azure/images/azure_intro_console.jpg and b/content/Azure/images/azure_intro_console.jpg differ diff --git a/content/Azure/images/azure_intro_console_orig.jpg b/content/Azure/images/azure_intro_console_orig.jpg new file mode 100644 index 0000000..c3cd325 Binary files /dev/null and b/content/Azure/images/azure_intro_console_orig.jpg differ diff --git a/content/Azure/intro_to_Azure_Essentials.ipynb b/content/Azure/intro_to_Azure_Essentials.ipynb index c7ec784..2984146 100644 --- a/content/Azure/intro_to_Azure_Essentials.ipynb +++ b/content/Azure/intro_to_Azure_Essentials.ipynb @@ -2,7 +2,7 @@ "cells": [ { "cell_type": "markdown", - "id": "21ff6943", + "id": "30c652f8", "metadata": {}, "source": [ "# Microsoft Azure\n", @@ -10,6 +10,7 @@ "The modules in this lesson are: \n", "\n", "1. [Introduction to the Azure Cloud Portal](./01_intro_to_cloud_console)\n", + "2. [Introduction to the Azure CLI](./02_intro_to_cli.ipynb)\n", "2. [Introduction to Cloud Compute (Azure Virtual Machines)](./02_intro_to_compute)\n", "3. [Introduction to the Azure CLI](./04_intro_to_cli)\n", "4. [Introduction to Cloud Storage (Blob Storage)](./05_intro_to_cloud_storage)\n", @@ -17,11 +18,19 @@ "6. [Monitoring Costs](./07_monitoring_costs)\n", "7. [Cleaning up Resources and Best Practices](./08_cleaning_up_resources)" ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "92d161a5", + "metadata": {}, + "outputs": [], + "source": [] } ], "metadata": { "kernelspec": { - "display_name": "Python 3 (ipykernel)", + "display_name": "Python 3", "language": "python", "name": "python3" }, @@ -35,7 +44,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.8.10" + "version": "3.8.8" } }, "nbformat": 4, diff --git a/content/_toc.yml b/content/_toc.yml index 450f420..3f1f58b 100644 --- a/content/_toc.yml +++ b/content/_toc.yml @@ -25,12 +25,15 @@ parts: - file: Azure/intro_to_Azure_Essentials sections: - file: Azure/01_intro_to_cloud_console - - file: Azure/02_intro_to_compute - - file: Azure/04_intro_to_cli + - file: Azure/02_intro_to_cli + - file: Azure/03_intro_to_compute + - file: Azure/04_intro_to_compute_cli - file: Azure/05_intro_to_cloud_storage - - file: Azure/06_running_analysis + - file: Azure/06_intro_to_cloud_storage_cli - file: Azure/07_monitoring_costs - - file: Azure/08_cleaning_up_resources + - file: Azure/08_running_analysis + - file: Azure/09_cleaning_up_resources + - file: Azure/10_basic_networking_extra - file: GCP/intro_to_GCP_Essentials sections: