diff --git a/content/Azure/01_intro_to_cloud_console.ipynb b/content/Azure/01_intro_to_cloud_console.ipynb index c4b5219..8cfb88d 100644 --- a/content/Azure/01_intro_to_cloud_console.ipynb +++ b/content/Azure/01_intro_to_cloud_console.ipynb @@ -132,7 +132,7 @@ }, { "cell_type": "markdown", - "id": "43cd9544", + "id": "842ad945", "metadata": {}, "source": [ "## Everything Else\n", @@ -162,7 +162,7 @@ }, { "cell_type": "markdown", - "id": "a06f390f", + "id": "ebc30bae", "metadata": {}, "source": [ "## Open a Cloud Shell\n", @@ -172,7 +172,7 @@ "To use the Cloud Shell:\n", " * Open up a Cloud Shell by clicking on the **Cloud Shell** icon in the right side of the top blue bar (number 3 above).\n", " * The first time you open cloud shell you will be prompted for the following:\n", - "![welsome-to-cloud-shell-bashl](images/welsome-to-cloud-shell-bash.png)\n", + "![welsome-to-cloud-shell-bashl](images/welcome-to-cloud-shell-bash.png)\n", " * Choose *bash* to continue.\n", " * Then it will ask you to create a storage account for persistant storage for your cloud shell experience.\n", "![welcome-to-cloud-shell-storage](images/welcome-to-cloud-shell-storage.png)\n", @@ -210,7 +210,7 @@ }, { "cell_type": "markdown", - "id": "c58c867b", + "id": "1d7f2ca7", "metadata": {}, "source": [ "This prompt is asking you to create a .ssh directory in your home directory and generate ssh public/private keys named id_rsa.pub/id_rsa (default) respectively.\n", @@ -227,8 +227,8 @@ "\n", "After successfully retyping the passphrase the key generation process will continue showing a similar message as follows (the fingerprint and art will look different):\n", "```\n", - "Your identification has been saved in /home/learner/.ssh/id_rsa.\n", - "Your public key has been saved in /home/learner/.ssh/id_rsa.pub.\n", + "Your identification has been saved in /home//.ssh/id_rsa.\n", + "Your public key has been saved in /home//.ssh/id_rsa.pub.\n", "The key fingerprint is:\n", "SHA256:D9yj1nZEUigahirvjkIFoCP7RfjDu67Ku88M7nHQftI learner@cs-748001990186-default-boost-r9phq\n", "The key's randomart image is:\n", @@ -249,6 +249,20 @@ "create proces you will be asked to Cut & Paste the public key into a form. " ] }, + { + "cell_type": "markdown", + "id": "8bb7ae75", + "metadata": {}, + "source": [ + "## Overview of what we will create\n", + "\n", + "We are now ready to move on with the rest of the Azure essentials lessons. Here is an overview of what we are \n", + "going to do. First we will create a Virtual Machine (VM) and in doing so create a Resource Group. Then we will \n", + "create a Storage Account and in that a blob storage container. After that we will create a blob and upload it to\n", + "the blob contaioner.\n", + "![Azure-essentails-outline](images/azure-essentails-outline.png)" + ] + }, { "cell_type": "markdown", "id": "779c8715", diff --git a/content/Azure/02_intro_to_compute.ipynb b/content/Azure/02_intro_to_compute.ipynb index 3b526a0..b95664b 100644 --- a/content/Azure/02_intro_to_compute.ipynb +++ b/content/Azure/02_intro_to_compute.ipynb @@ -47,26 +47,17 @@ }, { "cell_type": "markdown", - "id": "0ef69f07", + "id": "a4dd13ae", "metadata": {}, "source": [ - "Before we begin this lesson/episode and create a Virtual Machine (VM), we need to create a ssh \n", - "key so that we can securely login to the VM we are about to create. We need use a ssh public \n", - "key when we create the VM.\n", - "\n", - "**Note:** Step 1. below is not required if you have already created a ssh key in the \n", - "[Intro to Cloud Console](02_intro_to_compute.ipynb) lesson. If you have already \n", - "created a key go directly to step 2.\n", + "Before we begin this lesson, we need to create a *ssh key pair* so that we can securely login to the Virtual Machone \n", + "(VM) we are about to create. We will need a ssh public key when we create the VM.\n", "\n", "**Step 1.**\n", "\n", - "If you have not used the cloud shell to connect to a *VM Instance* before, you will need to \n", - "create a new *ssh key*. The VM will use this key to allow you to access the *VM instance* \n", - "in a secure manner. \n", - "\n", - "Open up a Cloud Shell by clicking on the Cloud Shell icon in the right side of the top \n", - "blue bar (the icon looks like box with a greater than sign followed by an underscore) as \n", - "described in the [Intro to Cloud Console](01_intro_to_cloud_console.ipynb) lesson.\n", + "The Cloud Shell should already be open from the previous lesson [Intro to Cloud Console](01_intro_to_cloud_console.ipynb).\n", + "If not, Open a Cloud Shell by clicking on the Cloud Shell icon in the right side of the top \n", + "blue bar (the icon looks like box with a greater than sign followed by an underscore).\n", "\n", "At the $ prompt in the Cloud Shell, enter ssh-keygen and follow the prompts.\n" ] @@ -85,7 +76,7 @@ }, { "cell_type": "markdown", - "id": "164cdb8c", + "id": "0950e691", "metadata": {}, "source": [ "This prompt is asking you to create a .ssh directory in your home directory and generate ssh \n", @@ -103,8 +94,8 @@ "After successfully retyping the passphrase the key generation process will continue showing a \n", "similar message as follows (the fingerprint and art will look different):\n", "````\n", - "Your identification has been saved in /home/learner/.ssh/id_rsa.\n", - "Your public key has been saved in /home/learner/.ssh/id_rsa.pub.\n", + "Your identification has been saved in /home//.ssh/id_rsa.\n", + "Your public key has been saved in /home//.ssh/id_rsa.pub.\n", "The key fingerprint is:\n", "SHA256:D9yj1nZEUigahirvjkIFoCP7RfjDu67Ku88M7nHQftI learner@cs-748001990186-default-boost-r9phq\n", "The key's randomart image is:\n", @@ -142,7 +133,7 @@ "metadata": {}, "outputs": [], "source": [ - "cat /home/learner/.ssh/id_rsa.pub" + "cat /home//.ssh/id_rsa.pub" ] }, { diff --git a/content/Azure/03_intro_to_cloud_storage.ipynb b/content/Azure/03_intro_to_cloud_storage.ipynb index 161be7f..8bfd866 100644 --- a/content/Azure/03_intro_to_cloud_storage.ipynb +++ b/content/Azure/03_intro_to_cloud_storage.ipynb @@ -65,7 +65,7 @@ }, { "cell_type": "markdown", - "id": "af639990", + "id": "06c38751", "metadata": {}, "source": [ "## Create Resource Group (Note sure if this is needed)\n", @@ -76,6 +76,7 @@ "* Navigate to Resource Groups\n", "* Click \"+ create\" in the upper left \n", "* Select a name for your Resource Group\n", + " * Use the Resource Group we create earlier during the VM creation process\n", "* Leave the rest as default\n", "* Click Review & Create\n", "* After passing validation, click create\n" @@ -110,19 +111,21 @@ }, { "cell_type": "markdown", - "id": "4860da70", + "id": "4ac4e95a", "metadata": { "tags": [] }, "source": [ - "### Allocate Azure Cloud Storage\n", + "## Allocate Azure Cloud blob Storage\n", "\n", "To create a blob storage container in the Azure portal, follow these steps:\n", "\n", "* Navigate to your new storage account in the Azure portal.\n", - "* In the left menu for the storage account, scroll to the Data storage section, then select containers.\n", - "* Select the + Container button.\n", - "* Type a name for your new container.\n", + "* In the left menu of the storage account, scroll to the Data storage section, then select containers.\n", + "* Select the + Container button to create a new blob storage container.\n", + "* Over on the right side\n", + " * Type a name for your new container.\n", + " * Under the *Public access level* pulldown menu select *Blob (annonymous read access for blobs only)*\n", "* Select OK to create the container.\n", "\n", "![vm-create-container](images/04_azure_storage_create_container.png)\n" diff --git a/content/Azure/04_intro_to_cli.ipynb b/content/Azure/04_intro_to_cli.ipynb index f6d80c4..f6efbaa 100644 --- a/content/Azure/04_intro_to_cli.ipynb +++ b/content/Azure/04_intro_to_cli.ipynb @@ -166,17 +166,21 @@ ] }, { - "cell_type": "markdown", - "id": "a90996d2", + "cell_type": "code", + "execution_count": null, + "id": "12c48d46", "metadata": {}, + "outputs": [], "source": [ "# date" ] }, { - "cell_type": "markdown", - "id": "20a5bdaa", + "cell_type": "code", + "execution_count": null, + "id": "9bbeed89", "metadata": {}, + "outputs": [], "source": [ "# echo \"Date: $(date)\"" ] @@ -290,15 +294,21 @@ ], "metadata": { "kernelspec": { - "display_name": "Bash", - "language": "bash", - "name": "bash" + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" }, "language_info": { - "codemirror_mode": "shell", - "file_extension": ".sh", - "mimetype": "text/x-sh", - "name": "bash" + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.7" } }, "nbformat": 4, diff --git a/content/Azure/05_intro_to_cloud_storage_cli.ipynb b/content/Azure/05_intro_to_cloud_storage_cli.ipynb index 7161f00..3c4638f 100644 --- a/content/Azure/05_intro_to_cloud_storage_cli.ipynb +++ b/content/Azure/05_intro_to_cloud_storage_cli.ipynb @@ -51,9 +51,8 @@ "source": [ "RG=\"demo_rg\"\n", "LOCATION=\"eastus\"\n", - "BLOB_CON=\"demo_blob_cont\"\n", - "STGE_ACCT=\"demo_stor_ac\"\n", - "CTR=\"demo_stge_ctr\"" + "STGE_ACCT=\"demostorac\"\n", + "BLOB_CONT=\"demo-blob-cont\"" ] }, { @@ -83,18 +82,9 @@ }, { "cell_type": "code", - "execution_count": 2, - "metadata": {}, - "outputs": [ - { - "ename": "SyntaxError", - "evalue": "invalid syntax (838669738.py, line 1)", - "output_type": "error", - "traceback": [ - "\u001b[0;36m File \u001b[0;32m\"/var/folders/ty/h84mzh251vdbfwf3zsph_xrh0000gq/T/ipykernel_80687/838669738.py\"\u001b[0;36m, line \u001b[0;32m1\u001b[0m\n\u001b[0;31m az storage account create --name $STGE_ACCT --resource-group $RG --location $LOCATION --sku Standard_ZRS --encryption-services blob\u001b[0m\n\u001b[0m ^\u001b[0m\n\u001b[0;31mSyntaxError\u001b[0m\u001b[0;31m:\u001b[0m invalid syntax\n" - ] - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "az storage account create --name $STGE_ACCT --resource-group $RG --location $LOCATION --sku Standard_ZRS --encryption-services blob" ] @@ -133,7 +123,7 @@ "metadata": {}, "outputs": [], "source": [ - "az storage container create --account-name $STGE_ACCT --name $CTR --auth-mode login" + "az storage container create --account-name $STGE_ACCT --name $BLOB_CONT --auth-mode login" ] }, { @@ -156,7 +146,7 @@ "metadata": {}, "outputs": [], "source": [ - "echo \"Demo text\" > demo_file" + "echo \"Demo text\" > demo_file.txt" ] }, { @@ -172,7 +162,7 @@ "metadata": {}, "outputs": [], "source": [ - "az storage blob upload --account-name $STGE_ACCT --container-name $CTR --name demo_file --file demo_file --auth-mode login" + "az storage blob upload --account-name $STGE_ACCT --container-name $BLOB_CONT --name demo_file_upload.txt --file demo_file.txt --auth-mode login" ] }, { @@ -188,7 +178,7 @@ "metadata": {}, "outputs": [], "source": [ - "az storage blob list --account-name $STGE_ACCT --container-name $CTR --output table --auth-mode login" + "az storage blob list --account-name $STGE_ACCT --container-name $BLOB_CONT --output table --auth-mode login" ] }, { @@ -204,7 +194,7 @@ "metadata": {}, "outputs": [], "source": [ - "az storage blob download --account-name $STGE_ACCT --container-name $CTR --name demo_file --file demo_file_download file --auth-mode login" + "az storage blob download --account-name $STGE_ACCT --container-name $BLOB_CONT --name demo_file_download.txt --file demo_file_upload.txt --auth-mode login" ] }, { @@ -220,7 +210,7 @@ "metadata": {}, "outputs": [], "source": [ - "ls demo_file_download" + "ls demo_file_download.txt" ] }, { @@ -236,7 +226,7 @@ "metadata": {}, "outputs": [], "source": [ - "diff demo_file demo_file_download" + "diff demo_file.txt demo_file_download.txt" ] }, { @@ -262,7 +252,7 @@ "metadata": {}, "outputs": [], "source": [ - "az storage blob delete --account-name $STGE_ACCT --container-name $CTR --name demo_file --auth-mode login" + "az storage blob delete --account-name $STGE_ACCT --container-name $BLOB_CONT --name demo_file --auth-mode login" ] }, { @@ -278,7 +268,7 @@ "metadata": {}, "outputs": [], "source": [ - "az storage blob list --account-name $STGE_ACCT --container-name $CTR --output table --auth-mode login" + "az storage blob list --account-name $STGE_ACCT --container-name $BLOB_CONT --output table --auth-mode login" ] }, { diff --git a/content/Azure/06_running_analysis.ipynb b/content/Azure/06_running_analysis.ipynb index e6227b3..402ddc3 100644 --- a/content/Azure/06_running_analysis.ipynb +++ b/content/Azure/06_running_analysis.ipynb @@ -43,12 +43,12 @@ }, { "cell_type": "markdown", - "id": "3934bac4", + "id": "68d90090", "metadata": {}, "source": [ "## Create a VM\n", "\n", - "For this exercise we will use the VM that we created in the [Intro to Cloud Console](01_intro_to_cloud_console.ipynb) \n", + "For this exercise we will use the VM that we created in the [Intro to Cloud Compute](02_intro_to_compute.ipynb) \n", "lesson." ] }, @@ -120,22 +120,13 @@ }, { "cell_type": "markdown", - "id": "da999ac0", + "id": "43c7d723", "metadata": {}, "source": [ "## Get Example Code\n", "\n", - "We will now install `git` and use it to download the example code into your home directory. For those of you who are unfamiliar with git, it is a way to collaboratively manage files and we will only be using it to download the example that we will be using. " - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "0fc21993", - "metadata": {}, - "outputs": [], - "source": [ - "sudo apt-get install --yes git" + "We will now use git to download the example *landsat* analysis code into your home directory. For those of you \n", + "who are unfamiliar with git, it is a way to collaboratively manage files." ] }, { @@ -160,10 +151,10 @@ }, { "cell_type": "markdown", - "id": "f3133c48", + "id": "ab9cc790", "metadata": {}, "source": [ - "We now change the current directory to the `azure-landsat` directory in the `CLASS-Examples` directory that was just created by the previous git command." + "Now change directory to `CLASS-Examples/azure-landsat` where our analysis code is." ] }, { @@ -218,43 +209,19 @@ "metadata": {}, "outputs": [], "source": [ - "#./landsat-env" + "./landsat-env" ] }, { "cell_type": "markdown", - "id": "6be82619", - "metadata": {}, - "source": [ - "This will install the following packages." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "e3930da6", - "metadata": {}, - "outputs": [], - "source": [ - "sudo apt install python3-pip -y\n", - "pip3 install --upgrade pip\n", - "pip3 install numpy\n", - "pip3 install matplotlib\n", - "pip3 install pandas\n", - "pip3 install rasterio\n", - "pip3 install azure.storage.blob" - ] - }, - { - "cell_type": "markdown", - "id": "9d459eb2", + "id": "8fdf8aec", "metadata": {}, "source": [ "## Get and Process landsat data\n", "\n", - "Lets take a quick look at the python scrip *landsat-hls-azure.py*. This code will find the HLS tile coresponding \n", - "to a given latatue and longitude at a specific time. It will then produce a .tiff image of the tile. We will then combine images with different \n", - "bands to create a composite image of the area of interest. " + "Lets take a quick look at the python scrip *landsat-hls-azure.py*. This code will find the HLS tiles coresponding \n", + "to a given latatue and longitude at a specific time. Using the tiles stored on Azure, we will combine different *bands*\n", + "to create a image file of the satellite dat in (png) formate." ] }, { @@ -272,15 +239,15 @@ "metadata": {}, "outputs": [], "source": [ - "#./landsat-hls-azure.py" + "./landsat-hls-azure.py" ] }, { "cell_type": "markdown", - "id": "6ea30b3c", + "id": "3af52b33", "metadata": {}, "source": [ - "This will product the file *test.png*. This is an image file of the tile we reqeusted." + "This will product the file *test.png*. This is an image file of the combined tiles we reqeusted." ] }, { @@ -305,33 +272,10 @@ }, { "cell_type": "markdown", - "id": "19158868", - "metadata": {}, - "source": [ - "Install the azure CLI by executing the the curl command below.\n", - "\n", - "This script does three fundamental things: #\n", - " 1. Add Microsoft's GPG Key has a trusted source of apt packages. #\n", - " 2. Add Microsoft's repositories as a source for apt packages. #\n", - " 3. Installs the Azure CLI from those repositories. " - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "dca9b76f", - "metadata": {}, - "outputs": [], - "source": [ - "curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash" - ] - }, - { - "cell_type": "markdown", - "id": "ab4c265a", + "id": "89e85f82", "metadata": {}, "source": [ - "Define some environment variables" + "Define some environment variables from the resources created in previous lessons." ] }, { @@ -341,82 +285,69 @@ "metadata": {}, "outputs": [], "source": [ - "RG=\"demo_rg\"\n", + "SUB\"\"\"\n", + "RG=\"\"\n", "LOCATION=\"eastus\"\n", - "BLOB_CON=\"demo_blob_cont\"\n", - "STGE_ACCT=\"demo_stor_ac\"\n", - "CTR=\"demo_stge_ctr\"" + "STGE_ACCT=\"\"\n", + "BLOB_CONT=\"\"" ] }, { "cell_type": "markdown", - "id": "3b4f9136", + "id": "11e22dae", "metadata": {}, "source": [ - "login into azure and verify info" + "Install the azure CLI by executing the the curl command below. " ] }, { "cell_type": "code", "execution_count": null, - "id": "cdcbd1e2", + "id": "dca9b76f", "metadata": {}, "outputs": [], "source": [ - "az login\n", - "az group list –output table\n", - "az storage account list --output table\n", - "az ad signed-in-user show --query objectId -o tsv | az role assignment create --role \"Storage Blob Data Contributor\" --assignee @- --scope \"/subscriptions/ae186240-1b93-42b5-bb33-e9b8c4e527de/resourceGroups/demo_rg/providers/Microsoft.Storage/storageAccounts/landsatdemosa\"" + "curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash" ] }, { "cell_type": "code", "execution_count": null, - "id": "3b89ddb9", + "id": "30ffa151", "metadata": {}, "outputs": [], "source": [ - "Use the existing container created ealier or create a new one.\n", - "\n", - "List existing containers" + "login into azure" ] }, { "cell_type": "code", "execution_count": null, - "id": "ab37a316", + "id": "cdcbd1e2", "metadata": {}, "outputs": [], "source": [ - "az storage container list --account-name $STGE_ACCT --auth-mode login" + "az login\n", + "az ad signed-in-user show --query objectId -o tsv | az role assignment create --role \"Storage Blob Data Contributor\" --assignee @- --scope \"/subscriptions/$SUB/resourceGroups/$RG/providers/Microsoft.Storage/storageAccounts/$STGE_ACCT\"" ] }, { "cell_type": "markdown", - "id": "7c9f97db", - "metadata": {}, - "source": [ - "Create storage account if one doen't already exist" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "19ea43e7", + "id": "58a0e9ba", "metadata": {}, - "outputs": [], "source": [ - "az storage container create --account-name $STGE_ACCT --name $BLOB_CON --auth-mode login" + "Now we need to asign ourselves the role of Storage Blob Data Contributor so we can interact with the \n", + "blob container." ] }, { "cell_type": "code", "execution_count": null, - "id": "c6047945", + "id": "8a639fcc", "metadata": {}, "outputs": [], "source": [ - "az storage container list --account-name $STGE_ACCT" + "az ad signed-in-user show --query objectId -o tsv | az role assignment create --role \"Storage Blob Data Contributor\" --assignee @- --scope \"/subscriptions/$SUB/resourceGroups/$RG/providers/Microsoft.Storage/storageAccounts/$STGE_ACCT\"" ] }, { @@ -434,7 +365,7 @@ "metadata": {}, "outputs": [], "source": [ - "az storage blob upload --account-name $STGE_ACCT --container-name $BLOB_CON --name test.png --file test.png --auth-mode login" + "az storage blob upload --account-name $STGE_ACCT --container-name $BLOB_CONT --name test.png --file test.png --auth-mode login" ] }, { @@ -452,7 +383,7 @@ "metadata": {}, "outputs": [], "source": [ - "az storage blob list --account-name $STGE_ACCT --container-name $BLOB_CON --output table --auth-mode login" + "az storage blob list --account-name $STGE_ACCT --container-name $BLOB_CONT --output table --auth-mode login" ] }, { @@ -487,11 +418,9 @@ ] }, { - "cell_type": "code", - "execution_count": null, - "id": "e6faacab", + "cell_type": "markdown", + "id": "f8602b41", "metadata": {}, - "outputs": [], "source": [ "Delete the storage container" ] @@ -503,7 +432,7 @@ "metadata": {}, "outputs": [], "source": [ - "az storage container delete --account-name $STGE_ACCT --name $BLOB_CON --auth-mode login" + "az storage container delete --account-name $STGE_ACCT --name $BLOB_CONT --auth-mode login" ] }, { diff --git a/content/Azure/images/azure-essentails-outline.png b/content/Azure/images/azure-essentails-outline.png new file mode 100644 index 0000000..acdec2e Binary files /dev/null and b/content/Azure/images/azure-essentails-outline.png differ