diff --git a/content/Azure/02_intro_to_compute_old.ipynb b/content/Azure/02_intro_to_compute_old.ipynb deleted file mode 100644 index 41e2c89..0000000 --- a/content/Azure/02_intro_to_compute_old.ipynb +++ /dev/null @@ -1,279 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "id": "fcfca89e", - "metadata": {}, - "source": [ - "# Introduction to Virtual Machines (VM) in Azure\n", - "\n", - "\n", - "```{admonition} Overview\n", - ":class: tip\n", - "\n", - "**Teaching:** 45 mins\n", - "\n", - "**Exercises:** 10 mins\n", - "\n", - "**Questions:**\n", - "* What is an EC2 instance?\n", - "* When would I use an EC2 instance?\n", - "* How do I launch an EC2 instance?\n", - "\n", - "**Objectives:**\n", - "* Understand the concept of virtual servers.\n", - "* Understand what an Elastic Cloud Compute (EC2) instance is.\n", - "\n", - "```\n", - "\n", - "There are 7 steps to walk through to create a new EC2 instance; we will go through each in detail: \n", - "1. Select an AMI\n", - "2. Choose Instance Type\n", - "3. Configure Instance \n", - "4. Add Storage\n", - "5. Add Tags\n", - "6. Configure Security Group\n", - "7. Review/Launch\n", - "\n", - "\n", - "# Tutoroial to depoy VMs and show commands\n", - "# https://docs.microsoft.com/en-us/cli/azure/azure-cli-vm-tutorial\n", - "\n", - "# Create resource group\n", - "az group create --name TutorialResources --location eastus\n", - "\n", - "# Create VM\n", - "az vm create --resource-group TutorialResources \\\n", - " --name TutorialVM1 \\\n", - " --image UbuntuLTS \\\n", - " --generate-ssh-keys \\\n", - " --output json \\\n", - " --verbose \n", - "\n", - "# Login into VM\n", - "ssh \n", - "\n", - "# Show VM details (all)\n", - "az vm show --name TutorialVM1 --resource-group TutorialResources\n", - "\n", - "# NIC object ID\n", - "az vm show --name TutorialVM1 \\\n", - " --resource-group TutorialResources \\\n", - " --query 'networkProfile.networkInterfaces[].id' \\\n", - " --output tsv\n", - "\n", - "# Assign NIC ID to env variable \n", - "NIC_ID=$(az vm show -n TutorialVM1 -g TutorialResources \\\n", - " --query 'networkProfile.networkInterfaces[].id' \\\n", - " -o tsv)\n", - "\n", - "# Show NIC details\n", - "az network nic show --ids $NIC_ID\n", - "\n", - "# Since two values on separate lines are displayed, the read command delimiter must be set \n", - "# to the empty string rather than the default of non-newline whitespace.\n", - "read -d '' IP_ID SUBNET_ID <<< $(az network nic show \\\n", - " --ids $NIC_ID \\\n", - " --query '[ipConfigurations[].publicIpAddress.id, ipConfigurations[].subnet.id]' \\\n", - " -o tsv)\n", - "\n", - "# Assign IP to env variable\n", - "VM1_IP_ADDR=$(az network public-ip show --ids $IP_ID \\\n", - " --query ipAddress \\\n", - " -o tsv)\n", - "\n", - "# Show IP address\n", - "echo $VM1_IP_ADDR\n", - "\n", - "# Create another VM and assign to existing subnet\n", - "VM2_IP_ADDR=$(az vm create -g TutorialResources \\\n", - " -n TutorialVM2 \\\n", - " --image UbuntuLTS \\\n", - " --generate-ssh-keys \\\n", - " --subnet $SUBNET_ID \\\n", - " --query publicIpAddress \\\n", - " -o tsv)\n", - "\n", - "# Login to second VM2\n", - "ssh $VM2_IP_ADDR\n", - "\n", - "# Delete resource group (& all above)\n", - "az group delete --name TutorialResources --no-wait\n", - "\n", - "# The --no-wait parameter keeps the CLI from blocking while the deletion takes place. \n", - "az group wait --name TutorialResources --deleted" - ] - }, - { - "cell_type": "markdown", - "id": "81c865bc", - "metadata": {}, - "source": [ - "Hamburger in upper left -> Virtual Machines -> Create (+) in upper left -> virtual machines ->\n", - "In the middle of the page under \"Create Virtual Machine\" heading there are menu options from left to right: Basic, Disk, Networking, Management, Advanced, Tags, Review & Create\n", - "\n", - "**Basic:**\n", - "* Check subscription \n", - "* select or create Resource Group\n", - "* VM name\n", - "\n", - "**Disk**\n", - "\n", - "**Networking**\n", - "\n", - "**Management**\n", - "\n", - "**Advanced**\n", - "\n", - "**Tags**\n", - "\n", - "**Review & Create**\n", - "\n", - "Recall that the two fundamental components of cloud computing is compute and storage. On AWS, a \"virtual server\" or \"virtual computer\" is known as an **Elastic Cloud Compute (EC2) instance**; sometimes it's called \"EC2\", sometimes it's called an \"instance\" to denote that the ability to build and terminate this server instantaneously, but they all mean the same thing. An EC2 instance is no different from a server that sits under your desk, or your local departmental cluster, or even your local HPC cluster. You even boot up an EC2 instance through the web console, install software and then shut down your instance just like you would a real computer, except that Amazon takes care of the physical machinery while you are in charge of process of creating the computer. In some sense, you can think of utilizing an EC2 instance as renting a server or computer from Amazon! \n", - "\n", - "In cloud jargon, the term **elasticity** denotes the ability to quickly expand or decrease computer processing, memory, and storage resources to meet changing demands. In that way, you can expand the size of your CPU, RAM and disk size on your EC2 instance almost instantenously. Since EC2 forms the backbone of most of AWS's core infrastructure, it is an important part of your cloud journey. \n", - " \n", - "\n", - "Let's walk through some of the steps on getting an EC2 instance up and running. \n" - ] - }, - { - "cell_type": "markdown", - "id": "99fc77c6", - "metadata": {}, - "source": [ - "We begin with the AWS console again. Under the \"Build a Solution\" panel, select `Launch a Virtual Machine`\n", - "\n", - "\n", - "\n", - "
Figure 1: Start page for the AWS console

\n", - "\n", - "This will then lead you through a series of steps to get a **Free Tier** EC2 instance up and running. \n", - "\n", - "```{admonition} Note\n", - ":class: note\n", - "\n", - "AWS Free Tier refers to several of the services that AWS offers to help users gain more hands on experience on the AWS platform without being charged. [Click here](https://aws.amazon.com/free/?all-free-tier.sort-by=item.additionalFields.SortRank&all-free-tier.sort-order=asc&awsf.Free%20Tier%20Types=*all&awsf.Free%20Tier%20Categories=*all) for more info about the AWS Free Tier [external link] . \n", - "```\n", - "\n", - "There are 7 steps to walk through to create a new EC2 instance; we will go through each in detail: \n", - "1. Select an AMI\n", - "2. Choose Instance Type\n", - "3. Configure Instance \n", - "4. Add Storage\n", - "5. Add Tags\n", - "6. Configure Security Group\n", - "7. Review/Launch" - ] - }, - { - "cell_type": "markdown", - "id": "72dc7149", - "metadata": {}, - "source": [ - "## 1. Select an AMI\n", - "\n", - "An Amazon Machine Image (AMI) is a template that Amazon uses to describe the operating system, disk type and all the software configuration that is needed to make sure a computer runs smoothly. Imagine that you are purchasing a new laptop; fresh out of the box, the laptop is pre-configured with an operating system (e.g. Windows, Mac OS, Ubuntu etc.), configuration files that tells the laptop what peripherals are attached, and pre-installed software like Adobe PDF reader. An AMI contains all this information so that your EC2 instance runs exactly like it would a new laptop out of the box! There is much more to learn about AMIs and how they can used for collaboration and data sharing but that is not within the scope of CLASS Essentials. \n", - "\n", - "As you scroll through the AMI list (Figure 2) you will notice that the list contains offerings from various vendors (e.g. Amazon, RedHat, Windows, etc.). We will be choosing the Ubuntu operating system for flexibility and versatility (can be used outside of the AWS ecosystem). \n", - "\n", - "To list all the Free Tier AMIs, check the box on the right that says ```Free tier only```.\n", - "\n", - "\n", - "\n", - "
Figure 2: Step 1 - Select an AMI - Free Tier Only

\n", - "\n", - "Scroll to ```Ubuntu Server 20.04 LTS(HVM), SSD Volume Type``` (Figure 3). Select ```64-bit(x86)```. \n", - "\n", - "\n", - "\n", - "
Figure 3: Step 1 - Select an AMI - Operating System Selection

" - ] - }, - { - "cell_type": "markdown", - "id": "17597535", - "metadata": {}, - "source": [ - "## Step 2: Choose an Instance Type\n", - "\n", - "Choosing an instance type is choosing the hardware for your computing system: you get to pick the number of CPUs and memory size for your instance. \n", - "\n", - "Instance types are group by [**families**](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-types.html) and denotes whether, for example, an instance is optimized for batch processing (compute-optimized, C-family), optimized for databases (memory-optimized, R-family) or has accelerated hardware (GPUs) for AI or Machine Learning pipelines. \n", - "\n", - "When you choose an Instance Type (Figure 3), the screen show additional information about the selected instance type including the number of CPUs, the memory size, the type of storage and information about networking. \n", - "\n", - "In the Instance Storage (GB) column, you will notice a term called **EBS**. EBS is the acronym for **Elastic Block Storage** and is analogous to the hard disk or boot drive on your personal computer or laptop. More details about EBS and different kinds of disk storage on EC2 instances are beyond the scope of CLASS Essentials. \n", - "\n", - "```{admonition} Note\n", - ":class: note\n", - "The four most common types of storage you will encounter on AWS are: Elastic Block Storage (EBS), Elastic File Storage (EFS), Simple Storage Service (s3) and s3 Glacier. In the simplest terms, EBS is analogous to a computer hard drive and EFS is analogous to a network file system (NFS) or shared file system. s3 is AWS's object storage which is discussed [here](./intro_to_s3). s3 Glacier is a cost-effective way of storing s3 files that you do not need to access frequently. \n", - "```\n", - "\n", - "Here will will select a ```t2.micro``` instance which is Free Tier Eligible but only has 1vCPU and 1 GiB of memory. The cost of running a t2.micro instance is " - ] - }, - { - "cell_type": "markdown", - "id": "4ee655a3", - "metadata": {}, - "source": [ - "\n", - "\n", - "
Figure 3: Step 2 - Choose an Instance Type

\n", - "\n", - "Select ```Next: Configure Instance Details```.\n", - "\n", - "## Step 3: Configure Instance Details\n", - "Step 3 in creating an EC2 instance involves a rudimentary understanding of several key AWS and cloud jargon (Figure 4). While delving deeper into some of the terminology is outside of the scope of CLASS Essentials, we go will through these terms in brief as we learn how to configure your EC2 instance. \n", - "\n", - "\n", - "\n", - "
Figure 4: Step 3 - Configure Instance Details

\n", - "\n", - "**Number of instances** : This indicates how many instances you want to create at the same time. Here, we will leave the value as '1' but in actuality, you can can have up to 20 instances per region. \n", - "\n", - "```{admonition} Note\n", - ":class: note\n", - "Recall that we learned about regions in the [previous chapter](./intro_to_cloud_console). \n", - "```\n", - "\n", - "**Purchasing Options** : Throughout your AWS journey, you will hear the term [**Spot Instances**](). Spot instances make use of the servers that go unused in AWS data centers to minimize costs. Recall that AWS has many data centers spread across the globe and not all their servers are utilized at 100% capacity at all times. Amazon uses Spot Instances as a flexible way to profit from extra capacity. Users have access to Spot Instances through a bidding process, sometimes users can save up to 90% off the on-deman compute instance this way! We will not expand much more on Spot Instances in CLASS Essentials but if you are interested, I2's CLASS Intermediate talks more " - ] - }, - { - "cell_type": "markdown", - "id": "9c874bad", - "metadata": {}, - "source": [ - "```{admonition} Exercise\n", - ":class: attention\n", - "\n", - "* What kind of information is contained in an AMI? \n", - "* How do Spot Instances help you optimize costs?\n", - "````" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.9.7" - } - }, - "nbformat": 4, - "nbformat_minor": 5 -} diff --git a/content/Azure/02_intro_to_compute_part1.ipynb b/content/Azure/02_intro_to_compute_part1.ipynb deleted file mode 100644 index a6c4161..0000000 --- a/content/Azure/02_intro_to_compute_part1.ipynb +++ /dev/null @@ -1,186 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "id": "dc57021c", - "metadata": {}, - "source": [ - "# Introduction to Elastic Cloud Compute (EC2)\n", - "\n", - "\n", - "```{admonition} Overview\n", - ":class: tip\n", - "\n", - "**Teaching:** 45 mins\n", - "\n", - "**Exercises:** 10 mins\n", - "\n", - "**Questions:**\n", - "* What is an EC2 instance?\n", - "* When would I use an EC2 instance?\n", - "* How do I launch an EC2 instance?\n", - "\n", - "**Objectives:**\n", - "* Understand the concept of virtual servers.\n", - "* Understand what an Elastic Cloud Compute (EC2) instance is.\n", - "\n", - "```" - ] - }, - { - "cell_type": "markdown", - "id": "502f2360", - "metadata": {}, - "source": [ - "Recall that the two fundamental components of cloud computing is compute and storage. On AWS, a \"virtual server\" or \"virtual computer\" is known as an **Elastic Cloud Compute (EC2) instance**; sometimes it's called \"EC2\", sometimes it's called an \"instance\" to denote that the ability to build and terminate this server instantaneously, but they all mean the same thing. An EC2 instance is no different from a server that sits under your desk, or your local departmental cluster, or even your local HPC cluster. You even boot up an EC2 instance through the web console, install software and then shut down your instance just like you would a real computer, except that Amazon takes care of the physical machinery while you are in charge of process of creating the computer. In some sense, you can think of utilizing an EC2 instance as renting a server or computer from Amazon! \n", - "\n", - "In cloud jargon, the term **elasticity** denotes the ability to quickly expand or decrease computer processing, memory, and storage resources to meet changing demands. In that way, you can expand the size of your CPU, RAM and disk size on your EC2 instance almost instantenously. Since EC2 forms the backbone of most of AWS's core infrastructure, it is an important part of your cloud journey. \n", - " \n", - "\n", - "Let's walk through some of the steps on getting an EC2 instance up and running. \n" - ] - }, - { - "cell_type": "markdown", - "id": "bc5d082d", - "metadata": {}, - "source": [ - "We begin with the AWS console again. Under the \"Build a Solution\" panel, select `Launch a Virtual Machine`\n", - "\n", - "\n", - "\n", - "
Figure 1: Start page for the AWS console

\n", - "\n", - "This will then lead you through a series of steps to get a **Free Tier** EC2 instance up and running. \n", - "\n", - "```{admonition} Note\n", - ":class: note\n", - "\n", - "AWS Free Tier refers to several of the services that AWS offers to help users gain more hands on experience on the AWS platform without being charged. [Click here](https://aws.amazon.com/free/?all-free-tier.sort-by=item.additionalFields.SortRank&all-free-tier.sort-order=asc&awsf.Free%20Tier%20Types=*all&awsf.Free%20Tier%20Categories=*all) for more info about the AWS Free Tier [external link] . \n", - "```\n", - "\n", - "There are 7 steps to walk through to create a new EC2 instance; we will go through each in detail: \n", - "1. Select an AMI\n", - "2. Choose Instance Type\n", - "3. Configure Instance \n", - "4. Add Storage\n", - "5. Add Tags\n", - "6. Configure Security Group\n", - "7. Review/Launch" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "1e02813b", - "metadata": {}, - "outputs": [], - "source": [ - "## 1. Select an AMI\n", - "\n", - "An Amazon Machine Image (AMI) is a template that Amazon uses to describe the operating system, disk type and all the software configuration that is needed to make sure a computer runs smoothly. Imagine that you are purchasing a new laptop; fresh out of the box, the laptop is pre-configured with an operating system (e.g. Windows, Mac OS, Ubuntu etc.), configuration files that tells the laptop what peripherals are attached, and pre-installed software like Adobe PDF reader. An AMI contains all this information so that your EC2 instance runs exactly like it would a new laptop out of the box! There is much more to learn about AMIs and how they can used for collaboration and data sharing but that is not within the scope of CLASS Essentials. \n", - "\n", - "As you scroll through the AMI list (Figure 2) you will notice that the list contains offerings from various vendors (e.g. Amazon, RedHat, Windows, etc.). We will be choosing the Ubuntu operating system for flexibility and versatility (can be used outside of the AWS ecosystem). \n", - "\n", - "To list all the Free Tier AMIs, check the box on the right that says ```Free tier only```.\n", - "\n", - "\n", - "\n", - "
Figure 2: Step 1 - Select an AMI - Free Tier Only

\n", - "\n", - "Scroll to ```Ubuntu Server 20.04 LTS(HVM), SSD Volume Type``` (Figure 3). Select ```64-bit(x86)```. \n", - "\n", - "\n", - "\n", - "
Figure 3: Step 1 - Select an AMI - Operating System Selection

" - ] - }, - { - "cell_type": "markdown", - "id": "17597535", - "metadata": {}, - "source": [ - "## Step 2: Choose an Instance Type\n", - "\n", - "Choosing an instance type is choosing the hardware for your computing system: you get to pick the number of CPUs and memory size for your instance. \n", - "\n", - "Instance types are group by [**families**](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-types.html) and denotes whether, for example, an instance is optimized for batch processing (compute-optimized, C-family), optimized for databases (memory-optimized, R-family) or has accelerated hardware (GPUs) for AI or Machine Learning pipelines. \n", - "\n", - "When you choose an Instance Type (Figure 3), the screen show additional information about the selected instance type including the number of CPUs, the memory size, the type of storage and information about networking. \n", - "\n", - "In the Instance Storage (GB) column, you will notice a term called **EBS**. EBS is the acronym for **Elastic Block Storage** and is analogous to the hard disk or boot drive on your personal computer or laptop. More details about EBS and different kinds of disk storage on EC2 instances are beyond the scope of CLASS Essentials. \n", - "\n", - "```{admonition} Note\n", - ":class: note\n", - "The four most common types of storage you will encounter on AWS are: Elastic Block Storage (EBS), Elastic File Storage (EFS), Simple Storage Service (s3) and s3 Glacier. In the simplest terms, EBS is analogous to a computer hard drive and EFS is analogous to a network file system (NFS) or shared file system. s3 is AWS's object storage which is discussed [here](./intro_to_s3). s3 Glacier is a cost-effective way of storing s3 files that you do not need to access frequently. \n", - "```\n", - "\n", - "Here will will select a ```t2.micro``` instance which is Free Tier Eligible but only has 1vCPU and 1 GiB of memory. The cost of running a t2.micro instance is " - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "f4247acd", - "metadata": {}, - "outputs": [], - "source": [ - "\n", - "\n", - "
Figure 3: Step 2 - Choose an Instance Type

\n", - "\n", - "Select ```Next: Configure Instance Details```.\n", - "\n", - "## Step 3: Configure Instance Details\n", - "Step 3 in creating an EC2 instance involves a rudimentary understanding of several key AWS and cloud jargon (Figure 4). While delving deeper into some of the terminology is outside of the scope of CLASS Essentials, we go will through these terms in brief as we learn how to configure your EC2 instance. \n", - "\n", - "\n", - "\n", - "
Figure 4: Step 3 - Configure Instance Details

\n", - "\n", - "**Number of instances** : This indicates how many instances you want to create at the same time. Here, we will leave the value as '1' but in actuality, you can can have up to 20 instances per region. \n", - "\n", - "```{admonition} Note\n", - ":class: note\n", - "Recall that we learned about regions in the [previous chapter](./intro_to_cloud_console). \n", - "```\n", - "\n", - "**Purchasing Options** : Throughout your AWS journey, you will hear the term [**Spot Instances**](). Spot instances make use of the servers that go unused in AWS data centers to minimize costs. Recall that AWS has many data centers spread across the globe and not all their servers are utilized at 100% capacity at all times. Amazon uses Spot Instances as a flexible way to profit from extra capacity. Users have access to Spot Instances through a bidding process, sometimes users can save up to 90% off the on-deman compute instance this way! We will not expand much more on Spot Instances in CLASS Essentials but if you are interested, I2's CLASS Intermediate talks more " - ] - }, - { - "cell_type": "markdown", - "id": "9c874bad", - "metadata": {}, - "source": [ - "```{admonition} Exercise\n", - ":class: attention\n", - "\n", - "* What kind of information is contained in an AMI? \n", - "* How do Spot Instances help you optimize costs?\n", - "````" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.9.7" - } - }, - "nbformat": 4, - "nbformat_minor": 5 -} diff --git a/content/Azure/03_intro_to_compute_part2.ipynb b/content/Azure/03_intro_to_compute_part2.ipynb deleted file mode 100644 index f81f13b..0000000 --- a/content/Azure/03_intro_to_compute_part2.ipynb +++ /dev/null @@ -1,151 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "id": "dc57021c", - "metadata": {}, - "source": [ - "# Instance Storage, Tags & Security Groups\n", - "\n", - "\n", - "```{admonition} Overview\n", - ":class: tip\n", - "\n", - "**Teaching:** 45 mins\n", - "\n", - "**Exercises:** 10 mins\n", - "\n", - "**Questions:**\n", - "* What is an EC2 instance?\n", - "* When would I use an EC2 instance?\n", - "* How do I launch an EC2 instance?\n", - "\n", - "**Objectives:**\n", - "* Understand the concept of virtual servers.\n", - "* Understand what an Elastic Cloud Compute (EC2) instance is.\n", - "\n", - "```" - ] - }, - { - "cell_type": "markdown", - "id": "502f2360", - "metadata": {}, - "source": [ - "Recall that the two fundamental components of cloud computing is compute and storage. On AWS, a \"virtual server\" or \"virtual computer\" is known as an **Elastic Cloud Compute (EC2) instance**; sometimes it's called \"EC2\", sometimes it's called an \"instance\" to denote that the ability to build and terminate this server instantaneously, but they all mean the same thing. An EC2 instance is no different from a server that sits under your desk, or your local departmental cluster, or even your local HPC cluster. You even boot up an EC2 instance through the web console, install software and then shut down your instance just like you would a real computer, except that Amazon takes care of the physical machinery while you are in charge of process of creating the computer. In some sense, you can think of utilizing an EC2 instance as renting a server or computer from Amazon! \n", - "\n", - "In cloud jargon, the term **elasticity** denotes the ability to quickly expand or decrease computer processing, memory, and storage resources to meet changing demands. In that way, you can expand the size of your CPU, RAM and disk size on your EC2 instance almost instantenously. Since EC2 forms the backbone of most of AWS's core infrastructure, it is an important part of your cloud journey. \n", - " \n", - "\n", - "Let's walk through some of the steps on getting an EC2 instance up and running. \n" - ] - }, - { - "cell_type": "markdown", - "id": "bc5d082d", - "metadata": {}, - "source": [ - "We begin with the AWS console again. Under the \"Build a Solution\" panel, select `Launch a Virtual Machine`\n", - "\n", - "\n", - "\n", - "
Figure 1: Start page for the AWS console

\n", - "\n", - "This will then lead you through a series of steps to get a **Free Tier** EC2 instance up and running. \n", - "\n", - "```{admonition} Note\n", - ":class: note\n", - "\n", - "AWS Free Tier refers to several of the services that AWS offers to help users gain more hands on experience on the AWS platform without being charged. [Click here](https://aws.amazon.com/free/?all-free-tier.sort-by=item.additionalFields.SortRank&all-free-tier.sort-order=asc&awsf.Free%20Tier%20Types=*all&awsf.Free%20Tier%20Categories=*all) for more info about the AWS Free Tier [external link] . \n", - "```\n", - "\n", - "There are 7 steps to walk through to create a new EC2 instance; we will go through each in detail: \n", - "1. Select an AMI\n", - "2. Choose Instance Type\n", - "3. Configure Instance \n", - "4. Add Storage\n", - "5. Add Tags\n", - "6. Configure Security Group\n", - "7. Review/Launch" - ] - }, - { - "cell_type": "markdown", - "id": "b9809503", - "metadata": {}, - "source": [ - "## 1. Select an AMI\n", - "\n", - "An Amazon Machine Image (AMI) is a template that Amazon uses to describe the operating system, disk type and all the software configuration that is needed to make sure a computer runs smoothly. Imagine that you are purchasing a new laptop; fresh out of the box, the laptop is pre-configured with an operating system (e.g. Windows, Mac OS, Ubuntu etc.), configuration files that tells the laptop what peripherals are attached, and pre-installed software like Adobe PDF reader. An AMI contains all this information so that your EC2 instance runs exactly like it would a new laptop out of the box! There is much more to learn about AMIs and how they can used for collaboration and data sharing but that is not within the scope of CLASS Essentials. \n", - "\n", - "As you scroll through the AMI list (Figure 2) you will notice that the list contains offerings from various vendors (e.g. Amazon, RedHat, Windows, etc.). We will be choosing the Ubuntu operating system for flexibility and versatility (can be used outside of the AWS ecosystem). \n", - "\n", - "To list all the Free Tier AMIs, check the box on the right that says ```Free tier only```.\n", - "\n", - "\n", - "\n", - "
Figure 2: Step 1 - Select an AMI - Free Tier Only

\n", - "\n", - "Scroll to ```Ubuntu Server 20.04 LTS(HVM), SSD Volume Type``` (Figure 3). Select ```64-bit(x86)```. \n", - "\n", - "\n", - "\n", - "
Figure 3: Step 1 - Select an AMI - Operating System Selection

" - ] - }, - { - "cell_type": "markdown", - "id": "17597535", - "metadata": {}, - "source": [ - "## Step 2: Choose an Instance Type\n", - "\n", - "Choosing an instance type is choosing the hardware for your computing system: you get to pick the number of CPUs and memory size for your instance. \n", - "\n", - "Instance types are group by [**families**](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-types.html) and denotes whether, for example, an instance is optimized for batch processing (compute-optimized, C-family), optimized for databases (memory-optimized, R-family) or has accelerated hardware (GPUs) for AI or Machine Learning pipelines. \n", - "\n", - "When you choose an Instance Type (Figure 3), the screen show additional information about the selected instance type including the number of CPUs, the memory size, the type of storage and information about networking. \n", - "\n", - "In the Instance Storage (GB) column, you will notice a term called **EBS**. EBS is the acronym for **Elastic Block Storage** and is analogous to the hard disk or boot drive on your personal computer or laptop. More details about EBS and different kinds of disk storage on EC2 instances are beyond the scope of CLASS Essentials. \n", - "\n", - "```{admonition} Note\n", - ":class: note\n", - "The four most common types of storage you will encounter on AWS are: Elastic Block Storage (EBS), Elastic File Storage (EFS), Simple Storage Service (s3) and s3 Glacier. In the simplest terms, EBS is analogous to a computer hard drive and EFS is analogous to a network file system (NFS) or shared file system. s3 is AWS's object storage which is discussed [here](./intro_to_s3). s3 Glacier is a cost-effective way of storing s3 files that you do not need to access frequently. \n", - "```\n" - ] - }, - { - "cell_type": "markdown", - "id": "4ee655a3", - "metadata": {}, - "source": [ - "\n", - "\n", - "
Figure 3: Step 2 - Choose an Instance Type

\n", - "\n", - "Select ```Next: Configure Instance Details``` and we'll move on to the next chapter. " - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.9.7" - } - }, - "nbformat": 4, - "nbformat_minor": 5 -} diff --git a/content/Azure/05_intro_to_cloud_storage_gcp.ipynb b/content/Azure/05_intro_to_cloud_storage_gcp.ipynb deleted file mode 100644 index 98030c8..0000000 --- a/content/Azure/05_intro_to_cloud_storage_gcp.ipynb +++ /dev/null @@ -1,205 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "id": "677549a8", - "metadata": {}, - "source": [ - "# Introduction to Azure Cloud Storage\n", - "\n", - "Learner Questions\n", - " * How do I store data in the cloud?\n", - "\n", - "Learning Objectives:\n", - " * Navigate the Azure Cloud Storage service and terminology\n", - " * Understand the roles and permissions needed to use Azure Cloud Storage\n", - " * Allocate storage in Azure Cloud Storage account\n", - " * Allocate storage in Azure Cloud Storage\n", - " * De-allocate Azure Cloud Storage storage" - ] - }, - { - "cell_type": "markdown", - "id": "09e26920", - "metadata": {}, - "source": [ - "Drew, after working with a Research Computing and Data (RCD) facilitator, has decided to continue his cloud journey with storage.\n", - "\n", - "One of the most common and economic ways to store data in the cloud is to use object storage.\n" - ] - }, - { - "cell_type": "markdown", - "id": "d46f786b", - "metadata": { - "tags": [] - }, - "source": [ - "## Azure Cloud Storage\n", - "\n", - "To learn more about the Azure blob Storage service we will use the web console to create, explore, and destroy a storage container. The ability to quickly create, explore, and then discard resources in the web-console is a powerful tool to explore a service. After an initial exploration with the web-console, programmatic access using scripts and programming languages should be used.\n", - "\n", - "Unfortunately, the terminology used for object storage differs from one commercial cloud provider to another. Azure uses the terms Storage Container. Data is stored in Objects and Objects are stored in Containers. Objects (data) are stored as a key-value pair, which is similar to a Python dictionary but persistent. Values are stored and retrieved using a unique key and the value can contain any information (more on this later).\n", - "\n", - "We now take Drew through the process of creating a Google Cloud Storage bucket." - ] - }, - { - "cell_type": "markdown", - "id": "89b2cc65", - "metadata": { - "tags": [] - }, - "source": [ - "### Security (Still need to update for Azure)\n", - "\n", - "Everything in the cloud requires permission (authorization). Let's first verify that we have the permissions to create a bucket. A Bucket (a resource) is created within a project and inheres permissions from it.\n", - "\n", - "We are interested in what permissions that *your* account has for *your* project. To do this navigate to the IAM page (**Navigation Menu -> IAM & Admin -> IAM -> Permissions -> View By: Principals**). This shows the permissions for the project.\n", - "\n", - "*Note: There is a powerful filter box to limit the permissions shown.*\n", - "\n", - "You should see a row with your account shown in the Principal column. Here you should see the \"Owner\" Role in the Role column. A *role* is a collection of permissions managed by Google or someone else. The **Owner** or the **Storage Admin** role for a project will *allow* *you* to create, access, and delete Buckets *in* the project.\n", - "\n", - "There are three important pieces of information that work together to form the **IAM policy**. The permission (role), the identity (principal or member), and the resource (project)." - ] - }, - { - "cell_type": "markdown", - "id": "3e396a67", - "metadata": {}, - "source": [ - "### Create Resource Group (Note sure if this is needed)\n", - "\n", - "Alocating any resource in Azure requires a Resource Goup. Resources Groups scope your resources under a common entity for \n", - "resource interaction, IAM, billing (technically subsription but a resource group is under a subscription), ..., etc. \n", - "\n", - "* Navigate to Resource Groups\n", - "* Click \"+ create\" in the upper left \n", - "* Select a name for your Resource Group\n", - "* Leave the rest as default\n", - "* Click Review & Create\n", - "* After passing validation, click create\n" - ] - }, - { - "cell_type": "markdown", - "id": "4a8f2f30", - "metadata": {}, - "source": [ - "### Create storage account\n", - "\n", - "Like all things in Azure you will need an account to manage your service. In this case we need a storage account. To create a storage account, follow these steps.\n", - "\n", - "* Navigate to storage accounts\n", - "* Click \"+ create\" in the upper left \n", - "\n", - "\n", - "\n", - "* Select (or create) a Resource Group\n", - "* Select a name for your Storage account\n", - "* Leave the rest as default\n", - "* Click Review & Create\n", - "\n", - "\n", - "\n", - "* After passing validation, click create\n", - "* The page will indicate when the account has been deployed. Click on \"go to resource\"\n", - "\n", - "\n" - ] - }, - { - "cell_type": "markdown", - "id": "4860da70", - "metadata": { - "tags": [] - }, - "source": [ - "### Allocate Azure Cloud Storage\n", - "\n", - "To create a blob storage container in the Azure portal, follow these steps:\n", - "\n", - "* Navigate to your new storage account in the Azure portal.\n", - "* In the left menu for the storage account, scroll to the Data storage section, then select containers.\n", - "* Select the + Container button.\n", - "* Type a name for your new container.\n", - "* Select OK to create the container.\n", - "\n", - "\n" - ] - }, - { - "cell_type": "markdown", - "id": "d7bdd31c", - "metadata": {}, - "source": [ - "**Now click on the container**\n", - "* This is where your blob objects can be viewed, uploaded, downloaded, & deleted.\n", - "* Click \"Upload\"\n", - "* In the Upload blob menu, either type in a file name or click on the icon to the right to browse your local filesystem.\n", - "* Choose a file from your local filesystem\n", - "* click upload" - ] - }, - { - "cell_type": "markdown", - "id": "21adbab1", - "metadata": { - "tags": [] - }, - "source": [ - "### Enumerate the Buckets\n", - "\n", - "Now lets find and examine the bucket. To view a bucket we do the following:\n", - "\n", - "* Navigate to Storage Accounts in the Azure portal and select your new storage account.\n", - "* In your storage account select containers\n", - "* Under containers you should see the container you created in the previous step\n", - "* This verifies that a storage container was deployed\n" - ] - }, - { - "cell_type": "markdown", - "id": "9f28062e", - "metadata": { - "jp-MarkdownHeadingCollapsed": true, - "tags": [] - }, - "source": [ - "### Delete the Storage\n", - "\n", - "To complete the resource life-cycle we simply delete the storage container created in this lesson. All blobs in the container will also be deleted. This will leave you with a storage account that can be used again for additional stoarge services.\n", - "\n", - "To delete the container:\n", - "\n", - "* In the Azure portal, navigate to the list of containers in your storage account.\n", - "* Select the container to delete.\n", - "* Select the More button (...), and select Delete.\n", - "* Confirm that you want to delete the container.\n", - "\n" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.9.7" - } - }, - "nbformat": 4, - "nbformat_minor": 5 -} diff --git a/content/Azure/05_intro_to_cloud_storage_old.ipynb b/content/Azure/05_intro_to_cloud_storage_old.ipynb deleted file mode 100644 index f3f90ef..0000000 --- a/content/Azure/05_intro_to_cloud_storage_old.ipynb +++ /dev/null @@ -1,60 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "id": "dc57021c", - "metadata": {}, - "source": [ - "# Introduction to Cloud Storage\n", - "\n", - "\n", - "```{admonition} Overview\n", - ":class: tip\n", - "\n", - "\n", - "```" - ] - }, - { - "cell_type": "markdown", - "id": "502f2360", - "metadata": {}, - "source": [ - "```{code-cell} ipython3\n", - ":tags: [mytag]\n", - "\n", - "print(\"A python cell\")\n", - "```" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "c543528f", - "metadata": {}, - "outputs": [], - "source": [] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.9.5" - } - }, - "nbformat": 4, - "nbformat_minor": 5 -}