diff --git a/content/AWS/01_intro_to_cloud_console.ipynb b/content/AWS/01_intro_to_cloud_console.ipynb index ff5b77d..85c2618 100644 --- a/content/AWS/01_intro_to_cloud_console.ipynb +++ b/content/AWS/01_intro_to_cloud_console.ipynb @@ -2,7 +2,7 @@ "cells": [ { "cell_type": "markdown", - "id": "64325d0f", + "id": "ab4463e6", "metadata": {}, "source": [ "# Introduction to the AWS Console\n", @@ -28,43 +28,48 @@ }, { "cell_type": "markdown", - "id": "54c59223", + "id": "5ee1b145", "metadata": {}, "source": [] }, { "cell_type": "markdown", - "id": "2477a2d7", + "id": "660d1a7a", "metadata": {}, "source": [ "## Setup\n", "\n", - "The console is the point of entry where you will start your AWS journey. If you are participating in the CLASS Essentials Workshop, you will have received a csv file via email. This csv file contains your **username**, **temporary password** and **console login link**. \n", + "The console is the point of entry where you will start your AWS journey. If you are participating in the CLASS Essentials Workshop, you will be provisioned an AWS Academy Account. \n", "\n", "```{admonition} Be Aware\n", ":class: danger\n", - "If you are doing self-paced learning, your login information may be different!\n", + "If you are doing self-paced learning, you will need to have your own AWS account, some of the screencaps or instructions may differ!\n", "```" ] }, { "cell_type": "markdown", - "id": "c72b28be", + "id": "21873aae", "metadata": {}, "source": [ - "## Logging in to the console\n", - "Paste your console login link into your web browser. You will see the AWS site requesting you to sign in as an IAM user. Fill in your username and password that was assigned to you in the csv file. You will be asked to change your password the first time you sign in to the console. Fill in your old password and then choose a new password that you can remember! \n", + "## Using AWS Academy \n", "\n", - "You are now logged on. Welcome to the first step of your cloud journey! \n", + "The URL to AWS Academy is https://awsacademy.instructure.com/login/canvas. Once you sign in, the steps to get to the AWS console where we will begin our journey are as follows:\n", "\n", - "Below is what your screen should look like when you log in to the AWS console for the first time. If you encounter any pop up boxes, you can exit them. Next, let's take a closer look at some key concepts and components of the AWS console. \n", + "1. Click on 'Modules' on the left menu bar. \n", + "2. Select Learner Lab - Foundational Service \n", + "3. Click Start Lab on the top right corner\n", + "4. Click “AWS” button once the red dot turns to green, this will open the AWS Management Console” page\n", "\n", - "![First time logging in to the AWS console](images/intro_to_cloud_console/console_start.png)" + "```{admonition} Note\n", + ":class: attention\n", + "The pages may take some time to load especially if this your first time logging in to AWS Academy. If you encounter a blank page for a few seconds, please be patient. \n", + "```" ] }, { "cell_type": "markdown", - "id": "06ba8d29", + "id": "8a0329cb", "metadata": {}, "source": [ "## Key concepts and components of the AWS console\n", @@ -74,7 +79,7 @@ }, { "cell_type": "markdown", - "id": "b9a21e7b", + "id": "8528d175", "metadata": {}, "source": [ "Figure 2 lists the basic components you will see when you first log in to the AWS console. \n", @@ -83,7 +88,7 @@ ": If you click on the dropdown menu labeled \"Services\" you will see the entire list of services that is offered by AWS. Currently, AWS has over 200 services ranging from compute to machine learning to networking to data storage. The list grows frequently!\n", "\n", "**2. Search bar**\n", - ": You can also use the search bar to find a particular service or offering that you are interested in. Cloud vendors have proprietary vocabulary or jargon, so it is often useful to be aware of those as you start exploring more in your cloud adoption journey. [Internet2's CLASS Core documentation](https://github.internet2.edu/pages/CLASS/CLASS-Essentials-AWS/vendor_vocabulary.html) provides a basic list of vendor specific cloud terminology. For example, if you wanted to provision or utilize a \"virtual server\", you can try using the search bar to search for \"EC2\" which stands for \"Elastic Cloud Compute\" and is the AWS vocabulary for virtual computers.\n", + ": You can also use the search bar to find a particular service or offering that you are interested in. Cloud vendors have proprietary vocabulary or jargon, so it is often useful to be aware of those as you start exploring more in your cloud adoption journey. For example, if you wanted to provision or utilize a \"virtual server\", you can try using the search bar to search for \"EC2\" which stands for \"Elastic Cloud Compute\" and is the AWS vocabulary for virtual computers.\n", "\n", "**3. IAM Username and Account**\n", ": The menu bar on the console also lists your IAM username and account number. IAM is short for Identity and Access Management. Identity is a mechanism to authenticate users (e.g. password) and Access is a mechanism for authorizing what kinds of services users have access to. For example, in this scenario, student1 is an IAM user on the account \"4414-3982-1395\". In the creation of the IAM user \"student1\", the administrator (overlord) of the account assigned a particular permission scope (in AWS this is known as roles) to ensure that student1 only has access to a particular set of services and also assigned a password to authenticate user login. IAM is a key component in managing security on the cloud.\n", @@ -97,7 +102,7 @@ }, { "cell_type": "markdown", - "id": "14396589", + "id": "df94798a", "metadata": {}, "source": [ "```{admonition} Exercise\n", @@ -108,6 +113,14 @@ "* What are the three ways you can find an AWS service on the console?\n", "````" ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0e261320", + "metadata": {}, + "outputs": [], + "source": [] } ], "metadata": { diff --git a/content/AWS/02_intro_to_compute.ipynb b/content/AWS/02_intro_to_compute.ipynb deleted file mode 100644 index 0e43bcb..0000000 --- a/content/AWS/02_intro_to_compute.ipynb +++ /dev/null @@ -1,241 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "id": "99642f5e", - "metadata": {}, - "source": [ - "# Introduction to Elastic Cloud Compute (EC2) - Part 1\n", - "\n", - "\n", - "```{admonition} Overview\n", - ":class: tip\n", - "\n", - "**Teaching:** 45 mins\n", - "\n", - "**Exercises:** 10 mins\n", - "\n", - "**Questions:**\n", - "* What is an EC2 instance?\n", - "* When would I use an EC2 instance?\n", - "* How do I launch an EC2 instance?\n", - "\n", - "**Objectives:**\n", - "* Understand the concept of virtual servers.\n", - "* Understand what an Elastic Cloud Compute (EC2) instance is.\n", - "* Understand how to launch an EC2 instance. \n", - "\n", - "```" - ] - }, - { - "cell_type": "markdown", - "id": "fbcbf455", - "metadata": {}, - "source": [ - "Recall that the two fundamental components of cloud computing is compute and storage. On AWS, a \"virtual server\" or \"virtual computer\" is known as an **Elastic Cloud Compute (EC2) instance**; sometimes it's called \"EC2\", sometimes it's called an \"instance\" to denote that the ability to build and terminate this server instantaneously, but they all mean the same thing. An EC2 instance is no different from a server that sits under your desk, or your local departmental cluster, or even your local HPC cluster. You even boot up an EC2 instance through the web console, install software and then shut down your instance just like you would a real computer, except that Amazon takes care of the physical machinery while you are in charge of process of creating the computer. In some sense, you can think of utilizing an EC2 instance as renting a server or computer from Amazon! \n", - "\n", - "In cloud jargon, the term **elasticity** denotes the ability to quickly expand or decrease computer processing, memory, and storage resources to meet changing demands. In that way, you can expand the size of your CPU, RAM and disk size on your EC2 instance almost instantenously. Since EC2 forms the backbone of most of AWS's core infrastructure, it is an important part of your cloud journey. \n", - " \n", - "\n", - "Let's walk through some of the steps on getting an EC2 instance up and running. \n" - ] - }, - { - "cell_type": "markdown", - "id": "17329430", - "metadata": {}, - "source": [ - "We begin with the AWS console again. Under the \"Build a Solution\" panel, select `Launch a Virtual Machine`\n", - "\n", - "![Start page for the AWS console](images/console_ec2.png)\n", - "\n", - "This will then lead you through a series of steps to get a **Free Tier** EC2 instance up and running. \n", - "\n", - "```{admonition} Note\n", - ":class: note\n", - "\n", - "AWS Free Tier refers to several of the services that AWS offers to help users gain more hands on experience on the AWS platform without being charged. [Click here](https://aws.amazon.com/free/?all-free-tier.sort-by=item.additionalFields.SortRank&all-free-tier.sort-order=asc&awsf.Free%20Tier%20Types=*all&awsf.Free%20Tier%20Categories=*all) for more info about the AWS Free Tier [external link] . \n", - "```\n", - "\n", - "There are 7 steps to walk through to create a new EC2 instance; we will go through each in detail: \n", - "1. Select an AMI\n", - "2. Choose Instance Type\n", - "3. Configure Instance \n", - "4. Add Storage\n", - "5. Add Tags\n", - "6. Configure Security Group\n", - "7. Review/Launch" - ] - }, - { - "cell_type": "markdown", - "id": "efcd0d76", - "metadata": {}, - "source": [ - "## 1. Select an AMI\n", - "\n", - "An Amazon Machine Image (AMI) is a template that Amazon uses to describe the operating system, disk type and all the software configuration that is needed to make sure a computer runs smoothly. Imagine that you are purchasing a new laptop; fresh out of the box, the laptop is pre-configured with an operating system (e.g. Windows, Mac OS, Ubuntu etc.), configuration files that tells the laptop what peripherals are attached, and pre-installed software like Adobe PDF reader. An AMI contains all this information so that your EC2 instance runs exactly like it would a new laptop out of the box! There is much more to learn about AMIs and how they can used for collaboration and data sharing but that is not within the scope of CLASS Essentials. \n", - "\n", - "As you scroll through the AMI list (Figure 2) you will notice that the list contains offerings from various vendors (e.g. Amazon, RedHat, Windows, etc.). We will be choosing the Ubuntu operating system for flexibility and versatility (can be used outside of the AWS ecosystem). \n", - "\n", - "To list all the Free Tier AMIs, check the box on the right that says ```Free tier only```.\n", - "\n", - "\n", - "\n", - "
Figure 2: Step 1 - Select an AMI - Free Tier Only

\n", - "\n", - "Scroll to ```Ubuntu Server 20.04 LTS(HVM), SSD Volume Type``` (Figure 3). Select ```64-bit(x86)```. \n", - "\n", - "\n", - "\n", - "
Figure 3: Step 1 - Select an AMI - Operating System Selection

" - ] - }, - { - "cell_type": "markdown", - "id": "20372414", - "metadata": {}, - "source": [ - "## Step 2: Choose an Instance Type\n", - "\n", - "Choosing an instance type is choosing the hardware for your computing system: you get to pick the number of CPUs and memory size for your instance. \n", - "\n", - "Instance types are group by [**families**](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-types.html) and denotes whether, for example, an instance is optimized for batch processing (compute-optimized, C-family), optimized for databases (memory-optimized, R-family) or has accelerated hardware (GPUs) for AI or Machine Learning pipelines. \n", - "\n", - "When you choose an Instance Type (Figure 3), the screen show additional information about the selected instance type including the number of CPUs, the memory size, the type of storage and information about networking. \n", - "\n", - "In the Instance Storage (GB) column, you will notice a term called **EBS**. EBS is the acronym for **Elastic Block Storage** and is analogous to the hard disk or boot drive on your personal computer or laptop. More details about EBS and different kinds of disk storage on EC2 instances are beyond the scope of CLASS Essentials. \n", - "\n", - "```{admonition} Note\n", - ":class: note\n", - "The four most common types of storage you will encounter on AWS are: Elastic Block Storage (EBS), Elastic File Storage (EFS), Simple Storage Service (s3) and s3 Glacier. In the simplest terms, EBS is analogous to a computer hard drive and EFS is analogous to a network file system (NFS) or shared file system. s3 is AWS's object storage which is discussed [here](05_intro_to_cloud_storage). s3 Glacier is a cost-effective way of storing s3 files that you do not need to access frequently. \n", - "```\n", - "\n", - "Here will will select a ```t2.micro``` instance which is Free Tier Eligible but only has 1vCPU and 1 GiB of memory. The cost of running a t2.micro instance is " - ] - }, - { - "cell_type": "markdown", - "id": "ca6a7991", - "metadata": {}, - "source": [ - "![Choose an Instance Type](images/ec2-type.png)\n", - "\n", - "Select ```Next: Configure Instance Details```.\n", - "\n", - "## Step 3: Configure Instance Details\n", - "Step 3 in creating an EC2 instance involves a rudimentary understanding of several key AWS and cloud jargon (Figure 4). While delving deeper into some of the terminology is outside of the scope of CLASS Essentials, we go will through these terms in brief as we learn how to configure your EC2 instance. For the most part, we will **leave the settings as default**. CLASS Intermediate offers a more in depth discussion on cloud concepts. \n", - "\n", - "![Configure Instance Details](images/ec2-configure.png)\n", - "\n", - "**Number of instances** : This indicates how many instances you want to create at the same time. Here, we will leave the value as '1' but in actuality, you can can have up to 20 instances per region. \n", - "\n", - "```{admonition} Note\n", - ":class: note\n", - "Recall that we learned about regions in the [previous chapter](./01_intro_to_cloud_console). \n", - "```\n", - "\n", - "**Purchasing Options** : Throughout your AWS journey, you will hear the term **Spot Instances**. Spot instances make use of the servers that go unused in AWS data centers to minimize costs. Recall that AWS has many data centers spread across the globe and not all their servers are utilized at 100% capacity at all times. Amazon uses Spot Instances as a flexible way to profit from extra capacity. Users have access to Spot Instances through a bidding process, sometimes users can save up to 90% off the on-deman compute instance this way! We will not expand much more on Spot Instances in CLASS Essentials but if you are interested, I2's CLASS Intermediate talks more " - ] - }, - { - "cell_type": "markdown", - "id": "97271ddd", - "metadata": {}, - "source": [ - "```{admonition} Exercise\n", - ":class: attention\n", - "\n", - "* What kind of information is contained in an AMI? \n", - "* How do Spot Instances help you optimize costs?\n", - "````" - ] - }, - { - "cell_type": "markdown", - "id": "32d19e24", - "metadata": {}, - "source": [] - }, - { - "cell_type": "markdown", - "id": "7820264f", - "metadata": {}, - "source": [ - "## Step 4. Add Storage\n", - "\n", - "Storage on an EC2 instance is akin to a hard drive. Here we will leave the default settings but it is important to know that a hard drive on an EC2 instance is known as [Elastic Block Storage](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AmazonEBS.html). EBS volumes behave like raw, unformatted block devices.\n", - "\n", - "" - ] - }, - { - "cell_type": "markdown", - "id": "309bdc6d", - "metadata": {}, - "source": [ - "## Step 5: Add Tags\n", - "\n", - "[Tags](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Using_Tags.html) are labels that you assign to an AWS resource. Each tag consists of a key and an optional value, both of which you define.Tags enable you to categorize your AWS resources in different ways, for example, by purpose, owner, or environment. e\n", - "\n", - "\n" - ] - }, - { - "cell_type": "markdown", - "id": "2835744d", - "metadata": {}, - "source": [ - "## Step 6: Configure Security Group\n", - "\n", - "A [security group](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html) acts as a virtual firewall for your EC2 instances to control inbound and outbound traffic. Again, we will not delve too much into networking protocols in CLASS Essentials. We will leave the default values of opening port 22 so that we can securely log in to the EC2 instance that we create. \n", - "\n", - "" - ] - }, - { - "cell_type": "markdown", - "id": "56318ce3", - "metadata": {}, - "source": [ - "## Step 7: Review Instance Launch\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "0a851856", - "metadata": {}, - "outputs": [], - "source": [] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.10.2" - } - }, - "nbformat": 4, - "nbformat_minor": 5 -} diff --git a/content/AWS/02_intro_to_compute_part1.ipynb b/content/AWS/02_intro_to_compute_part1.ipynb index 285db29..adb5998 100644 --- a/content/AWS/02_intro_to_compute_part1.ipynb +++ b/content/AWS/02_intro_to_compute_part1.ipynb @@ -2,7 +2,7 @@ "cells": [ { "cell_type": "markdown", - "id": "ab9943d4", + "id": "57d9a12e", "metadata": {}, "source": [ "# Introduction to Elastic Cloud Compute (EC2)\n", @@ -30,7 +30,7 @@ }, { "cell_type": "markdown", - "id": "eea6fe0f", + "id": "5105c0ed", "metadata": {}, "source": [ "Recall that the two fundamental components of cloud computing is compute and storage. On AWS, a \"virtual server\" or \"virtual computer\" is known as an **Elastic Cloud Compute (EC2) instance**; sometimes it's called \"EC2\", sometimes it's called an \"instance\" to denote that the ability to build and terminate this server instantaneously, but they all mean the same thing. An EC2 instance is no different from a server that sits under your desk, or your local departmental cluster, or even your local HPC cluster. You even boot up an EC2 instance through the web console, install software and then shut down your instance just like you would a real computer, except that Amazon takes care of the physical machinery while you are in charge of process of creating the computer. In some sense, you can think of utilizing an EC2 instance as renting a server or computer from Amazon! \n", @@ -43,7 +43,7 @@ }, { "cell_type": "markdown", - "id": "ef47ad97", + "id": "8ec0c433", "metadata": {}, "source": [ "We begin with the AWS console again. Under the \"Build a Solution\" panel, select `Launch a Virtual Machine`\n", @@ -70,7 +70,7 @@ }, { "cell_type": "markdown", - "id": "a6f5824f", + "id": "d8201a47", "metadata": {}, "source": [ "## 1. Select an AMI\n", @@ -94,7 +94,7 @@ }, { "cell_type": "markdown", - "id": "3712a9db", + "id": "53271781", "metadata": {}, "source": [ "## Step 2: Choose an Instance Type\n", @@ -117,7 +117,7 @@ }, { "cell_type": "markdown", - "id": "c0e03ae3", + "id": "4095a194", "metadata": {}, "source": [ "![Choose an Instance Type](images/ec2-type.png)\n", @@ -125,23 +125,17 @@ "Select ```Next: Configure Instance Details```.\n", "\n", "## Step 3: Configure Instance Details\n", - "Step 3 in creating an EC2 instance involves a rudimentary understanding of several key AWS and cloud jargon (Figure 4). While delving deeper into some of the terminology is outside of the scope of CLASS Essentials, we go will through these terms in brief as we learn how to configure your EC2 instance. For the most part, we will **leave the settings as default**. CLASS Intermediate offers a more in depth discussion on cloud concepts. \n", + "Step 3 in creating an EC2 instance involves a rudimentary understanding of several key AWS and cloud jargon. Here we will **leave all the settings as default** except one. We will change the IAM role to `LabInstanceProfile`. \n", "\n", - "![Configure Instance Details](images/ec2-configure.png)\n", - "\n", - "**Number of instances** : This indicates how many instances you want to create at the same time. Here, we will leave the value as '1' but in actuality, you can can have up to 20 instances per region. \n", - "\n", - "```{admonition} Note\n", - ":class: note\n", - "Recall that we learned about regions in the [previous chapter](./01_intro_to_cloud_console). \n", - "```\n", + "Dlving deeper into some of the terminology is outside of the scope of CLASS Essentials, we go will through some of these terms in brief as we learn how to configure your EC2 instance. You can also click the grey \"i\" icon in the configuration list for a description. For the most part, we will **leave the settings as default**. \n", "\n", - "**Purchasing Options** : Throughout your AWS journey, you will hear the term **Spot Instances**. Spot instances make use of the servers that go unused in AWS data centers to minimize costs. Recall that AWS has many data centers spread across the globe and not all their servers are utilized at 100% capacity at all times. Amazon uses Spot Instances as a flexible way to profit from extra capacity. Users have access to Spot Instances through a bidding process, sometimes users can save up to 90% off the on-deman compute instance this way! We will not expand much more on Spot Instances in CLASS Essentials but if you are interested, I2's CLASS Intermediate talks more " + "![Configure Instance Details](images/ec2-configure.png)\n", + "\n" ] }, { "cell_type": "markdown", - "id": "c7029aff", + "id": "b5207387", "metadata": {}, "source": [ "```{admonition} Exercise\n", @@ -154,53 +148,76 @@ }, { "cell_type": "markdown", - "id": "8a741315", + "id": "23a8b7bf", "metadata": {}, - "source": [] + "source": [ + "Some important terms to note: \n", + "\n", + "**Number of instances** : This indicates how many instances you want to create at the same time. Here, we will leave the value as '1' but in actuality, you can can have up to 20 instances per region. \n", + "\n", + "```{admonition} Note\n", + ":class: note\n", + "Recall that we learned about regions in the [previous chapter](./01_intro_to_cloud_console). \n", + "```\n", + "\n", + "**Purchasing Options** : Throughout your AWS journey, you will hear the term **Spot Instances**. Spot instances make use of the servers that go unused in AWS data centers to minimize costs. Recall that AWS has many data centers spread across the globe and not all their servers are utilized at 100% capacity at all times. Amazon uses Spot Instances as a flexible way to profit from extra capacity. Users have access to Spot Instances through a bidding process, sometimes users can save up to 90% off the on-demand compute instance this way! \n", + "\n", + "**IAM Role** : An IAM role is an identity that you can create in your account that has specific permissions. An IAM role has some similarities to an IAM user. Roles and users are both AWS identities with permissions policies that determine what the identity can and cannot do in AWS. \n" + ] }, { "cell_type": "markdown", - "id": "36f10e0d", + "id": "daa16890", "metadata": {}, "source": [ "## Step 4. Add Storage\n", "\n", - "Storage on an EC2 instance is akin to a hard drive. Here we will leave the default settings but it is important to know that a hard drive on an EC2 instance is known as [Elastic Block Storage](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AmazonEBS.html). EBS volumes behave like raw, unformatted block devices.\n", + "Storage on an EC2 instance is akin to a hard drive. Here we will leave the default settings but it is important to know that a hard drive on an EC2 instance is known as [Elastic Block Storage](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AmazonEBS.html). EBS volumes behave like raw, unformatted block devices. Sometimes an EBS volume is also known as a persistent disk. \n", "\n", "" ] }, { "cell_type": "markdown", - "id": "3e2da435", + "id": "673573ec", "metadata": {}, "source": [ "## Step 5: Add Tags\n", "\n", - "[Tags](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Using_Tags.html) are labels that you assign to an AWS resource. Each tag consists of a key and an optional value, both of which you define.Tags enable you to categorize your AWS resources in different ways, for example, by purpose, owner, or environment. e\n", + "[Tags](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Using_Tags.html) are labels that you assign to an AWS resource. Each tag consists of a key and an optional value, both of which you define.Tags enable you to categorize your AWS resources in different ways, for example, by purpose, owner, or environment. Tags are an important mechanism to help you manage costs.\n", "\n", "\n" ] }, { "cell_type": "markdown", - "id": "6dda5e17", + "id": "c4290fa7", "metadata": {}, "source": [ "## Step 6: Configure Security Group\n", "\n", - "A [security group](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html) acts as a virtual firewall for your EC2 instances to control inbound and outbound traffic. Again, we will not delve too much into networking protocols in CLASS Essentials. We will leave the default values of opening port 22 so that we can securely log in to the EC2 instance that we create. \n", + "A [security group](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html) acts as a virtual firewall for your EC2 instances to control inbound and outbound traffic. Again, we will not delve too much into networking protocols in CLASS Essentials. We will leave the default values of opening port 22. \n", "\n", "" ] }, { "cell_type": "markdown", - "id": "49aa09c0", + "id": "8bd0eb43", "metadata": {}, "source": [ "## Step 7: Review Instance Launch\n", "\n", + "The final step before your instance is provisioned, is for you to review your setup. You will also be asked to `Create a new key pair`. This step is important if you will securely assessing your instance from **outside** the AWS console. Go ahead and give your key pair a name and select `Download Key Pair`. The `Launch Instance` button will then turn blue once you have downloaded your key pair. \n", + "\n", + "When you have launched the instance you will be immediately taken to the Launch Status page where you can see your instance being initiated. \n" + ] + }, + { + "cell_type": "markdown", + "id": "657919d4", + "metadata": {}, + "source": [ "\n", "\n", "\n", @@ -211,7 +228,7 @@ { "cell_type": "code", "execution_count": null, - "id": "40dc7b7d", + "id": "d99812c2", "metadata": {}, "outputs": [], "source": [] diff --git a/content/AWS/03_intro_to_cloud_storage.ipynb b/content/AWS/03_intro_to_cloud_storage.ipynb index 6bd8643..1a41321 100644 --- a/content/AWS/03_intro_to_cloud_storage.ipynb +++ b/content/AWS/03_intro_to_cloud_storage.ipynb @@ -2,7 +2,7 @@ "cells": [ { "cell_type": "markdown", - "id": "458bf7cc", + "id": "28255d12", "metadata": {}, "source": [ "# Introduction to Cloud Storage\n", @@ -11,9 +11,9 @@ "```{admonition} Overview\n", ":class: tip\n", "\n", - "**Teaching: 20 mins**\n", + "**Teaching: 15 mins**\n", "\n", - "**Exercises: 10 mins**\n", + "**Exercises: -**\n", "\n", "**Questions: **\n", "* What are the different kinds of storage on the cloud?\n", @@ -29,7 +29,7 @@ }, { "cell_type": "markdown", - "id": "9a0023d6", + "id": "a56a7c17", "metadata": {}, "source": [ "There are three types of [cloud data storage](https://aws.amazon.com/what-is-cloud-storage/): object storage, file storage, and block storage. In this module, we will focus on object storage (e.g. Amazon Simple Storage Service (S3)). Object storage is a technology that manages data as objects. All data is stored in one large repository which may be distributed across multiple physical storage devices, instead of being divided into files or folders.\n", @@ -44,7 +44,7 @@ }, { "cell_type": "markdown", - "id": "e347eb69", + "id": "cf62cd7c", "metadata": {}, "source": [ "Here we will click into the s3 service page. Note that the region here is Global. s3 namespaces(the name of the buckets) are global. This means that no two buckets can have identical names even if they reside in a different regions. \n", @@ -56,7 +56,7 @@ }, { "cell_type": "markdown", - "id": "3228bf08", + "id": "1bad1353", "metadata": {}, "source": [ "This will bring you to the Create Bucket page. Here we will choose a name for our new bucket - it will need to be a unique global namespace. Here I will use my identifying IAM (user1783892) to create a bucket. We will name my bucket ```bucket-user1783892```, leave the region as us-east-1 as well as all the default settings and click ```Create Bucket```\n", @@ -66,17 +66,17 @@ }, { "cell_type": "markdown", - "id": "1504f939", + "id": "2d2f51f3", "metadata": {}, "source": [ - "When your bucket is successfully created, you will see it pop up in the s3 console. \n", + "When your bucket is successfully created, you will see it pop up in the s3 console. Note the Access policy for this bucket. \n", "\n", "" ] }, { "cell_type": "markdown", - "id": "99e53e32", + "id": "65643baa", "metadata": {}, "source": [ "In the next lesson, we will learn about the AWS CLI and how we can use that to manipulate both the EC2 and s3 bucket we have created. " @@ -85,7 +85,7 @@ { "cell_type": "code", "execution_count": null, - "id": "839aede6", + "id": "fb65d8d5", "metadata": {}, "outputs": [], "source": [] diff --git a/content/AWS/04_intro_to_cli.ipynb b/content/AWS/04_intro_to_cli.ipynb index 8347c83..5783e2c 100644 --- a/content/AWS/04_intro_to_cli.ipynb +++ b/content/AWS/04_intro_to_cli.ipynb @@ -2,7 +2,7 @@ "cells": [ { "cell_type": "markdown", - "id": "b0cf1425", + "id": "c8344d92", "metadata": {}, "source": [ "# Introduction to the AWS CLI\n", @@ -26,7 +26,7 @@ }, { "cell_type": "markdown", - "id": "a25aac4e", + "id": "5a6a87bf", "metadata": {}, "source": [ "# The AWS CloudShell\n", @@ -35,7 +35,7 @@ "\n", "AWS CloudShell is a browser-based shell that makes it easy to securely manage, explore, and interact with your AWS resources. With CloudShell, you can quickly run scripts with the AWS Command Line Interface (AWS CLI). You can use CloudShell right from your browser and at no additional cost. \n", "\n", - "While your s3 bucket from the previously lesson is launching, we will navigate to the AWS CloudShell. You can find it in the top toolbar of the console. Clicking the icon will launch the AWS CloudShell. \n", + "You can find the AWS CloudShell on the top toolbar of the console. Clicking the icon will launch the AWS CloudShell. \n", "\n", "If this is your first time using the cloud console, you may encounter a pop up. It is fine to dismiss it. \n", "\n", @@ -52,7 +52,7 @@ }, { "cell_type": "markdown", - "id": "5ca7a3af", + "id": "c56f7b07", "metadata": {}, "source": [ "# AWS CLI and s3 buckets\n", @@ -129,7 +129,7 @@ { "cell_type": "code", "execution_count": null, - "id": "47313c4f", + "id": "91630f3f", "metadata": {}, "outputs": [], "source": [] diff --git a/content/AWS/05_running_analysis.ipynb b/content/AWS/05_running_analysis.ipynb index 9427a42..edd93a7 100644 --- a/content/AWS/05_running_analysis.ipynb +++ b/content/AWS/05_running_analysis.ipynb @@ -2,7 +2,7 @@ "cells": [ { "cell_type": "markdown", - "id": "7deb7a74", + "id": "1467f539", "metadata": {}, "source": [ "# Putting It All Together\n", @@ -25,7 +25,7 @@ }, { "cell_type": "markdown", - "id": "f9d82db5", + "id": "fa523b0d", "metadata": {}, "source": [ "# AWS CLI and EC2\n", @@ -70,7 +70,7 @@ }, { "cell_type": "markdown", - "id": "0992a27a", + "id": "5bb7a316", "metadata": {}, "source": [ "# Get example code onto EC2 instance\n", @@ -108,7 +108,7 @@ }, { "cell_type": "markdown", - "id": "ca96288c", + "id": "75d8670c", "metadata": {}, "source": [ "# Public s3 buckets \n", @@ -147,7 +147,7 @@ }, { "cell_type": "markdown", - "id": "45b0304b", + "id": "4ae6f4fd", "metadata": {}, "source": [ "# Running an Analysis\n", @@ -245,7 +245,7 @@ }, { "cell_type": "markdown", - "id": "ab907e17", + "id": "63ab55d9", "metadata": {}, "source": [ "We see that in 'process_sat.py' we will be opening the Red and Near Infrared band overview files using rasterio, then calcuting the Normalized Difference Vegetation Index (NDVI) which is a measure of changes in vegetation or landcover. \n", @@ -287,7 +287,7 @@ }, { "cell_type": "markdown", - "id": "db5a1d74", + "id": "ad0f8861", "metadata": {}, "source": [ "Let's check if we obtained an image in our directory:\n", @@ -318,14 +318,6 @@ "> 2022-02-08 05:51:01 1837900 LC08_L1TP_042034_20170616_20170629_01_T1_NDVI_OVIEW.tif \n", "2022-02-04 06:45:30 26 hemingway.txt" ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "426245b5", - "metadata": {}, - "outputs": [], - "source": [] } ], "metadata": { diff --git a/content/AWS/06_monitoring_costs.ipynb b/content/AWS/06_monitoring_costs.ipynb index e469660..d585729 100644 --- a/content/AWS/06_monitoring_costs.ipynb +++ b/content/AWS/06_monitoring_costs.ipynb @@ -2,7 +2,7 @@ "cells": [ { "cell_type": "markdown", - "id": "ea763c0e", + "id": "cddbc6bb", "metadata": {}, "source": [ "# Monitoring Costs\n", @@ -11,7 +11,7 @@ "```{admonition} Overview\n", ":class: tip\n", "\n", - "**Teaching:** 15 mins\n", + "**Teaching:** 20 mins\n", "\n", "**Exercises:** -\n", "\n", @@ -28,7 +28,7 @@ }, { "cell_type": "markdown", - "id": "bcf032a4", + "id": "09fb6b50", "metadata": {}, "source": [ "# Ways to monitor resources on your AWS account\n", @@ -36,7 +36,7 @@ "There are several ways to monitor what kinds of services that are running on your AWS account. We will focus on two: The AWS Tag Editor and the EC2 global view. \n", "\n", "\n", - "The AWS Tag Editor is a offers a way to look at resources that ar running in your account. Recall in Episode 2 that tagging resources is a best practice to manage costs. The AWS Tag Editor finds resources on your account and allows you tag them if you haven't already done so. It is a good tool to list ALL resources that are running on your account, or to filter them using a specific criteria. Advanced use of the AWS Tag Editor is beyond the scope of CLASS Essentials. Here we will use the AWS Tag Editor to list all the resources on our account. \n", + "The AWS Tag Editor is a offers a way to look at resources that ar running in your account. Recall in Module 2 that tagging resources is a best practice to manage costs. The AWS Tag Editor finds resources on your account and allows you tag them if you haven't already done so. It is a good tool to list ALL resources that are running on your account, or to filter them using a specific criteria. Advanced use of the AWS Tag Editor is beyond the scope of CLASS Essentials. Here we will use the AWS Tag Editor to list all the resources on our account. \n", "\n", "1. Search for `Tag Editor` in the navigation bar of the AWS console\n", "2. On the sidebar menu, click `Tag Editor`\n", @@ -55,6 +55,9 @@ " the region the resource is provisioned in \n", " all of the tags on the resource. \n", " Note that you can click on the badge with the number of tags to display the resource's tags\n", + " \n", + "Notice that when you do this, resources that you didn't even provision show up here. The AWS tag editor shows **all** the resources that have been provisioned by the entire organization! Imagine trying to filter through the entire list to find out what resources you have running. This is why Tagging and Regions are an important part of managing costs as it allows you to quickly filter through a list of projects, users and other identifying information. \n", + "\n", "\n", "The AWS EC2 global view is another way to check if you have EC2 instances running in *any* region. \n", "\n", @@ -74,7 +77,7 @@ }, { "cell_type": "markdown", - "id": "71c88189", + "id": "4d497066", "metadata": {}, "source": [ "# The AWS billing dashboard" @@ -82,16 +85,26 @@ }, { "cell_type": "markdown", - "id": "4319f2f6", + "id": "ab8830f7", + "metadata": {}, + "source": [ + "The AWS billing is disabled for AWS Academy. The granularity to which you can view your billing details varies by institution. However, if you are able to view the Billing Dashboard, AWS offers a detailed view of resources and estimated spend. The image below shows an example of the AWS Billing Dashboard. You can get to the Billing Dashboard by clicking your username on the top right corner of the navigation bar. \n", + "\n", + "" + ] + }, + { + "cell_type": "markdown", + "id": "2e12259f", "metadata": {}, "source": [ - "The AWS billing is disabled for AWS Academy. The granularity to which you can view your billing details varies by institution. However, if you are able to view the Billing Dashboard, AWS offers a detailed view of resources and estimated spend. The image below shows an example of the AWS Billing Dashboard. You can get to the Billing Dashboard by clicking your username on the top right corner of the navigation bar. " + "# The AWS Calculator" ] }, { "cell_type": "code", "execution_count": null, - "id": "fa28c172", + "id": "01e4d87d", "metadata": {}, "outputs": [], "source": [] diff --git a/content/AWS/07_cleaning_up_resources.ipynb b/content/AWS/07_cleaning_up_resources.ipynb index 5d96ba5..5928348 100644 --- a/content/AWS/07_cleaning_up_resources.ipynb +++ b/content/AWS/07_cleaning_up_resources.ipynb @@ -2,7 +2,7 @@ "cells": [ { "cell_type": "markdown", - "id": "e3dd2396", + "id": "1829d23b", "metadata": {}, "source": [ "# Cleaning Up Resources\n", @@ -25,12 +25,89 @@ ] }, { - "cell_type": "code", - "execution_count": null, - "id": "40ee5bca", + "cell_type": "markdown", + "id": "1969eb76", "metadata": {}, - "outputs": [], - "source": [] + "source": [ + "# Deleting a virtual machine\n", + "\n", + "We will use the console to delete our virtual machine. Let's return to the EC2 console. The steps to delete a virtual machine are:\n", + "\n", + "1. In the navigation pane, choose EC2.\n", + "2. Select the instance, and choose Instance state > Terminate instance.\n", + "3. Choose Terminate when prompted for confirmation.\n", + "\n", + "\n", + "```{admonition} Exercise\n", + ":class: attention\n", + "What is the difference between `Stopping` and `Terminating` an instance? \n", + "```\n", + "\n", + "```{dropdown} Answer\n", + "\n", + " The key difference between stopping and terminating an instance is that the attached bootable EBS volume will not be deleted. The data on your EBS volume will remain after stopping while all information on the local (ephemeral) hard drive will be lost as usual. The volume will continue to persist in its availability zone. Standard charges for EBS volumes will apply. \n", + "\n", + "```\n" + ] + }, + { + "cell_type": "markdown", + "id": "cab7ac33", + "metadata": {}, + "source": [ + "# Deleting an s3 bucket\n", + "\n", + "You can also use the AWS console to delete the s3 bucket you created earlier, but we will explore how the AWS CLI can be used to achieve it similiarly. \n", + "\n", + "1. Open the AWS CloudShell\n", + "2. List your s3 buckets\n", + "\n", + "```bash\n", + "aws s3 ls\n", + "```\n", + "\n", + "3. Try to remove the bucket by typing this command:\n", + "```bash\n", + "aws s3 rb s3://bucket-userXXXXXXXX\n", + "```\n", + "\n", + "You will receive this error:\n", + "\n", + "> remove_bucket failed: s3://user1824906-bucket An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: The bucket you tried to delete is not empty. \n", + "\n", + "This is because the bucket cannot be deleted if it is not empty. Let's try deleting the files inside the bucket. \n", + "\n", + "4. Delete files inside the bucket\n", + "```bash\n", + "aws s3 rm s3://bucket-userXXXXXXXX/*\n", + "```\n", + "\n", + "5. Delete bucket\n", + "aws s3 rb s3://bucket-userXXXXXXXX\n", + "\n", + "Finally let's check that all the resources we have provision throughout the lesson has been deleted. How would you do that?\n", + "\n", + "```{admonition} Exercise\n", + ":class: attention\n", + "How would you check if your resources have been deleted?\n", + "```\n", + "\n", + "```{dropdown} Answer\n", + "You can use the AWS Tag Editor to check if your instance and s3 bucket has been deleted. You can also use the EC2 Global View and s3 console to double check. \n", + "\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "7a42bcfb", + "metadata": {}, + "source": [ + "# Additional Steps if using AWS Academy \n", + "\n", + "1. Return to your AWS Academy Homepage\n", + "2. Stop Lab" + ] } ], "metadata": { diff --git a/content/AWS/07_monitoring_costs.ipynb b/content/AWS/07_monitoring_costs.ipynb deleted file mode 100644 index 5aaf0dd..0000000 --- a/content/AWS/07_monitoring_costs.ipynb +++ /dev/null @@ -1,47 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "id": "dc57021c", - "metadata": {}, - "source": [ - "# Monitoring Costs\n", - "\n", - "\n", - "```{admonition} Overview\n", - ":class: tip\n", - "\n", - "**Teaching:**\n", - "\n", - "**Exercises:**\n", - "\n", - "**Questions:**\n", - "\n", - "**Objectives:**\n", - "\n", - "```" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.7.10" - } - }, - "nbformat": 4, - "nbformat_minor": 5 -} diff --git a/content/AWS/08_cleaning_up_resources.ipynb b/content/AWS/08_cleaning_up_resources.ipynb deleted file mode 100644 index 665eaa7..0000000 --- a/content/AWS/08_cleaning_up_resources.ipynb +++ /dev/null @@ -1,47 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "id": "dc57021c", - "metadata": {}, - "source": [ - "# Cleaning Up Resources\n", - "\n", - "\n", - "```{admonition} Overview\n", - ":class: tip\n", - "\n", - "**Teaching:**\n", - "\n", - "**Exercises:**\n", - "\n", - "**Questions:**\n", - "\n", - "**Objectives:**\n", - "\n", - "```" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.7.10" - } - }, - "nbformat": 4, - "nbformat_minor": 5 -} diff --git a/content/AWS/images/billing_dashboard_1.png b/content/AWS/images/billing_dashboard_1.png new file mode 100644 index 0000000..32f509f Binary files /dev/null and b/content/AWS/images/billing_dashboard_1.png differ diff --git a/content/_config.yml b/content/_config.yml index fc1ed47..0bb2c65 100644 --- a/content/_config.yml +++ b/content/_config.yml @@ -3,6 +3,7 @@ title: Internet2 Cloud Learning And Skills Sessions author: Internet2 Research Engagement +copyright: "2022" # Copyright year to be placed in the footer logo: CLASS-logo.png # Force re-execution of notebooks on each build. diff --git a/content/intro.md b/content/intro.md index 0615376..a60b237 100644 --- a/content/intro.md +++ b/content/intro.md @@ -22,10 +22,6 @@ If you are taking the CLASS Essentials Workshop offered by Internet2, cloud acco For self-paced learning, you will need an institutional account, an agency-funded account (e.g. via Cloudbank), or a personal cloud account. Please note that we are unable to support or troubleshoot problems or errors that occur for self-paced learning. -**Knowledge of a POSIX compliant shell, such as bash, zsh, etc.** - -You will be using a bash shell to securely log on to a virtual machine and to run some analysis. As such, you will need to have a command line shell installed on your computer. Please follow the instructions [here](https://carpentries.github.io/workshop-template/#shell). For further information on the UNIX Shell, please see the [UNIX Shell Lesson on Software Carpentry](https://swcarpentry.github.io/shell-novice/). - ## CLASS Cloud Adoption Framework @@ -36,18 +32,12 @@ Cloud computing is a utility. Since you pay for the resources you use, it can be The CLASS Cloud Adoption Framework is a collection of documentation, best practices, tutorials, workflows, and cloud adoption pathways built by researchers and research and data (RCD) professionals ***for*** researchers and RCD professionals. In "adopting" the cloud, the onus is on users to manage costs, to understand the myriad cloud computing jargon, and to rethink how to adapt traditional workflows to a more "cloud-native" solution. -## More Resources - -If you are new to cloud computing for research, you may want to begin with [CLASS Core: A Primer to Cloud Computing for Research](https://github.internet2.edu/pages/CLASS/CLASS-core/intro.html). - ## Issues -Please create a [Github issue](https://github.internet2.edu/CLASS/CLASS-Essentials-AWS/issues) or contact - if you have questions, suggestions or comments. - +Contact [the CLASS team](mailto:class@internet2.edu?subject=Question%20about%20CLASS-Essentials-AWS) if you have questions, suggestions or comments. ## Acknowledgements -Content for this site was put together by John Hicks, Tim Middelkoop, Ananya Ravipati and Amanda Tan at Internet2 in collaboration with the CLASS and research community. [Find out more here](https://internet2.edu/community/research-engagement/internet2-research-engagement-team/)! +Content for this site was authored by John Hicks, Tim Middelkoop, Ananya Ravipati and Amanda Tan at Internet2 in collaboration with the CLASS and research community. [Find out more here](https://internet2.edu/community/research-engagement/internet2-research-engagement-team/)!