Skip to content
This repository was archived by the owner on Dec 12, 2025. It is now read-only.

Commit

Permalink
major overhaul 05-analysis & 02-compute
Browse files Browse the repository at this point in the history
  • Loading branch information
@jhicks
jhicks committed Feb 25, 2022
1 parent 5e114ed commit 85c7a66
Show file tree
Hide file tree
Showing 6 changed files with 621 additions and 368 deletions.
2 changes: 1 addition & 1 deletion content/AWS/05_running_analysis.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -336,7 +336,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.2"
"version": "3.9.7"
}
},
"nbformat": 4,
Expand Down
5 changes: 3 additions & 2 deletions content/Azure/01_intro_to_cloud_console.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -210,7 +210,7 @@
},
{
"cell_type": "markdown",
"id": "a1c0e980",
"id": "c58c867b",
"metadata": {},
"source": [
"This prompt is asking you to create a .ssh directory in your home directory and generate ssh public/private keys named id_rsa.pub/id_rsa (default) respectively.\n",
Expand Down Expand Up @@ -245,7 +245,8 @@
"+----[SHA256]-----+\n",
"```\n",
"\n",
"The ssh key generation is now complete. This will only happen once for your account. Later during the VM create proces you will be asked to Cut & Paste the public key into a form. "
"The ssh key generation is now complete. You will only need to do this once for your account. Later during the VM \n",
"create proces you will be asked to Cut & Paste the public key into a form. "
]
},
{
Expand Down
195 changes: 109 additions & 86 deletions content/Azure/02_intro_to_compute.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -30,59 +30,81 @@
},
{
"cell_type": "markdown",
"id": "92622861",
"id": "cb09d264",
"metadata": {},
"source": [
"Drew, after working with a Research Computing and Data (RCD) facilitator, has decided to start their cloud journey with compute, specifically a Virtual Machine, to process data in the cloud with a large and powerful machine to do the analysis. "
"Drew, after working with a Research Computing and Data (RCD) facilitator, has decided to start their cloud journey\n",
"with compute to process data in the cloud with a Virtual Machine to do the analysis. "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b85b77ed",
"cell_type": "markdown",
"id": "485a754a",
"metadata": {},
"outputs": [],
"source": [
"## Azure Virtual Machines"
]
},
{
"cell_type": "markdown",
"id": "ce935ec7",
"id": "0ef69f07",
"metadata": {},
"source": [
"Before we begin this lesson/episode we need to create a ssh key in order to login into the VM we about to create. We will use the public key when we create the VM.\n",
"Before we begin this lesson/episode and create a Virtual Machine (VM), we need to create a ssh \n",
"key so that we can securely login to the VM we are about to create. We need use a ssh public \n",
"key when we create the VM.\n",
"\n",
"**Note:** Step 1. below is not required if you have already created a ssh key in the [Intro to Cloud Console](01_intro_to_cloud_console.ipynb) lesson. If you have already created a key go directly to step 2.\n",
"**Note:** Step 1. below is not required if you have already created a ssh key in the \n",
"[Intro to Cloud Console](02_intro_to_compute.ipynb) lesson. If you have already \n",
"created a key go directly to step 2.\n",
"\n",
"**Step 1.**\n",
"\n",
"If you have not used the cloud shell to connect to a *VM Instance* before you will need to create a new *ssh key*. The Compute Engine will use this key to allow you to access the *VM instance* in a secure manner. If this is the case you will see a message similar to the following:\n",
"```\n",
"WARNING: The private SSH key file for gcloud does not exist.\n",
"WARNING: The public SSH key file for gcloud does not exist.\n",
"WARNING: You do not have an SSH key for gcloud.\n",
"WARNING: SSH keygen will be executed to generate a key.\n",
"This tool needs to create the directory [/home/learner/.ssh] before\n",
" being able to generate SSH keys.\n",
"If you have not used the cloud shell to connect to a *VM Instance* before, you will need to \n",
"create a new *ssh key*. The VM will use this key to allow you to access the *VM instance* \n",
"in a secure manner. \n",
"\n",
"Do you want to continue (Y/n)?\n",
"```\n",
"Open up a Cloud Shell by clicking on the Cloud Shell icon in the right side of the top \n",
"blue bar (the icon looks like box with a greater than sign followed by an underscore) as \n",
"described in the [Intro to Cloud Console](01_intro_to_cloud_console.ipynb) lesson.\n",
"\n",
"At the $ prompt in the Cloud Shell, enter ssh-keygen and follow the prompts.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c5889478",
"metadata": {},
"outputs": [],
"source": [
"$ssh-keygen\n",
"Generating public/private rsa key pair.\n",
"Enter file in which to save the key (/home/john/.ssh/id_rsa):"
]
},
{
"cell_type": "markdown",
"id": "164cdb8c",
"metadata": {},
"source": [
"This prompt is asking you to create a .ssh directory in your home directory and generate ssh \n",
"public/private keys named id_rsa.pub/id_rsa (default) respectively.\n",
"\n",
"Press enter to continue to create the key. You will see another message asking you for a \n",
"passphrase, similar to the following:\n",
"\n",
"Press enter to continue to create the key. You will see another message asking you for a passphrase, similar to the following:\n",
"```\n",
"Generating public/private rsa key pair.\n",
"Enter passphrase (empty for no passphrase):\n",
"```\n",
"Here you should enter a passphrase (a series of words) to protect the key. You will be asked to re-enter the passphrase as shown below:\n",
"```\n",
"Enter same passphrase again:\n",
"```\n",
"Here you should enter a passphrase (a series of words) to protect the key. You will be asked \n",
"to re-enter the passphrase as shown below:\n",
"\n",
"After successfully retyping the passphrase the key generation process will continue showing similar message as follows (the fingerprint and art will look different):\n",
"```\n",
"Your identification has been saved in /home/learner/.ssh/azure_compute_engine.\n",
"Your public key has been saved in /home/learner/.ssh/azure_compute_engine.pub.\n",
"Enter same passphrase again:\n",
"After successfully retyping the passphrase the key generation process will continue showing a \n",
"similar message as follows (the fingerprint and art will look different):\n",
"````\n",
"Your identification has been saved in /home/learner/.ssh/id_rsa.\n",
"Your public key has been saved in /home/learner/.ssh/id_rsa.pub.\n",
"The key fingerprint is:\n",
"SHA256:D9yj1nZEUigahirvjkIFoCP7RfjDu67Ku88M7nHQftI learner@cs-748001990186-default-boost-r9phq\n",
"The key's randomart image is:\n",
Expand All @@ -97,86 +119,92 @@
"|+.B o. . . . |\n",
"|=O**o |\n",
"+----[SHA256]-----+\n",
"```\n",
"\n",
"The ssh key generation is now complete. This will only happen once for your account. Later during the VM create proces you will be asked to Cut & Paste the public key into a form. \n",
"````\n",
"\n",
"\n",
"**Step 2.**\n",
"Copy the public key to your clip board future connections.\n"
"The ssh key generation is now complete. You will only need to do this once for your account. \n",
"Next, you will need to Cut (copy) and later Paste the public key during the VM creation process."
]
},
{
"cell_type": "markdown",
"id": "0f47f212",
"id": "d2237859",
"metadata": {},
"source": [
"**Step 2.**\n",
"\n",
"To Cut (copy) your public key, run the following command."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e4971654",
"metadata": {},
"outputs": [],
"source": [
"cat /home/learner/.ssh/id_rsa.pub"
]
},
{
"cell_type": "markdown",
"id": "d7f0bf3c",
"id": "b758abd3",
"metadata": {},
"source": [
"Highlight the output string with your cursor and use the copy command (depending on OS) to put it in your edit buffer."
"Highlight the output string with your cursor starting with ssh-rsa until the end of the string (white test) and \n",
"before you come to the green command prompt. With the public key highlighted, use the copy command (depending on OS) \n",
"to put it in your edit buffer (i.e. for mac use command c)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0d5b8abd",
"cell_type": "markdown",
"id": "84ddff97",
"metadata": {},
"outputs": [],
"source": [
"## Create a VM\n",
"\n",
"Hamburger in upper left -> Virtual Machines -> Create (+) in upper left -> virtual machines ->\n",
"In the middle of the page under \"Create Virtual Machine\" heading there are menu options from left to right: Basic, Disk, Networking, Management, Advanced, Tags, Review & Create\n",
"From the hamburger menu in the upper left of the top blue bar, click *Virtual Machines*. On the *Virtual machines* \n",
"page click Create (+) on the menu in upper left of the page. Choose *Virtual machines*.\n",
"In the middle of the page under the \"Create Virtual Machine\" heading there are menu options from left to right: \n",
"Basic, Disk, Networking, Management, Advanced, Tags, Review & Create. We will start with the Basic tab (default).\n",
"\n",
"**Part 1**\n",
"* Check subscription \n",
"\n",
"Under the Basic tab (default)\n",
"\n",
"* Check subscription\n",
" * Use the pull down menu to change subscriptions if needed\n",
"* select or create Resource Group\n",
"* VM name\n",
" * Click the create a new Resource Group and enter a name that conforms to the azure naming convention (only alphanumeric characters, periods, underscores, hyphens, and parenthesis and can not end in a period)\n",
"* Virtual machine name\n",
" * Enter a name that does not contain special characters including blah or begin *_* or end with *.* or *-* \n",
"* Image\n",
"* Spot\n",
" * Use default image (Ubuntu 20.04 - Gen2)\n",
"* Size\n",
" * Standard_D2s_v3 (default)\n",
"\n",
"![vm-create](images/01_CLASS_Azure_VM_basic_01.png)\n",
"\n",
"**Part 2**\n",
"\n",
"Before we begin part 2, we need to supply a public key when we create the VM so we can securely login. Using the key we created earlier (assume - /home/learner/.ssh/azure_compute_engine.pub), we will copy the key by doing the following."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c8fa1b24",
"metadata": {},
"outputs": [],
"source": [
"cat /home/learner/.ssh/azure_compute_engine.pub"
"![vm-create](images/01_CLASS_Azure_VM_basic_01.png)"
]
},
{
"cell_type": "markdown",
"id": "5da66e3b",
"id": "18d099ed",
"metadata": {},
"source": [
"Highlight the output string with your cursor and use the copy command (depending on OS) to put it in your edit buffer.\n",
"**Part 2**\n",
"\n",
"* Authentication type \n",
" * ssh public key\n",
"* Username\n",
" * Choose usename (default: azeruser)\n",
"* SSH public key source\n",
" * Select use existing public key\n",
" * Copy the key we create earlier and paste it in the red box below.\n",
" * Use the key we create and Cut (copied) earlier, paste (depending on OS) it into the red box below (i.e for mac use command v). \n",
"* Key pair name\n",
" * Choose name for key pair (default: auto generated based on VM name)\n",
"* Ports (none)\n",
" * Open inbound ports (default: none)\n",
"* Public inbound ports\n",
" * Select Allow selected ports (default)\n",
"* Select inbound ports\n",
" * Select SSH (22) from the pull down menu (default)\n",
"\n",
"![vm-create](images/01_CLASS_Azure_VM_basic_02.png)"
]
Expand All @@ -201,7 +229,7 @@
},
{
"cell_type": "markdown",
"id": "007e8359",
"id": "4feb08e9",
"metadata": {},
"source": [
"Now let's continue to the networking configuration by clicking Next:Networking> on the bottum or on the top heading menu choose the Networking option.\n",
Expand All @@ -216,13 +244,15 @@
" * Allow selected port\n",
"* Select inbound port(s)\n",
" * Use 22 for ssh access\n",
"* Delete public IP and NIC when VM is deleted\n",
" * Check this box\n",
"\n",
"![vm-networking](images/03_CLASS_Azure_VM_networking.png)"
]
},
{
"cell_type": "markdown",
"id": "a1561d9f",
"id": "1f163321",
"metadata": {},
"source": [
"Now let's continue to provision the VM. The next three menu options across the top are outside the scope of this tutorial so now we are ready to click Review & Create.\n",
Expand Down Expand Up @@ -255,36 +285,29 @@
"\n",
"![vm-networking](images/04_CLASS_Azure_VM_verify.png)\n",
"\n",
"On the VM page, you will be looking at the overview page. The Overview page contains essential information like OS, public IP, RG, status, subscriptioin as well as other properties and information. Find and record the Public IP address. "
]
},
{
"cell_type": "markdown",
"id": "f8afd914",
"metadata": {},
"source": [
"## Security (need to azurize)\n",
"\n",
"Everything in the cloud requires permission (authorization). Ordinary we would configure and check security first but in the case of exploring services it is often easier to do things out of order. We noted that the *VM instance* was created with the *Compute Engine default service account*, and if the \"Allow full access to all Cloud Api's\" scope is enable, then everyone on the VM has access to all the resources in your project."
"You will be looking at the overview page. The Overview page contains essential information like OS, public IP, \n",
"RG, status, subscriptioin as well as other properties and information. Find and record the Public IP address. "
]
},
{
"cell_type": "markdown",
"id": "0d1d75a2",
"id": "62c6abd4",
"metadata": {},
"source": [
"## Enumerate the VM Instances\n",
"\n"
"## Security\n",
"## Enumerate the VM Instances"
]
},
{
"cell_type": "markdown",
"id": "89f6a274",
"id": "dd25a15a",
"metadata": {},
"source": [
"## Connect to the VM Instance\n",
"\n",
"To connect to the *VM Instance* we will use the cloud shell in the web console. Open up a shell by clicking on the **Activate Cloud Shell** icon in the top blue bar if it is not already open. In Cloud Shell ssh to the VM you just created. (default username \"azureuser\")"
"To connect to the *VM Instance* we will use the cloud shell in the web console. Open up a shell by clicking on the **Activate Cloud Shell** icon (on the top blue bar click\n",
"button the looks like a box with a greater than sign followed by an underscore) in the top blue bar if it is not \n",
"already open. In Cloud Shell ssh to the VM you just created. (default username \"azureuser\")"
]
},
{
Expand Down
Loading

0 comments on commit 85c7a66

Please sign in to comment.