Skip to content
This repository was archived by the owner on Dec 12, 2025. It is now read-only.

Commit

Permalink
Merge pull request #35 from CLASS/gcp-dev
Browse files Browse the repository at this point in the history 
GCP: Minor updates for workshop.
  • Loading branch information
@amanda-tan
amanda-tan authored Feb 16, 2022
2 parents 1679bfe + c1318ad commit 9dc0a64
Show file tree
Hide file tree
Showing 11 changed files with 270 additions and 270 deletions.
13 changes: 13 additions & 0 deletions Build.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,19 @@ To publish to the `gh-pages` of the `origin` (or other) remote run.

This copies the contents of `content/_build/html/` to `docs/` so GitHub Pages will render the document. GitHub pages should be set to the `gh-pages` and `docs/` folder. The script pushes a new branch overwriting the old branch and will not track changes.

## VM Development Setup

Have your ~/.ssh/config source the file the *-create.sh scripts create. This is a one-time per machine setup.

```
echo "include ~/.ssh/essentials.config" >> ~/.ssh/config
```

Verify your name and email are correct (locally) as they are copied over to the VM
```
git config user.name
git config user.email
```

## GCP

Expand Down
20 changes: 10 additions & 10 deletions content/GCP/01_intro_to_cloud_console.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
"```{admonition} Overview\n",
":class: tip\n",
"\n",
"**Teaching:** 10 min.\n",
"**Teaching:** 15 min.\n",
"\n",
"**Exercises:** 6 min.\n",
"\n",
Expand All @@ -32,7 +32,7 @@
"id": "10be57f6",
"metadata": {},
"source": [
"## The Who, What, and Where of the Cloud\n",
"## The Who, Where, and What of the Cloud\n",
"\n",
"Before we get started we must first define, and find, a few key pieces of information about your environment that will be used throughout this lesson. This information is also important to ensure that this information is what you expect, otherwise you may use the wrong account or run or store things in the wrong place. It is important to be clear about these terms as they are often different in other cloud providers. We provide a [glossary of GCP terms](glossary.ipynb) for your reference."
]
Expand Down Expand Up @@ -82,18 +82,18 @@
"source": [
"## Projects\n",
"\n",
"Almost everything you will do within Google Cloud Platform must be associated with a **Project**. This is the \"Where\" of the cloud. You must have at least one Project and you can manage multiple Projects within a single (Google) Account. Always make sure you are working in the correct project!\n",
"Almost everything you will do within Google Cloud Platform must be associated with a **Project**. This is the \"**Where**\" of the cloud. You must have at least one Project and you can manage multiple Projects within a single (Google) Account. Always make sure you are working in the correct project!\n",
"\n",
"To find more information and change project settings\n",
" * Click on the \"Settings and Utilities button (kabob on the top right - see below) and then click on \"Project Settings\"\n",
" * Click on the \"Settings and Utilities button (**kabob** on the top right - see below) and then click on \"Project Settings\"\n",
" ![kabob-project](img/kabob-project.png)\n",
" * The Project Name is the human friendly description and can be changed.\n",
" * The Project ID is set on project creation and cannot be changed. The Project ID is almost always used when specifying a project.\n",
" * The Project Number is used internally by Google Services and is usually used indirectly. It is a good idea to be able to recognize this number.\n",
" \n",
"For a Project to do anything useful it must also have an enabled Billing Account associated with it. An enabled Billing Account is a prerequisite for this lesson.\n",
"\n",
"A Project, just like the word, should be associated with a real world project (for example a research effort or grant, lab, or your Ph.D. Project. etc). For Drew, this is the image processing project. In this way it is easier to track and allocate costs and to manage permissions and access for resources within the project.\n",
"A Project, just like the word, should be associated with a real world project (for example a research effort or grant, lab, or your Ph.D. Project. etc). For Drew, this is the image processing project. In this way it is easier to track and allocate costs and to manage permissions and access for resources within the project. Work in a project should have similar **people**, **lifecycle**, and **funding**.\n",
"\n",
"The active project is also shown next to the project icon (three hexagons) and clicking it brings up the project selection dialog. The current project has a check mark and the active project can be changed by clicking on the project name or double clicking the row.\n",
" ![select-project](img/select-project.png) "
Expand Down Expand Up @@ -128,15 +128,15 @@
"\n",
"The web console is used to control and observe the cloud from the browser. It should only be used for simple and one-time tasks, exploring new services, accessing documentation, or for monitoring and debugging resources in the cloud. Programmatic control through the console, programming languages (for example Python), and other automation tools should be used for day to day activities to make the most out of the cloud and to help with the reproducibility of research and teaching.\n",
"\n",
"The Navigation Menu (often called the hamburger) is used to navigate to the various products, which are also sometimes called services.\n",
"The Navigation Menu (often called the **hamburger**) is used to navigate to the various products, which are also sometimes called services.\n",
"\n",
"![hamburger-navigation](img/hamburger-navigation.png)\n",
"\n",
" * Click on a product to navigate the product page\n",
" * You can pin frequently used items on the top of this page by clicking on the pin icon. \n",
" * Click the hamburger again to hide the Left Sidebar.\n",
"\n",
"All the different products and services are the \"What\" of the cloud. We will start with compute, called *Google Compute Engine* in the next Episode. "
"All the different products and services are the \"**What**\" of the cloud. We will start with compute, called *Google Compute Engine* in the next Episode. "
]
},
{
Expand All @@ -149,9 +149,9 @@
"```{admonition} Exercise\n",
"\n",
"Take a few moments to navigate a few key services.\n",
" * Navigate to the \"Compute Engine\" service under the \"Compute\" product group.\n",
" * You will probably need to \"Enable\" this service first by clicking on the \"Enable\" button on the \"Compute Engine API\" page. This will only need to be done once per project.\n",
" * Navigate to \"Cloud Storage\" under the \"Storage\" product group and enable the service if necessary.\n",
" * Navigate to \"Cloud Storage\" under the \"Storage\" product group.\n",
" * You will probably need to \"Enable\" this service first by clicking on the \"Enable\" button on the \"Cloud Storage API\" page. This will only need to be done once per project.\n",
" * Navigate to the \"Compute Engine\" service under the \"Compute\" product group and enable the service if necessary.\n",
"```"
]
},
Expand Down
26 changes: 13 additions & 13 deletions content/GCP/02_intro_to_compute.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
"```{admonition} Overview\n",
":class: tip\n",
"\n",
"**Teaching:** 45 min.\n",
"**Teaching:** 30 min.\n",
"\n",
"**Exercises:** 6 min\n",
"\n",
Expand Down Expand Up @@ -67,7 +67,7 @@
"To create a VM Instance we do the following:\n",
" * Click **Navigation Menu** -> **Compute Engine** (under Compute) -> **VM Instances** -> **+Create Instance** (just under the blue bar) to open the *Create an instance* page.\n",
" * In the **New VM instance** tab on the left (selected by default) configure the *VM instance* as follows:\n",
" * For **Name**, enter a unique name for the instances (example: \"essentials-instance-1\")\n",
" * For **Name**, enter a unique name for the instances (example: \"**essentials-test-1**\")\n",
" * For **Region** leave the default or select your \"home\" region. The region is the physical location where your data will reside. Your \"home\" region should be close to your work and should be the region you use most of the time.\n",
" * For **Zone** leave the default (note how the name is constructed and that it is a separate data center) some zones have different capabilities.\n",
" * In the **Machine configuration** section:\n",
Expand Down Expand Up @@ -97,7 +97,7 @@
"id": "41c63432-a614-4a1e-9967-f49b68f9069e",
"metadata": {},
"source": [
"## Security\n",
"## More on Security (Optional)\n",
"\n",
"Everything in the cloud requires permission (authorization). Ordinary we would configure and check security first but in the case of exploring services it is often easier to do things out of order. We noted that the *VM instance* was created with the *Compute Engine default service account*, and if the \"Allow full access to all Cloud Api's\" scope is enable, then everyone on the VM has access to all the resources in your project.\n",
"\n",
Expand Down Expand Up @@ -125,9 +125,9 @@
"tags": []
},
"source": [
"## Follow the VM Allocation\n",
"## Track VM Instance Creation\n",
"\n",
"Just as with security, we will audit (follow) the *VM instance* creation by examining at the project *activity*.\n",
"We can track what is going on in our project by following the *VM instance* creation by examining at the project *activity* page on the project dashboard.\n",
"\n",
"To view the project activity we do the following:\n",
"\n",
Expand All @@ -147,7 +147,7 @@
"tags": []
},
"source": [
"## Enumerate the VM Instances\n",
"## Find the VM Instance\n",
"\n",
"Now lets find and connect to the *VM Instance*.\n",
" * Navigate to the Google Compute Engine page by clicking **Navigation Menu** -> **Compute Engine** (under Compute) -> **Instances**.\n",
Expand All @@ -174,7 +174,7 @@
"\n",
"To connect to the *VM instance* we enter the following command in the cloud shell:\n",
"```\n",
"gcloud compute ssh essentials-instance-1\n",
"gcloud compute ssh essentials-test-1\n",
"```\n",
"\n",
"If you have not used the cloud shell to connect to a *VM Instance* before you will probably be asked to create a new *ssh key*. The Compute Engine will use this key to allow you to access the *VM instance* in a secure manner. If this is the case you will see a message similar to the following:\n",
Expand Down Expand Up @@ -223,15 +223,15 @@
"\n",
"At this point the command will attempt to connect to the *VM Instance* and will ask the following question:\n",
"```\n",
"Did you mean zone [us-central1-a] for instance: [essentials-instance-1] (Y/n)? n\n",
"Did you mean zone [us-central1-a] for instance: [essentials-test-1] (Y/n)? n\n",
"```\n",
"Answer \"n\".\n",
"\n",
"The command will now configure the instance to allow your ssh key and connect to it.\n",
"\n",
"```\n",
"No zone specified. Using zone [us-central1-a] for instance: [essentials-instance-1].\n",
"Updating project ssh metadata...working..Updated [https://www.googleapis.com/compute/v1/projects/class-essentials-instance-1].\n",
"No zone specified. Using zone [us-central1-a] for instance: [essentials-test-1].\n",
"Updating project ssh metadata...working..Updated [https://www.googleapis.com/compute/v1/projects/class-essentials-test-1].\n",
"Updating project ssh metadata...done.\n",
"Waiting for SSH key to propagate.\n",
"Warning: Permanently added 'compute.74517428106645607' (ECDSA) to the list of known hosts.\n",
Expand All @@ -240,15 +240,15 @@
"\n",
"Once connected you will see the machine login banner and prompt similar to the following:\n",
"```\n",
"Linux instance-1 4.19.0-17-cloud-amd64 #1 SMP Debian 4.19.194-3 (2021-07-18) x86_64\n",
"Linux essentials-test-1 4.19.0-17-cloud-amd64 #1 SMP Debian 4.19.194-3 (2021-07-18) x86_64\n",
"\n",
"The programs included with the Debian GNU/Linux system are free software;\n",
"the exact distribution terms for each program are described in the\n",
"individual files in /usr/share/doc/*/copyright.\n",
"\n",
"Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent\n",
"permitted by applicable law.\n",
"learner@essentials-instance-1:~$\n",
"learner@essentials-test-1:~$\n",
"```\n",
"\n",
"Congratulations you have now created a *VM Instance* and connected to it.\n",
Expand Down Expand Up @@ -288,7 +288,7 @@
"```{admonition} Exercise\n",
"\n",
"* Try to delete the ***VM instance*** on your own\n",
"* Try to follow the process we just learned (hint: *follow* and *enumerate*)\n",
"* Try to follow the process we just learned (hint: *track* and *list*)\n",
"```"
]
},
Expand Down
70 changes: 10 additions & 60 deletions content/GCP/03_intro_to_cloud_storage.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
"```{admonition} Overview\n",
":class: tip\n",
"\n",
"**Teaching:** 40 min\n",
"**Teaching:** 20 min\n",
"\n",
"**Exercises:** 5 min\n",
"\n",
Expand All @@ -19,7 +19,6 @@
"\n",
"**Objectives:**\n",
"* Navigate the Google Cloud Storage service and terminology\n",
"* Understand the roles and permissions needed to use Google Cloud Storage in projects\n",
"* Allocate storage in Google Cloud Storage\n",
"* Find the cost estimator for Google Cloud Storage\n",
"* Recognize that resources have a \"location\"\n",
Expand Down Expand Up @@ -50,41 +49,6 @@
"We now take Drew through the process of creating a Google Cloud Storage bucket."
]
},
{
"cell_type": "markdown",
"id": "07fb9096-2b40-4995-a742-be7bd9b2797c",
"metadata": {
"tags": []
},
"source": [
"## Security\n",
"\n",
"Everything in the cloud requires permission (authorization). Let's first verify that we have the permissions to create a bucket. A Bucket (a resource) is created within a project and inheres permissions from it.\n",
"\n",
"We are interested in what permissions that *your* account has for *your* project. To do this navigate to the IAM page (**Navigation Menu -> IAM & Admin -> IAM -> Permissions -> View By: Principals**). This shows the permissions for the project.\n",
"\n",
"*Note: There is a powerful filter box to limit the permissions shown.*\n",
"\n",
"You should see a row with your account shown in the Principal column. Here you should see the \"Editor\" Role in the Role column. A *role* is a collection of permissions managed by Google or someone else. The **Editor**, **Owner**, or the **Storage Admin** role for a project will *allow* *you* to create, access, and delete Buckets *in* the project.\n",
"\n",
"There are three important pieces of information that work together to form the **IAM policy**. The permission (role), the identity (principal), and the resource (project). This is another who (identity), what (permission), and where (resource)."
]
},
{
"cell_type": "markdown",
"id": "9acf29cf-660b-4922-bcb8-89fd9080fdea",
"metadata": {
"tags": []
},
"source": [
"```{admonition} Exercise\n",
"\n",
"Answer the following questions:\n",
" * What is the \"Who, What, Where\" of the IAM policy that allows you to use your project?\n",
" * What else has permissions to do things in your project and state the \"Who, What, Where\"?\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "c5430b40-1a5f-40df-9e13-529ef3ece4ce",
Expand All @@ -94,12 +58,12 @@
"source": [
"## Allocate Google Cloud Storage\n",
"\n",
"Now that we have verified the permissions we can now create a bucket. Buckets are where objects are stored and have a globally unique name.\n",
"Buckets are where objects are stored and have a globally unique name.\n",
"\n",
"To create a bucket we do the following:\n",
" * Click **Navigation Menu** -> **Cloud Storage** (under Storage) -> **Browser** -> **+Create Bucket** (just under the blue bar) to open the *Create a bucket* page.\n",
" * In *Name your bucket*:\n",
" * For **Name**, enter a globally unique name for the bucket (example \"essentials-test-myname-2021-01-01\")\n",
" * For **Name**, enter a globally unique name for the bucket (example \"**essentials-test-myname-2022-01-01**\")\n",
" * Click **Continue**\n",
" * In *Choose where to store your data*:\n",
" * For *Location Type* select **Region** (cheapest and fastest)\n",
Expand Down Expand Up @@ -129,9 +93,9 @@
"tags": []
},
"source": [
"## Follow the Storage Allocation\n",
"## Track the Storage Allocation\n",
"\n",
"Just as with compute, we will audit (follow) the bucket creation by examining at the project *activity*.\n",
"Just as with compute, we will track (follow) the bucket creation by examining at the project *activity*.\n",
"\n",
"To view the project activity we do the following:\n",
"\n",
Expand All @@ -150,15 +114,13 @@
"tags": []
},
"source": [
"## Enumerate the Buckets\n",
"## List the Buckets\n",
"\n",
"Now lets find and examine the bucket. To view a bucket we do the following:\n",
"\n",
" * Navigate to the Google Storage page by clicking **Navigation Menu** -> **Cloud Storage** (under Storage) -> **Browser**. \n",
" * **Find** the bucket you just created. You can use the filter to find a bucket if there are a lot of them.\n",
" * Click on the bucket name to open the **bucket details** (it will display as a hyperlink when you hover over the bucket name).\n",
"\n",
"Navigate to the **dashboard** and you will now see \"Storage\" in the *resources* card under. You can click on this to quickly navigate to the storage page."
" * Click on the bucket name to open the **bucket details** (it will display as a hyperlink when you hover over the bucket name).\n"
]
},
{
Expand All @@ -170,12 +132,12 @@
"source": [
"## Review what is Important\n",
"\n",
"It is always important to review what is important to you. It may be cost, or keeping the data secure. Later on we will show how to monitor overall costs.\n",
"It is always important to review what is important to you. It may be cost, or keeping the data secure. Later on we will show how to monitor overall costs. We will also learn how to use the \"info panel\" to show more information about a bucket.\n",
"\n",
"For Drew, we will review that the bucket **public access** is *not public* by doing the following:\n",
" * Go to **Navigation Menu -> Cloud Storage -> Browser**\n",
" * Select the bucket of interest by **checking the box** next to the Bucket name.\n",
" * In the Right Side Bar (open if necessary) in the **Permissions** tab in the **Public Access** card you should see **Not Public**. This means that public access prevention is turned on.\n",
" * In the **Info Panel** (click show \"Info Panel\" if necessary) in the **Permissions** tab in the **Public Access** card you should see **Not Public**. This means that public access prevention is turned on.\n",
" * You can also see the **permissions** for the bucket in the bottom of the bar."
]
},
Expand Down Expand Up @@ -225,23 +187,11 @@
"\n",
"![storage-delete-bucket](img/storage-delete-bucket.png)\n",
"\n",
"Did you \"Follow\" the bucket by looking at the **activity** page as discussed above?\n",
"Did you \"Track\" the bucket by looking at the **activity** page as discussed above?\n",
"\n",
"Since we care about paying for resources we are not using we review our project by visiting the *compute storage* service and reviewing that we no longer have any *Buckets* allocated. "
]
},
{
"cell_type": "markdown",
"id": "3a28e28d-1d70-44fa-a952-4f3506ea85ec",
"metadata": {},
"source": [
"## Discussion (Optional)\n",
"\n",
"* What does the words \"Secure\", \"Allocate\", \"Follow\", and \"Enumerate\" spell?\n",
"* What happens when you add the \"R\" in \"Review?\"\n",
"* Is this useful?"
]
},
{
"cell_type": "markdown",
"id": "97d7ebc5-4a81-4f1a-aaf3-517adf70640a",
Expand Down
Loading

0 comments on commit 9dc0a64

Please sign in to comment.