Skip to content
This repository was archived by the owner on Dec 12, 2025. It is now read-only.

Commit

Permalink
GCP: Cleanup and move to Web Console for Cleanup
Browse files Browse the repository at this point in the history
  • Loading branch information
@tmiddelkoop
tmiddelkoop committed Jan 21, 2022
1 parent 967608a commit e0fa9ac
Show file tree
Hide file tree
Showing 2 changed files with 58 additions and 294 deletions.
2 changes: 1 addition & 1 deletion content/GCP/07_monitoring_costs.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -392,7 +392,7 @@
"id": "45681123-b27a-48d8-ac4a-30dc3b46d2c1",
"metadata": {},
"source": [
"## (CLI - Extra)\n",
"## Label Resources (CLI - Extra)\n",
"\n",
"In order to track resources it is important to Label them. Labeling places key-value pairs (key=value) in the resources metadata (information about the resource) and most of the Google Cloud tools can use this information to search, filter, and group information. In the literature, the example most used is the \"env\" Label and indicates if the resource is in development (dev), testing (test), or production (prod); the lables would be env=test, env=dev and env=prod, respectively. \n",
"\n",
Expand Down
350 changes: 57 additions & 293 deletions content/GCP/08_cleaning_up_resources.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,9 @@
"```{admonition} Overview\n",
":class: tip\n",
"\n",
"**Teaching:**\n",
"**Teaching:** 5 min\n",
"\n",
"**Exercises:**\n",
"**Exercises:** 10 min\n",
"\n",
"**Questions:**\n",
"* How do I clean up after I'm done for the day?\n",
Expand All @@ -27,317 +27,81 @@
},
{
"cell_type": "markdown",
"id": "7621ba3b-c110-48aa-8300-14bbe5ee845f",
"id": "65fb0615-f942-4af0-9087-8f0696df95f9",
"metadata": {},
"source": [
"## Enumerating Existing Resources\n",
"## Cleanup VMs\n",
"\n",
"We will first use the method from the previous lesson (see if you can do this on your own first) to check which resources are currently running so we can verify that they get removed."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "c8ee7307-4ed5-437f-b4e2-400b64190399",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"//storage.googleapis.com/essentials-learner-2021-12-16\n",
"//compute.googleapis.com/projects/essentials-project-335018/zones/us-west2-c/instances/essentials\n",
"//compute.googleapis.com/projects/essentials-project-335018/zones/us-west2-c/disks/essentials\n"
]
}
],
"source": [
"gcloud asset search-all-resources --asset-types='compute.googleapis.com/Disk,compute.googleapis.com/Instance,storage.googleapis.com/Bucket' --format='value(name)'"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "67f75321-d684-4930-bd16-8112d4ced868",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"//cloudresourcemanager.googleapis.com/projects/essentials-project-335018\n",
"//compute.googleapis.com/projects/essentials-project-335018\n",
"//compute.googleapis.com/projects/essentials-project-335018/zones/us-west2-c/disks/essentials\n",
"//compute.googleapis.com/projects/essentials-project-335018/zones/us-west2-c/instances/essentials\n",
"//storage.googleapis.com/essentials-learner-2021-12-16\n"
]
}
],
"source": [
"gcloud asset search-all-resources --format='value(name)' |grep -vE '/(services|networks|routes|subnetworks|firewalls|serviceAccounts)/' | sort"
]
},
{
"cell_type": "markdown",
"id": "d43bd86d-acf7-416a-9b35-506f5625f2c3",
"metadata": {},
"source": [
"## Cleanup Buckets\n",
"```{admonition} Danger\n",
":class: danger\n",
"\n",
"First set the environment variables and verify them. If they are previously set you can just verify them. You can find the zone from the previous commands."
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "03226ee4-00d3-4944-b60c-bd1ad62f1077",
"metadata": {},
"outputs": [],
"source": [
"ACCOUNT=$(gcloud config list --format='value(core.account)')\n",
"PROJECT=$(gcloud config list --format='value(core.project)')\n",
"BUCKET=\"essentials-${USER}-*\"\n",
"REGION=\"us-west2\"\n",
"ZONE=\"us-west2-c\""
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "75bf0dd4-4a62-497a-b491-2bc80285aa44",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Account: learner@class.internet2.edu, Project: essentials-project-335018, Region: us-west2 Zone: us-west2-c\n",
"Bucket: essentials-learner-*\n"
]
}
],
"source": [
"echo \"Account: $ACCOUNT, Project: $PROJECT, Region: $REGION Zone: $ZONE\"\n",
"echo \"Bucket: $BUCKET\""
"This will destroy the VM instance and and ALL the data in it! You cannot undelete an instance!\n",
"```\n",
"\n",
"```{admonition} Exercise\n",
"\n",
" * Try to delete the **VM Instance** on your own\n",
"```\n",
"\n",
"Navigate to the **Compute Engine** service and in the **VM Instances** page and remove the *VM Instance*.\n"
]
},
{
"cell_type": "markdown",
"id": "b4b93fcb-d72b-4b7c-ab40-5e5ad83fce86",
"id": "92321a60-13c6-4c87-aff5-92fb708e8bff",
"metadata": {},
"source": [
"## Cleanup Buckets\n",
"\n",
"We will now list the buckets that we will be deleting. Note the wild-card (GLOB) from above will match all dates (and all strings)."
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "93add660-5bdb-4393-a05a-031ec301d7f8",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"gs://essentials-learner-2021-12-16/\n"
]
}
],
"source": [
"gsutil ls -b \"gs://$BUCKET\""
]
},
{
"cell_type": "markdown",
"id": "8daa2ec9-d64d-4586-8ab9-33f03c1b1138",
"metadata": {},
"source": [
"**WARNING: All of the above buckets will be deleted if the following commands are run**"
]
},
{
"cell_type": "markdown",
"id": "be020ee5-d3af-4ada-a674-51775648f373",
"metadata": {},
"source": [
"**Delete** all the objects first"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "24616c20-ae7e-4d0f-9f9c-6c8752891fa7",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Removing gs://essentials-learner-2021-12-16/1#1639684995415818...\n",
"/ [1/1 objects] 100% Done \n",
"Operation completed over 1 objects. \n"
]
}
],
"source": [
"gsutil -m rm -r \"gs://$BUCKET/*\""
]
},
{
"cell_type": "markdown",
"id": "194c0181-6a47-43c5-af40-29230d858164",
"metadata": {},
"source": [
"Delete the empty buckets"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "a1310b76-6212-48fa-b023-0a15e06d4332",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Removing gs://essentials-learner-2021-12-16/...\n"
]
}
],
"source": [
"gsutil rb \"gs://$BUCKET/\""
]
},
{
"cell_type": "markdown",
"id": "4bf0ed5b-b4f3-4061-ae65-50bf9d2500f7",
"metadata": {},
"source": [
"## Enumerating Resources\n",
"```{admonition} Danger\n",
":class: danger\n",
"\n",
"Now we will check that all the resources were removed."
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "b088681d-633d-4991-ba7d-2ecfb41c36dd",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"//compute.googleapis.com/projects/essentials-project-335018/zones/us-west2-c/instances/essentials\n",
"//compute.googleapis.com/projects/essentials-project-335018/zones/us-west2-c/disks/essentials\n"
]
}
],
"source": [
"gcloud asset search-all-resources --asset-types='compute.googleapis.com/Disk,compute.googleapis.com/Instance,storage.googleapis.com/Bucket' --format='value(name)'"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "74d0e8e7-f097-41c5-88fe-929edcb0c1c1",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"//cloudresourcemanager.googleapis.com/projects/essentials-project-335018\n",
"//compute.googleapis.com/projects/essentials-project-335018\n",
"//compute.googleapis.com/projects/essentials-project-335018/zones/us-west2-c/disks/essentials\n",
"//compute.googleapis.com/projects/essentials-project-335018/zones/us-west2-c/instances/essentials\n"
]
}
],
"source": [
"gcloud asset search-all-resources --format='value(name)' |grep -vE '/(services|networks|routes|subnetworks|firewalls|serviceAccounts)/' | sort"
]
},
{
"cell_type": "markdown",
"id": "f275a53a-a3d1-4a0f-a44c-21e2ba7186ba",
"metadata": {},
"source": [
"Looks like we forgot to delete the VM."
]
},
{
"cell_type": "markdown",
"id": "74fd626a-6102-4815-a8c2-27aade6ebc3b",
"metadata": {},
"source": [
"## Cleanup VMs\n",
"We can also create and delete VMs using the `gcloud` command. Note, the `--quiet` disables conformation. **WARNING: This command will immediately delete the VM called essentials**"
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "2ab8b339-b923-4623-942c-ddd424e3bcf7",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Deleted [https://www.googleapis.com/compute/v1/projects/essentials-project-335018/zones/us-west2-c/instances/essentials].\n"
]
}
],
"source": [
"gcloud compute instances delete essentials --zone=$ZONE --quiet"
]
},
{
"cell_type": "markdown",
"id": "4a215b73-ac5e-4fa6-88b7-8af26ee4090f",
"metadata": {},
"source": [
"Check again"
]
},
{
"cell_type": "code",
"execution_count": 14,
"id": "e53cb55c-3195-4fc3-9aab-cabad44d865e",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"//cloudresourcemanager.googleapis.com/projects/essentials-project-335018\n",
"//compute.googleapis.com/projects/essentials-project-335018\n"
]
}
],
"source": [
"gcloud asset search-all-resources --format='value(name)' |grep -vE '/(services|networks|routes|subnetworks|firewalls|serviceAccounts)/' | sort"
"This will destroy the storage *Bucket* and and ALL the data in it! You cannot undelete a Bucket!\n",
"```\n",
"\n",
"```{admonition} Exercise\n",
"\n",
" * Try to delete the storage **bucket* on your own\n",
"```\n",
"\n",
"Navigate to the **Cloud Storage** service and in the **browser** page and remove the *essentials* Bucket."
]
},
{
"cell_type": "markdown",
"id": "631a08af-88bb-463c-b388-8993021ee216",
"id": "6753b415-1fa9-4716-8c41-8d98228af51c",
"metadata": {},
"source": [
"Success! The only resource left is the project (both are project resources)"
"## Verify Cleanup\n",
"\n",
"```{admonition} Exercise\n",
"\n",
" * There are at least three ways to verify resource removal, what are they?\n",
" * Try to verify that the resources were removed.\n",
"```\n",
"\n",
"```{admonition} Tip\n",
":class: tip\n",
"You can also quickly navigate to resource page for most services from the **Resources** tab on the project **Dashboard**\n",
"```\n",
"\n",
"To verify that the resources have been deleted do the following:\n",
" * Navigate to the **VM Instances** page and **Cloud Storage** **Browser** to verify the resources have been deleted.\n",
" * Navigate to the **IAM & Admin** service and in the **Asset Inventory** page to verify that the resources have been deleted.\n",
" * Navigate to the **Activity** page and verify the deletion events. **Note:** the information collected in the asset inventory is often delayed.\n",
"\n",
"```{admonition} Tip\n",
":class: tip\n",
"\n",
"If this is your own account, remember to visit the Billing Report page that you bookmarked daily.\n",
"```\n",
"\n",
"```{admonition} Tip\n",
":class: tip\n",
"\n",
"You can also create, list, and destroy resources easily from the cli with the `gcloud` and `gsutil` commands.\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "cac28557-fec2-43f4-aaa5-4f82b2257c55",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
Expand Down

0 comments on commit e0fa9ac

Please sign in to comment.