Skip to content
This repository was archived by the owner on Dec 12, 2025. It is now read-only.

Commit

Permalink
GCP: Cleanup and add images to monitoring
Browse files Browse the repository at this point in the history
  • Loading branch information
tmiddelkoop committed Jan 21, 2022
1 parent 40fb8dc commit 967608a
Show file tree
Hide file tree
Showing 4 changed files with 31 additions and 11 deletions.
42 changes: 31 additions & 11 deletions content/GCP/07_monitoring_costs.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,12 @@
"```{admonition} Overview\n",
":class: tip\n",
"\n",
"**Teaching:**\n",
"**Teaching:** 30 min\n",
"\n",
"**Exercises:**\n",
"**Exercises:** none\n",
"\n",
"**Questions:**\n",
"* How do I find out how much I have spent and what things cost?\n",
"* How do I find out how much I have spent?\n",
"* How do I find what is running and stored in my project?\n",
"\n",
"**Objectives:**\n",
Expand All @@ -24,9 +24,9 @@
"* Determine the permissions you have on the Billing Account\n",
"* Find and monitor your spend with billing reports\n",
"* Find where to set Billing Alerts.\n",
"* List all resources in a Project\n",
"* List resources in a service\n",
"* Optional: Understand how labels are used\n",
"* Find all resources in a Project\n",
"* Extra: List assets with the CLI\n",
"* Extra: Understand how labels are used\n",
" \n",
"```"
]
Expand All @@ -45,11 +45,12 @@
" * Double check that the Project ID and name in the \"Project Info\" card is correct.\n",
" * Note the billing card, but do not click on it.\n",
" * **Navigation Menu -> Billing** \n",
" * If you get a \"You have multiple billing accounts\" select \"Go to linked billing account\"\n",
" * If you get a \"You have multiple billing accounts\" select \"**Go to linked billing account**\"\n",
" * If you get a \"This project has no billing account\" you cannot incur any charges on this account. Since you do not have a billing account you have no charges to monitor or set alerts on. You will need to associate or create a billing account to do anything.\n",
" * Select **Account Management** on the *Left Sidebar* of the Billing page.\n",
" * Note down the \"Billing Account ID\" near the top left of the billing page It should be in the form *01ABC2-A0B34C-DE5F6*. This information should not be made public.\n",
" * Open the right sidebar Info Panel if closed. To do so click **Show Info Panel** button on top right under the blue bar (don't click the \"Hide Info Panel\" button!)\n",
" ![billing-management](img/billing-management.png)\n",
" * If you get \"you don't have permission to view the permissions of the selected resource\" in the side bar you have no privileges and cannot set Billing Alerts.\n",
" * If you have permission to view permissions (you did not get the previous message):\n",
" * Ensure **Show inherited permissions** is \"on\" (blue and to the right)\n",
Expand Down Expand Up @@ -84,7 +85,9 @@
" * Bar chart (bar chart graph on the right, \"Daily\" is on the left)\n",
" * **Save** view\n",
" * If you cannot save the view you can also click \"Share\" and use the URL to create a bookmark for easy access.\n",
" \n",
"\n",
"![billing-report](img/billing-report.png)\n",
"\n",
"This is the most reliable and direct way to monitor costs and these charts should be monitored daily for active projects and weekly for ongoing projects. For projects that are inactive the billing should be disabled or the project should be deleted.\n",
"\n",
"There reports do not show consumption in real time. In order to get an idea of what is generating costs (and you can estimate these costs based on past experience), later we will show how to *enumerate* running resources. "
Expand All @@ -106,12 +109,29 @@
"There are also 3rd party cost monitoring services that your institution may use that can set billing alerts, contact your research computing and data professional for more information."
]
},
{
"cell_type": "markdown",
"id": "1c382ee4-6fbc-4616-8154-4396fe724fa6",
"metadata": {},
"source": [
"## Asset Inventory\n",
"\n",
"In order to get an overview of what is running and stored in your account and where we use the *Asset Inventory* page in the *IAM* Service. To see what is in your *Project* do the following:\n",
"\n",
" * Go to **Navigation Menu** -> **IAM & Admin** -> **Asset Inventory** page.\n",
" * Filter important resources by navigating to **overview** -> **Asset type: View More** and selecting important resources (compute.Instance, compute.Disk, storage.Bucket) and click **Apply** on on the bottom.\n",
" ![iam-asset-navigation](img/iam-asset-navigation.png)\n",
"\n",
"Use the *Overview* and *Resource* to learn what is running and where (in the world) it is running.\n",
"\n"
]
},
{
"cell_type": "markdown",
"id": "cdfc0b3b-8bc3-44a3-a712-1005531c9d30",
"metadata": {},
"source": [
"## Enumerate All Resources\n",
"## Enumerate All Resources (CLI - Extra)\n",
"\n",
"Before we clean up we should always make sure that we can see the resources we created and that after destroying them they are actually removed. This is the most powerful tool for managing cost as it gives a real-time indication on what is running.\n",
"\n",
Expand Down Expand Up @@ -306,7 +326,7 @@
"id": "a670902f-53d3-4fce-acb0-0ace9608cd5d",
"metadata": {},
"source": [
"## Enumerate Service Resources\n",
"## Enumerate Service Resources (CLI - Extra)\n",
"\n",
"To get up-to-date information about a service the `gcloud` command can be used to list allocated resources. It is good to write a simple script to show all the resources in all the services you use. This, combined with the `gcloud asset` command is a powerful tool for "
]
Expand Down Expand Up @@ -372,7 +392,7 @@
"id": "45681123-b27a-48d8-ac4a-30dc3b46d2c1",
"metadata": {},
"source": [
"## Optional: Labeling Resources\n",
"## (CLI - Extra)\n",
"\n",
"In order to track resources it is important to Label them. Labeling places key-value pairs (key=value) in the resources metadata (information about the resource) and most of the Google Cloud tools can use this information to search, filter, and group information. In the literature, the example most used is the \"env\" Label and indicates if the resource is in development (dev), testing (test), or production (prod); the lables would be env=test, env=dev and env=prod, respectively. \n",
"\n",
Expand Down
Binary file added content/GCP/img/billing-management.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added content/GCP/img/billing-report.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added content/GCP/img/iam-asset-navigation.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 967608a

Please sign in to comment.