diff --git a/content/GCP/05_cli_storage.ipynb b/content/GCP/05_cli_storage.ipynb index 948f9ca..1a70c23 100644 --- a/content/GCP/05_cli_storage.ipynb +++ b/content/GCP/05_cli_storage.ipynb @@ -110,7 +110,7 @@ }, { "cell_type": "code", - "execution_count": 4, + "execution_count": 1, "id": "255eb914-a06a-4bfd-adbb-7ca2f94678e7", "metadata": {}, "outputs": [ @@ -118,7 +118,7 @@ "name": "stdout", "output_type": "stream", "text": [ - "essentials-student31-2021-10-26\n" + "essentials-learner-2021-12-16\n" ] } ], @@ -129,7 +129,7 @@ }, { "cell_type": "code", - "execution_count": 5, + "execution_count": 2, "id": "b0661b4c-b56b-4d6d-ba3c-e1c5f38699c6", "metadata": {}, "outputs": [ @@ -137,7 +137,7 @@ "name": "stdout", "output_type": "stream", "text": [ - "Creating gs://essentials-student31-2021-10-26/...\n" + "Creating gs://essentials-learner-2021-12-16/...\n" ] } ], diff --git a/content/GCP/07_monitoring_costs.ipynb b/content/GCP/07_monitoring_costs.ipynb index e4d1f57..3e7f5a0 100644 --- a/content/GCP/07_monitoring_costs.ipynb +++ b/content/GCP/07_monitoring_costs.ipynb @@ -15,14 +15,75 @@ "**Exercises:**\n", "\n", "**Questions:**\n", - " * \n", + " * How do I find out how much I am spending?\n", "\n", "**Objectives:**\n", - " * \n", - "\n", + " * Learn about Billing Accounts\n", + " * Find information about the Billing Account associated with your project\n", + " * Determine the permissions you have on the Billing Account\n", + " * Find and monitor your spend with billing reports\n", "```" ] }, + { + "cell_type": "markdown", + "id": "f6f39fb5-4425-4e46-b2f8-c50f56697b5c", + "metadata": {}, + "source": [ + "## Billing\n", + "\n", + "Projects and Billing Accounts are two different things in Google Cloud. In order for a Project operate it must have a Billing Account to charge any expenses to. Even if a project is utilizing credits it must have a Billing Account associated with it. Projects within an organization will often have the Billing Account controlled directly by the organization.\n", + "\n", + "To find out more about the Billing Account associated with your project do the following:\n", + " * **Navigation Menu -> Home** (this will ensure your project is selected)\n", + " * Double check that the Project ID and name in the \"Project Info\" card is correct.\n", + " * Note the billing card, but do not click on it.\n", + " * **Navigation Menu -> Billing** \n", + " * If you get a \"You have multiple billing accounts\" select \"Go to linked billing account\"\n", + " * If you get a \"This project has no billing account\" you cannot incur any charges on this account. Since you do not have a billing account you have no charges to monitor or set alerts on. You will need to associate or create a billing account to do anything.\n", + " * Select **Account Management** on the *Left Sidebar* of the Billing page.\n", + " * Note down the \"Billing Account ID\" near the top left of the billing page It should be in the form *01ABC2-A0B34C-DE5F6*. This information should not be made public.\n", + " * Open the right sidebar Info Panel if closed. To do so click **Show Info Panel** button on top right under the blue bar (don't click the \"Hide Info Panel\" button!)\n", + " * If you get \"you don't have permission to view the permissions of the selected resource\" in the side bar you have no privileges and cannot set Billing Alerts.\n", + " * If you have permission to view permissions (you did not get the previous message):\n", + " * Ensure **Show inherited permissions** is \"on\" (blue and to the right)\n", + " * **Expand** the all the \"arrows\" on panel (for example \"Billing Account Administrator\") for each Role.\n", + " * When you see your Account/User (Member) in a section you have that permission.\n", + " * *Check* if you have **Billing Account Administrator** or **Billing Account Cost Manager** Role you have the permission (Role) to set alerts.\n", + "\n", + "The Billing Account Administrator Role on the Billing Account gives you full control over the Billing Account. The Billing Account Cost Manager Role allows you to view information and set Billing Alerts." + ] + }, + { + "cell_type": "markdown", + "id": "b6ac1492-727f-4bda-8ecb-78b8ca13f222", + "metadata": { + "tags": [] + }, + "source": [ + "## Billing Reports\n", + "\n", + "To monitor spending we look at the billing reports and configure to to see the information we need. Specifically we will turn off discounts, promotions, and credits to see activity, even if we are not billed for it. We will also switch to the bar-chart to see easily spikes and constant expenses.\n", + "\n", + " * Navigate to **Billing -> Billing Account -> Reports** (see billing section)\n", + " * Get detailed reports for the past 30 days.\n", + " * In the **filters** on the Right Sidebar (open if necessary)\n", + " * Time Range \"Last 30 days\"\n", + " * Group By \"Sku\"\n", + " * Projects: All or your current project (should be already selected)\n", + " * All Services (should be already selected)\n", + " * All SKUs (should be already selected)\n", + " * In the main chart change options on the top of the **graph** to\n", + " * \"Daily\"\n", + " * Bar chart (bar chart graph on the right, \"Daily\" is on the left)\n", + " * **Save** view\n", + " * If you cannot save the view you can also click \"Share\" and use the URL to create a bookmark for easy access.\n", + " \n", + "These charts should be monitored daily for active projects and weekly for ongoing projects. For projects that are inactive the billing should be disabled or the project should be deleted.\n", + "\n", + "There reports do not show consumption in real time. In order to get an idea of what is generating costs (and you can estimate these costs based on past expereince), we will next show how to *enumerate* running resources. " + ] + }, { "cell_type": "markdown", "id": "cdfc0b3b-8bc3-44a3-a712-1005531c9d30", @@ -161,7 +222,7 @@ }, { "cell_type": "code", - "execution_count": 5, + "execution_count": 7, "id": "7ac47880-3760-4da0-a56d-8d59a0a51f24", "metadata": {}, "outputs": [ @@ -169,8 +230,9 @@ "name": "stdout", "output_type": "stream", "text": [ - "//compute.googleapis.com/projects/essentials-project-335018/zones/us-west2-c/instances/essentials\n", - "//compute.googleapis.com/projects/essentials-project-335018/zones/us-west2-c/disks/essentials\n" + "//storage.googleapis.com/essentials-learner-2021-12-16\n", + "//compute.googleapis.com/projects/just-armor-301114/zones/us-west2-c/instances/essentials\n", + "//compute.googleapis.com/projects/just-armor-301114/zones/us-west2-c/disks/essentials\n" ] } ], @@ -188,7 +250,7 @@ }, { "cell_type": "code", - "execution_count": 6, + "execution_count": 10, "id": "57caf876-c8b0-4263-bb96-19adcbba779a", "metadata": {}, "outputs": [ @@ -196,10 +258,44 @@ "name": "stdout", "output_type": "stream", "text": [ - "//cloudresourcemanager.googleapis.com/projects/essentials-project-335018\n", - "//compute.googleapis.com/projects/essentials-project-335018\n", - "//compute.googleapis.com/projects/essentials-project-335018/zones/us-west2-c/disks/essentials\n", - "//compute.googleapis.com/projects/essentials-project-335018/zones/us-west2-c/instances/essentials\n" + "//appengine.googleapis.com/apps/just-armor-301114\n", + "//bigquery.googleapis.com/projects/just-armor-301114/datasets/SAFE_Billing\n", + "//bigquery.googleapis.com/projects/just-armor-301114/datasets/SAFE_Billing/tables/cloud_pricing_export\n", + "//bigquery.googleapis.com/projects/just-armor-301114/datasets/SAFE_Billing/tables/gcp_billing_export_v1_01FBF6_A0C37F_BD97C8\n", + "//cloudresourcemanager.googleapis.com/projects/just-armor-301114\n", + "//compute.googleapis.com/projects/just-armor-301114\n", + "//compute.googleapis.com/projects/just-armor-301114/zones/us-west2-c/disks/essentials\n", + "//compute.googleapis.com/projects/just-armor-301114/zones/us-west2-c/instances/essentials\n", + "//container.googleapis.com/projects/just-armor-301114/zones/us-west2-c/clusters/grove-container-cluster/k8s/namespaces/kube-system/apps/deployments/metrics-server-v0.4.4\n", + "//container.googleapis.com/projects/just-armor-301114/zones/us-west2-c/clusters/grove-container-cluster/k8s/namespaces/kube-system/pods/fluentbit-gke-qqs9p\n", + "//container.googleapis.com/projects/just-armor-301114/zones/us-west2-c/clusters/grove-container-cluster/k8s/namespaces/kube-system/pods/kube-dns-697dc8fc8b-2rhmf\n", + "//container.googleapis.com/projects/just-armor-301114/zones/us-west2-c/clusters/grove-container-cluster/k8s/namespaces/kube-system/rbac.authorization.k8s.io/rolebindings/gce:cloud-provider\n", + "//container.googleapis.com/projects/just-armor-301114/zones/us-west2-c/clusters/grove-container-cluster/k8s/namespaces/kube-system/rbac.authorization.k8s.io/rolebindings/pdcsi-leaderelection-binding\n", + "//container.googleapis.com/projects/just-armor-301114/zones/us-west2-c/clusters/grove-container-cluster/k8s/nodes/gke-grove-container-clus-default-pool-5149d1fc-34zk\n", + "//container.googleapis.com/projects/just-armor-301114/zones/us-west2-c/clusters/grove-container-cluster/k8s/rbac.authorization.k8s.io/clusterrolebindings/npd-binding\n", + "//container.googleapis.com/projects/just-armor-301114/zones/us-west2-c/clusters/grove-container-cluster/k8s/rbac.authorization.k8s.io/clusterrolebindings/storage-version-migration-migrator-v2\n", + "//container.googleapis.com/projects/just-armor-301114/zones/us-west2-c/clusters/grove-container-cluster/k8s/rbac.authorization.k8s.io/clusterrolebindings/storage-version-migration-trigger\n", + "//container.googleapis.com/projects/just-armor-301114/zones/us-west2-c/clusters/grove-container-cluster/k8s/rbac.authorization.k8s.io/clusterrolebindings/system:controller:pod-garbage-collector\n", + "//container.googleapis.com/projects/just-armor-301114/zones/us-west2-c/clusters/grove-container-cluster/k8s/rbac.authorization.k8s.io/clusterrolebindings/system:controller:pv-protection-controller\n", + "//container.googleapis.com/projects/just-armor-301114/zones/us-west2-c/clusters/grove-container-cluster/k8s/rbac.authorization.k8s.io/clusterrolebindings/system:controller:resourcequota-controller\n", + "//container.googleapis.com/projects/just-armor-301114/zones/us-west2-c/clusters/grove-container-cluster/k8s/rbac.authorization.k8s.io/clusterrolebindings/system:controller:root-ca-cert-publisher\n", + "//container.googleapis.com/projects/just-armor-301114/zones/us-west2-c/clusters/grove-container-cluster/k8s/rbac.authorization.k8s.io/clusterrolebindings/system:controller:service-controller\n", + "//container.googleapis.com/projects/just-armor-301114/zones/us-west2-c/clusters/grove-container-cluster/k8s/rbac.authorization.k8s.io/clusterrolebindings/system:gke-controller\n", + "//container.googleapis.com/projects/just-armor-301114/zones/us-west2-c/clusters/grove-container-cluster/k8s/rbac.authorization.k8s.io/clusterrolebindings/system:gke-master-healthcheck\n", + "//container.googleapis.com/projects/just-armor-301114/zones/us-west2-c/clusters/grove-container-cluster/k8s/rbac.authorization.k8s.io/clusterrolebindings/system:gke-master-resourcequota\n", + "//container.googleapis.com/projects/just-armor-301114/zones/us-west2-c/clusters/grove-container-cluster/k8s/rbac.authorization.k8s.io/clusterrolebindings/system:kube-controller-manager\n", + "//container.googleapis.com/projects/just-armor-301114/zones/us-west2-c/clusters/grove-container-cluster/k8s/rbac.authorization.k8s.io/clusterrolebindings/system:kube-dns-autoscaler\n", + "//container.googleapis.com/projects/just-armor-301114/zones/us-west2-c/clusters/grove-container-cluster/k8s/rbac.authorization.k8s.io/clusterroles/konnectivity-agent-cpha\n", + "//container.googleapis.com/projects/just-armor-301114/zones/us-west2-c/clusters/grove-container-cluster/k8s/rbac.authorization.k8s.io/clusterroles/system:basic-user\n", + "//container.googleapis.com/projects/just-armor-301114/zones/us-west2-c/clusters/grove-container-cluster/k8s/rbac.authorization.k8s.io/clusterroles/system:certificates.k8s.io:certificatesigningrequests:selfnodeclient\n", + "//container.googleapis.com/projects/just-armor-301114/zones/us-west2-c/clusters/grove-container-cluster/k8s/rbac.authorization.k8s.io/clusterroles/system:controller:endpointslice-controller\n", + "//container.googleapis.com/projects/just-armor-301114/zones/us-west2-c/clusters/grove-container-cluster/k8s/rbac.authorization.k8s.io/clusterroles/system:controller:replicaset-controller\n", + "//container.googleapis.com/projects/just-armor-301114/zones/us-west2-c/clusters/grove-container-cluster/k8s/rbac.authorization.k8s.io/clusterroles/system:node\n", + "//container.googleapis.com/projects/just-armor-301114/zones/us-west2-c/clusters/grove-container-cluster/k8s/rbac.authorization.k8s.io/clusterroles/system:node-problem-detector\n", + "//dns.googleapis.com/projects/just-armor-301114/managedZones/5691889021211349439\n", + "//pubsub.googleapis.com/projects/just-armor-301114/topics/Bucket\n", + "//secretmanager.googleapis.com/projects/1002111293252/secrets/class-k8-bootstrap/versions/1\n", + "//storage.googleapis.com/essentials-learner-2021-12-16\n" ] } ], @@ -214,6 +310,78 @@ "source": [ "Now we can see that the list of resources is what we expect. Of interest here is the \"disk\" associated with the virtual machine. If these are not removed as well they can run up costs if you don't know where to look." ] + }, + { + "cell_type": "markdown", + "id": "a670902f-53d3-4fce-acb0-0ace9608cd5d", + "metadata": {}, + "source": [ + "To get up-to-date information about a service the `gcloud` command can be used to list allocated resources." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "d364da55-40d5-48d8-9c15-71cd08acb080", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "NAME ZONE MACHINE_TYPE PREEMPTIBLE INTERNAL_IP EXTERNAL_IP STATUS\n", + "essentials us-west2-c e2-highcpu-4 true 10.168.0.23 34.94.154.34 RUNNING\n" + ] + } + ], + "source": [ + "gcloud compute instances list" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "42057591-6252-431b-aaa9-1ad7560de021", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "NAME LOCATION LOCATION_SCOPE SIZE_GB TYPE STATUS\n", + "essentials us-west2-c zone 10 pd-standard READY\n" + ] + } + ], + "source": [ + "gcloud compute disks list" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "10a54a93-7b4c-41e8-a44e-7f1ea83ab923", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "gs://essentials-learner-2021-12-16/\n" + ] + } + ], + "source": [ + "gcloud alpha storage ls" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d815546b-5a0a-4280-af29-aa032a790f1d", + "metadata": {}, + "outputs": [], + "source": [] } ], "metadata": {