diff --git a/Build.md b/Build.md index cd47601..4a61cca 100644 --- a/Build.md +++ b/Build.md @@ -73,3 +73,12 @@ Expect that `aws` is installed locally. A ssh-key named 'learner' is required t ## Azure Expect that `az` is installed and configured locally. It will utilize your ~/.ssh/id_rsa.pub key. + +## Bugs and Fixes + +The bash kernel and Python do not like the new "bracketed paste" mode of the new bash/readline. To fix you can disable "bracketed paste" mode by doing +```bash +echo "set enable-bracketed-paste off" > ~/.inputrc +``` +Note, this removes some copy/paste protection and recommended only for disposable VM's. + diff --git a/content/GCP/01_intro_to_cloud_console.ipynb b/content/GCP/01_intro_to_cloud_console.ipynb index cde498b..3e27a2b 100644 --- a/content/GCP/01_intro_to_cloud_console.ipynb +++ b/content/GCP/01_intro_to_cloud_console.ipynb @@ -9,7 +9,7 @@ "source": [ "# Introduction to the GCP Cloud Console\n", "\n", - "```{admonition} Overview\n", + ":::{admonition} Overview\n", ":class: tip\n", "\n", "**Teaching:** 15 min.\n", @@ -17,14 +17,16 @@ "**Exercises:** 6 min.\n", "\n", "**Questions:**\n", - "* How do I find things in the web console?\n", - "* How do I access the cloud through the command line?\n", + " * How do I find things in the web console?\n", + " * How do I access the cloud through the command line?\n", "\n", "**Objectives:**\n", - "* Login to the Google Cloud web console\n", - "* Find and verify important information about your cloud environment.\n", - "* Find and open the Cloud Shell\n", - "```" + " * Login to the Google Cloud web console\n", + " * Find and verify important information about your cloud environment.\n", + " * Learn important Google Cloud terminology\n", + " * Find and open the Cloud Shell\n", + "\n", + ":::" ] }, { @@ -64,15 +66,17 @@ "tags": [] }, "source": [ - "```{admonition} Exercise\n", + ":::{admonition} Exercise\n", + ":class: exercise\n", "\n", "Take a few moments to ask yourself the following questions:\n", - " * What account are you currently using in the Google Cloud web console?\n", - " * Are you logged into any other accounts?\n", - " * How do you know which account is being used?\n", + " * What account are you currently using in the Google Cloud web console?\n", + " * Are you logged into any other accounts?\n", + " * How do you know which account is being used?\n", " \n", " If you are not currently using the correct account, switch to it.\n", - "```" + "\n", + ":::" ] }, { @@ -104,7 +108,8 @@ "id": "182e3a9a", "metadata": {}, "source": [ - "```{admonition} Exercise\n", + ":::{admonition} Exercise\n", + ":class: exercise\n", "\n", "Take a few moments to ask yourself the following questions:\n", " * What is the *name* of the Project you are currently using in the Google Cloud Platform web console?\n", @@ -114,7 +119,8 @@ " * How do you know which project you are currently using?\n", "\n", "If you are currently not using the correct project for this lesson, switch to the correct one.\n", - "```" + "\n", + ":::" ] }, { @@ -124,19 +130,20 @@ "tags": [] }, "source": [ - "## Services\n", + "## Products\n", "\n", - "The web console is used to control and observe the cloud from the browser. It should only be used for simple and one-time tasks, exploring new services, accessing documentation, or for monitoring and debugging resources in the cloud. Programmatic control through the console, programming languages (for example Python), and other automation tools should be used for day to day activities to make the most out of the cloud and to help with the reproducibility of research and teaching.\n", - "\n", - "The Navigation Menu (often called the **hamburger**) is used to navigate to the various products, which are also sometimes called services.\n", + "**Products** are the various services offered on Google Cloud such as *Google Compute Engine* and *Cloud Storage*. Within each product you create **resources** that you can use, such virtual machines. The Navigation Menu (often called the **hamburger**) is used to navigate to the various products.\n", "\n", "![hamburger-navigation](img/hamburger-navigation.png)\n", "\n", " * Click on a product to navigate the product page\n", " * You can pin frequently used items on the top of this page by clicking on the pin icon. \n", + " * A product is accessed and controlled via an API and usually must be enabled (once per project). To enable a product API click on the \"ENABLE\" button shown below.\n", " * Click the hamburger again to hide the Left Sidebar.\n", "\n", - "All the different products and services are the \"**What**\" of the cloud. We will start with compute, called *Google Compute Engine* in the next Episode. " + "![enable-compute-api](img/enable-compute-api.png)\n", + "\n", + "All the different products and resources are the \"**What**\" of the cloud. We will start with compute, called *Google Compute Engine* in the next Episode. " ] }, { @@ -146,13 +153,17 @@ "tags": [] }, "source": [ - "```{admonition} Exercise\n", + "*Instructor: place instructions on the screen*\n", + "\n", + ":::{admonition} Exercise 1\n", + ":class: exercise\n", "\n", - "Take a few moments to navigate a few key services.\n", - " * Navigate to \"Cloud Storage\" under the \"Storage\" product group.\n", - " * You will probably need to \"Enable\" this service first by clicking on the \"Enable\" button on the \"Cloud Storage API\" page. This will only need to be done once per project.\n", - " * Navigate to the \"Compute Engine\" service under the \"Compute\" product group and enable the service if necessary.\n", - "```" + "Take a few moments to navigate to a few key products and enable them.\n", + " * Navigate to \"Cloud Storage\" under the \"Storage\" product group.\n", + " * You may need to \"Enable\" this product first by clicking on the \"Enable\" button on the \"Cloud Storage API\" page. This will only need to be done once per project.\n", + " * Navigate to the \"Compute Engine\" product under the \"Compute\" product group and enable the product as well.\n", + "\n", + ":::" ] }, { @@ -170,8 +181,8 @@ "\n", "From left to right of the blue top navigation bar we have the following items (hover over the item with your mouse to see the name):\n", " 1. Navigation Menu (often called the hamburger) allows you to go to each project and services page (shown below).\n", - " 2. Dashboard (\"Google Cloud Platform\") provides an overview of your project.\n", - " * Click it and note resources, billing, and platform Status\n", + " 2. Welcome (\"Google Cloud Platform\") provides an overview of your project.\n", + " * Click \"Dashboard\" it and note resources, billing, and platform Status\n", " * This page is customizable and may look different.\n", " 3. Project selection button showing the current project name with a drop-down arrow.\n", " 4. Search bar - type in service or keywords\n", @@ -213,28 +224,43 @@ }, { "cell_type": "markdown", - "id": "118254c6-a5d9-4bd0-8a0f-64d4f72b3cf0", + "id": "9470567b-9966-4c07-a7ad-a94358530480", "metadata": { "tags": [] }, "source": [ - "```{admonition} Exercise\n", + "## Putting it Together\n", "\n", - "Before we move on, do the following:\n", - "* Double check that you are in the correct account and project that you will be using for this lesson.\n", - "* Take a moment to reflect on the \"Who, What, and Where\" of the cloud. \n", - " * What is your Who, What, and Where?\n", - " * Why is this important?\n", - "```" + "Now that we have the basics, we can provide a bit larger picture of the cloud and all the pieces:\n", + "\n", + "![gcp-overview](img/CLASS-Essentials-GCP-Overview.svg)\n", + "\n", + " * Project - a container to house and manage resources.\n", + " * Products - The different types of resources or services within Google Cloud.\n", + " * Resources - individual products running or stored within Google cloud such as compute, storage, networking, databases, etc. Resources can belong to only one Project.\n", + " * Billing Account - used to pay for resources and services consumed. Each project has only one billing account.\n", + " * Account - the identity that is used to accesses resources within Google Cloud. Note: this is *NOT* the Billing Account.\n", + " * Identity Access and Management (IAM) - manages access (permissions) to a resource by an account (also referred to as a principal).\n", + " * Organization - projects may optionally be housed in an organization (arranged in folders), which can add permissions and restrictions in a hierarchical manner. Projects can only belong to one Organization." ] }, { - "cell_type": "code", - "execution_count": null, - "id": "530f7a45-5708-4213-b120-71c6328f7026", - "metadata": {}, - "outputs": [], - "source": [] + "cell_type": "markdown", + "id": "118254c6-a5d9-4bd0-8a0f-64d4f72b3cf0", + "metadata": { + "tags": [] + }, + "source": [ + ":::{admonition} Exercise\n", + ":class: exercise\n", + "\n", + "Before we move on, do the following:\n", + " * Double check that you are in the correct account and project that you will be using for this lesson.\n", + " * Take a moment to reflect on the \"Who, Where, and What\" of the cloud. \n", + " * What is your Who, Where, and What?\n", + " * Why is this important?\n", + ":::" + ] } ], "metadata": { @@ -253,7 +279,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.9.7" + "version": "3.9.2" } }, "nbformat": 4, diff --git a/content/GCP/02_intro_to_compute.ipynb b/content/GCP/02_intro_to_compute.ipynb index 815c971..9bdafaf 100644 --- a/content/GCP/02_intro_to_compute.ipynb +++ b/content/GCP/02_intro_to_compute.ipynb @@ -9,7 +9,7 @@ "source": [ "# Introduction to Cloud Compute (Google Compute Engine)\n", "\n", - "```{admonition} Overview\n", + ":::{admonition} Overview\n", ":class: tip\n", "\n", "**Teaching:** 30 min.\n", @@ -17,16 +17,17 @@ "**Exercises:** 6 min\n", "\n", "**Questions:**\n", - "* How do I create my own computer in the cloud?\n", + " * How do I create my own computer in the cloud?\n", "\n", "**Objectives:**\n", - "* Navigate the Google Compute Engine service and terminology.\n", - "* Allocate a virtual machine in Google Compute Engine.\n", - "* Choose a machine type based on the application.\n", - "* Use the cost estimator for Google Compute Engine.\n", - "* Learn what a service account is and how they are used.\n", - "* Learn how to connect to a virtual machine using ssh in the Cloud Shell with the web console.\n", - "```" + " * Navigate the Google Compute Engine product and terminology.\n", + " * Allocate a virtual machine in Google Compute Engine.\n", + " * Choose a machine type based on the application.\n", + " * Use the cost estimator for Google Compute Engine.\n", + " * Learn what a service account is and how they are used.\n", + " * Learn how to connect to a virtual machine using ssh in the Cloud Shell with the web console.\n", + "\n", + ":::" ] }, { @@ -46,7 +47,7 @@ "source": [ "## Google Compute Engine\n", "\n", - "To learn more about the Google Compute Engine service we will use the web console to create, explore, and destroy a Virtual Machine. The ability to quickly create, explore, and then discard resources in the web-console is a powerful tool to learn more about a service. After an initial exploration with the web-console, programmatic access using scripts or configuration management systems should be used.\n", + "To learn more about the Google Compute Engine product we will use the web console to create, explore, and destroy a Virtual Machine. The ability to quickly create, explore, and then discard resources in the web-console is a powerful tool to learn more about a product. After an initial exploration with the web-console, programmatic access using scripts or configuration management systems should be used.\n", "\n", "Unfortunately, the terminology used for cloud computing differs from one commercial cloud provider to another. Google Compute Engine uses the term **VM instance** to refer to a running virtual machine, **image** for the boot disk containing the operating system, and **disk** for the writable storage used for a running *VM Instance*.\n", "\n", @@ -55,14 +56,14 @@ }, { "cell_type": "markdown", - "id": "c5430b40-1a5f-40df-9e13-529ef3ece4ce", + "id": "05f4a1dd-79eb-49bc-a52e-98e2b77c6ade", "metadata": { "tags": [] }, "source": [ "## Allocate a VM Instance\n", "\n", - "We will first create an *VM instance* with defaults to explore the Google Compute Engine service.\n", + "We will first create an *VM instance* with defaults to explore the Google Compute Engine product.\n", "\n", "To create a VM Instance we do the following:\n", " * Click **Navigation Menu** -> **Compute Engine** (under Compute) -> **VM Instances** -> **+Create Instance** (just under the blue bar) to open the *Create an instance* page.\n", @@ -76,9 +77,7 @@ " * In the **Identity and API access** section note that for the **Service Accounts** -> **Service account** option that the **Compute Engine default service account** is selected. This allows the *VM Instance* to access resources in the project. Note: the *\"Allow full access to all Cloud APIs\"* allows *everyone* on the VM to *access and edit all resources in the project*.\n", " * Leave the remaining options as default.\n", " * Note the **Monthly estimate** in the right side panel. This is only for running the *VM instance* and does not include other storage, networking, compute, and other charges that may occur by running software. Click **Details** show a breakdown of the *VM instance* only costs.\n", - "![compute-create-instance](img/compute-create-instance.png) \n", - " * Finally, click **Create** to create the *VM Instance* (allocate the resource).\n", - "![compute-create-instance-ok](img/compute-create-instance-ok.png)\n" + "![compute-create-instance](img/compute-create-instance.png) " ] }, { @@ -86,10 +85,25 @@ "id": "6737d8ba-e6d4-437f-a652-5d9d4477b34c", "metadata": {}, "source": [ - "```{admonition} Exercise\n", - "* Find the most expensive VM. **DO NOT actually create! Just cancel!**\n", - "* Find the least expensive VM.\n", - "```" + ":::{admonition} Exercise\n", + ":class: exercise\n", + "\n", + " * Find the most expensive VM. **DO NOT actually create!**\n", + " * Find the least expensive VM.\n", + " * Select `e2-medium`\n", + "\n", + ":::" + ] + }, + { + "cell_type": "markdown", + "id": "92263998-5a0b-4e37-9fc6-605f666309ac", + "metadata": { + "tags": [] + }, + "source": [ + " * Finally, click **Create** to create the *VM Instance* (allocate the resource).\n", + "![compute-create-instance-ok](img/compute-create-instance-ok.png)" ] }, { @@ -99,7 +113,7 @@ "source": [ "## More on Security (Optional)\n", "\n", - "Everything in the cloud requires permission (authorization). Ordinary we would configure and check security first but in the case of exploring services it is often easier to do things out of order. We noted that the *VM instance* was created with the *Compute Engine default service account*, and if the \"Allow full access to all Cloud Api's\" scope is enable, then everyone on the VM has access to all the resources in your project.\n", + "Everything in the cloud requires permission (authorization). Ordinary we would configure and check security first but in the case of exploring products it is often easier to do things out of order. We noted that the *VM instance* was created with the *Compute Engine default service account*, and if the \"Allow full access to all Cloud Api's\" scope is enable, then everyone on the VM has access to all the resources in your project.\n", "\n", "In the case of Drew, this is useful since they will be storing results of the analysis in Cloud Storage and the VM that will be doing the processing will need to have access to the storage. This is the mechanism that allows this.\n", "\n", @@ -113,9 +127,9 @@ "\n", "There are three important pieces of information that work together to form the **IAM policy**. The permission (role), the identity (principal or member), and the resource (project).\n", "\n", - "There is one additional tool that is used, **scope**. Scope describes the services that are accessible by an identity and is a simple way of limiting access to services. \n", + "There is one additional tool that is used, **scope**. Scope describes the products that are accessible by an identity and is a simple way of limiting access to products. \n", "\n", - "When we previously created the virtual machine we selected the default **scope**, which limits what services the virtual machine has access to. In other words, these scopes can only *limit* what a virtual machine instance can do, it can not add any permissions that are not already granted to the service account through IAM. In general, access should be managed through IAM policy, not scope.\n" + "When we previously created the virtual machine we selected the default **scope**, which limits what products the virtual machine has access to. In other words, these scopes can only *limit* what a virtual machine instance can do, it can not add any permissions that are not already granted to the service account through IAM. In general, access should be managed through IAM policy, not scope.\n" ] }, { @@ -131,7 +145,7 @@ "\n", "To view the project activity we do the following:\n", "\n", - " * Navigate to the project **Dashboard** by clicking on \"Google Cloud Platform\" on the left side of the top blue bar.\n", + " * Navigate to the project **Dashboard** by clicking on \"Google Cloud Platform\" on the left side of the top blue bar and clicking the \"*Dashboard*\" link.\n", " * Select **Activity** tab just below the top blue bar.\n", " * Look for the row with **Create VM** on the top of the table. This shows that the *VM Instance* was created.\n", " * Expand the detail by clicking the **down arrow** on the right side of the row. You can see what identity created the bucket and the name. The rest of the information details the initial settings that you configured when creating the virtual machine. The details are not really important at this time, just know they exist.\n", @@ -269,11 +283,12 @@ "source": [ "## Delete the VM Instance\n", "\n", - "```{admonition} Danger\n", + ":::{admonition} Danger\n", ":class: danger\n", "\n", "This will destroy the VM instance and and ALL the data in it! You cannot undelete an instance!\n", - "```" + "\n", + ":::" ] }, { @@ -285,11 +300,13 @@ "source": [ "Now that we have made that clear, we will now complete the resource life-cycle by deleting the *VM Instance* we just created. First, try to do this on your own.\n", "\n", - "```{admonition} Exercise\n", + ":::{admonition} Exercise\n", + ":class: exercise\n", + "\n", + " * Try to delete the ***VM instance*** on your own\n", + " * Try to follow the process we just learned (hint: *track* and *list*)\n", "\n", - "* Try to delete the ***VM instance*** on your own\n", - "* Try to follow the process we just learned (hint: *track* and *list*)\n", - "```" + ":::" ] }, { @@ -308,7 +325,7 @@ "\n", "Did you \"Follow\" the VM instance by looking at the **activity** page as discussed above?\n", "\n", - "Since we care about paying for resources we are not using we review our project by visiting the *compute engine* service and reviewing that we no longer have any *VM instances* running. \n", + "Since we care about paying for resources we are not using we review our project by visiting the *compute engine* product and reviewing that we no longer have any *VM instances* running. \n", " * Go to **Navigation Menu** -> **Compute Engine** (under Compute) -> **Instances** and look for your virtual machine. It should be gone!\n", "\n", "In a later episode we will show how to easily list all the resources in a project." @@ -340,7 +357,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.9.7" + "version": "3.9.2" } }, "nbformat": 4, diff --git a/content/GCP/03_intro_to_cloud_storage.ipynb b/content/GCP/03_intro_to_cloud_storage.ipynb index a8e3028..42ddb9c 100644 --- a/content/GCP/03_intro_to_cloud_storage.ipynb +++ b/content/GCP/03_intro_to_cloud_storage.ipynb @@ -7,7 +7,7 @@ "source": [ "# Introduction to Google Cloud Storage\n", "\n", - "```{admonition} Overview\n", + ":::{admonition} Overview\n", ":class: tip\n", "\n", "**Teaching:** 20 min\n", @@ -15,16 +15,25 @@ "**Exercises:** 5 min\n", "\n", "**Questions:**\n", - "* How do I store data in the cloud?\n", + " * How do I store data in the cloud?\n", "\n", "**Objectives:**\n", - "* Navigate the Google Cloud Storage service and terminology\n", - "* Allocate storage in Google Cloud Storage\n", - "* Find the cost estimator for Google Cloud Storage\n", - "* Recognize that resources have a \"location\"\n", - "* Recognize danger of public access\n", - "* De-allocate Google Cloud Storage storage\n", - "```" + " * Navigate the Google Cloud Storage service and terminology\n", + " * Allocate storage in Google Cloud Storage\n", + " * Find the cost estimator for Google Cloud Storage\n", + " * Recognize that resources have a \"location\"\n", + " * Recognize danger of public access\n", + " * De-allocate Google Cloud Storage storage\n", + "\n", + ":::" + ] + }, + { + "cell_type": "markdown", + "id": "9358d09a-8eb9-4b6e-bd27-16f1beab6c0c", + "metadata": {}, + "source": [ + "Now that Drew has a basic handle on how to create a virtual machine instance and has played around with it they are now ready to figure out where to store the data. Drew was advised that for storing and sharing data outside a virtual machine instance that Object storage is the first thing to try." ] }, { @@ -32,7 +41,9 @@ "id": "9897048a-6aa8-4d85-a557-d85b802f3f1d", "metadata": {}, "source": [ - "There are many storage services in the cloud. One of the most common and economic ways to store data in the cloud is to use object storage. In GCP object storage is called *Google Cloud Storage*, which is similar to the Simple Storage Service, also known as S3, on Amazon Web Services (AWS). For object storage, information is stored as a collection of key-value pairs. This is different to how data is commonly stored on laptops and high performance computing clusters (supercomputers)." + "## Storage\n", + "\n", + "There are many storage products and services in the cloud. One of the most common and economic ways to store data in the cloud is to use object storage. In GCP object storage is called *Google Cloud Storage*, which is similar to the Simple Storage Service, also known as S3, on Amazon Web Services (AWS). For object storage, information is stored as a collection of key-value pairs. This is different to how data is commonly stored on laptops and high performance computing clusters (supercomputers)." ] }, { @@ -99,7 +110,7 @@ "\n", "To view the project activity we do the following:\n", "\n", - " * Navigate to the project **Dashboard** by clicking on \"Google Cloud Platform\" on the left side of the top blue bar.\n", + " * Navigate to the project **Dashboard** by clicking on \"Google Cloud Platform\" on the left side of the top blue bar and clicking on the \"*Dashboard*\" link.\n", " * Select **Activity** tab just below the top blue bar.\n", " * Look for the row with **Create Bucket** on the top of the table. This shows that the bucket was created.\n", " * Expand the detail by clicking the **down arrow** on the right side of the row. You can see what identity created the bucket and the name. The rest of the information details the initial settings that you configured when creating the bucket. The details are not really important at this time, just know they exist.\n", @@ -150,27 +161,38 @@ "source": [ "## Delete the Storage\n", "\n", - "```{admonition} Danger\n", + ":::{admonition} Danger\n", ":class: danger\n", "\n", "This will destroy the storage *Bucket* and and ALL the data in it! You cannot undelete a Bucket!\n", - "```" + "\n", + ":::" + ] + }, + { + "cell_type": "markdown", + "id": "619c0350-6058-4162-9b4b-3a6aaf96041f", + "metadata": { + "tags": [] + }, + "source": [ + "We will now complete the resource life-cycle by deleting the bucket we just created. Think about the process we did for creating the bucket and follow a similar one." ] }, { "cell_type": "markdown", - "id": "59005efb-2aad-4dcd-ac7b-f153f4295abc", + "id": "9a6973f1-07f4-4c72-bd92-1a6b9741eed9", "metadata": { "tags": [] }, "source": [ - "We will now complete the resource life-cycle by deleting the bucket we just created. Think about the process we did for creating the bucket and follow a similar one.\n", + ":::{admonition} Exercise\n", + ":class: exercise\n", "\n", - "```{admonition} Exercise\n", + " * Try to delete the storage bucket on your own\n", + " * Verify that the storage bucket has been deleted.\n", "\n", - " * Try to delete the storage bucket on your own\n", - " * Verify that the storage bucket has been deleted.\n", - "```" + ":::" ] }, { @@ -237,7 +259,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.7.3" + "version": "3.9.2" } }, "nbformat": 4, diff --git a/content/GCP/04_intro_to_cli.ipynb b/content/GCP/04_intro_to_cli.ipynb index 957463e..33404b9 100644 --- a/content/GCP/04_intro_to_cli.ipynb +++ b/content/GCP/04_intro_to_cli.ipynb @@ -7,7 +7,7 @@ "source": [ "# Introduction to the Command Line Interface (CLI)\n", "\n", - "```{admonition} Overview\n", + ":::{admonition} Overview\n", ":class: tip\n", "\n", "**Teaching:** 20 min\n", @@ -15,16 +15,17 @@ "**Exercises:** 5 min\n", "\n", "**Questions:**\n", - "* How do I use the Cloud CLI?\n", - "* How do I use `gcloud`?\n", + " * How do I use the Cloud CLI?\n", + " * How do I use `gcloud`?\n", "\n", "**Objectives:**\n", - "* Find and use the \"Cloud Shell\" in the web console.\n", - "* Use basic cloud CLI commands (`gcloud` and `gsutil`).\n", - "* Verify basic settings.\n", - "* Use environment variables for configuration.\n", - "* Understand the importance of using variables reproducibility and automation.\n", - "``` " + " * Find and use the \"Cloud Shell\" in the web console.\n", + " * Use basic cloud CLI commands (`gcloud` and `gsutil`).\n", + " * Verify basic settings.\n", + " * Use environment variables for configuration.\n", + " * Understand the importance of using variables reproducibility and automation.\n", + "\n", + ":::" ] }, { @@ -61,11 +62,11 @@ "id": "8eb44568-1108-4963-aebf-7f6b98ddaffc", "metadata": {}, "source": [ - "```{admonition} Tip\n", - ":class: tip\n", + ":::{tip}\n", "\n", "You can access the cloud shell from your laptop via `gcloud cloud-shell ssh` if you have the [Google Cloud SDK installed](https://cloud.google.com/sdk/docs/install). Installing the Google Cloud SDK will also allow you to run `gcloud` and `gsutil` commands directly from your laptop, workstation, or cluster. *Pro-tip, you cannot use a service account to access the cloud shell.*\n", - "```" + "\n", + ":::" ] }, { @@ -147,6 +148,17 @@ "gcloud config get-value project" ] }, + { + "cell_type": "markdown", + "id": "379aa938-04e0-4663-b402-e79b03b474da", + "metadata": {}, + "source": [ + "We will first use the `gsutil` command, which is similar to `gcloud` but older but more comprehensive. We will first check to see if the command working by showing the help by issuing the following command:\n", + "```\n", + "gcloud help\n", + "```" + ] + }, { "cell_type": "markdown", "id": "1389ca4f-7234-4ea3-9ad8-85914d88ede5", @@ -186,7 +198,7 @@ }, { "cell_type": "markdown", - "id": "598b5d12-9bea-49de-a772-4113727e0859", + "id": "fe9c7d78-edbe-4c84-81d1-a83c20773181", "metadata": {}, "source": [ "### Environment Variables\n", @@ -195,13 +207,19 @@ "\n", "*To make the point even clearer, we use environment variables in the Cloud Essentials notebooks instead of hard coding them, which makes it easy for anyone to run, use, and edit and thus collaborate and contribute to them!*\n", "\n", - "We now demonstrate the basic use environment variables. As a reminder, environment variables are used to store configuration information that can be easily passed between programs and programming languages.\n", - "\n", - "```{admonition} Advanced Tip\n", - ":class: tip\n", + "We now demonstrate the basic use environment variables. As a reminder, environment variables are used to store configuration information that can be easily passed between programs and programming languages." + ] + }, + { + "cell_type": "markdown", + "id": "b843dba2-3276-4e2e-9619-8598b2718e4d", + "metadata": {}, + "source": [ + ":::{tip}\n", "\n", "Although we do not always need to, we should always enclose shell variables in double quotes and enclose the variable name in curly braces (funny things may happen otherwise) when writing scripts, for example `\"${TEST}\"`. In most cases it is easier to do this every time than try to figure out when it is possible to use the short form. We only use the short form (`$TEST`) when it is used frequently and known to always work (most cases here), in our case it is `$PROJECT` and `$BUCKET`. When in doubt, use the explicit form.\n", - "```" + "\n", + ":::" ] }, { @@ -402,7 +420,7 @@ "source": [ "Note the \"Your active configuration is: ...\" line is a \"warning\" and not actually part of the output of the program.\n", "\n", - "````{admonition} Advanced Tip\n", + "::::{admonition} Advanced Tip\n", ":class: tip\n", "\n", "You can also use \n", @@ -410,7 +428,8 @@ "PROJECT=$(gcloud config list --format='value(core.project)')\n", "```\n", "to get the project id without the warning.\n", - "````" + "\n", + "::::" ] }, { @@ -451,11 +470,98 @@ "id": "270e0abb-d5e9-476a-af91-060cc0ea0349", "metadata": {}, "source": [ - "```{admonition} Exercise\n", + ":::{admonition} Exercise\n", + ":class: exercise\n", "\n", - " * What command shows the account? (hint: we used it already)\n", - " * Set the `ACCOUNT` environment variable with the correct value. (hint: be careful with punctuation (single and double quotes) and matching brackets and punctuation)\n", - "```" + " * What command shows the account? (hint: we used it already)\n", + " * Set the `ACCOUNT` environment variable with the correct value. (hint: be careful with punctuation (single and double quotes) and matching brackets and punctuation)\n", + "\n", + ":::" + ] + }, + { + "cell_type": "markdown", + "id": "4ba1b3a1-15fc-40f3-ad7e-f260a85ce0a3", + "metadata": {}, + "source": [ + "### Naming Resources\n", + "\n", + "Many times we will create resources (which have names) that we will need to use later. In the next episode we will be creating a *Cloud Storage* bucket (a resource). Bucket names are globally unique, so here we use \"essentials\" with your Cloud Shell username (the username part of the Google Account) as a prefix to name the bucket something unique and easy to understand where it came from." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "247a515d-04ed-4dd1-bd7b-aa52df948ce8", + "metadata": {}, + "outputs": [], + "source": [ + "export BUCKET=\"essentials-$USER-$(date +%F)\"" + ] + }, + { + "cell_type": "markdown", + "id": "17a58166-e9e2-419d-9a3c-5ce746e0bf8e", + "metadata": {}, + "source": [ + "As usual, we will verify that it was set correctly." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "13a99343-4de3-436b-9f45-6b4f4eff4895", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "essentials-learner-2022-05-02\n" + ] + } + ], + "source": [ + "echo $BUCKET" + ] + }, + { + "cell_type": "markdown", + "id": "bde1ca1b-5904-4aa2-92ac-a31185d42d35", + "metadata": {}, + "source": [ + "## Environment Configuration\n", + "\n", + "We will use what we just learned to get ready for the next episode by configuring the environment in one step. These commands can be included in your scripts, however, we will be simply copy and pasting them in the Cloud Shell for the remainder of the lesson." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "95ac29b1-2bf0-49e8-a409-dff8f77b754f", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Account: 998995103712-compute@developer.gserviceaccount.com Project: essentials-learner Bucket: essentials-learner-2022-05-02\n" + ] + } + ], + "source": [ + "PROJECT=$(gcloud config get-value project)\n", + "ACCOUNT=$(gcloud config get-value account)\n", + "BUCKET=\"essentials-$USER-$(date +%F)\"\n", + "echo \"Account: $ACCOUNT Project: $PROJECT Bucket: $BUCKET\"" + ] + }, + { + "cell_type": "markdown", + "id": "24cf2c5f-742c-4b00-9f3a-05251a8889fb", + "metadata": {}, + "source": [ + "You must verify the output since this will be the \"Who, Where, and What\" of what you are doing." ] }, { diff --git a/content/GCP/05_cli_storage.ipynb b/content/GCP/05_cli_storage.ipynb index 878850e..8e8507e 100644 --- a/content/GCP/05_cli_storage.ipynb +++ b/content/GCP/05_cli_storage.ipynb @@ -8,7 +8,7 @@ "# Managing Cloud Storage from the Command Line\n", "\n", "\n", - "```{admonition} Overview\n", + ":::{admonition} Overview\n", ":class: tip\n", "\n", "**Teaching:** 20 min\n", @@ -16,13 +16,14 @@ "**Exercises:** 2 min\n", "\n", "**Questions:**\n", - "* How do I store data in a Bucket?\n", + " * How do I store data in a Bucket?\n", "\n", "**Objectives:**\n", - "* Manage buckets using the command line.\n", - "* Create a bucket with a sensible configuration.\n", - "* Learn to manage data (add, list, retrieve, delete) in a bucket using the command line.\n", - "```\n" + " * Manage buckets using the command line.\n", + " * Create a bucket with a sensible configuration.\n", + " * Learn to manage data (add, list, retrieve, delete) in a bucket using the command line.\n", + "\n", + ":::" ] }, { @@ -40,7 +41,7 @@ "source": [ "## Configuration\n", "\n", - "It is important to verify the Account and Project information every time you start interacting with the cloud. We also use this opportunity to set the configuration environment variables (`PROJECT`) for the Episode.\n" + "We will use what we learned in the previous episode to set the configuration by entering the commands below. Remember, it is important to verify the Account and Project information every time you start interacting with the cloud. " ] }, { @@ -53,70 +54,27 @@ "name": "stdout", "output_type": "stream", "text": [ - "Your active configuration is: [cloudshell-27767]\n", - "learner@class.internet2.edu\n" + "Account: 998995103712-compute@developer.gserviceaccount.com Project: essentials-learner Bucket: essentials-learner-2022-05-02\n" ] } ], "source": [ - "gcloud config get-value account" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "id": "9db0f3ca-2402-4002-b9d3-e634580e8f7f", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Your active configuration is: [cloudshell-27767]\n", - "essentials-learner\n" - ] - } - ], - "source": [ - "gcloud config get-value project" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "id": "44f0b255-508a-4488-abde-87aae67cd5b3", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Your active configuration is: [cloudshell-27767]\n", - "learner@class.internet2.edu\n" - ] - } - ], - "source": [ - "ACCOUNT=$(gcloud config get-value account) ; echo $ACCOUNT" + "PROJECT=$(gcloud config get-value project)\n", + "ACCOUNT=$(gcloud config get-value account)\n", + "BUCKET=\"essentials-$USER-$(date +%F)\"\n", + "echo \"Account: $ACCOUNT Project: $PROJECT Bucket: $BUCKET\"" ] }, { - "cell_type": "code", - "execution_count": 4, - "id": "36fcc48b-8a5b-4158-9704-f1ea7c9951eb", + "cell_type": "markdown", + "id": "0727548c-bb05-4664-8ed5-cc33ba1b8021", "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Your active configuration is: [cloudshell-27767]\n", - "essentials-learner\n" - ] - } - ], "source": [ - "PROJECT=$(gcloud config get-value project) ; echo $PROJECT" + ":::{tip}\n", + "\n", + "If you need to setup a new Cloud Shell you can copy and paste the above commands into the new shell.\n", + "\n", + ":::" ] }, { @@ -126,7 +84,7 @@ "source": [ "## Create a Bucket\n", "\n", - "We will first make a new bucket (mb) with mostly default values. We will explore the newly created bucket with default values then immediately destroy it. This is a typical pattern when learning a new service by reducing chances to make a mistake (errors can be hard to understand). In most cases resources should not be created with all default values. We will re-create the bucket in the next section with more sensible values. " + "We will first make a new bucket (mb) with the default values. We will explore the newly created bucket with default values then immediately destroy it. This is a typical pattern when learning a new product by reducing chances to make a mistake (errors can be hard to understand). In most cases resources should not be created with all default values. We will re-create the bucket in the next section with more sensible values. " ] }, { @@ -134,12 +92,12 @@ "id": "658e1324", "metadata": {}, "source": [ - "We first generate a bucket name and store it in an environment variable for future use. Bucket names are globally unique, so here we use \"essentials\" with your Cloud Shell username (the username part of the Google Account) as a prefix to name the bucket something unique and easy to understand where it came from." + "We first verify that `$BUCKET` is set correctly." ] }, { "cell_type": "code", - "execution_count": 5, + "execution_count": 2, "id": "255eb914-a06a-4bfd-adbb-7ca2f94678e7", "metadata": {}, "outputs": [ @@ -147,15 +105,22 @@ "name": "stdout", "output_type": "stream", "text": [ - "essentials-learner-2022-02-03\n" + "essentials-learner-2022-05-02\n" ] } ], "source": [ - "BUCKET=\"essentials-$USER-$(date +%F)\"\n", "echo $BUCKET" ] }, + { + "cell_type": "markdown", + "id": "9a5789df-ed6b-44a5-8f06-7937c62d6035", + "metadata": {}, + "source": [ + "And now we create the bucket. Note that most times we must use `gs://` to indicate that it is a bucket. " + ] + }, { "cell_type": "code", "execution_count": 6, @@ -179,9 +144,7 @@ "id": "42584633-f77f-43b4-bb23-49ae4364af07", "metadata": {}, "source": [ - "If the bucket creation fails with a \"ServiceException: 409 A Cloud Storage bucket named ... already exists.\" it means that the name is not unique or you have already created the bucket. You will need to either delete the bucket or choose another name.\n", - "\n", - "*Advanced Note: Bucket creation may fail if you are not using the Cloud Shell and the machine you are using has generic account name such as `pi`, `admin`, or other standard username. If this is the case just set `BUCKET` to some unique value.*" + "If the bucket creation fails with a \"ServiceException: 409 A Cloud Storage bucket named ... already exists.\" it means that the name is not unique or you have already created the bucket. You will need to either delete the bucket or choose another name." ] }, { @@ -191,7 +154,7 @@ "source": [ "## Show the Bucket\n", "\n", - "We will now list (Enumerate) the resource to verify that the resource (bucket) has been created by using the `gsutil ls` command to list all the buckets." + "We will now list all the buckets (resources) to verify that the bucket has been created by using the `gsutil ls` command." ] }, { @@ -282,7 +245,11 @@ "cell_type": "code", "execution_count": 9, "id": "8fc85600-a793-413b-b139-642e4cf08a8e", - "metadata": {}, + "metadata": { + "tags": [ + "remove-cell" + ] + }, "outputs": [], "source": [ "sleep 2 # used to ensure that the next command runs correctly in Jupyter" @@ -334,7 +301,7 @@ "id": "e9e48571-80fa-40cc-838f-ec5ea3000bd7", "metadata": {}, "source": [ - "````{admonition} Note\n", + "::::{admonition} Note\n", ":class: note\n", "\n", "You will probably need to enable the logging api to continue, press `y` to accept as shown below:\n", @@ -343,7 +310,7 @@ "like to enable and retry (this will take a few minutes)? (y/N)?\n", "```\n", "\n", - "````" + "::::" ] }, { @@ -640,7 +607,7 @@ } ], "source": [ - "gsutil ls \"gs://$BUCKET\"" + "gsutil ls gs://$BUCKET" ] }, { @@ -648,10 +615,12 @@ "id": "1427e837-f65f-4d35-8088-6a82677b5ed6", "metadata": {}, "source": [ - "```{admonition} Exercise\n", + ":::{admonition} Exercise\n", + ":class: exercise\n", + "\n", + " * Display the object with the date in the bucket.\n", "\n", - "* Display the object with the date in the bucket.\n", - "```" + ":::" ] }, { diff --git a/content/GCP/06_running_analysis.ipynb b/content/GCP/06_running_analysis.ipynb index 5a725e7..d33b896 100644 --- a/content/GCP/06_running_analysis.ipynb +++ b/content/GCP/06_running_analysis.ipynb @@ -7,7 +7,7 @@ "source": [ "# Putting It All Together\n", "\n", - "```{admonition} Overview\n", + ":::{admonition} Overview\n", ":class: tip\n", "\n", "**Teaching:** 60 min\n", @@ -15,21 +15,21 @@ "**Exercises:** 8 min\n", "\n", "**Questions:**\n", - " * Can you show me an example?\n", + " * Can you show me an example?\n", "\n", "**Objectives:**\n", - " * Create a simple workflow using a cloud VM and cloud object storage.\n", - " * Update a VM Instance software for important security updates.\n", - " * Create a VM Instance with the appropriate storage scope.\n", - " * Create a private regional storage bucket with appropriate security settings.\n", - " * Using the CLI to install software.\n", - " * Download source code using git\n", - " * Retrieve data from a bucket\n", - " * Run the python analysis code\n", - " * Store results in a bucket\n", - " * View the results in the Cloud Storage browser.\n", + " * Create a simple workflow using a cloud VM and cloud object storage.\n", + " * Update a VM Instance software for important security updates.\n", + " * Create a VM Instance with the appropriate storage scope.\n", + " * Create a private regional storage bucket with appropriate security settings.\n", + " * Using the CLI to install software.\n", + " * Download source code using git\n", + " * Retrieve data from a bucket\n", + " * Run the python analysis code\n", + " * Store results in a bucket\n", + " * View the results in the Cloud Storage browser.\n", "\n", - "```" + ":::" ] }, { @@ -42,6 +42,23 @@ "Drew needs to do some analysis on the data. They need data (satellite images stored in the cloud), computational resources (a virtual machine), some software (we will supply this), and a place to store the results (Cloud Storage). We will assemble and process all these parts in the cloud with a simple example.\n" ] }, + { + "cell_type": "markdown", + "id": "03938799-181e-4755-961d-e2f837af7db7", + "metadata": {}, + "source": [ + ":::{tip}\n", + "\n", + "If you get disconnected, you will need to reconnect to the virtual machine (`gcloud compute ssh essentials`) and re-run the following commands:\n", + "```\n", + "BUCKET=\"essentials-$USER-$(date +%F)\"\n", + "REGION=\"us-west2\"\n", + "echo \"bucket: $BUCKET region: $REGION\"\n", + "cd ~/CLASS-Examples/landsat\n", + "```\n", + ":::" + ] + }, { "cell_type": "markdown", "id": "245ffbcb-bf77-4ad1-8bde-1c7d2717ab46", @@ -51,7 +68,19 @@ "\n", "Since we only create resources as we need them in the cloud, we will now create a new virtual machine (VM)instance for Drew to use for their analysis.\n", "\n", - "We will do this as an exercise to give you practice in creating resources. Since the virtual machine will need access to storage on your behalf, you will need to change the **access scope** to give **Full** access to the **Storage** API to the virtual machine." + "We will do this as an exercise to give you practice in creating resources. Since the virtual machine will need access to storage on your behalf, you will need to change the **access scope** to give **Full** access to the **Storage** API to the virtual machine.\n", + "\n", + "Before you do anything, think about (and check) your **who**, **where**, and **what**!\n" + ] + }, + { + "cell_type": "markdown", + "id": "276d6d6e-5d3e-4fb7-ae4c-a70470d7af4b", + "metadata": {}, + "source": [ + "*Instructor: place the exercise instructions below on the screen*\n", + "\n", + "*When you are done feel free to connect to the virtual machine on your own for additional practice. Once everyone has created their VM we will connect to the machine as described below.*" ] }, { @@ -59,24 +88,22 @@ "id": "45cb5d1d-c28a-4e0e-a0f3-bf68add065be", "metadata": {}, "source": [ - "````{admonition} Exercise\n", + ":::{admonition} Exercise 6\n", + ":class: exercise\n", "\n", "Using the console navigate to the \"Compute Engine\" service and create a new VM with the following properties.\n", - " * Call the VM instance \"essentials\"\n", - " * Allow the VM instance \"Full\" access to \"Storage\". This can be found under \"Identity and API\" on the \"create an instance\" page and then selecting \"Set access for each API\" and change \"Storage\" to \"Full\". **This will allow the VM to create, read, write, and delete all storage buckets in the project\"**\n", - " * Select a bit larger VM instance by changing the machine type to \"e2-standard-2\".\n", - "````" + " * Set the VM instance **name** to \"**essentials**\"\n", + " * Select a bit larger VM instance by changing the machine **type** to \"**e2-standard-2**\".\n", + " * Set the VM instance API access for \"**Storage**\" to \"**Full**\". This can be found under \"Identity and API access\" on the \"create an instance\" page and then selecting \"**Set access for each API**\" and change \"Storage\" to \"Full\". *This will allow the VM to create, read, write, and delete all storage buckets in the project\"*\n", + " \n", + ":::" ] }, { "cell_type": "markdown", - "id": "009fbfee-f1ce-4046-8223-0aa11b21bd61", + "id": "4bfb5021-d07d-43c2-b651-5dd6fa498620", "metadata": {}, "source": [ - "*Instructor: place the above exercise instructions on the screen*\n", - "\n", - "*When you are done feel free to connect to the virtual machine on your own for additional practice. Once everyone has created their VM we will connect to the machine as described below.*\n", - "\n", "Please verify that the virtual machine was created as above. If you are unsure delete the virtual machine instance and create it again.\n", "\n", "Verify that the **Compute Engine default service account** is being used.\n", @@ -86,8 +113,7 @@ "![compute-iam-scope-top](img/compute-iam-scope-top.png)\n", "\n", "And set **Storage** to **Full**.\n", - "![compute-iam-scope-storage-full](img/compute-iam-scope-storage-full.png)\n", - "\n" + "![compute-iam-scope-storage-full](img/compute-iam-scope-storage-full.png)" ] }, { @@ -97,7 +123,7 @@ "source": [ "## Connect to the VM\n", "\n", - "Now login to the new virtual machine instance by opening up the Cloud Shell and by running the following command:\n", + "Now login to the new virtual machine instance by opening up the Cloud Shell (don't forget to check your **who**, **where**, and **what**) and then running the following command:\n", "```\n", "gcloud compute ssh essentials\n", "```\n", @@ -267,19 +293,23 @@ }, { "cell_type": "code", - "execution_count": 5, + "execution_count": 1, "id": "05fadd30-4f35-4a93-b713-cc132131b948", - "metadata": {}, + "metadata": { + "tags": [ + "hide-output" + ] + }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Reading package lists... Done\n", - "Building dependency tree \n", + "Building dependency tree... Done\n", "Reading state information... Done\n", - "git is already the newest version (1:2.20.1-2+deb10u3).\n", - "0 upgraded, 0 newly installed, 0 to remove and 2 not upgraded.\n" + "git is already the newest version (1:2.30.2-1).\n", + "0 upgraded, 0 newly installed, 0 to remove and 11 not upgraded.\n" ] } ], @@ -289,7 +319,7 @@ }, { "cell_type": "code", - "execution_count": 6, + "execution_count": 2, "id": "96db6a66-3fbf-419a-b8c8-dbb27639e990", "metadata": {}, "outputs": [], @@ -299,7 +329,7 @@ }, { "cell_type": "code", - "execution_count": 7, + "execution_count": 3, "id": "36554c99-ba08-4733-8ef2-e68d42d0d2b7", "metadata": {}, "outputs": [ @@ -308,9 +338,12 @@ "output_type": "stream", "text": [ "Cloning into 'CLASS-Examples'...\n", - "remote: Enumerating objects: 86, done.\u001b[K\n", - "remote: Total 86 (delta 0), reused 0 (delta 0), pack-reused 86\u001b[K\n", - "Unpacking objects: 100% (86/86), done.\n" + "remote: Enumerating objects: 117, done.\u001b[K\n", + "remote: Counting objects: 100% (10/10), done.\u001b[K\n", + "remote: Compressing objects: 100% (9/9), done.\u001b[K\n", + "remote: Total 117 (delta 1), reused 9 (delta 1), pack-reused 107\u001b[K\n", + "Receiving objects: 100% (117/117), 20.66 KiB | 542.00 KiB/s, done.\n", + "Resolving deltas: 100% (44/44), done.\n" ] } ], @@ -328,7 +361,7 @@ }, { "cell_type": "code", - "execution_count": 8, + "execution_count": 4, "id": "90c1cda7-60d4-44bb-84f8-e776a77a94ab", "metadata": {}, "outputs": [], @@ -349,7 +382,7 @@ }, { "cell_type": "code", - "execution_count": 9, + "execution_count": 5, "id": "55b628d5-6e5c-45a5-9cd3-c129db9cdcd2", "metadata": {}, "outputs": [ @@ -357,14 +390,15 @@ "name": "stdout", "output_type": "stream", "text": [ - "total 28\n", - "-rw-r--r-- 1 learner learner 964 Feb 14 20:30 ReadMe.md\n", - "-rw-r--r-- 1 learner learner 72 Feb 14 20:30 clean.sh\n", - "-rw-r--r-- 1 learner learner 280 Feb 14 20:30 download.sh\n", - "-rw-r--r-- 1 learner learner 314 Feb 14 20:30 get-index.sh\n", - "-rw-r--r-- 1 learner learner 613 Feb 14 20:30 process_sat.py\n", - "-rw-r--r-- 1 learner learner 76 Feb 14 20:30 search.json\n", - "-rw-r--r-- 1 learner learner 783 Feb 14 20:30 search.py\n" + "total 32\n", + "-rw-r--r-- 1 learner learner 964 May 20 17:41 ReadMe.md\n", + "-rw-r--r-- 1 learner learner 72 May 20 17:41 clean.sh\n", + "-rw-r--r-- 1 learner learner 280 May 20 17:41 download.sh\n", + "-rw-r--r-- 1 learner learner 113 May 20 17:41 get-data.sh\n", + "-rw-r--r-- 1 learner learner 345 May 20 17:41 get-index.sh\n", + "-rw-r--r-- 1 learner learner 613 May 20 17:41 process_sat.py\n", + "-rw-r--r-- 1 learner learner 95 May 20 17:41 search.json\n", + "-rw-r--r-- 1 learner learner 851 May 20 17:41 search.py\n" ] } ], @@ -386,7 +420,7 @@ }, { "cell_type": "code", - "execution_count": 10, + "execution_count": 6, "id": "e56ab74a-ae6d-4602-a26b-4a2656bd40cd", "metadata": {}, "outputs": [ @@ -435,7 +469,7 @@ }, { "cell_type": "code", - "execution_count": 11, + "execution_count": 7, "id": "bbe85b75-c7cd-40ed-a3b0-37cbd0a5f52e", "metadata": {}, "outputs": [ @@ -453,7 +487,7 @@ }, { "cell_type": "code", - "execution_count": 12, + "execution_count": 8, "id": "18a9b71c-5871-4ce2-a202-b48ad04e8d38", "metadata": {}, "outputs": [ @@ -462,12 +496,7 @@ "output_type": "stream", "text": [ "Copying gs://gcp-public-data-landsat/index.csv.gz...\n", - "==> NOTE: You are downloading one or more large file(s), which would \n", - "run significantly faster if you enabled sliced object downloads. This\n", - "feature is enabled by default but requires that compiled crcmod be\n", - "installed (see \"gsutil help crcmod\").\n", - "\n", - "/ [1 files][731.9 MiB/731.9 MiB] \n", + "\\ [1 files][731.9 MiB/731.9 MiB] 54.4 MiB/s \n", "Operation completed over 1 objects/731.9 MiB. \n" ] } @@ -486,7 +515,7 @@ }, { "cell_type": "code", - "execution_count": 13, + "execution_count": 9, "id": "2cdaf24c-c4aa-4e80-9236-939e7c982916", "metadata": {}, "outputs": [], @@ -504,7 +533,7 @@ }, { "cell_type": "code", - "execution_count": 14, + "execution_count": 10, "id": "b005876c-f9af-43d6-80c6-f04295413b9b", "metadata": {}, "outputs": [ @@ -513,7 +542,7 @@ "output_type": "stream", "text": [ "total 2.5G\n", - "-rw-r--r-- 1 learner learner 2.5G Feb 14 20:32 index.csv\n" + "-rw-r--r-- 1 learner learner 2.5G May 20 17:42 index.csv\n" ] } ], @@ -531,7 +560,7 @@ }, { "cell_type": "code", - "execution_count": 15, + "execution_count": 11, "id": "ffe969db-d207-44fe-8957-8d129c76ee8f", "metadata": {}, "outputs": [ @@ -542,12 +571,18 @@ "SCENE_ID,PRODUCT_ID,SPACECRAFT_ID,SENSOR_ID,DATE_ACQUIRED,COLLECTION_NUMBER,COLLECTION_CATEGORY,SENSING_TIME,DATA_TYPE,WRS_PATH,WRS_ROW,CLOUD_COVER,NORTH_LAT,SOUTH_LAT,WEST_LON,EAST_LON,TOTAL_SIZE,BASE_URL\n", "LE71800592011134ASN00,LE07_L1TP_180059_20110514_20161209_01_T1,LANDSAT_7,ETM,2011-05-14,01,T1,2011-05-14T08:50:07.5251363Z,L1TP,180,59,74.0,2.39913,0.50961,18.09062,20.2487,181281962,gs://gcp-public-data-landsat/LE07/01/180/059/LE07_L1TP_180059_20110514_20161209_01_T1\n", "LT51360422008226BKT00,LT05_L1GS_136042_20080813_20161030_01_T2,LANDSAT_5,TM,2008-08-13,01,T2,2008-08-13T04:03:49.0450690Z,L1GS,136,42,92.0,26.9495,25.03915,91.40541,93.81099,141994748,gs://gcp-public-data-landsat/LT05/01/136/042/LT05_L1GS_136042_20080813_20161030_01_T2\n", - "LE71760312020339NSG00,LE07_L1TP_176031_20201204_20201230_01_T1,LANDSAT_7,ETM,2020-12-04,01,T1,2020-12-04T07:41:11.6084536Z,L1TP,176,31,3.0,42.75649,40.7935,33.66313,36.65653,188511155,gs://gcp-public-data-landsat/LE07/01/176/031/LE07_L1TP_176031_20201204_20201230_01_T1\n" + "LE71760312020339NSG00,LE07_L1TP_176031_20201204_20201230_01_T1,LANDSAT_7,ETM,2020-12-04,01,T1,2020-12-04T07:41:11.6084536Z,L1TP,176,31,3.0,42.75649,40.7935,33.66313,36.65653,188511155,gs://gcp-public-data-landsat/LE07/01/176/031/LE07_L1TP_176031_20201204_20201230_01_T1\n", + "LE71930302019087NSG00,LE07_L1TP_193030_20190328_20190423_01_T1,LANDSAT_7,ETM,2019-03-28,01,T1,2019-03-28T09:57:50.8833259Z,L1TP,193,30,1.0,44.17399,42.20768,7.88666,10.97109,154211367,gs://gcp-public-data-landsat/LE07/01/193/030/LE07_L1TP_193030_20190328_20190423_01_T1\n", + "LM10420361973161AAA05,LM01_L1TP_042036_19730610_20180428_01_T2,LANDSAT_1,MSS,1973-06-10,01,T2,1973-06-10T17:50:11.5000000Z,L1TP,42,36,11.0,35.7197,33.79417,-116.97493,-114.47118,27823097,gs://gcp-public-data-landsat/LM01/01/042/036/LM01_L1TP_042036_19730610_20180428_01_T2\n", + "LT51780171998140KIS00,LT05_L1TP_178017_19980520_20161224_01_T1,LANDSAT_5,TM,1998-05-20,01,T1,1998-05-20T08:04:57.6290500Z,L1TP,178,17,88.0,62.53031,60.38833,39.29647,44.07323,110078792,gs://gcp-public-data-landsat/LT05/01/178/017/LT05_L1TP_178017_19980520_20161224_01_T1\n", + "LE71150302015314ASN00,LE07_L1GT_115030_20151110_20161017_01_T2,LANDSAT_7,ETM,2015-11-10,01,T2,2015-11-10T02:03:53.8882257Z,L1GT,115,30,94.0,44.16222,42.18812,128.52473,131.58518,199231999,gs://gcp-public-data-landsat/LE07/01/115/030/LE07_L1GT_115030_20151110_20161017_01_T2\n", + "LE71620272018219NPA00,LE07_L1TP_162027_20180807_20180902_01_T1,LANDSAT_7,ETM,2018-08-07,01,T1,2018-08-07T06:51:16.8070952Z,L1TP,162,27,16.0,48.44773,46.45227,57.25936,60.58404,242206283,gs://gcp-public-data-landsat/LE07/01/162/027/LE07_L1TP_162027_20180807_20180902_01_T1\n", + "LC81240642016196LGN02,LC08_L1TP_124064_20160714_20180523_01_T1,LANDSAT_8,OLI_TIRS,2016-07-14,01,T1,2016-07-14T03:12:23.6248870Z,L1TP,124,64,41.35,-4.73633,-6.8367,103.1169,105.17739,916587606,gs://gcp-public-data-landsat/LC08/01/124/064/LC08_L1TP_124064_20160714_20180523_01_T1\n" ] } ], "source": [ - "head --lines=4 data/index.csv" + "head data/index.csv" ] }, { @@ -555,14 +590,13 @@ "id": "588ddd30-af8f-4378-b543-290c4c6f0840", "metadata": {}, "source": [ - "````{admonition} Tip\n", - ":class: tip\n", - "To run the above commands in one step run\n", + "::::{tip}\n", "\n", + "To run the above commands in one step run\n", "```\n", "bash get-index.sh\n", "```\n", - "````" + "::::" ] }, { @@ -570,10 +604,12 @@ "id": "f98c38de-87fa-4e66-9d4b-186fbf81b3b2", "metadata": {}, "source": [ - "````{admonition} Break (Optional)\n", + ":::{admonition} Break (Optional)\n", + ":class: exercise\n", "\n", "Now our virtual machine instance is ready and we can access the code and data. Now is a great time to take a short break.\n", - "````" + "\n", + ":::" ] }, { @@ -583,12 +619,12 @@ "source": [ "## Getting the Data\n", "\n", - "We can see the data is well formed and what we expect. We will now use this data to download data related to a specific point and for the Landsat 8. The following script does a simple filter." + "We can see the data is well formed and what we expect. We will now use this data to download data related to a specific point and limit it to only Landsat 8. The following script does a simple filter." ] }, { "cell_type": "code", - "execution_count": 16, + "execution_count": 12, "id": "c5e300c3-e1f3-4cd4-9679-77725e61c4db", "metadata": {}, "outputs": [ @@ -606,6 +642,7 @@ "config=json.load(open(\"search.json\"))\n", "lat,lon=config['lat'],config['lon']\n", "landsat=config['landsat']\n", + "limit=config['limit']\n", "\n", "reader=csv.reader(sys.stdin)\n", "header=next(reader) # skip header\n", @@ -614,7 +651,8 @@ " west,east=float(WEST_LON),float(EAST_LON)\n", " north,south=float(NORTH_LAT),float(SOUTH_LAT)\n", " if SPACECRAFT_ID==landsat and north >= lat and south <= lat and west <= lon and east >= lon:\n", - " print(BASE_URL) # output BASE_URL\n" + " print(BASE_URL) # output BASE_URL\n", + " limit and sys.exit(0) # limit results\n" ] } ], @@ -632,7 +670,7 @@ }, { "cell_type": "code", - "execution_count": 17, + "execution_count": 13, "id": "c9872510-4265-4b0e-aeb5-5a829ff69b24", "metadata": {}, "outputs": [ @@ -643,7 +681,8 @@ "{\n", " \"lat\": 38.899313,\n", " \"lon\": -92.464562,\n", - " \"landsat\": \"LANDSAT_8\"\n", + " \"landsat\": \"LANDSAT_8\",\n", + " \"limit\": true\n", "}\n" ] } @@ -657,12 +696,12 @@ "id": "cbb27235-6bc4-4eb6-b668-5c30427a28b8", "metadata": {}, "source": [ - "Now lets test this on a subset of the data." + "Now lets test this on a subset of the data (note the 'limit' option)." ] }, { "cell_type": "code", - "execution_count": 18, + "execution_count": 14, "id": "6912a9ec-0f9b-4500-ba20-d4280592b323", "metadata": {}, "outputs": [ @@ -670,15 +709,12 @@ "name": "stdout", "output_type": "stream", "text": [ - "gs://gcp-public-data-landsat/LC08/01/025/033/LC08_L1TP_025033_20201007_20201016_01_T1\n", - "gs://gcp-public-data-landsat/LC08/01/025/033/LC08_L1TP_025033_20210519_20210528_01_T1\n", - "gs://gcp-public-data-landsat/LC08/01/025/033/LC08_L1TP_025033_20200601_20200608_01_T1\n", - "gs://gcp-public-data-landsat/LC08/01/025/033/LC08_L1GT_025033_20150111_20170302_01_T2\n" + "gs://gcp-public-data-landsat/LC08/01/025/033/LC08_L1TP_025033_20201007_20201016_01_T1\n" ] } ], "source": [ - "head --lines=200000 data/index.csv | python3 search.py" + "python3 search.py < data/index.csv" ] }, { @@ -691,7 +727,7 @@ }, { "cell_type": "code", - "execution_count": 19, + "execution_count": 15, "id": "3572c518-df83-4906-bfa6-a37bde2a5063", "metadata": {}, "outputs": [ @@ -721,16 +757,18 @@ "id": "a02100bd-f8c5-42dd-975e-a9eb5369bc81", "metadata": {}, "source": [ - "Get the first 2 datasets" + "Get the first dataset (limit option)" ] }, { "cell_type": "code", - "execution_count": 20, + "execution_count": 16, "id": "cccec3e1-0dcd-4e3b-a059-a884f5219b66", "metadata": { "scrolled": true, - "tags": [] + "tags": [ + "hide-output" + ] }, "outputs": [ { @@ -739,53 +777,26 @@ "text": [ "+++ gs://gcp-public-data-landsat/LC08/01/025/033/LC08_L1TP_025033_20201007_20201016_01_T1\n", "Copying gs://gcp-public-data-landsat/LC08/01/025/033/LC08_L1TP_025033_20201007_20201016_01_T1/LC08_L1TP_025033_20201007_20201016_01_T1_ANG.txt...\n", - "Copying gs://gcp-public-data-landsat/LC08/01/025/033/LC08_L1TP_025033_20201007_20201016_01_T1/LC08_L1TP_025033_20201007_20201016_01_T1_B1.TIF...\n", - "Copying gs://gcp-public-data-landsat/LC08/01/025/033/LC08_L1TP_025033_20201007_20201016_01_T1/LC08_L1TP_025033_20201007_20201016_01_T1_B11.TIF...\n", "Copying gs://gcp-public-data-landsat/LC08/01/025/033/LC08_L1TP_025033_20201007_20201016_01_T1/LC08_L1TP_025033_20201007_20201016_01_T1_B10.TIF...\n", - "Copying gs://gcp-public-data-landsat/LC08/01/025/033/LC08_L1TP_025033_20201007_20201016_01_T1/LC08_L1TP_025033_20201007_20201016_01_T1_B2.TIF...\n", + "Copying gs://gcp-public-data-landsat/LC08/01/025/033/LC08_L1TP_025033_20201007_20201016_01_T1/LC08_L1TP_025033_20201007_20201016_01_T1_B1.TIF...\n", "Copying gs://gcp-public-data-landsat/LC08/01/025/033/LC08_L1TP_025033_20201007_20201016_01_T1/LC08_L1TP_025033_20201007_20201016_01_T1_B3.TIF...\n", + "Copying gs://gcp-public-data-landsat/LC08/01/025/033/LC08_L1TP_025033_20201007_20201016_01_T1/LC08_L1TP_025033_20201007_20201016_01_T1_B4.TIF...\n", + "Copying gs://gcp-public-data-landsat/LC08/01/025/033/LC08_L1TP_025033_20201007_20201016_01_T1/LC08_L1TP_025033_20201007_20201016_01_T1_B11.TIF...\n", "Copying gs://gcp-public-data-landsat/LC08/01/025/033/LC08_L1TP_025033_20201007_20201016_01_T1/LC08_L1TP_025033_20201007_20201016_01_T1_B5.TIF...\n", "Copying gs://gcp-public-data-landsat/LC08/01/025/033/LC08_L1TP_025033_20201007_20201016_01_T1/LC08_L1TP_025033_20201007_20201016_01_T1_B6.TIF...\n", - "Copying gs://gcp-public-data-landsat/LC08/01/025/033/LC08_L1TP_025033_20201007_20201016_01_T1/LC08_L1TP_025033_20201007_20201016_01_T1_B4.TIF...\n", + "Copying gs://gcp-public-data-landsat/LC08/01/025/033/LC08_L1TP_025033_20201007_20201016_01_T1/LC08_L1TP_025033_20201007_20201016_01_T1_B2.TIF...\n", "Copying gs://gcp-public-data-landsat/LC08/01/025/033/LC08_L1TP_025033_20201007_20201016_01_T1/LC08_L1TP_025033_20201007_20201016_01_T1_B7.TIF...\n", "Copying gs://gcp-public-data-landsat/LC08/01/025/033/LC08_L1TP_025033_20201007_20201016_01_T1/LC08_L1TP_025033_20201007_20201016_01_T1_B8.TIF...\n", - "==> NOTE: You are downloading one or more large file(s), which would \n", - "run significantly faster if you enabled sliced object downloads. This\n", - "feature is enabled by default but requires that compiled crcmod be\n", - "installed (see \"gsutil help crcmod\").\n", - "\n", "Copying gs://gcp-public-data-landsat/LC08/01/025/033/LC08_L1TP_025033_20201007_20201016_01_T1/LC08_L1TP_025033_20201007_20201016_01_T1_B9.TIF...\n", "Copying gs://gcp-public-data-landsat/LC08/01/025/033/LC08_L1TP_025033_20201007_20201016_01_T1/LC08_L1TP_025033_20201007_20201016_01_T1_BQA.TIF...\n", "Copying gs://gcp-public-data-landsat/LC08/01/025/033/LC08_L1TP_025033_20201007_20201016_01_T1/LC08_L1TP_025033_20201007_20201016_01_T1_MTL.txt...\n", - "\\ [14/14 files][952.9 MiB/952.9 MiB] 100% Done \n", - "Operation completed over 14 objects/952.9 MiB. \n", - "+++ gs://gcp-public-data-landsat/LC08/01/025/033/LC08_L1TP_025033_20210519_20210528_01_T1\n", - "Copying gs://gcp-public-data-landsat/LC08/01/025/033/LC08_L1TP_025033_20210519_20210528_01_T1/LC08_L1TP_025033_20210519_20210528_01_T1_B1.TIF...\n", - "Copying gs://gcp-public-data-landsat/LC08/01/025/033/LC08_L1TP_025033_20210519_20210528_01_T1/LC08_L1TP_025033_20210519_20210528_01_T1_B10.TIF...\n", - "Copying gs://gcp-public-data-landsat/LC08/01/025/033/LC08_L1TP_025033_20210519_20210528_01_T1/LC08_L1TP_025033_20210519_20210528_01_T1_B4.TIF...\n", - "Copying gs://gcp-public-data-landsat/LC08/01/025/033/LC08_L1TP_025033_20210519_20210528_01_T1/LC08_L1TP_025033_20210519_20210528_01_T1_B3.TIF...\n", - "Copying gs://gcp-public-data-landsat/LC08/01/025/033/LC08_L1TP_025033_20210519_20210528_01_T1/LC08_L1TP_025033_20210519_20210528_01_T1_ANG.txt...\n", - "Copying gs://gcp-public-data-landsat/LC08/01/025/033/LC08_L1TP_025033_20210519_20210528_01_T1/LC08_L1TP_025033_20210519_20210528_01_T1_B11.TIF...\n", - "Copying gs://gcp-public-data-landsat/LC08/01/025/033/LC08_L1TP_025033_20210519_20210528_01_T1/LC08_L1TP_025033_20210519_20210528_01_T1_B5.TIF...\n", - "Copying gs://gcp-public-data-landsat/LC08/01/025/033/LC08_L1TP_025033_20210519_20210528_01_T1/LC08_L1TP_025033_20210519_20210528_01_T1_B6.TIF...\n", - "Copying gs://gcp-public-data-landsat/LC08/01/025/033/LC08_L1TP_025033_20210519_20210528_01_T1/LC08_L1TP_025033_20210519_20210528_01_T1_B7.TIF...\n", - "Copying gs://gcp-public-data-landsat/LC08/01/025/033/LC08_L1TP_025033_20210519_20210528_01_T1/LC08_L1TP_025033_20210519_20210528_01_T1_B2.TIF...\n", - "Copying gs://gcp-public-data-landsat/LC08/01/025/033/LC08_L1TP_025033_20210519_20210528_01_T1/LC08_L1TP_025033_20210519_20210528_01_T1_B8.TIF...\n", - "==> NOTE: You are downloading one or more large file(s), which would \n", - "run significantly faster if you enabled sliced object downloads. This\n", - "feature is enabled by default but requires that compiled crcmod be\n", - "installed (see \"gsutil help crcmod\").\n", - "\n", - "Copying gs://gcp-public-data-landsat/LC08/01/025/033/LC08_L1TP_025033_20210519_20210528_01_T1/LC08_L1TP_025033_20210519_20210528_01_T1_B9.TIF...\n", - "Copying gs://gcp-public-data-landsat/LC08/01/025/033/LC08_L1TP_025033_20210519_20210528_01_T1/LC08_L1TP_025033_20210519_20210528_01_T1_BQA.TIF...\n", - "Copying gs://gcp-public-data-landsat/LC08/01/025/033/LC08_L1TP_025033_20210519_20210528_01_T1/LC08_L1TP_025033_20210519_20210528_01_T1_MTL.txt...\n", - "| [14/14 files][ 1.0 GiB/ 1.0 GiB] 100% Done 37.1 MiB/s ETA 00:00:00 \n", - "Operation completed over 14 objects/1.0 GiB. \n" + "| [14/14 files][952.9 MiB/952.9 MiB] 100% Done 18.8 MiB/s ETA 00:00:00 \n", + "Operation completed over 14 objects/952.9 MiB. \n" ] } ], "source": [ - "head --lines=200000 data/index.csv | python3 search.py | head --lines=2 | bash download.sh" + "python3 search.py < data/index.csv | bash download.sh" ] }, { @@ -798,7 +809,7 @@ }, { "cell_type": "code", - "execution_count": 21, + "execution_count": 17, "id": "a37c1567-14b5-4dc7-bc27-d1b84411fce1", "metadata": {}, "outputs": [ @@ -806,10 +817,9 @@ "name": "stdout", "output_type": "stream", "text": [ - "total 2564796\n", - "drwxr-xr-x 2 learner learner 4096 Feb 14 20:33 \u001b[0m\u001b[01;34mLC08_L1TP_025033_20201007_20201016_01_T1\u001b[0m\n", - "drwxr-xr-x 2 learner learner 4096 Feb 14 20:33 \u001b[01;34mLC08_L1TP_025033_20210519_20210528_01_T1\u001b[0m\n", - "-rw-r--r-- 1 learner learner 2626336574 Feb 14 20:32 index.csv\n" + "total 2564792\n", + "drwxr-xr-x 2 learner learner 4096 May 20 17:42 \u001b[0m\u001b[01;34mLC08_L1TP_025033_20201007_20201016_01_T1\u001b[0m\n", + "-rw-r--r-- 1 learner learner 2626336574 May 20 17:42 index.csv\n" ] } ], @@ -822,14 +832,15 @@ "id": "0ca2e2ff-8276-4092-8a9e-ca754db5078e", "metadata": {}, "source": [ - "````{admonition} Tip\n", - ":class: tip\n", + "::::{tip}\n", + "\n", "To run the above analysis in one step run\n", "\n", "```\n", "bash get-data.sh\n", "```\n", - "````" + "\n", + "::::" ] }, { @@ -844,7 +855,7 @@ }, { "cell_type": "code", - "execution_count": 22, + "execution_count": 18, "id": "0c027e92-ae6f-4152-b8d6-5a70172de3e2", "metadata": {}, "outputs": [ @@ -888,7 +899,7 @@ }, { "cell_type": "code", - "execution_count": 25, + "execution_count": 19, "id": "0d6f55c9-3e2c-47d7-b4d0-b984fedc110e", "metadata": {}, "outputs": [ @@ -906,7 +917,7 @@ }, { "cell_type": "code", - "execution_count": 23, + "execution_count": 20, "id": "77999a80-7cfd-46d3-86a6-05d199f7e66d", "metadata": {}, "outputs": [ @@ -915,7 +926,7 @@ "output_type": "stream", "text": [ "Traceback (most recent call last):\n", - " File \"process_sat.py\", line 3, in \n", + " File \"/home/learner/CLASS-Examples/landsat/process_sat.py\", line 3, in \n", " import rasterio\n", "ModuleNotFoundError: No module named 'rasterio'\n" ] @@ -931,17 +942,18 @@ "id": "52ada3ec-26cd-4e8b-94c2-8ed33e98be6e", "metadata": {}, "source": [ - "Oops, let's install the library (note: the output will be slightly different due to how this Lesson is built)\n", - "and while we are at it we also need to create the output directory." + "Oops, let's install the library (note: the output will be slightly different due to how this Lesson is built)." ] }, { "cell_type": "code", - "execution_count": 24, + "execution_count": 21, "id": "b5a2b29b-9c1d-4376-a3eb-4a3dc4bac160", "metadata": { "scrolled": true, - "tags": [] + "tags": [ + "hide-output" + ] }, "outputs": [ { @@ -949,455 +961,453 @@ "output_type": "stream", "text": [ "Reading package lists... Done\n", - "Building dependency tree \n", + "Building dependency tree... Done\n", "Reading state information... Done\n", "The following additional packages will be installed:\n", - " fontconfig-config fonts-dejavu-core gdal-data libaec0 libarmadillo9\n", - " libarpack2 libblas3 libcharls2 libdap25 libdapclient6v5 libdapserver7v5\n", - " libepsilon1 libfontconfig1 libfreexl1 libfyba0 libgdal20 libgeos-3.7.1\n", - " libgeos-c1v5 libgeotiff2 libgfortran5 libgif7 libhdf4-0-alt libhdf5-103\n", - " libicu63 libjbig0 libjpeg62-turbo libkmlbase1 libkmlconvenience1 libkmldom1\n", - " libkmlengine1 libkmlregionator1 libkmlxsd1 liblapack3 liblcms2-2 libltdl7\n", - " libmariadb3 libminizip1 libnetcdf13 libnspr4 libnss3 libodbc1 libogdi3.2\n", - " libopenjp2-7 libpoppler82 libpq5 libproj13 libqhull7 libspatialite7\n", - " libsuperlu5 libsz2 libtiff5 liburiparser1 libwebp6 libxerces-c3.2 libxml2\n", - " libxslt1.1 mariadb-common mysql-common odbcinst odbcinst1debian2\n", - " poppler-data proj-bin proj-data python3-affine python3-attr python3-bs4\n", - " python3-chardet python3-click python3-click-plugins python3-cligj\n", - " python3-colorama python3-html5lib python3-lxml python3-numpy\n", - " python3-pyparsing python3-snuggs python3-soupsieve python3-webencodings\n", + " gdal-data libaec0 libaom0 libarmadillo10 libarpack2 libblas3 libcfitsio9\n", + " libcharls2 libdap27 libdapclient6v5 libdav1d4 libde265-0 libepsilon1\n", + " libfreexl1 libfyba0 libgdal28 libgeos-3.9.0 libgeos-c1v5 libgeotiff5\n", + " libgfortran5 libgif7 libhdf4-0-alt libhdf5-103-1 libhdf5-hl-100 libheif1\n", + " libicu67 libkmlbase1 libkmldom1 libkmlengine1 liblapack3 liblcms2-2 libltdl7\n", + " libmariadb3 libminizip1 libnetcdf18 libnspr4 libnss3 libnuma1 libodbc1\n", + " libogdi4.1 libopenjp2-7 libpoppler102 libpq5 libproj19 libqhull8.0\n", + " librttopo1 libspatialite7 libsuperlu5 libsz2 liburiparser1 libx265-192\n", + " libxerces-c3.2 libxml2 libxslt1.1 mariadb-common mysql-common odbcinst\n", + " odbcinst1debian2 poppler-data proj-bin proj-data python3-affine python3-attr\n", + " python3-bs4 python3-certifi python3-chardet python3-click\n", + " python3-click-plugins python3-cligj python3-colorama python3-html5lib\n", + " python3-lxml python3-numpy python3-pyparsing python3-six python3-snuggs\n", + " python3-soupsieve python3-webencodings\n", "Suggested packages:\n", " geotiff-bin gdal-bin libgeotiff-epsg libhdf4-doc libhdf4-alt-dev hdf4-tools\n", " liblcms2-utils libmyodbc odbc-postgresql tdsodbc unixodbc-bin ogdi-bin\n", " poppler-utils ghostscript fonts-japanese-mincho | fonts-ipafont-mincho\n", " fonts-japanese-gothic | fonts-ipafont-gothic fonts-arphic-ukai\n", " fonts-arphic-uming fonts-nanum python-attr-doc python3-genshi\n", - " python3-lxml-dbg python-lxml-doc gfortran python-numpy-doc python3-pytest\n", - " python3-numpy-dbg python-pyparsing-doc\n", + " python3-lxml-dbg python-lxml-doc gfortran python-numpy-doc python3-numpy-dbg\n", + " python3-pytest python-pyparsing-doc\n", "The following NEW packages will be installed:\n", - " fontconfig-config fonts-dejavu-core gdal-data libaec0 libarmadillo9\n", - " libarpack2 libblas3 libcharls2 libdap25 libdapclient6v5 libdapserver7v5\n", - " libepsilon1 libfontconfig1 libfreexl1 libfyba0 libgdal20 libgeos-3.7.1\n", - " libgeos-c1v5 libgeotiff2 libgfortran5 libgif7 libhdf4-0-alt libhdf5-103\n", - " libicu63 libjbig0 libjpeg62-turbo libkmlbase1 libkmlconvenience1 libkmldom1\n", - " libkmlengine1 libkmlregionator1 libkmlxsd1 liblapack3 liblcms2-2 libltdl7\n", - " libmariadb3 libminizip1 libnetcdf13 libnspr4 libnss3 libodbc1 libogdi3.2\n", - " libopenjp2-7 libpoppler82 libpq5 libproj13 libqhull7 libspatialite7\n", - " libsuperlu5 libsz2 libtiff5 liburiparser1 libwebp6 libxerces-c3.2 libxml2\n", - " libxslt1.1 mariadb-common mysql-common odbcinst odbcinst1debian2\n", - " poppler-data proj-bin proj-data python3-affine python3-attr python3-bs4\n", - " python3-chardet python3-click python3-click-plugins python3-cligj\n", - " python3-colorama python3-html5lib python3-lxml python3-numpy\n", - " python3-pyparsing python3-rasterio python3-snuggs python3-soupsieve\n", - " python3-webencodings\n", - "0 upgraded, 79 newly installed, 0 to remove and 2 not upgraded.\n", - "Need to get 46.8 MB of archives.\n", - "After this operation, 172 MB of additional disk space will be used.\n", - "Get:1 http://security.debian.org/debian-security buster/updates/main amd64 libicu63 amd64 63.1-6+deb10u2 [8300 kB]\n", - "Get:2 http://deb.debian.org/debian buster/main amd64 poppler-data all 0.4.9-2 [1473 kB]\n", - "Get:3 http://deb.debian.org/debian buster/main amd64 fonts-dejavu-core all 2.37-1 [1068 kB]\n", - "Get:4 http://deb.debian.org/debian buster/main amd64 fontconfig-config all 2.13.1-2 [280 kB]\n", - "Get:5 http://deb.debian.org/debian buster/main amd64 gdal-data all 2.4.0+dfsg-1 [744 kB]\n", - "Get:6 http://deb.debian.org/debian buster/main amd64 libaec0 amd64 1.0.2-1 [20.1 kB]\n", - "Get:7 http://deb.debian.org/debian buster/main amd64 libgfortran5 amd64 8.3.0-6 [581 kB]\n", - "Get:8 http://deb.debian.org/debian buster/main amd64 libblas3 amd64 3.8.0-2 [148 kB]\n", - "Get:9 http://deb.debian.org/debian buster/main amd64 liblapack3 amd64 3.8.0-2 [2110 kB]\n", - "Get:10 http://deb.debian.org/debian buster/main amd64 libarpack2 amd64 3.7.0-2 [102 kB]\n", - "Get:11 http://deb.debian.org/debian buster/main amd64 libsuperlu5 amd64 5.2.1+dfsg1-4 [161 kB]\n", - "Get:12 http://deb.debian.org/debian buster/main amd64 libarmadillo9 amd64 1:9.200.7+dfsg-1 [88.6 kB]\n", - "Get:13 http://deb.debian.org/debian buster/main amd64 libcharls2 amd64 2.0.0+dfsg-1 [64.3 kB]\n", - "Get:14 http://deb.debian.org/debian buster/main amd64 libxml2 amd64 2.9.4+dfsg1-7+deb10u2 [689 kB]\n", - "Get:15 http://deb.debian.org/debian buster/main amd64 libdap25 amd64 3.20.3-1 [557 kB]\n", - "Get:16 http://deb.debian.org/debian buster/main amd64 libdapclient6v5 amd64 3.20.3-1 [202 kB]\n", - "Get:17 http://deb.debian.org/debian buster/main amd64 libdapserver7v5 amd64 3.20.3-1 [131 kB]\n", - "Get:18 http://deb.debian.org/debian buster/main amd64 libepsilon1 amd64 0.9.2+dfsg-4 [42.0 kB]\n", - "Get:19 http://deb.debian.org/debian buster/main amd64 libfontconfig1 amd64 2.13.1-2 [346 kB]\n", - "Get:20 http://deb.debian.org/debian buster/main amd64 libfreexl1 amd64 1.0.5-3 [34.1 kB]\n", - "Get:21 http://deb.debian.org/debian buster/main amd64 libfyba0 amd64 4.1.1-6 [114 kB]\n", - "Get:22 http://security.debian.org/debian-security buster/updates/main amd64 libtiff5 amd64 4.1.0+git191117-2~deb10u3 [271 kB]\n", - "Get:23 http://deb.debian.org/debian buster/main amd64 libgeos-3.7.1 amd64 3.7.1-1 [735 kB]\n", - "Get:24 http://security.debian.org/debian-security buster/updates/main amd64 liburiparser1 amd64 0.9.1-1+deb10u1 [48.1 kB]\n", - "Get:25 http://deb.debian.org/debian buster/main amd64 libgeos-c1v5 amd64 3.7.1-1 [299 kB]\n", - "Get:26 http://security.debian.org/debian-security buster/updates/main amd64 libnss3 amd64 2:3.42.1-1+deb10u5 [1160 kB]\n", - "Get:27 http://deb.debian.org/debian buster/main amd64 proj-data all 5.2.0-1 [6986 kB]\n", - "Get:28 http://security.debian.org/debian-security buster/updates/main amd64 libpq5 amd64 11.14-0+deb10u1 [171 kB]\n", - "Get:29 http://security.debian.org/debian-security buster/updates/main amd64 python3-lxml amd64 4.3.2-1+deb10u4 [1163 kB]\n", - "Get:30 http://deb.debian.org/debian buster/main amd64 libproj13 amd64 5.2.0-1 [225 kB]\n", - "Get:31 http://deb.debian.org/debian buster/main amd64 libjbig0 amd64 2.1-3.1+b2 [31.0 kB]\n", - "Get:32 http://deb.debian.org/debian buster/main amd64 libjpeg62-turbo amd64 1:1.5.2-2+deb10u1 [133 kB]\n", - "Get:33 http://deb.debian.org/debian buster/main amd64 libwebp6 amd64 0.6.1-2+deb10u1 [261 kB]\n", - "Get:34 http://deb.debian.org/debian buster/main amd64 libgeotiff2 amd64 1.4.3-1 [72.0 kB]\n", - "Get:35 http://deb.debian.org/debian buster/main amd64 libgif7 amd64 5.1.4-3 [43.3 kB]\n", - "Get:36 http://deb.debian.org/debian buster/main amd64 libhdf4-0-alt amd64 4.2.13-4 [277 kB]\n", - "Get:37 http://deb.debian.org/debian buster/main amd64 libsz2 amd64 1.0.2-1 [6676 B]\n", - "Get:38 http://deb.debian.org/debian buster/main amd64 libhdf5-103 amd64 1.10.4+repack-10 [1325 kB]\n", - "Get:39 http://deb.debian.org/debian buster/main amd64 libminizip1 amd64 1.1-8+b1 [20.4 kB]\n", - "Get:40 http://deb.debian.org/debian buster/main amd64 libkmlbase1 amd64 1.3.0-7 [48.9 kB]\n", - "Get:41 http://deb.debian.org/debian buster/main amd64 libkmldom1 amd64 1.3.0-7 [172 kB]\n", - "Get:42 http://deb.debian.org/debian buster/main amd64 libkmlengine1 amd64 1.3.0-7 [78.4 kB]\n", - "Get:43 http://deb.debian.org/debian buster/main amd64 libkmlconvenience1 amd64 1.3.0-7 [50.1 kB]\n", - "Get:44 http://deb.debian.org/debian buster/main amd64 libkmlregionator1 amd64 1.3.0-7 [25.1 kB]\n", - "Get:45 http://deb.debian.org/debian buster/main amd64 libkmlxsd1 amd64 1.3.0-7 [34.6 kB]\n", - "Get:46 http://deb.debian.org/debian buster/main amd64 mysql-common all 5.8+1.0.5 [7324 B]\n", - "Get:47 http://deb.debian.org/debian buster/main amd64 mariadb-common all 1:10.3.31-0+deb10u1 [32.7 kB]\n", - "Get:48 http://deb.debian.org/debian buster/main amd64 libmariadb3 amd64 1:10.3.31-0+deb10u1 [173 kB]\n", - "Get:49 http://deb.debian.org/debian buster/main amd64 libnetcdf13 amd64 1:4.6.2-1 [403 kB]\n", - "Get:50 http://deb.debian.org/debian buster/main amd64 libltdl7 amd64 2.4.6-9 [390 kB]\n", - "Get:51 http://deb.debian.org/debian buster/main amd64 libodbc1 amd64 2.3.6-0.1 [223 kB]\n", - "Get:52 http://deb.debian.org/debian buster/main amd64 libogdi3.2 amd64 3.2.1+ds-4 [241 kB]\n", - "Get:53 http://deb.debian.org/debian buster/main amd64 libopenjp2-7 amd64 2.3.0-2+deb10u2 [158 kB]\n", - "Get:54 http://deb.debian.org/debian buster/main amd64 liblcms2-2 amd64 2.9-3 [145 kB]\n", - "Get:55 http://deb.debian.org/debian buster/main amd64 libnspr4 amd64 2:4.20-1 [112 kB]\n", - "Get:56 http://deb.debian.org/debian buster/main amd64 libpoppler82 amd64 0.71.0-5 [1507 kB]\n", - "Get:57 http://deb.debian.org/debian buster/main amd64 libqhull7 amd64 2015.2-4 [196 kB]\n", - "Get:58 http://deb.debian.org/debian buster/main amd64 libspatialite7 amd64 4.3.0a-5+b2 [1256 kB]\n", - "Get:59 http://deb.debian.org/debian buster/main amd64 libxerces-c3.2 amd64 3.2.2+debian-1+deb10u1 [886 kB]\n", - "Get:60 http://deb.debian.org/debian buster/main amd64 odbcinst amd64 2.3.6-0.1 [48.4 kB]\n", - "Get:61 http://deb.debian.org/debian buster/main amd64 odbcinst1debian2 amd64 2.3.6-0.1 [78.1 kB]\n", - "Get:62 http://deb.debian.org/debian buster/main amd64 libgdal20 amd64 2.4.0+dfsg-1+b1 [6171 kB]\n", - "Get:63 http://deb.debian.org/debian buster/main amd64 libxslt1.1 amd64 1.1.32-2.2~deb10u1 [237 kB]\n", - "Get:64 http://deb.debian.org/debian buster/main amd64 proj-bin amd64 5.2.0-1 [105 kB]\n", - "Get:65 http://deb.debian.org/debian buster/main amd64 python3-affine all 2.2.2-1 [14.5 kB]\n", - "Get:66 http://deb.debian.org/debian buster/main amd64 python3-attr all 18.2.0-1 [37.3 kB]\n", - "Get:67 http://deb.debian.org/debian buster/main amd64 python3-soupsieve all 1.8+dfsg-1 [27.6 kB]\n", - "Get:68 http://deb.debian.org/debian buster/main amd64 python3-bs4 all 4.7.1-1 [94.1 kB]\n", - "Get:69 http://deb.debian.org/debian buster/main amd64 python3-chardet all 3.0.4-3 [80.5 kB]\n", - "Get:70 http://deb.debian.org/debian buster/main amd64 python3-colorama all 0.3.7-1 [18.1 kB]\n", - "Get:71 http://deb.debian.org/debian buster/main amd64 python3-click all 7.0-1 [73.6 kB]\n", - "Get:72 http://deb.debian.org/debian buster/main amd64 python3-click-plugins all 1.0.4-1 [7540 B]\n", - "Get:73 http://deb.debian.org/debian buster/main amd64 python3-cligj all 0.5.0-1 [8212 B]\n", - "Get:74 http://deb.debian.org/debian buster/main amd64 python3-webencodings all 0.5.1-1 [10.9 kB]\n", - "Get:75 http://deb.debian.org/debian buster/main amd64 python3-html5lib all 1.0.1-1 [89.5 kB]\n", - "Get:76 http://deb.debian.org/debian buster/main amd64 python3-numpy amd64 1:1.16.2-1 [2119 kB]\n", - "Get:77 http://deb.debian.org/debian buster/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [89.6 kB]\n", - "Get:78 http://deb.debian.org/debian buster/main amd64 python3-snuggs all 1.4.3-1 [7228 B]\n", - "Get:79 http://deb.debian.org/debian buster/main amd64 python3-rasterio amd64 1.0.21-1 [818 kB]\n", - "Fetched 46.8 MB in 1s (79.1 MB/s) \n", + " gdal-data libaec0 libaom0 libarmadillo10 libarpack2 libblas3 libcfitsio9\n", + " libcharls2 libdap27 libdapclient6v5 libdav1d4 libde265-0 libepsilon1\n", + " libfreexl1 libfyba0 libgdal28 libgeos-3.9.0 libgeos-c1v5 libgeotiff5\n", + " libgfortran5 libgif7 libhdf4-0-alt libhdf5-103-1 libhdf5-hl-100 libheif1\n", + " libicu67 libkmlbase1 libkmldom1 libkmlengine1 liblapack3 liblcms2-2 libltdl7\n", + " libmariadb3 libminizip1 libnetcdf18 libnspr4 libnss3 libnuma1 libodbc1\n", + " libogdi4.1 libopenjp2-7 libpoppler102 libpq5 libproj19 libqhull8.0\n", + " librttopo1 libspatialite7 libsuperlu5 libsz2 liburiparser1 libx265-192\n", + " libxerces-c3.2 libxml2 libxslt1.1 mariadb-common mysql-common odbcinst\n", + " odbcinst1debian2 poppler-data proj-bin proj-data python3-affine python3-attr\n", + " python3-bs4 python3-certifi python3-chardet python3-click\n", + " python3-click-plugins python3-cligj python3-colorama python3-html5lib\n", + " python3-lxml python3-numpy python3-pyparsing python3-rasterio python3-six\n", + " python3-snuggs python3-soupsieve python3-webencodings\n", + "0 upgraded, 79 newly installed, 0 to remove and 11 not upgraded.\n", + "Need to get 53.2 MB of archives.\n", + "After this operation, 211 MB of additional disk space will be used.\n", + "Get:1 http://security.debian.org/debian-security bullseye-security/main amd64 libpq5 amd64 13.7-0+deb11u1 [180 kB]\n", + "Get:2 http://deb.debian.org/debian bullseye/main amd64 poppler-data all 0.4.10-1 [1602 kB]\n", + "Get:3 http://deb.debian.org/debian bullseye/main amd64 gdal-data all 3.2.2+dfsg-2+deb11u1 [462 kB]\n", + "Get:4 http://deb.debian.org/debian bullseye/main amd64 libaec0 amd64 1.0.4-1 [20.3 kB]\n", + "Get:5 http://deb.debian.org/debian bullseye/main amd64 libaom0 amd64 1.0.0.errata1-3 [1158 kB]\n", + "Get:6 http://deb.debian.org/debian bullseye/main amd64 libblas3 amd64 3.9.0-3 [153 kB]\n", + "Get:7 http://deb.debian.org/debian bullseye/main amd64 libgfortran5 amd64 10.2.1-6 [727 kB]\n", + "Get:8 http://deb.debian.org/debian bullseye/main amd64 liblapack3 amd64 3.9.0-3 [2166 kB]\n", + "Get:9 http://deb.debian.org/debian bullseye/main amd64 libarpack2 amd64 3.8.0-1 [103 kB]\n", + "Get:10 http://deb.debian.org/debian bullseye/main amd64 libsuperlu5 amd64 5.2.2+dfsg1-2 [163 kB]\n", + "Get:11 http://deb.debian.org/debian bullseye/main amd64 libarmadillo10 amd64 1:10.1.2+dfsg-6+b1 [98.8 kB]\n", + "Get:12 http://deb.debian.org/debian bullseye/main amd64 libcfitsio9 amd64 3.490-3 [554 kB]\n", + "Get:13 http://deb.debian.org/debian bullseye/main amd64 libcharls2 amd64 2.2.0+dfsg-2 [79.4 kB]\n", + "Get:14 http://deb.debian.org/debian bullseye/main amd64 libicu67 amd64 67.1-7 [8622 kB]\n", + "Get:15 http://deb.debian.org/debian bullseye/main amd64 libxml2 amd64 2.9.10+dfsg-6.7+deb11u1 [693 kB]\n", + "Get:16 http://deb.debian.org/debian bullseye/main amd64 libdap27 amd64 3.20.7-6 [566 kB]\n", + "Get:17 http://deb.debian.org/debian bullseye/main amd64 libdapclient6v5 amd64 3.20.7-6 [208 kB]\n", + "Get:18 http://deb.debian.org/debian bullseye/main amd64 libdav1d4 amd64 0.7.1-3 [333 kB]\n", + "Get:19 http://deb.debian.org/debian bullseye/main amd64 libde265-0 amd64 1.0.8-1 [242 kB]\n", + "Get:20 http://deb.debian.org/debian bullseye/main amd64 libepsilon1 amd64 0.9.2+dfsg-5 [41.9 kB]\n", + "Get:21 http://deb.debian.org/debian bullseye/main amd64 libfreexl1 amd64 1.0.6-1 [34.2 kB]\n", + "Get:22 http://deb.debian.org/debian bullseye/main amd64 libfyba0 amd64 4.1.1-7 [114 kB]\n", + "Get:23 http://deb.debian.org/debian bullseye/main amd64 libgeos-3.9.0 amd64 3.9.0-1 [921 kB]\n", + "Get:24 http://deb.debian.org/debian bullseye/main amd64 libgeos-c1v5 amd64 3.9.0-1 [377 kB]\n", + "Get:25 http://deb.debian.org/debian bullseye/main amd64 proj-data all 7.2.1-1 [7940 kB]\n", + "Get:26 http://deb.debian.org/debian bullseye/main amd64 libproj19 amd64 7.2.1-1 [1132 kB]\n", + "Get:27 http://deb.debian.org/debian bullseye/main amd64 libgeotiff5 amd64 1.6.0-1 [70.3 kB]\n", + "Get:28 http://deb.debian.org/debian bullseye/main amd64 libgif7 amd64 5.1.9-2 [45.1 kB]\n", + "Get:29 http://deb.debian.org/debian bullseye/main amd64 libhdf4-0-alt amd64 4.2.15-3 [278 kB]\n", + "Get:30 http://deb.debian.org/debian bullseye/main amd64 libsz2 amd64 1.0.4-1 [6760 B]\n", + "Get:31 http://deb.debian.org/debian bullseye/main amd64 libhdf5-103-1 amd64 1.10.6+repack-4+deb11u1 [1189 kB]\n", + "Get:32 http://deb.debian.org/debian bullseye/main amd64 libnuma1 amd64 2.0.12-1+b1 [26.3 kB]\n", + "Get:33 http://deb.debian.org/debian bullseye/main amd64 libx265-192 amd64 3.4-2 [1095 kB]\n", + "Get:34 http://deb.debian.org/debian bullseye/main amd64 libheif1 amd64 1.11.0-1 [191 kB]\n", + "Get:35 http://deb.debian.org/debian bullseye/main amd64 libminizip1 amd64 1.1-8+b1 [20.4 kB]\n", + "Get:36 http://deb.debian.org/debian bullseye/main amd64 liburiparser1 amd64 0.9.4+dfsg-1+deb11u1 [45.9 kB]\n", + "Get:37 http://deb.debian.org/debian bullseye/main amd64 libkmlbase1 amd64 1.3.0-9 [48.7 kB]\n", + "Get:38 http://deb.debian.org/debian bullseye/main amd64 libkmldom1 amd64 1.3.0-9 [156 kB]\n", + "Get:39 http://deb.debian.org/debian bullseye/main amd64 libkmlengine1 amd64 1.3.0-9 [77.7 kB]\n", + "Get:40 http://deb.debian.org/debian bullseye/main amd64 mysql-common all 5.8+1.0.7 [7464 B]\n", + "Get:41 http://deb.debian.org/debian bullseye/main amd64 mariadb-common all 1:10.5.15-0+deb11u1 [36.7 kB]\n", + "Get:42 http://deb.debian.org/debian bullseye/main amd64 libmariadb3 amd64 1:10.5.15-0+deb11u1 [176 kB]\n", + "Get:43 http://deb.debian.org/debian bullseye/main amd64 libhdf5-hl-100 amd64 1.10.6+repack-4+deb11u1 [81.8 kB]\n", + "Get:44 http://deb.debian.org/debian bullseye/main amd64 libnetcdf18 amd64 1:4.7.4-1 [399 kB]\n", + "Get:45 http://deb.debian.org/debian bullseye/main amd64 libltdl7 amd64 2.4.6-15 [391 kB]\n", + "Get:46 http://deb.debian.org/debian bullseye/main amd64 libodbc1 amd64 2.3.6-0.1+b1 [224 kB]\n", + "Get:47 http://deb.debian.org/debian bullseye/main amd64 libogdi4.1 amd64 4.1.0+ds-5 [213 kB]\n", + "Get:48 http://deb.debian.org/debian bullseye/main amd64 libopenjp2-7 amd64 2.4.0-3 [172 kB]\n", + "Get:49 http://deb.debian.org/debian bullseye/main amd64 liblcms2-2 amd64 2.12~rc1-2 [150 kB]\n", + "Get:50 http://deb.debian.org/debian bullseye/main amd64 libnspr4 amd64 2:4.29-1 [112 kB]\n", + "Get:51 http://deb.debian.org/debian bullseye/main amd64 libnss3 amd64 2:3.61-1+deb11u2 [1306 kB]\n", + "Get:52 http://deb.debian.org/debian bullseye/main amd64 libpoppler102 amd64 20.09.0-3.1 [1675 kB]\n", + "Get:53 http://deb.debian.org/debian bullseye/main amd64 libqhull8.0 amd64 2020.2-3 [246 kB]\n", + "Get:54 http://deb.debian.org/debian bullseye/main amd64 librttopo1 amd64 1.1.0-2 [180 kB]\n", + "Get:55 http://deb.debian.org/debian bullseye/main amd64 libspatialite7 amd64 5.0.1-2 [1796 kB]\n", + "Get:56 http://deb.debian.org/debian bullseye/main amd64 libxerces-c3.2 amd64 3.2.3+debian-3 [870 kB]\n", + "Get:57 http://deb.debian.org/debian bullseye/main amd64 odbcinst amd64 2.3.6-0.1+b1 [48.7 kB]\n", + "Get:58 http://deb.debian.org/debian bullseye/main amd64 odbcinst1debian2 amd64 2.3.6-0.1+b1 [78.6 kB]\n", + "Get:59 http://deb.debian.org/debian bullseye/main amd64 libgdal28 amd64 3.2.2+dfsg-2+deb11u1 [7255 kB]\n", + "Get:60 http://deb.debian.org/debian bullseye/main amd64 libxslt1.1 amd64 1.1.34-4 [239 kB]\n", + "Get:61 http://deb.debian.org/debian bullseye/main amd64 proj-bin amd64 7.2.1-1 [189 kB]\n", + "Get:62 http://deb.debian.org/debian bullseye/main amd64 python3-affine all 2.3.0-2 [15.0 kB]\n", + "Get:63 http://deb.debian.org/debian bullseye/main amd64 python3-attr all 20.3.0-1 [52.9 kB]\n", + "Get:64 http://deb.debian.org/debian bullseye/main amd64 python3-soupsieve all 2.2.1-1 [34.7 kB]\n", + "Get:65 http://deb.debian.org/debian bullseye/main amd64 python3-bs4 all 4.9.3-1 [112 kB]\n", + "Get:66 http://deb.debian.org/debian bullseye/main amd64 python3-certifi all 2020.6.20-1 [151 kB]\n", + "Get:67 http://deb.debian.org/debian bullseye/main amd64 python3-chardet all 4.0.0-1 [99.0 kB]\n", + "Get:68 http://deb.debian.org/debian bullseye/main amd64 python3-colorama all 0.4.4-1 [28.5 kB]\n", + "Get:69 http://deb.debian.org/debian bullseye/main amd64 python3-click all 7.1.2-1 [75.7 kB]\n", + "Get:70 http://deb.debian.org/debian bullseye/main amd64 python3-click-plugins all 1.1.1-3 [10.3 kB]\n", + "Get:71 http://deb.debian.org/debian bullseye/main amd64 python3-cligj all 0.7.1-1 [9024 B]\n", + "Get:72 http://deb.debian.org/debian bullseye/main amd64 python3-six all 1.16.0-2 [17.5 kB]\n", + "Get:73 http://deb.debian.org/debian bullseye/main amd64 python3-webencodings all 0.5.1-2 [11.0 kB]\n", + "Get:74 http://deb.debian.org/debian bullseye/main amd64 python3-html5lib all 1.1-3 [93.0 kB]\n", + "Get:75 http://deb.debian.org/debian bullseye/main amd64 python3-lxml amd64 4.6.3+dfsg-0.1+deb11u1 [1093 kB]\n", + "Get:76 http://deb.debian.org/debian bullseye/main amd64 python3-numpy amd64 1:1.19.5-1 [2693 kB]\n", + "Get:77 http://deb.debian.org/debian bullseye/main amd64 python3-pyparsing all 2.4.7-1 [109 kB]\n", + "Get:78 http://deb.debian.org/debian bullseye/main amd64 python3-snuggs all 1.4.7-2 [7780 B]\n", + "Get:79 http://deb.debian.org/debian bullseye/main amd64 python3-rasterio amd64 1.2.0-1 [839 kB]\n", + "Fetched 53.2 MB in 1s (104 MB/s) \n", "Extracting templates from packages: 100%\n", - "Preconfiguring packages ...\n", "Selecting previously unselected package poppler-data.\n", - "(Reading database ... 57852 files and directories currently installed.)\n", - "Preparing to unpack .../00-poppler-data_0.4.9-2_all.deb ...\n", - "Unpacking poppler-data (0.4.9-2) ...\n", - "Selecting previously unselected package fonts-dejavu-core.\n", - "Preparing to unpack .../01-fonts-dejavu-core_2.37-1_all.deb ...\n", - "Unpacking fonts-dejavu-core (2.37-1) ...\n", - "Selecting previously unselected package fontconfig-config.\n", - "Preparing to unpack .../02-fontconfig-config_2.13.1-2_all.deb ...\n", - "Unpacking fontconfig-config (2.13.1-2) ...\n", + "(Reading database ... 63527 files and directories currently installed.)\n", + "Preparing to unpack .../00-poppler-data_0.4.10-1_all.deb ...\n", + "Unpacking poppler-data (0.4.10-1) ...\n", "Selecting previously unselected package gdal-data.\n", - "Preparing to unpack .../03-gdal-data_2.4.0+dfsg-1_all.deb ...\n", - "Unpacking gdal-data (2.4.0+dfsg-1) ...\n", + "Preparing to unpack .../01-gdal-data_3.2.2+dfsg-2+deb11u1_all.deb ...\n", + "Unpacking gdal-data (3.2.2+dfsg-2+deb11u1) ...\n", "Selecting previously unselected package libaec0:amd64.\n", - "Preparing to unpack .../04-libaec0_1.0.2-1_amd64.deb ...\n", - "Unpacking libaec0:amd64 (1.0.2-1) ...\n", - "Selecting previously unselected package libgfortran5:amd64.\n", - "Preparing to unpack .../05-libgfortran5_8.3.0-6_amd64.deb ...\n", - "Unpacking libgfortran5:amd64 (8.3.0-6) ...\n", + "Preparing to unpack .../02-libaec0_1.0.4-1_amd64.deb ...\n", + "Unpacking libaec0:amd64 (1.0.4-1) ...\n", + "Selecting previously unselected package libaom0:amd64.\n", + "Preparing to unpack .../03-libaom0_1.0.0.errata1-3_amd64.deb ...\n", + "Unpacking libaom0:amd64 (1.0.0.errata1-3) ...\n", "Selecting previously unselected package libblas3:amd64.\n", - "Preparing to unpack .../06-libblas3_3.8.0-2_amd64.deb ...\n", - "Unpacking libblas3:amd64 (3.8.0-2) ...\n", + "Preparing to unpack .../04-libblas3_3.9.0-3_amd64.deb ...\n", + "Unpacking libblas3:amd64 (3.9.0-3) ...\n", + "Selecting previously unselected package libgfortran5:amd64.\n", + "Preparing to unpack .../05-libgfortran5_10.2.1-6_amd64.deb ...\n", + "Unpacking libgfortran5:amd64 (10.2.1-6) ...\n", "Selecting previously unselected package liblapack3:amd64.\n", - "Preparing to unpack .../07-liblapack3_3.8.0-2_amd64.deb ...\n", - "Unpacking liblapack3:amd64 (3.8.0-2) ...\n", + "Preparing to unpack .../06-liblapack3_3.9.0-3_amd64.deb ...\n", + "Unpacking liblapack3:amd64 (3.9.0-3) ...\n", "Selecting previously unselected package libarpack2:amd64.\n", - "Preparing to unpack .../08-libarpack2_3.7.0-2_amd64.deb ...\n", - "Unpacking libarpack2:amd64 (3.7.0-2) ...\n", + "Preparing to unpack .../07-libarpack2_3.8.0-1_amd64.deb ...\n", + "Unpacking libarpack2:amd64 (3.8.0-1) ...\n", "Selecting previously unselected package libsuperlu5:amd64.\n", - "Preparing to unpack .../09-libsuperlu5_5.2.1+dfsg1-4_amd64.deb ...\n", - "Unpacking libsuperlu5:amd64 (5.2.1+dfsg1-4) ...\n", - "Selecting previously unselected package libarmadillo9.\n", - "Preparing to unpack .../10-libarmadillo9_1%3a9.200.7+dfsg-1_amd64.deb ...\n", - "Unpacking libarmadillo9 (1:9.200.7+dfsg-1) ...\n", + "Preparing to unpack .../08-libsuperlu5_5.2.2+dfsg1-2_amd64.deb ...\n", + "Unpacking libsuperlu5:amd64 (5.2.2+dfsg1-2) ...\n", + "Selecting previously unselected package libarmadillo10.\n", + "Preparing to unpack .../09-libarmadillo10_1%3a10.1.2+dfsg-6+b1_amd64.deb ...\n", + "Unpacking libarmadillo10 (1:10.1.2+dfsg-6+b1) ...\n", + "Selecting previously unselected package libcfitsio9:amd64.\n", + "Preparing to unpack .../10-libcfitsio9_3.490-3_amd64.deb ...\n", + "Unpacking libcfitsio9:amd64 (3.490-3) ...\n", "Selecting previously unselected package libcharls2:amd64.\n", - "Preparing to unpack .../11-libcharls2_2.0.0+dfsg-1_amd64.deb ...\n", - "Unpacking libcharls2:amd64 (2.0.0+dfsg-1) ...\n", - "Selecting previously unselected package libicu63:amd64.\n", - "Preparing to unpack .../12-libicu63_63.1-6+deb10u2_amd64.deb ...\n", - "Unpacking libicu63:amd64 (63.1-6+deb10u2) ...\n", + "Preparing to unpack .../11-libcharls2_2.2.0+dfsg-2_amd64.deb ...\n", + "Unpacking libcharls2:amd64 (2.2.0+dfsg-2) ...\n", + "Selecting previously unselected package libicu67:amd64.\n", + "Preparing to unpack .../12-libicu67_67.1-7_amd64.deb ...\n", + "Unpacking libicu67:amd64 (67.1-7) ...\n", "Selecting previously unselected package libxml2:amd64.\n", - "Preparing to unpack .../13-libxml2_2.9.4+dfsg1-7+deb10u2_amd64.deb ...\n", - "Unpacking libxml2:amd64 (2.9.4+dfsg1-7+deb10u2) ...\n", - "Selecting previously unselected package libdap25:amd64.\n", - "Preparing to unpack .../14-libdap25_3.20.3-1_amd64.deb ...\n", - "Unpacking libdap25:amd64 (3.20.3-1) ...\n", + "Preparing to unpack .../13-libxml2_2.9.10+dfsg-6.7+deb11u1_amd64.deb ...\n", + "Unpacking libxml2:amd64 (2.9.10+dfsg-6.7+deb11u1) ...\n", + "Selecting previously unselected package libdap27:amd64.\n", + "Preparing to unpack .../14-libdap27_3.20.7-6_amd64.deb ...\n", + "Unpacking libdap27:amd64 (3.20.7-6) ...\n", "Selecting previously unselected package libdapclient6v5:amd64.\n", - "Preparing to unpack .../15-libdapclient6v5_3.20.3-1_amd64.deb ...\n", - "Unpacking libdapclient6v5:amd64 (3.20.3-1) ...\n", - "Selecting previously unselected package libdapserver7v5:amd64.\n", - "Preparing to unpack .../16-libdapserver7v5_3.20.3-1_amd64.deb ...\n", - "Unpacking libdapserver7v5:amd64 (3.20.3-1) ...\n", + "Preparing to unpack .../15-libdapclient6v5_3.20.7-6_amd64.deb ...\n", + "Unpacking libdapclient6v5:amd64 (3.20.7-6) ...\n", + "Selecting previously unselected package libdav1d4:amd64.\n", + "Preparing to unpack .../16-libdav1d4_0.7.1-3_amd64.deb ...\n", + "Unpacking libdav1d4:amd64 (0.7.1-3) ...\n", + "Selecting previously unselected package libde265-0:amd64.\n", + "Preparing to unpack .../17-libde265-0_1.0.8-1_amd64.deb ...\n", + "Unpacking libde265-0:amd64 (1.0.8-1) ...\n", "Selecting previously unselected package libepsilon1:amd64.\n", - "Preparing to unpack .../17-libepsilon1_0.9.2+dfsg-4_amd64.deb ...\n", - "Unpacking libepsilon1:amd64 (0.9.2+dfsg-4) ...\n", - "Selecting previously unselected package libfontconfig1:amd64.\n", - "Preparing to unpack .../18-libfontconfig1_2.13.1-2_amd64.deb ...\n", - "Unpacking libfontconfig1:amd64 (2.13.1-2) ...\n", + "Preparing to unpack .../18-libepsilon1_0.9.2+dfsg-5_amd64.deb ...\n", + "Unpacking libepsilon1:amd64 (0.9.2+dfsg-5) ...\n", "Selecting previously unselected package libfreexl1:amd64.\n", - "Preparing to unpack .../19-libfreexl1_1.0.5-3_amd64.deb ...\n", - "Unpacking libfreexl1:amd64 (1.0.5-3) ...\n", + "Preparing to unpack .../19-libfreexl1_1.0.6-1_amd64.deb ...\n", + "Unpacking libfreexl1:amd64 (1.0.6-1) ...\n", "Selecting previously unselected package libfyba0:amd64.\n", - "Preparing to unpack .../20-libfyba0_4.1.1-6_amd64.deb ...\n", - "Unpacking libfyba0:amd64 (4.1.1-6) ...\n", - "Selecting previously unselected package libgeos-3.7.1:amd64.\n", - "Preparing to unpack .../21-libgeos-3.7.1_3.7.1-1_amd64.deb ...\n", - "Unpacking libgeos-3.7.1:amd64 (3.7.1-1) ...\n", + "Preparing to unpack .../20-libfyba0_4.1.1-7_amd64.deb ...\n", + "Unpacking libfyba0:amd64 (4.1.1-7) ...\n", + "Selecting previously unselected package libgeos-3.9.0:amd64.\n", + "Preparing to unpack .../21-libgeos-3.9.0_3.9.0-1_amd64.deb ...\n", + "Unpacking libgeos-3.9.0:amd64 (3.9.0-1) ...\n", "Selecting previously unselected package libgeos-c1v5:amd64.\n", - "Preparing to unpack .../22-libgeos-c1v5_3.7.1-1_amd64.deb ...\n", - "Unpacking libgeos-c1v5:amd64 (3.7.1-1) ...\n", + "Preparing to unpack .../22-libgeos-c1v5_3.9.0-1_amd64.deb ...\n", + "Unpacking libgeos-c1v5:amd64 (3.9.0-1) ...\n", "Selecting previously unselected package proj-data.\n", - "Preparing to unpack .../23-proj-data_5.2.0-1_all.deb ...\n", - "Unpacking proj-data (5.2.0-1) ...\n", - "Selecting previously unselected package libproj13:amd64.\n", - "Preparing to unpack .../24-libproj13_5.2.0-1_amd64.deb ...\n", - "Unpacking libproj13:amd64 (5.2.0-1) ...\n", - "Selecting previously unselected package libjbig0:amd64.\n", - "Preparing to unpack .../25-libjbig0_2.1-3.1+b2_amd64.deb ...\n", - "Unpacking libjbig0:amd64 (2.1-3.1+b2) ...\n", - "Selecting previously unselected package libjpeg62-turbo:amd64.\n", - "Preparing to unpack .../26-libjpeg62-turbo_1%3a1.5.2-2+deb10u1_amd64.deb ...\n", - "Unpacking libjpeg62-turbo:amd64 (1:1.5.2-2+deb10u1) ...\n", - "Selecting previously unselected package libwebp6:amd64.\n", - "Preparing to unpack .../27-libwebp6_0.6.1-2+deb10u1_amd64.deb ...\n", - "Unpacking libwebp6:amd64 (0.6.1-2+deb10u1) ...\n", - "Selecting previously unselected package libtiff5:amd64.\n", - "Preparing to unpack .../28-libtiff5_4.1.0+git191117-2~deb10u3_amd64.deb ...\n", - "Unpacking libtiff5:amd64 (4.1.0+git191117-2~deb10u3) ...\n", - "Selecting previously unselected package libgeotiff2:amd64.\n", - "Preparing to unpack .../29-libgeotiff2_1.4.3-1_amd64.deb ...\n", - "Unpacking libgeotiff2:amd64 (1.4.3-1) ...\n", + "Preparing to unpack .../23-proj-data_7.2.1-1_all.deb ...\n", + "Unpacking proj-data (7.2.1-1) ...\n", + "Selecting previously unselected package libproj19:amd64.\n", + "Preparing to unpack .../24-libproj19_7.2.1-1_amd64.deb ...\n", + "Unpacking libproj19:amd64 (7.2.1-1) ...\n", + "Selecting previously unselected package libgeotiff5:amd64.\n", + "Preparing to unpack .../25-libgeotiff5_1.6.0-1_amd64.deb ...\n", + "Unpacking libgeotiff5:amd64 (1.6.0-1) ...\n", "Selecting previously unselected package libgif7:amd64.\n", - "Preparing to unpack .../30-libgif7_5.1.4-3_amd64.deb ...\n", - "Unpacking libgif7:amd64 (5.1.4-3) ...\n", + "Preparing to unpack .../26-libgif7_5.1.9-2_amd64.deb ...\n", + "Unpacking libgif7:amd64 (5.1.9-2) ...\n", "Selecting previously unselected package libhdf4-0-alt.\n", - "Preparing to unpack .../31-libhdf4-0-alt_4.2.13-4_amd64.deb ...\n", - "Unpacking libhdf4-0-alt (4.2.13-4) ...\n", + "Preparing to unpack .../27-libhdf4-0-alt_4.2.15-3_amd64.deb ...\n", + "Unpacking libhdf4-0-alt (4.2.15-3) ...\n", "Selecting previously unselected package libsz2:amd64.\n", - "Preparing to unpack .../32-libsz2_1.0.2-1_amd64.deb ...\n", - "Unpacking libsz2:amd64 (1.0.2-1) ...\n", - "Selecting previously unselected package libhdf5-103:amd64.\n", - "Preparing to unpack .../33-libhdf5-103_1.10.4+repack-10_amd64.deb ...\n", - "Unpacking libhdf5-103:amd64 (1.10.4+repack-10) ...\n", + "Preparing to unpack .../28-libsz2_1.0.4-1_amd64.deb ...\n", + "Unpacking libsz2:amd64 (1.0.4-1) ...\n", + "Selecting previously unselected package libhdf5-103-1:amd64.\n", + "Preparing to unpack .../29-libhdf5-103-1_1.10.6+repack-4+deb11u1_amd64.deb ...\n", + "Unpacking libhdf5-103-1:amd64 (1.10.6+repack-4+deb11u1) ...\n", + "Selecting previously unselected package libnuma1:amd64.\n", + "Preparing to unpack .../30-libnuma1_2.0.12-1+b1_amd64.deb ...\n", + "Unpacking libnuma1:amd64 (2.0.12-1+b1) ...\n", + "Selecting previously unselected package libx265-192:amd64.\n", + "Preparing to unpack .../31-libx265-192_3.4-2_amd64.deb ...\n", + "Unpacking libx265-192:amd64 (3.4-2) ...\n", + "Selecting previously unselected package libheif1:amd64.\n", + "Preparing to unpack .../32-libheif1_1.11.0-1_amd64.deb ...\n", + "Unpacking libheif1:amd64 (1.11.0-1) ...\n", "Selecting previously unselected package libminizip1:amd64.\n", - "Preparing to unpack .../34-libminizip1_1.1-8+b1_amd64.deb ...\n", + "Preparing to unpack .../33-libminizip1_1.1-8+b1_amd64.deb ...\n", "Unpacking libminizip1:amd64 (1.1-8+b1) ...\n", "Selecting previously unselected package liburiparser1:amd64.\n", - "Preparing to unpack .../35-liburiparser1_0.9.1-1+deb10u1_amd64.deb ...\n", - "Unpacking liburiparser1:amd64 (0.9.1-1+deb10u1) ...\n", + "Preparing to unpack .../34-liburiparser1_0.9.4+dfsg-1+deb11u1_amd64.deb ...\n", + "Unpacking liburiparser1:amd64 (0.9.4+dfsg-1+deb11u1) ...\n", "Selecting previously unselected package libkmlbase1:amd64.\n", - "Preparing to unpack .../36-libkmlbase1_1.3.0-7_amd64.deb ...\n", - "Unpacking libkmlbase1:amd64 (1.3.0-7) ...\n", + "Preparing to unpack .../35-libkmlbase1_1.3.0-9_amd64.deb ...\n", + "Unpacking libkmlbase1:amd64 (1.3.0-9) ...\n", "Selecting previously unselected package libkmldom1:amd64.\n", - "Preparing to unpack .../37-libkmldom1_1.3.0-7_amd64.deb ...\n", - "Unpacking libkmldom1:amd64 (1.3.0-7) ...\n", + "Preparing to unpack .../36-libkmldom1_1.3.0-9_amd64.deb ...\n", + "Unpacking libkmldom1:amd64 (1.3.0-9) ...\n", "Selecting previously unselected package libkmlengine1:amd64.\n", - "Preparing to unpack .../38-libkmlengine1_1.3.0-7_amd64.deb ...\n", - "Unpacking libkmlengine1:amd64 (1.3.0-7) ...\n", - "Selecting previously unselected package libkmlconvenience1:amd64.\n", - "Preparing to unpack .../39-libkmlconvenience1_1.3.0-7_amd64.deb ...\n", - "Unpacking libkmlconvenience1:amd64 (1.3.0-7) ...\n", - "Selecting previously unselected package libkmlregionator1:amd64.\n", - "Preparing to unpack .../40-libkmlregionator1_1.3.0-7_amd64.deb ...\n", - "Unpacking libkmlregionator1:amd64 (1.3.0-7) ...\n", - "Selecting previously unselected package libkmlxsd1:amd64.\n", - "Preparing to unpack .../41-libkmlxsd1_1.3.0-7_amd64.deb ...\n", - "Unpacking libkmlxsd1:amd64 (1.3.0-7) ...\n", + "Preparing to unpack .../37-libkmlengine1_1.3.0-9_amd64.deb ...\n", + "Unpacking libkmlengine1:amd64 (1.3.0-9) ...\n", "Selecting previously unselected package mysql-common.\n", - "Preparing to unpack .../42-mysql-common_5.8+1.0.5_all.deb ...\n", - "Unpacking mysql-common (5.8+1.0.5) ...\n", + "Preparing to unpack .../38-mysql-common_5.8+1.0.7_all.deb ...\n", + "Unpacking mysql-common (5.8+1.0.7) ...\n", "Selecting previously unselected package mariadb-common.\n", - "Preparing to unpack .../43-mariadb-common_1%3a10.3.31-0+deb10u1_all.deb ...\n", - "Unpacking mariadb-common (1:10.3.31-0+deb10u1) ...\n", + "Preparing to unpack .../39-mariadb-common_1%3a10.5.15-0+deb11u1_all.deb ...\n", + "Unpacking mariadb-common (1:10.5.15-0+deb11u1) ...\n", "Selecting previously unselected package libmariadb3:amd64.\n", - "Preparing to unpack .../44-libmariadb3_1%3a10.3.31-0+deb10u1_amd64.deb ...\n", - "Unpacking libmariadb3:amd64 (1:10.3.31-0+deb10u1) ...\n", - "Selecting previously unselected package libnetcdf13:amd64.\n", - "Preparing to unpack .../45-libnetcdf13_1%3a4.6.2-1_amd64.deb ...\n", - "Unpacking libnetcdf13:amd64 (1:4.6.2-1) ...\n", + "Preparing to unpack .../40-libmariadb3_1%3a10.5.15-0+deb11u1_amd64.deb ...\n", + "Unpacking libmariadb3:amd64 (1:10.5.15-0+deb11u1) ...\n", + "Selecting previously unselected package libhdf5-hl-100:amd64.\n", + "Preparing to unpack .../41-libhdf5-hl-100_1.10.6+repack-4+deb11u1_amd64.deb ...\n", + "Unpacking libhdf5-hl-100:amd64 (1.10.6+repack-4+deb11u1) ...\n", + "Selecting previously unselected package libnetcdf18:amd64.\n", + "Preparing to unpack .../42-libnetcdf18_1%3a4.7.4-1_amd64.deb ...\n", + "Unpacking libnetcdf18:amd64 (1:4.7.4-1) ...\n", "Selecting previously unselected package libltdl7:amd64.\n", - "Preparing to unpack .../46-libltdl7_2.4.6-9_amd64.deb ...\n", - "Unpacking libltdl7:amd64 (2.4.6-9) ...\n", + "Preparing to unpack .../43-libltdl7_2.4.6-15_amd64.deb ...\n", + "Unpacking libltdl7:amd64 (2.4.6-15) ...\n", "Selecting previously unselected package libodbc1:amd64.\n", - "Preparing to unpack .../47-libodbc1_2.3.6-0.1_amd64.deb ...\n", - "Unpacking libodbc1:amd64 (2.3.6-0.1) ...\n", - "Selecting previously unselected package libogdi3.2.\n", - "Preparing to unpack .../48-libogdi3.2_3.2.1+ds-4_amd64.deb ...\n", - "Unpacking libogdi3.2 (3.2.1+ds-4) ...\n", + "Preparing to unpack .../44-libodbc1_2.3.6-0.1+b1_amd64.deb ...\n", + "Unpacking libodbc1:amd64 (2.3.6-0.1+b1) ...\n", + "Selecting previously unselected package libogdi4.1.\n", + "Preparing to unpack .../45-libogdi4.1_4.1.0+ds-5_amd64.deb ...\n", + "Unpacking libogdi4.1 (4.1.0+ds-5) ...\n", "Selecting previously unselected package libopenjp2-7:amd64.\n", - "Preparing to unpack .../49-libopenjp2-7_2.3.0-2+deb10u2_amd64.deb ...\n", - "Unpacking libopenjp2-7:amd64 (2.3.0-2+deb10u2) ...\n", + "Preparing to unpack .../46-libopenjp2-7_2.4.0-3_amd64.deb ...\n", + "Unpacking libopenjp2-7:amd64 (2.4.0-3) ...\n", "Selecting previously unselected package liblcms2-2:amd64.\n", - "Preparing to unpack .../50-liblcms2-2_2.9-3_amd64.deb ...\n", - "Unpacking liblcms2-2:amd64 (2.9-3) ...\n", + "Preparing to unpack .../47-liblcms2-2_2.12~rc1-2_amd64.deb ...\n", + "Unpacking liblcms2-2:amd64 (2.12~rc1-2) ...\n", "Selecting previously unselected package libnspr4:amd64.\n", - "Preparing to unpack .../51-libnspr4_2%3a4.20-1_amd64.deb ...\n", - "Unpacking libnspr4:amd64 (2:4.20-1) ...\n", + "Preparing to unpack .../48-libnspr4_2%3a4.29-1_amd64.deb ...\n", + "Unpacking libnspr4:amd64 (2:4.29-1) ...\n", "Selecting previously unselected package libnss3:amd64.\n", - "Preparing to unpack .../52-libnss3_2%3a3.42.1-1+deb10u5_amd64.deb ...\n", - "Unpacking libnss3:amd64 (2:3.42.1-1+deb10u5) ...\n", - "Selecting previously unselected package libpoppler82:amd64.\n", - "Preparing to unpack .../53-libpoppler82_0.71.0-5_amd64.deb ...\n", - "Unpacking libpoppler82:amd64 (0.71.0-5) ...\n", + "Preparing to unpack .../49-libnss3_2%3a3.61-1+deb11u2_amd64.deb ...\n", + "Unpacking libnss3:amd64 (2:3.61-1+deb11u2) ...\n", + "Selecting previously unselected package libpoppler102:amd64.\n", + "Preparing to unpack .../50-libpoppler102_20.09.0-3.1_amd64.deb ...\n", + "Unpacking libpoppler102:amd64 (20.09.0-3.1) ...\n", "Selecting previously unselected package libpq5:amd64.\n", - "Preparing to unpack .../54-libpq5_11.14-0+deb10u1_amd64.deb ...\n", - "Unpacking libpq5:amd64 (11.14-0+deb10u1) ...\n", - "Selecting previously unselected package libqhull7:amd64.\n", - "Preparing to unpack .../55-libqhull7_2015.2-4_amd64.deb ...\n", - "Unpacking libqhull7:amd64 (2015.2-4) ...\n", + "Preparing to unpack .../51-libpq5_13.7-0+deb11u1_amd64.deb ...\n", + "Unpacking libpq5:amd64 (13.7-0+deb11u1) ...\n", + "Selecting previously unselected package libqhull8.0:amd64.\n", + "Preparing to unpack .../52-libqhull8.0_2020.2-3_amd64.deb ...\n", + "Unpacking libqhull8.0:amd64 (2020.2-3) ...\n", + "Selecting previously unselected package librttopo1:amd64.\n", + "Preparing to unpack .../53-librttopo1_1.1.0-2_amd64.deb ...\n", + "Unpacking librttopo1:amd64 (1.1.0-2) ...\n", "Selecting previously unselected package libspatialite7:amd64.\n", - "Preparing to unpack .../56-libspatialite7_4.3.0a-5+b2_amd64.deb ...\n", - "Unpacking libspatialite7:amd64 (4.3.0a-5+b2) ...\n", + "Preparing to unpack .../54-libspatialite7_5.0.1-2_amd64.deb ...\n", + "Unpacking libspatialite7:amd64 (5.0.1-2) ...\n", "Selecting previously unselected package libxerces-c3.2:amd64.\n", - "Preparing to unpack .../57-libxerces-c3.2_3.2.2+debian-1+deb10u1_amd64.deb ...\n", - "Unpacking libxerces-c3.2:amd64 (3.2.2+debian-1+deb10u1) ...\n", + "Preparing to unpack .../55-libxerces-c3.2_3.2.3+debian-3_amd64.deb ...\n", + "Unpacking libxerces-c3.2:amd64 (3.2.3+debian-3) ...\n", "Selecting previously unselected package odbcinst.\n", - "Preparing to unpack .../58-odbcinst_2.3.6-0.1_amd64.deb ...\n", - "Unpacking odbcinst (2.3.6-0.1) ...\n", + "Preparing to unpack .../56-odbcinst_2.3.6-0.1+b1_amd64.deb ...\n", + "Unpacking odbcinst (2.3.6-0.1+b1) ...\n", "Selecting previously unselected package odbcinst1debian2:amd64.\n", - "Preparing to unpack .../59-odbcinst1debian2_2.3.6-0.1_amd64.deb ...\n", - "Unpacking odbcinst1debian2:amd64 (2.3.6-0.1) ...\n", - "Selecting previously unselected package libgdal20.\n", - "Preparing to unpack .../60-libgdal20_2.4.0+dfsg-1+b1_amd64.deb ...\n", - "Unpacking libgdal20 (2.4.0+dfsg-1+b1) ...\n", + "Preparing to unpack .../57-odbcinst1debian2_2.3.6-0.1+b1_amd64.deb ...\n", + "Unpacking odbcinst1debian2:amd64 (2.3.6-0.1+b1) ...\n", + "Selecting previously unselected package libgdal28.\n", + "Preparing to unpack .../58-libgdal28_3.2.2+dfsg-2+deb11u1_amd64.deb ...\n", + "Unpacking libgdal28 (3.2.2+dfsg-2+deb11u1) ...\n", "Selecting previously unselected package libxslt1.1:amd64.\n", - "Preparing to unpack .../61-libxslt1.1_1.1.32-2.2~deb10u1_amd64.deb ...\n", - "Unpacking libxslt1.1:amd64 (1.1.32-2.2~deb10u1) ...\n", + "Preparing to unpack .../59-libxslt1.1_1.1.34-4_amd64.deb ...\n", + "Unpacking libxslt1.1:amd64 (1.1.34-4) ...\n", "Selecting previously unselected package proj-bin.\n", - "Preparing to unpack .../62-proj-bin_5.2.0-1_amd64.deb ...\n", - "Unpacking proj-bin (5.2.0-1) ...\n", + "Preparing to unpack .../60-proj-bin_7.2.1-1_amd64.deb ...\n", + "Unpacking proj-bin (7.2.1-1) ...\n", "Selecting previously unselected package python3-affine.\n", - "Preparing to unpack .../63-python3-affine_2.2.2-1_all.deb ...\n", - "Unpacking python3-affine (2.2.2-1) ...\n", + "Preparing to unpack .../61-python3-affine_2.3.0-2_all.deb ...\n", + "Unpacking python3-affine (2.3.0-2) ...\n", "Selecting previously unselected package python3-attr.\n", - "Preparing to unpack .../64-python3-attr_18.2.0-1_all.deb ...\n", - "Unpacking python3-attr (18.2.0-1) ...\n", + "Preparing to unpack .../62-python3-attr_20.3.0-1_all.deb ...\n", + "Unpacking python3-attr (20.3.0-1) ...\n", "Selecting previously unselected package python3-soupsieve.\n", - "Preparing to unpack .../65-python3-soupsieve_1.8+dfsg-1_all.deb ...\n", - "Unpacking python3-soupsieve (1.8+dfsg-1) ...\n", + "Preparing to unpack .../63-python3-soupsieve_2.2.1-1_all.deb ...\n", + "Unpacking python3-soupsieve (2.2.1-1) ...\n", "Selecting previously unselected package python3-bs4.\n", - "Preparing to unpack .../66-python3-bs4_4.7.1-1_all.deb ...\n", - "Unpacking python3-bs4 (4.7.1-1) ...\n", + "Preparing to unpack .../64-python3-bs4_4.9.3-1_all.deb ...\n", + "Unpacking python3-bs4 (4.9.3-1) ...\n", + "Selecting previously unselected package python3-certifi.\n", + "Preparing to unpack .../65-python3-certifi_2020.6.20-1_all.deb ...\n", + "Unpacking python3-certifi (2020.6.20-1) ...\n", "Selecting previously unselected package python3-chardet.\n", - "Preparing to unpack .../67-python3-chardet_3.0.4-3_all.deb ...\n", - "Unpacking python3-chardet (3.0.4-3) ...\n", + "Preparing to unpack .../66-python3-chardet_4.0.0-1_all.deb ...\n", + "Unpacking python3-chardet (4.0.0-1) ...\n", "Selecting previously unselected package python3-colorama.\n", - "Preparing to unpack .../68-python3-colorama_0.3.7-1_all.deb ...\n", - "Unpacking python3-colorama (0.3.7-1) ...\n", + "Preparing to unpack .../67-python3-colorama_0.4.4-1_all.deb ...\n", + "Unpacking python3-colorama (0.4.4-1) ...\n", "Selecting previously unselected package python3-click.\n", - "Preparing to unpack .../69-python3-click_7.0-1_all.deb ...\n", - "Unpacking python3-click (7.0-1) ...\n", + "Preparing to unpack .../68-python3-click_7.1.2-1_all.deb ...\n", + "Unpacking python3-click (7.1.2-1) ...\n", "Selecting previously unselected package python3-click-plugins.\n", - "Preparing to unpack .../70-python3-click-plugins_1.0.4-1_all.deb ...\n", - "Unpacking python3-click-plugins (1.0.4-1) ...\n", + "Preparing to unpack .../69-python3-click-plugins_1.1.1-3_all.deb ...\n", + "Unpacking python3-click-plugins (1.1.1-3) ...\n", "Selecting previously unselected package python3-cligj.\n", - "Preparing to unpack .../71-python3-cligj_0.5.0-1_all.deb ...\n", - "Unpacking python3-cligj (0.5.0-1) ...\n", + "Preparing to unpack .../70-python3-cligj_0.7.1-1_all.deb ...\n", + "Unpacking python3-cligj (0.7.1-1) ...\n", + "Selecting previously unselected package python3-six.\n", + "Preparing to unpack .../71-python3-six_1.16.0-2_all.deb ...\n", + "Unpacking python3-six (1.16.0-2) ...\n", "Selecting previously unselected package python3-webencodings.\n", - "Preparing to unpack .../72-python3-webencodings_0.5.1-1_all.deb ...\n", - "Unpacking python3-webencodings (0.5.1-1) ...\n", + "Preparing to unpack .../72-python3-webencodings_0.5.1-2_all.deb ...\n", + "Unpacking python3-webencodings (0.5.1-2) ...\n", "Selecting previously unselected package python3-html5lib.\n", - "Preparing to unpack .../73-python3-html5lib_1.0.1-1_all.deb ...\n", - "Unpacking python3-html5lib (1.0.1-1) ...\n", + "Preparing to unpack .../73-python3-html5lib_1.1-3_all.deb ...\n", + "Unpacking python3-html5lib (1.1-3) ...\n", "Selecting previously unselected package python3-lxml:amd64.\n", - "Preparing to unpack .../74-python3-lxml_4.3.2-1+deb10u4_amd64.deb ...\n", - "Unpacking python3-lxml:amd64 (4.3.2-1+deb10u4) ...\n", + "Preparing to unpack .../74-python3-lxml_4.6.3+dfsg-0.1+deb11u1_amd64.deb ...\n", + "Unpacking python3-lxml:amd64 (4.6.3+dfsg-0.1+deb11u1) ...\n", "Selecting previously unselected package python3-numpy.\n", - "Preparing to unpack .../75-python3-numpy_1%3a1.16.2-1_amd64.deb ...\n", - "Unpacking python3-numpy (1:1.16.2-1) ...\n", + "Preparing to unpack .../75-python3-numpy_1%3a1.19.5-1_amd64.deb ...\n", + "Unpacking python3-numpy (1:1.19.5-1) ...\n", "Selecting previously unselected package python3-pyparsing.\n", - "Preparing to unpack .../76-python3-pyparsing_2.2.0+dfsg1-2_all.deb ...\n", - "Unpacking python3-pyparsing (2.2.0+dfsg1-2) ...\n", + "Preparing to unpack .../76-python3-pyparsing_2.4.7-1_all.deb ...\n", + "Unpacking python3-pyparsing (2.4.7-1) ...\n", "Selecting previously unselected package python3-snuggs.\n", - "Preparing to unpack .../77-python3-snuggs_1.4.3-1_all.deb ...\n", - "Unpacking python3-snuggs (1.4.3-1) ...\n", + "Preparing to unpack .../77-python3-snuggs_1.4.7-2_all.deb ...\n", + "Unpacking python3-snuggs (1.4.7-2) ...\n", "Selecting previously unselected package python3-rasterio.\n", - "Preparing to unpack .../78-python3-rasterio_1.0.21-1_amd64.deb ...\n", - "Unpacking python3-rasterio (1.0.21-1) ...\n", - "Setting up liblcms2-2:amd64 (2.9-3) ...\n", - "Setting up python3-attr (18.2.0-1) ...\n", - "Setting up mysql-common (5.8+1.0.5) ...\n", + "Preparing to unpack .../78-python3-rasterio_1.2.0-1_amd64.deb ...\n", + "Unpacking python3-rasterio (1.2.0-1) ...\n", + "Setting up liblcms2-2:amd64 (2.12~rc1-2) ...\n", + "Setting up python3-attr (20.3.0-1) ...\n", + "Setting up mysql-common (5.8+1.0.7) ...\n", "update-alternatives: using /etc/mysql/my.cnf.fallback to provide /etc/mysql/my.cnf (my.cnf) in auto mode\n", - "Setting up proj-data (5.2.0-1) ...\n", - "Setting up python3-colorama (0.3.7-1) ...\n", - "Setting up libcharls2:amd64 (2.0.0+dfsg-1) ...\n", + "Setting up proj-data (7.2.1-1) ...\n", + "Setting up libicu67:amd64 (67.1-7) ...\n", + "Setting up libogdi4.1 (4.1.0+ds-5) ...\n", + "Setting up libqhull8.0:amd64 (2020.2-3) ...\n", + "Setting up python3-colorama (0.4.4-1) ...\n", + "Setting up libcharls2:amd64 (2.2.0+dfsg-2) ...\n", "Setting up libminizip1:amd64 (1.1-8+b1) ...\n", - "Setting up libproj13:amd64 (5.2.0-1) ...\n", - "Setting up libpq5:amd64 (11.14-0+deb10u1) ...\n", - "Setting up proj-bin (5.2.0-1) ...\n", - "Setting up libqhull7:amd64 (2015.2-4) ...\n", - "Setting up python3-click (7.0-1) ...\n", - "Setting up libepsilon1:amd64 (0.9.2+dfsg-4) ...\n", - "Setting up libjbig0:amd64 (2.1-3.1+b2) ...\n", - "Setting up python3-webencodings (0.5.1-1) ...\n", - "Setting up libgeos-3.7.1:amd64 (3.7.1-1) ...\n", - "Setting up libaec0:amd64 (1.0.2-1) ...\n", - "Setting up libicu63:amd64 (63.1-6+deb10u2) ...\n", - "Setting up gdal-data (2.4.0+dfsg-1) ...\n", - "Setting up poppler-data (0.4.9-2) ...\n", - "Setting up mariadb-common (1:10.3.31-0+deb10u1) ...\n", + "Setting up libaom0:amd64 (1.0.0.errata1-3) ...\n", + "Setting up libpq5:amd64 (13.7-0+deb11u1) ...\n", + "Setting up python3-click (7.1.2-1) ...\n", + "Setting up libepsilon1:amd64 (0.9.2+dfsg-5) ...\n", + "Setting up python3-webencodings (0.5.1-2) ...\n", + "Setting up libaec0:amd64 (1.0.4-1) ...\n", + "Setting up gdal-data (3.2.2+dfsg-2+deb11u1) ...\n", + "Setting up poppler-data (0.4.10-1) ...\n", + "Setting up libcfitsio9:amd64 (3.490-3) ...\n", + "Setting up mariadb-common (1:10.5.15-0+deb11u1) ...\n", "update-alternatives: using /etc/mysql/mariadb.cnf to provide /etc/mysql/my.cnf (my.cnf) in auto mode\n", - "Setting up python3-chardet (3.0.4-3) ...\n", - "Setting up libjpeg62-turbo:amd64 (1:1.5.2-2+deb10u1) ...\n", - "Setting up python3-pyparsing (2.2.0+dfsg1-2) ...\n", - "Setting up libnspr4:amd64 (2:4.20-1) ...\n", - "Setting up libwebp6:amd64 (0.6.1-2+deb10u1) ...\n", - "Setting up libgeos-c1v5:amd64 (3.7.1-1) ...\n", - "Setting up python3-click-plugins (1.0.4-1) ...\n", - "Setting up libmariadb3:amd64 (1:10.3.31-0+deb10u1) ...\n", - "Setting up fonts-dejavu-core (2.37-1) ...\n", - "Setting up python3-html5lib (1.0.1-1) ...\n", - "Setting up libltdl7:amd64 (2.4.6-9) ...\n", - "Setting up libgfortran5:amd64 (8.3.0-6) ...\n", - "Setting up libhdf4-0-alt (4.2.13-4) ...\n", - "Setting up libgif7:amd64 (5.1.4-3) ...\n", - "Setting up liburiparser1:amd64 (0.9.1-1+deb10u1) ...\n", - "Setting up libfreexl1:amd64 (1.0.5-3) ...\n", - "Setting up libfyba0:amd64 (4.1.1-6) ...\n", - "Setting up libkmlbase1:amd64 (1.3.0-7) ...\n", - "Setting up libopenjp2-7:amd64 (2.3.0-2+deb10u2) ...\n", - "Setting up libtiff5:amd64 (4.1.0+git191117-2~deb10u3) ...\n", - "Setting up python3-affine (2.2.2-1) ...\n", - "Setting up libxml2:amd64 (2.9.4+dfsg1-7+deb10u2) ...\n", - "Setting up python3-soupsieve (1.8+dfsg-1) ...\n", - "Setting up libsz2:amd64 (1.0.2-1) ...\n", - "Setting up libkmlxsd1:amd64 (1.3.0-7) ...\n", - "Setting up libkmldom1:amd64 (1.3.0-7) ...\n", - "Setting up libspatialite7:amd64 (4.3.0a-5+b2) ...\n", - "Setting up libogdi3.2 (3.2.1+ds-4) ...\n", - "Setting up libxerces-c3.2:amd64 (3.2.2+debian-1+deb10u1) ...\n", - "Setting up libkmlengine1:amd64 (1.3.0-7) ...\n", - "Setting up libkmlconvenience1:amd64 (1.3.0-7) ...\n", - "Setting up fontconfig-config (2.13.1-2) ...\n", - "Setting up python3-cligj (0.5.0-1) ...\n", - "Setting up libdap25:amd64 (3.20.3-1) ...\n", - "Setting up libnss3:amd64 (2:3.42.1-1+deb10u5) ...\n", - "Setting up libdapserver7v5:amd64 (3.20.3-1) ...\n", - "Setting up python3-bs4 (4.7.1-1) ...\n", - "Setting up libgeotiff2:amd64 (1.4.3-1) ...\n", - "Setting up libblas3:amd64 (3.8.0-2) ...\n", + "Setting up python3-six (1.16.0-2) ...\n", + "Setting up libblas3:amd64 (3.9.0-3) ...\n", "update-alternatives: using /usr/lib/x86_64-linux-gnu/blas/libblas.so.3 to provide /usr/lib/x86_64-linux-gnu/libblas.so.3 (libblas.so.3-x86_64-linux-gnu) in auto mode\n", - "Setting up libkmlregionator1:amd64 (1.3.0-7) ...\n", - "Setting up libhdf5-103:amd64 (1.10.4+repack-10) ...\n", - "Setting up libodbc1:amd64 (2.3.6-0.1) ...\n", - "Setting up libxslt1.1:amd64 (1.1.32-2.2~deb10u1) ...\n", - "Setting up libdapclient6v5:amd64 (3.20.3-1) ...\n", - "Setting up libfontconfig1:amd64 (2.13.1-2) ...\n", - "Setting up liblapack3:amd64 (3.8.0-2) ...\n", + "Setting up python3-chardet (4.0.0-1) ...\n", + "Setting up python3-pyparsing (2.4.7-1) ...\n", + "Setting up python3-certifi (2020.6.20-1) ...\n", + "Setting up libnspr4:amd64 (2:4.29-1) ...\n", + "Setting up python3-click-plugins (1.1.1-3) ...\n", + "Setting up libmariadb3:amd64 (1:10.5.15-0+deb11u1) ...\n", + "Setting up python3-html5lib (1.1-3) ...\n", + "Setting up libltdl7:amd64 (2.4.6-15) ...\n", + "Setting up libgfortran5:amd64 (10.2.1-6) ...\n", + "Setting up libhdf4-0-alt (4.2.15-3) ...\n", + "Setting up libgif7:amd64 (5.1.9-2) ...\n", + "Setting up liburiparser1:amd64 (0.9.4+dfsg-1+deb11u1) ...\n", + "Setting up libnuma1:amd64 (2.0.12-1+b1) ...\n", + "Setting up libfreexl1:amd64 (1.0.6-1) ...\n", + "Setting up libgeos-3.9.0:amd64 (3.9.0-1) ...\n", + "Setting up libfyba0:amd64 (4.1.1-7) ...\n", + "Setting up libkmlbase1:amd64 (1.3.0-9) ...\n", + "Setting up libdav1d4:amd64 (0.7.1-3) ...\n", + "Setting up libopenjp2-7:amd64 (2.4.0-3) ...\n", + "Setting up python3-affine (2.3.0-2) ...\n", + "Setting up libde265-0:amd64 (1.0.8-1) ...\n", + "Setting up libproj19:amd64 (7.2.1-1) ...\n", + "Setting up libxml2:amd64 (2.9.10+dfsg-6.7+deb11u1) ...\n", + "Setting up python3-soupsieve (2.2.1-1) ...\n", + "Setting up libsz2:amd64 (1.0.4-1) ...\n", + "Setting up libkmldom1:amd64 (1.3.0-9) ...\n", + "Setting up liblapack3:amd64 (3.9.0-3) ...\n", "update-alternatives: using /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3 to provide /usr/lib/x86_64-linux-gnu/liblapack.so.3 (liblapack.so.3-x86_64-linux-gnu) in auto mode\n", - "Setting up libarpack2:amd64 (3.7.0-2) ...\n", - "Setting up libsuperlu5:amd64 (5.2.1+dfsg1-4) ...\n", - "Setting up libpoppler82:amd64 (0.71.0-5) ...\n", - "Setting up libnetcdf13:amd64 (1:4.6.2-1) ...\n", - "Setting up python3-numpy (1:1.16.2-1) ...\n", - "Setting up python3-lxml:amd64 (4.3.2-1+deb10u4) ...\n", - "Setting up libarmadillo9 (1:9.200.7+dfsg-1) ...\n", - "Setting up python3-snuggs (1.4.3-1) ...\n", - "Setting up odbcinst (2.3.6-0.1) ...\n", - "Setting up odbcinst1debian2:amd64 (2.3.6-0.1) ...\n", - "Setting up libgdal20 (2.4.0+dfsg-1+b1) ...\n", - "Setting up python3-rasterio (1.0.21-1) ...\n", - "Processing triggers for libc-bin (2.28-10) ...\n", - "Processing triggers for man-db (2.8.5-2) ...\n" + "Setting up libxerces-c3.2:amd64 (3.2.3+debian-3) ...\n", + "Setting up libkmlengine1:amd64 (1.3.0-9) ...\n", + "Setting up python3-cligj (0.7.1-1) ...\n", + "Setting up libarpack2:amd64 (3.8.0-1) ...\n", + "Setting up libx265-192:amd64 (3.4-2) ...\n", + "Setting up libsuperlu5:amd64 (5.2.2+dfsg1-2) ...\n", + "Setting up proj-bin (7.2.1-1) ...\n", + "Setting up libnss3:amd64 (2:3.61-1+deb11u2) ...\n", + "Setting up python3-bs4 (4.9.3-1) ...\n", + "Setting up libgeotiff5:amd64 (1.6.0-1) ...\n", + "Setting up libdap27:amd64 (3.20.7-6) ...\n", + "Setting up libgeos-c1v5:amd64 (3.9.0-1) ...\n", + "Setting up libodbc1:amd64 (2.3.6-0.1+b1) ...\n", + "Setting up python3-numpy (1:1.19.5-1) ...\n", + "Setting up libhdf5-103-1:amd64 (1.10.6+repack-4+deb11u1) ...\n", + "Setting up librttopo1:amd64 (1.1.0-2) ...\n", + "Setting up libpoppler102:amd64 (20.09.0-3.1) ...\n", + "Setting up libxslt1.1:amd64 (1.1.34-4) ...\n", + "Setting up libdapclient6v5:amd64 (3.20.7-6) ...\n", + "Setting up libhdf5-hl-100:amd64 (1.10.6+repack-4+deb11u1) ...\n", + "Setting up libspatialite7:amd64 (5.0.1-2) ...\n", + "Setting up libarmadillo10 (1:10.1.2+dfsg-6+b1) ...\n", + "Setting up libheif1:amd64 (1.11.0-1) ...\n", + "Setting up python3-lxml:amd64 (4.6.3+dfsg-0.1+deb11u1) ...\n", + "Setting up libnetcdf18:amd64 (1:4.7.4-1) ...\n", + "Setting up python3-snuggs (1.4.7-2) ...\n", + "Setting up odbcinst (2.3.6-0.1+b1) ...\n", + "Setting up odbcinst1debian2:amd64 (2.3.6-0.1+b1) ...\n", + "Setting up libgdal28 (3.2.2+dfsg-2+deb11u1) ...\n", + "Setting up python3-rasterio (1.2.0-1) ...\n", + "Processing triggers for libc-bin (2.31-13+deb11u3) ...\n", + "Processing triggers for man-db (2.9.4-2) ...\n" ] } ], @@ -1407,7 +1417,7 @@ }, { "cell_type": "code", - "execution_count": 26, + "execution_count": 22, "id": "b9e367a0-26ce-42ce-bb04-6a432f41876e", "metadata": {}, "outputs": [ @@ -1417,8 +1427,7 @@ "text": [ "LC08_L1TP_025033_20201007_20201016_01_T1 1\n", "LC08_L1TP_025033_20201007_20201016_01_T1 2\n", - "LC08_L1TP_025033_20201007_20201016_01_T1 3\n", - "ERROR 4: output/result-LC08_L1TP_025033_20201007_20201016_01_T1.png: No such file or directory\n" + "LC08_L1TP_025033_20201007_20201016_01_T1 3\n" ] } ], @@ -1431,7 +1440,7 @@ "id": "4199910a-1056-455c-ae9d-669296441aed", "metadata": {}, "source": [ - "Note, the \"ERROR 4\" is expected." + "*Note: if you get an \"ERROR 4\" it is expected in earlier versions of rasterio.*" ] }, { @@ -1444,7 +1453,7 @@ }, { "cell_type": "code", - "execution_count": 27, + "execution_count": 23, "id": "db9f26aa-6317-4834-8bf1-972c8b3cc032", "metadata": {}, "outputs": [ @@ -1453,8 +1462,8 @@ "output_type": "stream", "text": [ "total 192M\n", - "-rw-r--r-- 1 learner learner 192M Feb 14 20:36 \u001b[0m\u001b[01;35mresult-LC08_L1TP_025033_20201007_20201016_01_T1.png\u001b[0m\n", - "-rw-r--r-- 1 learner learner 910 Feb 14 20:36 result-LC08_L1TP_025033_20201007_20201016_01_T1.png.aux.xml\n" + "-rw-r--r-- 1 learner learner 192M May 20 17:43 \u001b[0m\u001b[01;35mresult-LC08_L1TP_025033_20201007_20201016_01_T1.png\u001b[0m\n", + "-rw-r--r-- 1 learner learner 941 May 20 17:43 result-LC08_L1TP_025033_20201007_20201016_01_T1.png.aux.xml\n" ] } ], @@ -1612,11 +1621,12 @@ "id": "4de32b26-f4cb-4f33-b7d5-e3852769840a", "metadata": {}, "source": [ - "````{admonition} Exercise\n", + ":::{admonition} Exercise\n", + ":class: exercise\n", "\n", - " * Try to find and view the results on your own\n", + " * Try to find and view the results on your own\n", "\n", - "````" + ":::" ] }, { @@ -1640,11 +1650,12 @@ "\n", "We will now leave the resources running in order to learn more about sharing and monitoring costs and will clean up all the resources as the end of Lesson. \n", "\n", - "```{admonition} Danger\n", + ":::{admonition} Danger\n", ":class: danger\n", "\n", "**Do not forget to do remove the Cloud Storage Bucket and the Compute Engine Instance (Virtual Machine) when you are done with this Lesson!** Running resources and stored data costs money! Cleanup when you are done!\n", - "```" + "\n", + ":::" ] } ], diff --git a/content/GCP/06_sharing_results.ipynb b/content/GCP/06_sharing_results.ipynb index edc6e0b..b5a9cc0 100644 --- a/content/GCP/06_sharing_results.ipynb +++ b/content/GCP/06_sharing_results.ipynb @@ -7,7 +7,7 @@ "source": [ "# Sharing Data (Optional)\n", "\n", - "```{admonition} Overview\n", + ":::{admonition} Overview\n", ":class: tip\n", "\n", "**Teaching:** 30 min\n", @@ -15,12 +15,13 @@ "**Exercises:** 5 min (optional)\n", "\n", "**Questions:**\n", - "* How do I share my results?\n", + " * How do I share my results?\n", "\n", "**Objectives:**\n", - "* Understand the basics of Identity and Access Management (IAM)\n", - "* Add collaborators to a Bucket with appropriate permissions.\n", - "```" + " * Understand the basics of Identity and Access Management (IAM)\n", + " * Add collaborators to a Bucket with appropriate permissions.\n", + "\n", + ":::" ] }, { @@ -50,12 +51,14 @@ "tags": [] }, "source": [ - "```{admonition} Exercise\n", + ":::{admonition} Exercise\n", + ":class: exercise\n", "\n", "Answer the following questions:\n", " * What is the \"Who, What, Where\" of the IAM policy that allows you to use your project?\n", " * What else has permissions to do things in your project and state the \"Who, What, Where\"?\n", - "```" + "\n", + ":::" ] }, { @@ -126,10 +129,12 @@ "id": "4b93362c-c22b-453c-910b-db4a14ca58af", "metadata": {}, "source": [ - "```{admonition} Exercise\n", + ":::{admonition} Exercise\n", + ":class: exercise\n", "\n", "*Instructors: You may want to have students share these buckets for the example to reduce screen flipping and involve the students.*\n", - "```" + "\n", + ":::" ] } ], diff --git a/content/GCP/07_monitoring_costs.ipynb b/content/GCP/07_monitoring_costs.ipynb index 781ba4e..521ca09 100644 --- a/content/GCP/07_monitoring_costs.ipynb +++ b/content/GCP/07_monitoring_costs.ipynb @@ -7,7 +7,7 @@ "source": [ "# Monitoring Costs\n", "\n", - "```{admonition} Overview\n", + ":::{admonition} Overview\n", ":class: tip\n", "\n", "**Teaching:** 10 min\n", @@ -15,20 +15,20 @@ "**Exercises:** none\n", "\n", "**Questions:**\n", - "* How do I find out how much I have spent?\n", - "* How do I find what is running and stored in my project?\n", + " * How do I find out how much I have spent?\n", + " * How do I find what is running and stored in my project?\n", "\n", "**Objectives:**\n", - "* Learn about Billing Accounts\n", - "* Find information about the Billing Account associated with your project\n", - "* Determine the permissions you have on the Billing Account\n", - "* Find and monitor your spend with billing reports\n", - "* Find where to set Billing Alerts.\n", - "* Find all resources in a Project\n", - "* Extra: List assets with the CLI\n", - "* Extra: Understand how labels are used\n", + " * Learn about Billing Accounts\n", + " * Find information about the Billing Account associated with your project\n", + " * Determine the permissions you have on the Billing Account\n", + " * Find and monitor your spend with billing reports\n", + " * Find where to set Billing Alerts.\n", + " * Find all resources in a Project\n", + " * Extra: List assets with the CLI\n", + " * Extra: Understand how labels are used\n", " \n", - "```" + ":::" ] }, { @@ -41,10 +41,9 @@ "Projects and Billing Accounts are two different things in Google Cloud. In order for a Project operate it must have a Billing Account to charge any expenses to. Even if a project is utilizing credits it must have a Billing Account associated with it. Projects within an organization will often have the Billing Account controlled directly by the organization.\n", "\n", "To find out more about the Billing Account associated with your project do the following:\n", - " * **Navigation Menu -> Home** (this will ensure your project is selected)\n", - " * Double check that the Project ID and name in the \"Project Info\" card is correct.\n", - " * Note the billing card, but do not click on it.\n", - " * **Navigation Menu -> Billing** \n", + " * Click **Google Cloud Platform** (this will ensure your project is selected)\n", + " * Double check that the Project ID and name is correct.\n", + " * Select **Navigation Menu -> Billing** \n", " * If you get a \"You have multiple billing accounts\" select \"**Go to linked billing account**\"\n", " * If you get a \"This project has no billing account\" you cannot incur any charges on this account. Since you do not have a billing account you have no charges to monitor or set alerts on. You will need to associate or create a billing account to do anything.\n", " * Select **Account Management** on the *Left Sidebar* of the Billing page.\n", @@ -84,7 +83,7 @@ " * \"Daily\"\n", " * Bar chart (bar chart graph on the right, \"Daily\" is on the left)\n", " * **Save** view\n", - " * If you cannot save the view you can also click \"Share\" and use the URL to create a bookmark for easy access.\n", + " * If you cannot save the view you can also click \"**Share**\" and use the URL to create a bookmark for easy access.\n", "\n", "![billing-report](img/billing-report.png)\n", "\n", @@ -152,7 +151,9 @@ "id": "bee03b96-53f6-47c0-bcbf-0bf73fe99f73", "metadata": { "scrolled": true, - "tags": [] + "tags": [ + "hide-output" + ] }, "outputs": [ { @@ -389,16 +390,27 @@ }, { "cell_type": "markdown", - "id": "45681123-b27a-48d8-ac4a-30dc3b46d2c1", + "id": "9b34734b-4039-4589-a98b-e8101f9b96c6", "metadata": {}, "source": [ "## Label Resources (CLI - Extra)\n", "\n", "In order to track resources it is important to Label them. Labeling places key-value pairs (key=value) in the resources metadata (information about the resource) and most of the Google Cloud tools can use this information to search, filter, and group information. In the literature, the example most used is the \"env\" Label and indicates if the resource is in development (dev), testing (test), or production (prod); the lables would be env=test, env=dev and env=prod, respectively. \n", "\n", - "For research and teaching, Labels could be used to track funding, grants, students, or other uses of resources. For large efforts (projects) it is better to use different Projects to separate access and billing and protect the projects from each other.\n", + "For research and teaching, Labels could be used to track funding, grants, students, or other uses of resources. For large efforts (projects) it is better to use different Projects to separate access and billing and protect the projects from each other." + ] + }, + { + "cell_type": "markdown", + "id": "2ee4b84d-80b2-4013-a258-bf709768430b", + "metadata": {}, + "source": [ + ":::{admonition} Pro Tip\n", + ":class: tip\n", + "\n", + "Labeling is called Tagging on AWS and Azure - tagging in GCP is related to firewalls).\n", "\n", - "*Pro-tip: Labeling is called Tagging on AWS and Azure - tagging in GCP is related to firewalls).*\n" + ":::" ] }, { diff --git a/content/GCP/08_cleaning_up_resources.ipynb b/content/GCP/08_cleaning_up_resources.ipynb index f4b7bc1..388edb8 100644 --- a/content/GCP/08_cleaning_up_resources.ipynb +++ b/content/GCP/08_cleaning_up_resources.ipynb @@ -7,7 +7,7 @@ "source": [ "# Cleaning Up Resources\n", "\n", - "```{admonition} Overview\n", + ":::{admonition} Overview\n", ":class: tip\n", "\n", "**Teaching:** 5 min\n", @@ -15,92 +15,166 @@ "**Exercises:** 10 min\n", "\n", "**Questions:**\n", - "* How do I clean up after I'm done for the day?\n", - "* How do I make sure everything is gone?\n", + " * How do I clean up after I'm done for the day?\n", + " * How do I make sure everything is gone?\n", "\n", "**Objectives:**\n", - "* Cleanup resources created by the Lesson.\n", - "* Verify that all the resources have been removed.\n", + " * Cleanup resources created by the Lesson.\n", + " * Verify that all the resources have been removed.\n", "\n", - "```" + ":::" ] }, { "cell_type": "markdown", - "id": "65fb0615-f942-4af0-9087-8f0696df95f9", + "id": "29608fbe-70fa-474e-bcab-9f1fe3754503", "metadata": {}, "source": [ - "## Cleanup VMs\n", - "\n", - "```{admonition} Danger\n", + "## Cleanup Virtual Machine Instances" + ] + }, + { + "cell_type": "markdown", + "id": "e20eead6-05a0-4936-a27a-bb3377ae3cc0", + "metadata": {}, + "source": [ + ":::{admonition} Danger\n", ":class: danger\n", "\n", "This will destroy the VM instance and and ALL the data in it! You cannot undelete an instance!\n", - "```\n", "\n", - "```{admonition} Exercise\n", + ":::" + ] + }, + { + "cell_type": "markdown", + "id": "33ff4396-b718-4cba-8547-398ff48ea914", + "metadata": {}, + "source": [ + ":::{admonition} Exercise\n", + ":class: exercise\n", "\n", - " * Try to delete the **VM Instance** on your own\n", - "```\n", + " * Try to delete the **VM Instance** on your own\n", "\n", - "Navigate to the **Compute Engine** service and in the **VM Instances** page and remove the *essentials* *VM Instance*.\n" + ":::" ] }, { "cell_type": "markdown", - "id": "92321a60-13c6-4c87-aff5-92fb708e8bff", + "id": "63cf29e5-f3bb-426d-9a9c-2a9269caf7f4", "metadata": {}, "source": [ - "## Cleanup Buckets\n", - "\n", - "```{admonition} Danger\n", + "Navigate to the **Compute Engine** service and in the **VM Instances** page and remove the *essentials* *VM Instance*." + ] + }, + { + "cell_type": "markdown", + "id": "b7bb9c0c-98e5-41c6-a1a2-3cde35a2e994", + "metadata": {}, + "source": [ + "## Cleanup Buckets" + ] + }, + { + "cell_type": "markdown", + "id": "1e686ed7-f126-4941-a0bd-94b36d2e2ab5", + "metadata": {}, + "source": [ + ":::{admonition} Danger\n", ":class: danger\n", "\n", "This will destroy the storage *Bucket* and and ALL the data in it! You cannot undelete a Bucket!\n", - "```\n", - "\n", - "```{admonition} Exercise\n", + ":::" + ] + }, + { + "cell_type": "markdown", + "id": "4c6552f8-7b49-4414-8c44-8e739ec2ab44", + "metadata": {}, + "source": [ + ":::{admonition} Exercise\n", + ":class: exercise\n", "\n", - " * Try to delete the storage **bucket** on your own\n", - "```\n", + " * Try to delete the storage **bucket** on your own\n", "\n", + ":::" + ] + }, + { + "cell_type": "markdown", + "id": "440b89c3-52e1-44bb-945c-d323fccbf2e0", + "metadata": {}, + "source": [ "Navigate to the **Cloud Storage** service and in the **browser** page and remove the *essentials* Bucket." ] }, { "cell_type": "markdown", - "id": "6753b415-1fa9-4716-8c41-8d98228af51c", + "id": "f3749cf1-90d3-43db-980f-fb815c6363c0", "metadata": {}, "source": [ - "## Verify Cleanup\n", - "\n", - "```{admonition} Exercise\n", + "## Verify Cleanup" + ] + }, + { + "cell_type": "markdown", + "id": "2ed44c12-2592-433f-8ace-5f96d2ff7adf", + "metadata": {}, + "source": [ + ":::{admonition} Exercise\n", + ":class: exercise\n", "\n", " * There are at least three ways to verify resource removal, what are they?\n", " * Try to verify that the resources were removed.\n", - "```\n", "\n", - "```{admonition} Tip\n", - ":class: tip\n", + ":::" + ] + }, + { + "cell_type": "markdown", + "id": "b0f3766f-4ec2-487d-bc82-178d275387c0", + "metadata": {}, + "source": [ + ":::{tip}\n", + "\n", "You can also quickly navigate to resource page for most services from the **Resources** tab on the project **Dashboard**\n", - "```\n", "\n", + ":::" + ] + }, + { + "cell_type": "markdown", + "id": "9a4e2677-a8f0-4090-bfc7-4777b42e97c9", + "metadata": {}, + "source": [ "To verify that the resources have been deleted do the following:\n", " * Navigate to the **VM Instances** page and **Cloud Storage** **Browser** to verify the resources have been deleted.\n", " * Navigate to the **IAM & Admin** service and in the **Asset Inventory** page to verify that the resources have been deleted. **Note:** the information collected in the asset inventory is often delayed.\n", - " * Navigate to the **Activity** page and verify the deletion events. \n", - "\n", - "```{admonition} Tip\n", - ":class: tip\n", + " * Navigate to the **Activity** page and verify the deletion events. " + ] + }, + { + "cell_type": "markdown", + "id": "82a53b80-9600-482d-a41a-7be912059ffa", + "metadata": {}, + "source": [ + ":::{tip}\n", "\n", "If this is your own account, remember to visit the Billing Report page that you bookmarked daily.\n", - "```\n", "\n", - "```{admonition} Tip\n", - ":class: tip\n", + ":::" + ] + }, + { + "cell_type": "markdown", + "id": "753d0da7-d327-481e-a28a-333f14153e5d", + "metadata": {}, + "source": [ + ":::{tip}\n", "\n", "You can also create, list, and destroy resources easily from the cli with the `gcloud` and `gsutil` commands.\n", - "```" + "\n", + ":::" ] } ], diff --git a/content/GCP/glossary.ipynb b/content/GCP/glossary.ipynb index b0ef7a6..af1f059 100644 --- a/content/GCP/glossary.ipynb +++ b/content/GCP/glossary.ipynb @@ -13,6 +13,7 @@ " * **Bucket** - A collection of key value pairs (similar to a Python dictionary) and is often called object storage. Object storage in Google is called Cloud Storage and is similar to Amazon S3.\n", " * **Cloud Shell** - A Linux command line interface (CLI) running inside the GCP web console. \n", " * **Disk Image** - The operating system disk for a **VM Instance** such as Ubuntu, Debian, and other Linux distributions.\n", + " * **Organization** - A top level container for projects in an institution, usually one per institution.\n", " * **Project** - Resources and services in Google Cloud are created within a single Project. \n", " * **Project Name** - The long human readable name of your project. You can change this.\n", " * **Project ID** - an alpha-numeric string identifier for the project. This is not changeable.\n", diff --git a/content/GCP/img/CLASS-Essentials-GCP-Overview.svg b/content/GCP/img/CLASS-Essentials-GCP-Overview.svg new file mode 100644 index 0000000..999cc8a --- /dev/null +++ b/content/GCP/img/CLASS-Essentials-GCP-Overview.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/content/GCP/img/enable-compute-api.png b/content/GCP/img/enable-compute-api.png new file mode 100644 index 0000000..9a3cebb Binary files /dev/null and b/content/GCP/img/enable-compute-api.png differ diff --git a/content/_config.yml b/content/_config.yml index 0bb2c65..2d38905 100644 --- a/content/_config.yml +++ b/content/_config.yml @@ -48,3 +48,4 @@ parse: # don't forget to list any other extensions you want enabled, # including those that are enabled by default! - html_image + - colon_fence diff --git a/scripts/gcp-create.sh b/scripts/gcp-create.sh index aaaefed..2d7f8de 100755 --- a/scripts/gcp-create.sh +++ b/scripts/gcp-create.sh @@ -41,8 +41,9 @@ done echo "+++ configuring VM" gcloud compute ssh --zone=$ZONE $NAME@$VM --ssh-flag='-A' < .ssh/known_hosts +echo "set enable-bracketed-paste off" > ~/.inputrc git config --global color.ui auto git config --global push.default simple git config --global pull.ff only diff --git a/scripts/jupyter-lab.sh b/scripts/jupyter-lab.sh index 5f131d6..a2ae7b4 100755 --- a/scripts/jupyter-lab.sh +++ b/scripts/jupyter-lab.sh @@ -10,6 +10,8 @@ echo "+++ installing jupyter" python3 -m venv .venv . .venv/bin/activate +ln -sfv /usr/lib/python3/dist-packages .venv/lib/python3.*/ # simulate VM behavior for apt-get install python3-rasterio + python3 -m pip install pip --upgrade python3 -m pip install --upgrade jupyterlab