Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Browse files
Browse the repository at this point in the history
Added more notebooks
- Loading branch information
Amanda Tan
committed
Sep 24, 2021
1 parent
9bd449e
commit d3a720a
Showing
90 changed files
with
3,245 additions
and
439 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,33 @@ | ||
<!-- #region --> | ||
# A Primer to Cloud Computing for Research | ||
|
||
This online "book" aims to enable pathways to adoption of cloud computing for research. You will discover how the public cloud can foster innovation, accelerate scientific discovery and learn about best practices for leveraging the cloud. | ||
|
||
As a researcher or research facilitator, there are three main considerations in utilizing the cloud for scientific workflows: | ||
|
||
**(1) How do I know if my research belongs in the cloud?** | ||
|
||
**(2) Which cloud provider and which services should I utilize?** | ||
|
||
**(3) How do I estimate costs and resources needed for porting my work to the cloud?** | ||
|
||
Whether you are new to the cloud journey or are a seasoned cyberinfrastructure professional looking to communicate and address common concerns faced by researchers, this guide is intended to help you make an informed decision on whether a research workflow belongs in the public cloud, and to demystify some of the more obscure features around cloud adoption. | ||
|
||
|
||
## Cloud Adoption Framework | ||
|
||
```{attention} | ||
Cloud computing is a utility. You pay for the resources that you use! | ||
``` | ||
|
||
The CLASS Cloud Adoption Framework is a series of documentation, best practices, tutorials, workflows and cloud adoption pathways built by researchers and research and data (RCD) professionals ***for*** researchers and RCD professionals. | ||
|
||
## Issues | ||
Please contact class@internet2.edu with issues or questions | ||
|
||
|
||
Content for this site was put together by the Research Engagement team at Internet2. [Find out more here](https://internet2.edu/community/research-engagement/internet2-research-engagement-team/)! | ||
|
||
|
||
<!-- #endregion --> |
Binary file not shown.
Binary file renamed
BIN
+2.29 KB
...octrees/content/05_best_practices.doctree → .../.doctrees/content/best_practices.doctree
Binary file not shown.
Binary file renamed
BIN
+2.47 KB
...ees/content/02_is_the_cloud_right.doctree → ...d/.doctrees/content/class_program.doctree
Binary file not shown.
Binary file renamed
BIN
+2.31 KB
...doctrees/content/03_cost_analysis.doctree → ...d/.doctrees/content/cost_analysis.doctree
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file renamed
BIN
+2.33 KB
.../.doctrees/content/04_openscience.doctree → _build/.doctrees/content/openscience.doctree
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,4 @@ | ||
# Sphinx build info version 1 | ||
# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done. | ||
config: 04c13d88eea40c9fbb10e7c5003b91b1 | ||
config: 6526d3c866653ffeb0150924af4e4d0d | ||
tags: 645f666f9bcd5a90fca523b33c5a78b7 |
This file was deleted.
Oops, something went wrong.
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,46 @@ | ||
{ | ||
"cells": [ | ||
{ | ||
"cell_type": "markdown", | ||
"id": "90e130db", | ||
"metadata": {}, | ||
"source": [ | ||
"# Cloud Computing for Research\n", | ||
"\n", | ||
"Cloud computing is an on-demand computing resource that is scalable and follows a pay-as-you-go model. Instead of a singular data center or super-computing center, large cloud providers have data centers spanning multiple locations. The largest cloud computing providers are [Microsoft (Azure)](https://azure.microsoft.com/), [Amazon (Amazon Web Services, AWS)](https://aws.amazon.com/) and [Google (Google Cloud Platform, GCP)](https://cloud.google.com/). Together, they are often referred to as **\"public\"** or **\"commercial\"** cloud providers. \n", | ||
"\n", | ||
"\n", | ||
"For researchers, it is often helpful to be aware of these five key components related to cloud computing infrastructure: \n", | ||
"\n", | ||
"<img src=\"./images/cloudcomponents.svg\" width=600 align=\"center\"><br/> \n", | ||
"\n", | ||
"In contrast to buying your own desktop or laptop computer, a cluster of machines, or with buying external storage devices (such as a RAID, redundant array of independent disks), cloud computing allows you to provision computing and storage on machines that only available to you through an intermediated interface (such as a web-browser or through ssh). \n", | ||
"\n", | ||
"In some sense, cloud computing is akin to computing resources that would you obtain through institutional high-perfomance computing clusters or the National Science Foundation (NSF)-funded platforms such as [XSEDE](https://www.xsede.org/). However, the fundamental difference is that the commercial cloud follows a pay-as-you-go model as opposed to a funding allocation model which you need to apply and be approved for. Commercial cloud providers are also frequently inventing, updating and improving their catalog of services; some researchers may find that the services offered on the commercial cloud can help accelerate their scientific workflows especially when dealing with big data. \n", | ||
"\n", | ||
"We will explore more about the services offered by the public cloud vendors in the next few chapters. " | ||
] | ||
} | ||
], | ||
"metadata": { | ||
"kernelspec": { | ||
"display_name": "Python 3", | ||
"language": "python", | ||
"name": "python3" | ||
}, | ||
"language_info": { | ||
"codemirror_mode": { | ||
"name": "ipython", | ||
"version": 3 | ||
}, | ||
"file_extension": ".py", | ||
"mimetype": "text/x-python", | ||
"name": "python", | ||
"nbconvert_exporter": "python", | ||
"pygments_lexer": "ipython3", | ||
"version": "3.9.5" | ||
} | ||
}, | ||
"nbformat": 4, | ||
"nbformat_minor": 5 | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,49 @@ | ||
{ | ||
"cells": [ | ||
{ | ||
"cell_type": "markdown", | ||
"id": "1b6fcbc0", | ||
"metadata": {}, | ||
"source": [ | ||
"### Is the Cloud Right for Me?\n", | ||
"\n", | ||
"Now that you have an idea of what cloud computing is, the question that is most pertinent to researchers is whether or not you ***should*** move your work to the cloud. The following chart outlines some questions to ask: \n", | ||
"\n", | ||
"<img src=\"./images/acceleratescience.svg\" width=500 align=\"center\"><br/> \n", | ||
"\n", | ||
"Many researchers move to the commercial cloud simply because their local compute resources (local HPC clusters, or departmental clusters) are insufficient to deal with the volume of data and type of computation. With the cloud, there is no wait time to obtain the computing resources you need. With sufficient funds, you may be able to obtain a near infinite number of CPUs, RAM and GPUs. \n", | ||
"\n", | ||
"```{note}\n", | ||
"Scalable computing is the ability to obtain more computers(horizontal scaling) and/or to obtain more powerful computers(vertical scaling)\n", | ||
"```\n", | ||
"\n", | ||
"Other factors that will play in your decision to move to the cloud include knowing where common large datasets are hosted. The core idea in working on the cloud involves a paradigm shift: researchers should no longer bring their data to the compute (i.e. downloading data) but should instead bring their compute to the data! If you think about it, why spend hours on end to download data and find a place to store it when you can work directly with the data? \n", | ||
"\n", | ||
"If your collaborators are already working in the cloud i.e. hosting data on the cloud or building cloud-based workflows, it would also make sense for you to bring your work to the cloud. Go here for more on Open Science and Collaboration. \n", | ||
"\n", | ||
"In the next chapter, we will discuss the million dollar question: \"But which cloud provider should I choose?!\"" | ||
] | ||
} | ||
], | ||
"metadata": { | ||
"kernelspec": { | ||
"display_name": "Python 3", | ||
"language": "python", | ||
"name": "python3" | ||
}, | ||
"language_info": { | ||
"codemirror_mode": { | ||
"name": "ipython", | ||
"version": 3 | ||
}, | ||
"file_extension": ".py", | ||
"mimetype": "text/x-python", | ||
"name": "python", | ||
"nbconvert_exporter": "python", | ||
"pygments_lexer": "ipython3", | ||
"version": "3.9.5" | ||
} | ||
}, | ||
"nbformat": 4, | ||
"nbformat_minor": 5 | ||
} |
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
{ | ||
"cells": [ | ||
{ | ||
"cell_type": "markdown", | ||
"id": "a71a9499", | ||
"metadata": {}, | ||
"source": [ | ||
"# Funding Your Cloud Journey" | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": null, | ||
"id": "57bc18a6", | ||
"metadata": {}, | ||
"outputs": [], | ||
"source": [] | ||
} | ||
], | ||
"metadata": { | ||
"kernelspec": { | ||
"display_name": "Python 3", | ||
"language": "python", | ||
"name": "python3" | ||
}, | ||
"language_info": { | ||
"codemirror_mode": { | ||
"name": "ipython", | ||
"version": 3 | ||
}, | ||
"file_extension": ".py", | ||
"mimetype": "text/x-python", | ||
"name": "python", | ||
"nbconvert_exporter": "python", | ||
"pygments_lexer": "ipython3", | ||
"version": "3.9.5" | ||
} | ||
}, | ||
"nbformat": 4, | ||
"nbformat_minor": 5 | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
{ | ||
"cells": [ | ||
{ | ||
"cell_type": "markdown", | ||
"id": "dbc2e7e6", | ||
"metadata": {}, | ||
"source": [ | ||
"# Choosing Cloud Providers" | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": null, | ||
"id": "7ff14b2c", | ||
"metadata": {}, | ||
"outputs": [], | ||
"source": [] | ||
} | ||
], | ||
"metadata": { | ||
"kernelspec": { | ||
"display_name": "Python 3", | ||
"language": "python", | ||
"name": "python3" | ||
}, | ||
"language_info": { | ||
"codemirror_mode": { | ||
"name": "ipython", | ||
"version": 3 | ||
}, | ||
"file_extension": ".py", | ||
"mimetype": "text/x-python", | ||
"name": "python", | ||
"nbconvert_exporter": "python", | ||
"pygments_lexer": "ipython3", | ||
"version": "3.9.5" | ||
} | ||
}, | ||
"nbformat": 4, | ||
"nbformat_minor": 5 | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,20 +1,33 @@ | ||
<!-- #region --> | ||
#### Overview | ||
# A Primer to Cloud Computing for Research | ||
|
||
This online "book" aims to enable pathways to adoption of cloud computing for research. You will discover how the public cloud can foster innovation, accelerate scientific discovery and learn about best practices for leveraging the cloud. | ||
|
||
As a researcher or research facilitator, there are three main considerations in utilizing the cloud for scientific workflows: | ||
|
||
#### Issues | ||
Please contact class@internet2.edu with issues or questions | ||
**(1) How do I know if my research belongs in the cloud?** | ||
|
||
Content for this site was put together by the Research Engagement team at Internet2. [Find out more here](https://internet2.edu/community/research-engagement/internet2-research-engagement-team/)! | ||
**(2) Which cloud provider and which services should I utilize?** | ||
|
||
**(3) How do I estimate costs and resources needed for porting my work to the cloud?** | ||
|
||
<!-- #endregion --> | ||
Whether you are new to the cloud journey or are a seasoned cyberinfrastructure professional looking to communicate and address common concerns faced by researchers, this guide is intended to help you make an informed decision on whether a research workflow belongs in the public cloud, and to demystify some of the more obscure features around cloud adoption. | ||
|
||
```python | ||
|
||
``` | ||
## Cloud Adoption Framework | ||
|
||
```python | ||
```{attention} | ||
Cloud computing is a utility. You pay for the resources that you use! | ||
``` | ||
|
||
The CLASS Cloud Adoption Framework is a series of documentation, best practices, tutorials, workflows and cloud adoption pathways built by researchers and research and data (RCD) professionals ***for*** researchers and RCD professionals. | ||
|
||
## Issues | ||
Please contact class@internet2.edu with issues or questions | ||
|
||
|
||
Content for this site was put together by the Research Engagement team at Internet2. [Find out more here](https://internet2.edu/community/research-engagement/internet2-research-engagement-team/)! | ||
|
||
|
||
<!-- #endregion --> |
Oops, something went wrong.