added i2cc workflows for each CSP

i2cc-workflow/AWS-DirectConnect.md

@@ -0,0 +1,30 @@
+# Amazon Web Services
+
+AWS calls a hosted cloud connect connection a "hosted connection".
+
+Workflow:
+
+- Create a new connection in Virtual Networks (Choose "A cloud provider", then "AWS"), specifying an AWS account number that will hold the hosted connection.
+  - Select the region the hosted connection will directly connect to (us-east-1 (which we interconnect in Ashburn VA and Dallas TX), us-east-2 (we interconnect in Chicago) or us-west-1 (we interconnect in San Jose CA). Note that you can get to us-west-2 from any other region.
+  - Next pick an interface with sufficient capacity. Choosing different devices for multiple connections provides additional redundancy. **We ask that you pick a 10G interface if you are allocating less than a 1G hosted connection, and 100G otherwise** (where 100G is available - Ashburn and San Jose currently)
+  - Finally you are given the opportunity to select the bandwidth, and then BGP peering parameters for Virtual Router connections.
+  - When the virtual network is saved, the AWS Console associated with the AWS account number is offered a new hosted connection.
+  - The size of the AWS hosted connection is specified in the Virtual Networks console initially, and **cannot be changed**. You must create a new hosted connection to change the bandwidth.
+- Accept the hosted connection in the AWS console
+- Attach a AWS Virtual Interface (VIF) to the hosted connection. This will specify the IP addresses, local ASN (from a virtual gateway or direct connect gateway), and remote ASN (55038 for Internet2 connections with a virtual router), if BFD is going to be used, and if a BGP key is going to be used.
+
+> [!TIP]
+>Virtual Networks currently (Sept 2023) has the restriction that IPv4 and IPv6 BGP keys be identical. If you create both IPv4 and IPv6 peerings, if you let AWS choose the BGP key for the first peering, remember to copy it into the second peering you create for the other address family. If you let AWS choose both keys, they will be different and one will not come up.
+
+To change a BGP key within AWS, you need to delete the old peering, wait for it to actually delete, then add a new peering for that address family. If you do this, the other address family (the one not changed) peering will remain up and passing traffic. The delete is traffic impacting for the family deleted.
+
+If you delete and recreate an IPv6 peering, the addresses AWS chooses for the pering remain the same, even if the BGP key changes (at least it has in our tests). Be aware that it _could_ also change, though.
+:::
+
+- For connections to a virtual router, return to Virtual Networks and update the AWS peering with any information that was auto-generated by AWS (IP addresses, BGP key, ASN...) or updated
+  - The MTU must match the AWS VIF (1500 for standard Ethernet, 9001 for virtual private gateways with jumbo frames or 8500 for transit gateways with Jumbo frames)
+- You must select `Go Live` for the connection to be configured on the Internet2 Network.
+
+## Deprovisioning
+
+Before Virtual Networks can delete a hosted connection, the VIF must be deleted from the connection in the AWS Console. If this not done, an error will be shown in the Provisioner pane of the Details page for the connection.

i2cc-workflow/Google Cloud Partner Interconnect.md

@@ -0,0 +1,51 @@
+# Google Cloud Partner Interconnect
+
+Internet2 PoPs are not bound to particular GCP regions, unlike AWS or Oracle. Any GCP region is accessable from any Virtual Networks interconnect site (the point of presence, or PoP). Generally you create the VLAN attachment in the same region as your GCP Cloud Router. Choose the closest Internet2 PoP that is on the way to that GCP region from the connection to the Internet2 network, or another if you want additional resiliency from redundant attachments.
+
+| Internet2 PoP      | Google facility name | Low-latency region |
+| ------------------ | -------------------- | ------------------ |
+| ASHB - Ashburn VA  | iad-zone{1,2}-1      | us-east4           |
+| EQCH - Chicago IL  | ord-zone{1,2}-7      |                    |
+| DALL3 - Dallas TX  | dfw-zone{1,2}-4      | us-south1          |
+| SANJ - San Jose CA | sjc-zone{1,2}-6      |                    |
+
+See [Google region and zone documentation](https://cloud.google.com/compute/docs/regions-zones) and [Google colocation facility documentation](https://cloud.google.com/network-connectivity/docs/interconnect/concepts/choosing-colocation-facilities) for more information.
+
+Each Internet2 PoP has interconnects to two GCP zones. When you create a VLAN attachment in the UI, GCP chooses the zone, which is encoded in the pairing key. If you create a pair of VLAN attachments (which GCP recommends when you need the highest resiliency) it will create two pairing keys, each in a different zone. If the VLAN attachments are requested using the GCP API (or CLI), you can specify the zone or say that either zone is fine (and the key ends in "/any".
+
+Workflow:
+
+In the GCP Console:
+
+- Create a VLAN attachment (or a pair of VLAN attachments) in the Google Cloud console.
+  - Note that the MTU should match that of your network attached to the cloud router (1440, 1460, 1500 or 8896 bytes)
+  - Copy the pairing key(s).
+    - note that the ending of the key specifies the zone; /1 specifies zone 1, and /2 specifies zone 2. In the API, but not the console, you can also say "any zone", in which case /any specifies any zone.
+
+In the Virtual Networks console, do the following for each pairing key. If you created two VLAN attachments, we do not recommend connecting both to the same virtual switch. For virtual routers, standard practice is to connect both VLAN attachments to the same virtual router (which in turn will be connected two two different physical devices, one for each zone). You may also connect each VLAN attachment to a separate virtual router.
+
+- Use the pairing key in Virtual Networks after selecting cloud provier and Google Cloud Platform to create a connection to the associated VLAN attachment.
+- Select an interface that has enough remaining bandwidth free. **We ask that you select a 10G interface if you are requesting less than 1G of bandwidth, otherwise select a 100G interface**
+  - bandwidth is specified in Virtual Networks in the next step
+  - interfaces are offered that are in the Zone your key requests, but are unordered with respect to location. Typically you select an interface close to the region on the path from your connection to the Internet2 network to that region (although again the interfaces are not bound to region, so you can for example request an Ashburn VA interface to get to the us-west2 region).
+  - Select the VLAN you would like to use.
+  - Select the `Continue` arrow.
+  - For Virtual Switch connections you specify the bandwidth
+  - For Virtual Router connections you can specify bandwdith and whether or not you would like to use BFD, and the BGP key. The rest of the parameters are specified by GCP, and are read by Virtual Networks.
+    - You can update whether BFD is used and the BGP key later.
+    - In GCP, the BGP key is specified in the peering, so it may be easier to do in a subsequent step after you verify the peering comes up without a key.
+
+> [!TIP]
+> GCP allows different keys for IPv4 and IPv6 peerings, but Virtual Networks currently only allows a single key for both. Therefore if you allow GCP to pick a key for one peering, you need to copy that key to the other peering (if it exists) as well as copying it in to Virtual Networks.
+
+  - Select `Save Draft`; at this time GCP is contacted to find out the rest of the parameters associated with the service key, and to instantiate the VLAN attachment.
+  - When provisioning is finished, you will either see `Connection provisioned, please activate in your GCP account` or `[Provisioned]` if the connection was pre-activated (see below).
+  - You must select `Go Live` for the connection to be configured on the Internet2 Network.
+
+Back in the GCP Console:
+
+- Go back to the Google Cloud console and activate the VLAN attachment (acknowledging you are willing to pay for the specified bandwidth). You can also "pre-activate", and avoid this step.
+
+## Deprovisioning
+
+If you delete the connection in Virtual Networks, the state goes to `[GCP] [PENDING] - Tearing down connection`, and is removed after a few minutes. In the Google console, the VLAN attachment becomes `defunct`. You then need to delete the VLAN attachment.

i2cc-workflow/Microsoft Azure ExpressRoute.md

@@ -0,0 +1,49 @@
+# Microsoft Azure ExpressRoute
+
+> [!TIP]
+>Both Azure Government and Azure commercial ExpressRoute connections are completed by selecting Microsoft Azure as the cloud provider. Virtual Networks will query both provider sites and proceed using the site where the Service Key is valid.
+
+An Azure ExpressRoute has two connections to diverse routers by default. ExpressRoute calls these "Primary" and "Secondary", although they are treated like equal-cost multipath peers. The first time through the Virtual Neworks workflow, the primary connection is created. You re-use the Service Key and go through the workflow a second time to create the secondary connection. (Creating the secondary connection is not required, but is encouraged for resiliency.)
+
+We do not recommend connecting the primary and secondary connections to the same virtual switch. For virtual routers, standard practice is to connect the primary and secondary to the same virtual router (although using separate virtual routers is also possible).
+
+Workflow:
+
+In the Azure Portal:
+
+- Create an ExpressRoute in the Azure portal. Choose Internet2 as the provider, and the Peering Location where the ExpressRoute is created.
+
+  | Azure Peering Location | Internet2 PoP       |
+  | ---------------------- | ------------------- |
+  | Washington DC          | Ashburn VA (ASHB)   |
+  | Chicago                | Chicago IL (EQCH)   |
+  | Dallas                 | Dallas TX (DALL3)   |
+  | Silicon Valley         | San Jose, CA (SANJ) |
+
+  - The size of the ExpressRoute connection is only specified inside the Azure portal. It can be raised, but never lowered in the future. (One can always create a new ExpressRoute with a lower bandwidth and delete the old ExpressRoute.)
+
+- Copy the "Service Key" once the ExpressRoute circuit has been created
+
+In Virtual Networks (repeat for the secondary connection):
+
+- Use the Service Key in Virtual Networks after selecting cloud provider and Microsoft Azure to create a connection to an ExpressRoute interface
+- Azure dictates which interface is used
+- For Virtual Switch connections, select the `Continue` arrow, and then you can optionally enter a description, then cick `Save Draft`
+- For Virtual Router connections, you can enter an inner VLAN ID (this is arbitrary, so it can be anything from 1 to 4095; often schools match a VLAN number facing the school), select the `Continue` arrow, and then you are brought to a pane to configure the BGP peering information. Enter the information and then select `Save Draft` to continue. **Virtual Networks will only configure private peering for ExpressRoute Virtual Router connections.**
+- Azure is contacted as soon as the draft is saved to find the peering location, interface used, and bandwidth configured.
+- You must select `Go Live` for the connection to be configured on the Internet2 Network.
+
+**Note:** When configuring IPv4 peer addresses, Azure expects a /30 for both the primary and secondary connections. The first address will be used by the peer and the second will be used by Azure. For example, if 192.2.0.248/30 is used, 192.2.0.249/30 will be used by the peer and 192.2.0.250/30 will be used by Azure.
+
+Similarly, when configuring IPv6 peer addresses, Azure expects a /126 for both the primary and secondary connections (not a /64). The first usable address is used for your equipment (or Internet2) and the second useable address is used by Azure. For example, if 2001:db8:468::0/126 is used, 2001:db8:468::1/126 will be used by the peer and 2001:db8:468::2/126 will be used by Azure. For more information, as of this writing see [IP Addresses Used for Azure Peering](https://learn.microsoft.com/en-us/azure/expressroute/expressroute-routing#ip-addresses-used-for-peering).
+
+> [!WARNING]
+> As of this writing Azure does not allow configuring a BGP Shared Key, so that is not available in Virtual Networks.
+
+## Deprovisioning
+
+In order for Virtual Networks to be able to deprovision an ExpressRoute connection, all attachments must be removed from the ExpressRoute connection in the Azure console. This includes any virtual gateways, Global Reach connections, and even authorizations on the ExpressRoute circuit. Only then will Virtual Networks be allowed to deprovision the ExpressRoute connection completely.
+
+Once that is done, you are able to delete the ExpressRoute in the Azure console and stop Azure billing.
+
+Unlike other providers, Azure allows a deprovisioned ExpressRoute to be used again if it is not deleted. To reuse the ExpressRoute, start the workflow over from the top, once again using the Service Key for the ExpressRoute circuit.

i2cc-workflow/Oracle Cloud FastConnect.md

@@ -0,0 +1,38 @@
+# Oracle Cloud FastConnect
+
+> [!TIP]
+>Both Oracle Government cloud (OC2) and Oracle commercial cloud (OC1) FastConnect connections are completed by selecting Oracle as the cloud provider. The cloud type is encoded in the FastConnect OCID, so Virtual Networks will use the site where the FastConnect OCID is valid.
+
+A note on peering addresses:
+
+- Oracle IPv4 BGP peering addresses are in a block with subnet mask ranging from `/28` to `/31`. See the instructions in [FastConnect: With an Oracle Partner](https://docs.oracle.com/en-us/iaas/Content/Network/Concepts/fastconnectprovider.htm#FastConnect_With_an_Oracle_Partner)
+- Oracle IPv6 BGP peering addresses must be in a block with a subnet mask of `/64`, `/96`, `/126` or `/127`. See [FastConnect and IPv6](https://docs.oracle.com/en-us/iaas/Content/Network/Concepts/ipv6.htm#fastconnect)
+
+Workflow:
+
+In the Oracle Cloud console:
+
+- Under Networking>FastConnect, create a new FastConnect (FastConnect Partner) using either `Internet2: Internet2 L2` or `Internet2: Internet2 L3` as the provider, depending on if you want a Layer 2 (connection to a Virtual Switch) or Layer 3 (connection to a Virtual Router) virtual network within Internet2.
+  - For Layer 2 you will be asked to input all BGP parameters in the Oracle console
+    - IPv4 settings are required, IPv6 settings are optional
+    - The BGP key will be the same for IPv4 and IPv6
+  - For Layer 3 all BGP parameters are specified within Virtual Networks
+  - Choose Private virtual circuit
+  - Select the desired bandwidth and MTU
+    - bandwidth is also specified in the Virtual Networks console; they should match, but if they differ the Virtual Networks specification wins.
+    - if you later edit the bandwidth in the Oracle Console, Virtual Networks should pick up the change within an hour
+- Copy the OCID of the FastConnect and wait for the virtual circuit (VC) to go to PENDING PARTNER from PROVISIONING.
+
+Similar to Azure, for Layer 3 connections you can re-use the FastConnect OCID once and create a secondary connection. If you want a resilient Layer 2 connection you need to create a second FastConnect. The BFD setting is common between the primary and secondary; the BGP key could be different between primary and secondary (but is common between any IPv4 and IPv6 peering on the same connection).
+
+In the Virtual Networks section of Internet2 Insight Console:
+
+- Create a new connection to A cloud provider, choose Oracle, and provide the FastConnect OCID.
+- Select the `Continue` Arrow
+- Select the region the FastConnect was created in
+- Choose the interface and VLAN you would like to use (typically this is the VLAN toward your infrastructure, but there is no need for it to be the same, the Internet2 Network will translate), and select the `Continue` arrow again
+- for Virtual Router (Layer 3) connections, Specify the IPv4 and optionally IPv6 peering addresses, subject to the rules above. The MTU and max bandwidth of the connection are specified as well; note that the bandwidth specified here should match the bandwidth specified in the Oracle console, and if it is different it overrides that in the Oracle console (although you can also update the bandwidth of an existing connection in the Oracle console). You can also specify a BGP key.
+
+## Deprovisioning
+
+Deleting a connection in Virtual Networks will result in the Oracle FastConnect VC terminating, ending in it being completely deleted as well.