Skip to content
Permalink
Browse files
First commit.
First commit.
  • Loading branch information
skoranda committed Nov 7, 2019
0 parents commit e19dbe2284a300d05ab1d8e32b32f66bb81babd7
Showing with 4,065 additions and 0 deletions.
  1. +9 −0 .gitignore
  2. +215 −0 README.md
  3. +494 −0 ansible.cfg
  4. +25 −0 comanage_registry_training.yml
  5. +222 −0 ec2.ini
  6. +11 −0 hostnames.yml
  7. +92 −0 idp_node.yml
  8. +5 −0 roles/common/handlers/main.yml
  9. +16 −0 roles/common/tasks/main.yml
  10. +72 −0 roles/common/tasks/users.yml
  11. +32 −0 roles/idp/files/attribute-filter.xml
  12. +60 −0 roles/idp/files/attribute-resolver.xml
  13. +155 −0 roles/idp/files/config-always-01.ldif
  14. +7 −0 roles/idp/files/config-always-all-olcAccess.ldif
  15. +26 −0 roles/idp/files/idp-encryption.crt
  16. +130 −0 roles/idp/files/idp-encryption.key
  17. +26 −0 roles/idp/files/idp-signing.crt
  18. +130 −0 roles/idp/files/idp-signing.key
  19. +17 −0 roles/idp/files/metadata-providers.xml
  20. +60 −0 roles/idp/files/relying-party.xml
  21. BIN roles/idp/files/sealer.jks
  22. +24 −0 roles/idp/files/server.xml
  23. +63 −0 roles/idp/files/shibboleth-idp-stack.yml
  24. +166 −0 roles/idp/tasks/main.yml
  25. +212 −0 roles/idp/templates/idp.properties
  26. +63 −0 roles/idp/templates/ldap.properties
  27. +92 −0 roles/idp/templates/registry-metadata.xml
  28. +23 −0 roles/idp/vars/main.yml
  29. +3 −0 roles/swarm/files/daemon.json
  30. +11 −0 roles/swarm/handlers/main.yml
  31. +101 −0 roles/swarm/tasks/main.yml
  32. +11 −0 roles/training/files/attribute-map.xml
  33. +112 −0 roles/training/files/comanage-registry-stack.yml
  34. +29 −0 roles/training/files/config-always-01.ldif
  35. +7 −0 roles/training/files/config-always-all-olcAccess.ldif
  36. +80 −0 roles/training/files/idp-metadata.xml
  37. +53 −0 roles/training/files/shibboleth2.xml
  38. +127 −0 roles/training/tasks/main.yml
  39. +53 −0 roles/training/templates/000-comanage.conf
  40. +452 −0 roles/training/vars/main.yml
  41. +121 −0 ssh_bastion.yml
  42. +21 −0 ssh_config.j2
  43. +252 −0 training_nodes.yml
  44. +48 −0 vars/global.yml
  45. +137 −0 vpc.yml
@@ -0,0 +1,9 @@
AWS-Trng-1.pem
bin
include
lib
share
ssh_config
ssh_mux*
ec2.py
.*.swp
215 README.md
@@ -0,0 +1,215 @@
# Ansible Deployment for InCommon COmanage Registry Training

This repository contains the necessary Ansible and other files for
deploying the InCommon COmanage Registry Training environment.

The primary Ansible playbook when run will create

* a AWS Virtual Private Cloud (VPC) with the name `comanage_training`.
All infrastructure is created within the VPC and can be deprovisioned by
deleting the VPC.

* an internet gateway (IG) to connect the VPC to the internet.

* public and private subnets within the VPC.

* NATs to allow virtual machines in the private subnets to open
connections to the internet (e.g. to execute `yum update`).

* appropriate security groups.

* SSH bastion hosts (one per public subnet).

* a host for a Shibboleth IdP. The IdP is deployed using the TAP image
and a Docker Swarm service stack (compose) file, and includes an LDAP server
pre-populated with user accounts for SAML authentication.

* N hosts for trainees. Each host is a single-node Docker Swarm
pre-populated with most details necessary for deploying COmanage Registry
using the TAP image.

* Target groups and an application load balancer (ALB) that terminates
TLS and is configured to route web traffic to the IdP and the COmanage
Registry hosts.

* Route53 DNS configurations so that the IdP and the training nodes can
all be easily reached.

## Secrets

There are no unencrypted secrets in this repository. All secrets,
including SAML keys, are encrypted using the Ansible vault tooling.
Refer to the Ansible documentation for details on how to manage the
encrypted files and strings.

## Prerequisites

You will need to have an AWS access key and AWS secret access key provisioned
by an administrator for the internet2-training AWS account.

You will need to have the Ansible vault password used with this ansible
deployment.

You will need to have the AWS-Trng-1.pem (or other approved key) used
for the initial login access to virtual machines.

You will need to use the AWS Console to access the Certificate Manager
and provision (or renew) an X.509 wildcard certificate for the domain
`*.comanage.incommon.training`.

## Set up Environment

To set up the environment for ansible the first time:

```
git clone https://github.com/cilogon/comanage-registry-ansible.git
cd comanage-registry-training-deployment
virtualenv -p python3.7 ./
source bin/activate
pip install --upgrade pip
pip install git+https://github.com/ansible/ansible.git@devel
pip install boto
pip install boto3
wget https://raw.githubusercontent.com/ansible/ansible/devel/contrib/inventory/ec2.py
chmod 755 ec2.py
cp /path/to/AWS-Trng-1.pem .
```

Some ansible files are encrypted using `ansible-vault`. When running
a playbook ansible needs to be able to find the password for the
vault.

Create a file outside of the clone of this repository to hold
the vault password, e.g.

```
touch ~/.vault_pass.txt
chmod 600 ~/.vault_pass.txt
```
Find the vault password from and enter it into the file you just created.

## Initialization Before Running Playbooks

Do this each time to run ansible commands or playbooks
to set up the environment:

```
cd comanage-registry-training-deployment
source bin/activate
export AWS_ACCESS_KEY_ID='XXXXXXXX'
export AWS_SECRET_ACCESS_KEY='XXXXXXXX'
export ANSIBLE_VAULT_PASSWORD_FILE=~/.vault_pass.txt
export AWS_REGION=us-west-2
rm ./ssh_mux_*
kill $SSH_AGENT_PID
unset SSH_AUTH_SOCK
eval `ssh-agent -s`
ssh-add ./AWS-Trng-1.pem
```

## Configuration

Most of the configurable details, including the number of training nodes to
deploye, are set in the file

```
vars/global.yml
```

Review that file before running the playbook.


## Provision the COmanage Training Infrastructure

To provision the infrastructure execute the playbook:

```
ansible-playbook -i ./ec2.py comanage_registry_training.yml
```

## SSH Access

Trainers may use their provisioned SSH keys to access all nodes. Each trainer
has a dedicated account on each node.

Trainees may SSH using the account `training` and the provisioned password.

Begin by logging into the bastion node, e.g.

```
$ ssh training@ssh.comanage.incommon.training
training@ssh.comanage.incommon.training's password:
Last login: Thu Nov 7 15:12:40 2019 from some/host
[training@ssh ~]$
```

From there each trainee may SSH into their assigned host:

```
[training@ssh ~]$ ssh registry1-private
training@registry1-private's password:
Last login: Thu Nov 7 17:43:27 2019 from ip-192-168-10-10.us-west-2.compute.internal
[training@registry1-private ~]$
```

Only trainers may SSH into the IdP node:

```
skoranda@paprika:~$ ssh -A ssh.comanage.incommon.training
Last login: Thu Nov 7 15:01:48 2019 from some.host
[skoranda@ssh ~]$ ssh login-private
Last login: Thu Nov 7 17:43:56 2019 from ip-192-168-10-10.us-west-2.compute.internal
```

## Deploying the IdP

The Ansible tooling does not automatically start the IdP service stack.
To start the stack log into the IdP node and execute

```
docker stack deploy --compose-file /opt/shibboleth-idp-stack.yml idp
```

Useful Docker Swarm commands for the IdP node are

```
docker stack ls
docker service ls
docker service ps idp_shibboleth-idp
docker service ps idp_ldap
docker service logs -f idp_shibboleth-idp
docker service logs -f idp_ldap
docker stack rm idp
```

## Deploying COmanage Registry

Each trainee is expected to SSH to the bastion host and then to their
assigned node. In the home directory for the `training` user the trainee
will find the Docker Swarm services stack (compose) file for deploying
COmanage Registry, a MariaDB database, and an LDAP server.

Before deploying the service stack the trainee must first, as an exercise,
create some Docker Swarm secrets (see the training materials for details).
Most secrets have been pre-populated using Ansible to save time, but the
trainee is expected to create a few secrets.

Once successfully deployed, COmanage Registry is available at the URL

```
https://registry1.comanage.incommon.training
```

for node 1, and

```
https://registry2.comanage.incommon.training
```

for node 2, and so on.

0 comments on commit e19dbe2

Please sign in to comment.