README.md

GNU Mailman 3 for COmanage Registry Docker

What it is

Docker version of GNU Mailman 3 for use with COmanage Registry.

The instructions below detail how to build and then run a suite of services to deploy GNU Mailman 3 for use with a separate COmanage Registry deployment. The suite of services include:

1. mailman-core: GNU Mailman 3 core services including the REST API server.

2. mailman-web: GNU Mailman 3 web interface.

3. database: Relational database, currently PostgreSQL, required by GNU Mailman 3.

4. postfix: MTA needed for sending and receiving mail.

5. apache: Apache HTTP Server with Shibboleth SP as a web proxy for GNU Mailman 3 REST and web interface.

How To

• Install Docker. These instructions require version 17.03.1 or higher.

• The instructions below assume you are deploying the service stack using either Docker Swarm or Docker Compose. If you are deploying using Docker Compose then install the Docker Compose script. These instructions require version 1.13.0 or higher.

• Clone this repository:

git clone https://github.com/Internet2/comanage-registry-docker.git
cd comanage-registry-docker

• Build and tag the images used for each of the services (you may use your own repository instead of "sphericalcowgroup")

pushd comanage-registry-mailman/core
docker build -t sphericalcowgroup/mailman-core:0.2.1 .
popd

pushd comanage-registry-mailman/web
docker build -t sphericalcowgroup/mailman-web:0.2.1 .
popd

pushd comanage-registry-mailman/apache-shib
docker build -t sphericalcowgroup/mailman-core-apache-shib .
popd

pushd comanage-registry-mailman/postfix
docker build -t sphericalcowgroup/mailman-postfix .
popd

• Gather or create the following secrets and other information to be injected (be sure to substitute your own secrets and do not use the examples below):

  • A password for the PostgreSQL user, eg. gECPnaqXVID80TlRS5ZG.

  • An API key for GNU Mailman 3 Hyperkitty (web front end), eg. HbTKLdrhRxUX96f5bD2g.

  • A secret key used by Django for signing cookies, eg. fPe7d9e0PKF8ryySOow0.

  • A password for the GNU Mailman 3 REST user, eg. K6gfcC9uHQMXr448Kmdi.

  • An X.509 certificate for HTTPS for Apache HTTP Server (Apache). The server certificate and any subordinate CA signing certificates (except for the trust root) should be in a single file, eg. fullchain.pem.

  • The associated private key for the X.509 HTTPS certificate, eg. privkey.pem.

• Create the directory structure on the Docker engine hosts needed for the services to save local state, eg.

mkdir -p /srv/docker/opt/mailman/core
mkdir -p /srv/docker/opt/mailman/web
mkdir -p /srv/docker/opt/mailman/database
mkdir -p /srv/docker/opt/mailman/shib

• If you are using Docker Compose to deploy the service stack copy the file docker-compose.yaml. Review the services configuration. You MUST make at least the following changes (some environment variables are set for more than one service):

  • MAILMAN_DATABASE_URL: URL of the form postgres://mailman:PASSWORD@database/mailmandb where PASSWORD is the password for the PostgreSQL database.

  • HYPERKITT_API_KEY: API key for GNU Mailman 3 Hyperkitty (web front end)

  • MAILMAN_REST_PASSWORD: A password for the GNU Mailman 3 REST user

  • SERVE_FROM_DOMAIN: The domain name from which Django (web front end) will be served.

  • MAILMAN_ADMIN_EMAIL: The email address of the first GNU Mailman 3 administrator.

  • MAILMAN_WEB_SECRET_KEY: A secret key used by Django for signing cookies.

  • POSTGRES_PASSWORD: The PostgreSQL database password.

  • POSTFIX_MAILNAME: The domain name from which Postfix will be receiving and sending mail.

  • Copy the Nginx X.509 certificate chain file, private key, and DH param files and edit the volumes for the ngnix service as necessary to mount the files into the container.

• If you are using Docker Swarm to deploy the service stack copy the file mailman-stack.yml. Review the services configuration. Create the necessary Swarm secrets, eg.

echo "postgres://mailman:gECPnaqXVID80TlRS5ZG@database/mailmandb" | docker secret create mailman_database_url -
echo "HbTKLdrhRxUX96f5bD2g" | docker secret create hyperkitty_api_key -
echo "K6gfcC9uHQMXr448Kmdi" | docker secret create mailman_rest_password -
echo "fPe7d9e0PKF8ryySOow0" | docker secret create mailman_web_secret_key -
echo "gECPnaqXVID80TlRS5ZG" | docker secret create postgres_password -
docker secret create https_cert_file fullchain.pem
docker secret create https_key_file privkey.pem

Additionally you MUST also make at least the following changes to the stack compose file mailman-stack.yml:

• MAILMAN_ADMIN_EMAIL: The email address of the first GNU Mailman 3 administrator.

• POSTFIX_MAILNAME: The domain name from which Postfix will be receiving and sending mail.

• Start the services. If you are using Docker Compose then run

docker-compose up -d

If you are using Docker Swarm then run

docker stack deploy --compose-file mailman-stack.yml mailman

• It can take as long as 30 seconds for the GNU Mailman 3 core service to be ready. The other services wait until detecting that core is ready. Monitor the apache service with

docker-compose logs -f --tail=100 apache

or

docker service logs --tail=100 -f mailman_apache

until Apache is ready. You should see something like

2019-04-03 12:27:55,389 CRIT Set uid to user 0
2019-04-03 12:27:55,391 INFO supervisord started with pid 1
2019-04-03 12:27:56,394 INFO spawned: 'shibd' with pid 8
2019-04-03 12:27:56,399 INFO spawned: 'apache2' with pid 9
Waiting for Mailman core container...
2019-04-03 12:27:57,491 INFO success: shibd entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2019-04-03 12:27:57,492 INFO success: apache2 entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
Waiting for Mailman core container...
Waiting for Mailman core container...
Waiting for Mailman core container...
Waiting for Mailman web container...
Waiting for Mailman web container...
Waiting for Mailman web container...
[Wed Apr 03 13:48:41.263252 2019] [mpm_event:notice] [pid 9:tid 140569797922880] AH00489: Apache/2.4.38 (Unix) OpenSSL/1.1.0j configured -- resuming normal operations
[Wed Apr 03 13:48:41.284857 2019] [core:notice] [pid 9:tid 140569797922880] AH00094: Command line: 'httpd -D FOREGROUND'

• Browse to port 443 on the host and authenticate using an identity provider (IdP) federated with the Shibboleth SP.

• Visit the COmanage wiki to learn how to enable and configure the Mailman Provisioning Plugin for COmanage Registry.

• To stop the services:

docker-compose down

or

docker stack rm mailman

Useful commands for Docker Compose

docker-compose up -d 

docker-compose ps

docker-compose logs mailman-core

docker-compose logs database

docker-compose logs mailman-web

docker-compose logs postfix

docker-compose logs apache

docker-compose logs -f --tail=100 mailman-core

docker-compose logs -f --tail=100 database

docker-compose logs -f --tail=100 mailman-web

docker-compose logs -f --tail=100 postfix

docker-compose logs -f --tail=100 apache

docker-compose down

Useful commands for Docker Swarm

docker stack deploy --compose-file mailman-stack.yml mailman

docker stack ls

docker service ls

docker stack ps mailman

docker service logs mailman_mailman-core

docker service logs mailman_mailman-web

docker service logs mailman_apache

docker service logs mailman_database

docker service logs mailman_postfix

docker service logs --tail=100 -f mailman_mailman-core

docker service logs --tail=100 -f mailman_mailman-web

docker service logs --tail=100 -f mailman_apache

docker service logs --tail=100 -f mailman_postfix

docker stack rm mailman