diff --git a/comanage-registry-mailman/README.md b/comanage-registry-mailman/README.md new file mode 100644 index 0000000..bdaba4f --- /dev/null +++ b/comanage-registry-mailman/README.md @@ -0,0 +1,285 @@ + + +# GNU Mailman 3 for COmanage Registry Docker + +## What it is +Docker version of [GNU Mailman 3](http://www.list.org/) for use with +[COmanage Registry](https://spaces.internet2.edu/display/COmanage/Home). + +The instructions below detail how to build and then run a suite of +services to deploy GNU Mailman 3 for use with a separate COmanage Registry +deployment. The suite of services include: + +1. **mailman-core**: GNU Mailman 3 core services including the REST API server. + +1. **mailman-web**: GNU Mailman 3 web interface. + +1. **database**: Relational database, currently PostgreSQL, required by GNU Mailman 3. + +1. **postfix**: MTA needed for sending and receiving mail. + +1. **nginx**: Web proxy for GNU Mailman 3 REST and web interface. + +## How To + +* Install Docker. These instructions require version 17.03.1 or higher. + +* The instructions below assume you are deploying the service stack using either Docker Swarm +or [Docker Compose](https://docs.docker.com/compose/). If you are deploying using Docker Compose then +install the Docker Compose script. These instructions require version 1.13.0 or higher. + +* Clone this repository: + +``` +git clone https://github.com/Internet2/comanage-registry-docker.git +cd comanage-registry-docker +``` + +* Build and tag the images used for each of the services +(you may use your own repository instead of "sphericalcowgroup") + +``` +pushd comanage-registry-mailman/core +docker build -t sphericalcowgroup/mailman-core:0.1.7 . +popd + +pushd comanage-registry-mailman/web +docker build -t sphericalcowgroup/mailman-web:0.1.7 . +popd + +pushd comanage-registry-mailman/nginx +docker build -t sphericalcowgroup/mailman-core-nginx . +popd + +pushd comanage-registry-mailman/postfix +docker build -t sphericalcowgroup/mailman-postfix . +popd + +``` + +* Gather or create the following secrets and other information to be injected (be sure +to substitute your own secrets and do not use the examples below): + + * A password for the PostgreSQL user, eg. `gECPnaqXVID80TlRS5ZG`. + + * An API key for GNU Mailman 3 Hyperkitty (web front end), eg. `HbTKLdrhRxUX96f5bD2g`. + + * A secret key used by Django for signing cookies, eg. `fPe7d9e0PKF8ryySOow0`. + + * A password for the GNU Mailman 3 REST user, eg. `K6gfcC9uHQMXr448Kmdi`. + + * An X.509 certificate for HTTPS for Nginx. The server certificate and any subordinate +CA signing certificates (except for the trust root) should be in a single file, eg. `fullchain.pem`. + + * The associated private key for the X.509 HTTPS certificate, eg. `privkey.pem`. + + * A DH parameters file for Nginx. You can generate one by doing + +``` +openssl dhparam -out dhparam.pem 2048 +``` + +* Create the directory structure on the Docker engine hosts needed for the services +to save local state, eg. + +``` +mkdir -p /opt/mailman/core +mkdir -p /opt/mailman/web +mkdir -p /opt/mailman/database +mkdir -p /opt/mailman/nginx +``` + +* If you are using Docker Compose to deploy the service stack copy the file +`docker-compose.yaml`. Review the services configuration. You MUST make at least the following +changes (some environment variables are set for more than one service): + + * `MAILMAN_DATABASE_URL`: URL of the form `postgres://mailman:PASSWORD@database/mailmandb` where `PASSWORD` is the + password for the PostgreSQL database. + + * `HYPERKITT_API_KEY`: API key for GNU Mailman 3 Hyperkitty (web front end) + + * `MAILMAN_REST_PASSWORD`: A password for the GNU Mailman 3 REST user + + * `SERVE_FROM_DOMAIN`: The domain name from which Django (web front end) will be served. + + * `MAILMAN_ADMIN_EMAIL`: The email address of the first GNU Mailman 3 administrator. + + * `MAILMAN_WEB_SECRET_KEY`: A secret key used by Django for signing cookies. + + * `POSTGRES_PASSWORD`: The PostgreSQL database password. + + * `POSTFIX_MAILNAME`: The domain name from which Postfix will be receiving and sending mail. + + * Copy the Nginx X.509 certificate chain file, private key, and DH param files and edit the + `volumes` for the `ngnix` service as necessary to mount the files into the container. + +* If you are using Docker Swarm to deploy the service stack copy the file `mailman-stack.yml`. +Review the services configuration. Create the necessary Swarm secrets, eg. + +``` +echo "postgres://mailman:gECPnaqXVID80TlRS5ZG@database/mailmandb" | docker secret create mailman_database_url - +echo "HbTKLdrhRxUX96f5bD2g" | docker secret create hyperkitty_api_key - +echo "K6gfcC9uHQMXr448Kmdi" | docker secret create mailman_rest_password - +echo "fPe7d9e0PKF8ryySOow0" | docker secret create mailman_web_secret_key - +echo "gECPnaqXVID80TlRS5ZG" | docker secret create postgres_password - +docker secret create nginx_https_cert_file fullchain.pem +docker secret create nginx_https_key_file privkey.pem +docker secret create nginx_dh_param_file dhparam.pem +``` + +Additionally you MUST also make at least the following changes to the stack compose file `mailman-stack.yml`: + + * `MAILMAN_ADMIN_EMAIL`: The email address of the first GNU Mailman 3 administrator. + + * `POSTFIX_MAILNAME`: The domain name from which Postfix will be receiving and sending mail. + +* Start the services. If you are using Docker Compose then run + +``` +docker-compose up -d +``` + +If you are using Docker Swarm then run + +``` +docker stack deploy --compose-file mailman-stack.yml mailman +``` + +* It can take as long as 30 seconds for the GNU Mailman 3 core service to be ready. The other +services wait until detecting that core is ready. Monitor the `nginx` service with + +``` +docker-compose logs -f --tail=100 nginx +``` + +or + +``` +docker service logs --tail=100 -f mailman_nginx +``` + +until Nginx is ready. You should see something like + +``` +mailman-nginx | Waiting for Mailman core container... +mailman-nginx | Waiting for Mailman core container... +mailman-nginx | Waiting for Mailman core container... +mailman-nginx | Waiting for Mailman core container... +mailman-nginx | 2018/04/23 18:07:27 [notice] 1#1: using the "epoll" event method +mailman-nginx | 2018/04/23 18:07:27 [notice] 1#1: nginx/1.10.3 +mailman-nginx | 2018/04/23 18:07:27 [notice] 1#1: OS: Linux 4.9.0-6-amd64 +mailman-nginx | 2018/04/23 18:07:27 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576 +mailman-nginx | 2018/04/23 18:07:27 [notice] 1#1: start worker processes +mailman-nginx | 2018/04/23 18:07:27 [notice] 1#1: start worker process 48 +``` + +* Browse to port 443 on the host. + +* Click `Login` and then `Forgot Password?`. Enter the email address you injected as the first +Mailman 3 administrator and then click `Reset My Password`. + +* You will receive an email at that administrator password with a link. Follow the link to reset +the administrator password. + +* Browse again to port 443 on the host and click `Login`. Enter the administrator name you injected +and the password you just set. Click `Sign In`. You will be sent another email with a link in it to verify the account. +Follow the link to verify the account. + +* Browse again to port 443 on the host and click `Login`. Enter the administrator name and password +you just verified. Click `Sign In`. + +* Visit the [COmanage wiki](https://spaces.internet2.edu/display/COmanage/Mailman+Provisioning+Plugin) +to learn how to enable and configure the Mailman Provisioning Plugin for COmanage Registry. + +* To stop the services: + +``` +docker-compose down +``` + +or + +``` +docker stack rm mailman +``` + +## Useful commands for Docker Compose + +``` +docker-compose up -d + +docker-compose ps + +docker-compose logs mailman-core + +docker-compose logs database + +docker-compose logs mailman-web + +docker-compose logs postfix + +docker-compose logs nginx + +docker-compose logs -f --tail=100 mailman-core + +docker-compose logs -f --tail=100 database + +docker-compose logs -f --tail=100 mailman-web + +docker-compose logs -f --tail=100 postfix + +docker-compose logs -f --tail=100 nginx + +docker-compose down +``` + +## Useful commands for Docker Swarm + +``` +docker stack deploy --compose-file mailman-stack.yml mailman + +docker stack ls + +docker service ls + +docker stack ps mailman + +docker service logs mailman_mailman-core + +docker service logs mailman_mailman-web + +docker service logs mailman_nginx + +docker service logs mailman_database + +docker service logs mailman_postfix + +docker service logs --tail=100 -f mailman_mailman-core + +docker service logs --tail=100 -f mailman_mailman-web + +docker service logs --tail=100 -f mailman_nginx + +docker service logs --tail=100 -f mailman_postfix + +docker stack rm mailman +``` diff --git a/comanage-registry-mailman/core/Dockerfile b/comanage-registry-mailman/core/Dockerfile new file mode 100644 index 0000000..1d62ffc --- /dev/null +++ b/comanage-registry-mailman/core/Dockerfile @@ -0,0 +1,62 @@ +# GNU Mailman 3 for COmanage Registry Dockerfile +# +# Portions licensed to the University Corporation for Advanced Internet +# Development, Inc. ("UCAID") under one or more contributor license agreements. +# See the NOTICE file distributed with this work for additional information +# regarding copyright ownership. +# +# UCAID licenses this file to you under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with the +# License. You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +FROM python:3.6-stretch + + +#Install all required packages, add user for executing mailman and set execution rights for startup script +RUN apt-get update \ + && apt-get install -y \ + gcc \ + libcurl4-openssl-dev \ + libmariadbclient-dev \ + libpq-dev \ + postgresql-client \ + python3-dev \ + && pip install psycopg2 \ + mailman==3.1.1 \ + mailman-hyperkitty==1.1.0 \ + pymysql \ + && adduser --system mailman + +COPY su-exec.c /usr/local/src/ +COPY Makefile /usr/local/src/ + +RUN cd /usr/local/src \ + && make \ + && cp su-exec /usr/local/bin/ \ + && chmod 755 /usr/local/bin/su-exec + +# Change the working directory. +WORKDIR /opt/mailman + +#Add startup script to container +COPY docker-entrypoint.sh /usr/local/bin/ + +#Expose the ports for the api (8001) and lmtp (8024) +EXPOSE 8001 8024 + +ENV MAILMAN_CONFIG_FILE /etc/mailman.cfg + +# Patch REST API for COmanage provisioning functionality until patches +# are accepted upstream. +COPY addresses.py /usr/local/lib/python3.6/site-packages/mailman/rest/addresses.py +COPY users.py /usr/local/lib/python3.6/site-packages/mailman/rest/users.py + +ENTRYPOINT ["docker-entrypoint.sh"] +CMD ["master", "--force"] diff --git a/comanage-registry-mailman/core/Makefile b/comanage-registry-mailman/core/Makefile new file mode 100644 index 0000000..9af2c73 --- /dev/null +++ b/comanage-registry-mailman/core/Makefile @@ -0,0 +1,18 @@ +# Taken from https://github.com/ncopa/su-exec + +CFLAGS ?= -Wall -Werror -g +LDFLAGS ?= + +PROG := su-exec +SRCS := $(PROG).c + +all: $(PROG) + +$(PROG): $(SRCS) + $(CC) $(CFLAGS) -o $@ $^ $(LDFLAGS) + +$(PROG)-static: $(SRCS) + $(CC) $(CFLAGS) -o $@ $^ -static $(LDFLAGS) + +clean: + rm -f $(PROG) $(PROG)-static diff --git a/comanage-registry-mailman/core/addresses.py b/comanage-registry-mailman/core/addresses.py new file mode 100644 index 0000000..a249053 --- /dev/null +++ b/comanage-registry-mailman/core/addresses.py @@ -0,0 +1,283 @@ +# Copyright (C) 2011-2017 by the Free Software Foundation, Inc. +# +# This file is part of GNU Mailman. +# +# GNU Mailman is free software: you can redistribute it and/or modify it under +# the terms of the GNU General Public License as published by the Free +# Software Foundation, either version 3 of the License, or (at your option) +# any later version. +# +# GNU Mailman is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or +# FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for +# more details. +# +# You should have received a copy of the GNU General Public License along with +# GNU Mailman. If not, see . +# +# This file includes patches needed for integration with COmanage Registry. +# Inclusion of the patches in the upstream GNU Mailman distribution is pending. + +"""REST for addresses.""" + +from lazr.config import as_boolean +from mailman.interfaces.address import ( + ExistingAddressError, InvalidEmailAddressError) +from mailman.interfaces.usermanager import IUserManager +from mailman.rest.helpers import ( + BadRequest, CollectionMixin, NotFound, bad_request, child, created, etag, + no_content, not_found, okay) +from mailman.rest.members import MemberCollection +from mailman.rest.preferences import Preferences +from mailman.rest.validator import Validator +from mailman.utilities.datetime import now +from operator import attrgetter +from public import public +from zope.component import getUtility + + +class _AddressBase(CollectionMixin): + """Shared base class for address representations.""" + + def _resource_as_dict(self, address): + """See `CollectionMixin`.""" + # The canonical url for an address is its lower-cased version, + # although it can be looked up with either its original or lower-cased + # email address. + representation = dict( + email=address.email, + original_email=address.original_email, + registered_on=address.registered_on, + self_link=self.api.path_to('addresses/{}'.format(address.email)), + ) + # Add optional attributes. These can be None or the empty string. + if address.display_name: + representation['display_name'] = address.display_name + if address.verified_on: + representation['verified_on'] = address.verified_on + if address.user: + uid = self.api.from_uuid(address.user.user_id) + representation['user'] = self.api.path_to('users/{}'.format(uid)) + if address == address.user.preferred_address: + representation['preferred'] = True + return representation + + def _get_collection(self, request): + """See `CollectionMixin`.""" + return sorted(getUtility(IUserManager).addresses, + key=attrgetter('original_email')) + + +@public +class AllAddresses(_AddressBase): + """The addresses.""" + + def on_get(self, request, response): + """/addresses""" + resource = self._make_collection(request) + okay(response, etag(resource)) + + +class _VerifyResource: + """A helper resource for verify/unverify POSTS.""" + + def __init__(self, address, action): + self._address = address + self._action = action + assert action in ('verify', 'unverify') + + def on_post(self, request, response): + # We don't care about the POST data, just do the action. + if self._action == 'verify' and self._address.verified_on is None: + self._address.verified_on = now() + elif self._action == 'unverify': + self._address.verified_on = None + no_content(response) + + +@public +class AnAddress(_AddressBase): + """An address.""" + + def __init__(self, email): + """Get an address by either its original or lower-cased email. + + :param email: The email address of the `IAddress`. + :type email: string + """ + self._address = getUtility(IUserManager).get_address(email) + + def on_get(self, request, response): + """Return a single address.""" + if self._address is None: + not_found(response) + else: + okay(response, self._resource_as_json(self._address)) + + def on_delete(self, request, response): + if self._address is None: + not_found(response) + else: + getUtility(IUserManager).delete_address(self._address) + no_content(response) + + @child() + def memberships(self, context, segments): + """/addresses//memberships""" + if len(segments) != 0: + return NotFound(), [] + if self._address is None: + return NotFound(), [] + return AddressMemberships(self._address) + + @child() + def preferences(self, context, segments): + """/addresses//preferences""" + if len(segments) != 0: + return NotFound(), [] + if self._address is None: + return NotFound(), [] + child = Preferences( + self._address.preferences, + 'addresses/{}'.format(self._address.email)) + return child, [] + + @child() + def verify(self, context, segments): + """/addresses//verify""" + if len(segments) != 0: + return BadRequest(), [] + if self._address is None: + return NotFound(), [] + child = _VerifyResource(self._address, 'verify') + return child, [] + + @child() + def unverify(self, context, segments): + """/addresses//verify""" + if len(segments) != 0: + return BadRequest(), [] + if self._address is None: + return NotFound(), [] + child = _VerifyResource(self._address, 'unverify') + return child, [] + + @child() + def user(self, context, segments): + """/addresses//user""" + if self._address is None: + return NotFound(), [] + # Avoid circular imports. + from mailman.rest.users import AddressUser + return AddressUser(self._address) + + +@public +class UserAddresses(_AddressBase): + """The addresses of a user.""" + + def __init__(self, user): + super().__init__() + self._user = user + + def _get_collection(self, request): + """See `CollectionMixin`.""" + return sorted(self._user.addresses, + key=attrgetter('original_email')) + + def on_get(self, request, response): + """/addresses""" + assert self._user is not None + okay(response, etag(self._make_collection(request))) + + def on_post(self, request, response): + """POST to /addresses + + Add a new address to the user record. + """ + assert self._user is not None + + preferred = None + + user_manager = getUtility(IUserManager) + validator = Validator(email=str, + display_name=str, + preferred=as_boolean, + absorb_existing=bool, + _optional=('display_name', 'absorb_existing', 'preferred')) + try: + data = validator(request) + + # We cannot set the address to be preferred when it is + # created so remove it from the arguments here and + # set it below. + preferred = data.pop('preferred', False) + except ValueError as error: + bad_request(response, str(error)) + return + absorb_existing = data.pop('absorb_existing', False) + try: + address = user_manager.create_address(**data) + except InvalidEmailAddressError: + bad_request(response, b'Invalid email address') + return + except ExistingAddressError: + # If the address is not linked to any user, link it to the current + # user and return it. Since we're linking to an existing address, + # ignore any given display_name attribute. + address = user_manager.get_address(data['email']) + if address.user is None: + address.user = self._user + location = self.api.path_to( + 'addresses/{}'.format(address.email)) + created(response, location) + return + elif not absorb_existing: + bad_request(response, 'Address belongs to other user') + return + else: + # The address exists and is linked but we can merge the users. + address = user_manager.get_address(data['email']) + self._user.absorb(address.user) + else: + # Link the address to the current user and return it. + address.user = self._user + + # Set the preferred address here if we were signalled to do so. + if preferred: + address.verified_on = now() + self._user.preferred_address = address + + location = self.api.path_to('addresses/{}'.format(address.email)) + created(response, location) + + +def membership_key(member): + # Sort first by mailing list, then by address, then by role. + return member.list_id, member.address.email, member.role.value + + +@public +class AddressMemberships(MemberCollection): + """All the memberships of a particular email address.""" + + def __init__(self, address): + super().__init__() + self._address = address + + def _get_collection(self, request): + """See `CollectionMixin`.""" + # XXX Improve this by implementing a .memberships attribute on + # IAddress, similar to the way IUser does it. + # + # Start by getting the IUser that controls this address. For now, if + # the address is not controlled by a user, return the empty set. + # Later when we address the XXX comment, it will return some + # memberships. But really, it should not be legal to subscribe an + # address to a mailing list that isn't controlled by a user -- maybe! + user = getUtility(IUserManager).get_user(self._address.email) + if user is None: + return [] + return sorted((member for member in user.memberships.members + if member.address == self._address), + key=membership_key) diff --git a/comanage-registry-mailman/core/docker-entrypoint.sh b/comanage-registry-mailman/core/docker-entrypoint.sh new file mode 100755 index 0000000..23b343d --- /dev/null +++ b/comanage-registry-mailman/core/docker-entrypoint.sh @@ -0,0 +1,229 @@ +#! /bin/bash + +# GNU Mailman 3 Core for COmanage Registry Dockerfile entrypoint +# +# This bash script borrows heavily from a script by Abhilash Raj for +# GNU Mailman 3. See https://github.com/maxking/docker-mailman +# +# Portions licensed to the University Corporation for Advanced Internet +# Development, Inc. ("UCAID") under one or more contributor license agreements. +# See the NOTICE file distributed with this work for additional information +# regarding copyright ownership. +# +# UCAID licenses this file to you under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with the +# License. You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +set -e + +function wait_for_postgres () { + # Check if the postgres database is up and accepting connections before + # moving forward. + # TODO: Use python's psycopg2 module to do this in python instead of + # installing postgres-client in the image. + until psql $MAILMAN_DATABASE_URL -c '\l'; do + >&2 echo "Postgres is unavailable - sleeping" + sleep 1 + done + >&2 echo "Postgres is up - continuing" +} + +function wait_for_mysql () { + # Check if MySQL is up and accepting connections. + HOSTNAME=$(python3 -c "from urllib.parse import urlparse; o = urlparse('$MAILMAN_DATABASE_URL'); print(o.hostname);") + until mysqladmin ping --host "$HOSTNAME" --silent; do + >&2 echo "MySQL is unavailable - sleeping" + sleep 1 + done + >&2 echo "MySQL is up - continuing" +} + +# Configuration details that may be injected through environment +# variables or the contents of files. + +injectable_config_vars=( + HYPERKITTY_API_KEY + MAILMAN_DATABASE_URL + MAILMAN_REST_PASSWORD +) + +# If the file associated with a configuration variable is present then +# read the value from it into the appropriate variable. + +for config_var in "${injectable_config_vars[@]}" +do + eval file_name=\$"${config_var}_FILE"; + + if [ -e "$file_name" ]; then + declare "${config_var}"=`cat $file_name` + fi +done + +# Empty the config file. +echo "# This file is autogenerated at container startup." > /etc/mailman.cfg + +# Check if $MM_HOSTNAME is set, if not, set it to the value returned by +# `hostname -i` command to set it to whatever IP address is assigned to the +# container. +if [[ ! -v MM_HOSTNAME ]]; then + export MM_HOSTNAME=`hostname -i` +fi + +if [[ ! -v SMTP_HOST ]]; then + export SMTP_HOST='172.19.199.1' +fi + +if [[ ! -v SMTP_PORT ]]; then + export SMTP_PORT=25 +fi + +# Check if REST port, username, and password are set, if not, set them +# to default values. +if [[ ! -v MAILMAN_REST_PORT ]]; then + export MAILMAN_REST_PORT='8001' +fi + +if [[ ! -v MAILMAN_REST_USER ]]; then + export MAILMAN_REST_USER='restadmin' +fi + +if [[ ! -v MAILMAN_REST_PASSWORD ]]; then + export MAILMAN_REST_PASSWORD='restpass' +fi + +function setup_database () { + if [[ ! -v MAILMAN_DATABASE_URL ]] + then + echo "Environemnt variable MAILMAN_DATABASE_URL should be defined..." + exit 1 + fi + + # Translate mysql:// urls to mysql+mysql:// backend: + if [[ "$MAILMAN_DATABASE_URL" == mysql://* ]]; then + MAILMAN_DATABASE_URL="mysql+pymysql://${MAILMAN_DATABASE_URL:8}" + echo "Database URL was automatically rewritten to: $MAILMAN_DATABASE_URL" + fi + + # If MAILMAN_DATABASE_CLASS is not set, guess it for common databases: + if [ -z "$MAILMAN_DATABASE_CLASS" ]; then + if [[ ("$MAILMAN_DATABASE_URL" == mysql:*) || + ("$MAILMAN_DATABASE_URL" == mysql+*) ]]; then + MAILMAN_DATABASE_CLASS=mailman.database.mysql.MySQLDatabase + fi + if [[ ("$MAILMAN_DATABASE_URL" == postgres:*) || + ("$MAILMAN_DATABASE_URL" == postgres+*) ]]; then + MAILMAN_DATABASE_CLASS=mailman.database.postgresql.PostgreSQLDatabase + fi + fi + + cat >> /etc/mailman.cfg <> /etc/mailman.cfg < /etc/postfix-mailman.cfg <> /etc/mailman.cfg +fi + + +if [[ ! -v HYPERKITTY_API_KEY ]]; then + echo "HYPERKITTY_API_KEY not defined, please set this environment variable..." + echo "exiting..." + exit 1 +fi + +if [[ ! -v HYPERKITTY_URL ]]; then + echo "HYPERKITTY_URL not set, using the default value of http://mailman-web:8000/hyperkitty" + export HYPERKITTY_URL="http://mailman-web:8000/hyperkitty/" +fi + +# Generate a basic mailman-hyperkitty.cfg. +cat > /etc/mailman-hyperkitty.cfg < + +#include +#include +#include +#include +#include +#include +#include +#include + +static char *argv0; + +static void usage(int exitcode) +{ + printf("Usage: %s user-spec command [args]\n", argv0); + exit(exitcode); +} + +int main(int argc, char *argv[]) +{ + char *user, *group, **cmdargv; + char *end; + + uid_t uid = getuid(); + gid_t gid = getgid(); + + argv0 = argv[0]; + if (argc < 3) + usage(0); + + user = argv[1]; + group = strchr(user, ':'); + if (group) + *group++ = '\0'; + + cmdargv = &argv[2]; + + struct passwd *pw = NULL; + if (user[0] != '\0') { + pw = getpwnam(user); + uid_t nuid = strtol(user, &end, 10); + if (*end == '\0') + uid = nuid; + } + if (pw == NULL) { + pw = getpwuid(uid); + } + if (pw != NULL) { + uid = pw->pw_uid; + gid = pw->pw_gid; + } + + setenv("HOME", pw != NULL ? pw->pw_dir : "/", 1); + + if (group && group[0] != '\0') { + /* group was specified, ignore grouplist for setgroups later */ + pw = NULL; + + struct group *gr = getgrnam(group); + if (gr == NULL) { + gid_t ngid = strtol(group, &end, 10); + if (*end == '\0') { + gr = getgrgid(ngid); + if (gr == NULL) + gid = ngid; + } + } + if (gr != NULL) + gid = gr->gr_gid; + } + + if (pw == NULL) { + if (setgroups(1, &gid) < 0) + err(1, "setgroups(%i)", gid); + } else { + int ngroups = 0; + gid_t *glist = NULL; + + while (1) { + int r = getgrouplist(pw->pw_name, gid, glist, &ngroups); + + if (r >= 0) { + if (setgroups(ngroups, glist) < 0) + err(1, "setgroups"); + break; + } + + glist = realloc(glist, ngroups * sizeof(gid_t)); + if (glist == NULL) + err(1, "malloc"); + } + } + + if (setgid(gid) < 0) + err(1, "setgid(%i)", gid); + + if (setuid(uid) < 0) + err(1, "setuid(%i)", uid); + + execvp(cmdargv[0], cmdargv); + err(1, "%s", cmdargv[0]); + + return 1; +} diff --git a/comanage-registry-mailman/core/users.py b/comanage-registry-mailman/core/users.py new file mode 100644 index 0000000..4813ac9 --- /dev/null +++ b/comanage-registry-mailman/core/users.py @@ -0,0 +1,524 @@ +# Copyright (C) 2011-2017 by the Free Software Foundation, Inc. +# +# This file is part of GNU Mailman. +# +# GNU Mailman is free software: you can redistribute it and/or modify it under +# the terms of the GNU General Public License as published by the Free +# Software Foundation, either version 3 of the License, or (at your option) +# any later version. +# +# GNU Mailman is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or +# FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for +# more details. +# +# You should have received a copy of the GNU General Public License along with +# GNU Mailman. If not, see . +# +# This file includes patches needed for integration with COmanage Registry. +# Inclusion of the patches in the upstream GNU Mailman distribution is pending. + +"""REST for users.""" + +from functools import lru_cache +from lazr.config import as_boolean +from mailman.config import config +from mailman.interfaces.address import ExistingAddressError +from mailman.interfaces.usermanager import IUserManager +from mailman.rest.addresses import UserAddresses +from mailman.rest.helpers import ( + BadRequest, CollectionMixin, GetterSetter, NotFound, bad_request, child, + conflict, created, etag, forbidden, no_content, not_found, okay) +from mailman.rest.preferences import Preferences +from mailman.rest.validator import ( + PatchValidator, ReadOnlyPATCHRequestError, UnknownPATCHRequestError, + Validator, list_of_strings_validator) +from mailman.utilities.datetime import now +from passlib.utils import generate_password as generate +from public import public +from zope.component import getUtility + + +# Attributes of a user which can be changed via the REST API. +@public +class PasswordEncrypterGetterSetter(GetterSetter): + def __init__(self): + super().__init__(config.password_context.encrypt) + + def get(self, obj, attribute): + assert attribute == 'cleartext_password' + super().get(obj, 'password') + + def put(self, obj, attribute, value): + assert attribute == 'cleartext_password' + super().put(obj, 'password', value) + +@public +class PreferredAddressDecoder(object): + """ + Callable for decoding a web request preferred email value string + into an instance of Address that can then be set as an attribute + on the User instance. See the definition of the GetterSetter class + for how this is called. + """ + def __init__(self): + pass + def __call__(self, email): + address = getUtility(IUserManager).get_address(email) + + if not address: + raise ValueError('Unknown email') + + # We mark the address as verified since we are directly + # setting a preferred address. + address.verified_on = now() + + return address + +class PreferredAddressGetterSetter(GetterSetter): + """ + Subclass for getting and setting the preferred_address + attribute on the User instance since it is not a simple + type. See the super GetterSetter class for details. + """ + def __init__(self): + # Use the special decoder with the logic to take an incoming + # email as a string and convert to an instance of Address. + super().__init__(PreferredAddressDecoder()) + + def get(self, obj, attribute): + assert attribute == 'preferred_address' + # Return the simple string representation of the Address. + super().get(obj, 'preferred_address').email + + def put(self, obj, attribute, value): + assert attribute == 'preferred_address' + # The conversion of the simple string to an instance of Address + # will have already been done by the decoder when the Validator + # instance calls this method so just put the value directly. + super().put(obj, 'preferred_address', value) + +@public +class ListOfDomainOwners(GetterSetter): + def get(self, domain, attribute): + assert attribute == 'owner', ( + 'Unexpected attribute: {}'.format(attribute)) + def sort_key(owner): # noqa: E306 + return owner.addresses[0].email + return sorted(domain.owners, key=sort_key) + + def put(self, domain, attribute, value): + assert attribute == 'owner', ( + 'Unexpected attribute: {}'.format(attribute)) + domain.add_owners(value) + + +ATTRIBUTES = dict( + cleartext_password=PasswordEncrypterGetterSetter(), + display_name=GetterSetter(str), + is_server_owner=GetterSetter(as_boolean), + preferred_address=PreferredAddressGetterSetter() + ) + + +CREATION_FIELDS = dict( + display_name=str, + email=str, + is_server_owner=as_boolean, + password=str, + _optional=('display_name', 'password', 'is_server_owner'), + ) + + +def create_user(api, arguments, response): + """Create a new user.""" + # We can't pass the 'password' argument to the user creation method, so + # strip that out (if it exists), then create the user, adding the password + # after the fact if successful. + password = arguments.pop('password', None) + is_server_owner = arguments.pop('is_server_owner', False) + user_manager = getUtility(IUserManager) + try: + user = user_manager.create_user(**arguments) + except ExistingAddressError as error: + # The address already exists. If the address already has a user + # linked to it, raise an error, otherwise create a new user and link + # it to this address. + email = arguments.pop('email') + user = user_manager.get_user(email) + if user is None: + address = user_manager.get_address(email) + user = user_manager.create_user(**arguments) + user.link(address) + else: + bad_request( + response, 'User already exists: {}'.format(error.address)) + return None + if password is None: + # This will have to be reset since it cannot be retrieved. + password = generate(int(config.passwords.password_length)) + user.password = config.password_context.encrypt(password) + user.is_server_owner = is_server_owner + user_id = api.from_uuid(user.user_id) + location = api.path_to('users/{}'.format(user_id)) + created(response, location) + return user + + +class _UserBase(CollectionMixin): + """Shared base class for user representations.""" + + def _resource_as_dict(self, user): + """See `CollectionMixin`.""" + # The canonical URL for a user is their unique user id, although we + # can always look up a user based on any registered and validated + # email address associated with their account. The user id is a UUID, + # but we serialize its integer equivalent. + user_id = self.api.from_uuid(user.user_id) + resource = dict( + created_on=user.created_on, + is_server_owner=user.is_server_owner, + self_link=self.api.path_to('users/{}'.format(user_id)), + user_id=user_id, + ) + # Add the password attribute, only if the user has a password. Same + # with the real name. These could be None or the empty string. + if user.password: + resource['password'] = user.password + if user.display_name: + resource['display_name'] = user.display_name + if user.preferred_address: + resource['preferred_address'] = user.preferred_address.email + return resource + + def _get_collection(self, request): + """See `CollectionMixin`.""" + return list(getUtility(IUserManager).users) + + +@public +class AllUsers(_UserBase): + """The users.""" + + def on_get(self, request, response): + """/users""" + resource = self._make_collection(request) + okay(response, etag(resource)) + + def on_post(self, request, response): + """Create a new user.""" + try: + validator = Validator(**CREATION_FIELDS) + arguments = validator(request) + except ValueError as error: + bad_request(response, str(error)) + return + create_user(self.api, arguments, response) + + +@public +class AUser(_UserBase): + """A user.""" + + def __init__(self, user_identifier): + """Get a user by various type of identifiers. + + :param user_identifier: The identifier used to retrieve the user. The + identifier may either be an email address controlled by the user + or the UUID of the user. The type of identifier is auto-detected + by looking for an `@` symbol, in which case it's taken as an email + address, otherwise it's assumed to be a UUID. However, UUIDs in + API 3.0 are integers, while in 3.1 are hex. + :type user_identifier: string + """ + self._user_identifier = user_identifier + # Defer calculation of the user until the API object is set, since + # that will determine how to interpret the user identifier. For ease + # of code migration, use an _user caching property (see below). + + @property + @lru_cache(1) + def _user(self): + user_manager = getUtility(IUserManager) + if '@' in self._user_identifier: + return user_manager.get_user(self._user_identifier) + else: + # The identifier is the string representation of a UUID, either an + # int in API 3.0 or a hex in API 3.1. + try: + user_id = self.api.to_uuid(self._user_identifier) + except ValueError: + return None + else: + return user_manager.get_user_by_id(user_id) + + def on_get(self, request, response): + """Return a single user end-point.""" + if self._user is None: + not_found(response) + else: + okay(response, self._resource_as_json(self._user)) + + @child() + def addresses(self, context, segments): + """/users//addresses""" + if self._user is None: + return NotFound(), [] + return UserAddresses(self._user) + + def on_delete(self, request, response): + """Delete the named user and all associated resources.""" + if self._user is None: + not_found(response) + return + for member in self._user.memberships.members: + member.unsubscribe() + user_manager = getUtility(IUserManager) + user_manager.delete_user(self._user) + no_content(response) + + @child() + def preferences(self, context, segments): + """/users//preferences""" + if len(segments) != 0: + return BadRequest(), [] + if self._user is None: + return NotFound(), [] + child = Preferences( + self._user.preferences, + 'users/{}'.format(self.api.from_uuid(self._user.user_id))) + return child, [] + + def on_patch(self, request, response): + """Patch the user's configuration (i.e. partial update).""" + if self._user is None: + not_found(response) + return + try: + validator = PatchValidator(request, ATTRIBUTES) + validator.update(self._user, request) + except UnknownPATCHRequestError as error: + bad_request( + response, b'Unknown attribute: {0}'.format(error.attribute)) + except ReadOnlyPATCHRequestError as error: + bad_request( + response, b'Read-only attribute: {0}'.format(error.attribute)) + except ValueError as error: + bad_request(response, str(error)) + else: + no_content(response) + + def on_put(self, request, response): + """Put the user's configuration (i.e. full update).""" + if self._user is None: + not_found(response) + return + validator = Validator(**ATTRIBUTES) + try: + validator.update(self._user, request) + except UnknownPATCHRequestError as error: + bad_request( + response, b'Unknown attribute: {0}'.format(error.attribute)) + except ReadOnlyPATCHRequestError as error: + bad_request( + response, b'Read-only attribute: {0}'.format(error.attribute)) + except ValueError as error: + bad_request(response, str(error)) + else: + no_content(response) + + @child() + def login(self, context, segments): + """Log the user in, sort of, by verifying a given password.""" + if self._user is None: + return NotFound(), [] + return Login(self._user) + + +@public +class AddressUser(_UserBase): + """The user linked to an address.""" + + def __init__(self, address): + self._address = address + self._user = address.user + + def on_get(self, request, response): + """Return a single user end-point.""" + if self._user is None: + not_found(response) + else: + okay(response, self._resource_as_json(self._user)) + + def on_delete(self, request, response): + """Delete the named user, all her memberships, and addresses.""" + if self._user is None: + not_found(response) + return + self._user.unlink(self._address) + no_content(response) + + def on_post(self, request, response): + """Link a user to the address, and create it if needed.""" + if self._user: + conflict(response) + return + # When creating a linked user by POSTing, the user either must already + # exist, or it can be automatically created, if the auto_create flag + # is given and true (if missing, it defaults to true). However, in + # this case we do not accept 'email' as a POST field. + fields = CREATION_FIELDS.copy() + del fields['email'] + fields['user_id'] = self.api.to_uuid + fields['auto_create'] = as_boolean + fields['_optional'] = fields['_optional'] + ( + 'user_id', 'auto_create', 'is_server_owner') + try: + validator = Validator(**fields) + arguments = validator(request) + except ValueError as error: + bad_request(response, str(error)) + return + user_manager = getUtility(IUserManager) + if 'user_id' in arguments: + user_id = arguments['user_id'] + user = user_manager.get_user_by_id(user_id) + if user is None: + bad_request(response, 'No user with ID {}'.format( + self.api.from_uuid(user_id))) + return + okay(response) + else: + auto_create = arguments.pop('auto_create', True) + if auto_create: + # This sets the 201 or 400 status. + user = create_user(self.api, arguments, response) + if user is None: + return + else: + forbidden(response) + return + user.link(self._address) + + def on_put(self, request, response): + """Set or replace the addresses's user.""" + if self._user: + self._user.unlink(self._address) + # Process post data and check for an existing user. + fields = CREATION_FIELDS.copy() + fields['user_id'] = self.api.to_uuid + fields['_optional'] = fields['_optional'] + ( + 'user_id', 'email', 'is_server_owner') + try: + validator = Validator(**fields) + arguments = validator(request) + except ValueError as error: + bad_request(response, str(error)) + return + user_manager = getUtility(IUserManager) + if 'user_id' in arguments: + user_id = arguments['user_id'] + user = user_manager.get_user_by_id(user_id) + if user is None: + not_found(response, b'No user with ID {}'.format(user_id)) + return + okay(response) + else: + user = create_user(self.api, arguments, response) + if user is None: + return + user.link(self._address) + + +@public +class Login: + """/users//login""" + + def __init__(self, user): + assert user is not None + self._user = user + + def on_post(self, request, response): + # We do not want to encrypt the plaintext password given in the POST + # data. That would hash the password, but we need to have the + # plaintext in order to pass into passlib. + validator = Validator(cleartext_password=GetterSetter(str)) + try: + values = validator(request) + except ValueError as error: + bad_request(response, str(error)) + return + is_valid, new_hash = config.password_context.verify( + values['cleartext_password'], self._user.password) + if is_valid: + if new_hash is not None: + self._user.password = new_hash + no_content(response) + else: + forbidden(response) + + +@public +class OwnersForDomain(_UserBase): + """Owners for a particular domain.""" + + def __init__(self, domain): + self._domain = domain + + def on_get(self, request, response): + """/domains//owners""" + if self._domain is None: + not_found(response) + return + resource = self._make_collection(request) + okay(response, etag(resource)) + + def on_post(self, request, response): + """POST to /domains//owners """ + if self._domain is None: + not_found(response) + return + validator = Validator( + owner=ListOfDomainOwners(list_of_strings_validator)) + try: + validator.update(self._domain, request) + except ValueError as error: + bad_request(response, str(error)) + return + return no_content(response) + + def on_delete(self, request, response): + """DELETE to /domains//owners""" + if self._domain is None: + not_found(response) + try: + # No arguments. + Validator()(request) + except ValueError as error: + bad_request(response, str(error)) + return + owner_email = [ + owner.addresses[0].email + for owner in self._domain.owners + ] + for email in owner_email: + self._domain.remove_owner(email) + return no_content(response) + + def _get_collection(self, request): + """See `CollectionMixin`.""" + return list(self._domain.owners) + + +@public +class ServerOwners(_UserBase): + """All server owners.""" + + def on_get(self, request, response): + """/owners""" + resource = self._make_collection(request) + okay(response, etag(resource)) + + def _get_collection(self, request): + """See `CollectionMixin`.""" + return list(getUtility(IUserManager).server_owners) diff --git a/comanage-registry-mailman/docker-compose.yaml b/comanage-registry-mailman/docker-compose.yaml new file mode 100644 index 0000000..e2e69ea --- /dev/null +++ b/comanage-registry-mailman/docker-compose.yaml @@ -0,0 +1,131 @@ +# Docker Compose file for Mailman 3 for COmanage Registry +# +# Portions licensed to the University Corporation for Advanced Internet +# Development, Inc. ("UCAID") under one or more contributor license agreements. +# See the NOTICE file distributed with this work for additional information +# regarding copyright ownership. +# +# UCAID licenses this file to you under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with the +# License. You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# This is an example compose file. Be sure to modify it as necessary +# for your own deployment. + +version: '2' + +services: + mailman-core: + image: sphericalcowgroup/mailman-core:0.1.7 + container_name: mailman-core + hostname: mailman-core + volumes: + - /opt/mailman/core:/opt/mailman/ + stop_grace_period: 30s + links: + - database:database + depends_on: + - database + environment: + - MAILMAN_DATABASE_URL=postgres://mailman:gECPnaqXVID80TlRS5ZG@database/mailmandb + - MAILMAN_DATABASE_TYPE=postgres + - MAILMAN_DATABASE_CLASS=mailman.database.postgresql.PostgreSQLDatabase + - HYPERKITTY_API_KEY=HbTKLdrhRxUX96f5bD2g + - MAILMAN_REST_USER=restadmin + - MAILMAN_REST_PASSWORD=K6gfcC9uHQMXr448Kmdi + - SMTP_HOST=postfix + - SMTP_PORT=25 + expose: + - "8001" + networks: + mailman: + ipv4_address: 172.19.199.2 + + mailman-web: + image: sphericalcowgroup/mailman-web:0.1.7 + container_name: mailman-web + hostname: mailman-web + depends_on: + - database + links: + - mailman-core:mailman-core + - database:database + volumes: + - /opt/mailman/web:/opt/mailman-web-data + environment: + - MAILMAN_DATABASE_TYPE=postgres + - MAILMAN_DATABASE_URL=postgres://mailman:gECPnaqXVID80TlRS5ZG@database/mailmandb + - HYPERKITTY_API_KEY=HbTKLdrhRxUX96f5bD2g + - SERVE_FROM_DOMAIN=lists-dev.sphericalcowgroup.com + - MAILMAN_ADMIN_USER=mailman_admin + - MAILMAN_ADMIN_EMAIL=skoranda@gmail.com + - MAILMAN_WEB_SECRET_KEY=fPe7d9e0PKF8ryySOow0 + - MAILMAN_REST_USER=restadmin + - MAILMAN_REST_PASSWORD=K6gfcC9uHQMXr448Kmdi + - SMTP_HOST=postfix + - SMTP_PORT=25 + networks: + mailman: + ipv4_address: 172.19.199.3 + + database: + image: postgres:9.6 + container_name: mailman-database + environment: + - POSTGRES_DB=mailmandb + - POSTGRES_USER=mailman + - POSTGRES_PASSWORD=gECPnaqXVID80TlRS5ZG + restart: always + volumes: + - /opt/mailman/database:/var/lib/postgresql/data + networks: + mailman: + ipv4_address: 172.19.199.4 + + postfix: + image: sphericalcowgroup/mailman-postfix + container_name: mailman-postfix + volumes: + - /opt/mailman:/opt/mailman + environment: + - POSTFIX_MAILNAME=lists-dev.sphericalcowgroup.com + depends_on: + - mailman-core + ports: + - "25:25" + networks: + mailman: + ipv4_address: 172.19.199.5 + + nginx: + image: sphericalcowgroup/mailman-core-nginx + container_name: mailman-nginx + volumes: + - /opt/mailman/web:/opt/mailman-web-data + - /opt/mailman/nginx/fullchain.pem:/etc/nginx/https.crt + - /opt/mailman/nginx/privkey.pem:/etc/nginx/https.key + - /opt/mailman/nginx/dhparam.pem:/etc/nginx/dhparam.pem + depends_on: + - mailman-core + ports: + - "80:80" + - "443:443" + networks: + mailman: + ipv4_address: 172.19.199.6 + +networks: + mailman: + driver: bridge + ipam: + driver: default + config: + - subnet: 172.19.199.0/24 diff --git a/comanage-registry-mailman/mailman-stack.yml b/comanage-registry-mailman/mailman-stack.yml new file mode 100644 index 0000000..3e4d81a --- /dev/null +++ b/comanage-registry-mailman/mailman-stack.yml @@ -0,0 +1,143 @@ +# Docker Swarm stack compose file for Mailman 3 for COmanage Registry +# +# Portions licensed to the University Corporation for Advanced Internet +# Development, Inc. ("UCAID") under one or more contributor license agreements. +# See the NOTICE file distributed with this work for additional information +# regarding copyright ownership. +# +# UCAID licenses this file to you under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with the +# License. You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# This is an example compose file. Be sure to modify it as necessary +# for your own deployment. + +version: '3.2' + +services: + mailman-core: + image: sphericalcowgroup/mailman-core:0.1.7 + volumes: + - /srv/docker/mailman/core:/opt/mailman/ + environment: + - MAILMAN_DATABASE_URL_FILE=/run/secrets/mailman_database_url + - MAILMAN_DATABASE_TYPE=postgres + - MAILMAN_DATABASE_CLASS=mailman.database.postgresql.PostgreSQLDatabase + - HYPERKITTY_API_KEY_FILE=/run/secrets/hyperkitty_api_key + - MAILMAN_REST_USER=restadmin + - MAILMAN_REST_PASSWORD_FILE=/run/secrets/mailman_rest_password + - SMTP_HOST=postfix + - SMTP_PORT=25 + stop_grace_period: 30s + networks: + - default + secrets: + - hyperkitty_api_key + - mailman_database_url + - mailman_rest_password + + mailman-web: + image: sphericalcowgroup/mailman-web:0.1.7 + volumes: + - /srv/docker/mailman/web:/opt/mailman-web-data + environment: + - MAILMAN_DATABASE_URL_FILE=/run/secrets/mailman_database_url + - MAILMAN_DATABASE_TYPE=postgres + - HYPERKITTY_API_KEY_FILE=/run/secrets/hyperkitty_api_key + - SERVE_FROM_DOMAIN=lists-dev.sphericalcowgroup.com + - MAILMAN_ADMIN_USER=mailman_admin + - MAILMAN_ADMIN_EMAIL=skoranda@gmail.com + - MAILMAN_WEB_SECRET_KEY_FILE=/run/secrets/mailman_web_secret_key + - MAILMAN_REST_USER=restadmin + - MAILMAN_REST_PASSWORD_FILE=/run/secrets/mailman_rest_password + - SMTP_HOST=postfix + - SMTP_PORT=25 + networks: + - default + secrets: + - hyperkitty_api_key + - mailman_database_url + - mailman_rest_password + - mailman_web_secret_key + + database: + image: postgres:9.6 + volumes: + - /srv/docker/mailman/database:/var/lib/postgresql/data + environment: + - POSTGRES_DB=mailmandb + - POSTGRES_USER=mailman + - POSTGRES_PASSWORD_FILE=/run/secrets/postgres_password + networks: + - default + secrets: + - postgres_password + + postfix: + image: sphericalcowgroup/mailman-postfix + volumes: + - /srv/docker/mailman:/opt/mailman + environment: + - POSTFIX_MAILNAME=lists-dev.sphericalcowgroup.com + ports: + - "25:25" + networks: + - default + + nginx: + image: sphericalcowgroup/mailman-core-nginx + volumes: + - /srv/docker/mailman/web:/opt/mailman-web-data + environment: + - NGINX_HTTPS_CERT_FILE=/run/secrets/nginx_https_cert_file + - NGINX_HTTPS_KEY_FILE=/run/secrets/nginx_https_key_file + - NGINX_DH_PARAM_FILE=/run/secrets/nginx_dh_param_file + secrets: + - nginx_https_cert_file + - nginx_https_key_file + - nginx_dh_param_file + networks: + - default + ports: + - target: 443 + published: 443 + protocol: tcp + # 'host' mode is necessary for nginx to receive the browser IP address + # instead of the ingress network IP address. This is only a useful workaround + # when not truly leveraging the load balancing capabilities of swarm mode. + # Normally with more than a single mode swarm the upstream load balancer in + # front of the swarm would be used for the definitive access log. + # See discussion at https://github.com/moby/moby/issues/25526 . + mode: host + - target: 80 + published: 80 + protocol: tcp + mode: host + deploy: + replicas: 1 + +secrets: + hyperkitty_api_key: + external: true + mailman_database_url: + external: true + mailman_rest_password: + external: true + mailman_web_secret_key: + external: true + nginx_https_cert_file: + external: true + nginx_https_key_file: + external: true + nginx_dh_param_file: + external: true + postgres_password: + external: true diff --git a/comanage-registry-mailman/nginx/Dockerfile b/comanage-registry-mailman/nginx/Dockerfile new file mode 100644 index 0000000..407fe6c --- /dev/null +++ b/comanage-registry-mailman/nginx/Dockerfile @@ -0,0 +1,41 @@ +# Nginx for GNU Mailman 3 for COmanage Registry Dockerfile +# +# Portions licensed to the University Corporation for Advanced Internet +# Development, Inc. ("UCAID") under one or more contributor license agreements. +# See the NOTICE file distributed with this work for additional information +# regarding copyright ownership. +# +# UCAID licenses this file to you under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with the +# License. You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +FROM debian:stretch + +RUN apt-get update \ + && apt-get install --no-install-recommends --no-install-suggests -y \ + netcat-traditional \ + nginx \ + nginx-extras + +# forward request and error logs to docker log collector +RUN ln -sf /dev/stdout /var/log/nginx/access.log \ + && ln -sf /dev/stderr /var/log/nginx/error.log + +EXPOSE 80 + +COPY nginx.conf /etc/nginx/nginx.conf +COPY start.sh /usr/local/sbin/nginx-start.sh + +ENTRYPOINT ["/usr/local/sbin/nginx-start.sh"] + +STOPSIGNAL SIGTERM + +CMD ["nginx", "-g", "daemon off;"] diff --git a/comanage-registry-mailman/nginx/nginx.conf b/comanage-registry-mailman/nginx/nginx.conf new file mode 100644 index 0000000..d09fc47 --- /dev/null +++ b/comanage-registry-mailman/nginx/nginx.conf @@ -0,0 +1,114 @@ +# Nginx configuration for GNU Mailman 3 for COmanage Registry Dockerfile +# +# Portions licensed to the University Corporation for Advanced Internet +# Development, Inc. ("UCAID") under one or more contributor license agreements. +# See the NOTICE file distributed with this work for additional information +# regarding copyright ownership. +# +# UCAID licenses this file to you under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with the +# License. You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +user www-data; +worker_processes 1; +include /etc/nginx/modules-enabled/*.conf; + +error_log /var/log/nginx/error.log info; +pid /var/run/nginx.pid; + +events { + worker_connections 1024; +} + + +http { + include /etc/nginx/mime.types; + default_type application/octet-stream; + + log_format main '$remote_addr - $remote_user [$time_local] "$request" ' + '$status $body_bytes_sent "$http_referer" ' + '"$http_user_agent" "$http_x_forwarded_for"'; + + access_log /var/log/nginx/access.log main; + + sendfile on; + + keepalive_timeout 65; + + include /etc/nginx/conf.d/*.conf; + + upstream mailman-core { + # The name of the server is 'mailman-core' and matches the name + # of the service in the compose file. + server mailman-core:8001 fail_timeout=0; + } + + upstream mailman-web { + # The name of the server is 'mailman-core' and matches the name + # of the service in the compose file. + server mailman-web:8000 fail_timeout=0; + } + + server { + listen 80; + server_name lists-dev.sphericalcowgroup.com; + return 301 https://$server_name$request_uri; + } + + server { + listen 443 ssl; + server_name lists-dev.sphericalcowgroup.com; + ssl_certificate /etc/nginx/https.crt; + ssl_certificate_key /etc/nginx/https.key; + ssl_protocols TLSv1 TLSv1.1 TLSv1.2; + ssl_prefer_server_ciphers on; + ssl_ciphers AES256+EECDH:AES256+EDH:!aNULL; + ssl_dhparam /etc/nginx/dhparam.pem; + ssl_session_cache shared:SSL:10m; + ssl_session_timeout 24h; + ssl_buffer_size 1400; + ssl_session_tickets off; + ssl_stapling on; + ssl_stapling_verify on; + resolver 8.8.4.4 8.8.8.8 valid=300s; + resolver_timeout 10s; + + add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always; + add_header X-Frame-Options DENY; + add_header X-Content-Type-Options nosniff; + + location /api/ { + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto https; + proxy_set_header Host $http_host; + proxy_redirect off; + proxy_pass http://mailman-core/; + proxy_intercept_errors on; + + subs_filter_types application/json; + subs_filter '"self_link": "http://[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}:8001' '"self_link": "https://$host/api' [r]; + } + + location /static/ { + alias /opt/mailman-web-data/static/; + } + + location / { + include uwsgi_params; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto https; + proxy_set_header Host $http_host; + proxy_redirect off; + proxy_pass http://mailman-web; + } + + } +} diff --git a/comanage-registry-mailman/nginx/start.sh b/comanage-registry-mailman/nginx/start.sh new file mode 100755 index 0000000..ac8f085 --- /dev/null +++ b/comanage-registry-mailman/nginx/start.sh @@ -0,0 +1,73 @@ +#! /bin/bash + +# Nginx for GNU Mailman 3 Core for COmanage Registry Dockerfile entrypoint +# +# Portions licensed to the University Corporation for Advanced Internet +# Development, Inc. ("UCAID") under one or more contributor license agreements. +# See the NOTICE file distributed with this work for additional information +# regarding copyright ownership. +# +# UCAID licenses this file to you under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with the +# License. You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# exit immediately on failure +set -e + +# Configuration details that may be injected through environment +# variables or the contents of files. + +injectable_config_vars=( + MAILMAN_CORE_HOST + MAILMAN_CORE_PORT +) + +# Default values. +MAILMAN_CORE_HOST="mailman-core" +MAILMAN_CORE_PORT="8001" + +# If the file associated with a configuration variable is present then +# read the value from it into the appropriate variable. + +for config_var in "${injectable_config_vars[@]}" +do + eval file_name=\$"${config_var}_FILE"; + + if [ -e "$file_name" ]; then + declare "${config_var}"=`cat $file_name` + fi +done + +# Copy HTTPS certificate and key into place. +if [ -n "${NGINX_HTTPS_CERT_FILE}" ] && [ -n "${NGINX_HTTPS_KEY_FILE}" ]; then + cp "${NGINX_HTTPS_CERT_FILE}" /etc/nginx/https.crt + cp "${NGINX_HTTPS_KEY_FILE}" /etc/nginx/https.key + chmod 644 /etc/nginx/https.crt + chmod 600 /etc/nginx/https.key + chown www-data /etc/nginx/https.key +fi + +# Copy DH parameters for EDH ciphers into place +if [ -n "${NGINX_DH_PARAM_FILE}" ]; then + cp "${NGINX_DH_PARAM_FILE}" /etc/nginx/dhparam.pem + chmod 600 /etc/nginx/dhparam.pem + chown www-data /etc/nginx/dhparam.pem +fi + +# Wait for the mailman core container to be ready. +until nc -z -w 1 "${MAILMAN_CORE_HOST}" "${MAILMAN_CORE_PORT}" +do + echo "Waiting for Mailman core container..." + sleep 1 +done + +# Start nginx. +exec nginx -g 'daemon off;' diff --git a/comanage-registry-mailman/postfix/Dockerfile b/comanage-registry-mailman/postfix/Dockerfile new file mode 100644 index 0000000..d7aa8d2 --- /dev/null +++ b/comanage-registry-mailman/postfix/Dockerfile @@ -0,0 +1,41 @@ +# Postfix for Mailman 3 for COmanage Registry Dockerfile +# +# Portions licensed to the University Corporation for Advanced Internet +# Development, Inc. ("UCAID") under one or more contributor license agreements. +# See the NOTICE file distributed with this work for additional information +# regarding copyright ownership. +# +# UCAID licenses this file to you under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with the +# License. You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +FROM debian:stretch + + +RUN apt-get update \ + && apt-get install -y --no-install-recommends \ + postfix \ + rsyslog \ + supervisor \ + #syslog-ng \ + && apt-get clean + +COPY supervisord.conf /usr/local/etc/ +COPY docker-postfix-entrypoint /usr/local/bin/ + +RUN ln -sfT /dev/stdout /var/log/mail.log \ + && ln -sfT /dev/stdout /var/log/mail.info \ + && ln -sfT /dev/stdout /var/log/mail.warn \ + && ln -sfT /dev/stdout /var/log/mail.err + +COPY main.cf /etc/postfix/main.cf +COPY master.cf /etc/postfix/master.cf + +ENTRYPOINT ["/usr/bin/supervisord", "-c", "/usr/local/etc/supervisord.conf"] diff --git a/comanage-registry-mailman/postfix/docker-postfix-entrypoint b/comanage-registry-mailman/postfix/docker-postfix-entrypoint new file mode 100755 index 0000000..37b0e8f --- /dev/null +++ b/comanage-registry-mailman/postfix/docker-postfix-entrypoint @@ -0,0 +1,65 @@ +#!/bin/bash + +# Postfix Dockerfile entrypoint +# +# Portions licensed to the University Corporation for Advanced Internet +# Development, Inc. ("UCAID") under one or more contributor license agreements. +# See the NOTICE file distributed with this work for additional information +# regarding copyright ownership. +# +# UCAID licenses this file to you under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with the +# License. You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +OUTPUT=/dev/stdout + +# Configuration details that may be injected through environment +# variables or the contents of files. + +injectable_config_vars=( + POSTFIX_MAILNAME + POSTFIX_MYHOSTNAME +) + +# If the file associated with a configuration variable is present then +# read the value from it into the appropriate variable. So for example +# if the variable COMANAGE_REGISTRY_DATASOURCE_FILE exists and its +# value points to a file on the file system then read the contents +# of that file into the variable COMANAGE_REGISTRY_DATASOURCE. + +for config_var in "${injectable_config_vars[@]}" +do + eval file_name=\$"${config_var}_FILE"; + + if [ -e "$file_name" ]; then + declare "${config_var}"=`cat $file_name` + fi +done + +# Create the /etc/mailname file +if [ -n "${POSTFIX_MAILNAME}" ]; then + MAILNAME=${POSTFIX_MAILNAME} +else + MAILNAME=`/bin/hostname -f` +fi + +echo "${MAILNAME}" > /etc/mailname +chmod 644 /etc/mailname + +if [ -n "${POSTFIX_MYHOSTNAME}" ]; then + MAILNAME=${POSTFIX_MYHOSTNAME} +else + MAILNAME=`/bin/hostname -f` +fi + + + +exec /usr/lib/postfix/sbin/master -c /etc/postfix -d diff --git a/comanage-registry-mailman/postfix/main.cf b/comanage-registry-mailman/postfix/main.cf new file mode 100644 index 0000000..8584da1 --- /dev/null +++ b/comanage-registry-mailman/postfix/main.cf @@ -0,0 +1,73 @@ +# Postfix configuration for GNU Mailman 3 for COmanage Registry Dockerfile +# +# Portions licensed to the University Corporation for Advanced Internet +# Development, Inc. ("UCAID") under one or more contributor license agreements. +# See the NOTICE file distributed with this work for additional information +# regarding copyright ownership. +# +# UCAID licenses this file to you under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with the +# License. You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# See /usr/share/postfix/main.cf.dist for a commented, more complete version + +# Debian specific: Specifying a file name will cause the first +# line of that file to be used as the name. The Debian default +# is /etc/mailname. +#myorigin = /etc/mailname + +smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU) +biff = no + +# appending .domain is the MUA's job. +append_dot_mydomain = no + +# Uncomment the next line to generate "delayed mail" warnings +#delay_warning_time = 4h + +readme_directory = no + +# See http://www.postfix.org/COMPATIBILITY_README.html -- default to 2 on +# fresh installs. +compatibility_level = 2 + +# TLS parameters +smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem +smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key +smtpd_use_tls=yes +smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache +smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache + +# See /usr/share/doc/postfix/TLS_README.gz in the postfix-doc package for +# information on enabling SSL in the smtp client. + +smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination +myhostname = lists-dev.sphericalcowgroup.com +alias_maps = hash:/etc/aliases +alias_database = hash:/etc/aliases +mydestination = $myhostname, localhost.localdomain, localhost +relayhost = +mynetworks = 10.0.0.0/8 172.16.0.0/12 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 +mailbox_size_limit = 0 +recipient_delimiter = + +inet_interfaces = all +inet_protocols = ipv4 + +recipient_delimiter = + +unknown_local_recipient_reject_code = 550 +owner_request_special = no + +transport_maps = + regexp:/opt/mailman/core/var/data/postfix_lmtp +local_recipient_maps = + regexp:/opt/mailman/core/var/data/postfix_lmtp +relay_domains = + regexp:/opt/mailman/core/var/data/postfix_domains diff --git a/comanage-registry-mailman/postfix/master.cf b/comanage-registry-mailman/postfix/master.cf new file mode 100644 index 0000000..c74b5b1 --- /dev/null +++ b/comanage-registry-mailman/postfix/master.cf @@ -0,0 +1,143 @@ +# Postfix configuration for GNU Mailman 3 for COmanage Registry Dockerfile +# +# Portions licensed to the University Corporation for Advanced Internet +# Development, Inc. ("UCAID") under one or more contributor license agreements. +# See the NOTICE file distributed with this work for additional information +# regarding copyright ownership. +# +# UCAID licenses this file to you under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with the +# License. You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# +# Postfix master process configuration file. For details on the format +# of the file, see the master(5) manual page (command: "man 5 master" or +# on-line: http://www.postfix.org/master.5.html). +# +# Do not forget to execute "postfix reload" after editing this file. +# +# ========================================================================== +# service type private unpriv chroot wakeup maxproc command + args +# (yes) (yes) (no) (never) (100) +# ========================================================================== +smtp inet n - n - - smtpd +#smtp inet n - y - 1 postscreen +#smtpd pass - - y - - smtpd +#dnsblog unix - - y - 0 dnsblog +#tlsproxy unix - - y - 0 tlsproxy +#submission inet n - y - - smtpd +# -o syslog_name=postfix/submission +# -o smtpd_tls_security_level=encrypt +# -o smtpd_sasl_auth_enable=yes +# -o smtpd_reject_unlisted_recipient=no +# -o smtpd_client_restrictions=$mua_client_restrictions +# -o smtpd_helo_restrictions=$mua_helo_restrictions +# -o smtpd_sender_restrictions=$mua_sender_restrictions +# -o smtpd_recipient_restrictions= +# -o smtpd_relay_restrictions=permit_sasl_authenticated,reject +# -o milter_macro_daemon_name=ORIGINATING +#smtps inet n - y - - smtpd +# -o syslog_name=postfix/smtps +# -o smtpd_tls_wrappermode=yes +# -o smtpd_sasl_auth_enable=yes +# -o smtpd_reject_unlisted_recipient=no +# -o smtpd_client_restrictions=$mua_client_restrictions +# -o smtpd_helo_restrictions=$mua_helo_restrictions +# -o smtpd_sender_restrictions=$mua_sender_restrictions +# -o smtpd_recipient_restrictions= +# -o smtpd_relay_restrictions=permit_sasl_authenticated,reject +# -o milter_macro_daemon_name=ORIGINATING +#628 inet n - y - - qmqpd +pickup unix n - n 60 1 pickup +cleanup unix n - n - 0 cleanup +qmgr unix n - n 300 1 qmgr +#qmgr unix n - n 300 1 oqmgr +tlsmgr unix - - n 1000? 1 tlsmgr +rewrite unix - - n - - trivial-rewrite +bounce unix - - n - 0 bounce +defer unix - - n - 0 bounce +trace unix - - n - 0 bounce +verify unix - - n - 1 verify +flush unix n - n 1000? 0 flush +proxymap unix - - n - - proxymap +proxywrite unix - - n - 1 proxymap +smtp unix - - n - - smtp +relay unix - - n - - smtp +# -o smtp_helo_timeout=5 -o smtp_connect_timeout=5 +showq unix n - n - - showq +error unix - - n - - error +retry unix - - n - - error +discard unix - - n - - discard +local unix - n n - - local +virtual unix - n n - - virtual +lmtp unix - - n - - lmtp +anvil unix - - n - 1 anvil +scache unix - - n - 1 scache +# +# ==================================================================== +# Interfaces to non-Postfix software. Be sure to examine the manual +# pages of the non-Postfix software to find out what options it wants. +# +# Many of the following services use the Postfix pipe(8) delivery +# agent. See the pipe(8) man page for information about ${recipient} +# and other message envelope options. +# ==================================================================== +# +# maildrop. See the Postfix MAILDROP_README file for details. +# Also specify in main.cf: maildrop_destination_recipient_limit=1 +# +maildrop unix - n n - - pipe + flags=DRhu user=vmail argv=/usr/bin/maildrop -d ${recipient} +# +# ==================================================================== +# +# Recent Cyrus versions can use the existing "lmtp" master.cf entry. +# +# Specify in cyrus.conf: +# lmtp cmd="lmtpd -a" listen="localhost:lmtp" proto=tcp4 +# +# Specify in main.cf one or more of the following: +# mailbox_transport = lmtp:inet:localhost +# virtual_transport = lmtp:inet:localhost +# +# ==================================================================== +# +# Cyrus 2.1.5 (Amos Gouaux) +# Also specify in main.cf: cyrus_destination_recipient_limit=1 +# +#cyrus unix - n n - - pipe +# user=cyrus argv=/cyrus/bin/deliver -e -r ${sender} -m ${extension} ${user} +# +# ==================================================================== +# Old example of delivery via Cyrus. +# +#old-cyrus unix - n n - - pipe +# flags=R user=cyrus argv=/cyrus/bin/deliver -e -m ${extension} ${user} +# +# ==================================================================== +# +# See the Postfix UUCP_README file for configuration details. +# +uucp unix - n n - - pipe + flags=Fqhu user=uucp argv=uux -r -n -z -a$sender - $nexthop!rmail ($recipient) +# +# Other external delivery methods. +# +ifmail unix - n n - - pipe + flags=F user=ftn argv=/usr/lib/ifmail/ifmail -r $nexthop ($recipient) +bsmtp unix - n n - - pipe + flags=Fq. user=bsmtp argv=/usr/lib/bsmtp/bsmtp -t$nexthop -f$sender $recipient +scalemail-backend unix - n n - 2 pipe + flags=R user=scalemail argv=/usr/lib/scalemail/bin/scalemail-store ${nexthop} ${user} ${extension} +mailman unix - n n - - pipe + flags=FR user=list argv=/usr/lib/mailman/bin/postfix-to-mailman.py + ${nexthop} ${user} + diff --git a/comanage-registry-mailman/postfix/supervisord.conf b/comanage-registry-mailman/postfix/supervisord.conf new file mode 100644 index 0000000..7ec3413 --- /dev/null +++ b/comanage-registry-mailman/postfix/supervisord.conf @@ -0,0 +1,36 @@ +; COmanage Registry Docker supervisord configuration +; +; Portions licensed to the University Corporation for Advanced Internet +; Development, Inc. ("UCAID") under one or more contributor license agreements. +; See the NOTICE file distributed with this work for additional information +; regarding copyright ownership. +; +; UCAID licenses this file to you under the Apache License, Version 2.0 +; (the "License"); you may not use this file except in compliance with the +; License. You may obtain a copy of the License at: +; +; http://www.apache.org/licenses/LICENSE-2.0 +; +; Unless required by applicable law or agreed to in writing, software +; distributed under the License is distributed on an "AS IS" BASIS, +; WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +; See the License for the specific language governing permissions and +; limitations under the License. + +[supervisord] +nodaemon=true +user=root + +[program:rsyslog] +command=/usr/sbin/rsyslogd -n +stdout_logfile=/dev/stdout +stdout_logfile_maxbytes=0 +stderr_logfile=/dev/stderr +stderr_logfile_maxbytes=0 + +[program:postfix] +command=/usr/local/bin/docker-postfix-entrypoint +stdout_logfile=/dev/stdout +stdout_logfile_maxbytes=0 +stderr_logfile=/dev/stderr +stderr_logfile_maxbytes=0 diff --git a/comanage-registry-mailman/web/Dockerfile b/comanage-registry-mailman/web/Dockerfile new file mode 100644 index 0000000..c819670 --- /dev/null +++ b/comanage-registry-mailman/web/Dockerfile @@ -0,0 +1,64 @@ +# GNU Mailman 3 for COmanage Registry Dockerfile +# +# Portions licensed to the University Corporation for Advanced Internet +# Development, Inc. ("UCAID") under one or more contributor license agreements. +# See the NOTICE file distributed with this work for additional information +# regarding copyright ownership. +# +# UCAID licenses this file to you under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with the +# License. You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +FROM python:2.7-stretch + +RUN apt-get update \ + && apt-get install -y --no-install-recommends \ + gcc \ + libc-dev \ + libcurl4-openssl-dev \ + libmariadbclient-dev \ + netcat-traditional \ + postgresql-client \ + sassc \ + && pip install -U django==1.11 pip \ + && pip install mailmanclient==3.1.1 \ + postorius==1.1.2 \ + hyperkitty==1.1.4 \ + django-mailman3==1.1.0 \ + whoosh \ + uwsgi \ + psycopg2 \ + dj-database-url \ + mysqlclient \ + typing \ + && adduser --system --no-create-home --group mailman + +# Add needed files for uwsgi server + settings for django +COPY mailman-web /opt/mailman-web + +RUN chown -R mailman:mailman /opt/mailman-web/ \ + && chmod u+x /opt/mailman-web/manage.py + +# Add startup script to container +COPY docker-entrypoint.sh /usr/local/bin/ + +WORKDIR /opt/mailman-web + +# Expose port 8000 for http and port 8080 for uwsgi +# (see web/mailman-web/uwsgi.ini#L2-L4) +EXPOSE 8000 8080 + +# Use stop signal for uwsgi server +STOPSIGNAL SIGINT + +ENTRYPOINT ["docker-entrypoint.sh"] + +CMD ["uwsgi", "--ini", "/opt/mailman-web/uwsgi.ini"] diff --git a/comanage-registry-mailman/web/docker-entrypoint.sh b/comanage-registry-mailman/web/docker-entrypoint.sh new file mode 100755 index 0000000..5fa07d7 --- /dev/null +++ b/comanage-registry-mailman/web/docker-entrypoint.sh @@ -0,0 +1,202 @@ +#! /bin/bash + +# GNU Mailman 3 Core for COmanage Registry Dockerfile entrypoint +# +# This bash script borrows heavily from a script by Abhilash Raj for +# GNU Mailman 3. See https://github.com/maxking/docker-mailman +# +# Portions licensed to the University Corporation for Advanced Internet +# Development, Inc. ("UCAID") under one or more contributor license agreements. +# See the NOTICE file distributed with this work for additional information +# regarding copyright ownership. +# +# UCAID licenses this file to you under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with the +# License. You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +set -e + +function wait_for_postgres () { + # Check if the postgres database is up and accepting connections before + # moving forward. + # TODO: Use python's psycopg2 module to do this in python instead of + # installing postgres-client in the image. + until psql $MAILMAN_DATABASE_URL -c '\l'; do + >&2 echo "Postgres is unavailable - sleeping" + sleep 1 + done + >&2 echo "Postgres is up - continuing" +} + +function wait_for_mysql () { + # Check if MySQL is up and accepting connections. + HOSTNAME=$(python <&2 echo "MySQL is unavailable - sleeping" + sleep 1 + done + >&2 echo "MySQL is up - continuing" +} + + +function check_or_create () { + # Check if the path exists, if not, create the directory. + if [[ ! -e dir ]]; then + echo "$1 does not exist, creating ..." + mkdir "$1" + fi +} + +# Configuration details that may be injected through environment +# variables or the contents of files. + +injectable_config_vars=( + HYPERKITTY_API_KEY + MAILMAN_DATABASE_URL + MAILMAN_REST_PASSWORD + MAILMAN_WEB_SECRET_KEY +) + +# If the file associated with a configuration variable is present then +# read the value from it into the appropriate variable. + +for config_var in "${injectable_config_vars[@]}" +do + eval file_name=\$"${config_var}_FILE"; + + if [ -e "$file_name" ]; then + declare "${config_var}"=`cat $file_name` + export "${config_var}"=`cat $file_name` + fi +done + +# Django needs DATABASE_URL. +if [[ MAILMAN_DATABASE_URL ]]; then + export DATABASE_URL="$MAILMAN_DATABASE_URL" +fi + +# Wait for the mailman core container to be ready. +if [[ ! -v MAILMAN_CORE_HOST ]]; +then + export MAILMAN_CORE_HOST="mailman-core" +fi + +if [[ ! -v MAILMAN_CORE_PORT ]]; +then + export MAILMAN_CORE_PORT=8001 +fi + +until nc -z -w 1 "${MAILMAN_CORE_HOST}" "${MAILMAN_CORE_PORT}" +do + echo "Waiting for Mailman core container..." + sleep 1 +done + +# Set SECRET_KEY which is required by the Django code. +if [[ -v MAILMAN_WEB_SECRET_KEY ]]; then + export SECRET_KEY="$MAILMAN_WEB_SECRET_KEY" +fi + +# Check if $SECRET_KEY is defined, if not, bail out. +if [[ ! -v SECRET_KEY ]]; then + echo "SECRET_KEY is not defined. Aborting." + exit 1 +fi + +# Check if $MAILMAN_DATABASE_URL is defined, if not, use a standard sqlite database. +# +# If the $MAILMAN_DATABASE_URL is defined and is postgres, check if it is available +# yet. Do not start the container before the postgresql boots up. +# +# If the $MAILMAN_DATABASE_URL is defined and is mysql, check if the database is +# available before the container boots up. +# +# TODO: Check the database type and detect if it is up based on that. For now, +# assume that postgres is being used if MAILMAN_DATABASE_URL is defined. + +if [[ ! -v MAILMAN_DATABASE_URL ]]; then + echo "MAILMAN_DATABASE_URL is not defined. Using sqlite database..." + export MAILMAN_DATABASE_URL=sqlite://mailmanweb.db + export MAILMAN_DATABASE_TYPE='sqlite' +fi + +if [[ "$MAILMAN_DATABASE_TYPE" = 'postgres' ]] +then + wait_for_postgres +elif [[ "$MAILMAN_DATABASE_TYPE" = 'mysql' ]] +then + wait_for_mysql +fi + +# Check if we are in the correct directory before running commands. +if [[ ! $(pwd) == '/opt/mailman-web' ]]; then + echo "Running in the wrong directory...switching to /opt/mailman-web" + cd /opt/mailman-web +fi + +# Check if the logs directory is setup. +if [[ ! -e /opt/mailman-web-data/logs/mailmanweb.log ]]; then + echo "Creating log file for mailman web" + mkdir -p /opt/mailman-web-data/logs/ + touch /opt/mailman-web-data/logs/mailmanweb.log +fi + +if [[ ! -e /opt/mailman-web-data/logs/uwsgi.log ]]; then + echo "Creating log file for uwsgi.." + touch /opt/mailman-web-data/logs/uwsgi.log +fi + +# Check if the settings_local.py file exists, if yes, copy it too. +if [[ -e /opt/mailman-web-data/settings_local.py ]]; then + echo "Copying settings_local.py ..." + cp /opt/mailman-web-data/settings_local.py /opt/mailman-web/settings_local.py +fi + +# Collect static for the django installation. +python manage.py collectstatic --noinput + +# Migrate all the data to the database if this is a new installation, otherwise +# this command will upgrade the database. +python manage.py migrate + +# If MAILMAN_ADMIN_USER and MAILMAN_ADMIN_EMAIL is defined create a new +# superuser for Django. There is no password setup so it can't login yet unless +# the password is reset. +if [[ -v MAILMAN_ADMIN_USER ]] && [[ -v MAILMAN_ADMIN_EMAIL ]]; +then + echo "Creating admin user $MAILMAN_ADMIN_USER ..." + python manage.py createsuperuser --noinput --username "$MAILMAN_ADMIN_USER"\ + --email "$MAILMAN_ADMIN_EMAIL" 2> /dev/null || \ + echo "Superuser $MAILMAN_ADMIN_USER already exists" +fi + +# If SERVE_FROM_DOMAIN is defined then rename the default `example.com` +# domain to the defined domain. +if [[ -v SERVE_FROM_DOMAIN ]]; +then + echo "Setting $SERVE_FROM_DOMAIN as the default domain ..." + python manage.py shell -c \ + "from django.contrib.sites.models import Site; Site.objects.filter(domain='example.com').update(domain='$SERVE_FROM_DOMAIN', name='$SERVE_FROM_DOMAIN')" +fi + +# Create a mailman user with the specific UID and GID and do not create home +# directory for it. Also chown the logs directory to write the files. +chown mailman:mailman /opt/mailman-web-data -R + +exec $@ diff --git a/comanage-registry-mailman/web/mailman-web/__init__.py b/comanage-registry-mailman/web/mailman-web/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/comanage-registry-mailman/web/mailman-web/manage.py b/comanage-registry-mailman/web/mailman-web/manage.py new file mode 100755 index 0000000..f9726f9 --- /dev/null +++ b/comanage-registry-mailman/web/mailman-web/manage.py @@ -0,0 +1,10 @@ +#!/usr/bin/env python +import os +import sys + +if __name__ == "__main__": + os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settings") + + from django.core.management import execute_from_command_line + + execute_from_command_line(sys.argv) diff --git a/comanage-registry-mailman/web/mailman-web/settings.py b/comanage-registry-mailman/web/mailman-web/settings.py new file mode 100644 index 0000000..cd31115 --- /dev/null +++ b/comanage-registry-mailman/web/mailman-web/settings.py @@ -0,0 +1,410 @@ +# -*- coding: utf-8 -*- +# Copyright (C) 1998-2016 by the Free Software Foundation, Inc. +# +# This file is part of Mailman Suite. +# +# Mailman Suite is free sofware: you can redistribute it and/or modify it +# under the terms of the GNU General Public License as published by the Free +# Software Foundation, either version 3 of the License, or (at your option) +# any later version. +# +# Mailman Suite is distributed in the hope that it will be useful, but +# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY +# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License +# for more details. + +# You should have received a copy of the GNU General Public License along +# with Mailman Suite. If not, see . +""" +Django Settings for Mailman Suite (hyperkitty + postorius) + +For more information on this file, see +https://docs.djangoproject.com/en/1.8/topics/settings/ + +For the full list of settings and their values, see +https://docs.djangoproject.com/en/1.8/ref/settings/ +""" + +# Build paths inside the project like this: os.path.join(BASE_DIR, ...) +import os +import dj_database_url +import sys + +BASE_DIR = os.path.dirname(os.path.abspath(__file__)) + +# SECURITY WARNING: keep the secret key used in production secret! +SECRET_KEY = os.environ.get('SECRET_KEY') + +# SECURITY WARNING: don't run with debug turned on in production! +DEBUG = False + +ADMINS = ( + ('Mailman Suite Admin', 'root@localhost'), +) + +SITE_ID = 1 + +# Hosts/domain names that are valid for this site; required if DEBUG is False +# See https://docs.djangoproject.com/en/1.8/ref/settings/#allowed-hosts +ALLOWED_HOSTS = [ + "localhost", # Archiving API from Mailman, keep it. + # "lists.your-domain.org", + # Add here all production URLs you may have. + "mailman-web", + "172.19.199.3", + os.environ.get('SERVE_FROM_DOMAIN'), + os.environ.get('DJANGO_ALLOWED_HOSTS'), +] + +# Mailman API credentials +MAILMAN_REST_API_URL = os.environ.get('MAILMAN_REST_URL', 'http://mailman-core:8001') +MAILMAN_REST_API_USER = os.environ.get('MAILMAN_REST_USER', 'restadmin') +MAILMAN_REST_API_PASS = os.environ.get('MAILMAN_REST_PASSWORD', 'restpass') +MAILMAN_ARCHIVER_KEY = os.environ.get('HYPERKITTY_API_KEY') +MAILMAN_ARCHIVER_FROM = os.environ.get('MAILMAN_HOST_IP', '172.19.199.2') + +# Application definition + +INSTALLED_APPS = ( + 'hyperkitty', + 'postorius', + 'django_mailman3', + # Uncomment the next line to enable the admin: + 'django.contrib.admin', + # Uncomment the next line to enable admin documentation: + # 'django.contrib.admindocs', + 'django.contrib.auth', + 'django.contrib.contenttypes', + 'django.contrib.sessions', + 'django.contrib.sites', + 'django.contrib.messages', + 'django.contrib.staticfiles', + 'rest_framework', + 'django_gravatar', + 'paintstore', + 'compressor', + 'haystack', + 'django_extensions', + 'django_q', + 'allauth', + 'allauth.account', + 'allauth.socialaccount', + 'django_mailman3.lib.auth.fedora', + 'allauth.socialaccount.providers.openid', + 'allauth.socialaccount.providers.github', + 'allauth.socialaccount.providers.gitlab', + 'allauth.socialaccount.providers.google', +) + +_MIDDLEWARE = ( + 'django.contrib.sessions.middleware.SessionMiddleware', + 'django.middleware.common.CommonMiddleware', + 'django.middleware.csrf.CsrfViewMiddleware', + 'django.middleware.locale.LocaleMiddleware', + 'django.contrib.auth.middleware.AuthenticationMiddleware', + 'django.contrib.auth.middleware.SessionAuthenticationMiddleware', + 'django.contrib.messages.middleware.MessageMiddleware', + 'django.middleware.clickjacking.XFrameOptionsMiddleware', + 'django.middleware.security.SecurityMiddleware', + 'django_mailman3.middleware.TimezoneMiddleware', + 'postorius.middleware.PostoriusMiddleware', +) + +# Use old-style Middleware class in Python 2 and released versions of +# Django-mailman3 don't support new style middlewares. + +if sys.version_info < (3, 0): + MIDDLEWARE_CLASSES = _MIDDLEWARE +else: + MIDDLEWARE = _MIDDLEWARE + +ROOT_URLCONF = 'urls' + + +TEMPLATES = [ + { + 'BACKEND': 'django.template.backends.django.DjangoTemplates', + 'DIRS': [], + 'APP_DIRS': True, + 'OPTIONS': { + 'context_processors': [ + 'django.template.context_processors.debug', + 'django.template.context_processors.i18n', + 'django.template.context_processors.media', + 'django.template.context_processors.static', + 'django.template.context_processors.tz', + 'django.template.context_processors.csrf', + 'django.template.context_processors.request', + 'django.contrib.auth.context_processors.auth', + 'django.contrib.messages.context_processors.messages', + 'django_mailman3.context_processors.common', + 'hyperkitty.context_processors.common', + 'postorius.context_processors.postorius', + ], + }, + }, +] + +WSGI_APPLICATION = 'wsgi.application' + + +# Database +# https://docs.djangoproject.com/en/1.8/ref/settings/#databases + + +# This uses $DATABASE_URL from the environment variable to create a +# django-style-config-dict. +# https://github.com/kennethreitz/dj-database-url +DATABASES = { + 'default': dj_database_url.config(conn_max_age=600) +} + +# If you're behind a proxy, use the X-Forwarded-Host header +# See https://docs.djangoproject.com/en/1.8/ref/settings/#use-x-forwarded-host +USE_X_FORWARDED_HOST = True + + +# Password validation +# https://docs.djangoproject.com/en/1.9/ref/settings/#auth-password-validators +AUTH_PASSWORD_VALIDATORS = [ + { + 'NAME': +'django.contrib.auth.password_validation.UserAttributeSimilarityValidator', + }, + { + 'NAME': +'django.contrib.auth.password_validation.MinimumLengthValidator', + }, + { + 'NAME': +'django.contrib.auth.password_validation.CommonPasswordValidator', + }, + { + 'NAME': +'django.contrib.auth.password_validation.NumericPasswordValidator', + }, +] + +# Internationalization +# https://docs.djangoproject.com/en/1.8/topics/i18n/ + +LANGUAGE_CODE = 'en-us' + +TIME_ZONE = 'UTC' + +USE_I18N = True + +USE_L10N = True + +USE_TZ = True + +STATIC_ROOT = '/opt/mailman-web-data/static' + +STATIC_URL = '/static/' + +# Additional locations of static files + + +# List of finder classes that know how to find static files in +# various locations. +STATICFILES_FINDERS = ( + 'django.contrib.staticfiles.finders.FileSystemFinder', + 'django.contrib.staticfiles.finders.AppDirectoriesFinder', + 'compressor.finders.CompressorFinder', +) + + +SESSION_SERIALIZER = 'django.contrib.sessions.serializers.PickleSerializer' + +LOGIN_URL = 'account_login' +LOGIN_REDIRECT_URL = 'list_index' +LOGOUT_URL = 'account_logout' + + +# Use SERVE_FROM_DOMAIN as the default domain in the email. +hostname = os.environ.get('SERVE_FROM_DOMAIN', 'localhost.local') +DEFAULT_FROM_EMAIL = 'postorius@{}'.format(hostname) +SERVER_EMAIL = 'root@{}'.format(hostname) + +# Change this when you have a real email backend +EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend' +EMAIL_HOST = os.environ.get('SMTP_HOST', '172.19.199.1') +EMAIL_PORT = os.environ.get('SMTP_PORT', 25) +EMAIL_HOST_USER = '' +EMAIL_HOST_PASSWORD = '' +EMAIL_USE_TLS = False + +# Compatibility with Bootstrap 3 +from django.contrib.messages import constants as messages # flake8: noqa +MESSAGE_TAGS = { + messages.ERROR: 'danger' +} + + +# +# Social auth +# +AUTHENTICATION_BACKENDS = ( + 'django.contrib.auth.backends.ModelBackend', + 'allauth.account.auth_backends.AuthenticationBackend', +) + +# Django Allauth +ACCOUNT_AUTHENTICATION_METHOD = "username_email" +ACCOUNT_EMAIL_REQUIRED = True +ACCOUNT_EMAIL_VERIFICATION = "mandatory" +# You probably want https in production, but this is a dev setup file +ACCOUNT_DEFAULT_HTTP_PROTOCOL = "https" +ACCOUNT_UNIQUE_EMAIL = True + +SOCIALACCOUNT_PROVIDERS = { + 'openid': { + 'SERVERS': [ + dict(id='yahoo', + name='Yahoo', + openid_url='http://me.yahoo.com'), + ], + }, + 'google': { + 'SCOPE': ['profile', 'email'], + 'AUTH_PARAMS': {'access_type': 'online'}, + }, + 'facebook': { + 'METHOD': 'oauth2', + 'SCOPE': ['email'], + 'FIELDS': [ + 'email', + 'name', + 'first_name', + 'last_name', + 'locale', + 'timezone', + ], + 'VERSION': 'v2.4', + }, +} + + +# django-compressor +# https://pypi.python.org/pypi/django_compressor +# +COMPRESS_PRECOMPILERS = ( + ('text/less', 'lessc {infile} {outfile}'), + ('text/x-scss', 'sassc -t compressed {infile} {outfile}'), + ('text/x-sass', 'sassc -t compressed {infile} {outfile}'), +) + +# On a production setup, setting COMPRESS_OFFLINE to True will bring a +# significant performance improvement, as CSS files will not need to be +# recompiled on each requests. It means running an additional "compress" +# management command after each code upgrade. +# http://django-compressor.readthedocs.io/en/latest/usage/#offline-compression +# COMPRESS_OFFLINE = True + +# +# Full-text search engine +# +HAYSTACK_CONNECTIONS = { + 'default': { + 'ENGINE': 'haystack.backends.whoosh_backend.WhooshEngine', + 'PATH': "/opt/mailman-web-data/fulltext_index", + # You can also use the Xapian engine, it's faster and more accurate, + # but requires another library. + # http://django-haystack.readthedocs.io/en/v2.4.1/installing_search_engines.html#xapian + # Example configuration for Xapian: + #'ENGINE': 'xapian_backend.XapianEngine' + }, +} + +import sys +# A sample logging configuration. The only tangible logging +# performed by this configuration is to send an email to +# the site admins on every HTTP 500 error when DEBUG=False. +# See http://docs.djangoproject.com/en/dev/topics/logging for +# more details on how to customize your logging configuration. +LOGGING = { + 'version': 1, + 'disable_existing_loggers': False, + 'filters': { + 'require_debug_false': { + '()': 'django.utils.log.RequireDebugFalse' + } + }, + 'handlers': { + 'mail_admins': { + 'level': 'ERROR', + 'filters': ['require_debug_false'], + 'class': 'django.utils.log.AdminEmailHandler' + }, + 'file':{ + 'level': 'INFO', + 'class': 'logging.handlers.RotatingFileHandler', + #'class': 'logging.handlers.WatchedFileHandler', + 'filename': os.environ.get('DJANGO_LOG_URL','/opt/mailman-web-data/logs/mailmanweb.log'), + 'formatter': 'verbose', + }, + 'console': { + 'class': 'logging.StreamHandler', + 'formatter': 'simple', + 'level': 'INFO', + 'stream': sys.stdout, + }, + # TODO: use an environment variable $DJ_LOG_URL to configure the logging + # using an environment variable. + }, + 'loggers': { + 'django.request': { + 'handlers': ['mail_admins', 'file'], + 'level': 'INFO', + 'propagate': True, + }, + 'django': { + 'handlers': ['file'], + 'level': 'INFO', + 'propagate': True, + }, + 'hyperkitty': { + 'handlers': ['file'], + 'level': 'INFO', + 'propagate': True, + }, + 'postorius': { + 'handlers': ['file'], + 'level': 'INFO', + 'propagate': True + }, + }, + 'formatters': { + 'verbose': { + 'format': '%(levelname)s %(asctime)s %(process)d %(name)s %(message)s' + }, + 'simple': { + 'format': '%(levelname)s %(message)s' + }, + }, + #'root': { + # 'handlers': ['file'], + # 'level': 'INFO', + #}, +} + + +if os.environ.get('LOG_TO_CONSOLE') == 'yes': + LOGGING['loggers']['django']['handlers'].append('console') + LOGGING['loggers']['django.request']['handlers'].append('console') +# HyperKitty-specific +# +# Only display mailing-lists from the same virtual host as the webserver +FILTER_VHOST = False + + +Q_CLUSTER = { + 'timeout': 300, + 'save_limit': 100, + 'orm': 'default', +} + +try: + from settings_local import * +except ImportError: + pass diff --git a/comanage-registry-mailman/web/mailman-web/settings_local.py b/comanage-registry-mailman/web/mailman-web/settings_local.py new file mode 100644 index 0000000..d6b5009 --- /dev/null +++ b/comanage-registry-mailman/web/mailman-web/settings_local.py @@ -0,0 +1,23 @@ +""" +Django Settings for Mailman Suite (hyperkitty + postorius) + +For more information on this file, see +https://docs.djangoproject.com/en/1.8/topics/settings/ + +For the full list of settings and their values, see +https://docs.djangoproject.com/en/1.8/ref/settings/ +""" + +# Try to get the address of Mailman Core automatically. +import os +import socket +MAILMAN_HOST_IP_AUTO = socket.gethostbyname('mailman-core') + +# Mailman API credentials +MAILMAN_REST_API_URL = os.environ.get('MAILMAN_REST_URL', 'http://mailman-core:8001') +MAILMAN_REST_API_USER = os.environ.get('MAILMAN_REST_USER', 'restadmin') +MAILMAN_REST_API_PASS = os.environ.get('MAILMAN_REST_PASSWORD', 'restpass') +MAILMAN_ARCHIVER_KEY = os.environ.get('HYPERKITTY_API_KEY') +MAILMAN_ARCHIVER_FROM = (MAILMAN_HOST_IP_AUTO, os.environ.get('MAILMAN_HOST_IP', '172.19.199.2')) + + diff --git a/comanage-registry-mailman/web/mailman-web/urls.py b/comanage-registry-mailman/web/mailman-web/urls.py new file mode 100644 index 0000000..a6b9174 --- /dev/null +++ b/comanage-registry-mailman/web/mailman-web/urls.py @@ -0,0 +1,35 @@ +# -*- coding: utf-8 -*- +# Copyright (C) 1998-2016 by the Free Software Foundation, Inc. +# +# This file is part of Postorius. +# +# Postorius is free software: you can redistribute it and/or modify it under +# the terms of the GNU General Public License as published by the Free +# Software Foundation, either version 3 of the License, or (at your option) +# any later version. +# +# Postorius is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or +# FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for +# more details. +# +# You should have received a copy of the GNU General Public License along with +# Postorius. If not, see . + + +from django.conf.urls import include, url +from django.contrib import admin +from django.core.urlresolvers import reverse_lazy +from django.views.generic import RedirectView + +urlpatterns = [ + url(r'^$', RedirectView.as_view( + url=reverse_lazy('list_index'), + permanent=True)), + url(r'^postorius/', include('postorius.urls')), + url(r'^hyperkitty/', include('hyperkitty.urls')), + url(r'', include('django_mailman3.urls')), + url(r'^accounts/', include('allauth.urls')), + # Django admin + url(r'^admin/', include(admin.site.urls)), +] diff --git a/comanage-registry-mailman/web/mailman-web/uwsgi.ini b/comanage-registry-mailman/web/mailman-web/uwsgi.ini new file mode 100644 index 0000000..4a24c52 --- /dev/null +++ b/comanage-registry-mailman/web/mailman-web/uwsgi.ini @@ -0,0 +1,48 @@ +[uwsgi] +# Port on which uwsgi will be listening. +uwsgi-socket = 0.0.0.0:8080 +http-socket = 0.0.0.0:8000 + +#Enable threading for python +enable-threads = true + +# Move to the directory wher the django files are. +chdir = /opt/mailman-web + +# Use the wsgi file provided with the django project. +wsgi-file = wsgi.py + +# Setup default number of processes and threads per process. +master = true +process = 2 +threads = 2 + +# Drop privielges and don't run as root. +uid = mailman +gid = mailman + +# Setup the django_q related worker processes. +attach-daemon = ./manage.py qcluster + +# Setup hyperkitty's cron jobs. +unique-cron = -1 -1 -1 -1 -1 ./manage.py runjobs minutely +unique-cron = -15 -1 -1 -1 -1 ./manage.py runjobs quarter_hourly +unique-cron = 0 -1 -1 -1 -1 ./manage.py runjobs hourly +unique-cron = 0 0 -1 -1 -1 ./manage.py runjobs daily +unique-cron = 0 0 1 -1 -1 ./manage.py runjobs monthly +unique-cron = 0 0 -1 -1 0 ./manage.py runjobs weekly +unique-cron = 0 0 1 1 -1 ./manage.py runjobs yearly + +# Setup the request log. +req-logger = file:/opt/mailman-web-data/logs/uwsgi.log + +# Log cron seperately. +logger = cron file:/opt/mailman-web-data/logs/uwsgi-cron.log +log-route = cron uwsgi-cron + +# Log qcluster commands seperately. +logger = qcluster file:/opt/mailman-web-data/logs/uwsgi-qcluster.log +log-route = qcluster uwsgi-daemons + +# Last log and it logs the rest of the stuff. +logger = file:/opt/mailman-web-data/logs/uwsgi-error.log diff --git a/comanage-registry-mailman/web/mailman-web/wsgi.py b/comanage-registry-mailman/web/mailman-web/wsgi.py new file mode 100755 index 0000000..8e59cd8 --- /dev/null +++ b/comanage-registry-mailman/web/mailman-web/wsgi.py @@ -0,0 +1,38 @@ +""" +WSGI config for HyperKitty project. + +It exposes the WSGI callable as a module-level variable named ``application``. + +For more information on this file, see +https://docs.djangoproject.com/en/{{ docs_version }}/howto/deployment/wsgi/ +""" + +import os + +# import sys +# import site + +# For some unknown reason, sometimes mod_wsgi fails to set the python paths to +# the virtualenv, with the 'python-path' option. You can do it here too. +# +# # Remember original sys.path. +# prev_sys_path = list(sys.path) +# # Add here, for the settings module +# site.addsitedir(os.path.abspath(os.path.dirname(__file__))) +# # Add the virtualenv +# venv = os.path.join(os.path.abspath(os.path.dirname(__file__)), +# '..', 'lib', 'python2.6', 'site-packages') +# site.addsitedir(venv) +# # Reorder sys.path so new directories at the front. +# new_sys_path = [] +# for item in list(sys.path): +# if item not in prev_sys_path: +# new_sys_path.append(item) +# sys.path.remove(item) +# sys.path[:0] = new_sys_path + +from django.core.wsgi import get_wsgi_application + +os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settings") + +application = get_wsgi_application()