Tier secrets

README.md

@@ -141,15 +141,23 @@ There are three primary ways to provide Grouper and additional configuration fil
 
 Docker Config and Docker Secrets are Docker's way of providing configurations files to a container at runtime. The primary difference between the Config and Secrets functionality is that Secrets is designed to protect resources/files that are sensitive.
 
-This container will make any secrets with secret names prepended with `grouper_` available to the appropriate Grouper component's conf directory (i.e. `<GROUPER_HOME>/conf` or `WEB-INF/classes`). Any secrets with secret names starting with `shib_` will be available in the Shibboleth SP `/etc/shibboleth/` directory. Any secrets with secret names starting with `httpd_` will be available to `/etc/httpd/conf.d` directory. Finally, if a secret with the name of `host-key.pem` will be mapped to the httpd TLS cert used by Grouper UI, Grouper WS, and Grouper SCIM Server containers. These files will supercede any found in the underlying image.
+For passing full files into the container, this container will make any secrets with secret names prepended with `grouper_` available to the appropriate Grouper component's conf directory (i.e. `<GROUPER_HOME>/conf` or `WEB-INF/classes`). Any secrets with secret names starting with `shib_` will be available in the Shibboleth SP `/etc/shibboleth/` directory. Any secrets with secret names starting with `httpd_` will be available to `/etc/httpd/conf.d` directory. Finally, if a secret with the name of `host-key.pem` will be mapped to the httpd TLS cert used by Grouper UI, Grouper WS, and Grouper SCIM Server containers. These files will supercede any found in the underlying image.
+
+Docker Secrets can also be used to pass in strings, such as a database connection string password, into the component config. To pass in the Grouper database connection string, one might set the property and value as such:
+
+```text
+hibernate.connection.password.elConfig = ${java.lang.System.getenv().get('GROUPER_DATABASE_PASSWORD_FILE') != null ? org.apache.commons.io.FileUtils.readFileToString(java.lang.System.getenv().get('GROUPER_DATABASE_PASSWORD_FILE'), "utf-8") : java.lang.System.getenv().get('GROUPER_DATABASE_PASSWORD') }
+```
+
+Note that the default property name has been changed by appending `.elConfig`. (This causes Grouper to evaluate the string before saving the value.) The expression allows deployers to use a file containing only the database password as a Docker Secret and reference the file name via the `GROUPER_DATABASE_PASSWORD_FILE` environment property. This allows the config files to be baked into the image, if desired. Also, but not recommended, the database password could just be set in the Docker Service definition as an environment variable, `GROUPER_DATABASE_PASSWORD`. (Technically the expression can be broken up and just the desired functionality used.) Of course, using Grouper's MorphString functionality is supported and likely is the best option, but does require more effort in setting it up.
 
 Secrets can be managed using the `docker secret` command: `docker secret create grouper_grouper.hibernate.properties ./grouper.hibernate.properties`. This will securely store the file in the swarm. Secrets can then be assigned to the service `docker service create -d --name daemon --secret grouper_grouper.hibernate.properties --secret grouper_sources.xml tier/grouper daemon`.
 
-> `docker run` does not support secrets; Bind mounts need to be used instead.
+> `docker run` does not support secrets; Bind mounts need to be used instead, which is technically what Docker Compose does when not running against a Swarm.
 
 ### Bind Mounts
 
-Bind mounts can be used to connect files/folders on the Docker host into the container's file system. Unless running in swarm mode, the secrets are not supported, so we can use a bind mount to provide the container with the configuration files.
+Bind mounts can be used to connect files/folders on the Docker host into the container's file system. Unless running in swarm mode, Docker Secrets are not supported, so we can use a bind mount to provide the container with the configuration files.
 
 ```console
 $ docker run --detach --name daemon \
@@ -215,6 +223,7 @@ Here is a list of significant directories and files that deployers should be awa
 To examine baseline image files, one might run `docker run --name=temp -it tier/grouper bash` and browse through these file system endpoints. While the container is running one may copy files out of the image/container using something like `docker cp containerId:/opt/grouper/grouper.api/conf/grouper.properties .`, which will copy the `grouper.properties` to the Docker client's present working directory. These files can then be edited and applied via the mechanisms outlined above.
 
 # Web Application Endpoints
+
 Here is a list of significant web endpoints that deployers should be aware of:
 
 - `/grouper/`: location of the Grouper UI application

container_files/tier-support/supervisord-tomee.conf

@@ -1,5 +1,5 @@
 [supervisord]
-logfile=/tmp/logpipe                               ; supervisord log file
+logfile=/tmp/logsuperd                               ; supervisord log file
 logfile_maxbytes=0                           ; maximum size of logfile before rotation
 loglevel=error                                  ; info, debug, warn, trace
 nodaemon=true                                  ; run supervisord as a daemon
@@ -16,16 +16,16 @@ serverurl=unix:///tmp/supervisor.sock         ; use a unix:// URL  for a unix so
 
 [program:httpd]
 command=httpd -DFOREGROUND
-stderr_logfile = /tmp/logpipe
+stderr_logfile = /tmp/loghttpd
 stderr_logfile_maxbytes=0
-stdout_logfile = /tmp/logpipe
+stdout_logfile = /tmp/loghttpd
 stdout_logfile_maxbytes=0
 
 [program:tomee]
 user=tomcat
 command=/opt/tomee/bin/catalina.sh run 
-stderr_logfile = /tmp/logpipe
+stderr_logfile = /tmp/logtomcat
 stderr_logfile_maxbytes=0
-stdout_logfile = /tmp/logpipe
+stdout_logfile = /tmp/logtomcat
 stdout_logfile_maxbytes=0
 

container_files/usr-local-bin/library.sh

@@ -90,4 +90,4 @@ prepWS() {
     fi
 
     cp /opt/tier-support/grouper-ws.xml /opt/tomcat/conf/Catalina/localhost/
-}
+}

test-compose/README.md

@@ -50,10 +50,11 @@ Note that when accessing the Grouper UI, Grouper WS, or Shibboleth IdP, your bro
 
 # Additional Notes
 
-- Docker `configs` are not supported by Docker Compose, so those are represented in the `docker-compose.yml` file as bind mount volumes.
-- The Grouper config files in the `data` image's `conf` directory are used to build the sample grouper database and ldap store. They are not used when the container is instantiated.
+- In this example, we use a variety of ways to pass in passwords (Grouper database, LDAP, Grouper Client, and RabbitMQ). The point is to demonstrate possibilities and not demonstrating what is required. (See the image readme for more details.)
+- Docker `configs` are not supported by Docker Compose (when run in a non-Swarm mode), so those are represented in the `docker-compose.yml` file as bind mount volumes.
+- The Grouper config files in the `data` image's `conf` directory are used to build the sample grouper database and ldap store. They are not used when the container is instantiated as there is no Grouper runtime in this container.
 - The containers will use Docker Secrets and bind mounts for non-sensitive files that are read from the `configs-ans-secrets` directory in the `test-compose` directory.
-- With regard to RabbitMQ, the deployer must manually add a queue named `sampleQueue` to see Grouper messages in RabbitMQ. Messages will be dropped by RabbitMQ until this occurs.
+- With regard to RabbitMQ, the deployer must manually add a queue named `sampleQueue` to see Grouper messages in RabbitMQ. Messages will be dropped by RabbitMQ (and the Grouper Deamon will log errors) until this occurs.
 - In this example, we don't care about the IdP secrets. They are baked into the overlay instead of using Docker Secrets. (This is not best practice for an IdP configuration, but that isn't the focus of this example.)
 
 # Future TODOs

test-compose/configs-and-secrets/grouper/grouper-loader.properties

@@ -11,13 +11,13 @@
 ldap.demo.url = ldap://data:389/dc=example,dc=edu
 
 #optional, if authenticated
-#ldap.personLdap.user = uid=someapp,ou=people,dc=myschool,dc=edu
+ldap.demo.user = cn=admin,dc=internet2,dc=edu
 
 #optional, if authenticated note the password can be stored encrypted in an external file
-#ldap.personLdap.pass = secret
+ldap.demo.pass = ${java.lang.System.getenv().get('SUBJECT_SOURCE_LDAP_PASSWORD_FILE') != null ? org.apache.commons.io.FileUtils.readFileToString(java.lang.System.getenv().get('SUBJECT_SOURCE_LDAP_PASSWORD_FILE'), "utf-8") : java.lang.System.getenv().get('SUBJECT_SOURCE_LDAP_PASSWORD')}
 
 #optional, if you are using tls, set this to true.  Generally you will not be using an SSL URL to use TLS...
-#ldap.personLdap.tls = false
+ldap.demo.tls = false
 
 #optional, if using sasl
 #ldap.personLdap.saslAuthorizationId =

test-compose/configs-and-secrets/grouper/grouper.client.properties

@@ -55,7 +55,7 @@ grouperClient.webService.login = banderson
 
 # password for shared secret authentication to web service
 # or you can put a filename with an encrypted password
-grouperClient.webService.password = password
+grouperClient.webService.password = ${java.lang.System.getenv().get('GROUPER_CLIENT_WEBSERVICE_PASSWORD_FILE') != null ? org.apache.commons.io.FileUtils.readFileToString(java.lang.System.getenv().get('GROUPER_CLIENT_WEBSERVICE_PASSWORD_FILE'), "utf-8") : java.lang.System.getenv().get('GROUPER_CLIENT_WEBSERVICE_PASSWORD') }
 
 
 ################################
@@ -98,9 +98,9 @@ grouper.messaging.system.rabbitmq.name = rabbitmq
 grouper.messaging.system.rabbitmq.defaultSystemName = rabbitmqSystem
 
 grouper.messaging.system.rabbitmq.user = guest
- 
+
 #pass
-grouper.messaging.system.rabbitmq.password = guest
+grouper.messaging.system.rabbitmq.password = ${java.lang.System.getenv().get('RABBITMQ_PASSWORD_FILE') != null ? org.apache.commons.io.FileUtils.readFileToString(java.lang.System.getenv().get('RABBITMQ_PASSWORD_FILE'), "utf-8") : java.lang.System.getenv().get('RABBITMQ_PASSWORD') }
 # set the following three properties if you want to use TLS connection to rabbitmq. All three need to be populated.
 # TLS Version
 #grouper.messaging.system.rabbitmqSystem.tlsVersion = TLSv1.1

test-compose/configs-and-secrets/grouper/grouper.hibernate.properties

@@ -26,5 +26,4 @@ hibernate.connection.username         = root
 # If you are using an empty password, depending upon your version of
 # Java and Ant you may need to specify a password of "".
 # Note: you can keep passwords external and encrypted: https://bugs.internet2.edu/jira/browse/GRP-122
-hibernate.connection.password         = 
-
+hibernate.connection.password.elConfig = ${java.lang.System.getenv().get('GROUPER_DATABASE_PASSWORD_FILE') != null ? org.apache.commons.io.FileUtils.readFileToString(java.lang.System.getenv().get('GROUPER_DATABASE_PASSWORD_FILE'), "utf-8") : java.lang.System.getenv().get('GROUPER_DATABASE_PASSWORD') }

test-compose/configs-and-secrets/grouper/rabbitmq_password.txt

@@ -0,0 +1 @@
+guest