Tier secrets (#12)

* POC for pulling in secrets into Grouper property files
* Adding more secrets to the POC.

README.md

@@ -141,15 +141,23 @@ There are three primary ways to provide Grouper and additional configuration fil
 
 Docker Config and Docker Secrets are Docker's way of providing configurations files to a container at runtime. The primary difference between the Config and Secrets functionality is that Secrets is designed to protect resources/files that are sensitive.
 
-This container will make any secrets with secret names prepended with `grouper_` available to the appropriate Grouper component's conf directory (i.e. `<GROUPER_HOME>/conf` or `WEB-INF/classes`). Any secrets with secret names starting with `shib_` will be available in the Shibboleth SP `/etc/shibboleth/` directory. Any secrets with secret names starting with `httpd_` will be available to `/etc/httpd/conf.d` directory. Finally, if a secret with the name of `host-key.pem` will be mapped to the httpd TLS cert used by Grouper UI, Grouper WS, and Grouper SCIM Server containers. These files will supercede any found in the underlying image.
+For passing full files into the container, this container will make any secrets with secret names prepended with `grouper_` available to the appropriate Grouper component's conf directory (i.e. `<GROUPER_HOME>/conf` or `WEB-INF/classes`). Any secrets with secret names starting with `shib_` will be available in the Shibboleth SP `/etc/shibboleth/` directory. Any secrets with secret names starting with `httpd_` will be available to `/etc/httpd/conf.d` directory. Finally, if a secret with the name of `host-key.pem` will be mapped to the httpd TLS cert used by Grouper UI, Grouper WS, and Grouper SCIM Server containers. These files will supercede any found in the underlying image.
+
+Docker Secrets can also be used to pass in strings, such as a database connection string password, into the component config. To pass in the Grouper database connection string, one might set the property and value as such:
+
+```text
+hibernate.connection.password.elConfig = ${java.lang.System.getenv().get('GROUPER_DATABASE_PASSWORD_FILE') != null ? org.apache.commons.io.FileUtils.readFileToString(java.lang.System.getenv().get('GROUPER_DATABASE_PASSWORD_FILE'), "utf-8") : java.lang.System.getenv().get('GROUPER_DATABASE_PASSWORD') }
+```
+
+Note that the default property name has been changed by appending `.elConfig`. (This causes Grouper to evaluate the string before saving the value.) The expression allows deployers to use a file containing only the database password as a Docker Secret and reference the file name via the `GROUPER_DATABASE_PASSWORD_FILE` environment property. This allows the config files to be baked into the image, if desired. Also, but not recommended, the database password could just be set in the Docker Service definition as an environment variable, `GROUPER_DATABASE_PASSWORD`. (Technically the expression can be broken up and just the desired functionality used.) Of course, using Grouper's MorphString functionality is supported and likely is the best option, but does require more effort in setting it up.
 
 Secrets can be managed using the `docker secret` command: `docker secret create grouper_grouper.hibernate.properties ./grouper.hibernate.properties`. This will securely store the file in the swarm. Secrets can then be assigned to the service `docker service create -d --name daemon --secret grouper_grouper.hibernate.properties --secret grouper_sources.xml tier/grouper daemon`.
 
-> `docker run` does not support secrets; Bind mounts need to be used instead.
+> `docker run` does not support secrets; Bind mounts need to be used instead, which is technically what Docker Compose does when not running against a Swarm.
 
 ### Bind Mounts
 
-Bind mounts can be used to connect files/folders on the Docker host into the container's file system. Unless running in swarm mode, the secrets are not supported, so we can use a bind mount to provide the container with the configuration files.
+Bind mounts can be used to connect files/folders on the Docker host into the container's file system. Unless running in swarm mode, Docker Secrets are not supported, so we can use a bind mount to provide the container with the configuration files.
 
 ```console
 $ docker run --detach --name daemon \
@@ -215,6 +223,7 @@ Here is a list of significant directories and files that deployers should be awa
 To examine baseline image files, one might run `docker run --name=temp -it tier/grouper bash` and browse through these file system endpoints. While the container is running one may copy files out of the image/container using something like `docker cp containerId:/opt/grouper/grouper.api/conf/grouper.properties .`, which will copy the `grouper.properties` to the Docker client's present working directory. These files can then be edited and applied via the mechanisms outlined above.
 
 # Web Application Endpoints
+
 Here is a list of significant web endpoints that deployers should be aware of:
 
 - `/grouper/`: location of the Grouper UI application

container_files/tier-support/supervisord-tomee.conf

@@ -1,5 +1,5 @@
 [supervisord]
-logfile=/tmp/logpipe                               ; supervisord log file
+logfile=/tmp/logsuperd                               ; supervisord log file
 logfile_maxbytes=0                           ; maximum size of logfile before rotation
 loglevel=error                                  ; info, debug, warn, trace
 nodaemon=true                                  ; run supervisord as a daemon
@@ -16,16 +16,16 @@ serverurl=unix:///tmp/supervisor.sock         ; use a unix:// URL  for a unix so
 
 [program:httpd]
 command=httpd -DFOREGROUND
-stderr_logfile = /tmp/logpipe
+stderr_logfile = /tmp/loghttpd
 stderr_logfile_maxbytes=0
-stdout_logfile = /tmp/logpipe
+stdout_logfile = /tmp/loghttpd
 stdout_logfile_maxbytes=0
 
 [program:tomee]
 user=tomcat
 command=/opt/tomee/bin/catalina.sh run 
-stderr_logfile = /tmp/logpipe
+stderr_logfile = /tmp/logtomcat
 stderr_logfile_maxbytes=0
-stdout_logfile = /tmp/logpipe
+stdout_logfile = /tmp/logtomcat
 stdout_logfile_maxbytes=0
 

container_files/usr-local-bin/library.sh

@@ -90,4 +90,4 @@ prepWS() {
     fi
 
     cp /opt/tier-support/grouper-ws.xml /opt/tomcat/conf/Catalina/localhost/
-}
+}

test-compose/README.md

@@ -50,10 +50,11 @@ Note that when accessing the Grouper UI, Grouper WS, or Shibboleth IdP, your bro
 
 # Additional Notes
 
-- Docker `configs` are not supported by Docker Compose, so those are represented in the `docker-compose.yml` file as bind mount volumes.
-- The Grouper config files in the `data` image's `conf` directory are used to build the sample grouper database and ldap store. They are not used when the container is instantiated.
+- In this example, we use a variety of ways to pass in passwords (Grouper database, LDAP, Grouper Client, and RabbitMQ). The point is to demonstrate possibilities and not demonstrating what is required. (See the image readme for more details.)
+- Docker `configs` are not supported by Docker Compose (when run in a non-Swarm mode), so those are represented in the `docker-compose.yml` file as bind mount volumes.
+- The Grouper config files in the `data` image's `conf` directory are used to build the sample grouper database and ldap store. They are not used when the container is instantiated as there is no Grouper runtime in this container.
 - The containers will use Docker Secrets and bind mounts for non-sensitive files that are read from the `configs-ans-secrets` directory in the `test-compose` directory.
-- With regard to RabbitMQ, the deployer must manually add a queue named `sampleQueue` to see Grouper messages in RabbitMQ. Messages will be dropped by RabbitMQ until this occurs.
+- With regard to RabbitMQ, the deployer must manually add a queue named `sampleQueue` to see Grouper messages in RabbitMQ. Messages will be dropped by RabbitMQ (and the Grouper Deamon will log errors) until this occurs.
 - In this example, we don't care about the IdP secrets. They are baked into the overlay instead of using Docker Secrets. (This is not best practice for an IdP configuration, but that isn't the focus of this example.)
 
 # Future TODOs

test-compose/configs-and-secrets/grouper/grouper-loader.properties

@@ -11,13 +11,13 @@
 ldap.demo.url = ldap://data:389/dc=example,dc=edu
 
 #optional, if authenticated
-#ldap.personLdap.user = uid=someapp,ou=people,dc=myschool,dc=edu
+ldap.demo.user = cn=admin,dc=internet2,dc=edu
 
 #optional, if authenticated note the password can be stored encrypted in an external file
-#ldap.personLdap.pass = secret
+ldap.demo.pass = ${java.lang.System.getenv().get('SUBJECT_SOURCE_LDAP_PASSWORD_FILE') != null ? org.apache.commons.io.FileUtils.readFileToString(java.lang.System.getenv().get('SUBJECT_SOURCE_LDAP_PASSWORD_FILE'), "utf-8") : java.lang.System.getenv().get('SUBJECT_SOURCE_LDAP_PASSWORD')}
 
 #optional, if you are using tls, set this to true.  Generally you will not be using an SSL URL to use TLS...
-#ldap.personLdap.tls = false
+ldap.demo.tls = false
 
 #optional, if using sasl
 #ldap.personLdap.saslAuthorizationId =

test-compose/configs-and-secrets/grouper/grouper.client.properties

@@ -55,7 +55,7 @@ grouperClient.webService.login = banderson
 
 # password for shared secret authentication to web service
 # or you can put a filename with an encrypted password
-grouperClient.webService.password = password
+grouperClient.webService.password = ${java.lang.System.getenv().get('GROUPER_CLIENT_WEBSERVICE_PASSWORD_FILE') != null ? org.apache.commons.io.FileUtils.readFileToString(java.lang.System.getenv().get('GROUPER_CLIENT_WEBSERVICE_PASSWORD_FILE'), "utf-8") : java.lang.System.getenv().get('GROUPER_CLIENT_WEBSERVICE_PASSWORD') }
 
 
 ################################
@@ -98,9 +98,9 @@ grouper.messaging.system.rabbitmq.name = rabbitmq
 grouper.messaging.system.rabbitmq.defaultSystemName = rabbitmqSystem
 
 grouper.messaging.system.rabbitmq.user = guest
- 
+
 #pass
-grouper.messaging.system.rabbitmq.password = guest
+grouper.messaging.system.rabbitmq.password = ${java.lang.System.getenv().get('RABBITMQ_PASSWORD_FILE') != null ? org.apache.commons.io.FileUtils.readFileToString(java.lang.System.getenv().get('RABBITMQ_PASSWORD_FILE'), "utf-8") : java.lang.System.getenv().get('RABBITMQ_PASSWORD') }
 # set the following three properties if you want to use TLS connection to rabbitmq. All three need to be populated.
 # TLS Version
 #grouper.messaging.system.rabbitmqSystem.tlsVersion = TLSv1.1

test-compose/configs-and-secrets/grouper/grouper.hibernate.properties

@@ -26,5 +26,4 @@ hibernate.connection.username         = root
 # If you are using an empty password, depending upon your version of
 # Java and Ant you may need to specify a password of "".
 # Note: you can keep passwords external and encrypted: https://bugs.internet2.edu/jira/browse/GRP-122
-hibernate.connection.password         = 
-
+hibernate.connection.password.elConfig = ${java.lang.System.getenv().get('GROUPER_DATABASE_PASSWORD_FILE') != null ? org.apache.commons.io.FileUtils.readFileToString(java.lang.System.getenv().get('GROUPER_DATABASE_PASSWORD_FILE'), "utf-8") : java.lang.System.getenv().get('GROUPER_DATABASE_PASSWORD') }

test-compose/configs-and-secrets/grouper/rabbitmq_password.txt

@@ -0,0 +1 @@
+guest

test-compose/configs-and-secrets/grouper/subject.properties

@@ -0,0 +1,75 @@
+subject.sources.xml.location =
+
+subjectApi.source.ldap.id = ldap
+subjectApi.source.ldap.name = EDU Ldap 
+subjectApi.source.ldap.types = person
+subjectApi.source.ldap.adapterClass = edu.internet2.middleware.grouper.subj.GrouperJndiSourceAdapter
+subjectApi.source.ldap.param.INITIAL_CONTEXT_FACTORY.value = com.sun.jndi.ldap.LdapCtxFactory
+subjectApi.source.ldap.param.PROVIDER_URL.value = ldap://data:389
+subjectApi.source.ldap.param.SECURITY_AUTHENTICATION.value = simple
+subjectApi.source.ldap.param.SECURITY_PRINCIPAL.value = cn=admin,dc=internet2,dc=edu
+subjectApi.source.ldap.param.SECURITY_CREDENTIALS.value.elConfig = ${java.lang.System.getenv().get('SUBJECT_SOURCE_LDAP_PASSWORD_FILE') != null ? org.apache.commons.io.FileUtils.readFileToString(java.lang.System.getenv().get('SUBJECT_SOURCE_LDAP_PASSWORD_FILE'), "utf-8") : java.lang.System.getenv().get('SUBJECT_SOURCE_LDAP_PASSWORD')}
+subjectApi.source.ldap.param.SubjectID_AttributeType.value = uid
+subjectApi.source.ldap.param.SubjectID_formatToLowerCase.value = false
+subjectApi.source.ldap.param.Name_AttributeType.value = cn
+subjectApi.source.ldap.param.Description_AttributeType.value = cn
+subjectApi.source.ldap.param.VTLDAP_VALIDATOR.value = ConnectLdapValidator
+subjectApi.source.ldap.param.subjectVirtualAttribute_0_searchAttribute0.value = ${subjectUtils.defaultIfBlank(subject.getAttributeValueOrCommaSeparated('uid'), "")},${subjectUtils.defaultIfBlank(subject.getAttributeValueOrCommaSeparated('cn'), "")},${subjectUtils.defaultIfBlank(subject.getAttributeValueOrCommaSeparated('exampleEduRegId'), "")}
+subjectApi.source.ldap.param.sortAttribute0.value = cn
+subjectApi.source.ldap.param.searchAttribute0.value = searchAttribute0
+
+# STATUS SECTION for searches to filter out inactives and allow
+# the user to filter by status with e.g. status=all
+# this is optional, and advanced
+#
+# field in database or ldap or endpoint that is the status field
+#subjectApi.source.example.param.statusDatastoreFieldName.value = status
+
+# search string from user which represents the status.  e.g. status=active
+#subjectApi.source.example.param.statusLabel.value = status
+
+# available statuses from screen (if not specified, any will be allowed). comma separated list.
+# Note, this is optional and you probably dont want to configure it, it is mostly necessary
+# when you have multiple sources with statuses...  if someone types an invalid status
+# and you have this configured, it will not filter by it
+#subjectApi.source.example.param.statusesFromUser.value = Active, Inactive, Pending, All
+
+# all label from the user
+#subjectApi.source.example.param.statusAllFromUser.value = All
+
+# if no status is specified, this will be used (e.g. for active only).  Note, the value should be of the
+# form the user would type in
+#subjectApi.source.example.param.statusSearchDefault.value = status=active
+
+# translate between screen values of status, and the data store value.  Increment the 0 to 1, 2, etc for more translations.
+# so the user could enter: status=active, and that could translate to status_col=A.  The 'user' is what the user types in,
+# the 'datastore' is what is in the datastore.  The user part is not case-sensitive.  Note, this could be a many to one
+#subjectApi.source.example.param.statusTranslateUser0.value = active
+#subjectApi.source.example.param.statusTranslateDatastore0.value = A
+
+# subject identifier to store in grouper's member table.  this is used to increase speed of loader and perhaps for provisioning
+# you can have up to max 1 subject identifier
+#subjectApi.source.example.param.subjectIdentifierAttribute0.value = uid
+
+#searchSubject: find a subject by ID.  ID is generally an opaque and permanent identifier, e.g. 12345678.
+#  Each subject has one and only on ID.  Returns one result when searching for one ID.
+subjectApi.source.ldap.search.searchSubject.param.filter.value = (&(uid=%TERM%)(objectclass=person))
+subjectApi.source.ldap.search.searchSubject.param.scope.value = SUBTREE_SCOPE
+subjectApi.source.ldap.search.searchSubject.param.base.value = ou=people,dc=internet2,dc=edu
+
+#searchSubjectByIdentifier: find a subject by identifier.  Identifier is anything that uniquely
+#  identifies the user, e.g. jsmith or jsmith@institution.edu.
+#  Subjects can have multiple identifiers.  Note: it is nice to have if identifiers are unique
+#  even across sources.  Returns one result when searching for one identifier.
+subjectApi.source.ldap.search.searchSubjectByIdentifier.param.filter.value = (&(|(uid=%TERM%)(employeeNumber=%TERM%))(objectclass=person))
+subjectApi.source.ldap.search.searchSubjectByIdentifier.param.scope.value = SUBTREE_SCOPE
+subjectApi.source.ldap.search.searchSubjectByIdentifier.param.base.value = ou=people,dc=internet2,dc=edu
+
+#   search: find subjects by free form search.  Returns multiple results.
+
+subjectApi.source.ldap.search.search.param.filter.value = (&(|(|(uid=%TERM%)(cn=*%TERM%*))(uid=%TERM%*))(objectclass=person))
+subjectApi.source.ldap.search.search.param.scope.value = SUBTREE_SCOPE
+subjectApi.source.ldap.search.search.param.base.value = ou=people,dc=internet2,dc=edu
+
+subjectApi.source.ldap.attributes = givenName, sn, uid, mail, employeeNumber
+subjectApi.source.ldap.internalAttributes = searchAttribute0

test-compose/data/container_files/conf/grouper.hibernate.properties

@@ -26,5 +26,4 @@ hibernate.connection.username         = root
 # If you are using an empty password, depending upon your version of
 # Java and Ant you may need to specify a password of "".
 # Note: you can keep passwords external and encrypted: https://bugs.internet2.edu/jira/browse/GRP-122
-hibernate.connection.password         = 
-
+hibernate.connection.password =

test-compose/data/container_files/conf/subject.properties

@@ -0,0 +1,75 @@
+subject.sources.xml.location =
+
+subjectApi.source.ldap.id = ldap
+subjectApi.source.ldap.name = EDU Ldap 
+subjectApi.source.ldap.types = person
+subjectApi.source.ldap.adapterClass = edu.internet2.middleware.grouper.subj.GrouperJndiSourceAdapter
+subjectApi.source.ldap.param.INITIAL_CONTEXT_FACTORY.value = com.sun.jndi.ldap.LdapCtxFactory
+subjectApi.source.ldap.param.PROVIDER_URL.value = ldap://localhost:389
+subjectApi.source.ldap.param.SECURITY_AUTHENTICATION.value = simple
+subjectApi.source.ldap.param.SECURITY_PRINCIPAL.value = cn=admin,dc=internet2,dc=edu
+subjectApi.source.ldap.param.SECURITY_CREDENTIALS.value = password
+subjectApi.source.ldap.param.SubjectID_AttributeType.value = uid
+subjectApi.source.ldap.param.SubjectID_formatToLowerCase.value = false
+subjectApi.source.ldap.param.Name_AttributeType.value = cn
+subjectApi.source.ldap.param.Description_AttributeType.value = cn
+subjectApi.source.ldap.param.VTLDAP_VALIDATOR.value = ConnectLdapValidator
+subjectApi.source.ldap.param.subjectVirtualAttribute_0_searchAttribute0.value = ${subjectUtils.defaultIfBlank(subject.getAttributeValueOrCommaSeparated('uid'), "")},${subjectUtils.defaultIfBlank(subject.getAttributeValueOrCommaSeparated('cn'), "")},${subjectUtils.defaultIfBlank(subject.getAttributeValueOrCommaSeparated('exampleEduRegId'), "")}
+subjectApi.source.ldap.param.sortAttribute0.value = cn
+subjectApi.source.ldap.param.searchAttribute0.value = searchAttribute0
+
+# STATUS SECTION for searches to filter out inactives and allow
+# the user to filter by status with e.g. status=all
+# this is optional, and advanced
+#
+# field in database or ldap or endpoint that is the status field
+#subjectApi.source.example.param.statusDatastoreFieldName.value = status
+
+# search string from user which represents the status.  e.g. status=active
+#subjectApi.source.example.param.statusLabel.value = status
+
+# available statuses from screen (if not specified, any will be allowed). comma separated list.
+# Note, this is optional and you probably dont want to configure it, it is mostly necessary
+# when you have multiple sources with statuses...  if someone types an invalid status
+# and you have this configured, it will not filter by it
+#subjectApi.source.example.param.statusesFromUser.value = Active, Inactive, Pending, All
+
+# all label from the user
+#subjectApi.source.example.param.statusAllFromUser.value = All
+
+# if no status is specified, this will be used (e.g. for active only).  Note, the value should be of the
+# form the user would type in
+#subjectApi.source.example.param.statusSearchDefault.value = status=active
+
+# translate between screen values of status, and the data store value.  Increment the 0 to 1, 2, etc for more translations.
+# so the user could enter: status=active, and that could translate to status_col=A.  The 'user' is what the user types in,
+# the 'datastore' is what is in the datastore.  The user part is not case-sensitive.  Note, this could be a many to one
+#subjectApi.source.example.param.statusTranslateUser0.value = active
+#subjectApi.source.example.param.statusTranslateDatastore0.value = A
+
+# subject identifier to store in grouper's member table.  this is used to increase speed of loader and perhaps for provisioning
+# you can have up to max 1 subject identifier
+#subjectApi.source.example.param.subjectIdentifierAttribute0.value = uid
+
+#searchSubject: find a subject by ID.  ID is generally an opaque and permanent identifier, e.g. 12345678.
+#  Each subject has one and only on ID.  Returns one result when searching for one ID.
+subjectApi.source.ldap.search.searchSubject.param.filter.value = (&(uid=%TERM%)(objectclass=person))
+subjectApi.source.ldap.search.searchSubject.param.scope.value = SUBTREE_SCOPE
+subjectApi.source.ldap.search.searchSubject.param.base.value = ou=people,dc=internet2,dc=edu
+
+#searchSubjectByIdentifier: find a subject by identifier.  Identifier is anything that uniquely
+#  identifies the user, e.g. jsmith or jsmith@institution.edu.
+#  Subjects can have multiple identifiers.  Note: it is nice to have if identifiers are unique
+#  even across sources.  Returns one result when searching for one identifier.
+subjectApi.source.ldap.search.searchSubjectByIdentifier.param.filter.value = (&(|(uid=%TERM%)(employeeNumber=%TERM%))(objectclass=person))
+subjectApi.source.ldap.search.searchSubjectByIdentifier.param.scope.value = SUBTREE_SCOPE
+subjectApi.source.ldap.search.searchSubjectByIdentifier.param.base.value = ou=people,dc=internet2,dc=edu
+
+#   search: find subjects by free form search.  Returns multiple results.
+
+subjectApi.source.ldap.search.search.param.filter.value = (&(|(|(uid=%TERM%)(cn=*%TERM%*))(uid=%TERM%*))(objectclass=person))
+subjectApi.source.ldap.search.search.param.scope.value = SUBTREE_SCOPE
+subjectApi.source.ldap.search.search.param.base.value = ou=people,dc=internet2,dc=edu
+
+subjectApi.source.ldap.attributes = givenName, sn, uid, mail, employeeNumber
+subjectApi.source.ldap.internalAttributes = searchAttribute0

test-compose/docker-compose.yml

@@ -8,21 +8,25 @@ services:
      - data
     environment:
      - ENV=dev
-     - USERTOKEN=nothing
+     - GROUPER_CLIENT_WEBSERVICE_PASSWORD_FILE=password
+     - GROUPER_DATABASE_PASSWORD_FILE=/run/secrets/database_password.txt
+     - RABBITMQ_PASSWORD_FILE=/run/secrets/rabbitmq_password.txt
+     - SUBJECT_SOURCE_LDAP_PASSWORD=password
+     - USERTOKEN=build-2
     logging:
       options:
-        tag: "daemon dev"
+        tag: "grouper daemon"
     networks:
      - back
     secrets:
+     - database_password.txt
+     - rabbitmq_password.txt
      - source: grouper.hibernate.properties
        target: grouper_grouper.hibernate.properties
      - source: grouper-loader.properties
        target: grouper_grouper-loader.properties
-     - source: ldap.properties
-       target: grouper_ldap.properties
-     - source: sources.xml
-       target: grouper_sources.xml
+     - source: subject.properties
+       target: grouper_subject.properties
     volumes:
      - type: bind
        source: ./configs-and-secrets/grouper/grouper.properties
@@ -31,33 +35,34 @@ services:
        source: ./configs-and-secrets/grouper/grouper.client.properties
        target: /opt/grouper/conf/grouper.client.properties
 
- 
+
   ui:
     build: ./ui/
     command: bash -c "while ! curl -s data:3306 > /dev/null; do echo waiting for mysql to start; sleep 3; done; while ! curl -s ldap://data:389 > /dev/null; do echo waiting for ldap to start; sleep 3; done; exec ui"
     depends_on:
      - data
     environment:
      - ENV=dev
-     - USERTOKEN=nothing
+     - GROUPER_DATABASE_PASSWORD_FILE=/run/secrets/database_password.txt
+     - SUBJECT_SOURCE_LDAP_PASSWORD=password
+     - USERTOKEN=build-2
     logging:
       options:
-         tag: "ui dev"
+         tag: "grouper ui"
     networks:
      - front
      - back
     ports:
      - "80:80"
      - "443:443"
     secrets:
+     - database_password.txt
      - source: grouper.hibernate.properties
        target: grouper_grouper.hibernate.properties
      - source: grouper-loader.properties
        target: grouper_grouper-loader.properties
-     - source: ldap.properties
-       target: grouper_ldap.properties
-     - source: sources.xml
-       target: grouper_sources.xml
+     - source: subject.properties
+       target: grouper_subject.properties
      - source: sp-key.pem
        target: shib_sp-key.pem
      - source: host-key.pem
@@ -92,24 +97,27 @@ services:
      - data
     environment:
      - ENV=dev
-     - USERTOKEN=nothing
+     - GROUPER_DATABASE_PASSWORD_FILE=/run/secrets/database_password.txt
+     - SUBJECT_SOURCE_LDAP_PASSWORD=password
+     - USERTOKEN=build-2
     logging:
       options:
-        tag: "ws dev"
+        tag: "grouoer ws"
     networks:
      - front
      - back
     ports:
      - "8443:443"
     secrets:
+     - database_password.txt
      - source: grouper.hibernate.properties
        target: grouper_grouper.hibernate.properties
      - source: grouper-loader.properties
        target: grouper_grouper-loader.properties
-     - source: ldap.properties
-       target: grouper_ldap.properties
-     - source: sources.xml
-       target: grouper_sources.xml
+     - source: subject.properties
+       target: grouper_subject.properties
+     - source: sp-key.pem
+       target: shib_sp-key.pem
      - source: host-key.pem
     volumes:
      - type: bind
@@ -129,19 +137,6 @@ services:
 #  scim:
 #    build: ./scim/
 #    command: bash -c "while ! curl -s data:3306 > /dev/null; do echo waiting for mysql to start; sleep 3; done; while ! curl -s ldap://data:389 > /dev/null; do echo waiting for ldap to start; sleep 3; done; exec scim"
-#    volumes:
-#     - type: bind
-#       source: ./configs-and-secrets/grouper/grouper.properties
-#       target: /opt/grouper/conf/grouper.properties
-#     - type: bind
-#       source: ./configs-and-secrets/grouper/grouper.client.properties
-#       target: /opt/grouper/conf/grouper.client.properties
-#     - type: bind
-#       source: ./configs-and-secrets/httpd/host-cert.pem
-#       target: /etc/pki/tls/certs/host-cert.pem
-#     - type: bind
-#       source: ./configs-and-secrets/httpd/host-cert.pem
-#       target: /etc/pki/tls/certs/cachain.pem
 #    depends_on:
 #     - data
 #    networks:
@@ -156,31 +151,46 @@ services:
 #       target: grouper_grouper-loader.properties
 #     - source: ldap.properties
 #       target: grouper_ldap.properties
-#     - source: sources.xml
-#       target: grouper_sources.xml
+#     - source: subject.properties
+#       target: grouper_subject.properties
 #     - source: host-key.pem
+#    volumes:
+#     - type: bind
+#       source: ./configs-and-secrets/grouper/grouper.properties
+#       target: /opt/grouper/conf/grouper.properties
+#     - type: bind
+#       source: ./configs-and-secrets/grouper/grouper.client.properties
+#       target: /opt/grouper/conf/grouper.client.properties
+#     - type: bind
+#       source: ./configs-and-secrets/httpd/host-cert.pem
+#       target: /etc/pki/tls/certs/host-cert.pem
+#     - type: bind
+#       source: ./configs-and-secrets/httpd/host-cert.pem
+#       target: /etc/pki/tls/certs/cachain.pem
+
 
   gsh:
     build: ./gsh/
     depends_on:
      - data
     environment:
      - ENV=dev
-     - USERTOKEN=nothing
+     - GROUPER_DATABASE_PASSWORD_FILE=/run/secrets/database_password.txt
+     - SUBJECT_SOURCE_LDAP_PASSWORD=password
+     - USERTOKEN=build-2
     logging:
       options:
-         tag: "gsh dev"
+         tag: "grouper gsh"
     networks:
      - back
     secrets:
+     - database_password.txt
      - source: grouper.hibernate.properties
        target: grouper_grouper.hibernate.properties
      - source: grouper-loader.properties
        target: grouper_grouper-loader.properties
-     - source: ldap.properties
-       target: grouper_ldap.properties
-     - source: sources.xml
-       target: grouper_sources.xml
+     - source: subject.properties
+       target: grouper_subject.properties
     volumes:
      - type: bind
        source: ./configs-and-secrets/grouper/grouper.properties
@@ -201,6 +211,7 @@ services:
      - "389:389"
      - "3306:3306"
 
+
   idp:
     build: ./idp/
     depends_on: 
@@ -215,6 +226,7 @@ services:
     ports:
      - "4443:4443"
 
+
   rabbitmq:
     image: rabbitmq:management
     environment:
@@ -226,23 +238,26 @@ services:
     ports:
      - "15672:15672"
 
+
 networks:
   front:
     driver: bridge
   back:    
     driver: bridge
 
+
 secrets:
+  database_password.txt:
+    file: ./configs-and-secrets/grouper/database_password.txt
   grouper.hibernate.properties:
     file: ./configs-and-secrets/grouper/grouper.hibernate.properties
   grouper-loader.properties:
     file: ./configs-and-secrets/grouper/grouper-loader.properties
-  sources.xml:
-    file: ./configs-and-secrets/grouper/sources.xml
-  ldap.properties:
-    file: ./configs-and-secrets/grouper/ldap.properties
-  sp-key.pem:
-    file: ./configs-and-secrets/shibboleth/sp-key.pem
   host-key.pem:
     file: ./configs-and-secrets/httpd/host-key.pem
-
+  rabbitmq_password.txt:
+    file: ./configs-and-secrets/grouper/rabbitmq_password.txt
+  subject.properties:
+    file: ./configs-and-secrets/grouper/subject.properties
+  sp-key.pem:
+    file: ./configs-and-secrets/shibboleth/sp-key.pem