Skip to content
Permalink
Newer
Older
100644 374 lines (237 sloc) 20.8 KB
1
[![Build Status](https://jenkins.testbed.tier.internet2.edu/buildStatus/icon?job=docker/grouper/master)](https://jenkins.testbed.tier.internet2.edu/buildStatus/icon?job=docker/grouper/master)
Feb 26, 2019
2
3
May 24, 2018
4
Dec 9, 2019
5
This repository contains the source code used to create the InCommon Trusted Access Platform Grouper container. This standalone container is pushed to Dockerhub, various tags are available at the following URL: https://hub.docker.com/r/tier/grouper/tags. This repo can also be cloned and the container built locally.
Dec 9, 2019
6
7
The test-compose directory contains an example Grouper environment that starts up the various Grouper components. This example demonstrates how one might go about customizing and deploying their Grouper containers, using the TIER Grouper image as a base image. If evaluating Grouper, this is a good place to start.
8
Sep 10, 2018
10
# Upgrading from 2.3 to 2.4
11
Sep 11, 2018
12
If upgrading from Grouper version 2.3 to 2.4 and using LDAP, modifications will be needed in subject.properties and grouper-loaders.proprties. Further details about this can be found at the following URL:
Sep 10, 2018
13
https://spaces.at.internet2.edu/display/Grouper/vt-ldap+to+ldaptive+migration+for+LDAP+access
14
Oct 1, 2018
15
In particular, in subject.properties, *.param.base.value should be adjusted to only contain the RDN (Relative Distinguished Name), not the full DN. For example, "OU=People", not "OU=People,DC=domain,DC=edu"
16
Sep 10, 2018
17
Additional upgrade information can be found at the following URL: https://spaces.at.internet2.edu/display/Grouper/v2.4+Upgrade+Instructions+from+v2.3
18
Dec 9, 2019
19
20
21
# Supported tags
Dec 9, 2019
24
- patch specific tags with date timestamp* (i.e. 2.4.0-80-u51-w10-p11-20191118)
25
26
\* Patch builds are routinely produced, but not necessarily for each patch release. The following monikers are used to construct the tag name:
27
28
- a = api patch number
29
- u = ui patch number
30
- w = ws patch number
31
- p = pspng patch number
Dec 9, 2019
32
- last field = the year, month and day the image was built
33
34
# Quick reference
35
36
- **Where to get help**:
37
[tier-packaging@internet2.edu](mailto:tier-packaging@internet2.edu?subject=Grouper%20Image%20Help)
38
39
- **Where to file issues**:
40
[https://github.internet2.edu/docker/grouper/issues](https://github.internet2.edu/docker/grouper/issues)
41
42
- **Maintained by**:
43
[TIER Packaging Working Group](https://spaces.internet2.edu/display/TPWG)
44
45
- **Supported Docker versions**:
46
[the latest release](https://github.com/docker/docker-ce/releases/latest) (down to 1.6 on a best-effort basis)
47
48
# What is Grouper?
49
50
Grouper is an enterprise access management system designed for the highly distributed management environment and heterogeneous information technology environment common to universities. Operating a central access management system that supports both central and distributed IT reduces risk.
51
52
> [www.internet2.edu/products-services/trust-identity/grouper/](https://www.internet2.edu/products-services/trust-identity/grouper/)
53
54
![logo](https://www.internet2.edu/media/medialibrary/2013/10/15/image_grouper_logowordmark_bw.png)
55
56
# How to use this image
57
58
This image provides support for each of the Grouper components/roles: Grouper Daemon/Loader, Grouper UI, Grouper Web Services, and Grouper SCIM Server.
59
60
## Starting each role
61
62
While TIER recommends/supports using Docker Swarm for orchestrating the Grouper environment, these containers can be run directly (or with other orchestration products). Both examples are shown below. It should be noted that these examples will not run independently, but required additional configuration to be provided before each container will start as expected.
63
64
### Daemon/Loader
65
May 24, 2018
66
Run the Grouper Daemon/Loader as a service. If the daemon/loader container dies unexpectedly, it may be due to memory contraints. Refer to the "Grouper Shell/Loader" section below for information on how to tweak memory settings.
67
68
```console
69
$ docker service create --detach --name grouper-daemon tier/grouper:latest daemon
70
```
71
72
Run the Grouper Daemon/Loader as a standalone container.
73
74
```console
75
$ docker run --detach --name grouper-daemon tier/grouper:latest daemon
76
```
77
78
### SCIM Server
79
80
Runs the Grouper SCIM Server as a service.
81
82
```console
83
$ docker service create --detach --publish 9443:443 --name grouper-ws tier/grouper:latest scim
84
```
85
86
Runs the Grouper Web Services in a standalone container.
87
88
```console
89
$ docker run --detach --publish 9443:443 --name grouper-daemon tier/grouper:latest scim
90
```
91
92
### UI
93
94
Runs the Grouper UI as a service.
95
96
```console
97
$ docker service create --detach --publish 443:443 --name grouper-ui tier/grouper:latest ui
98
```
99
100
Runs the Grouper UI in a standalone container.
101
102
```console
103
$ docker run --detach --name --publish 443:443 grouper-ui tier/grouper:latest ui
104
```
105
106
### Web Services
107
108
Runs the Grouper Web Services as a service.
109
110
```console
111
$ docker service create --detach --publish 8443:443 --name grouper-ws tier/grouper:latest ws
112
```
113
114
Runs the Grouper Web Services in a standalone container.
115
116
```console
117
$ docker run --detach --publish 8443:443 --name grouper-daemon tier/grouper:latest ws
118
```
119
120
### UI and Web Services
121
122
> This method is good when first starting to work with Grouper, but when scaling Grouper UI or Web Services it is advisable to use the individual roles noted above.
123
124
Runs the Grouper UI and Web Services as a combined service. (You should really run these as individual roles to take advantage of Docker service replicas.)
125
126
```console
127
$ docker service create --detach --publish 443:443 --name grouper-web tier/grouper:latest ui-ws
128
```
129
130
Runs the Grouper UI and Web Services in a combined container. This good when first starting to work with Grouper, but when scaling Grouper UI or Web Services it is advisable to use the individual roles noted above.
131
132
```console
133
$ docker run --detach --publish 443:443 --name grouper-web tier/grouper:latest ui-ws
134
```
135
136
### GSH
137
138
Runs the Grouper Shell in a throwaway container. This makes it easy to run Grouper commands and Grouper Shell scripts. Since it is interactive it does not run as a service.
139
140
```console
141
$ docker run -it --rm tier/grouper:latest bin/gsh <optional GSH args>
142
```
143
144
# Configuration
145
146
## Grouper Configurations
147
148
There are several things that are required for this image to successfully start. At a minimum, the `grouper.hibernate.properties` and `subject.properties` (or the old `sources.xml` equivalent) files need to be customized and available to the container at start-up.
149
150
Grouper config files maybe placed into `/opt/grouper/conf` and these files will be put into the appropriate location based on the role the container assumes. Docker Secrets starting with the name `grouper_` should take precedence over these files. (See below.)
151
152
## Web Apps Configuration
153
154
If starting the container to serve the Grouper UI, Grouper Web Services, Grouper SCIM Server components, a TLS key and cert(s) need to be applied to those containers.
155
156
The Grouper UI also requires some basic Shibboleth SP configuration. The `/etc/shibboleth/shibboleth2.xml` file should be modified to set:
157
- an entityId for the SP
158
- load IdP or federation metadata
159
- set the SP's encryption keys
160
- the identity attribute of the subject to be passed to Grouper
161
162
If encryption keys are defined in the `shibboleth2.xml` file, then the key/cert files should be provided as well. The `attribute-map.xml` file has most of the common identity attributes pre-configured, but it (and other Shibboleth SP files) can be overlaid/replaced as necessary.
163
164
(See the section below.)
165
166
## General Configuration Mechanism
167
168
There are three primary ways to provide Grouper and additional configuration files to the container: Docker Config/Secrets, customized images, and bind mounts. Depending upon your needs you may use a combination of two or three of these options.
169
170
### Secrets/Configs
171
172
Docker Config and Docker Secrets are Docker's way of providing configurations files to a container at runtime. The primary difference between the Config and Secrets functionality is that Secrets is designed to protect resources/files that are sensitive.
173
174
For passing full files into the container, this container will make any secrets with secret names prepended with `grouper_` available to the appropriate Grouper component's conf directory (i.e. `<GROUPER_HOME>/conf` or `WEB-INF/classes`). Any secrets with secret names starting with `shib_` will be available in the Shibboleth SP `/etc/shibboleth/` directory. Any secrets with secret names starting with `httpd_` will be available to `/etc/httpd/conf.d` directory. Finally, if a secret with the name of `host-key.pem` will be mapped to the httpd TLS cert used by Grouper UI, Grouper WS, and Grouper SCIM Server containers. These files will supercede any found in the underlying image.
175
176
Docker Secrets can also be used to pass in strings, such as a database connection string password, into the component config. To pass in the Grouper database connection string, one might set the property and value as such:
177
178
```text
179
hibernate.connection.password.elConfig = ${java.lang.System.getenv().get('GROUPER_DATABASE_PASSWORD_FILE') != null ? org.apache.commons.io.FileUtils.readFileToString(new("java.io.File", java.lang.System.getenv().get('GROUPER_DATABASE_PASSWORD_FILE')), "utf-8") : java.lang.System.getenv().get('GROUPER_DATABASE_PASSWORD') }
180
```
181
182
Note that the default property name has been changed by appending `.elConfig`. (This causes Grouper to evaluate the string before saving the value.) The expression allows deployers to use a file containing only the database password as a Docker Secret and reference the file name via the `GROUPER_DATABASE_PASSWORD_FILE` environment property. This allows the config files to be baked into the image, if desired. Also, but not recommended, the database password could just be set in the Docker Service definition as an environment variable, `GROUPER_DATABASE_PASSWORD`. (Technically the expression can be broken up and just the desired functionality used.) Of course, using Grouper's MorphString functionality is supported and likely is the best option, but does require more effort in setting it up.
183
184
Secrets can be managed using the `docker secret` command: `docker secret create grouper_grouper.hibernate.properties ./grouper.hibernate.properties`. This will securely store the file in the swarm. Secrets can then be assigned to the service `docker service create -d --name daemon --secret grouper_grouper.hibernate.properties --secret grouper_sources.xml tier/grouper daemon`.
185
186
> `docker run` does not support secrets; Bind mounts need to be used instead, which is technically what Docker Compose does when not running against a Swarm.
187
188
### Bind Mounts
189
190
Bind mounts can be used to connect files/folders on the Docker host into the container's file system. Unless running in swarm mode, Docker Secrets are not supported, so we can use a bind mount to provide the container with the configuration files.
191
192
```console
193
$ docker run --detach --name daemon \
194
--mount type=bind,src=$(pwd)/grouper.hibernate.properties,dst=/run/secrets/grouper_grouper.hibernate.properties \
195
--mount type=bind,src=$(pwd)/sources.xml,dst=/run/secrets/grouper_sources.xml \
196
tier/grouper daemon
197
```
198
199
### Customized Images
200
201
Deployers will undoubtedly want to add in their files to the container. Things like additional jar files defining Grouper Hooks, or things like images and css files. This can be accomplished by building custom images. **Deployers should NOT use this method to store sensitive configuration files.**
202
203
To add a favicon to the Grouper UI, we use the tier/grouper images as a base and `COPY` our local `favicon.ico` into the image. While we are at it, we define this image as a UI image by specifying the default commnd (i.e `CMD`) of `ui`.
204
205
```Dockerfile
206
FROM tier/grouper:latest
207
208
COPY favicon.ico /opt/grouper/grouper.ui/
209
210
CMD ui
211
```
212
213
To build our image:
214
215
```console
216
$ docker build --tag=org/grouper-ui .
217
```
218
219
This image can now be used locally or pushed to an organization's Docker repository.
220
221
222
## Environment Variables
223
224
Deployers can set runtime variables to both the Grouper Shell and Loader/Daemon and to Tomcat/Tomcat EE using environment variables. These can be set using the `docker run` and `docker service creates`'s `--env` paramater.
225
226
### Grouper Shell/Loader
227
228
The following environment variables are used by the Grouper Shell/Loader:
May 24, 2018
229
- MEM_START: corresponds to the java's `-Xms`. (default is 64m)
230
- MEM_MAX: corresponds to java's `-Xmx`. (default is 750m)
231
232
### Tomcat/TomEE
233
234
Amongst others variables defined in the `catalina.sh`, the following variables would like be useful for deployers:
235
- CATALINA_OPTS: Java runtime options to only be used by Tomcat itself.
236
237
# File System Endpoints
238
239
Here is a list of significant directories and files that deployers should be aware of:
240
241
- `/opt/grouper/conf/`: a common directory to place non-sensitive config files that will be placed into the appropriate location for each Grouper component at container start-up.
242
- `/opt/grouper/lib/`: a common directory to place additional jar files that will be placed into the appropriate location for each Grouper component at container start-up.
243
- `/opt/grouper/grouper.apiBinary/`: location to overlay Grouper GSH or Daemon/Loader files.
244
`/opt/grouper/grouper.scim/`: location for overlaying Grouper SCIM Server web application files (expanded `grouper-ws-scim.war`).
245
- `/opt/grouper/grouper.ui/`: location for overlaying Grouper UI web application files (expanded `grouper.war`).
246
- `/opt/grouper/grouper.ws/`: location for overlaying Grouper Web Services web application files (expanded `grouper-ws.war`).
247
- `/etc/httpd/conf.d/ssl-enabled.conf`: Can be overlaid to change the TLS settings when running Grouper UI or Web Servicse.
248
- `/etc/shibboleth/`: location to overlay the Shibboleth SP configuration files used by the image.
249
- `/opt/tomcat/`: used to run Grouper UI and Grouper WS
250
- `/opt/tomee/`: used to run the Grouper SCIM Server.
251
- `/var/run/secrets`: location where Docker Secrets are mounted into the container. Secrets starting with `grouper_`, `shib_`, and `httpd_` have special meaning. See `Secrets/Configs` above.
Jun 14, 2018
252
- `/usr/lib/jvm/zulu-8/jre/lib/security/cacerts`: location of the Java trust store.
253
254
To examine baseline image files, one might run `docker run --name=temp -it tier/grouper bash` and browse through these file system endpoints. While the container is running one may copy files out of the image/container using something like `docker cp containerId:/opt/grouper/grouper.api/conf/grouper.properties .`, which will copy the `grouper.properties` to the Docker client's present working directory. These files can then be edited and applied via the mechanisms outlined above.
255
256
# Web Application Endpoints
257
258
Here is a list of significant web endpoints that deployers should be aware of:
259
260
- `/grouper/`: location of the Grouper UI application
261
- `grouper-ws/`: location of the Grouper WS application.
262
- `/grouper-ws-scim/`: location of the Grouper SCIM Server application.
263
264
The endpoint that is available is dependent upon the role of the container.
265
266
# Provisioning a Grouper Database
267
268
Using standard methods, create a MariaDb Server and an empty Grouper database. Create a database user with privileges to create and populate schema objects. Set the appropriate database connection properties in `grouper.hibernate.properties`. Be sure to the user created with schema manipulation privileges as the db user.
269
270
Next populate the database by using the following command.
271
272
```console
273
$ docker run -it --rm \
274
--mount type=bind,src=$(pwd)/grouper.hibernate.properties,dst=/run/secrets/grouper_grouper.hibernate.properties \
275
tier/grouper gsh -registry -check -runscript -noprompt
276
```
277
278
Note: a less privileged database user maybe used when running the typical Grouper roles. This user needs SELECT, INSERT, UPDATE, and DELETE privileges on the schema objects.
279
280
# Provisioning a Grouper Database
281
jgasper
Apr 30, 2018
282
Using standard methods, create a MariaDb Server and an empty Grouper database. Create a database user with privileges to create and populate schema objects. Set the appropriate database connection properties in `grouper.hibernate.properties`. Be sure that the user is created with schema manipulation privileges.
283
284
Next populate the database by using the following command.
285
286
```console
287
$ docker container run -it --rm \
288
--mount type=bind,src=$(pwd)/grouper.hibernate.properties,dst=/run/secrets/grouper_grouper.hibernate.properties \
289
tier/grouper gsh -registry -check -runscript -noprompt
290
```
291
jgasper
Apr 30, 2018
292
Also, it is possible to just connect directly to the container, create the DDL, and copy it out. This is necessary if your DBAs would prefer to manually execute the DDL to create the schema objects:
293
294
```console
295
$ docker container run -it --name grouper \
296
--mount type=bind,src=$(pwd)/grouper.hibernate.properties,dst=/run/secrets/grouper_grouper.hibernate.properties \
297
tier/grouper
298
299
gsh -registry -check
300
301
exit
302
303
$ docker container cp grouper:/opt/grouper/grouper.apiBinary/ddlScripts/ .
304
$ docker container rm -f grouper
305
```
306
The generated DDL will be on the host in the `ddlScripts` directory.
307
jgasper
Apr 30, 2018
308
Note: A less privileged database user maybe used when running the typical Grouper roles. This user just needs SELECT, INSERT, UPDATE, and DELETE privileges on the tables and views. Running in this configuration requires DBAs to manually run the DDL scripts.
310
# Configuring the embedded Shibboleth SP
311
312
The Shibboleth SP needs to be configured to integrate with one or more SAML IdPs. Reference the Shibboleth SP documentation for specific instructions, but here is information on generating an encryption key/cert pair and mounting them (all of which are environment specific) and the shibboleth2.xml into the container.
313
314
1. Start a temporary container and generate the key/cert pair:
315
```
316
$ docker container run -it --name grouper \
317
tier/grouper bash
319
cd /etc/shibboleth
320
./keygen.sh -f -h <public_hostname>
321
exit
322
```
323
324
1. Copy the key, cert, and `shibboleth2.xml` files out of the container (and remove the container)
325
```console
326
$ docker container cp grouper:/etc/shibboleth/shibboleth2.xml .
327
$ docker container cp grouper:/etc/shibboleth/sp-cert.pem .
328
$ docker container cp grouper:/etc/shibboleth/sp-key.pem .
329
330
$ docker container rm grouper
331
```
332
333
1. After updating the `shibboleth2.xml` file, save the key, cert, and shibboleth2.xml as secrets/config:
334
```console
335
$ docker secret create sp-key.pem sp-key.pem
336
$ docker config create sp-cert.pem sp-cert.pem
337
$ docker config create shibboleth2.xml shibboleth2.xml
338
```
339
340
1. Add the following to the UI service creation command to mount the environment specific settings:
341
```
342
--secret source=sp-key.pem.pem,target=shib_sp-key.pem \
343
--config source=sp-cert.pem,target=/etc/shibboleth/sp-cert.pem \
344
--config source=shibboleth2.xml,target=/etc/shibboleth/shibboleth2.xml \
345
```
347
# Logging
348
349
This image outputs logs in a manner that is consistent with Docker Logging. Each log entry is prefaced with the submodule name (e.g. shibd, httpd, tomcat, grouper), the logfile name (e.g. access_log, grouper_error.log, catalina.out) and user definable environment name and a user definable token. Content found after the preface will be specific to the application ands its logging configuration.
350
351
> Note: If customizing a particular component's logging, it is recommended that the file be source from the image (`docker container cp`) or from the image's source repository.
352
353
To assign the "environment" string, set the environment variable `ENV` when defining the Docker service. For the "user defined token" string, use the environment variable of `USERTOKEN`.
354
355
An example might look like the following, with the env of "dev" and the usertoken of "build-2"
356
357
```text
358
shibd shibd.log dev build-2 2018-03-27 20:42:22 INFO Shibboleth.Listener : listener service starting
359
grouper-api grouper_event.log dev build-2 2018-03-27 21:10:00,046: [DefaultQuartzScheduler_Worker-1] INFO EventLog.info(156) - - [fdbb0099fe9e46e5be4371eb11250d39,'GrouperSystem','application'] session: start (0ms)
360
tomcat console dev build-2 Grouper starting up: version: 2.3.0, build date: null, env: <no label configured>
361
```
362
363
# Misc Notes
364
365
- [HTTP Strict Transport Security (HSTS)](https://en.wikipedia.org/wiki/HTTP_Strict_Transport_Security) is enabled on the Apache HTTP Server.
366
- morphStrings functionality in Grouper is supported. It is recommended that the various morphString files be associated with the containers as Docker Secrets. Set the configuration file properties to use `/var/run/secrets/secretname`.
367
- Grouper UI has been pre-configured to authenticate users via Shibboleth SP.
368
- By default, Grouper WS (hosted by `/opt/tomcat/`) and the Grouper SCIM Server (hosted by `/opt/tomee/`) use tomcat-users.xml for authentication, but by default no users are enabled. LDAP-backed authentication or other methods can be used and must be configured by the deployer.
369
370
# License
371
372
View [license information](https://www.apache.org/licenses/LICENSE-2.0) for the software contained in this image.
373
374
As with all Docker images, these likely also contain other software which may be under other licenses (such as Bash, etc from the base distribution, along with any direct or indirect dependencies of the primary software being contained).
You can’t perform that action at this time.