diff --git a/Documentation-as-DevOps.adoc b/Documentation-as-DevOps.adoc new file mode 100644 index 0000000..6e8d8a3 --- /dev/null +++ b/Documentation-as-DevOps.adoc @@ -0,0 +1,35 @@ +=== DevOps Approach to Documentation + +==== Proposal + +. Review Evolveum Documentation: https://docs.evolveum.com/, and their writing processes: https://docs.evolveum.com/about/writing-documentation/, https://docs.evolveum.com/about/asciidoc/ +. Use Hugo static site builder with Docsy module: www.docsy.dev/docs/get-started/docsy-as-module/example-site-as-template/ +. Create and edit new and existing files with markdown or asciidoc, push to the site repo +. Include support forPlanntUML, C4, and output from additional diagramming tools with an extension for kroki: https://kroki.io/ +. Trigger republication on commit using Github Actions + +--- + +.Docsy and the Four Types of Documentation +[width="70%",options="header"] +|======= +| 4 types of documentation | Docsy Example Site Documentation Page +| tutorials | Tutorials +| how-to's | Core Tasks +| explanation | Concepts +| reference | Reference +|======= + +--- + +==== Reference Materials +- Evolveum documentation site: https://docs.evolveum.com +- Hugo: https://gohugo.io/ +- Docsy: https://www.docsy.dev/ +- Docsy example site; https://example.docsy.dev/ +- Asciidoc: https://asciidoc.org +- Markdown: https://www.markdownguide.org/ +- Github Flavored Markdown: https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet +- Github Pages: https://pages.github.com (Auto Re-Publication on commit to repo) + + diff --git a/Screenshot 2023-02-23 at 14.55.33.png b/Screenshot 2023-02-23 at 14.55.33.png new file mode 100644 index 0000000..2128404 Binary files /dev/null and b/Screenshot 2023-02-23 at 14.55.33.png differ diff --git a/asciidocTemplate.adoc b/asciidocTemplate.adoc new file mode 100644 index 0000000..4818e8d --- /dev/null +++ b/asciidocTemplate.adoc @@ -0,0 +1,20 @@ += Document Title +Doc Writer + +An example of a basic https://asciidoc.org[AsciiDoc] document. + +== Introduction + +A paragraph followed by an unordered list + +* item 1 +* item 2 +* item 3 + +== Main + +"`Hello, World!`" + +== Conclusion + +That's all, folks! diff --git a/async-message-resources.adoc b/async-message-resources.adoc deleted file mode 100644 index 594705d..0000000 --- a/async-message-resources.adoc +++ /dev/null @@ -1,46 +0,0 @@ -# Async Message Resources - -#### The Protocol -https://www.amqp.org/specification/0-9-1/amqp-org-download - <= AMQP Protocol 0.9.1 + -http://docs.oasis-open.org/amqp/core/v1.0/amqp-core-complete-v1.0.pdf - <= AMQP Core Complete v1.0 (pdf) + - -#### Rabbit AMQP -https://rabbitmq.com/reliability.html - <= + -https://rabbitmq.com/download.html - <= download and install page + -https://github.com/rabbitmq/rabbitmq-server - <= + -https://rabbitmq.com/documentation.html - <= + -https://rabbitmq.com/configure.html - <= + -https://rabbitmq.com/queues.html - <= + - -#### Apache QPID -https://qpid.apache.org/index.html - <= + -https://github.com/amqphub/quarkus-qpid-jms-quickstart - <= + - -#### Intros and Tutorials - -https://ably.com/topic/intro-to-amqp-0-9-1 - <= AMQP 0.9.1 + -https://ably.com/topic/intro-to-amqp-1-0 - <= AMQP 1.0 + -https://jstobigdata.com/rabbitmq/complete-rabbitmq-tutorial-in-java/ - <= + -https://spring.io/projects/spring-amqp - <= + - -#### Challenges: Ordering of Event -https://medium.com/baseds/ordering-distributed-events-29c1dd9d1eff - <= + -https://medium.com/baseds/logical-time-and-lamport-clocks-part-1-d0317e407112 - <= + -https://medium.com/baseds/logical-time-and-lamport-clocks-part-2-272c097dcdda - <= + diff --git a/async-msg-resources.adoc b/async-msg-resources.adoc new file mode 100644 index 0000000..b52a457 --- /dev/null +++ b/async-msg-resources.adoc @@ -0,0 +1,31 @@ +=== async-msg-resources.adoc + +# Async Message Resources + +#### The Protocol +https://www.amqp.org/specification/0-9-1/amqp-org-download <- AMQP Protocol 0.9.1 + +http://docs.oasis-open.org/amqp/core/v1.0/amqp-core-complete-v1.0.pdf <- AMQP Core Complete v1.0 (pdf) + + +#### Rabbit AMQP +https://rabbitmq.com/reliability.html <- + +https://rabbitmq.com/download.html <- download and install page + +https://github.com/rabbitmq/rabbitmq-server <- + +https://rabbitmq.com/documentation.html <- + +https://rabbitmq.com/configure.html <- + +https://rabbitmq.com/queues.html <- + + +#### Apache QPID +https://qpid.apache.org/index.html <- + +https://github.com/amqphub/quarkus-qpid-jms-quickstart <- + + +#### Intros and Tutorials + +https://ably.com/topic/intro-to-amqp-0-9-1 <- AMQP 0.9.1 + +https://ably.com/topic/intro-to-amqp-1-0 <- AMQP 1.0 + +https://jstobigdata.com/rabbitmq/complete-rabbitmq-tutorial-in-java/ <- + +https://spring.io/projects/spring-amqp <- + + +#### Challenges: Ordering of Event +https://medium.com/baseds/ordering-distributed-events-29c1dd9d1eff <- + +https://medium.com/baseds/logical-time-and-lamport-clocks-part-1-d0317e407112 <- + +https://medium.com/baseds/logical-time-and-lamport-clocks-part-2-272c097dcdda <- + diff --git a/connector-howto.adoc b/connector-howto.adoc new file mode 100644 index 0000000..14fd6e8 --- /dev/null +++ b/connector-howto.adoc @@ -0,0 +1,231 @@ +=== connector-howto.adoc + +SEE workbench.adoc + +permalink: https://github.internet2.edu/internet2/iam-knowledge-bits/blob/main/connector-howto.adoc + +next] LDAP/AD provisioning with midPoint + +- - - +_2023-03-22 19:12:40 Connector Howto_ + +*- Reset Workbench to pristine state -* + +``` +ssh csprootuser@khazelton.workbench.incommon.org +cd /csp-tap/InCommonTAP-Examples/Workbench + +docker-compose down -v +docker-compose up -d +``` + +==== Add the Artists' Collective, a new CSV resource, to the Workbench + +*- First, Where is the MIDPOINT_HOME directory? -* + +` In the Workbench host for the containers, MIDPOINT_HOME is: + +` /csp-tap/InCommonTAP-Examples/Workbench/midpoint_server/container_files/mp-home + +` Everything in MIDPOINT_HOME is mapped to + +` /opt/midpoint/var + +` inside the running container + + +*- Artists Collective demo -* + +` User schema extension .xsd files go into MIDPOINT_HOME/schema + +` .csv data file for the Artist Collective resource also goes into MIDPOINT_HOME + + +``` +cd $MIDPOINT_HOME +ls -la schema +ls -la usertype_export.csv +``` + +*- Complete the Artists' Collective resource definition, 100-mnt0.xml -* + +` Edit resource definition file on local computer: + +` ~/opt/kh-wb/objects/resources/100-mnt0.xml + +` fill in the essential parameter definitions section by section: + +` Connector section + +` Schema section + +` Schema Handling section + +` Capabilities section + +` Synchronization section + + +*- The Artist Collective resource config goes into MIDPOINT_HOME/post-initial-objects/resources on the Workbench host -* + +` Use MidPoint Studio to connect to the workbench host and upload the above files to their proper locations + + +*- Once the files are in place, Open the Workbench UI and select Midpoint -* + +` go to Resources / Import Resource Definition, Browse to 100-mnt0.xml on local host + +` click Import + + +` Back to 'All Resources / TMNT System', select Accounts, scroll down and press the Import button + +` Open 'Accounts' on the new resource, Scroll down to the list of users and select one for viewing + +*- Now browse to the Users tab, scroll to 2nd page and note that Cesare Borgia from the Artists' Collective is now recognized as a User in midPoint + +*- This process can be used as a template for adding any other csv-based resource -* + +*- TBD: native install -* + +` replicate artist/hr resource on MBProMax: ~/opt/midpoint-4.6 + +` ~/Users/khazelton/opt/midpoint-4.6 <- MIDPOINT_HOME + +` MIDPOINT_HOME/bin <- All the start/stop/etc scripts + +``` +cd $MIDPOINT_HOME +./bin/start.sh <- To Start midPoint +``` +- - - + +next: replicate artist/hr resource on MBProMax:midpoint-4.6 + +- - - +_2023-03-22 14:36:48 csv connector, resource configuration_ + +/csp-tap/InCommonTAP-Examples/Workbench/midpoint_server/container_files/mp-home +/csp-tap/InCommonTAP-Examples/Workbench/midpoint_server/container_files/mp-home/post-initial-objects/ +resources/100-mnt.xml + +if you make changes in the midpoint files on the host running docker, you must bring the running containers down with the option that clears out the existing mapped files in the container: + +``` +docker-compose down -v +... +docker-compose up -d +``` +*- working tmnt resource def -* + +~/opt/kh.wb/objects/resources/SourceTMNTSystem.xml + +- - - +_2023-03-21 17:42:32 csv connector, resource configuration_ + +*- Host for containerize workbench -* + +``` +Workbench/midpoint_server/container_files/mp-home + +COPY ../Workbench/midpoint_server/container_files/mp-home/ /opt/midpoint/var/ + | | + base in container host base inside connector +``` + +*- in the container -* + +``` +├── container_files +│ ├── csv +│ │ ├── source-hr.csv +│ │ └── UserType_export.csv // <- +│ ├── httpd +│ │ ├── 00-shib.conf +│ │ ├── host-cert.pem +│ │ ├── host-key.pem +│ │ ├── midpoint-shib.conf +│ │ └── vhosts.conf +│ ├── mp-home +│ │ ├── config.xml +│ │ ├── cs-portal.csv +│ │ ├── faculty-portal.csv +│ │ ├── icf-connectors +│ │ │ ├── connector-grouper-rest-0.7.jar +│ │ │ ├── connector-rest-wordpress-.23-SNAPSHOT.jar +│ │ │ ├── connector-sympa-1.0.2-connector.jar +│ │ │ └── net.tirasa.connid.bundles.db.scriptedsql-2.2.-SNAPSHOT.jar +│ │ ├── lib +│ │ │ ├── mariadb-java-client-3.0.jar +│ │ │ └── mysql-connector-java-8.0.30.jar +│ │ ├── mailing-lists.csv +│ │ ├── post-initial-objects +│ │ │ ├── archetypes +│ │ │ │ ├── 300-archetype-affiliation.xml +│ │ │ │ ├── 300-archetype-course.xml +│ │ │ │ ├── 300-archetype-department.xml +│ │ │ │ ├── 300-archetype-external-person.xml +│ │ │ │ ├── 300-archetype-generic-grouper-group.xml +│ │ │ │ ├── 300-archetype-mailing-list.xml +│ │ │ │ ├── 300-archetype-midpoint-group.xml +│ │ │ │ ├── 300-archetype-sis-person.xml +│ │ │ │ ├── 400-archetype-guest-person.xml +│ │ │ │ └── 500-archetype-hr-person.xml +│ │ │ ├── bulkActions +│ │ │ │ ├── 900-test-all-resources.xml +│ │ │ │ └── 910-recompute-grouper-objects.xml +│ │ │ ├── functionLibraries +│ │ │ │ └── 100-function-library-grouper.xml +│ │ │ ├── objectTemplates +│ │ │ │ └── 100-template-user.xml +│ │ │ ├── ordering.txt +│ │ │ ├── orgs +│ │ │ │ ├── 100-org-affiliations.xml +│ │ │ │ ├── 100-org-courses.xml +│ │ │ │ ├── 100-org-departments.xml +│ │ │ │ ├── 100-org-generic-groups.xml +│ │ │ │ ├── 100-org-mailing-lists.xml +│ │ │ │ ├── 100-org-midpoint-groups.xml +│ │ │ │ ├── 400-org-grouper-sysadmin.xml +│ │ │ │ ├── 500-org-hr-sor.xml +│ │ │ │ └── 600-org-guest-sor.xml +│ │ │ ├── resources // <- +│ │ │ │ ├── 100-connector-rest-wordpress.xml +│ │ │ │ ├── 100-grouper.xml +│ │ │ │ ├── 100-guest-db.xml +│ │ │ │ ├── 100-ldap-main.xml +│ │ │ │ ├── 100-source-hr-sor.xml +│ │ │ │ ├── 100-source-sis-persons.xml +│ │ │ │ ├── 100-target-cs-portal.xml +│ │ │ │ ├── 100-target-faculty-portal.xml +│ │ │ │ └── 100-target-mailing-lists.xml +│ │ │ ├── roles +│ │ │ │ ├── 200-metarole-grouper-provided-group.xml +│ │ │ │ ├── 200-metarole-ldap-group.xml +│ │ │ │ ├── 200-metarole-role-wordpress-admin.xml +│ │ │ │ ├── 200-metarole-role-wordpress-editor.xml +│ │ │ │ ├── 200-role-ldap-basic.xml +│ │ │ │ ├── 300-direct-wordpress-role-admin.xml +│ │ │ │ └── 300-direct-wordpress-role-editor.xml +│ │ │ ├── securityPolicy +│ │ │ │ └── 000-security-policy.xml +│ │ │ ├── systemConfigurations +│ │ │ │ └── 010-system-configuration.xml +│ │ │ ├── tasks +│ │ │ │ ├── 300-task-hr-import.xml +│ │ │ │ ├── 300-task-import-guests-sql.xml +│ │ │ │ ├── 300-task-import-sis-persons.xml +│ │ │ │ ├── 500-task-import-guests-sql-livesync.xml +│ │ │ │ ├── 995-task-group-scavenger.xml +│ │ │ │ ├── 997-task-async-update-grouper.xml +│ │ │ │ ├── 998-task-reconciliation-grouper-groups.xml +│ │ │ │ └── 999-task-recomputation-users.xml +│ │ │ └── users +│ │ │ └── 600-user-banderson.xml +│ │ ├── res +│ │ │ └── sis-persons +│ │ │ ├── SchemaScript.groovy +│ │ │ ├── SearchScript.groovy +│ │ │ └── TestScript.groovy +│ │ ├── schema +│ │ │ ├── fanner.xsd +│ │ │ └── internet2.xsd +│ │ ├── source-external.csv +│ │ ├── staff-portal.csv +│ │ └── UserType_export.csv +│ ├── shibboleth +│ │ ├── attribute-map.xml +│ │ ├── idp-metadata.xml +│ │ ├── shibboleth2.xml +│ │ ├── shibd.logger +│ │ ├── sp-encrypt-cert.pem +│ │ ├── sp-encrypt-key.pem +│ │ ├── sp-signing-cert.pem +│ │ └── sp-signing-key.pem +│ └── system +│ └── setservername.sh +└── Dockerfile +``` + +- - - + diff --git a/connectors.adoc b/connectors.adoc index f23022b..c72a399 100644 --- a/connectors.adoc +++ b/connectors.adoc @@ -1,38 +1,113 @@ === connectors.adoc -https://docs.evolveum.com/connectors/connectors/com.evolveum.polygon.connector.csv.CsvConnector/ - <= CSV Connector Documentation + -next] 2020-03-18 10:07 spell out steps toward a how-to for LDAP/AD provisioning with midPoint +permalink: https://github.internet2.edu/internet2/iam-knowledge-bits/blob/main/connector-howto.adoc + +next] LDAP/AD provisioning with midPoint - - - -_2022-05-13 05:50 continue work on csv connector_ +_2023-02-08 11:38:56 database table connector configuration_ -working example: -/Users/khazelton/opt/non.adoc/source-hr.csv +*- Use workbench instance, guest resource, dbTable connector for a full working example -* + +demo in browser: + +` resource, guest db, configuration in UI, in XML + +` import task, operation statistics + +. + +- - - +_2023-02-08 11:37:46 references and links_ + +https://docs.evolveum.com/connectors/connectors/org.identityconnectors.databasetable.DatabaseTableConnector/ + +https://docs.evolveum.com/connectors/resources/databasetable/ + +https://docs.evolveum.com/midpoint/reference/resources/connector-setup/ + + +https://evolveum.com/blog/ + + +- - - +_2022-09-19 13:08 chad redman developing SCIM 2 server_ + +part of the Grouper roadmap for 2.7 is to rewrite the SCIM server. The current implementation uses a 3rd party library written for J2EE, which is why Grouper runs under TomEE and not regular Tomcat. There are a few options for replacement libraries, so this should be a reachable goal. + +If the Grouper SCIM server is rewritten, the endpoints should not change significantly, but the object data is likely to change. The current service expresses objects in ways that differ from the published SCIM RFC's [1][2], and a different solution would adhere more closely to the standards. An example of some ways SCIM in Grouper is non-standard and would change: + +- extensions are wrapped in an "extensions" node (includes group name or subject id, so essential fields) + +- userName is not present in user objects and is required + +- unknown attribute baseUrn + +- inconsistent use of group and subject ids vs. uuids + +- /Schemas endpoint is broken (infinite loop that eventually aborts) + +- no PATCH or BulkRequest support + +Changes would impact integrations already in production, so the Grouper team is looking to hear from current users of the SCIM server. + +Starting a conversation with the current users, as well as users holding back because of current limitations, would also be a good opportunity to make improvements to the system. BulkRequest isn't supported, so large change sets are inefficient. PATCH operations are not currently supported, which means memberships can't be managed through the group object. Instead, multiple calls potentially need to be made to look up uuids for the group, subject, and membership. That illustrates how cumbersome it is to work with uuids for groups and subjects in general, and maybe there is some opportunity to switch to more friendly subject ids and group names as resource keys. + +So, if you are using the SCIM service in Grouper, or want to use a more standard version, please comment or let the Grouper team know, so that the needs can be better known. + +- - - +_2022-09-14 17:37 schema mapping, csv connector template_ + +- - - +_2022-05-19 09:43 utility for prompted user input in CLI scripts (for use in soliciting configuration items and choices)_ + +https://github.com/SBoudrias/Inquirer.js + <- + +https://github.com/mokkabonna/inquirer-autocomplete-prompt + <- + + +- - - +_2022-05-15 17:10 continue work on csv connector_ TBD: SoR person to mP user schema mapping utility Next resource definition: develop, test, document SIS resource creation using the 100-student csv sample from BennO's mock data sets /Users/khazelton/opt/non.adoc/sis.csv +sorid +GivenName +MiddleInitial +Surname +Birthday +EmailAddress +TelephoneCountryCode +TelephoneNumber +NationalID +Occupation +Company + + + +- - - +_2022-05-13 05:50 continue work on csv connector_ + +working example: +/Users/khazelton/opt/non.adoc/source-hr.csv + - - - https://github.com/Evolveum/midpoint-samples/blob/master/samples/evolveum/object-template-user.xml - <= user template + + <- user template + A user template may be applied globally by including the following snippet in xref:/midpoint/reference/concepts/system-configuration-object/just after the "logging" element: +``` +``` System configuration xml; after logging element: - ``` UserType - + ``` That template ref, oid="8098b124-c20c-4965-8adf-e528abedf7a4", points to ../objects/objectTemplates/UserTemplate.xml which assigns the unique name and uid + ``` uid,firstname,lastname,department,mail,validFrom,validTo E600001,John R,Smith,HR_SOR,xjsmith@example.com,2018-01-01,9999-12-31 @@ -40,7 +115,6 @@ E600002,Alice,Anderson,HR_SOR,xaanderson@example.com,2016-03-15,9999-12-31 E600003,Ellen,Johnson,HR_SOR,xejohnson@example.com,2019-10-01,2019-12-31 E600004,Ron,Vasquez,HR_SOR,xrvasquez@example.com,2019-01-01,2019-10-31 ``` - csv resource def template: ../non.adoc/extCsvResourceA.xml Resource on Aktis: 'HR SOR Source' @@ -60,34 +134,35 @@ schema extension xsd's go in $MIDPOINT_HOME/schema ~/opt/InCommonTAP-Examples-current/Workbench/midpoint_server/container_files/mp-home$ +``` tree . -L 2 . ├── config.xml ├── cs-portal.csv ├── faculty-portal.csv ├── icf-connectors -│   ├── connector-grouper-rest-0.7.jar -│   ├── connector-rest-wordpress-.23-SNAPSHOT.jar -│   ├── connector-sympa-1.0.2-connector.jar -│   └── net.tirasa.connid.bundles.db.scriptedsql-2.2.6-SNAPSHOT.jar +│ ├── connector-grouper-rest-0.7.jar +│ ├── connector-rest-wordpress-.23-SNAPSHOT.jar +│ ├── connector-sympa-1.0.2-connector.jar +│ └── net.tirasa.connid.bundles.db.scriptedsql-2.2.6-SNAPSHOT.jar ├── mailing-lists.csv ├── post-initial-objects -│   ├── archetypes -│   ├── bulkActions -│   ├── functionLibraries -│   ├── objectTemplates -│   ├── ordering.txt -│   ├── orgs -│   ├── resources -│   ├── roles -│   ├── securityPolicy -│   ├── systemConfigurations -│   ├── tasks -│   └── users +│ ├── archetypes +│ ├── bulkActions +│ ├── functionLibraries +│ ├── objectTemplates +│ ├── ordering.txt +│ ├── orgs +│ ├── resources +│ ├── roles +│ ├── securityPolicy +│ ├── systemConfigurations +│ ├── tasks +│ └── users ├── res -│   └── sis-persons +│ └── sis-persons ├── schema -│   └── internet2.xsd <= example schema extension file ──────────────────────────────── +│ └── internet2.xsd <- example schema extension file ──────────────────────────────── ├── source-external.csv └── staff-portal.csv ``` @@ -117,7 +192,7 @@ _2022-04-27 18:44 CSV connector how-to_ https://www.evolveum.com/downloads/midpoint/4.1/midpoint-4.1-schemadoc/http---midpoint-evolveum-com-xml-ns-public-common-common-3/object/UserType.html -userAttr.ods <= midPoint User Attribute Groups + +userAttr.ods <- midPoint User Attribute Priority Categorization + ==== CSV Connector Work Plan @@ -141,27 +216,26 @@ Then click “UserType” Starting from a template xml file (TBD), map the information from the schema document into the matching XML elements in the sections on , , , , and (See sisSorResourceDef.xml) -Next step is to debug Resource Tasks -Import (and Reconcile) +Next step is to debug Resource Tasks + +Import (and Reconcile) + Recompute -TBD -Build and test Synchronization Task +TBD + +Build and test Synchronization Task + Live sync: Work with Ethan - - - _2022-05-13 05:48 references and links_ https://github.com/Evolveum/midpoint-samples - <= + + <- + https://github.com/Evolveum/midpoint-samples/tree/master/samples/contrib/bshp - <= Jason Everling, Bishop examples + + <- Jason Everling, Bishop examples + - - - _2022-04-05 13:59 csv connector how-to_ [source,xml] ----- + + diff --git a/its.adoc b/its.adoc new file mode 100644 index 0000000..7ffb3ca --- /dev/null +++ b/its.adoc @@ -0,0 +1,35 @@ +Static site generators as documentation substrate + +. MkDocs +. Jekyll +. GitHub Pages +. GitLab Pages +. Docusaurus +. Antora +. Hugo + + +=== Documentation-as-DevOps + +==== One Option + +. Review Evolveum Documentation: https://docs.evolveum.com/, and their writing processes: https://docs.evolveum.com/about/writing-documentation/, https://docs.evolveum.com/about/asciidoc/ +. Migrate existing content from Confluence XHTML to asciidoc: https://docs.asciidoctor.org/asciidoctor/latest/migrate/confluence-xhtml/ +. Create and edit new and existing files with asciidoc, push to a github repo +. Include PlanntUML, C4, and output from additional diagramming tools with the asciidoctor extension for kroki: https://github.com/ggrossetie/asciidoctor-kroki, https://kroki.io/ +. Create an Antora site to host the asciidoc files +. Anrtora Assembler: add-on library and extension that assembles content from multiple pages into aggregate documents, converts them to the specified output format, and publishes them alongside other files in the site https://gitlab.com/antora/antora-assembler +. Follow Antora guidance to create a pipeline to rebuild the site whenever content is added, edited, or deleted: https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&ved=2ahUKEwiB5IfittT-AhWVk4kEHR9zBwQQFnoECBcQAQ&url=https%3A%2F%2Fdocs.antora.org%2Fantora%2Flatest%2Fpublish-to-gitlab-pages%2F&usg=AOvVaw3vOCafAF8DhwsRqTQbeFcm + +==== DevOps approach to Documentation: Reference materials +- Evolveum documentation site: https://docs.evolveum.com +- Asciidoc: https://asciidoc.org +- Markdown: https://www.markdownguide.org/ +- Antora: https://antora.org/ "multi-repository documentation site generator for tech writers writing in AsciiDoc" +- Github Pages or Gitlab Pages (Auto Re-Publication on commit to repo) +- https://pages.github.com, https://docs.gitlab.com/ee/user/project/pages/ +- Gitlab Demo ‘source’: https://gitlab.com/antora/demo +- Gitlab Demo published version: https://antora.gitlab.io/demo/docs-site/component-b/2.0/index.html +- Pros and Cons of Markdown vs Asciidoc: https://www.slant.co/versus/1903/23065/~markdown_vs_asciidoc +- Jeff Sandberg, "I wish asciidoc was more popular": https://pdx.su/blog/2023-02-05-asciidoc-and-markdown/ + diff --git a/markdown-asciidoc.adoc b/markdown-asciidoc.adoc new file mode 100644 index 0000000..88c112d --- /dev/null +++ b/markdown-asciidoc.adoc @@ -0,0 +1,39 @@ +==== IV. Comparison of text entry, editing, preview, publication, and CI/CD tools: Markdown-based vs Asciidoc-based + +[cols=3*,stripes=odd] +|=== +^| *FEATURE/ASPECT* ^|*MARKDOWN* ^| *ASCIIDOC* + +|Basic text entry, editing, and preview +|https://code.visualstudio.com/docs/languages/markdown[VS Code support for Markdown] +|https://marketplace.visualstudio.com/items?itemName=asciidoctor.asciidoctor-vscode[Asciidoc support for VS Code] + +| Distinguishing features +| Broad adoption, greater volume of published content, simple syntax for basic features, enhanced features available in various markdown flavors + +| Complex tables, asides, definition lists, includes, sidebars, table of contents, live snippets from source code, anchors, in-file metadata, attribute/variable support, ifdef/ifndef for conditional inclusion + +| Flavors and Standards +| Markdown (MD) plus varieties: Github Flavored Markdown (GFM), GitLab Flavored Markdown (GLFM, based on https://commonmark.org/[CommonMark]) +| Asciidoc specification is governed by the https://asciidoc-wg.eclipse.org/[Asciidoc Working Group] under Eclipse + +|Publication process +|Markdown -> Jekyll -> GitHub Pages + +Markdown -> Jekyll -> GitLab Pages + +Output: HTML, pdf, more via Pandoc +|Asciidoc -> https://antora.org/[Antora] -> GitLab Pages + +Asciidoc -> Antora -> GitHub Pages + +Output: HTML, pdf, Docbook, man pages, EPUB3, Reveal.js via Asciidoctor CLI + +Multi-repository documentation sites via Antora + +|CI/CD support +|Jekyll to GitHub Pages via Github Actions; To GitLab pages via https://docs.gitlab.com/ee/user/markdown.html[GitLab Flavored Markdown] (based on Kramdown) + +|Antora to GitLab Pages via .gitlab-ci.yml playbook; to GitHub pages via Github Actions with .nojekyll directive + +|Conversion between markdown and asciidoc +|https://github.com/asciidoctor/kramdown-asciidoc[markdown to asciidoc] +|https://github.com/opendevise/downdoc[asciidoc to markdown] + +| +|HTML, pdf, Docbook, man pages, EPUB3, Reveal.js via Asciidoctor CLI + +|=== diff --git a/midpoint-db-table-res-def.adoc b/midpoint-db-table-res-def.adoc new file mode 100644 index 0000000..88eb652 --- /dev/null +++ b/midpoint-db-table-res-def.adoc @@ -0,0 +1,94 @@ += midPoint Database Table Resource Definition How-to + +=== Walkthrough midPont UI Screenshots during DB table Resource Defintion + + +.First page of resource definition + +image:/images/Screenshot 2023-02-23 at 14.03.59.png[ +"First page of resource definition",width=128, +link="/images/Screenshot 2023-02-23 at 14.03.59.png"] + +.Basic tab + +image:/images/Screenshot 2023-02-23 at 14.05.13.png[ +"First page of resource definition",width=128, +link="/images/Screenshot 2023-02-23 at 14.05.13.png"] + +.Connector configuration (top half of page) + +image:/images/Screenshot 2023-02-23 at 14.05.48.png[ +"First page of resource definition",width=128, +link="/images/Screenshot 2023-02-23 at 14.05.48.png"] + +.Connector configuration (bottom half of page) + +image:/images/Screenshot 2023-02-23 at 14.06.48.png[ +"First page of resource definition",width=128, +link="/images/Screenshot 2023-02-23 at 14.06.48.png"] + +.Accounts tab + +image:/images/Screenshot 2023-02-23 at 14.07.12.png[ +"First page of resource definition",width=128, +link="/images/Screenshot 2023-02-23 at 14.07.12.png"] + +XML version of resource definition + +image:/images/Screenshot 2023-02-23 at 14.55.33.png[ +"First page of resource definition",width=128, +link="/images/Screenshot 2023-02-23 at 14.55.33.png"][ + + +Basiic tab + +image:/images/Screenshot 2023-02-23 at 14.05.13.png[ +"First page of resource definition",width=128, +link="/images/Screenshot 2023-02-23 at 14.05.13.png"] + +Basiic tab + +image:/images/Screenshot 2023-02-23 at 14.05.13.png[ +"First page of resource definition",width=128, +link="/images/Screenshot 2023-02-23 at 14.05.13.png"] + +Basiic tab + +image:/images/Screenshot 2023-02-23 at 14.05.13.png[ +"First page of resource definition",width=128, +link="/images/Screenshot 2023-02-23 at 14.05.13.png"] + +Basiic tab + +image:/images/Screenshot 2023-02-23 at 14.05.13.png[ +"First page of resource definition",width=128, +link="/images/Screenshot 2023-02-23 at 14.05.13.png"] + +Basiic tab + +image:/images/Screenshot 2023-02-23 at 14.05.13.png[ +"First page of resource definition",width=128, +link="/images/Screenshot 2023-02-23 at 14.05.13.png"] + +Basiic tab + +image:/images/Screenshot 2023-02-23 at 14.05.13.png[ +"First page of resource definition",width=128, +link="/images/Screenshot 2023-02-23 at 14.05.13.png"] + +Basiic tab + +image:/images/Screenshot 2023-02-23 at 14.05.13.png[ +"First page of resource definition",width=128, +link="/images/Screenshot 2023-02-23 at 14.05.13.png"] + +Basiic tab + +image:/images/Screenshot 2023-02-23 at 14.05.13.png[ +"First page of resource definition",width=128, +link="/images/Screenshot 2023-02-23 at 14.05.13.png"] + + + + + diff --git a/perydAffilFwork.adoc b/perydAffilFwork.adoc new file mode 100644 index 0000000..4e53c8d --- /dev/null +++ b/perydAffilFwork.adoc @@ -0,0 +1,120 @@ +=== PeopleSoft Affiliation Framework at SMU +Pery Doan, 2 December 2022 + +I'm glad to know that we have some PeopleSoft developers on the call. I wasn't sure how this meeting was going to go, so I kind of prepared for doing a little bit more of a deep dive on how the affiliation process actually works, even though at SMU We've um gone with um, using some of the delivered functionality, but then also have some custom implementation that goes with it. + +So I will start with just showing you guys, some diagrams of our environment. Do a little bit of a quick demo just to show you some screens, so that hopefully some of the more boring slides that come after that which are a lot to describe how the Affiliation Framework is intended to work in PeopleSoft, and how you would configure it. Um would make a little bit more sense to you guys, as we're going through it. Um also my main goal in kind of throwing this slides a bunch of slides in here so that I can then share it with you guys afterwards, and you would have a roadmap. Um, if you wanted to try doing a like a proof of concept at your school, you would know where to start. It's not going to have everything in it, obviously, but I give you some references too at the end to some of the technical documents you can.find out there. + +So just for you PeopleSoft folks. This our environment at SMU. We have CS, HCM, And Financials, + +image::images/mySmu.png['PeopleSoft Packages', width=700] + +We try to keep current with updates and fixes on a quarterly basis, and we are an Oracle shop on release 19.16 and we use Campus Solutions as our portal, and we leverage unified navigation to pull our HCM and FS content into our CS home pages, and we do heavily use Gideon Taylor: Well, I say heavily, we now have it across all of our three environments, and we are starting to become more and more dependent on that framework. + +I just found this one diagram that I put together a while back, +just to show how we have different sources around campus that enter person data. + +image::images/identityFlow.png['Person Data Sources', width=700] + +So that's the departments in blue. We have different applications as well, like Slate and Blackbaud that create identity data in our PeopleSoft system. So that's this column here, where we have this person model and we are using the delivered service operations to sync our split databases which we split back in 2014, using the subscriber only integration model, and we have person basic sync and diversity sync, and all of these nice, fun delivered service operations enabled to do bidirectional syncing of our person data. And then we move on to the transaction data where we have employee.smu synchronizing job data. It's just the one-way integration for the workforce sync and position data. We also have user profiles getting synced, and that's not represented in this diagram. + +Our CS instance is our authoritative data source. So it it sends everything to uh employee.Smu. And also our setup data is authoritative in an employee.smu, and syncs over to our Campus Solutions instance. From there also you can see that we do have some external affiliation sources that we manage. So in these two boxes we have our employees and students, and we do have some sponsored identities that we select here based on instructor data. So when we don't have the authoritative job data to tell us this is an adjunct. We use the schedule of classes and the instructor/advisor table to figure out who's teaching, and then we go ahead and give them a sponsorship that we call instructor-sponsored access. + +we have other sponsorships represented that go into the AMA, which is our current custom, identity management bolt-on that we created in PeopleSoft and Campus Solutions which i'll talk more about. But then the end result is that we have our service provisioning, and even just physical access directed by the logic that is running here in this engine. + +So for our current state. we have PeopleSoft as our person registry. We have this custom uh Account Maintenance Automation, (AMA) bolt-on that determines the person, affiliation and service eligibility. And then we have Tommy's power shell scripts that actually performs the service provisioning. + +What we did with our Affiliation Framework project is replace a piece of what AMA does assigning what we call user types, and you will hear me interchangeably use the term user type and affiliation throughout. + +In our in our future state as we are implementing COmanage and Grouper and midpoint. We're still in process of doing all of these things. One of the very first things and foundational things that we needed to do was figure out, Well, how are we going to determine a person's affiliation? And we landed on trying to leverage the Affiliation Framework so that we can keep that logic and leverage all the work that we did with AMA to transition into Affiliation Framework, to assign and remove our user types, and that makes it a little bit easier. Now, with our integration with COmanage where we are not only sending the bio demo data, but we're also attaching a bit of the affiliation data with those identities. + +We realize that that relationship data was sort of critical to have in our new enterprise person registry. So that when we have a co-worker whose name is Eric Macy. And there was a student who had that same name. And so it's just really a lot. it's very important in Higher Ed to be able to distinguish a student from a staff, for example, at the person registry level. So we are sending a few sort of affiliation related attributes to COmanage, and then later, Grouper is going to leverage COmanage for determining service eligibility, assign and remove services, and also reach back into PeopleSoft itself, so it'll go back to employee in My SMU and grab whatever information it requires for additional provisioning that perhaps wouldn't be in the COmanage registry. Moving to the Affiliation Framework, we took that opportunity to sort of revisit, and broke out our benefit eligible employees, into more distinct affiliations. We also broke out the ones that we select even before their effective date. So if somebody gets hired ahead of time right now, we will go ahead and assign these user types, and we can distinguish them from people who are current versus future dated hires. So that kind of is a neat feature that I think we're gonna really like. We've been able to identify the non-degree students, which is nice. And we broke out the active students into undergraduate and graduate to give us a little more granularity. At this point I wanted to show you just quickly a couple of the pages. + +So our Account Maintenance Automation I have, an ID that I want to show yobuild time. We're still using this as we're implementing our new IAM components allows us to. As you can see, it selects the type of person this is, and then assigned services, and we've gotten very accustomed to being able to see where is that coming from. So I built into this bolt-on the ability for people to see why. This person is an adjunct faculty and a temporary staff. You can actually see that here, and I mean, I see here that they were terminated as a temp staff on twelve one. But this is a test environment, and I have not run it recently. So they still show as um active staff. But if I ran ANA for this person, they would lose that affiliation, and only retain their active temporary faculty position. I can also see that this person is in the instructor Advisor Table. and they are an instructor type of adjunct, and that they are attached to future classes. So if this person were a student, I would see, you know, student information here under these grids that the AMA is using itself as the source data to make these decisions. So that has been very helpful, because when there are any questions about, Why did this person not get picked up? Why did this person you know this or that? I'm? I'm not bombarded with a million people asking me questions. They can just go here and see what that may be. + +*- at the 21 minute mark -* + +So the reason I wanted to show you that is because in the Affiliation Framework I'm. Going to show you this person and i'm going to show you the delivered page. So this is what the delivered page would look like. It still shows me right that they're an adjunct that they're an instructor, temp staff, and a conversion former student, which is something that am, I does not pay attention to. So that's why this is kind of a new thing here. I can see that the affiliation is active, and I can click into the context fields, which is something. Um, that we're leveraging with the delivered framework to be able to say, Okay, what are some? You can define these, you know. You can tell it whatever record name you want and field name. We kind of piggybacked on that, and just are saying COmanage, because that's what we're using it for. For when we create our Json uh message over to COmanage, we are grabbing information from this table. And we're saying that this person for their adjunct faculty, affiliation has, you know, happens to be in the meadow School of the arts, and the department is news. The Cat Junks title is not going to be populated. Uh, we're not allowed to do that at the time, and then we have their uh primary affiliation, which is faculty and their manager. + +Those are the attributes that we're sending over to COmanage.This is all very helpful. But what I ended up doing is creating a custom page, which is this affiliation snapshot, which is very much modeled after what I did with AMA, so that at the top I can see. Okay, here's the active user types. Here might be any inactive ones, and these are really affiliations. We're just using the same terminology that users are used to seeing which is user type, and also because, you know, when you're talking about this affiliation, which is more sort of like a high level affiliation. There's the affiliation codes, and then there's this affiliation and primary relation it can. There's a lot of affiliation, and it can get confusing. So we've decided to try to call these affiliation codes user types, and then um, we have the revelation, and I can see right away. I don't have to click and drill in to see what those attributes are for that particular code. And I also have our data source details um included here, so that that is, uh one of the benefits of us having done it. Um, you know, with a bit of a custom component to this process, and then I can also access affiliation, history. Of course I haven't. Nothing happened to this person, but you would be able to see over time things that are activated or um inactivated, as as time goes on, and I can also see details about our COmanage Api, whether it has gone uh successfully. Or it's waiting to be processed, or you know, whatever it may be. Also as as part of our COmanage um integration. We have a put that sends information from affiliation, framework, and our person model over to come, manage, and then we receive this post which um updates the external system, identifiers and PeopleSoft, and we haven't run it in this test environment. But we would see here what those external IDs would be, so that we can understand and and see within PeopleSoft what that enterprise ID is, and what's the other one, Tommy? So just a bit of background on that When PeopleSoft sends. The put message to Co. Manage will respond back with identifiers that COmanage generates and so we're generate currently we're generating two. One is a net ID or a network ID and the other is the enterprise ID uh so a lot we could say about that. But that's uh that's the idea. So the other thing I wanted to show you here, while while I'm here is um. + +One other neat feature of the Affiliation Framework delivered application is that in any component where there are bio demo data type information about, you know, throughout PeopleSoft. There is an integrated icon which is this little house which is the appellation display. So, no matter where, like I, I was looking at um like, I think, academic program information on on a student, and it had this a little affiliation box. So you can actually see what other affiliations that person might have right right there at your fingertips. That's pretty cool. + +So now like to jump into the Affiliation Framework. You know there's some pretty basic things. We know that it tracks them a relationship with the institutions um over time, and it was delivered as part of the nineo feature pack one. This was back in two thousand and nine. And uh, it is a more robust method of tracking affiliations than the relationship with institution component within campus solutions. I don't know if you guys are familiar with it. We used to use that as well, and I thought it was going to replace it. But as I was looking at people's books recently found out that it's not. It didn't replace it. It's just more of a robust method of of tracking affiliations. This framework will allow you to update those manually and batch or triggered by affiliation events. +And then we already talked about the little icon. So it is part of the constituent web services and campus solutions. Not sure if you guys are familiar with it, but it is designed. The Cws is designed to allow and facilitate integration of people, self systems with external systems, and it also is heavily used with interaction. Hub and the external search match functionality, and of course, Oracle identity manager delivered a connector into CS. Which increase integrates with affiliation, framework, and um constituent web services. I've heard not very good things about that, but that is what it is. We also know that Um, the constituent transaction management framework, which is what we use for loading our slate applications into PeopleSoft. There's the data update rules. The affiliation override dependencies for that interface. So that's another reason why we wanted to. Um, you know, use the Affiliation Framework, and when we went live with a slate. + +I don't remember when it was, but it was a while back we tried to implement Affiliation Framework and had a lot of hurdles. So we ended up just so that we could meet this need delivering. I just created like a SQL script that the Dba's had on a cron job that would run every night to identify current employees, former employees, current students, and former students, so that when new apps came into the system it would check and see, because they did not want +those applications to update any bio demo data for existing current employees, for example, or existing current students and things like that. So that's where that comes in currently. We've gone live with Affiliation Framework like two weeks ago when it went Live any discrepancies, you know, do doing, reporting. We can um use it to continue our COmanage integration efforts and things like that. But I thought it would be interesting to show you guys, some of the counts that we have with our current um affiliations. And I, broke out the instructor, one, because that one is not dependent on job data, obviously right. So So the ones that have, for example, adjunct faculty. There's three hundred and thirty-eight. But as you can see here, Um, we have e instructor assigned to six hundred and sixty-eight that are set up as instructor type adjunct in the instructor visor table, and that is because we do have an issue with getting those higher forms in a timely fashion and things like that. So the instructor. Affiliation code is intended to be sort of a high level. We don't want to say that you are an adjunct, because you can be a teaching assistant, or a Gsi, or graduate student structure, full time, faculty, or whatever. And currently, unfortunately, in our PeopleSoft system. These instructor types are not maintained, So there's not a high degree of confidence, and using those as our as our source to say, Yes, you are now assigned right. We know of instances where, for example, we have full time faculty that come in, so they're in this table as full time faculty. They retire. They come back as adjuncts, and nobody updates their role to adjunct again. So we're moving in other projects to try to improve that data quality, and we have made a lot of progress in trying to sync it up with what job data says and things like that. But we're still not at one hundred percent confidence in those designations. But we're moving in the right direction, I feel, and I think and hope that by the time we're ready to go live with our new Im solution that things will be a lot more timely and accurate in our in our data. + +So how does the delivered Affiliation Framework process works. A lot of what you're gonna see are things that I took from the Campus Solutions Affiliation Developer Guide. I just copied this high level diagram of how it works where you have constituent information that's changing the system. For example, Admissions matriculates a student or HR terminates an employee, that's a change; or some self service utility that a student uses to change their address. + +Then you have Integration Broker messages that are published. And the way that Oracle developed. This process was really thinking, and and it really was developed forures I mean the batch concept came later on, and kind of was kind of forced into it, but it was intended, so that you know any time you have a change in the system, it would trigger a message which would then trigger the Affiliation Framework process to process that message and then assign or remove affiliations according to that data. + +If you want to talk to an external system like Om. Or something and the uh Affiliation Developer guide does not really talk in great depth about the constituent web services. They refer you to the constituent Web Services developer, guide for further information. If you're interested in and knowing more about that I to send you. Of course we're not doing that. We created a custom web service that leverages Affiliation Framework. Data.And since JSON message over to COmanage um that I that I shared earlier our put message. + +now i'm gonna talk through these steps kind of like in a little bit more detail. So the constituent information is changing in the system via different ways. Right? So the the relevance of of outlining. This is because you you've got to think about. If you're If you want to go with trigger based transactions, then you pretty much have to um update everywhere in the system where you're interested in having that trigger sent, you have to modify those components if they are not already sending some sort of message right? So like we wouldn't need to necessarily modify the job component, because it's already triggering work for sync into campus solutions so that could be leveraged as something that would then trigger the Affiliation Framework. But if I wanted to change somebody's academic program or um, you know, admissions matriculate somebody, or whatever it may be, those are things that don't trigger anything today, and camp solutions. So we would have to modify those components to have some sort of message triggerthe uh, And so that that's what these two like, whether still service or administrator makes online update. The other thing i s like batch updates. So if we have a program, for example, like I'm: sure pretty sure admissions, You know, matriculates people in bachelor. I'm gonna go one by one. So those batch program updates umhave to be. You have to use this specific pattern to trigger the appropriate messages so that those triggers happen. + +So. Integration, That's the second step. The Integration broker message is published, and so that's the code that's needed, and i'm sorry if i'm not being very good at this I put this presentation together yesterday, and I I haven't practiced it at all. So please forgive me upon all over the place. But this um just explain basically what I just said about the component triggers. You can also like, if you have a custom app engine program that makes updates or even delivered one. You can actually have a call, you know, and and trigger the messages, and then for the cobalt and sqrs. You would use this um delivered table, and that you'll publish programs to the message into the table If the Affiliation Framework processes the message. And this is where + +This is cool. I I like the fact that in affiliation firmware. There's really only three tables that are updated. You have your person affiliation. And then the context data that I showed you guys like there's one that has the data fields because it's um configurable. You can tell it what record and what field to use. Then it it has to have a way to track that, And then there's the value table. So it's. It's pretty simplistic. The thing that's not very simplistic is how they design this thing to work. And this is a diagram that um we. We hired a consultant to help us with this. He's very, very smart and very good with um at packages, so he really understood, after +doing our proof of concept how this thing works, and I have the example of a custom trigger event which is really what is described in the Developer guide is This is what you will see on how to, you know, set up a proof of concept and what it does, is it? Will you know you have to? I'll show you the steps, but you you create your own service operation, and then it that's what triggers through the handler, the Affiliation Framework. There's a custom adapter that's called, and then it returns to the Affiliation Framework. Then it does the implementation classthen it returns the information framework, and then it finally gets an answer whether it's affiliated or not. And this is done for one ID at a time right, because the way that this was designed was relief to work for triggers. So you're only processing one message at a time. Then um for the delivered trigger example. We have that workforce sync where you have an existing service operation that you can modify. You can go in there and add the handler to it,and then it will, of course, as soon as it triggers it will go ahead and trigger your Affiliation Framework. + +The batch process is run through Process scheduler, of course, but it um. It calls that message handler directly, so it does not generate Ib. Messages so you wouldn't see those an integration broker or anything like that, however, because it is working with just the way that this is designed, it's going to process one ID at a time instead of doing set-based processing. It's an app engine that is very inefficient, and it leverages the pop select query um forget what that framework is called, but it does leverage that, and It proved very inefficient for us. So that's one of the reasons why we are not using this delivered process. I know that there are two schools out there at least. Um. + +A long time ago I I found two presentations, one from two thousand and fifteen, one from two thousand and eighteen where um you know those schools are running it, and they're running the deliver process. So I think that it can be done. It just was not something that we did we deemed feasible for our institution. All right. So then, um step four is, you know, the affiliations getting assigned or removed. And I just have this to show you an example of a person with affiliations. And then here's the person table. You can see here that the start date is a key field. So you do have to provide that. And the way that the delivery program works, is it just like I said, it returns a true or false the start date. It's kind of determined by what that message had in in the effective date, or you can go around it by doing your own SQL to. If you wanted to try to figure that out there are in the Developer Handbook. + +Some examples and and some definite rules that you need to be aware of in terms of how it deals with the start, date, and end dates and those kinds of rules, which was another reason why, for smu it j ustwasn't going to handle some of our scenarios very well For example, the typical one is our Temp employee population, we have a higher form. It's a Gideon Taylor form THAT is submitted by departments, and really follows a business practice that peril has had over the years, which is whenever they get a they used to get the paper epaph, or now the electronic one. They know the start and end date of a temporary job. So they go ahead and create two job rows, the start and the end the termination. And so, in the sense of like the triggers that was gonna pose a lot of problems for us in terms of how this um start and end datel ogic works. And what would it allow? Also, you know if it it would go ahead and show somebody active if if they're um future dated it doesn't matter if they they would be active on in A. So then here's the context data example. And just to show you the the table structure. + +So we're at the implementation steps. There are five steps that they outline in the Developer Guide, which is, you know, creating your integration broker pieces. So this This will have several steps that go along with it. Then you have to develop and register the constituent event. Adapter. +You develop your affiliation handler. Then after that you go in to set up your affiliations, and then you test them. So the first thing is to create the um the message, and in my poc. I I just called it the same thing as the Developer Handbook, and I grab the Acad Prog table for my test. +I created the service, and then the service operation, and this is where you um go to the handler and tell it to use an application class. And this is where you um specify the delivered Affiliation Framework package and theulation Iv. Notify handler. + +You then have to go implement the publish the code to publish the message, and this is just the example taken from the people book, and i'm sorry from the Developer Guide, and which um you you can put it on like a safe uh pre-change or post change event to trigger on the components. So what I did is I did it on the academic program component, and then you test it. So at this point you're just making sure that you know your your web services working that, and it's pulling the information that you want from the component which in my case it did. + +That's all great and dandy, and my in my test. I set the person to a leap of absence, so I was able to see that it actually did it after you get that working, then you go in and do your application class. So in this case you are writing the people code, and the way that they describe it is you write the people code that marshals the message content into the constituent object, and to do that you have to extend the constituent event, abstract class, and in your code you have to marshal in that institution the constituent Id, the constituent type and the as of date variables. + +Then you go into a page here now in my notes, I have the navigation and stuff in the notes section. So when I share this, you'll have that information, but it's also in the guide page twenty You navigate to this page, and this is where you kind of do that association between your. What is this associating? Well, this is my custom root package, and It's associating it to my, to my service operation. That's what it is. So then you go in and develop your handler, and this is where you extend that affiliation abstract, that is going to have your is affiliated method that returns the to or false to to say whether or not somebody is active or an inactive for that particular affiliation. Then you go set of the affiliation, and and the reason you have to do it in this order is because, If you notice here at this point, you have to specify with your custom, packages and app class where that is resolved for that particular affiliation code, and then on the trigger tab. That is where you associated with the actual service operation, and in this case I had two tests. But, like this is The second one was one that I was using for my POC. + +Once you test the affiliation, you can go validate, obviously, that they, whether or not, they +got the affiliation assigned, and in terms of the batch processing. Right? So that's another way that we've talked about. This framework allows you to assign and remove affiliations. And this is where you can see this Pop: Select um query that I created. + +I have a section that you know just talks about Hi, what types of things you would need to consider when moving to a higher instance. And then I have a section here regarding the population selection query, because we ran into problems with that as well. So we have way more identities than fifty thousand. But our current uh setup for query is to return a maximum of fifty thousand. That was giving us an error that would have required the DBAs to go in and change a lot of configuration, and and it really goes against even what Oracle recommends. Right? So recommended maximum is fifty thousand people books, and they do allow you to change it. But we knew that we would get a little bit of resistance from tba team to do it. So that was another reason kind of a downside of how they how they do this process that they rely on the query tool, which is great for flexibility on the user side not so great when you're thinking through. This could have been written, you know, step based, and and not so intensive. + +I provided some links to some of the like the people books on how to set up the affiliations, how to manage them. And then um, just if you're interested in information about. When it was released, the Developer guide, the constituent web services, and then the two presentations that I found on HEUG. So are these available uh to uh people that aren't people to have customers, or are they? What's that? Some of these? So like the people books that you can just access that if you'd like um. Some of the stuff like uh these notes would be only with a login, and to my Oracle support, I think the developer guide is also behind that. Um, and the presentations are, If you have access to the HEUG website, I did it, guys? + +Laxmi Malladi: So how how long did it take to configure and do the customization stuff? That is a great question. Um, + +Pery Doan: Help me to remember + +Tommy Doan / SMU: When did you get started on the project? Well, I know that we engaged a consultant to bootstrap us. That was in the June timeframe, I believe. And then we went live last week. + +Pery Doan: And the consultant spent a lot of time, you know, looking at the delivered framework and just trying to figure out, Can we leverage it? But I don't remember at what point we made the decision, Tommy, that you know what let's just do this combined approach where we are leveraging the setup. I did not show that. But like we are leveraging the affiliation setup, obviouslyfor for our affiliations, and because we're not using any custom app classes or anything. This is kind of what it looks like. And the trigger. Just this automatically gets saved in here. I didn't do this, but when I put it in, it did it. + +On the context, data, since it's our custom program. I didn't see the need to set it up in here. We just have it in the program, and we know what it needs to to do. But all of that is set up, and we have our rankings as well, which the only, if if you look at the people book. It'll tell you that these rankings are really they're not meaningful and PeopleSoft in any way. They're just given to the customer, because they know that you'll probably want to rank for some other processing, and we do, we use it to determine the person's primary affiliation, and then we have an additional. So if I go, this is ur process. This is my uh, our custom. Run control to run, and +we have this set up where for every affiliation, I have a start SQL and SQL and a check data SQL. So this one checks to see if any of those um context data fields are different and updates them. + +All of that is to SQL definitions that the app engine is leveraging. So it cycles through, And so that allows us to add more, you know, affiliations in the future without having to really necessarily modifying the main program. All you have to do is kind of create your start and and and check SQLs for each. + +Laxmi Malladi: So the other question I had was The trigger is higher for a staff right hire record on the job. + +Pery Doan: I mean It's any any kind of change to Job. So, because it's the workforce sync gets triggered anytime you save the components, so you can, you know, transfer the person add a new effective data to to say they got compensated. Uh, you know a bonus, anytime it's saved. It's going to trigger that message. + +Laxmi Malladi: Yeah, What about the people with our job? Like person of interest type of? + +Pery Doan: We don't use that um so. But I would imagine, if they are maintained in the same component as job data that it would trigger that message, the workforce, think message. So the person of interest be anything similar to what we call an instructor-sponsored access. It is a specific thing in the workforce. So ou select somebody as staff, you would select them as just a person of interest. We don't do that, so I don't, + +for us It was not super hard to do all of this work, because we already had our current AMA system. Where this one is also kind of flexible. You know we have our our SQL for adding, say, a benefit. Eligible employee, you know. Here's our SQL, and it's all configuration based. So in this case I don't. I don't have that as SQL objects, I have it in a table that my app engine kind of cycles through to figure out how to how to find new people. So it's just a matter of taking a lot of the leg work that we had already done for AMA to now break it out into like these different, you know, by ample class or whatever. But of course it did take work because we had to create all this. But I didn't have to start from scratch like when we did. AMA going to Hr. Going to payroll, going to? You know, student records and talking through a bunch of details details about How do we know this selection criteria we should use? What is your business process? You know we didn't have to have those conversations because we already had, Am I in place? So that shortened our implementation window? + +Tommy Doan / SMU: Let me just jump in I wanted to ask is the silence because we're almost out of time, and any potential topic would take us over time. Or is the silence? Because uh periods covered everything? And uh, there's not much else to discuss at this point. I ask, because, uh, we don't yet have a an agenda for our next meeting. We typically meet every two weeks, and if people would like to have our next session, and at least one agenda topic for the next session, maybe just be intended for questions and response from Perry about this content. Then we could certainly do that. + +Perry said, mentioned this before, but we did spend some time looking for other campuses. Using it. We found two schools that use it to a degree one of which had implemented it, but the developer who had implemented it had left the campus. And so they, you know I don't want to say too much about that. But the story is, we didn't find many schools that were using it. So we may be one of very few. Yeah. I mean, I think that the way that people developed this project, this particular project, + +Pery: it's very clunky and complicated and inefficient. So I am so happy that we're not using their batch program, you know. Um, but I feel like we couldn't get around using batch, because you have things that change over time, even where there's not even a trigger in the system that would require that would that would alert anybody you know, like, for example, a student that never enrolls, that has to take, an we have to remove services for those people after census state. That kind of thing does not get a transaction in the system. It just it's a passage of time type of event. Also just having to put triggers all over the system, and even um just figuring out where and what processes trigger The kinds of data that we would care about seemed very overwhelming, and not not really something that I was excited about trying to do. So for for things that are obvious like, and that we already have like work for. Think. Well, that's great. But having to do all that for for all over the system just, I don't think that's doable, and so you have this batch that really should capture anything that you know your triggers Aren't capturing. But if it's not very efficient, and it's um the logic of it is, it's kind of um not working for us then, you know, having our own +process really helped, and I would be more than happy to share that with with schools. I think the the way that we'vedeveloped it was pretty much with the mind of How can we make it so that, you know, if other schools wanted to adopt it they could easily plug into it, you know. + +khazelton - Internet2: So if we do come back to this in some form uh in a couple of weeks. Um, you know this is out of scope for the presentation here. But What's the link between affiliations and uh service provisioning + +Tommy Doan / SMU: That will be a follow up conversation. I just thought it made logical sense to get started with Affiliation Framework Number one, because that's what we just put into production. And uh, it's also sequentially where we're sort of starting uh, and then we'll move forward to productionalize or operationalize COmanage, and then Grouper and then midPoint, so that we plan to, continue giving presentations and status updates as we progress through that. + +So yeah, if there are um topics that we should continue to explore around Affiliation Framework. I certainly want to make time for that. It may be that there are no other questions, or maybe people just need to digest this or go back and watch the recording. All of that is fine, If you'll please let me know if we should have this as Affiliation Framework as an ongoing agenda. Item, we'll make time for that but if not, we'll move forward to the question that Keith posed about how we leverage the Affiliation Framework data downstream. + diff --git a/perydAffilFwork.adoc1 b/perydAffilFwork.adoc1 new file mode 100644 index 0000000..9c25b47 --- /dev/null +++ b/perydAffilFwork.adoc1 @@ -0,0 +1,826 @@ +=== Peoplesoft Affiliation Framework at SMU +Pery Doan, 2 December 2022 + + +I'm glad to know that we have some people soft developers on the call. I wasn't sure how this meeting was going to go, so I kind of prepared for doing a little bit more of a deep dive on how the affiliation process actually works, even though at SMU We've um gone with um, using some of the delivered functionality, but then also have some custom implementation that goes with it. + +So I will start with just showing you guys, some diagrams of our environment. Do a little bit of a quick demo just to show you some screens, so that hopefully some of the more boring slides that come after that which are a lot to describe how the affiliation framework is intended to work in people soft, and how you would configure it. Um would make a little bit more sense to you guys, as we're going through it. Um also my main goal in kind of throwing this slides a bunch of slides in here. Um with, so that I can then share it with you guys afterwards, and you would have like sort of a a roadmap. Um, if you wanted to try doing a like a proof of concept um at your school, you would know where to start. It's not going to have everything in it, obviously, but I give you some references too at the end. Um to some of the technical documents you can. You can find out there. + +So just for you Peoplesoft folks. This is kind of our environment here. At some you we have CS HCM And Financials, + +image::images/mySmu.png['PeopleSoft Packages', width=700] + +and we try to keep current with um updates and fixes on a quarterly basis, and we are an oracle shop on release one thousand 19.16 and we use canvas solutions as our portal, and we leverage unified navigation to pull our Hcmfs content into our CS home pages, and we do heavily use uh Gideon Taylor: Well, I say heavily, we we now have it across all of three environments, and we um are starting to become more and more dependent on that uh framework. + +I just found this one diagram that I put together a while back, +just to show how we have like different sources around campus that enter data person data. + +image::images/identityFlow.png['Person Data Sources', width=700] + +The departments in blue. We have different applications as well, like slate and black bod that create Ids identity data in our People's cell system. So that's this column here, where we have this person model and we are using um the delivered service operations to sync our um fleet, split databases which you know we we split back in two thousand and fourteen, and use the subscriber only integration model, and we have um, you know, person basic sync and diversity sync, and all of this nice fun delivered um service operations enabled to do bidirectional syncing of our person data. And then um, we move on to the transaction data where we have employee.SMU: You synchronizing job data. It's just the one way integrationfor um, the workforce sync and position data. We also have user profiles getting synced, and that's not represented in this diagram, but for user. Profiles. + +Our CS instance is our um authoritative data source. So it it sends everything to uh employee.Smu. And also our setup data is authoritative in an employee Dot Smu, and syncs over to our campus solutions. Instance. From there also you can see that we do have some external affiliation sources that we manage. So in in these two boxes we have our employees and students, and we do have some sponsored identities that we select here based on um instructor data. So when we don't have the authoritative, you know job data to tell us this is an adjunct. We use the um schedule of classes and the instructor visor table to figure out who's teaching, and then we go ahead and give them a sponsorship that we call instructor-sponsored access. + +we have other sponsorships down here represented that go into this box here, which is our current custom, identity management type uh bolt-on that we created in peoplesoft and campus solutions which i'll talk more about. But then the end result is that we have our service provisioning, and even just physical access uh kind of directed by the logic that is running here in this engine. + +So for our current state. we have people soft as our person registry. We have this custom uh account maintenance automation, Ama bolt-on that determines the person, affiliation and service eligibility. And then we have Tommy's power shell script, or maybe more than one script that actually performs the service provisioning, + +What we did with our affiliation. Framework. Um project is sort of just replace +a piece of what Ama does, Which is this right here assigning what we call user types, and you will hear um me interchangeably use the term user type and affiliation. Um, throughout. + +In our in our future state as we are implementing um co-manage and grouper in midpoint. This is kind of Still, you know we're still in process of doing all of these things. One of the very first things and foundational things that we needed to do was, figure out. Well, how are we going to determine a person's affiliation? And we landed on trying to leverage the affiliation framework so that we can keep that logic and leverage all the work that we did with Ama Um to sort of transition into affiliation framework, to assign and remove our user types, and that makes it a little bit easier. Now, with our integration with kill, manage where we are, not only sending the bio demo data, but we're also attaching a bit of the affiliation data with those identities. We realize that that relationship data was sort of critical to have at the Ident in our new enterprise person registry. So that when we have somebody like we have a co-worker whose name is Eric macy. And there was a student who had that same name. And so it's just really a lot. it's very important in Higher Ed to be able to distinguish a student from a staff, for example, at the person registry level. So we are sending a few sort of affiliation related attributes to co-manage, and then, you know later, grouper is going to leverage co-manage for determining service. Eligibility assign and remove services, and, also, you know, reach back into people's soft itself, so it'll go back to employee. That's uh i'm sorry my smu and grab uh whatever information it requires for additional provisioning that perhaps wouldn't be in the co-manage registry. you know, moving to the affiliation framework, we took that opportunity to sort of revisit, and um broke out like our benefit, eligible employees, into more distinct affiliations. We also broke out the ones that we go ahead and select even before their effective date. So if somebody gets hired ahead of time right now Am. A will go ahead and assign these user types, and we can distinguish them really from people who are, you know, current versus future dated hires. So that kind of is a neat feature that I think we're gonna really like. We've also um been able to identify uh the non degree students, which is nice. And we broke out the active students into undergraduate and graduate to give us a little more granularity. At this point I wanted to show you just quickly couple of the pages. + +So our account maintenance automation I have, an Id that I want to show yobuild time. We're still using this as we're implementing our new. I am components allows us to. As you can see, it selects the type of person This is, and then assigned services, and we've gotten very accustomed to being able to see Well, where is that coming from. So I I built into this bolt on the ability for people to see why. This person is an adjunct faculty and a temporary staff. You can actually see that here, and I mean, I see here that they were terminated as a temp staff on twelve one. But this is a test environment, and I have not run. Am I? So? They still show as um active staff. But if I ran a for this person, they would lose that affiliation, and only retain their active temporary faculty. Um position. I can also see that this person is in the instructor Advisor Table. Um, and they are an adj. An instructor type of adjunct, and that they are attached to future classes. So if this person were a student, I would see, you know, student information here under these grids that kind of um. The ama is using itself as the source data to make these decisions. So that has been very helpful, because when there are any questions about, Why did this person not get picked up? Why did this person you know this or that? I'm? I'm not bombarded with a million people asking me questions. They can just go here and see what that what that may be. + +So the reason I wanted to show you that is because in the affiliation framework I'm. Going to show you this person who +and i'm going to show you the delivered page. So this is what the delivered page would look like. It still shows me right that they're an adjunct that they're an instructor, temp staff, and a conversion former student, which is something that am, I does not pay attention to. So that's why this is kind of a new thing here. I can see that the affiliation is active, and I can click into the context fields, which is something. Um, that we're leveraging with the delivered framework to be able to say, Okay, what are some? You can define these, you know. You can tell it whatever record name you want and field name. We kind of piggybacked on that, and just are saying co-manage, because that's what we're using it for. For when we create our Json uh message over to co-manage, we are grabbing information from this table. And we're saying that this person for their adjunct faculty, affiliation has, you know, happens to be in the meadow School of the arts, and the department is news. The Cat Junks title is not going to be populated. Uh, we're not allowed to do that at the time, and then we have their uh primary affiliation, which is faculty and their manager. + +Those are the attributes that we're sending over to co-manage.This is all very helpful. But what I ended up doing is creating a custom page, which is this affiliation snapshot, which is very much modeled after what I did with Ama, so that at the top I can see. Okay, here's the active user types. Here might be any inactive ones, and these are really affiliations. We're just using the same terminology that users are used to seeing which is user type, and also because, you know, when you're talking about this affiliation, which is more sort of like a high level affiliation. There's the affiliation codes, and then there's this affiliation and primary relation it can. There's a lot of affiliation, and it can get confusing. So we've decided to try to call these affiliation codes user types, and then um, we have the revelation, and I can see right away. I don't have to click and drill in to see what those attributes are for that particular code. And I also have our data source details um included here, so that that is, uh one of the benefits of us having done it. Um, you know, with a bit of a custom component to this process, and then I can also access affiliation, history. Of course I haven't. Nothing happened to this person, but you would be able to see over time things that are activated or um inactivated, as as time goes on, and I can also see details about our co-manage Api, whether it has gone uh successfully. Or it's waiting to be processed, or you know, whatever it may be. Also as as part of our co-manage um integration. We have a put that sends information from affiliation, framework, and our person model over to come, manage, and then we receive this post which um updates the external system, identifiers and people soft, and we haven't run it in this test environment. But we would see here what those external ids would be, so that we can understand and and see within people soft what that enterprise Id is, and what's the other one, Tommy? So just a bit of background on that When people soft sends. The put message to Co. Manage will respond back with identifiers that co-manage generates and so we're generate currently we're generating two. One is a net Id or a network Id and the other is the enterprise Id uh so a lot we could say about that. But that's uh that's the idea. So the other thing I wanted to show you here, while while I'm here is um. + +One other neat feature of the affiliation framework delivered application is that in any component where there are bio demo data type information about, you know, throughout people's soft. There is an integrated icon which is this little house which is the appellation display. So, no matter where, like I, I was looking at um like, I think, academic program information on on a student, and it had this a little affiliation box. So you can actually see what other affiliations that person might have right right there at your fingertips. That's pretty cool. + +So now like to jump into the affiliation framework. You know there's some pretty basic things. We know that it tracks them a relationship with the institutions um over time, and it was delivered as part of the nineo feature pack one. This was back in two thousand and nine. And uh, it is a more robust method of tracking affiliations than the relationship with institution component within campus solutions. I don't know if you guys are familiar with it. We used to use that as well, and I thought it was going to replace it. But as I was looking at people's books recently found out that it's not. It didn't replace it. It's just more of a robust method of of tracking affiliations. This framework will allow you to update those manually and batch or triggered by affiliation events. +And then we already talked about the little icon. So it is part of the constituent web services and campus solutions. Not sure if you guys are familiar with it, but it is designed. The Cws is designed to allow and facilitate integration of people, self systems with external systems, and it also is heavily used with interaction. Hub and the external search match functionality, and of course, oracle identity manager delivered a connector into CS. Which increase integrates with affiliation, framework, and um constituent web services. I've heard not very good things about that, but that is what it is. We also know that Um, the constituent transaction management framework, which is what we use for loading our slate applications into people soft. There's the data update rules. The affiliation override dependencies for that interface. So that's another reason why we wanted to. Um, you know, use the affiliation framework, and when we went live with a slate. + +I don't remember when it was, but it was a while back we tried to implement affiliation framework and had a lot of hurdles. So we ended up just so that we could meet this need delivering. I just created like a sql script that the Dba's had on a cron job that would run every night to identify current employees, former employees, current students, and former students, so that when new apps came into the system it would check and see, because they did not want +those applications to update any bio demo data for existing current employees, for example, or existing current students and things like that. So that's where that comes in currently. We've gone live with affiliation framework like two weeks ago when it went Live any discrepancies, you know, do doing, reporting. We can um use it to continue our co-manage integration efforts and things like that. But I thought it would be interesting to show you guys, some of the counts that we have with our current um affiliations. And I, broke out the instructor, one, because that one is not dependent on job data, obviously right. So So the ones that have, for example, adjunct faculty. There's three hundred and thirty-eight +but as you can see here, Um, we have e instructor assigned to six hundred and sixty-eight that are set up as instructor type adjunct in the instructor visor table, and that is because we do have an issue with getting those higher forms in a timely fashion and things like that. So the instructor. Affiliation code is intended to be sort of a high level. We don't want to say that you are an adjunct, because you can be a teaching assistant, or a Gsi, or graduate student structure, full time, faculty, or whatever. And currently, unfortunately, in our peoplesoft system. These instructor types are not maintained, So there's not a high degree of confidence, and using those as our as our source to say, Yes, you are now assigned right. We know of instances where, for example, we have full time faculty that come in, so they're in this table as full time faculty. They retire. They come back as adjuncts, and nobody updates their role to adjunct again. So we're moving in other projects to try to improve that data quality, and we have made a lot of progress in trying to sync it up with what job data says and things like that. But we're still not at one hundred percent confidence in those designations. But we're moving in the right direction, I feel, and I think and hope that by the time we're ready to go live with our new Im solution that things will be a lot more timely and accurate in our in our data. + +So how does the delivered Affiliation Framework process works. A lot of what you're gonna see are things that I took from the Campus Solutions Affiliation Developer Guide. I just copied this high level diagram of how it works where you have constituent information that's changing the system. For example, Admissions matriculates a student or HR terminates an employee, that's a change; or some self service utility that a student uses to change their address. + +Then you have Integration Broker messages that are published. And + +253 +00:33:42.740 --> 00:33:55.369 +Pery Doan: um the way that oracle developed. This process was really thinking, and and it really was developed forures I mean the batch concept came later on, and kind of was + +254 +00:33:55.950 --> 00:34:04.120 +Pery Doan: kind of forced into it, but it was intended, so that you know any time you have a change in the system, it would trigger a message + +255 +00:34:04.450 --> 00:34:06.020 +Pery Doan: which would then + +256 +00:34:06.250 --> 00:34:08.979 +Pery Doan: trigger the affiliation framework process + +257 +00:34:09.090 --> 00:34:14.549 +Pery Doan: to process that message and then assign or remove affiliations according to that data. + +258 +00:34:14.600 --> 00:34:21.400 +Pery Doan: And then this fifth uh optional, I guess message gets published, which is your um + +259 +00:34:22.120 --> 00:34:23.589 +Pery Doan: cws. + +260 +00:34:25.060 --> 00:34:29.540 +Pery Doan: If you want to talk to an external system like Om. Or something + +261 +00:34:29.929 --> 00:34:31.120 +Pery Doan: uh, + +262 +00:34:31.730 --> 00:34:32.850 +Pery Doan: let's see. + +263 +00:34:33.909 --> 00:34:50.479 +Pery Doan: Yeah. And and the uh Affiliation Developer guide does not really talk in great depth about the constituent web services. They refer you to the constituent Web Services developer, guide for further information. If you're interested in and knowing more about that, + +264 +00:34:50.489 --> 00:34:58.410 +Pery Doan: I to send you. Of course we're not doing that. We created a custom web service that leverages affiliation framework. Data. + +265 +00:34:58.520 --> 00:35:05.590 +Pery Doan: And since Jason message over to Co-manage um that I that I shared earlier our put message. + +266 +00:35:08.620 --> 00:35:10.349 +Pery Doan: So um + +267 +00:35:10.480 --> 00:35:26.620 +Pery Doan: now i'm gonna talk through these steps kind of like in a little bit more detail. So the constituent information is changing in the system via different ways. Right? So the the relevance of of outlining. This is because you you've got to think about. + +268 +00:35:26.820 --> 00:35:44.589 +Pery Doan: If you're If you want to go with trigger based transactions, then you pretty much have to um update everywhere in the system where you're interested in having that trigger sent, you have to modify those components if they are not already + +269 +00:35:44.980 --> 00:35:47.850 +Pery Doan: sending some sort of message right? So like + +270 +00:35:48.170 --> 00:35:50.910 +Pery Doan: we wouldn't need to necessarily + +271 +00:35:51.150 --> 00:35:59.239 +Pery Doan: modify the job component, because it's already triggering work for sync into campus solutions so + +272 +00:35:59.470 --> 00:36:12.750 +Pery Doan: that could be leveraged as something that would then trigger the affiliation framework. But if I wanted to change somebody's academic program or um, you know, admissions matriculate somebody, or whatever it may be, + +273 +00:36:12.970 --> 00:36:22.080 +Pery Doan: those are things that don't trigger anything today, and camp solutions. So we would have to modify those components to have some sort of message trigger + +274 +00:36:22.260 --> 00:36:40.500 +Pery Doan: the uh, And so that that's what these two like, whether still service or administrator makes online update. The other thing is like batch updates. So if we have a program, for example, like I'm: sure pretty sure admissions, You know, matriculates people in bachelor. I'm gonna go one by one. So those batch program updates um + +275 +00:36:41.150 --> 00:36:42.669 +Pery Doan: have to be. + +276 +00:36:42.820 --> 00:36:47.200 +Pery Doan: You have to use this specific pattern to trigger + +277 +00:36:47.470 --> 00:36:51.849 +Pery Doan: the appropriate messages so that those triggers happen. + +278 +00:36:55.610 --> 00:37:02.719 +Pery Doan: So. Integration, That's the second step. The Integration broker message is published, and so that's the code that's needed, and + +279 +00:37:02.840 --> 00:37:22.230 +Pery Doan: i'm sorry if i'm not being very good at this I put this presentation together yesterday, and I I haven't practiced it at all. So please forgive me upon all over the place. But this um just explain basically what I just said about the component triggers. You can also like, if you have a custom app engine program that makes updates + +280 +00:37:22.240 --> 00:37:36.959 +Pery Doan: or even delivered one. You can actually have a call, you know, and and trigger the messages, and then for the cobalt and sqrs. You would use this um delivered table, and that you'll publish programs to the message into the + +281 +00:37:38.160 --> 00:37:39.229 +Pery Doan: table. + +282 +00:37:41.190 --> 00:37:45.999 +Pery Doan: The If the affiliation framework processes the message. And this is where + +283 +00:37:46.850 --> 00:38:04.900 +Pery Doan: this is cool. I I like the fact that in affiliation firmware. There's really only three tables that are updated. You have your person affiliation. And then the context data that I showed you guys like there's one that has the data fields because it's um configurable. You can tell it what record and what field + +284 +00:38:04.910 --> 00:38:09.300 +Pery Doan: to use. Then it it has to have a way to track that, And then there's the value + +285 +00:38:09.420 --> 00:38:10.430 +uh + +286 +00:38:10.460 --> 00:38:23.659 +Pery Doan: table. So it's. It's pretty simplistic. The thing that's not very simplistic is how they design this thing to work. And this is a diagram that um we. We hired a consultant to help us with this. + +287 +00:38:23.970 --> 00:38:31.330 +Pery Doan: He's very, very smart and very good with um at packages, so he really understood, after + +288 +00:38:31.680 --> 00:38:33.919 +Pery Doan: doing our proof of concept + +289 +00:38:34.020 --> 00:38:36.909 +Pery Doan: sort of how this thing works, and + +290 +00:38:37.990 --> 00:38:50.940 +Pery Doan: I have the example of a custom trigger event which is really what is described in the Developer guide is This is what you will see on how to, you know, set up a proof of concept + +291 +00:38:51.190 --> 00:39:02.039 +Pery Doan: and what it does, is it? Will you know you have to? I'll show you the steps, but you you create your own service operation, and then it that's what triggers through the handler. + +292 +00:39:02.070 --> 00:39:13.139 +Pery Doan: Um, the affiliation framework. There's a custom adapter that's called, and then it returns to the affiliation framework. Then it does the implementation class + +293 +00:39:13.330 --> 00:39:22.120 +Pery Doan: then it returns the information framework, and then it finally gets an answer whether it's affiliated or not. And this is done + +294 +00:39:22.350 --> 00:39:25.980 +Pery Doan: for one Id at a time right, because the way that this was + +295 +00:39:26.190 --> 00:39:32.760 +Pery Doan: designed was relief to work for triggers. So you're only processing one message at a time. + +296 +00:39:33.410 --> 00:39:38.440 +Pery Doan: Then um for the delivered trigger example. We have that workforce sync + +297 +00:39:39.110 --> 00:39:44.939 +Pery Doan: where you have an existing service operation that you can modify. You can go in there and add the handler to it, + +298 +00:39:44.960 --> 00:39:50.909 +Pery Doan: and then it will, of course, as soon as it triggers it will go ahead and trigger your affiliation framework. + +299 +00:39:51.340 --> 00:39:53.289 +Pery Doan: The batch process + +300 +00:39:53.420 --> 00:40:07.359 +Pery Doan: is run through Process scheduler, of course, but it um. It calls that message handler directly, so it does not generate Ib. Messages so you wouldn't see those an integration broker or anything like that, however, + +301 +00:40:07.480 --> 00:40:13.289 +Pery Doan: because it is working with just the way that this is designed, it's going to + +302 +00:40:13.370 --> 00:40:24.280 +Pery Doan: process one Id at a time instead of doing set-based processing. It's an app engine that is very inefficient, and it leverages the pop select uh + +303 +00:40:24.700 --> 00:40:31.930 +Pery Doan: query um forget what that framework is called, but it does leverage that, and + +304 +00:40:32.170 --> 00:40:44.550 +Pery Doan: you know it just. It proved very inefficient for us. So that's one of the reasons why we are not using this delivered process. I know that there are two schools out there at least. Um. + +305 +00:40:44.740 --> 00:40:56.149 +Pery Doan: A long time ago I I found two presentations, one from two thousand and fifteen, one from two thousand and eighteen where um you know those schools are running it, and they're running the deliver process. So + +306 +00:40:56.710 --> 00:41:04.030 +Pery Doan: I I think that it can be done. It just was not something that we did we deemed feasible for our institution. + +307 +00:41:05.830 --> 00:41:13.099 +Pery Doan: All right. So then, um step four is, you know, the affiliations getting assigned or removed. And I just have this to show you + +308 +00:41:13.140 --> 00:41:15.229 +Pery Doan: an example of + +309 +00:41:15.690 --> 00:41:21.279 +Pery Doan: a person with affiliations. And then here's the person table. + +310 +00:41:21.490 --> 00:41:27.639 +Pery Doan: You can see here that the start date is a key field. So you do have to provide that. And + +311 +00:41:27.840 --> 00:41:36.780 +Pery Doan: the way that the delivery program works, is it just like I said, it returns a true or false the start date. + +312 +00:41:36.860 --> 00:41:39.130 +Pery Doan: It's kind of determined by + +313 +00:41:39.890 --> 00:41:41.790 +Pery Doan: what that message + +314 +00:41:42.180 --> 00:41:44.589 +Pery Doan: had in in the effective date, + +315 +00:41:44.730 --> 00:41:50.510 +Pery Doan: or you can go around it by doing your own sequel to. If you wanted to try to figure that out + +316 +00:41:50.720 --> 00:41:53.860 +Pery Doan: there are in the Developer Handbook. + +317 +00:41:54.170 --> 00:42:04.280 +Pery Doan: Um. Some examples and and some definite rules that you need to be aware of in terms of how it deals with the start, date, and end dates and those kinds of rules, + +318 +00:42:04.640 --> 00:42:09.139 +Pery Doan: which was another reason why, for smu + +319 +00:42:09.490 --> 00:42:13.709 +Pery Doan: it just wasn't going to handle some of our scenarios very well + +320 +00:42:14.180 --> 00:42:16.060 +Pery Doan: like. For example, + +321 +00:42:16.160 --> 00:42:22.120 +Pery Doan: the typical one is our Temp employee population, we have a higher form. + +322 +00:42:22.300 --> 00:42:25.309 +Pery Doan: It's a Gideon Taylor form + +323 +00:42:25.610 --> 00:42:26.709 +Pery Doan: that + +324 +00:42:27.080 --> 00:42:37.699 +Pery Doan: is submitted by departments, and really follows a business practice that peril has had over the years, which is whenever they get a they used to get the paper epaph, or now the electronic one. + +325 +00:42:38.050 --> 00:42:43.210 +Pery Doan: They know the start and end date of a temporary job. So + +326 +00:42:43.770 --> 00:42:51.909 +Pery Doan: they go ahead and create two job rows, the start and the end the termination. And so, + +327 +00:42:51.950 --> 00:43:00.780 +Pery Doan: in the sense of like the triggers that was gonna pose a lot of problems for us in terms of how this um start and end date + +328 +00:43:01.720 --> 00:43:07.119 +Pery Doan: logic works. And what would it allow? Also, you know if it it would + +329 +00:43:07.740 --> 00:43:15.959 +Pery Doan: go ahead and show somebody active if if they're um future dated it doesn't matter if they they would be active on in A. + +330 +00:43:16.590 --> 00:43:19.779 +Pery Doan: So then here's the context data + +331 +00:43:20.330 --> 00:43:22.969 +Pery Doan: example. And just to show you + +332 +00:43:23.050 --> 00:43:25.769 +Pery Doan: the the table structure. + +333 +00:43:29.140 --> 00:43:33.490 +Pery Doan: Okay. So we're at the implementation steps. Do we have any questions up to that point. + +334 +00:43:37.670 --> 00:43:43.950 +Pery Doan: Alright, so. Um! There are five steps that they outline in the Developer Guide, + +335 +00:43:44.360 --> 00:43:46.529 +Pery Doan: which is, you know, creating your + +336 +00:43:46.670 --> 00:43:51.830 +Pery Doan: integration broker pieces. So this This will have several steps that go along with it. + +337 +00:43:52.070 --> 00:43:57.009 +Pery Doan: Then you have to develop and register the constituent event. Adapter. + +338 +00:43:57.160 --> 00:43:59.939 +Pery Doan: You. Develop your affiliation handler. + +339 +00:44:00.080 --> 00:44:04.749 +Pery Doan: Then after that you go into set up your affiliations, and then you test them. + +340 +00:44:08.320 --> 00:44:23.730 +Pery Doan: So the first thing is to create the um the message, and in my poc. I I just called it the same thing as the Developer Handbook, and I grab the Acad Prague table for my test. + +341 +00:44:26.280 --> 00:44:28.239 +Pery Doan: I created the service, + +342 +00:44:30.420 --> 00:44:32.770 +Pery Doan: and then the service operation, + +343 +00:44:33.860 --> 00:44:44.300 +Pery Doan: and this is where you um go to the handler and tell it to use an application class. And this is where you um specify + +344 +00:44:44.610 --> 00:44:48.089 +Pery Doan: the delivered affiliation, framework, package + +345 +00:44:48.300 --> 00:44:49.750 +Pery Doan: and theulation. + +346 +00:44:49.880 --> 00:44:51.629 +Pery Doan: Iv. Notify handler. + +347 +00:44:53.830 --> 00:45:04.840 +Pery Doan: You then have to go implement the publish the code to publish the message, and this is just the example taken from the people book, and i'm sorry from the Developer Guide, + +348 +00:45:05.100 --> 00:45:12.039 +Pery Doan: and which um you you can put it on like a safe uh pre-change or post change event + +349 +00:45:12.080 --> 00:45:17.059 +Pery Doan: to trigger on the components. So what I did is I did it on the um + +350 +00:45:17.590 --> 00:45:20.169 +Pery Doan: academic program component, + +351 +00:45:22.430 --> 00:45:27.049 +Pery Doan: and then you test it. So at this point you're just making sure that + +352 +00:45:27.480 --> 00:45:34.800 +Pery Doan: you know your your web services working that, and it's pulling the information that you want from the component which in my case it did. + +353 +00:45:35.000 --> 00:45:43.660 +Pery Doan: That's all great and dandy, and my in my test. I set the person to a leap of absence, so I was able to see that it actually did it + +354 +00:45:44.370 --> 00:45:46.060 +Pery Doan: after you get that + +355 +00:45:46.290 --> 00:45:50.230 +Pery Doan: working, then you go in and do your + +356 +00:45:50.950 --> 00:45:52.740 +Pery Doan: application class. + +357 +00:45:54.740 --> 00:45:58.470 +Pery Doan: So in this case you are um + +358 +00:45:59.320 --> 00:46:08.960 +Pery Doan: writing the people code, and the way that they describe it is you write the people code that marshals the message content into the constituent object, + +359 +00:46:09.600 --> 00:46:13.929 +Pery Doan: and so to do that you have to extend the constituent event, abstract class, + +360 +00:46:14.150 --> 00:46:15.160 +Pery Doan: and + +361 +00:46:15.740 --> 00:46:20.879 +Pery Doan: in the in your code you have to marshal in that institution the constituent Id + +362 +00:46:20.900 --> 00:46:25.179 +Pery Doan: the constituent type and the as of date variables. + +363 +00:46:27.200 --> 00:46:29.870 +Pery Doan: Then you go into a + +364 +00:46:29.950 --> 00:46:41.759 +Pery Doan: a page here now in my notes, you I I have the navigation and stuff in the notes section. So when I share this, you'll have that information, but it's also in the guide page twenty um. + +365 +00:46:41.920 --> 00:46:46.149 +Pery Doan: You navigate to this page, and this is where you kind of do that association + +366 +00:46:48.190 --> 00:46:50.740 +Pery Doan: between um Your. + +367 +00:46:53.120 --> 00:46:55.009 +Pery Doan: What is this associating? + +368 +00:46:56.490 --> 00:47:02.200 +Pery Doan: Well, this is my custom root package, and It's associating it to my, to my service operation. That's what it is. Okay. + +369 +00:47:02.870 --> 00:47:21.079 +Pery Doan: So then you go in and develop your handler, and this is where you extend that affiliation abstract, that is going to have your is affiliated method that returns the to or false to to say whether or not somebody is active or an inactive for that particular affiliation. + +370 +00:47:24.590 --> 00:47:28.009 +Pery Doan: Then you go set of the affiliation, + +371 +00:47:28.680 --> 00:47:33.729 +Pery Doan: and and the reason you have to do it in this order is because, If you notice here + +372 +00:47:33.860 --> 00:47:39.809 +Pery Doan: at this point, you have to specify with your custom, packages and app class + +373 +00:47:39.890 --> 00:47:42.040 +Pery Doan: where that um + +374 +00:47:42.220 --> 00:47:46.179 +Pery Doan: is resolved for that particular affiliation code, + +375 +00:47:46.500 --> 00:47:56.489 +Pery Doan: and then on the trigger tab. That is where you associated with the actual service operation, and in this case I had two tests. But, like this is The second one was one that I + +376 +00:47:56.660 --> 00:47:58.830 +Pery Doan: that I was using for my poc. + +377 +00:48:00.120 --> 00:48:09.170 +Pery Doan: And I have some notes here. I'm not gonna go over them, but it's just as I was going through it. I was just making notes of different things. + +378 +00:48:11.760 --> 00:48:17.879 +Pery Doan: Once you test the affiliation, you can go validate, obviously, that they, whether or not, they + +379 +00:48:18.580 --> 00:48:20.740 +Pery Doan: they got the affiliation assigned, + +380 +00:48:22.170 --> 00:48:39.520 +Pery Doan: and in terms of the batch processing. Right? So that's another way that we've talked about. This framework allows you to assign and remove affiliations. And this is where you can see this Pop: Select um query that I created. + +381 +00:48:42.010 --> 00:48:43.450 +Pery Doan: Um, + +382 +00:48:44.630 --> 00:48:46.839 +Pery Doan: yeah, I I don't know what to say about that. + +383 +00:48:49.700 --> 00:48:54.029 +Pery Doan: Okay? Then. Um, I have a section that you know just talks about. + +384 +00:48:54.410 --> 00:49:01.709 +Pery Doan: Hi, what types of things you would need to consider when moving to a higher instance. + +385 +00:49:02.910 --> 00:49:20.889 +Pery Doan: And then I have a section here regarding the population selection query, because we ran into problems with that as well. So we have way more identities than fifty thousand. But our current uh setup for query is to return a maximum of fifty thousand. + +386 +00:49:20.900 --> 00:49:40.170 +Pery Doan: And so that was giving us an error that would have required the Dbas to go in and change a lot of configuration, and and it really goes against even what oracle recommends. Right? So recommended maximum is fifty thousand people books, and they do allow you to change it. But + +387 +00:49:40.300 --> 00:49:43.639 +Pery Doan: we knew that we would get a little bit of resistance from our + +388 +00:49:43.920 --> 00:49:48.169 +Pery Doan: tba team um to do it. So that was another reason kind of a + +389 +00:49:48.400 --> 00:49:50.109 +Pery Doan: downside of + +390 +00:49:50.360 --> 00:49:54.739 +Pery Doan: how they how they do this process that they rely on the query + +391 +00:49:54.840 --> 00:50:03.670 +Pery Doan: tool, which is great for flexibility on the user side not so great when you're thinking through. This could have been written, + +392 +00:50:04.050 --> 00:50:07.909 +Pery Doan: you know, step based, and and not so + +393 +00:50:08.650 --> 00:50:13.049 +Pery Doan: intensive. And then I provided some links to some of the + +394 +00:50:13.270 --> 00:50:17.020 +Pery Doan: like the people books on how to set up the affiliations, + +395 +00:50:17.470 --> 00:50:21.899 +Pery Doan: how to manage them. And then um, just if you're interested in + +396 +00:50:22.650 --> 00:50:26.600 +Pery Doan: information about. When it was released, the Developer guide, + +397 +00:50:27.040 --> 00:50:32.439 +Pery Doan: the constituent web services, and then the two presentations that I found on hug. + +398 +00:50:32.870 --> 00:50:50.539 +Pery Doan: So are these available uh to uh people that aren't people to have customers, or are they? What's that? Some of these? So like the people books that you can just access that if you'd like um. Some of the stuff like uh these notes would be + +399 +00:50:50.550 --> 00:50:53.389 +Pery Doan: only with a login, and to my oracle support, + +400 +00:50:54.510 --> 00:51:02.180 +Pery Doan: I think the developer guide is also behind that. Um, and the presentations are, If you have access to the HEUG website, + +401 +00:51:07.020 --> 00:51:08.690 +Pery Doan: I did it, guys? + +402 +00:51:09.070 --> 00:51:19.529 +Laxmi Malladi: So how how long did it take to configure and do the customization stuff? That is a great question. Um, + +403 +00:51:19.610 --> 00:51:21.140 +Pery Doan: tell me to remember + +404 +00:51:21.490 --> 00:51:24.019 +Tommy Doan / SMU: when you get started on the project? + +405 +00:51:24.180 --> 00:51:34.329 +Tommy Doan / SMU: Well, I know that we engaged a consultant uh to uh kind of bootstrap us. That was in the June timeframe, I believe. + +406 +00:51:34.390 --> 00:51:36.540 +Tommy Doan / SMU: And then we went live last week. + +407 +00:51:37.850 --> 00:51:45.009 +Pery Doan: Yeah. And he spent a lot of time, you know, looking at the delivered framework and just trying to figure out, Can we leverage it? + +408 +00:51:45.130 --> 00:51:53.169 +Pery Doan: And um so. But I don't remember at what point we made the decision, Tommy, that you know what let's just + +409 +00:51:53.560 --> 00:52:00.210 +Pery Doan: do this combined approach where we are leveraging the setup. Am I still sharing my screen? + +410 +00:52:00.330 --> 00:52:08.040 +Pery Doan: Okay, Because, Um, I did not show that. But like we are leveraging the affiliation setup, obviously + +411 +00:52:08.640 --> 00:52:23.550 +Pery Doan: for for our affiliations, and because we're not using any custom app classes or anything. This is kind of what it looks like. And the trigger. Just this automatically gets saved in here. I didn't do this, but when I + +412 +00:52:23.750 --> 00:52:38.120 +Pery Doan: put it in it, it did it. Um on the context, data, since it's our custom program. I didn't see the need to set it up in here. We just have it in the program, and we know what it needs to to do. But all of that is set up, and we have our rankings + +413 +00:52:38.690 --> 00:52:39.830 +Pery Doan: um + +414 +00:52:40.520 --> 00:52:59.279 +Pery Doan: as well, which the only, if if you look at the people book. It'll tell you that these rankings are really they're not meaningful and people soft in any way. They're just given to the customer, because they know that you'll probably want to rank for some other processing, and we do, we use it to + +415 +00:52:59.520 --> 00:53:02.390 +Pery Doan: determine the person's primary affiliation, + +416 +00:53:03.610 --> 00:53:08.599 +Pery Doan: and then um, we have an additional. So if I go, this is + +417 +00:53:09.080 --> 00:53:13.279 +Pery Doan: our process. This is my uh, our custom. Run control + +418 +00:53:13.680 --> 00:53:15.500 +Pery Doan: to run, and + +419 +00:53:15.830 --> 00:53:17.279 +Pery Doan: we have this + +420 +00:53:17.400 --> 00:53:19.540 +Pery Doan: set up where + +421 +00:53:20.040 --> 00:53:22.149 +Pery Doan: for every affiliation, + +422 +00:53:22.850 --> 00:53:26.299 +Pery Doan: I have a start sequel and Sql + +423 +00:53:26.360 --> 00:53:34.419 +Pery Doan: and a check data sequel. So this one checks to see if any of those um context data fields are different and updates them. + +424 +00:53:34.600 --> 00:53:37.830 +Pery Doan: And all of that is to sequel definitions that the + +425 +00:53:38.070 --> 00:53:52.010 +Pery Doan: app engine is leveraging. So it cycles through, And so that allows us to add more, you know, affiliations in the future without having to really necessarily modifying the main program. All you have to do is kind of create your start and and and check sequels for each. + +426 +00:53:58.270 --> 00:54:01.759 +Laxmi Malladi: So the other question I had was + +427 +00:54:02.260 --> 00:54:08.030 +Laxmi Malladi: so. The trigger is higher for a staff right higher record on the job. + +428 +00:54:08.330 --> 00:54:27.470 +Pery Doan: I mean It's any any kind of change to Job. So, because it's the workforce sync gets triggered anytime you save the components, so you can, you know, transfer the person add a new effective data to to say they got compensated. Uh, you know a bonus, anytime it's saved. It's going to trigger that message. + +429 +00:54:27.880 --> 00:54:32.209 +Laxmi Malladi: Yeah, What What about the people with our job? Like person of interest, + +430 +00:54:32.330 --> 00:54:33.609 +Laxmi Malladi: type of? + +431 +00:54:34.330 --> 00:54:36.490 +Laxmi Malladi: I think it. + +432 +00:54:36.560 --> 00:54:45.819 +Pery Doan: I don't we don't use that um so. But I would imagine, if they are maintained in the same component as job data that it would trigger that message, + +433 +00:54:48.540 --> 00:54:56.659 +Pery Doan: the workforce, think message. So the person of interest be anything similar to what we call an instructor-sponsored access. + +434 +00:54:58.070 --> 00:55:03.000 +Pery Doan: I mean it. It is a specific thing in the workforce. + +435 +00:55:03.420 --> 00:55:09.959 +Pery Doan: Yeah, So like you, you select somebody as staff, you would select them as just a person of interest. + +436 +00:55:11.370 --> 00:55:13.520 +Pery Doan: We don't do that, so I don't, + +437 +00:55:27.120 --> 00:55:28.470 +Pery Doan: and you know it + +438 +00:55:28.530 --> 00:55:30.879 +Pery Doan: for us. It was not + +439 +00:55:31.810 --> 00:55:37.460 +Pery Doan: super hard to do all of this work, because we already had + +440 +00:55:37.580 --> 00:55:40.689 +Pery Doan: our current ama system. Where + +441 +00:55:40.910 --> 00:55:42.549 +Pery Doan: this one is also + +442 +00:55:42.600 --> 00:55:58.909 +Pery Doan: kind of flexible. You know we have our our sequel for adding, say, a benefit. Eligible employee, you know. Here's our sequel, and it's all configuration based. So in this case I don't. I don't have that as sequel objects, I have it in a table that my app engine kind of cycles through to figure out how to how to find new people. So it's just a matter of taking a lot of the leg work that we had already done for Ama to now break it out into like these different, you know, by ample class or whatever. But of course it did take work because we had to create all this. But I didn't have to start from scratch like when we did. Ama going to Hr. Going to payroll, going to? You know, student records and talking through a bunch of details details about How do we know this selection criteria we should use? What is your business process? You know we didn't have to have those conversations because we already had, Am I in place? So that shortened our implementation window? + +Tommy Doan / SMU: Let me just jump in to say this was a lot of content and very uh a lot of directions uh questions could go, and I uh, I guess I I wanted to ask is the silence because we're almost out of time, and any potential topic would take us over time. Or is the silence? Because uh periods covered everything? And uh, there's not much else to discuss at this point. I ask, because, uh, we don't yet have a an agenda for our next meeting. We typically meet every two weeks, and if people would like to have our next session, and at least one agenda topic for the next session, maybe just be intended for questions and response from Perry about this content. Then we could certainly do that. + +Yeah, I was gonna mention learned. Yeah, Perry said, mentioned this before, but we did spend some time looking for other campuses. Using it. We found, like Perry mentioned. We found two schools that use it to a degree one of which had implemented it, and but the developer who had implemented it had left the campus. And so they, you know I don't want to say too much about that. But the the the story is, we didn't find many schools that were using it. So we may be one of very few. Yeah. I mean, I think that the way that people developed this project, this particular project, + +Pery: it's very clunky and complicated and inefficient. So I am so happy that we're not using their batch program, you know. Um, but I feel like we couldn't get around using batch, because you know, you. You have things that change over time, even where there's not even a trigger in the system that would require that would that would alert anybody you know, like, for example, a student that never enrolls, that has to take, an we have to remove services for those people after census state. That kind of thing does not get a transaction in the system. It just it's a passage of time type of event. Also just having to put triggers all over the system, and even um just figuring out where and what processes trigger The kinds of data that we would care about seemed very overwhelming, and not not really something that I was excited about trying to do. So for for things that are obvious like, and that we already have like work for. Think. Well, that's great. But having to do all that for for all over the system just, I don't think that's doable, and so you have this batch that really should capture anything that you know your triggers Aren't capturing. But if it's not very efficient, and it's um the logic of it is, it's kind of um not working for us then, you know, having our own +process really helped, and I would be more than happy to share that with with schools. I think the the way that we'vedeveloped it was pretty much with the mind of How can we make it so that, you know, if other schools wanted to adopt it they could easily plug into it, you know. + +====== + +khazelton - Internet2: So if we do come back to this in some form uh in a couple of weeks. Um, you know this is out of scope for the presentation here. But What's the link between affiliations and uh service, provisioning or determining which + +Tommy Doan / SMU: uh yes, that will be a follow up conversation. I just thought it made logical sense to get started with affiliation Framework Number one, because that's what we just put into production. And uh, it's also sequentially. That's where we're sort of starting uh, and then we'll move forward to sort of productionalize uh or operationalize Co. Manage, and then grouper and then midpoint, so that we plan to, uh continue giving presentations and status updates as we progress through that. So yeah, uh, again, if there are um topiCS that we should continue uh to explore around affiliate framework. I certainly want to make time for that. Um! It may be that again. Maybe there are no other questions, or maybe people just need to digest this or go back and watch the recording. All of that is fine, If you'll please let me know if we should have this as affiliation framework as an ongoing agenda. Item, we'll make time for that but if not, we'll move forward to the question that Keith posed about how we leverage the affiliation framework data downstream. + +=== end presentation diff --git a/ps2grouper.adoc b/ps2grouper.adoc index b753455..b096fd4 100644 --- a/ps2grouper.adoc +++ b/ps2grouper.adoc @@ -1,6 +1,6 @@ ==== Grouper Data Integration with PeopleSoft @ UNC Chapel Hill -Ethan Kromhout, 2 Nov 2022 +Ethan Kromhout, 2 November 2022 I wanted to talk about how our data currently gets from Peoplesoft to Grouper. There's maybe a story that I didn't include about how Grouper data gets indirectly to peoplesoft. But maybe I can comment about that at the end. @@ -10,7 +10,7 @@ We've got three main paths the data goes through to get from from Peoplesoft to The first path, the the largest and most complex, is our our homegrown Java app. We call it Directory Manager, but it's basically been the place where the business logic happens for deciding people's, affiliation states and things like that, and getting those things published out to, particularly, the OpenLDAP Directory, but also Active Directory. -image::images/unc3flows.jpg[Directory Manager] +image::images/unc3flows.jpg['The 3 paths from Peoplesoft to IAM infrastructure', width=700] So that path through Directory Manager includes our oldest integration, which is with Campus Solutions. We brought up Campus Solutions in 2009 well ahead of bringing up the HR And Finance modules. The integrations are messaging and the reason for those two different transports are that the original Campus Solutions integration was all messaging. @@ -46,7 +46,7 @@ and it's doing SQL queries into any of the big towers along with some other data push those in the group or via the group or web services. So a lot of the roles that our PeopleSoft security folks create are useful in other applications besides just inside PeopleSoft, so they they function. As for proxies to other data that people might have access to because of the roles that they have. And and one of the Peoplesoft powers. So, for example, they can go to a data warehouse. -image::images/uncInfMat.jpg[Informatica path] +image::images/uncInfMat.jpg['Informatica path', width=700] Instead of having a separate role structure for the data warehouse the people self-security folks would rather just replicate those roles that they have, and people solve the Grouper, and then ! It can be queried, or L. That can be queried to retrieve that information about those roles. We also do use this for some special group that our Hcm. People need to keep track of, anyway. So an example that I off the top of my head is who's in a HIPAA-related department? So are you in a HIPAA covered entity? @@ -62,7 +62,7 @@ A: I debated which way to point these arrows, but this is our SQL query, so that The third flow is just direct Grouper loader jobs that people saw from that group or a queries people. Probably the courses and course roles are the main use of these queries. -image::/images/uncLoader.jpg[Grouper loader jobs] +image::/images/uncLoader.jpg['Grouper loader job path', width=700] We do publish all of our courses out to Grouper each semester, and then break out the different roles inside the courses. So, student, faculty, primary diff --git a/recoveredDocToolRec.adoc b/recoveredDocToolRec.adoc new file mode 100644 index 0000000..ac94430 --- /dev/null +++ b/recoveredDocToolRec.adoc @@ -0,0 +1,55 @@ +=== docToolRec.adoc +:hide-uri-scheme: + +- - - +_2023-05-14 13:48:53 Draft documentation tool option comparisons_ + +==== I. Deliverables + +. A documentation entry system with source files under version control +. A process that creates a web accessible documentation site from the source repository +. A CI/CD pipeline that triggers the publication process on a push to the source repository + +==== II. Requirements + +What features of Confluence are used in our wiki content? Are there counterparts in the other options? + +==== III. Comparison of text entry and editing tools: Markdown vs Asciidoc + +[cols=3*,stripes=odd] +|=== +^| *FEATURE/ASPECT* ^|*MARKDOWN* ^| *ASCIIDOC* + +|Basic text entry, editing, and preview +|https://code.visualstudio.com/docs/languages/markdown[VS Code support for Markdown] +|https://marketplace.visualstudio.com/items?itemName=asciidoctor.asciidoctor-vscode[Asciidoc support for VS Code] + +| Distinguishing features +| Broad adoption, greater volume of published content, simple syntax for basic features, enhanced features available in various markdown flavors + +| Complex tables, asides, definition lists, includes, sidebars, table of contents, live snippets from source code, anchors, in-file metadata, attribute/variable support, ifdef/ifndef for conditional inclusion + +| Flavors and Standards +| Markdown (MD) plus varieties: Github Flavored Markdown (GFM), GitLab Flavored Markdown (GLFM, based on https://commonmark.org/[CommonMark]) +| Asciidoc specification is governed by the Asciidoc Working Group under Eclipse + +|Publication process +|Markdown -> Jekyll -> GitHub Pages + +Markdown -> Jekyll -> GitLab Pages + +Output: HTML, pdf, more via Pandoc +|Asciidoc -> https://antora.org/[Antora] -> GitLab Pages + +Asciidoc -> Antora -> GitLab Pages + +Output: HTML, pdf, Docbook, man pages, EPUB3, Reveal.js via Asciidoctor CLI + +Multi-repository documentation sites via Antora + +|CI/CD support +|Jekyll to GitHub Pages via Github Actions; To GitLab pages via https://docs.gitlab.com/ee/user/markdown.html[GitLab Flavored Markdown] (based on Kramdown) + +|Antora to GitLab Pages via .gitlab-ci.yml playbook; to GitHub pages via Github Actions with .nojekyll directive + +|Conversion between markdown and asciidoc +|https://github.com/asciidoctor/kramdown-asciidoc[markdown to asciidoc] +|https://github.com/opendevise/downdoc[asciidoc to markdown] + +| +|HTML, pdf, Docbook, man pages, EPUB3, Reveal.js via Asciidoctor CLI + +|=== diff --git a/siwg-2023.adoc b/siwg-2023.adoc new file mode 100644 index 0000000..72640bf --- /dev/null +++ b/siwg-2023.adoc @@ -0,0 +1,14 @@ +ongoing subbgroups: PS Int WG, midPoint UG, + +Using, reviewing, documenting new Grouper SQL-provisioning facility to redo Grouper/midPoint + +summaries of presentations in I2/InC into an easily searchable unified resource + +working with BTAA on their Provisioning Survey (summer-ish) + + + + + + + diff --git a/test-data-howto.adoc b/test-data-howto.adoc new file mode 100644 index 0000000..e31c31c --- /dev/null +++ b/test-data-howto.adoc @@ -0,0 +1,316 @@ +=== test-data-how-to.adoc +- - - +_2023-08-16 18:10:25 Importing a small sample csv SIS file_ + +*- CSV connector How-to: Bring a new resource into a Worbench instance -* + + +- (example modified from The Book, Chapter 8, HR section) + +- Start with the small csv resource definition from Chapter 8 of the Book + +- hr.csv contents: +``` +"empno","firstname","lastname" +"001","Alice","Anderson" +"002","Bob","Brown" +``` + +- Place the new comma-delimited hr.csv file in the proper place on the Workbench host + +- Replace {Workbench Directory} with the actual path to your Workbench folder + +``` +{Workbench Directory}/midpoint_server/container_files/mp-home/csv/hr.csv +``` + +- Add the new .csv data file to the volume bind section of the midpoint_server segment of the docker-compose.yml file + +``` + ... + midpoint_server: + ... + volumes: + - midpoint_home:/opt/midpoint/var + ... + - type: bind + source: ./midpoint_server/container_files/mp-home/csv/hr.csv + target: /opt/midpoint/csv/hr.csv + ... +``` + +--- + +*- The following template from Ch. 8 of THE BOOK can be used to create your own simple inbound csv resource definition -* + +``` + + + + + + HR System, Book ch 8 + + + HR resource using CSV connector. This is the HR feed (source) resource. + This is a sample used in the "Practical Identity Management with MidPoint" + book, chapter 8. + + + + + + c:connectorType + com.evolveum.polygon.connector.csv.CsvConnector + + + +``` +* In the connectorConfiguration section +** filepath should match the path defined in the docker-compose.yml for the target location of the bind element +** {filename.csv} should be your .csv file name +** uniqueAttribute should identify the field in the csv file that carries the unique identifier for each row + +``` + + + + /opt/midpoint/csv/{filename.csv} + utf-8 + , + ; + empno + + +``` +- NOTE: The section that goes here will be added automatically + when the first connection to the resource takes place + +- The schema handling section should use the attribute names that match the "element" definitions in the auto-loaded schema section of the resource definition + +- Near the top of the schema handling section, the element for the unique attribute should look like this + +``` + + + + Default Account + true + ri:AccountObjectClass + + ri:empno + Name (book hr empno) + + 0 + + true + true + true + + + + + $focus/employeeNumber + + + +``` +- Now the rest of the attribute mappings + +``` + + ri:firstname + First name + + + $focus/givenName + + + + + ri:lastname + Last name + + + $focus/familyName + + + + + + + + + + + + + weak + + + + + + + + + + + + none + + + + + + true +``` +- The correlation element should look like this: + +``` + + + employeeNumber + + $projection/attributes/empno + + + +``` + +- The rest of the synchronization element follows + +``` + + linked + true + + + deleted + true + + http://midpoint.evolveum.com/xml/ns/public/model/action-3#deleteFocus + + + + unlinked + true + + http://midpoint.evolveum.com/xml/ns/public/model/action-3#link + + + + unmatched + true + + http://midpoint.evolveum.com/xml/ns/public/model/action-3#addFocus + + + + + +``` + +*- The resource definition file is now ready to upload with midPoint Studio -* + +- - - +_2023-08-17 08:44:37 Import 50,000 users from source-sis-full.csv_ + +*- process for importing large csv files into fresh instance of midPoint -* + +Assumes you have already done this once and we start from that instance + +Save the existing edited docker-compose.yml with data file binds: + +/csp-tap/InCommonTAP-Examples-0816/Workbench/docker-compose.yml + +Bring the full test SIS population (50k) file into midPoint Workbench + +Save the contents of the ../csv directory + +``` +$ pwd +/csp-tap/InCommonTAP-Examples/Workbench/midpoint_server/container_files/mp-home/csv + +ls -la +-rw-r--r-- 1 csprootuser csprootuser 6826528 Aug 5 22:54 source-hrms.csv +-rw-r--r-- 1 csprootuser csprootuser 8386727 Aug 5 22:54 source-sis.csv +``` + +*- Wipe all existing Workbench images and files -* + +``` +docker stop $(docker ps -a -q); docker rm $(docker ps -a -q); docker rmi $(docker images -q) --force; docker volume rm $(docker volume ls -q) +``` + +Replace downloaded ../csv directory with saved version + +Replace repo version of docker-compose.yml with saved version + +*- Build and bring up the fresh Workbench -* + +``` +docker-compose up --build -d +``` + +- Create a new (midpoint) project in midPoint Studio and edit the default configuration to point to the new Workbench host + +*- import the full sis resource -* + +``` +*Schema mapping and extension attributes* + +CSV MIDPOINT USER +___ _____________ +sorid org +given givenName +surname familyName +email emailAddress +ph telephoneNumber +cntry-code ex-cntryCode +unid uid +occup title +dept orgUnit +refid ex-refid + +59614 (closed) Last object processed: 641-64-5552 + +8/17/23, 3:22:57 PM - + +8/17/23, 5:05:57 PM (01:42:59.164) +``` + +- The import task incorporates the synchronization process, so the sis records have created corresponding user records + +*- Next Steps -* + +Give all users the ldap-basic role which will provision all users into LDAP + +Have Grouper treat LDAP as its subject source and create HRMS and SIS Grouper groups with loader jobs + +Compare performance of LDAP subject source with a custom sql db table subject source + +Change groups and group memberships with Grouper and track the performance of the Grouper - midPoint connector + +---