Management of enterprise employees. Requires good RBAC, support for complex organizational structures and entitlements, excellent provisioning capabilities, reasonable reporting and governance
Management of enterprise customer identities. Requires scalability and good provisioning capabilities. Organizational structure and RBAC are much less important. Governance is usually only an obstacle here.
Use of IDM inside cloud service deployments, e.g. integrating applications in SaaS clouds or directly exposing functionality as IDaaS. Requires scalability. At least basic support for RBAC and organizational structure is also required. Multi-tenancy is critical.
Management of identities in the public sector. Usually a good support for organizational structures is required to model organizational structure of public agencies, hierarchy of regions/provinces for citizen identities, etc. Also reasonable support for RBAC, good authorizations and at least a basic governance is required. Public sector seems to be shifting to open source preference therefore a clean open source strategy is also important.
Management of Identities in the in Higher Education. Requires all types of identities: teachers, students, employees, visitors, researchers, collaborators, visitors etc., Usually support for very complex and parallel organizational structures is required. Ability for a parameterized membership in many organizational units is critical. As is the support for temporal conditions to limit student and visitor access) Clean open source strategy is also crucial.
Overall System Architecture
How good is the software architecture from the software engineering point of view. Is the system well divided into subsystems and components? Are there proper abstractions in place (such as interfaces)? Is the structure of the system appropriate and understandable?
Platform on which the system runs. E.G. specific operating system or hardware-independent platform
Framework (or other method) which is used to ‘wire’ the system together. Framework that binds the components together and forms the basic structure of the system.
What is this? Programming framework that was used to build GUI. This is crucial as the framework is very difficult to change. It usually means re-writing the entire GUI.
What is this? How easy is to use the system, how easy is to understand it. Is the system flooding user with information? Does it spread the information in a thousands of confusing tabs? Ergonomy, etc.
What is this? Does the user interface provide access to all functionality available in the system?
What is this? How quickly the GUI reacts to user actions. CustomizationWhat is this? How easily can be the GUI fuctionality be customized.
Role-Based Access Control (RBAC)
What is this? Ability to specify which accounts to create when a role is assigned to a user. Ability to define attribute values.
What is this? Ability to include one role in another role.
What is this? Ability to customize each role assignment with parameters. E.g. specify a tenant for which the assigned role applies). The assignment parameters are not part of role definition and neither they are part of user data. The parameters must be part of user-role relation (assignment).
What is this? Use parameters from user assignment or from a super role in the role expressions. E.g. parametrize the assignment of role assistant with an organizational unit or locality to which it applies.
What is this? Ability to "switch on and off" each role based on an arbitrary condition. Ability to assign temporal validity constraints (role valid from or to a specific date).
What is this? Roles that can be applied to roles themselves. E.g. ability to sort roles to groups or types (functional,business,IT,…) and specify the synchronization properties for each group using a unified policy (meta-role).
What is this? Assign a role owner who have more privileges over the role, e.g. ability to modify role definition.
What is this? Ability to guide the creation, modification and disposal of a role, e.g. using proper authorizations, workflow, approvals, etc.
What is this? Ability to create groups (or other objects) in the target systems as a reflection of a role. Also ability to create roles as a reflection of arbitrary resource objects.
What is this? Ability to support object that model organizational units such as companies, divisions, departments, projects, workgroups, teams, …
What is this? Ability to organize organizational units to a tree-like structures, ability to display them and efficiently browse them.
Parallel organizational structures
What is this? Ability to maintain several independent organizational structures. E.g. maintain functional organizational tree and a parallel flat project-oriented structure. Ability to assign the same user to each of them independently.
Organizational structure synchronization
What is this? Ability to create organizational units (or other objects) in the target systems as a reflection of organizational structure. Also the other way around. Ability to transform flat structures to tree structures, ability to reconstruct tree structure from flat string attributes, etc.
Provisioning and Synchronization
What is this? Ability to propagate data from the IDM system to the managed systems (resources).
What is this? Ability to synchronize data from managed systems to the IDM on an almost-real-time basis (delay in seconds).
What is this? Ability to compare data records in IDM and in the managed systems.
What is this? Ability of the IDM system to automatically trigger synchronization when needed. E.g. in case that an account is missing when IDM attempts to modify it, when existing account is present when a new account is being created, etc.
What is this? Ability to map attribute values between resource objects (object on managed systems) and the objects in the IDM system.
What is this? Ability to enforce uniqueness of attribute values (on managed systems) and to iteratively find a unique value, e.g. by trying identifiers in the form of jack001, jack002, …
Provisioning ordering and dependencies
What is this? Ability to enforce proper ordering of provisioning operations. E.g. if an application account depends on existence of operating system account. Also ability to properly pass attribute values between systems. E.g. create e-mail account first, pass the e-mail address value to user attribute, then create an AD account and properly set the e-mail address.
What is this? Notifications that announce success or failure of provisioning operations. Used mostly to deliver initial credentials and to notify system administrators about problems. Support for various channels (e-mail, SMS, …)
What is this? Ability of an IDM system to recover from provisioning failures such as timeouts and retries, compensation mechanisms, transactional guarantees, etc.
What is this? Support for management of entitlements on the resource side (in managed systems) such as LDAP groups, AD groups, privileges, ACLs, etc. Ability to display and synchronize them. Also ability to manage membership or association of accounts and entitlements.
What is this? Framework of mechanism used to manage and access provisioning connectors. LDAPWhat is this? Support for LDAP servers.
What is this? Support for Microsoft Active Directory.
What is this? Support for relational databases.
What is this? Connectors that can apply to many types of systems. Flat files, CSV, XML, scripting connectors, etc.
What is this? Connectors for UNIX-like systems such as Linux, Solaris, BSD, AIX, …
What is this? Connectors for HR systems such as SAP HR modules, PeopleSoft HRMS, …
ERP and business applications connectors
What is this? Connectors for ERP systems and various 'business' systems such as SAP ERP (R/3), Oracle applications, …
What is this? Connectors for cloud-based services such as SalesForce, Google apps, Office 365, …
Mainframe and mini connectors
What is this? Connectors for mainframe systems and 'minicomputers' such as z/OS, OS400, RACF, …
What is this? Can the connectors be used in other systems? Is there a support for legacy connector frameworks?
What is this? How easy is to develop a new connector.
What is this? Overall flexibility of the product: ability to change its behavior to satisfy the requirements.
Popular scripting languages
What is this? Support for other scripting languages.
What is this? Ability to extend existing object types with custom attributes. Ability to use the custom attribute in the same way as built-in attributes. Also ability of the attribute to be properly stored, indexed, displayed in forms, etc.
What is this? Ability to define new object types beyond those that are provided by default. Also ability for these new object types to behave as a first-class citizens.
What is this? Ability to synchronize any object with any other object.
What is this? Ability to place custom code to be executed at important points in request processing.
External interfaces (APIs)
Local native API
What is this? Local interface available in a primary language (e.g. Java). The goal is low overhead (local calls) and efficient development (e.g. use of callbacks, asynchronous invocation, etc.)
SOAP web service
What is this? Web service exposed by SOAP endpoint, WSDL definition, XSD schema, WS-Security support, etc.
What is this? RESTful resource-oriented interface with proper structure according to REST architectural style (Fielding) and WWW architecture.
What is this? A stand-alone component that can be linked to an application code and can be used to conveniently access the IDM system over the network.
Commercial relational databases
What is this? Ability to store data in commercial relational databases such as Oracle, Microsoft SQL Server, etc.
Opensource relational databases
What is this? Ability to store data in open source relational databases such as PostgreSQL, MariaDB, etc.
What is this? Ability to store data in NoSQL databases.
What is this? Ability for anonymous user to fill out a registration form which creates a user record. Also ability to control which fields are required, field validation, CAPTCHA, etc.
What is this? A dialog that allows user to change some of their own user profile details. Also ability to control which fields are displayed, which fields are editable, etc.
What is this? Ability for a user to change his own password (when the user still knows the old password). Also ability to select/filter resources, apply policies, etc.
What is this? Ability for a user to reset his own password when the old password is lost. Support for verification mail, security questions, etc.
What is this? Simple page that provides easily understandable information about user’s accounts, entitlements, group membership, etc.
What is this? Agents that capture cleartext passwords and sent them to IDM for distribution. E.g. agents for Active Directory, LDAP servers, etc.
Other self-service functionality
What is this? Flexibility of authentication mechanisms, integration with SSO systems, etc.
What is this? Ability to control who can do what. Overall authorization flexibility and architecture.
What is this? Ability to specify authorization policies on a fine granularity (e.g. on the attribute level)
What is this? Ability to delegate administrative tasks to specific user groups. E.g. ability to specify administrators for individual divisions, ability to delegate some functions to he call center, etc.
What is this? Ability to delegate privileges of one user to another user. E.g. allow one user to take all the responsibilities of another user during a vacation.
What is this? Ability to record all the operations of the users and the system down to a very fine details.
Workflow engine integration
What is this? How well is the workflow engine integrated into the system. Is it natural part of the system or was it added just as an afterthought? Are the workflow action items (such as approvals) reasonably integrated into the user interface?
Built-in approval workflow
What is this? Whether the product contains built-in or default approval workflow and what are the capabilities. Approval process is a usual part of IDM solutions and it is not entirely trivial to implement.
What is this? Can the workflow be customized? Can any type of custom workflow be plugged into the IDM processes?
What is this? Does the workflow support workflow standards (such as BPMN)?
Pluggable workflow engine
What is this? How easily can the default workflow engine be replaced? Can the product use a different engine? Or can it invoke remote workflow system instead?
Governance, risk assessment, compliance and forensic
Segregation of duties
What is this? Ability to exclude privileges or groups of privileges that cannot be assigned to the same identity at the same time.
What is this? Support for regular reviews and re-approvals of assigned privileges.
What is this? Support for automated analysis of privileges aiming at assisted design of RBAC structures. E.g. Role mining, role suggestions, etc.
What is this? Support for producing a well-formatted human-readable reports (e.g in HTML or PDF) that contain information from the IDM system and/or the resources. Also ability to easily configure custom report, modify the report design, etc. (Simple data export from a database is NOT considered to be reporting)
What is this? Support for storage of historical data and ability to analyze them. E.g. ability to report who had a particular role 6 moths ago.
Hardware resource efficiency
What is this? Systems that consume a lot of CPU, RAM or overload disks will have a low score here.
What is this? Whether the system actually works, all the time, reliably, without strange bugs.
What is this? Ability to work in clusters, geoclusters or other distributed configurations.
What is this? Ability to export all system data and import it to a different system. This is useful for configuration management, migrations (dev->test->prod), backup and restore, upgrades and variety of other reasons.
What is this? Ability to efficiently execute operations on a selected objects in a batch mode.
What is this? Ability to control what information is logged, ability to log debug and tracing information, whether the log messages are easy to understand, etc.
What is this? Documentation of architecture, subsystems, components, dependencies, modules, UML diagrams, …
What is this? Documentation describing system configuration, administration and customization
What is this? Documentation describing how the system is implemented, how to create plug-ins and other programming extensions, how to contribute to the project, etc.
Version control system
What is this? Where is the source code maintained? Is the history public? What are the technical obstacles to contribution?
What is this? Publicly shared information, e.g. in mailing lists, wiki, bugtracking, knowledge base, etc. Information that are only accessing for subscribers or behind a paywall are NOT considered to be community support.
What is this? Is project roadmap publicly available? Is product development planning transparent and predictable? Can roadmap be influenced by the community?
What is this? Is the code a product of a closed team in a single company or is it a group effort? How many independent groups or developers contribute to the project? This is a crucial aspect because the companies behind open source projects tend to be small and there is still a risk of failure. However if the project has a broad community it is very likely that the product development will continue even if the project founder fails.
What is this? How much is the project open to the public? Is the product design and architecture discussed in public? The the planning done in public? Is everything done in a clean and transparent open source way?