Ping Identity Data Governance Server Release Notes

Ping Identity Logo
  Return to Documentation Index

Notes for the following versions of the Ping Identity Data Governance Server are available in this document:

Critical Fixes

Updating to the latest version of the Data Governance Server addresses the following critical issues from previous versions. Affected servers should be updated.

Data Governance Server 7.0.1.4

Following are notes for version 7.0.1.4 of the Data Governance Server.

Resolved Issues

The following issues have been resolved with this release of the Data Governance Server:

No information is available

Data Governance Server 7.0.1.3

Resolved Issues

These issues were resolved with version 7.0.1.3 of the Data Governance Server:

  • Fixed a bug where the startIndex value for SCIM requests would be incorrect if the used LDAPSearch element had more than one baseDN defined in the scim-resources XML file. Issue:DS-38670 SF#:00643950

  • Critical: Fixed two issues in which the server could have exposed some clear-text passwords in files on the server filesystem.

    * When creating an encrypted backup of the alarms, alerts, configuration, encryption settings, schema, tasks, or trust store backends, the password used to generate the encryption key (which may have been obtained from an encryption settings definition) could have been inadvertently written into the backup descriptor. This problem does not affect local DB backends (like userRoot), the LDAP changelog backend, or the replication database.

    * When running certain command-line tools with an argument instructing the tool to read a password from a file, the password contained in that file could have been written into the server's tool invocation log instead of the path to that file. Affected tools include backup, create-initial-config, create-initial-proxy-config, dsreplication, enter-lockdown-mode, export-ldif, import-ldif, ldappasswordmodify, leave-lockdown-mode, manage-tasks, manage-topology, migrate-ldap-schema, parallel-update, prepare-endpoint-server, prepare-external-server, realtime-sync, rebuild-index, re-encode-entries, reload-http-connection-handler-certificates, reload-index, remove-defunct-server, restore, rotate-log, and stop-server. Other tools are not affected. Also note that this only includes passwords contained in files that were provided as command-line arguments; passwords included in the tools.properties file, or in a file referenced from tools.properties, would not have been exposed.

    In each of these cases, the files would have been written with permissions that make their contents only accessible to the system account used to run the server. Further, while administrative passwords may have been exposed in the tool invocation log, neither the passwords for regular users, nor any other data from their entries, should have been affected. We have introduced new automated tests to help ensure that such incidents do not occur in the future.

    We recommend changing any administrative passwords you fear may have been compromised as a result of this issue. If you are concerned that the passphrase for an encryption settings definition may have been exposed, then we recommend creating a new encryption settings definition that is preferred for all subsequent encryption operations, exporting your data to LDIF, and re-importing so that it will be encrypted with the new key. You also may wish to re-encrypt or destroy any existing backups, LDIF exports, or other data encrypted with a compromised key, and you may wish to sanitize or destroy any existing tool invocation log files that may contain clear-text passwords. Issues:DS-38897,DS-38908

Data Governance Server 7.0.1.0

Known Issues and Workarounds

These were known issues at the time of the release of version 7.0.1.0 of the Data Governance Server:

  • Servers to be monitored by the PingDataMetrics Server must have an instance name of less than 256 characters. A server's instance name is specified during setup. Issue:DS-36788

Resolved Issues

These issues were resolved with version 7.0.1.0 of the Data Governance Server:

  • Added a configuration option to allow a null serverFQDN for the GSSAPI SASL mechanism to allow an unbound SASL server connection. Issue:DS-36642 SF#:00637397

  • Updated the keys and values used in the monitoring JMX MBeans to conform with best practices. The keys "type" and "name" are now used in place of "Rdn1" and "Rdn2".

    To maintain backwards compatibility with existing monitoring solutions, installations upgrading to this release will retain the old behavior, but they can revert to the default behavior by changing the Global Configuration property jmx-use-legacy-mbean-names to false. Issue:DS-37235

Data Governance Server 7.0.0.0

Important Upgrade Considerations

Important upgrade considerations for version 7.0.0.0 of the Data Governance Server:

  • SCIM 2 error responses, including Config API error responses, now represent the "status" field as a JSON string rather than as a number. Clients written to expect the earlier version format will need to be updated. In particular, clients written using the SCIM 2 SDK for Java should upgrade to version 2.2.0 or higher.

  • After updating the first server in a multi-server Data Governance topology, "object class violation" errors will be logged in the errors log of the other servers. These errors do not affect any functionality and may be safely ignored. As each server is updated to 7.0, these messages will no longer be logged.

  • The Administrative Console now uses server information found in the topology registry to populate its server selection control. If the Console is used to manage a legacy server that does not use the topology registry, then the server selection control will not be populated. To manage a different server, the administrator will need to log out of the Console and provide the other server's connection details from the login page.

New Features

These features were added for version 7.0.0.0 of the Data Governance Server:

  • Simplified management tasks related to configuring servers in a large cluster topology or in an automated deployment. Most notably, servers can now be added to a cluster while other servers are offline.

  • Added management features for SSL/TLS certificates. The default certificates used in inter-server replication can be replaced; validation of client certificates for HTTPS-based services like the SCIM REST API can be configured; and you can reload from the trust store for HTTPS client certificates without restarting the server or the HTTP-based services.

  • Added support for these operating system versions: Ubuntu LTS 16.04, CentOS 7.4, RedHat Linux 7.4, SUSE Enterprise 12 SP3

  • Added API and policy to support storing and retrieving OpenBanking consents. Now an OpenBanking Account Servicing Payment Service Provider (ASPSP) can more easily support storing and retrieving account holder consents to sharing banking data. Consent records are persisted using the new PingDirectory consent API.

Known Issues and Workarounds

These were known issues at the time of the release of version 7.0.0.0 of the Data Governance Server:

  • Simultaneous cloning multiple PingDirectory Proxy, PingData Sync, and PingData Governance Servers from another server of the same type is not currently possible. To create server instances that are identical to a master server, cloning must be performed one at a time.

  • When updating a two-server Data Governance topology, a mirrored-subtree-manager-no-master-found alarm will be raised on each server while updating the other. The alarm will go away when both servers are updated, and does not affect any functionality. The alarm can be safely ignored.

Resolved Issues

These issues were resolved with version 7.0.0.0 of the Data Governance Server:

  • Support for the IBM JDK has been retired. Issue:DS-35536

  • Updated the JMX connection handler's monitor provider so that when a JMX connection is closed, it is removed from the list of established connections. After a JMX client disconnects, it may take the server a few minutes to detect the closure and update the monitor. Issue:DS-35576

  • The admin backend and the tool used to manage it, dsframework, have been replaced by the topology registry and dsconfig, respectively. The topology registry is automatically mirrored across all servers in the topology, so administrative information is kept in-sync on all servers at all times. Issues:DS-14281,DS-14282,DS-14283,DS-14284,DS-17197,DS-17366,DS-4570

  • Added a new manage-certificates tool that can be used to perform a number of functions related to TLS certificate management. Issue:DS-17891

  • Added a new Monitor Entry for SSL Cipher Suite and Protocol information. It is available under cn=SSL Context,cn=monitor. Issue:DS-35601

  • Updated the server to include an instance of the Periodic Stats Logger Plugin that is enabled out-of-the-box to aid in diagnosing support issues. The "Historical Stats Logger" plugin will log performance statistics to logs/monitor-history/historical-dsstats.csv every five minutes. This works in concert with the "Monitor History" plugin, which logs the full contents of cn=monitor to logs/monitor-history every five minutes. The tail of this csv file is automatically included in the output generated by collect-support-data. Issue:DS-35581

  • Added a default Consume Admin Alerts health check, which monitors the availability of a Ping Directory Server or Directory Proxy Server through administrative alerts. Also, updated the prepare-external-store tool to grant the Data Governance service account read-only access to the cn=alerts and cn=monitor backends of these external server types. This guards against cases in which the existing Get Root DSE health check may not be sufficient to detect the unavailability of external LDAP servers. Issue:DS-35949

  • Added the ability to configure data encryption during setup using a randomly generated key, a key generated from a user-supplied passphrase, or a key obtained from an export of another server's encryption settings database. When setting up multiple instances, providing the same encryption passphrase to each instance will ensure that all instances have the same encryption key.

    The encryption-settings tool has also been updated to allow creating encryption settings definitions from a passphrase, to allow providing a description when creating a new encryption settings definition, and to record a create timestamp for new definitions. It is now possible to create ciphers that use the Galois Counter Mode (GCM) cipher mode (for example, using a cipher transformation of "AES/GCM/PKCS5Padding") for authenticated encryption. Definitions created with with just a cipher algorithm but no transformation will now use stronger settings.

    The default encryption settings export format now provides stronger encryption. Newer server instances should be able to import encryption settings exported from other servers without issue. When exporting encryption settings for import into older servers, use the new --use-legacy-export-format argument. Issues:DS-15223,DS-35895

  • The create-systemd-script command now suggests placing the script created in "/etc/systemd/system." Issue:DS-35868

  • Added an encrypt-file tool that can encrypt and decrypt data with a user-supplied passphrase, an encryption settings definition, or a topology key shared among server instances. It includes support for decrypting the content in encrypted backups, LDIF exports, and log files. Issue:DS-36054

  • Fixed an issue with compressed logging that could leave some data buffered in memory and not actually written out to disk until the logger is closed. Issue:DS-36070 SF#:00628238

  • Added support for encrypted logging, using a key generated from an encryption settings definition. Encrypted log files may be decrypted with the encrypt-file tool. Issue:DS-6970

  • Made a number of improvements to backend backup and restore, and to LDIF export and import:

    * Added the ability to encrypt backups and LDIF exports with a key generated from a user-supplied passphrase or with a key generated from an encryption settings definition. Previously, encrypted backups and LDIF exports only used a secret key that was known only to servers within the replication topology. The new options make it easier to restore encrypted backups and import encrypted LDIF files in servers outside of the replication topology. The encrypt-file utility can be used to decrypt encrypted backups and LDIF exports, regardless of how the encryption key was obtained.

    * Added the ability to limit the rate at which backups and LDIF exports will be written to disk, which can help avoid performance problems that result from these operations saturating the disk subsystem.

    * Added new global configuration properties for automatically encrypting backups and LDIF exports by default, which will be set to true if data encryption is enabled during setup.

    * Added new global configuration properties that can specify which encryption settings definitions will be used to obtain the encryption keys for automatically encrypted backups and LDIF exports. If not specified, then the server will use its preferred encryption settings definition, or an internal topology key if no encryption settings definitions are available.

    * Added a new configuration property for automatically compressing encrypted LDIF exports.

    * Updated the backup tool to add new --promptForEncryptionPassphrase, --encryptionPassphraseFile, and --encryptionSettingsDefinitionID arguments that can be used to specify which key to use for encrypting the backup. Added a new --doNotEncrypt argument that can be used to force a backup to be unencrypted even if automatic encryption is enabled. Added a new --maxMegabytesPerSecond argument that can be used to impose a limit on the rate at which the backup may be written to disk.

    * Updated the restore tool to add new --promptForEncryptionPassphrase and --encryptionPassphraseFile arguments that can be used to provide a user-supplied passphrase for use in accessing the contents of an encrypted backup. For backups encrypted with an encryption settings definition or an internal topology key, the server will automatically be able to determine the correct key.

    * Updated the export-ldif tool to add new --promptForEncryptionPassphrase, --encryptionPassphraseFile, and --encryptionSettingsDefinitionID arguments that can be used to specify which key to use for encrypting the export. Added a new --doNotEncrypt argument that can be used to force an LDIF export to be unencrypted even if automatic encryption is enabled. Added a new --maxMegabytesPerSecond argument that can be used to impose a limit on the rate at which the LDIF file may be written to disk.

    * Updated the import-ldif tool to add new --promptForEncryptionPasshprase and --encryptionPassphraseFile arguments that can be used to provide a user-supplied passphrase for use in accessing the contents of an encrypted LDIF export. The --isEncrypted and --isCompressed arguments are no longer necessary, as the tool can automatically detect encryption and compression (although those arguments are still available to preserve backward] compatibility), and it can automatically identify the correct key for exports encrypted with a key obtained from an encryption settings definition or an internal topology key. Issues:DS-12157,DS-35896 SF#:3628

  • Updated setup to include key usage, extended key usage, and subject alternative name extensions in the self-signed certificates that it generates. Issues:DS-35727,DS-35728

  • Implemented invocation logging for several server tools, which will write to logs/tools/tool-invocation.log by default upon startup and shutdown. Some of the information recorded by log entries include the tool's start and completion times, the command-line arguments used to initialize them, and the name of the system account used to launch the tool. To modify this behavior, edit the config/tool-invocation-logging.properties file. Issue:DS-4406

  • Updated tools that interact with log or LDIF files to support reading from input files that are compressed and encrypted and writing to compressed and encrypted output files. Issue:DS-36075

  • Added the ability to generate administrative alert notifications when a task starts running, when it completes successfully, or when it fails to complete successfully. Also added the ability to send an email message to a specified set of users when a task starts running or completes successfully, which complements the existing ability to send an email message when a task fails to complete successfully or when it completes with any state, regardless of success or failure. Issue:DS-426

  • Provided the means to request that the server dynamically reload the certificate key and trust stores used by all HTTP connection handler instances that provide support for HTTPS. The request can be made using a new reload HTTP connection handler certificates task, the reload-http-connection-handler-certificates tool, or programmatically from a Server SDK extension using the ServerContext#reloadHTTPConnectionHandlerCertificates method. Issue:DS-35990 SF#:00629638

  • The update tool now enforces specification of a new product license when updating to a new major version. The license can be specified using the --licenseKeyFile command-line options, or by copying the license file to the top-level directory of the server package used to perform the update. Request a license key through the Ping Identity licensing website https://www.pingidentity.com/en/account/request-license-key.html, or contact sales@pingidentity.com. Issue:DS-35523

  • In addition to specifying an exact set of desired cipher suites for the LDAP and HTTP Connection Handlers, administrators can now specify inclusions to, or exclusions from, the set of cipher suites selected by the server. Issue:DS-36088

  • Added support for recurring tasks, which can be used to automatically invoke certain kinds of administrative tasks based on a specified schedule.

    At present, only certain kinds of tasks can be scheduled as recurring tasks. This includes both backups and LDIF exports, each of which provides retention support to limit the amount of disk space that the backups and LDIF files consume. It also includes support for any kind of task in which each instance of the task should use exactly the same values for all of the task-specific attributes. The Server SDK also provides an API for creating custom third-party recurring task implementations. Issue:DS-426

  • Updated the server to reduce contention when converting between strings and the bytes that comprise those strings. Issue:DS-36328 SF#:626850

  • Added a sanitize option to the Monitor History Plugin that, if enabled, will redact the small amount of potentially personally identifiable information that could appear in search filters and LDAP DNs within the monitor. This makes it easier to share the monitor history files with the support team in secure environments. Issue:DS-36545

  • Increased the default size of the queue used to hold alert notifications so they can be asynchronously processed by a background thread. This makes it less likely that the queue will become full if many alerts are generated in a short period of time, which would cause subsequent attempts to generate alerts to block while the server catches up. Also updated the server to log a message when the queue becomes full so that administrators will be aware of the problem and will have suggestions for addressing it. Issue:DS-36360 SF#:635134

  • Updated the dsconfig list subcommands to list objects of all complexity levels rather than requiring the --advanced flag to list advanced and expert objects. Issue:DS-16508

Data Governance Server 6.2.0.0

Resolved Issues

These issues were resolved with version 6.2.0.0 of the Data Governance Server:

  • Added a disabled-alert-type configuration property to the Alert Backend that can be used to suppress specific alert types from being added to the backend. Issue:DS-16906 SF#:3556

  • Fixed an issue where a SCIM PUT request that modified a multi-valued, complex attribute within an extension schema could fail with a 400 error code. Issue:DS-17006

  • The server now requires Java version 8. Issue:DS-17019

  • The SCIM interface now supports searches that filter on "id." Issue:DS-16997

  • Changed the format of encoded URLs for the auth UI to reduce their size. As a result, different versions of the Data Governance Broker (6.0.0.0 and 6.0.0.x) cannot co-exist in the same cluster as they will not be able to decode URLs generated by each other. Issue:DS-16614

  • Previously, the Data Governance Broker would reject access tokens containing a scope whose name was not known. The Data Governance Broker will now ignore the unknown scope, but may still accept the token and honor other scopes granted by that token. Issue:DS-16993

  • The authentication API now returns authenticator specific status and errors for all authenticators in the authentication chain with the following new response parameters.

    The behavior for the following authenticators has also been altered:

    ◾TOTP Authenticator no longer returns the "ready" response parameter. Clients should use the new "status" response parameter. ◾Account Lookup Authenticator no longer returns the "accountFound" response parameter. Clients should use the new "status" response parameter. The authenticator will now return the lookup parameters as submitted in the request. This will help in multi-step flows where the client must submit the same lookup parameters again. ◾reCAPTCHA Authenticator no longer returns the "validated" response parameter. Clients should use the new "status" response parameter. The authenticator will now return the recaptcha resonse as submitted in the request. This will help in multi-step flows where the client must submit the same recaptcha response again.

    Clients relying on any of the authenticator-specific response parameters will need to be updated to maintain compatibility. Issues:DS-16730,DS-16737

  • Fixed an issue with the Ping Data Governance Broker API Explorer using Safari. The Access Token feature will now correctly be set to "ON" after successfully authorizing a user. Issue:DS-16988

  • Authentication chain processing messages are now categorized under the authentication message type in the Trace Logger Publisher configuration. They are now enabled by default for the File-Based Trace Logger configuration. Issue:DS-17040

  • Creation of a Store Adapter Mapping no longer requires a case exact match of the store-adapter-attribute name with the attribute in the store adapter schema. Issue:DS-17142

  • A new XACML obligation type, 'add-filter', was added. This obligation allows a policy writer to pre-filter SCIM search results by requiring additional filters to be added to incoming SCIM requests. Issue:DS-17123

  • A single Authenticated Identity Scope may now be used to authorize access to one or more SCIM sub-resource types. The properties exposed in the scopes JSON object accessible under the applicable scopes XACML attribute category has changed slightly. The "scimSubResourceType" property is new replaced by the "scimSubResourceTypes" property, whose value is an array of strings. Issue:DS-16233

  • Updated the logic used to select which TLS cipher suites should be enabled by default, and the logic used to prioritize those cipher suites. The selection process has been updated to use the guidelines provided in the OWASP "Transport Layer Protection Cheat Sheet" document.

    Some of the changes include:

    - The server already preferred cipher suites that support forward secrecy over those that don't. It now prefers DHE over ECDHE, and avoids suites that use non-RSA keys.

    - The server already avoided cipher suites that used known-weak cryptographic weaknesses, including null encryption, the RC4 symmetric cipher, and the MD5 digest algorithm. It now also avoids anonymous encryption, the single-DES symmetric cipher, the IDEA symmetric cipher, and any suite using export-level encryption.

    - The server now prefers cipher suites that use the Galois/Counter Mode (GCM) over the Cipher Block Chaining (CBC) mode.

    - The server now prefers AES-based cipher suites with 256-bit keys over those that use 128-bit keys. For suites with equivalent key sizes, it prefers suites with a stronger message digest algorithm over suites with a weaker digest algorithm (e.g., SHA384 over SHA256 over SHA).

    - The server now provides better support for selecting and prioritizing ciphers when running on the IBM JVM. The IBM JVM uses somewhat different naming for its cipher suites than the Oracle implementation, which previously allowed certain desirable suites to not be included in the selected set. Issue:DS-17146

  • The Auth REST API no longer supports the ability for client applications to specify the message text of validation or authentication messages sent to end-users using email, text, or voice. Rather those messages are now specified by a Data Governance Broker administrator. The client application must now specify the authenticator parameter "codeRequested" with a value of "true" in order to have the Telephony or Email Delivered Code Authenticators send a new code to the end-user. Issue:DS-16678

  • SCIM resource schemas previously did not allow sub-attributes to be multi-valued. This is now permitted. Issue:DS-13954

  • Two new XACML attribute categories have been added which give policies access to the attributes of the current access token owner and session owner, respectively. This information was formerly available in the "subResource" attribute of the access-token and session XACML categories. The subResource attribute has been deprecated in both cases. Issue:DS-17178

  • SCIM PUT requests will now replace the value of a caseExact attribute whose only change is to case. Issue:DS-17304

  • HTTP TRACE requests have been disabled, and will now return an HTTP status code of 405 Method not allowed. Issue:DS-17298

  • Fixed an issue where incorrect names were displayed in the usage for the start scripts. Issue:DS-16593

  • Removed the default root password from the out-of-the-box configuration. This password was never actually used because it was replaced by the user-supplied password provided when running setup, and it has been removed for additional security. Issue:DS-17318

  • Updated the installer to discourage the use of weak root passwords.

    When run in interactive mode, setup will display a list of password quality recommendations before prompting for the initial root password, suggesting that it should be at least 12 characters long, should not be contained in a dictionary of English words, and should not be contained in a dictionary of commonly-used passwords. If the proposed password does not meet these constraints, then the user will be given the option of proceeding with the provided weak password or choosing a different password.

    When run in non-interactive mode, setup will exit with an error if the proposed initial root password does not satisfy the above constraints, unless the command line also includes the --allowWeakRootUserPassword argument.

    In either mode, when a strong initial root password is supplied, setup will also configure the root users' password policy to ensure that subsequent root user passwords will also be required to satisfy these constraints. Issue:DS-2074

  • Updated the access and audit loggers so that, when logging information about an internal operation that was triggered by an external client request, the log message will include the connection and operation ID for that request. Also updated the error logger so that when logging a message from a thread that is actively processing an operation, the log message will include the connection and operation ID for that operation. Issue:DS-16509 SF#:3536

  • Replaced the ldapsearch and ldapmodify tools with new versions. The new versions are backward-compatible, but offer a number of new features, including better connection handling, better output formatting, better support for bulk operations, support for referrals, support for additional request and response controls, and rate limiting. The ldapsearch tool now offers the ability to output results in JSON, CSV, or tab-delimited text as an alternative to LDIF, and provides support for a number of data transformations. The ldapmodify tool now supports the LDIF control syntax, as well as writing to output and reject files. Issues:DS-15861,DS-15862

  • Improved error reporting for the manage-extensions tool. Issue:DS-17080

  • The modifierName and modifyTimestamp attributes are now updated when offline configuration changes are made. Issue:DS-16858

  • Corrected the port number returned in the error message that is displayed when an administrator is trying to set up a server that is already running. Issue:DS-13721

  • The server now monitors important certificates used for client and inter-server communication. Certificate information is available in the Administrative Console and in the status tool output. An alarm is raised and alerts are sent when a monitored certificate is 30 days from expiration. Issue:DS-1029

  • The Administrative Console can be deployed in an external web container, such as Tomcat, using the contents of resource/admin-console.zip, located in the server root. Issue:DS-17544

  • Migration from previous releases is not supported. Issue:DS-17617

  • Added an optional reason parameter for dsconfig changes that will be automatically included in the server's config-audit.log file. Issue:DS-811

  • Updated the Server SDK to provide methods for obtaining a single LDAP connection or an LDAP connection pool with connections established to a specified LDAP external server defined in the server configuration.

    Also updated the server configuration to add support for obscured values. An obscured value is a general-purpose string that is stored in an obscured form in the configuration so that its plaintext value is not readily discernible to anyone looking at the configuration file and so that the value is not displayed in administrative interfaces. The Server SDK provides a method for obtaining the plaintext representation of an obscured value, and this mechanism can be used to store potentially sensitive values in the configuration for use in Server SDK extensions without the need to store those values in the clear. Issue:DS-10694

  • The Data Governance Server now supports the ability to configure multiple attribute pairs for correlation between a primary and second Store Adapter. The values of all specified attribute pairs must be equal in order for the data from the two Store Adapters to be correlated into a single SCIM resource. Issue:DS-17189

  • The Administrative Console is no longer compatible with older versions of the server. Issue:DS-17241

  • Updated the server to reduce the use of the SHA-1 message digest. The server will now use a 256-bit SHA-2 digest instead of a SHA-1 digest in all of the following cases:

    - When hashing or signing a backup. - When signing an LDIF export. - When signing log data. - When generating MACs for an encrypted collect-support-data archive. - When generating unique identifiers for encryption settings definitions. - When determining whether the configuration changed with the server offline.

    In all of the above cases, the server includes metadata in the output of the cryptographic processing to indicate the digest or MAC algorithm used for that processing, which ensures that the output remains compatible across server versions. For example, an LDIF export that uses a signature generated with the SHA-2 digest can be successfully imported into older versions of the server.

    Also, the fingerprint certificate mapper has been updated so that it can use the 256-bit SHA-2 digest when mapping a client certificate to the corresponding user entry. The previous MD5 and SHA-1 digests remain supported.

    Finally, the example enhanced password storage scheme provided with the UnboundID Server SDK has been updated so that it uses the 256-bit SHA-2 digest instead of a SHA-1 digest. Issue:DS-17444

  • The script files used to stop and start the server have been renamed stop-server and start-server. The older scripts are still present but may be removed in a future release of the product. Issue:DS-16789

  • The RequestContext class used by Policy Information Provider (PIP) extensions now provides a getJsonContent(String categoryId) method, allowing a PIP to access data handled by another PIP. Issue:DS-17761

  • Limited the ACI search on collect support data tool to only pull 100 entries. This will reduce the time the tool takes to run for organizations with a large number of ACIs. Issue:DS-17968

  • Added configuration options for setting the SSL Protocol and/or the SSL Cipher Suites to the HTTPS Connection Handler. Issue:DS-10748 SF#:00003622,614777

  • Enhanced the HTTPS Connection Handler to send a HTTP Strict Transport Security header by default in all responses. Issue:DS-14650

  • Updated PingDirectory, PingDirectoryProxy, PingDataSync, and PingDataGovernance with the capability to run as Windows Services. Issue:DS-4161

  • Fixed an issue in which the SCIM 2 schema checker could incorrectly reject modify operations involving binary attributes. Issue:DS-18091

  • Added support for the X-Forwarded-Prefix header to override the context path of operations processed by Http Servlet Extensions. Issue:DS-18016

  • Updated dsconfig batch mode to operate more efficiently over the WAN by consolidating the number of LDAP searches required to retrieve the full configuration when pre-validating configuration changes. Issue:DS-35495

  • A license key is required when setting up a server for the first time. Request a license key through the Ping Identity licensing website https://www.pingidentity.com/en/account/request-license-key.html or contact sales@pingidentity.com. Issue:DS-18100

  • Removed the ability to create custom HTTP trace loggers using the Server SDK. Issue:DS-18188

Data Governance Server 6.0.0.0

Important Upgrade Considerations

Important upgrade considerations for version 6.0.0.0 of the Data Governance Server:

  • This release of the Data Governance Broker contains numerous backwards-incompatible changes and cannot be upgraded from a previous release of the Data Broker. For migration assistance with a previous Data Broker version, contact Ping Identity Professional Services.

  • Note: The product names have been updated to reflect the UnboundID acquisition by Ping Identity. This is a naming and branding change only; the code base is the same as in prior releases and will continue to be maintained into the future.

  • The 6.0 release makes these changes to supported platforms:

    • CentOS 7.2 and RedHat 7.2 are now supported operating system versions.
    • SUSE 11 SP4 is now supported; SUSE 11 SP2 support has been retired.
    • Linux KVM and VMWare ESXi 6.x have been added as supported virtual machine environments.
    • Deprecation of JDKs 7.x. Customers are strongly advised to use JDK 8 with this release. A later release of the platform will remove support for JDK versions 7.x. At that time customers will be required to upgrade to JDK 8.x when upgrading servers. This will apply to all of our JDK flavors (OpenJDK, OracleJDK, IBM JDK) on all platforms.
    • WildFly 9.x (renamed from JBoss) is now a supported application server for all web applications, including the Administrative Console and sample applications. Support for JBoss 7.x has been retired. Tomcat support remains unchanged in this release.

  • The ID Token grant type has been removed and the resource owner grant type can no longer be used to obtain ID tokens.

  • The passwordQualityRequirement endpoint was renamed to passwordQualityRequirements (plural).

  • The resource type-level password quality requirements endpoint has been removed from the server.

New Features

These features were added for version 6.0.0.0 of the Data Governance Server:

  • Added support for two factor authentication supporting email, SMS, or TOTP authenticators. The two factor authentication feature is dynamically driven by configuration, policy, and user profiles.

  • Authenticators have been reworked to provide more possible authentication flows such as username/password, account recovery, account registration/verification, external identity provider login, email verification code delivery, SMS verification code delivery, time-based one-time password (TOTP) delivery, and reCAPTCHA validation. Custom authenticators can be created using the UnboundID Server SDK.

  • The token validation endpoint now ignores access tokens passed as query parameters, and only accepts access tokens passed as form parameters. Clients should avoid sending sensitive values as query parameters, because those values may be logged as part of the request URL.

  • Authentication context classes define specific security requirements for varying degrees of assurance or differentiated experiences for each OAuth2 client. These context classes can be assigned to scopes, meeting the requirements for specific authentication events prior to the scope being granted.

  • Added the Authentication Chain configuration object to build authentication flows that allow for two factor authentication, account registration, social login, and other authentication types.

  • Added an Authentication API to enable external applications to build local or native user experiences, while using a centralized authentication service. The API provides services for login and registration, second factor authentication, account recovery, and management of user's consent.

  • Account registration can now require users to verify their accounts either with a code sent over email or SMS.

  • Sessions can now be controlled through settings in the Authentication Service configuration and in each authentication context class. These ultimately determine how often a session is required to reauthenticate and can include login expiration intervals, second factor expiration intervals, and maximum session cookie lifetime.

  • Access tokens are now encoded as JWTs signed using a RSA private/public key pair. Signed tokens allow for external applications to verify the authenticity of the token’s claims. The key pair and algorithm specified by the new access-token-signing-key-pair and access-token-signing-algorithm properties, respectively, will be used to sign the access tokens. For verification, the algorithm and key pair originally used to sign the token will be used. Once a key pair configuration is deleted, any access tokens originally signed with that key pair will no longer be valid.

  • Access Token Validators are used to validate access tokens submitted by client applications for access to protected resources of the Data Governance Broker. An Access Token Validator is responsible for decoding an incoming access token and returning token metadata that is similar in content to that specified by RFC 7662, including whether the token is valid and what scopes are granted to the token. The default Access Token Validator validates access tokens that are issued by the Data Governance Broker itself. Access Token Validator extensions can be configured to allow the Data Governance Broker to accept access tokens issued by other identity providers.

  • A new external identity provider type has been added to the Data Governance Broker to accept SAML assertions from SAML IDP for the purposes of single sign-on. Currently only SAML providers which provide artifact resolution are supported.

  • For Java developers whose tools and workflows make use of Maven, the Server SDK jar has been deployed to Maven Central so that a developer can now add the Server SDK as a project dependency by adding a few lines to a project's pom.xml. Also, developers can now generate a Server SDK project that Maven-aware IDEs such as IntelliJ IDEA can package into an extension bundle with no special configuration needed. This benefit extends similarly to continuous integration systems such as Jenkins.

  • The dsconfig tool provides the ability to search for and quickly navigate to configuration objects and properties in which the name, synopsis, or description matches a provided pattern.

  • A new rotate-log tool and task have been added, which can be used to trigger rotation of one or more log files.

  • The Configuration API is now fully supported for all servers. In this release, the API was changed to match SCIM conventions for attribute naming, resource modeling, and the standard HTTP verbs. The UnboundID SCIM 2 SDK (available through GitHub) can now be used with the Configuration API.

  • All servers have an updated web Administrative Console, which includes:

    • New layouts for operational statistics, processing time, queues, all monitors, and the list of installed extensions.
    • Alert and alarm displays, summarizing the data in cn=alerts and cn=alarms and based on the configured gauges. Plus, filtering and searching for these.
    • A new LDAP Schema Editor for importing schema files, validity checking, creation and editing of object classes and attribute types. The editor also supports viewing of the attribute syntaxes, inheritance, and indexes that exist for each attribute and the dependencies between object classes and attributes.

  • The new Administrative Console can also be deployed to independent application servers instead of being co-hosted by the servers. This simplifies deployment models and increases separation between data and application layers.

  • To assist with situations where a very large number of changes may cause disk, memory, and server start time to increase unexpectedly, alerting and gauge features have been added to the Recent Changes Database.

  • Servers can now trigger events whenever log file rotation occurs. This includes "copy on rotate" and "summarize on rotate" listeners, as well as Server SDK support for creating custom log file rotation listeners.

  • It is now possible to create, change, and remove root user accounts across the topology using the dsconfig tool and Administrative Console.

  • A developer documentation site including a client developer guide, sample code, and API references is now available at https://developer.unboundid.com.

Known Issues and Workarounds

These were known issues at the time of the release of version 6.0.0.0 of the Data Governance Server:

  • When deploying the Administrative Console in Tomcat 8, and accessing the Administrative Console application using Tomcat's Web Application Manager, some browsers (including Safari and Firefox) will generate a path URL that encodes the dash in ubid-console. This results in a path such as http://localhost:8888/ubid%2Dconsole/, which causes session management errors. To workaround this issue, copy and paste the generated link into a browser, and replace the encoded dash with a dash (-) character.

  • The PermSize and MaxPermSize JVM properties are no longer supported in JDK 8, and will safely be ignored. These properties can be removed by modifying the config/java.properties file and running "bin/dsjavaproperties" while the server is offline.

  • Security criteria for root passwords with the default configuration will be increased in a future release. This might affect automated installation scripts that currently use less secure passwords. This will not affect existing root accounts.

  • The dsconfig tool and the Administrative Console enables creating and managing new Root DN users in this release. However, there is a limitation with changing the password of the currently logged in administrator. The ldappasswordmodify command can be used to change the administrator's password by providing the current and new password.

  • Invoking single log out using the OIDC logout endpoint revokes all access tokens rather than tokens associated with the user's session.

  • Deleting a session using the Session SCIM sub-resource does not revoke any tokens or codes associated with that session.

  • Username and Password recovery with a phone number is affected by a limitation present in the indexing of phoneNumbers.value as plain strings, as opposed to a telephone number format that ignores spaces and dashes. Any SCIM filter referencing phoneNumbers.value will require an exact string match with the values stored in the Directory Server.

  • The prepare-external-store and create-initial-broker-config commands no longer create the user store base DN entry on external LDAP stores. Deployers should create this entry if needed.

  • Users may be unable to change their password using the example client, unless all Directory Proxy Servers between the Data Governance Broker and external server are configured with an additional ACI. See resource/starter-schemas/README.txt for instructions.

Resolved Issues

These issues were resolved with version 6.0.0.0 of the Data Governance Server:

  • Updated the Data Governance Broker's sample application dsconfig batch scripts to specify an --applyChangeTo option necessary for Data Governance Broker instances that are part of a configuration server group. Issues:DS-14691,DS-14692

  • Fixed an issue where generation of a client secret for an OAuth2 Client in a dsconfig command could occasionally fail with an error message containing "IllegalBlockSizeException: Input length must be multiple of 8." Issue:DS-16549

  • The trace logger now recognizes requests of type application/x-www-form-urlencoded and correctly logs the request body. Issue:DS-16842

  • Updated command-line tools based on the LDAP SDK tool APIs to add the following features:

    • Tools can obtain default values for any arguments not provided on the command line from a properties file. If it exists, the server's config/tools.properties file will be used by default. Command-line arguments can be used to specify an alternate properties file or to indicate that no properties file should be used.

    • Tools can be launched in an interactive mode, in which the user is prompted for arguments used to establish and authenticate the connection, and for any other required arguments. The user can then use an interactive menu to specify values for any remaining arguments.
    Issue:DS-13700

  • Added a --prettyPrint option to the config-diff tool to make the output more human-readable. Issue:DS-14694

  • Added support for account usability information (notices, warnings, and errors) and account activation time to the account endpoint. Issue:DS-14328

  • Fixed an issue with the dsjavaproperties tool where java properties for PermSize and MaxPermSize could be added when using JDK 8, which no longer supports these options. Issue:DS-14857 SF#:3187

  • Updated the bcrypt, crypt, PBKDF2 and scrypt password storage schemes so they can be used to create new instances. Issue:DS-14923

  • Updated the Apache commons collections library to address the security vulnerability described by CVE 2015-4852. Issue:DS-14430 SF#:00003216

  • Fixed a case where attribute syntax configuration changes would not apply to undefined attributes, which rely on default attribute types. Issue:DS-14979

  • Removed SCIM endpoints for one-time passwords, password reset tokens, and delivery mechanisms. This only affects the Password Policy Management REST API, and will not affect the password reset functionality of client applications. Issue:DS-14962

  • Collect-support-data tool now captures Kerberos config and log information. Issue:DS-13823

  • Updated the server's support for the Twilio Messaging Service so that it uses the newer "Messages" API when sending SMS messages instead of the older "SMS" API. The older API has been deprecated, and Twilio now imposes a 120-character limit for messages sent via that API. The messages API allows the server to take advantage of the full 160 characters per SMS message. Issue:DS-14749

  • The --appsHttpsPort option has been removed from the setup tool. By default, the Administrative Console is hosted on the same port as the rest of the Data Governance APIs. The Administrative Console can be moved to a different port after setup by editing the HTTPS Connection Handler. Issues:DS-11247,DS-14988,DS-14990

  • Server SDK extensions are now built with a Java source version of 1.7 by default. Issue:DS-15015

  • SCIM Resource Type operations can now retrieve SCIM resources, where an entry on a secondary store does not exist. Update operations can now create an entry on a secondary store when necessary. Issue:DS-14789

  • There are several changes to the XACML request details:

    When the action-id is "grant," the grant-type attribute of the action category uses a dash instead of underscores, such as authorization-code instead of authorization_code.

    Changes to the urn:pingidentity:names:2.0:attribute-category:access-token category include:

    • The app and typ properties are removed.
    • The client property is added and contains the name of the OAuth 2 client for which the token was created.
    • The sub property is no longer present for app tokens.
    • All properties from the token validation endpoint are available.
    Issues:DS-12967,DS-13628,DS-14961,DS-14987

  • Changed interactive setup default value for HTTPS enablement. Issue:DS-15221

  • Enforce minimum length requirements for client secrets:

    • Minimum length of client secrets is now 32 characters, regardless of the signing-algorithm.
    • When using HS384 and HS512, the client secret length is 48 and 64 characters, respectively.
    • When the client-secret is generated, the length will be appropriate for the configured signing-algorithm.
    • The signing-algorithm used is now configured solely by the OAuth 2 Client configuration. The signing-algorithm property in the Identity Provider Service configuration is now deprecated.
    Issues:DS-12508,DS-15162,DS-15173,DS-15268,DS-15312,DS-15335

  • Updated the initial server configuration to improve security and usability. These changes apply only to new installations and will not be applied when updating an existing installation. Changes include:

    • Updated the default password policy to use a default password storage scheme that uses a salted 256-bit SHA-2 digest rather than a salted SHA-1 digest.

    • Updated the root password policy to use a default password storage scheme of PBKDF2 rather than salted 512-bit SHA-2.

    • Updated the secure password policy to use a default password storage scheme of PBKDF2 rather than a CRYPT variant that uses multiple rounds of 256-bit SHA-2.

    • Updated the password policy import plugin so that it will attempt to use the default password policy to select the password storage scheme(s) to use for entries that do not explicitly specify a password policy. The plugin will also fall back to using a salted 256-bit SHA-2 scheme instead of a salted SHA-1 scheme.

    • A number of weaker password storage schemes have been disabled by default, including base64, clear, unsalted MD5, salted MD5, 3DES, RC4, and unsalted SHA-1.

    • The default password policy has been updated to use a password generator that generates very strong yet memorable passphrases rather than a shorter and less-memorable string of randomly-selected characters.

    • Many of the server loggers have been updated to include additional log elements by default, including the instance name, requester DN, requester IP address, and request controls.

    • The exact match identity mapper has been updated to look at the mail attribute in addition to the uid attribute. When targeting a user with an authentication ID value (as when using SASL authentication or the proxied authorization v2 request control), it is now possible to specify an email address as an alternative to a user ID.

    • The UNBOUNDID-TOTP SASL mechanism handler has been updated to prevent TOTP password reuse by default.

    • Added new request criteria that make it possible to identify requests that target the root DSE or the subschema subentry. The global configuration has been updated so that requests targeting these entries will be in the default exceptions lists if the server is configured to reject insecure or unauthenticated requests.

    • Updated the template that setup generates for creating sample data to use a more logical and user-friendly numeric range. When the user requests N entries, setup would previously number the entries 0 through N-1 (for example, if the user requested 1000 entries, they would be numbered 0 through 999). It is logical for a user to expect them to be numbered 1 through 1000, but this change could break things that expecting to find an entry numbered with zero. To address this, if the user requests the server be populated with sample data, setup will create one more entry than actually requested so the numbering will go from 0 to N.
    Issues:DS-15183,DS-15220,DS-9407

  • Updated the default file permissions for new installations on UNIX-based systems. Files and directories included in the zip file will be only be accessible to their owner (the user that unzipped the file) by default.

    Newly-created files and directories will also be assigned permissions that allow them to be accessed only by the account used to run the server. Existing configuration options for setting file permissions (the log-file-permissions and db-directory-permissions properties) will continue to behave as before. The new config/server.umask file will control the default permissions for all other newly-created files and directories. Issues:DS-13571,DS-13860,DS-7505 SF#:2703

  • To help identify the originating client when a HTTP proxy or network load balancer is used, the Trace Logger and logged services will optionally log the originating client's protocol, host, and port. This is done by first examining the HTTP request "Forwarded" header and then the "X-Forwarded" header. If there are no such headers the current behavior is maintained. This optional feature can be turned on by setting the advanced "use-forwarded" property of HTTP connection handlers. Issue:DS-15028

  • Updated the global ACIs that ship with the server to use a separate ACI for each control or extended request to allow by default, rather than grouping all desired controls together in one ACI and all desired extended requests together in a second ACI. This change will only be reflected in new installations, and not when updating an existing deployment. Issue:DS-15417

  • Addressed an issue where dsconfig incorrectly allowed certain configuration objects to be deleted. Issue:DS-15400

  • Updated interactive setup to display default values, and improved the overall layout and appearance. Issues:DS-15361,DS-15363,DS-15434

  • Added more logging information when initializing web application and servlet extensions in case an extension causes conflicts or delays. Issue:DS-15466

  • Updated setup to encode the root password with the PBKDF2 password storage scheme instead of SSHA512. Issue:DS-15521

  • Updater tool will increase PermSize and MaxPermSize parameters to recommended value to prevent Java JVM pauses. Issue:DS-15522 SF#:00003324

  • Fixed an error that could occur during upgrade when the configuration can not be loaded due to missing custom schema. Issue:DS-15592 SF#:3340

  • Updated the Groovy Scripting Language version to 2.4.6. Issue:DS-15621

  • When creating SCIM resources backed by secondary store adapters, the server automatically sets the secondary correlation attribute value if it does not already have a value from the resource create request. Issue:DS-14544

  • Fixed an issue where SCIM schema attributes and sub-attributes marked 'Required' were not reflected as such via the /scim/v2/Schemas endpoint. Issue:DS-15741

  • Added a new rotate-log tool to request the rotation of one or more log files. Issue:DS-10464

  • The Data Governance Broker no longer returns account notices, warnings, and errors on authentication attempts using the resource owner password grant type. Instead, when authentication fails due to account issues, it returns an error_cause response parameter with either accountDisabled, accountLocked, or mustChangePassword. Additionally, when the password must be changed, the requirements that the new password must satisfy will also be returned. The client may then retry the token grant operation with the new_password request parameter to update the account's password. If the current password is correct and the new password satisfies all the requirements, the access token will be issued and returned. Issues:DS-13247,DS-15487,DS-15867

  • Operational attributes configured on an LDAP Store Adapter are now returned by default from a pass-through resource type without having to be requested. Issue:DS-15039

  • Improved the error messages produced by the manage-extensions tool when attempting to install invalid extensions. Issue:DS-15412

  • Improved the error messages and examples for create-rc-script and create-systemd-script by explicitly suggesting the use of sudo so that the scripts can modify protected files. Issue:DS-15178

  • The OAuth command line tools (oauth2-request and oauth2-request.bat) were removed. Any tool for performing http requests can be used in its place. For example, you could use curl to create, validate or invalidate tokens. Issue:DS-15942

  • The former suite of Administrative Console applications, each of which were tied to a particular product (for example the dsconsole.war for the Directory Server) are no longer available, and have been superceded by a new version of the Administrative Console capable of managing any server product. You can choose to access the Administrative Console by hosting it within a server, or by deploying it in an external servlet container. For the former, enable an HTTP Connection Handler and add the Administrative Console Web Application Extension to the handler. For the latter, download and unzip the management-console-[version].zip file, and install the ubid-console.war file according to your container's instructions. Issue:DS-15088

  • Root DN User configuration entries can now be fully managed through the configuration management interfaces such as dsconfig and the Administrative Console. Issue:DS-15422

  • Changed the way operations on SCIM sub-resources (i.e., consent, consent history, external identities, etc.) are authorized using OAuth 2 Scopes. Any single OAuth 2 Scope may now only apply to at most one SCIM sub-resource, which is specified using the scim-sub-resource-type property. Issue:DS-15990

  • Fixed an issue where a PUT of a SCIM user with a changed password resulted in the following error: "The password change would result in multiple password values in the user entry, which is not allowed." Issue:DS-15959

  • Added support for log file rotation listeners, which allow for custom processing whenever a log file is rotated out of service so that the server will no longer write to it. A copy listener (which will copy the rotated log file to an alternate location, optionally compressing it in the process), and a summarize listener (which will invoke the summarize-access-log tool on the rotated log file) are included. The Server SDK also includes an API for creating custom log file rotation listeners. Issue:DS-4235

  • The create-initial-broker-config setup script no longer supports an offline mode due to new requirements in this release. Issue:DS-15633

  • Improved the collect-support-data tool to include information provided by systemd on platforms that support it. Issue:DS-13401

  • Improved the error messages for create-rc-script and create-systemd-script when the directory in which the script will be created does not exist. Issue:DS-15337

  • Added the server's process ID to the output of the status tool. Issue:DS-10312

  • Added a monitor entry for each Server SDK extension. Issue:DS-14548

  • The Configuration API now returns unquoted, native Javascript values for integer, real number, and boolean properties. Duration and size property values, for example '1 w' or '100 G', continue to be represented as Javascript string types. Issue:DS-15175

  • Added the ability to create local constants in LDIF template files using the new 'local' keyword. Issue:DS-14213

  • Addressed a few issues in config-diff. In some situations, config-diff would not generate commands in an order that respected all dependencies. This has been fixed. Most expected warnings are now excluded by default but can be included in the output with the --includeAllWarnings option. The --sourceBindPasswordFile and --targetBindPassword are now applied in conjunction with the --targetConfigGroup and --sourceConfigGroup options. Issues:DS-10466,DS-10765,DS-14479,DS-15318,DS-16154

  • Added support for setting the request header size in the Jetty http configuration server properties. Issue:DS-12191 SF#:00002580

  • Updated the restore command so that it can no longer be used to restore a backup of the config backend. The command now points the administrator for safer ways to revert configuration changes, including using config-diff. Issue:DS-14704

  • Updated the sanitize-log tool to add support for JSON-formatted access and error log files. Issue:DS-16224

  • Added the ability to search for configuration objects and their properties by name with the dsconfig tool. Issue:DS-979

  • Added a --dry-run option to dsconfig, which can be used in batch mode to validate the configuration changes in a batch file without applying them. Issue:DS-10946

  • Tools used to prepare a server for access by another server, such as prepare-external-server, now validate base DN entries before any modifications are performed on the prepared server. Issue:DS-14807

  • Fixed an issue that prevented the deletion of disabled debug loggers. Issue:DS-15622

  • SCIM 2 POST, PATCH, and PUT operations that violate any attribute level uniqueness constraints now returns an error with 400 error code instead of 409. Issues:DS-16850,DS-16860

  • Fixed an issue where the Data Governance Broker could return a 400 error when creating a SCIM2 resource with complex attributes in extended schema. The error message in this case erroneously stated "The request contains attributes whose value(s) cannot be completely mapped to any store adapter: ", and did not specify which attributes could not be mapped. Issue:DS-16671

Data Governance Server 5.2.0.0

New Features

These features were added for version 5.2.0.0 of the Data Governance Server:

  • SCIM 2.0

    • Data Broker now supports the latest IETF SCIM protocol specification, which introduces new features that simplify managing user data. The /Me endpoint provides an easy reference to the currently authenticated user’s data. Expanded filtering operations including the ability for the client to explicitly include or exclude named attributes to provide greater control of the content of the response.
    • Changes to PATCH operations, and the addition of PUT, provide granular control of the result of the operation with respect to the data object being modified.
    • New filtering operations provide additional options to support broader type of queries including complex attribute filter groupings.
    Note: The SCIM 1.1 API for the Broker has been removed and is no longer included. Not all aspects of the SCIM 2 specification were implemented. The /Bulk and /Groups endpoints of SCIM 2.0 are not supported in this release. The SCIM 1.1 APIs for the Data Store, Proxy, and Sync Server products have not changed.

  • Resource Type (was Data View) Refactoring

    Data Broker can define custom resource views from LDAP and non-LDAP data stores. This includes the ability to map the attributes from a data store schema to a custom (SCIM) schema that will be used by Data Broker client applications. Simple mappings for like attribute types are supported to take complex object’s sub-attributes and map them to attributes in a data store.

    Also, a filter can be used on a multivalued complex object to map to a single complex object in the data store. Wild card mapping of all unmapped attributes to a single attribute has also been enabled.

    The mapping function can be by-passed to simply use the available data store schema.

  • Policy Engine Obligations and Advice

    Support for XACML-standard “obligation” and “advice” has been added, allowing policy rules to return instructions or information to a client application (PEP) as part of the policy decision response. An “obligation” is required to be executed by the PEP to exclude or include attributes. An “advice” is used to provide additional information on the decision that was delivered from the Policy Engine.

  • Policy Engine Support for Attribute Selectors using SCIM Paths

    Simplified resource level authorization of SCIM REST APIs using the Policy Engine with default OAuth 2.0 scope-based policy. This allows modeling simple privilege or role based access control using OAuth 2 scopes and without writing custom policies. Simplified policy-based scope granting by the OAuth 2.0 / OIDC authorization service.

  • JSON Attribute Support

    Data Broker now supports JSON attribute types, so that client applications can create and manage data objects without modifying the schema. This includes tighter integration between SCIM 2.0 and the OAuth2 protocol for data authorization.

  • Improved Logging

    Data Broker now provides simpler trace log and configuration log output from the Broker to better enable finding event details during pre-production testing.

  • Topology Registry for Data Broker

    The Data Broker configuration has been replaced by a new, central configuration registry that is distributed from a single (auto-nominated) master to all of the Data Broker servers in a topology. This allows configuration changes and settings to be made once at one server instance, and verified and pushed by the topology master to all associated server instances. The configuration also supports central configuration of administrative user accounts and storage of the public keys used in server-to-server communication.

  • New Data Broker Administrative Console

    The Data Broker Administrative Console now surfaces all Data Broker and UnboundID server configuration objects. All objects are also available with the dsconfig tool and the Configuration API.

  • Password API

    The Password API enables client applications to request the status of a user’s account, view password policy requirements, and perform other actions on the account, like password reset or set account status.

  • Consent and External Identity Provider APIs

    Added new SCIM-based REST APIs for consent management and external identity linking.

  • Social Registration

    End users can now review what data is being captured from their social provider through a populated registration, giving them the opportunity to change or correct their personal data.

Known Issues and Workarounds

These were known issues at the time of the release of version 5.2.0.0 of the Data Governance Server:

  • When deploying a .war file through the Web Application HTTP Servlet Extension, dependencies bundled in the file may conflict with the server's own dependencies if the server version differs from the version in the .war file. This may cause the Web Application HTTP Servlet Extension or the server itself to not start correctly. For reference, all server dependencies are available in /lib.

  • The Broker does not support signing OpenID Connect ID tokens using the RSA SHA-256 algorithm (RS256). The currently supported signing algorithms are HS256, HS384, and HS512.

  • The authorization endpoint does not support the max_age parameter defined by the OpenID Connect specification.

  • If a secondary store adapter is removed from a SCIM Resource Type, subsequent SCIM requests involving that resource type will fail if mappings to the secondary store adapter still exist. To resolve this, always remove related store adapter mappings when removing a secondary store adapter.

  • The new Data Broker Administrative Console cannot be run from a separate container, such as Tomcat.

  • The dsframework tool should not be used to create admin users for the Broker Administrative Console. The tool is available to manage the Administration Framework used by Data Store, Data Proxy, Data Sync and Metrics Engine.

Resolved Issues

These issues were resolved with version 5.2.0.0 of the Data Governance Server:

  • The setup tool has been updated to use HTTPS for initial configuration. Unsecure HTTP can be enabled post-setup, or by using non-interactive setup. Issue:DS-12182

  • Addressed cases where some messages may be suppressed in logs and alerts. Issue:DS-12287

  • Updated the Configuration API output where properties and their values are listed to include those that are undefined. Issue:DS-12123

  • Added support for running on Oracle Java 8 and OpenJDK 8 platforms. Issue:DS-12483

  • The Configuration API has been updated to support filtering, sorting, and paging for object list operations. See the Administration guide for usage. Issue:DS-12245

  • Added logging of all HTTP requests disallowed due to CORS. This should make it easier to debug HTTP 403/Forbidden errors. Issue:DS-12496

  • The server can now detect an "out of file handles" situation on the operating system, and shut down to prevent running in an unreliable state. Issue:DS-12579 SF#:2655

  • Update the Detailed HTTP Operation Log Publisher to log the correct return code (404 NOT FOUND) when a request is not handled by defined endpoints. Issue:DS-12576

  • Fixed an issue where changes to SMTP External Server configurations did not take effect until after a server restart. Issue:DS-12285

  • Improved server locking used by dsconfig in offline batch mode, so that the server lock is held for the entire batch duration, instead of for each invocation. Also, reduced the probability of contention for file locks used by server tools to determine the server status. Issue:DS-12969 SF#:2717

  • Fixed a rare condition that might cause the logger rotation and retention thread to exit under heavy file system load or a network file system outage. Issue:DS-12880

  • Added a gauge to monitor the number of available file descriptors. This Available File Descriptors gauge can detect if a server if running out of file descriptors and degrade the server appropriately. Issue:DS-12727

  • The setup and initial configuration tools now support offline modes that can be used to bootstrap the server configuration while it is not running. Also, files generated by theses tools are now saved to the server's resource directory. Issues:DS-12704,DS-8794,DS-9652

  • Addressed an issue where data definition language (DDL) log field mappings for the JDBC error log were not previously documented. Issue:DS-13163

  • Reduced the memory overhead of debug logging in high throughput environments by sharing logging buffers across multiple threads. Issue:DS-10010

  • Updated the server's JVM arguments to always log garbage collection information to a rotating set of log files stored within logs/jvm/gc.log.N. The file system usage is limited to 300MB. If the server had previously been configured with VERBOSE_GC, then garbage collection logging information will no longer be logged to logs/server.out. Issue:DS-11522

  • The following UnboundID product names have changed: - Identity Data Store to Data Store - Identity Proxy to Proxy Server - Identity Data Sync Server to Data Sync Server - Identity Broker to Data Broker Issue:DS-12799

  • Updated the prepare-external-server tool to suppress output when run with the --quiet option. Issue:DS-13242

  • Custom HTTP loggers are no longer permitted to modify the requests and responsesbeing logged. Calling a forbidden method will result in a subclass of UnsupportedOperationException. For requests, the forbidden methods are authenticate, getReader, login, logout and setCharacterEncoding. For responses, the forbidden methods are addCookie, addHeader, addIntHeader, flushBuffer, getOutputStream, getWriter, reset, sendError, sendRedirect, setBufferSize, setCharacterEncoding, setContentLength, setContentType, setHeader, setIntHeader, setLocale and setStatus. Issue:DS-10283

  • Fixed a bug where using the advanced arguments of some tools would result in changing the saved complexity settings for the dsconfig tool. Issue:DS-12897

  • Added a new search-logs tool. Similar to the command line tool 'grep,' this tool searches across log files to extract lines matching the provided pattern(s). The search-logs tool can handle multi-line log messages, extract log messages within a given time range, and include rotated log files. Issue:DS-3095

  • Updating prior versions of the Data Broker to version 5.2 is not supported. Issue:DS-12429

  • Updated the create-systemd-script tool by adding resource limits for available open file descriptors (NOFILE), and shared memory reservations (MEMLOCK). The generated script lists the recommended file descriptors limit and the resource limit setting for enabling large page support. The settings in the create-systemd-script output supersedes prior documentation for setting the number of open file descriptors on non-systemd systems. Issue:DS-13678

  • Updated interactive dsconfig to include an option to toggle between sorting similar properties together or sorting them alphabetically. Issue:DS-1706

  • The collect-support-data tool now has the option to collect logging information within a specified time range via the '--timeRange' argument. Issue:DS-1261

  • Broker now has a new logger, the trace logger. The trace logger will help with tracing transactions in the Broker. Issue:DS-13119

  • Improved the server's support for selecting TLS cipher suites. When the server is configured to use a specific set of cipher suites, it will now always validate that all of the configured suites are supported by the JVM. When the server is not configured to use a specific set of cipher suites, it will now customize the set of default suites to prioritize those using strong cryptography (especially those that offer forward secrecy), and exclude suites with known weaknesses. Issues:DS-12681,DS-13475

  • Updated the alert handler configuration to indicate whether the alert handler should be invoked asynchronously in a background thread rather than by the thread that generated the alert. For alerts generated during the course of processing an operation, invoking potentially time-consuming alert handlers in a background thread can avoid adversely impacting the response time for that operation while still ensuring that administrators are made aware of the issue that arose. Issue:DS-12833

  • Updated the server to provide support for SMTP connection pooling. When sending an email message, the server will attempt to reuse an existing SMTP connection rather than establishing a new connection for each message. Issue:DS-12833

  • Deprecated the INCREASE_PERM_SPACE jvmTuningParameter in the setup and dsjavaproperties tools, which was used to increase the size of the permanent generation in memory allocated for the server. Now PermGen is set to 256M whenever the heap allocated for the server is 1g or greater. If less than 1g, the PermGen is set to 128M with a maximum of 256M. Issues:DS-14020,DS-14066

  • Updated the prepare-* tools to avoid unnecessary confirmation for trust of the prepared server's certificate when the --trustStorePath argument specifies a trust store that establishes trust. Issue:DS-12616

  • Fixed a log publisher defect that would result in an unreadable file when both compression and signing were enabled at the same time. Issue:DS-13552

  • The Broker will now return account notices, warnings, and errors on successful and failed authentication attempts using the resource owner password grant type. Issue:DS-12450

  • The ldifsearch command now supports the option "---isCompressed" for LDIF files that have been compressed with gzip. Issue:DS-14140

  • Added properties to the task backend for limiting the number of log messages retained in task entries, in order to limit the size of the in-memory representation of those entries. All log messages generated by a task will still be recorded in the server error log, even if they are not all retained in the corresponding entry in the task backend. Issue:DS-11067 SF#:2282

  • Updated the configuration object names and file names of LDAP-centric logs to reflect that they apply to LDAP traffic only. Issue:DS-13122

Data Governance Server 5.1.5.1

Resolved Issues

These issues were resolved with version 5.1.5.1 of the Data Governance Server:

  • Fixed an issue in the Broker SCIM interface where null JSON values were converted to "null" strings. The JSON parser has been updated to ignore null attribute values, and to ignore attribute arrays without any non-null values. Also fixed an issue where it was possible to give a required SCIM attribute a value which is missing a required sub-attribute. Issue:DS-13815 SF#:00002916

Data Governance Server 5.1.5.0

Resolved Issues

These issues were resolved with version 5.1.5.0 of the Data Governance Server:

  • Added a gauge to monitor the number of available file descriptors. This Available File Descriptors gauge can detect if a server if running out of file descriptors and degrade the server appropriately. Issue:DS-12727

Data Governance Server 5.1.0.0

Important Upgrade Considerations

Important upgrade considerations for version 5.1.0.0 of the Data Governance Server:

  • The Identity Broker release will contain numerous changes in a future version and will not be backwards compatible with previous releases.

    • Upgrading from previous releases using update will not be supported.
    • The SCIM 1.1 REST API will be removed (along with XML support) for the Identity Broker only. This will be replaced with SCIM 2.0, which is JSON only. The SCIM 2.0 patch operation is not backwards compatible with SCIM 1.1.
    • The broker-admin command will be removed. All configuration will be performed using dsconfig.
    • Tags and trust-levels will no longer be configurable components in the Identity Broker. Tags will be a string property of applications and scopes.
    • Resources will no longer be a configurable component in the Identity Broker, and no longer need to be defined before used in policies.
    • The Identity Broker Access History REST API will be removed.
    • The Identity Broker consent-admin command will be removed.
    • Custom policies will need to be rewritten.

  • Use of the following new features require that the User Store be an UnboundID Identity Data Store version 5.1.0.0 or higher:

    • User registration
    • Username and password recovery
    • Account state and password management REST APIs

    When updating, the prepare-external-store tool must be run for both the Broker Store and the User Store, and additional Data Store configuration may be necessary. See "Updating the Identity Broker and the Broker Store" in the UnboundID Identity Broker Installation Guide for details.

  • The summarize-config tool is deprecated, and will be removed in future versions of the product. Use the config-diff tool with the "sourceBaseline" argument to list a summary of changes to the local server configuration.

New Features

These features were added for version 5.1.0.0 of the Data Governance Server:

  • User Registration (+reCAPTCHA) - The Identity Broker provides self-service registration pages that can be configured for customer use. The registration form includes fields to capture customer information, as well as the option to include Google reCAPTCHA to prevent bots from creating false accounts. The form can be customized to add or remove fields, and include company logos and website styles.

  • Password Reset - The Identity Broker delivers forms and functionality enabling customers to reset their account password. This is available on the login form, and is a reusable service that can be included in other web and mobile pages. This functionality leverages the Password Management API, which is new to this release.

  • Username Recovery - Similar to the password reset mechanism, customers can recover their account username with a set of administrator-defined properties.

  • Password Management API (Experimental) - The Password Management API validates candidate passwords to meet password policy, modifies passwords, and resets an account. The API can return the status of an account (such as locked or disabled) to a client application. This first release of the API is considered experimental. Generated documentation is available in the installation's /docs directory

  • The ID Token Grant type has been added for native application single sign-on use cases to provide an alternative to web views and embedded browsers. This should be used only where a well-trusted, native mobile application must retain control of the user interface for gathering a user ID and credentials. In all other cases, use the Web Views and Identity Broker interfaces for greater security, consistency, and reuse.

Known Issues and Workarounds

These were known issues at the time of the release of version 5.1.0.0 of the Data Governance Server:

  • The UnboundID Data Store 5.1.0.0 introduces support for a JSON attribute syntax and equality matching rule. The LDAP store adapter included with the UnboundID Identity Broker 5.1.0.0 does not support storing, reading, or mapping LDAP attributes using the JSON attribute syntax.

Resolved Issues

These issues were resolved with version 5.1.0.0 of the Data Governance Server:

  • The setup tool has been updated to use HTTPS for initial configuration. Unsecure HTTP can be enabled post-setup, or by using non-interactive setup. Issue:DS-12182

  • Addressed cases where some messages may be suppressed in logs and alerts. Issue:DS-12287

  • Updated the Configuration API output where properties and their values are listed to include those that are undefined. Issue:DS-12123

  • Added support for running on Oracle Java 8 and OpenJDK 8 platforms. Issue:DS-12483

  • Added logging of all HTTP requests disallowed due to CORS. This should make it easier to debug HTTP 403/Forbidden errors. Issue:DS-12496

  • Critical: The server can now detect an "out of file handles" situation on the operating system, and shut down to prevent running in an unreliable state. Issue:DS-12579 SF#:2655

  • Update the Detailed HTTP Operation Log Publisher to log the correct return code (404 NOT FOUND) when a request is not handled by defined endpoints. Issue:DS-12576

  • The Configuration API has been updated to support filtering, sorting, and paging for object list operations. See the Administration guide for usage. Issue:DS-12245

  • Fixed a possible security vulnerability where a UNC path could be set as the value of an Application or Identity Provider's URL field, or Icon URI field. These fields now require valid, absolute URL values, and the 'file' protocol scheme cannot be used. Issue:DS-12732

  • Fixed an issue where changes to SMTP External Server configurations did not take effect until after a server restart. Issue:DS-12285

  • Fixed a rare condition that might cause the logger rotation and retention thread to exit under heavy file system load or a network file system outage. Issue:DS-12880

  • Improved server locking used by dsconfig in offline batch mode, so that the server lock is held for the entire batch duration, instead of for each invocation. Also, reduced the probability of contention for file locks used by server tools to determine the server status. Issue:DS-12969 SF#:2717

  • Fixed a rare issue in which a NoSuchElementException could be thrown during login with valid credentials, causing the login attempt to fail. Issue:DS-12983

  • Reduced the memory overhead of debug logging in high throughput environments by sharing logging buffers across multiple threads. Issue:DS-10010

  • UnboundID Identity Data Stores configured as User Stores must be updated to version 5.1.0 prior to the installation of a new Identity Broker, or update of an existing Identity Broker installation. An update of the User Store also enables the following Identity Broker features:

    •User registration •Username recovery and password reset •Account state and password management REST APIs Issue:DS-13176

  • Fixed a problem in which an Identity Broker in a multi-server environment could incorrectly treat an authorization code issued by a different server as not yet valid, if server clocks were not synchronized. Issue:DS-13250

  • Fixed a potential security vulnerability in session management by using cookie-based session tracking in all default cases. The default session manager now only uses cookies for session tracking and always sets the HttpOnly cookie flag when a session manager is not explicitly configured in the HTTP Configuration. The in-memory session manager has also been updated to use only cookies for session tracking. Issue:DS-12734

  • Updated the default Velocity templates to omit exception details when errors occur. For troubleshooting purposes, display of these details may be enabled by setting the $debug flag in the template. Note that such exception details may be used by attackers to gather valid usernames or other information, so use of the default behavior is encouraged. Issue:DS-12735

Data Governance Server 5.0.1.0

Important Upgrade Considerations

Important upgrade considerations for version 5.0.1.0 of the Data Governance Server:

  • The Identity Broker can be upgraded from version 5.0.0, to version 5.0.1. See "Updating the Identity Broker and the Broker Store" in the UnboundID Identity Broker Administration Guide for details.

New Features

These features were added for version 5.0.1.0 of the Data Governance Server:

  • The Identity Broker now supports the mapping of multi-valued complex attributes that do not have a "type" sub-attribute (do not specify canonical types). The Identity Broker can map that attribute from a Data View to a multi-valued string attribute in a Store Adapter. Each complex value of the Data View attribute is encoded as a JSON string.

  • Custom attributes used in Policy requests can now be included in File-Based Authorization Log Publisher output, to support auditing of Policy decisions. This helps an auditor determine why a request was allowed or denied.

  • All SCIM PATCH operations are mapped to the "update" action when generating a XACML request. Previously, this meant that for multivalued SCIM attributes, there was no way for a Policy to determine whether the result of the PATCH was to add, remove, or replace existing values of the attribute. It is now possible to write a Policy that restricts the ability of clients to remove existing values of the attribute by checking for an "urn:pingidentity:names:1.0:update-operation” attribute with a value of “delete".

  • It is now possible for Policies to refer to HTTP request query and header parameters. This means that a client can provide additional information (metadata) along with the request. The Policy can then use that metadata as part of its evaluation. For example, a client or gateway can pass extra information about the context of a request — the type of client, the location or jurisdiction in which the data will be used, or additional data about the purpose of the request.

Known Issues and Workarounds

These were known issues at the time of the release of version 5.0.1.0 of the Data Governance Server:

  • When using the SCIM SDK to handle a complex, multivalued Data View attribute that has been mapped at the attribute level, the attribute's AttributeDescriptor will incorrectly report the presence of the normative sub-attributes (defined in section 3.2 of the SCIM 1.1 Core Schema spec), such as "type", even if those sub-attributes are not declared in the schema. However, attempts to read or set these undeclared sub-attributes, other than "operation," will fail.

Resolved Issues

These issues were resolved with version 5.0.1.0 of the Data Governance Server:

  • The setup tool has been updated to use HTTPS when configuring the HTTP Connection Handler(s). Unsecure HTTP can be enabled post-setup, or by using non-interactive setup. Issue:DS-12182

  • Addressed cases where some messages may be suppressed in logs and alerts. Issue:DS-12287

  • Updated the Configuration API output where properties and their values are listed to include those that are undefined. Issue:DS-12123

  • The server can now detect an "out of file handles" situation on the operating system, and shut down to prevent running in an unreliable state. Issue:DS-12579 SF#:2655

  • Fixed a possible security vulnerability in XACML processing due to expansion of externally defined entities by disabling support for external DTDs. This fix requires Oracle Java 7 Update 40. Issues:DS-12731,DS-12745

Data Governance Server 5.0.0.0

New Features

These features were added for version 5.0.0.0 of the Data Governance Server:

  • Java 7 is now required when setting up a new server or upgrading an existing server.

  • Enabled support for the SSLv2Hello TLS protocol by default in JVMs that support it. This does not enable support for the insecure SSLv2 protocol, but it can improve compatibility with clients running older versions of Java that may start TLS negotiation with an SSLv2 client hello packet before negotiating to a higher version of the TLS protocol. Support for SSLv2Hello in the initial phase of negotiation does not in any way compromise the strength of the integrity and/or confidentiality protection that is ultimately negotiated between the client and the server.

  • Added a Monitor History plugin that periodically records cn=monitor to timestamped files to aid in isolating intermittent problems. By default, it logs the full cn=monitor branch every five minutes to compressed files within logs/monitor-history/. Files are deleted automatically, but a sparse set of older files are kept to provide historical perspective on server performance. The collect-support-data tool has also been updated to collect a few of these files to aid in root cause analysis.

  • Introduced the Configuration HTTP Servlet Extension, which can be used for querying and updating the configuration over a REST API. This feature is currently experimental and is subject to change in the future. Your feedback is welcome.

Resolved Issues

These issues were resolved with version 5.0.0.0 of the Data Governance Server:

  • Fixed the gauge configuration manager to only re-initialize the gauge that was changed, and not any of the other gauges that did not change. Issue:DS-11472

  • Fixed the alarm manager to generate alarm-cleared alerts when internal alarms are cleared and the alarm manager's generated-alert-types property has the "alarm" value. Issue:DS-11541 SF#:2421

  • Removed the -hostname argument when running create-initial-broker-config from within setup, which was causing a deprecation warning during broker setup even when it was run with no arguments. Issue:DS-11513

  • Fixed the alarm manager to not include the details of the old alarm, (the alarm being cleared), in the "alarm-cleared" alert message. Issue:DS-11546

  • Fixed the dsconfig tool to suppress all stray output when run in batch mode with the --quiet option. Issue:DS-10460

  • Updated the status tool to use the host name specified at setup in URLs listed in the Active HTTP Extensions table. Issue:DS-11574

  • Updated the uninstall tool so that it unregisters the local server from any configured peer servers. Issue:DS-11564

  • Fixed an issue in which tools such as dsconfig, status, and dsreplication could not connect to the server over SSL or StartTLS. This occurred when a certificate was accepted with the 'Manually validate' option, while using the interactive LDAP connection menu. Issue:DS-11688

  • Updated the Velocity HTTP Servlet Extension and Velocity Context Providers to enable adding header fields to responses for pages and static content, such as images and scripts. Some default headings have been added to direct user agents on caching and frame display policies. Issue:DS-11649

  • Updated the alarm manager to not generate "alarm-normal" alert when a gauge's condition abates Issue:DS-11637

  • Reduced the severity of the "unrecognized alert type" message in the error log from SEVERE_WARNING to NOTICE. The message now states that this is expected if the server is reverted to a version prior to the implementation of these alert types. Issue:DS-11453

  • Removed the "alarm-normal" alert. Issue:DS-11730

  • Improved the Velocity Context Provider interface for HTTP method-specific requests. Context providers must now handle specific HTTP methods by overriding provider class methods. Provider implementations that handle HTTP methods, other than GET, must now be configured to handle them by updating the http-method configuration properties as well as overriding the appropriate handleXXX methods. Issue:DS-11650

  • Updated the alarm manager to not persist normal alarms. Issue:DS-11719

  • Updated the ExampleOverloadHandlerPlugin to monitor the alarm backend for delete actions, so that it can react appropriately to abating gauge conditions. Issue:DS-11719

  • Add an HTTP Configuration element with property include-stack-traces-in-error-pages that can be disabled in order to suppress stack traces included in web application and servlet error pages. Stack traces are helpful when diagnosing application errors, but in production they may reveal sensitive information. Issue:DS-11651

  • Critical: Disabled support for SSLv3 by default in the LDAP, HTTP, and JMX connection handlers, and for replication communication. The recently-discovered POODLE vulnerability could potentially allow a network attacker to determine the plaintext behind an SSLv3-encrypted session, which would effectively negate the primary benefit of the encryption.

    SSLv3 was initially defined in 1996, but was supplanted by the release of the TLSv1 definition in 1999 (and subsequently by TLSv1.1 in 2006 and TLSv1.2 in 2008). These newer TLS protocols are not susceptible to the POODLE vulnerability, and the server has supported them (and preferred them over SSLv3) for many years. The act of disabling SSLv3 by default should not have any adverse effect on clients that support any of the newer TLS protocols. However, if there are any legacy client applications that attempt to communicate securely but do not support the newer TLS protocols, they should be updated to support the newer protocols. In the event that there are known clients that do not support any security protocol newer than SSLv3 and that cannot be immediately updated to support a newer protocol, SSLv3 support can be re-enabled using the newly-introduced allowed-insecure-tls-protocol global configuration property. However, since communication using SSLv3 can no longer be considered secure, it is strongly recommended that every effort be made to update all known clients still using SSLv3.

    It is possible to use the server access log to identify LDAP clients that use SSLv3 to communicate with the server. Whenever an LDAP client establishes a secure connection to the server, or whenever a client uses the StartTLS extended operation to secure an existing plaintext connection, the server will generate a SECURITY-NEGOTIATION access log message. The "protocol" element of a SECURITY-NEGOTIATION access log message specifies the name of the security protocol that has been negotiated between the client and the server, and any SECURITY-NEGOTIATION messages with a protocol of "SSLv3" suggest that the associated client is vulnerable to the POODLE attack. In addition, if any connections are terminated for attempting to use the disallowed SSLv3 protocol, the access log message for that disconnect should include a message stating the reason for the termination. Issue:DS-11782

  • Updated the Web Console so that upon login, the user's old session is always invalidated. Issue:DS-11624

  • Updated the Web Console to suppress LDAP responses in user messages, such as when the server is unavailable or for authentication failures. Also added a context parameter to exclude stack traces and detailed error messages from appearing in the application's internal error page. Issues:DS-11629,DS-11645

  • Updated the HTTP Detailed Access logger to use timestamps with millisecond precision. Issue:DS-11755

  • Fixed incorrect property references for trustStorePassword and keyStorePasswordFile in tools.properties that corresponded to the wrong argument names. Issue:DS-11751

  • Return 409 from the linking service if the external identity is already linked to one or more other users. However all existing links will be removed before linking to the newly authenticated user during the link on login flow. Issue:DS-11802

  • Updated the setup tools to enable definition of external server instances that are configured to reject unauthenticated requests. Previously the tools would erroneously indicate these servers were unavailable. Issues:DS-11068,DS-11784,DS-11887

  • Fixed an issue where deleting values of a multi-valued attribute using SCIM PATCH could silently fail. Modifications in SCIM PATCH are now mapped directly to LDAP modifications to take advantage of the matching rules configured in the Identity Data Store, when matching deleted values. Since the SCIM PATCH is now applied by the Data Store, the Permissive Modify Request Control (1.2.840.113556.1.4.1413) is now required by the SCIM component. This will ensure that adding an existing value or deleting a non-existent value in the PATCH request will not result in an error.

    To continue using SCIM component after an upgrade of the Identity Data Store or Identity Proxy, access controls and configuration may need to be updated to allow access to the Permissive Modify Request Control.

    Identity Data Store:

    dsconfig set-access-control-handler-prop --remove 'global-aci:(targetcontrol="1.3.6.1.1.13.2 || 1.2.840.113556.1.4.473 || 1.2.840.113556.1.4.319 || 2.16.840.1.113730.3.4.9 || 1.3.6.1.1.12")(version 3.0;acl "Authenticated access to controls used by the SCIM servlet extension"; allow (all) userdn="ldap:///all";)'

    dsconfig set-access-control-handler-prop --add 'global-aci:(targetcontrol="1.3.6.1.1.13.2 || 1.2.840.113556.1.4.473 || 1.2.840.113556.1.4.319 || 2.16.840.1.113730.3.4.9 || 1.3.6.1.1.12 || 1.2.840.113556.1.4.1413")(version 3.0;acl "Authenticated access to controls used by the SCIM servlet extension"; allow (all) userdn="ldap:///all";)'

    Identity Proxy:

    dsconfig set-access-control-handler-prop --remove 'global-aci:(targetcontrol="1.3.6.1.1.13.2 || 1.2.840.113556.1.4.473 || 1.2.840.113556.1.4.319 || 2.16.840.1.113730.3.4.9 || 1.3.6.1.1.12")(version 3.0;acl "Authenticated access to controls used by the SCIM servlet extension"; allow (all) userdn="ldap:///all";)'

    dsconfig set-access-control-handler-prop --add 'global-aci:(targetcontrol="1.3.6.1.1.13.2 || 1.2.840.113556.1.4.473 || 1.2.840.113556.1.4.319 || 2.16.840.1.113730.3.4.9 || 1.3.6.1.1.12 || 1.2.840.113556.1.4.1413")(version 3.0;acl "Authenticated access to controls used by the SCIM servlet extension"; allow (all) userdn="ldap:///all";)'

    dsconfig set-request-processor-prop --processor-name dc_example_dc_com-req-processor --add supported-control-oid:1.2.840.113556.1.4.1413

    Note that "dc_example_dc_com-req-processor" is the default processor name and it may be different depending on your configuration.

    Identity Broker: For each Identity Data Store used as an user store, the following configuration changes are required:

    dsconfig set-access-control-handler-prop --remove 'global-aci:(targetcontrol="1.3.6.1.1.13.2||1.3.6.1.4.1.30221.2.5.3||1.3.6.1.4.1.30221.2.5.25||1.2.840.113556.1.4.1413||1.3.6.1.4.1.30221.2.5.5||2.16.840.1.113730.3.4.9||1.2.840.113556.1.4.473||1.2.840.113556.1.4.319")(version 3.0; acl "Broker User access to selected controls"; allow (read) userdn="ldap:///cn=Broker User,cn=Root DNs,cn=config";)'

    dsconfig set-access-control-handler-prop --add 'global-aci:(targetcontrol="1.3.6.1.1.13.2||1.3.6.1.4.1.30221.2.5.3||1.3.6.1.4.1.30221.2.5.25||1.2.840.113556.1.4.1413||1.3.6.1.4.1.30221.2.5.5||2.16.840.1.113730.3.4.9||1.2.840.113556.1.4.473||1.2.840.113556.1.4.319||1.2.840.113556.1.4.1413”)(version 3.0; acl "Broker User access to selected controls"; allow (read) userdn="ldap:///cn=Broker User,cn=Root DNs,cn=config";)'

    Note that the user DN "cn=Broker User,cn=Root DNs,cn=config" is default user name created when the external store is prepared. It may be different depending on your configuration. Issue:DS-11138

  • Disabled log rotation during startup to prevent potential problems with rotation dependencies on server components that have not yet been initialized. Issue:DS-10441

  • Added a gauge to the server to track JVM memory usage and alert if the amount of free memory gets low enough that it could impact server performance. Issue:DS-11993

  • Updated the server to make it easier to control the order of values in the ssl-protocol and ssl-cipher-suite properties in the LDAP connection handler and crypto manager configuration objects. Issue:DS-12147

  • The Identity Broker now uses TLS server validation to authenticate OpenID Connect Providers instead of validating ID token signatures. This enables the Identity Broker to support OpenID Connect providers, regardless of the algorithm used to sign ID tokens. Identity Broker deployers should not use the Blind Trust trust manager provider when configuring the Open ID Connect identity provider adapter. Issue:DS-11398

  • Fixed an issue in which a data view would become inoperable after updating an associated store adapter configuration. Configuration changes to a store adapter require a server restart before taking effect. Issue:DS-11460

  • Updated the HTTP Connection Handler to return a 404 Not Found response to requests for endpoints not handled by any servlet or web application extensions. Previously the hander would return a 200 OK with no response body. Issues:DS-12120,DS-8368

  • Fixed an issue where external Identity Provider login will fail when using an OpenID Connect Provider that omits the expires_in parameter in the token response. Issue:DS-12168

  • XACML policies, policy sets, and policy templates are now validated against the XACML schema definition during import. Issues:DS-12042,DS-12046

  • Updated the HTTP/HTTPS connection handler to Jetty 8.1.16.v20140903. Issue:DS-11959

  • The data view lookthroughLimit property now has a maximum value of 100000. Administrators should carefully consider the Identity Broker’s JVM configuration and the characteristics of its expected data set when configuring the lookthrough limit. Issue:DS-11803

  • Updated the governance tag and trust level behavior of resources. If a resource has no tags or trust levels, they will now be inherited from the parent resource. Issue:DS-11582

  • The NewUserCreation policy has been replaced by the UserCreateAndUpdate policy, which governs user creation and updates via the Identity Broker’s SCIM endpoint. Issue:DS-11481

Data Governance Server 4.7.0.0

New Features

These features were added for version 4.7.0.0 of the Data Governance Server:

  • New Profile Manager and Sign-In reference applications, which supersede the previously existing Privacy Preferences application.

  • OpenJDK 7 is now supported on Linux.

Known Issues and Workarounds

These were known issues at the time of the release of version 4.7.0.0 of the Data Governance Server:

  • Migration from previous releases of the Identity Broker is not supported.

  • JDK 6 is currently deprecated and will not be supported in the next major release.

  • UnboundID products are not supported on JDK 8.

Resolved Issues

These issues were resolved with version 4.7.0.0 of the Data Governance Server:

  • Updated the server preparation tools to use secure communication when setting up a Data Store for access over TLS. Previously the tools may fail when the server is configured to reject insecure requests. Issues:DS-11058,DS-6200

  • Added a result code tracker that maintains a monitor entry with counts and response times of results. Each result is categorized by operation type, post-response result code, and whether it is a failure or non-failure. Issue:DS-3270

  • Fixed an issue with HTTP Connection Handlers that allowed them to be configured with ports that were already in use. Now the server will not start if an HTTP Connection Handler is configured to use a port that is in use. Issue:DS-11202

  • The create-initial-broker-config and prepare-external-store tools now allow for non-OU base DNs for User Store and Broker Store configuration. Issue:DS-11286

  • Fixed a problem that prevented the server from starting if a TLS-enabled connection handler was configured with a certificate nickname that referenced a non-RSA certificate. Issue:DS-10949

  • Updated the dsjavaproperties tool so that the INCREASE_PERM_SPACE JVM tuning parameter is always included. This will prevent accidental misconfiguration that may harm performance. Issue:DS-11388

Data Governance Server 4.6.0.0

New Features

These features were added for version 4.6.0.0 of the Data Governance Server:

  • Identity Broker as Relying Party

    • Customers can support social login to their applications, from Facebook, Google and OIDC-compliant external Identity Providers.
    • Customers can deploy solutions that include both consent and social login.
    • Attributes can be captured from a consumer's social profile and stored in the user store.

Known Issues and Workarounds

These were known issues at the time of the release of version 4.6.0.0 of the Data Governance Server:

  • Using the manage_links scope, an application cannot delete all link values from the 'urn:pingidentity:schemas:broker:1.0.links' attribute by using the meta.attributes sub-attribute during a PATCH operation. Instead, each link value must be removed individually by adding an "operation" sub-attribute with the value "delete". For example:

     {
    "schemas": ["urn:pingidentity:schemas:broker:1.0"],
    "urn:pingidentity:schemas:broker:1.0": {
    "links": [
    {
    "providerId": "DF81",
    "providerName": "oidc-vm-small-83",
    "providerUserId": "9f8a23-b72ecd4b-34ac-3340-99fa-d0efacaf5d65",
    "operation": "delete"
    }
    ]
    }
    }
    Issue:DS-10945

  • UnboundID products, Java SE, and the JVM do not use OpenSSL libraries and are therefore not vulnerable to OpenSSL issues. Oracle has provided a statement on the April 2014 OpenSSL Heartbleed vulnerability at http://www.oracle.com/technetwork/topics/security/opensslheartbleedcve-2014-0160-2188454.html. Issue:DS-10807

  • The Relying Party feature cannot be used if the HTTP Session Manager Configuration session-tracking-mode property is set to "url." This is an advanced setting in dsconfig. Use of HTTP Session Managers is experimental and not supported. Issue:DS-10833

  • A SCIM request from a client application cannot contain null JSON elements in request bodies to represent attributes that have no values. Instead, the request should not include the attributes. Issue:DS-9048

  • If an application that was used in a consent operation is deleted, the Identity Broker's /privacy/v1/histories/{ownerCompositeKey}/accessHistory resource will show that application as null in its responses. This causes the following error with the Privacy Preferences application when requesting access history:
    An error occurred: "Unexpected AJAX error message format." Issue:DS-10203

  • Migration from previous releases of the Identity Broker is not supported.

Resolved Issues

These issues were resolved with version 4.6.0.0 of the Data Governance Server:

  • Updated the validate-file-signature tool to ensure that it will always display a final summary message to indicate whether any warnings or errors were encountered during processing. Issue:DS-10333

  • Updated the signed logging implementation to better handle any problems that may arise during cryptographic processing. If any such problem is encountered, the server will now include a message with information about the error in the signature block rather than suspending the logger with an exception recorded in the server.out log file. Issue:DS-10310

  • Fixed an issue in the Periodic Stats Logger, where no logging would occur when suppress-if-idle=true was configured, even when the server was not idle. Issue:DS-10387 SF#:2170

  • Metadata can now be retrieved from a modifies-as-creates store adapter if no entry exists. Issue:DS-10483

  • Added a new sanitize-log tool that can be used to remove sensitive information from server log files, including the file-based access log, the operation timing access log, the file-based error log, the file-based sync log, the file-based resync log, and the detailed HTTP operation log.

    The sanitization process operates on fields that consist of name-value pairs. The field name and equal sign will always be retained, but in cases where the value may contain sensitive data, that value may either be replaced with the string "---REDACTED---", or it may be tokenized. If the tokenized value is a DN or filter, then attribute names in that DN or filter will be preserved while the values will be replaced with a string consisting of a number inside curly braces. If the tokenized value is not a DN or filter, then the entire value will be replaced with a number inside curly braces. If a string to be tokenized appears multiple times in the log, the same replacement token will be used for each occurrence of that string to make it possible to correlate occurrences of that string without revealing the actual content.

    The sanitize-log tool has a default configuration that should be sufficient for many environments, allowing it to tokenize or redact sensitive information while preserving non-sensitive content for use in diagnosing problems or understanding usage patterns. However, this behavior can be customized using command-line arguments by indicating whether to preserve, tokenize, or redact a given log field. Issue:DS-10472

  • Custom Velocity templates should be updated to use Bootstrap 3.1.1 if they are using any of the shared original (shipped with the product) templates or scripts (such as _header.vm or dashboard.js). Issue:DS-10372

  • Fixed issues with the JDBC Access Logger that were related to Oracle Thin Client, where column values were "null" and disabling the logger resulted in losing a connection to the server when using the dsconfig command. Issue:DS-10485

  • Fixed an issue where calls to the /userinfo and /Self endpoints could fail when the user store contains consent for a deleted application. Issue:DS-10474

  • Fixed an issue so that collect-support-data now generates filename entries correctly. Previously, the tool would hang if the archiving of files following a symbolic link required generating a non-duplicating filename entry. Issue:DS-10582

  • Enabled the Host System Monitor Provider by default so that system CPU and memory utilization will be reported automatically through the server's monitoring framework. Disk and network monitoring can be enabled by configuring values for the disk-devices and network-devices configuration properties. Issue:DS-10562

  • Fixed an issue where replacing an expired consent with a new consent would retain the old timestamp. Issue:DS-10627

  • Fixed an issue where the Identity Broker could return an invalid SCIM ID when a SCIM resource was requested by its ID. This problem occurred when there were multiple store adapters with an ID attribute mapping, and the authoritative mapping was not associated with the first store adapter listed for the data view. Issue:DS-10516

  • The default timeout period for smtp-timeout was changed from none to two minutes to prevent non-responsive mail servers from disrupting administrative functions. Issue:DS-10230

  • To prevent corruption of the Broker Store, the Identity Broker now requires that the Broker Store be prepared by a current or later version of the Identity Broker installation. A Broker Store that is a version older than the current Identity Broker installation is not supported. Issues:DS-10093,DS-10563,DS-10613

  • Added ability to retrieve either owner or actor's attributes using the DataViewPolicyInformationProvider by using the attribute category.

    The following are valid attribute categories to use with the PIP:

    urn:pingidentity:names:1.0:attribute-category:resource-owner - Retrieve attribute from the owner resource. urn:pingidentity:names:1.0:attribute-category:actor - Retrieve attribute from the actor resource. urn:oasis:names:tc:xacml:3.0:attribute-category:resource - Retrieve attribute from the owner if available and the actor if not. (previous behavior) Issue:DS-10779

  • The setup command no longer saves user-provided key store and trust store passwords in PIN files. Passwords provided during setup are encrypted with the configuration data. If the administrator chooses to use PIN files to supply the passwords, the files are referenced in the server configuration by the key manager and trust manager. Issue:DS-10787

  • Updated the Periodic Stats Logger to include an empty value in the output rather than "infinity" in certain circumstances. This avoids problems plotting the output in a spreadsheet. Issue:DS-8842

  • Updated dsconfig to treat tabs as whitespace in batch files. Issue:DS-10549

  • Added Metrics Collection Size Limit Retention Policy to the metrics backend to allow up to 2 GB of metric data to be buffered locally, which allows the Metrics Engine to be offline for a longer time without missing collected data. Issue:DS-10156

  • Removed deprecated "lshal" command from Linux-specific processes performed by the collect-support-data tool and added similar command, "udevadm info --export-db" Issue:DS-10713

  • Updated the Replication Servers table produced by the dsreplication tool to omit unnecessary "Security" column. Issue:DS-10442

Data Governance Server 4.5.1.0

New Features

These features were added for version 4.5.1.0 of the Data Governance Server:

  • A new sample sign in application is now included with the Identity Broker. This application provides a template for how create an authentication UI with the Identity Broker's OpenID connect service.

  • The collect-support-data tool now refers to tools.properties for default command-line options.

  • The collect-support-data tool now supports an option to encrypt the data archive, to ensure protection of customer data while in transit, and an option to reduce the amount of potentially sensitive data that is collected.

  • Cross-origin Resource sharing (CORS) support is now included for HTTP Servlet Extensions, including the SCIM RESTful APIs.

  • Add support for SCIM resource versioning.

    • As required by the SCIM 1.1 REST API specification, bulk requests using the PUT, PATCH, or DELETE methods must include an operations.version value; existing client code may therefore need to be updated prior to enabling this feature.
    • Non-bulk modification requests (PUT and PATCH) must also specify the expected resource version via an If-Match header, according to the SCIM specification. However, the server does not enforce this.

Known Issues and Workarounds

These were known issues at the time of the release of version 4.5.1.0 of the Data Governance Server:

  • The SCIM REST client, included with the SCIM SDK, may hang while waiting for responses from the UnboundID SCIM implementation when using Java 6. This is due to a JDK issue, which was fixed in Java 7. If this condition is encountered, the latest version of Java 7 can be used with the client application to work around the issue. Issue:DS-10104

  • A SCIM request from a client application cannot contain null JSON elements in request bodies to represent attributes that have no values. Instead, the request should not include the attributes. Issue:DS-10105

  • If an application that was used in a consent operation is deleted, the Identity Broker's /privacy/v1/histories/{ownerCompositeKey}/accessHistory resource will show that application as null in its responses. This causes the following error with the Privacy Preferences application when requesting access history:
    An error occurred: "Unexpected AJAX error message format." Issue:DS-10203

  • When the Velocity servlet receives CORS-enabled requests and has a cross-origin policy in effect, it will return multiple Access-Control-* headers with duplicate values. This will cause cross-origin requests issued by web browsers to fail. Issue:DS-10205

Resolved Issues

These issues were resolved with version 4.5.1.0 of the Data Governance Server:

  • The setup tool's --aggressiveJVMTuning and --verboseGC command-line options have been deprecated. Instead, use --jvmTuningParameter AGGRESSIVE and --jvmTuningParameter VERBOSE_GC respectively. Issue:DS-9079

  • Update the server configuration to use a new default limit for duplicate alert suppression. The previous default imposed a maximum of 100 alerts of the same type per hour. The new default imposes a maximum of 10 alerts of the same type every ten minutes. This is more likely to suppress bursts in which the same alert is repeatedly generated over a short time without interfering with multiple occurrences of alerts of the same type over a longer period of time. Issue:DS-9259

  • Add a new load-balancing algorithm monitor entry that reports on the health of the load-balancing algorithm, including the aggregate state of the load-balancing algorithm, the number of AVAILABLE, DEGRADED, and UNAVAILABLE servers associated with the load-balancing algorithm, and the individual health check states of each server associated with the load-balancing algorithm. The status command has also been updated to report this information. Issue:DS-9026

  • Update the Velocity framework to better support customization out of the box. In order to customize Velocity templates or static content (CSS, Javascript files etc.) you should copy the original file from the config/velocity directory to the root velocity directory and modify the copied file. Files in the root velocity directory will override those in config/velocity. Issue:DS-9273

  • Update the server to improve the caching behavior for PIN files as used by key and trust manager providers. In the case that the keystore or truststore file has been updated to require a new PIN and the existing PIN file is updated without a configuration change to the associated key or trust manager provider, the server would previously keep trying to use the old PIN. It will now look for and use an updated PIN if a failure is encountered while using the old PIN. Issue:DS-10113 SF#:2123

  • Update the collect-support-data tool so that it can encrypt the data that is captured to protect it from unauthorized third parties. The encryption key is generated from a passphrase which may be read from a file, interactively provided by the user, or dynamically generated by the tool. This passphrase must be provided to support personnel (ideally over a different communication channel than the encrypted support data archive itself) for them to be able to access the information it contains.

    There is also a new option to decrypt an encrypted collect-support-data archive when provided with the encryption passphrase. Issue:DS-10129

  • Update the collect-support-data tool so that it is possible to configure default values for most arguments in the tools.properties file. Issue:DS-10178

  • Update the collect-support-data tool to further reduce the possibility of gathering sensitive information. Potentially sensitive data will be replaced with ---REDACTED--- in the output. A new "--securityLevel maximum" option can also be specified that redacts DNs and search filters, which might include personally identifiable information. Issue:DS-10115

Data Governance Server 4.5.0.0

New Features

These features were added for version 4.5.0.0 of the Data Governance Server:

  • The 4.5 Identity Broker, functioning as a Resource Server, can be configured to retrieve data from the UnboundID Identity Data Store and/or multiple back-end data stores in response to SCIM API requests. This means that the Broker's policy and consent enforcement can be applied to client requests, so that the returned data contains only those entries and attributes that pass the authorization rules.

  • When configured for multiple data stores, the Identity Broker can combine attributes into a single response, making the separate data stores look like a single one from the point of view of the client application. For each attribute, a specific data store can be marked as "authoritative" for read operations. Attribute-level control over write operations is provided also.

  • The Broker now supports the OpenID Connect protocol for incoming AuthN requests. These are checked against the credentials stored in the UnboundID Identity Data Store.

  • A new config-diff command line utility can compare two server configurations and produce the difference as a dsconfig batch file. The file can then be used to bring the source configuration in line with the target. Comparisons can be done between live servers or configuration files, and between current or legacy configurations. Run 'config-diff --help' to get more information including example use cases.

Known Issues and Workarounds

These were known issues at the time of the release of version 4.5.0.0 of the Data Governance Server:

  • Migration from 4.1 Identity Broker to 4.5 Identity Broker is NOT supported.

  • Java 1.7 has a synchronization bottleneck in HashMap that severely impacts performance. Use update 1.7u40, if possible, to avoid this issue. Issue:DS-9477

Resolved Issues

These issues were resolved with version 4.5.0.0 of the Data Governance Server:

  • Update SCIM and the Identity Access API to return a 400 status code when the id attribute is included in a PATCH request, as the id attribute is read-only. Issue:DS-9195

  • Update the OAuth authorization endpoint to require that the presented redirect URI exactly match one of the registered values for the client application. The previous behavior was to require that the presented redirect URI start with one of the registered values, and this behavior can still be obtained by issuing the following command followed by a server restart.

    dsconfig set-oauth-service-prop --set "redirect-resolver:Prefix Match Redirect Resolver" Issue:DS-8412

  • Add a --batch-file option to the broker-admin and consent-admin tools to allow multiple commands to be run with a single invocation of the tool. Issue:DS-8422

  • Fix a bug in the JDBC Access Logger that could cause incompatibility with some database versions and display a "Cannot commit when autoCommit is enabled" error message. Issue:DS-8750

  • Update the server startup process so that if no messages have been logged for at least five minutes, the server will generate and log a message about the current phase of startup processing. This can help reassure administrators that the server is still starting and provide information about what phase of startup may be taking so long. Issue:DS-7450

  • Update java.properties generation so that comments related to alternative JVM tunings are no longer present in the file. In most cases, rather than updating java.properties by hand you should use the dsjavaproperties tool to generate JVM options. Issue:DS-8339

  • Add an allow-insecure-local-jmx-access option to the global config that will expose JMX data via insecure local JVM connection Issue:DS-4300

  • Add a new alert handler that can use the Twilio service to deliver administrative alerts via SMS. Long alerts may be either truncated or split into multiple SMS messages. Issue:DS-5587

  • Update the configuration schema to make the ds-cfg-inherit-default-root-privileges attribute mandatory for object class ds-cfg-root-dn-user which is used to define Root User DNs. When this attribute is not present on Root DN User entries, the effect is for the root user to inherit default privileges. It has been made mandatory to make this behavior more explicit. During an update of the server, root DN user entries that do not explicitly declare a value for this attribute will be updated with a value of 'true'. Issue:DS-8450

  • Fix an issue that required create-initial-broker-config to be run independently of setup when using an external trust store. Issue:DS-8623

  • Add a WebLogic specific descriptor file for the web console to help with deployment compatibility. Issue:DS-8925 SF#:1915

  • The trust store password options have been deprecated for most tools that do not require read-write access to a trust store. Issue:DS-8789

  • Make a number of criteria-related improvements:

    - Add Server SDK support for creating custom connection, request, result, search entry, and search reference criteria implementations.

    - Update the simple request criteria type to make it possible to consider the search scope in determining whether a search operation matches the criteria.

    - Update the simple result criteria type to make it possible to consider the indexed/unindexed status in determining whether a search operation matches the criteria.

    - Add a new type of request criteria that may be used to more easily identify operations that target the server root DSE.

    - Add a new type of result criteria that may be used to classify operations based on replication assurance requirements and/or whether those requirements were satisfied.

    - Add a new allowed-insecure-request-criteria global configuration property that may be used to identify requests that the server should allow even if they are received over an insecure connection and the server is configured to reject insecure requests.

    - Add a new allowed-unauthenticated-request-criteria global configuration property that may be used to identify requests that the server should allow even if they are received over an unauthenticated connection and the server is configured to reject unauthenticated requests. Issues:DS-5079,DS-8168,DS-8770

  • Update the Broker's default approval page to add a section for optional scopes (scopes that have the 'required' property set to 'false'). Optional scopes appear in the approval form in a separate section from required scopes, accompanied by inline help that mentions that they can be unchecked to withhold approval. Issue:DS-8307

  • Configuration of many of the common beans used in spring-security-config.xml are now done via the configuration framework. The spring-security-config.xml file should only be modified for advanced use cases. Issue:DS-9027

  • Update error messages returned by the Identity Broker to distinguish between different types of authentication failures: - An authentication failure due to an incorrect client ID or client secret results in the following error message: Authentication client session failure: Bad credentials - An authentication failure due to an incorrect resource owner ID or password results in the following error message: Access token denied: Bad credentials Note that authentication failure error messages do not allow the client to distinguish between a failure due to an incorrect ID/username and a failure due to an incorrect secret/password. Issues:DS-8405,DS-8622

  • Add a new sign-log configuration property to file-based loggers that may be used to cause the server to digitally sign messages written by that logger. A new validate-file-signature tool may be used to verify signature information in signed log files, as well as LDIF files generated by signed LDIF exports. Issue:DS-8662

  • Conform to the OpenID Connect standard method for obtaining refresh tokens by using the "offline_access" scope. The access_type authorization parameter is no longer supported. Issue:DS-8792

  • The broker-admin tool now accepts and displays durations in the form "32w 12h" for application validity settings. The properties accessTokenValiditySeconds, refreshTokenValiditySeconds and consentValiditySeconds have been renamed accordingly to accessTokenValidityDuration, refreshTokenValidityDuration and consentValidityDuration. Issue:DS-9090

  • Remove the --offlineAccess option from the oauth2-request tool. To request a refresh token, include offline_access as one of the requested scopes. Issue:DS-9098

  • Add support for two new extended operations. A list configurations extended operation may be used to obtain information about the configurations that are available within to the server, including the currently-active configuration, the baseline configuration (i.e., the base configuration for an out-of-the-box installation of the current version), and all archived configurations that reflect configuration changes over time. A get configuration extended operation may be used to retrieve a specific configuration from the server. Issue:DS-9149

  • Update setup to fix a bug in which file path options specified as relative to the current directory may cause the server to be configured incorrectly or cause setup failure. Issue:DS-8389

  • Update the HTTP Connection handler to support configuration for tracking sessions either through HTTP cookies or by URL rewriting. Issues:DS-8639,DS-9128

  • Expose the Plugin type in the Server SDK, which is primarily useful for this server to have custom code run at server startup or shutdown using the 'startup' and 'shutdown' plugin types. Issue:DS-9165

  • Update the server to provide a degree of sandboxing around Server SDK extensions so that an unexpected exception thrown by an extension will be caught and result in an administrative alert rather than being caught further up in the stack and potentially causing other problems. Issue:DS-9247

  • In the rare cases where it is necessary to forcefully terminate the JVM from within the server itself, ensure that any files marked for deletion when the JVM shuts down are manually deleted before the JVM is terminated. This can help avoid problems like server shutdown not being detected properly because the server PID file hasn't been removed. Issue:DS-9267

  • Provide improved schema validation to detect additional cases in which certain misspelled tokens in the definition for a schema token could be silently interpreted as an extra property for that schema element. The server will now log a warning message about these unexpected tokens so that administrators can either correct them or prefix them with "X-" to indicate that they are an extra property provided for informational purposes. Issue:DS-9236

  • Reduce the time it takes the server to shut down in certain situations. Background threads sometimes missed a signal to wake up and had to wait for their next polling interval to see that a shut down had been requested. Issue:DS-9334

  • Update the default behavior of all file-based loggers to have include-thread-id=true. This will include a compact thread ID in all log messages. This can make it easier to correlate log messages generated by the same thread within a single log file or across different types of log files. Issue:DS-9352

  • Remove -XX:+UseMembar from the default set of generated JVM properties except on early JVM versions where this setting was required to work around a threading bug in the JVM.

  • Update the server JVM arguments generated by setup and dsjavaproperties to explicitly define -XX:MaxNewSize and -XX:NewSize for JVM's 1GB in size and larger. Also, add a comment to the generated java.properties file directing the administrator to use dsjavaproperties for making memory-related changes to this file rather than editing it directly. Issue:DS-9227

  • Add password file arguments to the scripts used to prepare external servers. Issue:DS-9406

  • Provide an example consumer-centric starter schema. This includes an LDAP schema, a make-ldif template for generating sample data, and a broker-admin batch file for mapping between the LDAP schema and a Data View. See resource/example-starter-schema/README.txt for more information. Issues:DS-8659,DS-9017

  • Update the setup and dsjavaproperties tools to permit maximum heap size values for memory that is not currently available on the host, though the value must still be less than the total amount of memory present on the host. Issue:DS-9111

  • Update the setup and dsjavapropeties tools to permit JVM heap size values to be as large as the amount of memory present on the system would permit. Issue:DS-9494

  • Update the Server SDK to provide the ability to run command line utilities within the server process. A ToolExecutor can be retrieved from the ServerContext. Currently, only the config-diff command is supported, but additional commands might be supported in the future. Issue:DS-9537 SF#:00001858

  • Enhance dsconfig to write to the config audit log when in offline mode. Issue:DS-1495

  • On Linux, issue a warning on startup and after a JVM pause if the kernel setting vm.swappiness is not 0 as this can cause the server to become unresponsive for several seconds when memory is paged back from disk during a garbage collection. Issue:DS-9070

  • Automatically record server monitor data at shutdown, as it may be useful for debugging purposes in cases where a problem was experienced within the server that was resolved by a restart. Issue:DS-9777

  • Improve the performance of certain monitor entry searches that target specific monitor entries by object class. In particular, this includes searches with AND or OR filters, as well as filters that target object classes not defined in the server schema. Issue:DS-9772

Data Governance Server 4.1.0.0

New Features

These features were added for version 4.1.0.0 of the Data Governance Server:

  • The UnboundID Identity Broker is the first of a new class of components for consumer and subscriber identity management architectures.

    As a stand-alone server, it provides authorization decisions for client applications, provisioning systems, API gateways, and analytical tools in any architecture involving personal, account, or sensitive identity data.

    Working together with the UnboundID Identity Data Store and Identity Proxy, the Identity Broker is designed to make high-volume and high-speed authorization decisions based on ever-changing consumer profile and consent data. Functionally, the Identity Broker is both the Policy Decision Point and the OAuth2 provider for externalized authorization. Performance-wise, the Identity Broker can support the request volumes driven by the complex, real-time interactions necessary to support today's consumer-facing mobile, social, and cloud ecosystems.

Resolved Issues

These issues were resolved with version 4.1.0.0 of the Data Governance Server:

  • Fix a bug where under certain error conditions the start server scripts could prompt user to overwrite existing file. Issue:DS-7268

  • Update the JMX connection handler to infer an appropriate Java type (e.g. Boolean, Long, Float, Date, or String) for JMX attributes from the underlying LDAP attribute type and value. The legacy behavior to return all JMX attributes as String values can be set if desired through the advanced global configuration property 'jmx-value-behavior'. Issue:DS-7635

  • Add the --noPropertiesFile option to the status command so that it does not fail when the option is provided to collect-support-data. Issue:DS-8390

  • Update setup to add a masters/peers trust-all argument so that the deployer must explicitly indicate that they trust the master/peer as well as any other masters/peers that are accessed during setup. In addition, if this argument is not specified a prompting trust store manager will be used instead of the previous behavior of using a trust-all manager all the time. If setup is in non-interactive mode and neither the trust-all argument nor the JKS trust store has been specified, and setup is accessing the master/peer over SSL or StartTLS setup will fail. Issue:DS-8381