Ping Identity Data Sync Server Release Notes
Ping Identity Data Sync Server Release Notes |
|
| Return to Documentation Index |
Notes for the following versions of the Ping Identity Data Sync Server are available in this document:
Updating to the latest version of the Data Sync Server addresses the following critical issues from previous versions. Affected servers should be updated.
Fixed two issues in which the server could have exposed some clear-text passwords in files on the server filesystem.
* When creating an encrypted backup of the alarms, alerts, configuration, encryption settings, schema, tasks, or trust store backends, the password used to generate the encryption key (which may have been obtained from an encryption settings definition) could have been inadvertently written into the backup descriptor. This problem does not affect local DB backends (like userRoot), the LDAP changelog backend, or the replication database.
* When running certain command-line tools with an argument instructing the tool to read a password from a file, the password contained in that file could have been written into the server's tool invocation log instead of the path to that file. Affected tools include backup, create-initial-config, create-initial-proxy-config, dsreplication, enter-lockdown-mode, export-ldif, import-ldif, ldappasswordmodify, leave-lockdown-mode, manage-tasks, manage-topology, migrate-ldap-schema, parallel-update, prepare-endpoint-server, prepare-external-server, realtime-sync, rebuild-index, re-encode-entries, reload-http-connection-handler-certificates, reload-index, remove-defunct-server, restore, rotate-log, and stop-server. Other tools are not affected. Also note that this only includes passwords contained in files that were provided as command-line arguments; passwords included in the tools.properties file, or in a file referenced from tools.properties, would not have been exposed.
In each of these cases, the files would have been written with permissions that make their contents only accessible to the system account used to run the server. Further, while administrative passwords may have been exposed in the tool invocation log, neither the passwords for regular users, nor any other data from their entries, should have been affected. We have introduced new automated tests to help ensure that such incidents do not occur in the future.
We recommend changing any administrative passwords you fear may have been compromised as a result of this issue. If you are concerned that the passphrase for an encryption settings definition may have been exposed, then we recommend creating a new encryption settings definition that is preferred for all subsequent encryption operations, exporting your data to LDIF, and re-importing so that it will be encrypted with the new key. You also may wish to re-encrypt or destroy any existing backups, LDIF exports, or other data encrypted with a compromised key, and you may wish to sanitize or destroy any existing tool invocation log files that may contain clear-text passwords.
The following enhancements were made to the topology manager to make it easier to diagnose the connection errors:
- Added monitoring information for all the failed outbound connections (including the time since it's been failing and the last error message seen when the failure occurred) from a server to one of its configured peers and the number of failed outbound connections.
- Added alarms/alerts for when a server fails to connect to a peer server within a configured grace period.
The topology manager will now raise a mirrored-subtree-manager-connection-asymmetry alarm when a server is able to establish outbound connections to its peer servers, but those peer servers are unable to establish connections back to the server within the configured grace period. The alarm is cleared as soon as there is connection symmetry.
The dsreplication tool has been fixed to work when the node being used to enable replication is currently out-of-sync with the topology master.
Fixed an issue with the sync connect and response timeouts being set with incorrect units of time.
Fixed an issue with the sync connect and response timeouts being set with incorrect units of time.
The server can now detect an "out of file handles" situation on the operating system, and shut down to prevent running in an unreliable state.
Disabled support for SSLv3 by default in the LDAP, HTTP, and JMX connection handlers, and for replication communication. The recently-discovered POODLE vulnerability could potentially allow a network attacker to determine the plaintext behind an SSLv3-encrypted session, which would effectively negate the primary benefit of the encryption.
SSLv3 was initially defined in 1996, but was supplanted by the release of the TLSv1 definition in 1999 (and subsequently by TLSv1.1 in 2006 and TLSv1.2 in 2008). These newer TLS protocols are not susceptible to the POODLE vulnerability, and the server has supported them (and preferred them over SSLv3) for many years. The act of disabling SSLv3 by default should not have any adverse effect on clients that support any of the newer TLS protocols. However, if there are any legacy client applications that attempt to communicate securely but do not support the newer TLS protocols, they should be updated to support the newer protocols. In the event that there are known clients that do not support any security protocol newer than SSLv3 and that cannot be immediately updated to support a newer protocol, SSLv3 support can be re-enabled using the newly-introduced allowed-insecure-tls-protocol global configuration property. However, since communication using SSLv3 can no longer be considered secure, it is strongly recommended that every effort be made to update all known clients still using SSLv3.
It is possible to use the server access log to identify LDAP clients that use SSLv3 to communicate with the server. Whenever an LDAP client establishes a secure connection to the server, or whenever a client uses the StartTLS extended operation to secure an existing plaintext connection, the server will generate a SECURITY-NEGOTIATION access log message. The "protocol" element of a SECURITY-NEGOTIATION access log message specifies the name of the security protocol that has been negotiated between the client and the server, and any SECURITY-NEGOTIATION messages with a protocol of "SSLv3" suggest that the associated client is vulnerable to the POODLE attack. In addition, if any connections are terminated for attempting to use the disallowed SSLv3 protocol, the access log message for that disconnect should include a message stating the reason for the termination.
Change the default behavior of the Synchronization Server to not lock entries across all Sync Pipes when processing changes.
The Sync Server has a specialized mutex that ensures that changes to the same entry are processed serially. The primary reason for this mutex is to ensure that the server can safely process changes in parallel to achieve high throughput. However, we also use this mutex to ensure that two Sync Pipes don't process the same entry at the same time for deployments that synchronize changes bi-directionally. A consequence of this locking is that if one Sync Pipe is failing (because the destination is unavailable) then it retains the lock on an entry, and when other Sync Pipes try to process changes to that entry they will block that change and all changes that follow it while they wait on the lock.
This change turns off using a shared mutex by default, but adds a new advanced configuration option on the Sync Pipe, shared-mutex-name, that specifies the name of a mutex that is shared by other Sync Pipes. This gives greater control over the locking so that two Sync Pipes that share end points can ensure that two changes to the same logical user are not processed concurrently, while not impacting other Sync Pipes.
See the shared-mutex-name property for more information.
This property is subject to change in a future release.
Following are notes for version 7.2.1.3 of the Data Sync Server.
The following issues have been resolved with this release of the Data Sync Server:
Addressed an issue where a resync operation might not synchronize all users if a PingOne Customer Sync Source was configured with a value for population-to-synchronize. Issue:DS-39177
Addressed an issue where a resync command could fail with a SEVERE_ERROR condition when run with multiple passes and the Sync Source was either Active Directory or PingOne for Customers. Issue:DS-39174
Fixed an issue that would throw an exception when trying to delete an entry containing uncached attributes if the LDAP changelog was enabled and using reversible form. Issue:DS-38957 SF#:00662848
These issues were resolved with version 7.2.1.1 of the Data Sync Server:
Critical: Fixed two issues in which the server could have exposed some clear-text passwords in files on the server filesystem.
* When creating an encrypted backup of the alarms, alerts, configuration, encryption settings, schema, tasks, or trust store backends, the password used to generate the encryption key (which may have been obtained from an encryption settings definition) could have been inadvertently written into the backup descriptor. This problem does not affect local DB backends (like userRoot), the LDAP changelog backend, or the replication database.
* When running certain command-line tools with an argument instructing the tool to read a password from a file, the password contained in that file could have been written into the server's tool invocation log instead of the path to that file. Affected tools include backup, create-initial-config, create-initial-proxy-config, dsreplication, enter-lockdown-mode, export-ldif, import-ldif, ldappasswordmodify, leave-lockdown-mode, manage-tasks, manage-topology, migrate-ldap-schema, parallel-update, prepare-endpoint-server, prepare-external-server, realtime-sync, rebuild-index, re-encode-entries, reload-http-connection-handler-certificates, reload-index, remove-defunct-server, restore, rotate-log, and stop-server. Other tools are not affected. Also note that this only includes passwords contained in files that were provided as command-line arguments; passwords included in the tools.properties file, or in a file referenced from tools.properties, would not have been exposed.
In each of these cases, the files would have been written with permissions that make their contents only accessible to the system account used to run the server. Further, while administrative passwords may have been exposed in the tool invocation log, neither the passwords for regular users, nor any other data from their entries, should have been affected. We have introduced new automated tests to help ensure that such incidents do not occur in the future.
We recommend changing any administrative passwords you fear may have been compromised as a result of this issue. If you are concerned that the passphrase for an encryption settings definition may have been exposed, then we recommend creating a new encryption settings definition that is preferred for all subsequent encryption operations, exporting your data to LDIF, and re-importing so that it will be encrypted with the new key. You also may wish to re-encrypt or destroy any existing backups, LDIF exports, or other data encrypted with a compromised key, and you may wish to sanitize or destroy any existing tool invocation log files that may contain clear-text passwords. Issues:DS-38897,DS-38908
Fixed an issue in which backups of the encryption settings database could be encrypted with a key from the encryption settings database. Issue:DS-38550
These issues were resolved with version 7.2.1.0 of the Data Sync Server:
Critical: The following enhancements were made to the topology manager to make it easier to diagnose the connection errors:
- Added monitoring information for all the failed outbound connections (including the time since it's been failing and the last error message seen when the failure occurred) from a server to one of its configured peers and the number of failed outbound connections.
- Added alarms/alerts for when a server fails to connect to a peer server within a configured grace period. Issue:DS-38334 SF#:00655578
Critical: The topology manager will now raise a mirrored-subtree-manager-connection-asymmetry alarm when a server is able to establish outbound connections to its peer servers, but those peer servers are unable to establish connections back to the server within the configured grace period. The alarm is cleared as soon as there is connection symmetry. Issue:DS-38344 SF#:00655578
Critical: The dsreplication tool has been fixed to work when the node being used to enable replication is currently out-of-sync with the topology master. Issue:DS-38335 SF#:00655578
Added a max-rate-per-second configuration property to the Sync Pipe, which can be used to throttle the rate that changes are applied to the destination. Issue:DS-38098
Added the PingOne for Customers Sync Source to Ping Data Sync. Identities can now be synchronized from PingOne for Customers to on premises identity stores. Issue:DS-38253
These features were added for version 7.2.0.0 of the Data Sync Server:
Introduced a Directory REST API to create, read, update and delete (CRUD) any object in the directory using JSON over HTTP. Compared to the SCIM-based Identity Access API (introduced in 4.0), the Directory REST API offers more capability without the configuration overhead and SCIM protocol limitations. See https://apidocs.pingidentity.com/pingdirectory/directory/v1/api/guide/ for more information.
Added support for Oracle Java JDK 11 and OpenJDK 11. Added support for RedHat 7.5, CentOS 7.5, and Ubuntu 18.04 LTS.
When running on JDK 11, we now configure G1GC as the default garbage collection algorithm. This eliminates long garbage collection pauses in most environments.
These were known issues at the time of the release of version 7.2.0.0 of the Data Sync Server:
There are known issues when running the server with Java 11.0.0. These are addressed in Java 11.0.1. In general, when using Java 11, we recommend using the latest available release. Issue:DS-38005
On Microsoft Windows systems, JVM arguments for verbose GC logging do not work as expected. So these arguments are not added to any of the server or client tools. Issue:DS-37798
These issues were resolved with version 7.2.0.0 of the Data Sync Server:
Added support for an exec task that can invoke commands on the server. There are several safeguards in place to prevent unauthorized users from invoking arbitrary commands on the server system, including a new exec-task privilege and a whitelist file that must be updated to include the absolute paths of the allowed commands. A new schedule-exec-task tool helps create an exec task from the command line, and the LDAP SDK has also been updated to allow interacting with exec tasks programmatically. Issue:DS-35873
Added support for recurring exec tasks. Issue:DS-35873
Added support for a delay task, which can be used on its own or as a recurring task. It is primarily intended to be used as a spacer between other tasks, and can sleep for a specified period of time, wait for the server to be idle (that is, there are no outstanding operations and all worker threads are idle), or wait for sets of search criteria to match at least one entry (for example, until a monitor entry indicates that the server is in a desired state). Issue:DS-36510
Added a new preFetch method to the LDAPSyncSourcePlugin Server SDK extensions. This method allows extension developers to modify the search request or source entry fetch behavior for LDAP Sync Sources. Issue:DS-36434
Added support for a new file retention task that can identify files in an indicated directory that match a given pattern and remove any matching files that fall outside of the specified retention criteria. You can specify the minimum number of files that should be retained, the minimum age of files that should be retained, the minimum aggregate size of files that should be retained, or any combination thereof. The files that match the pattern will be sorted by timestamp so that if any files are to be removed, the most recent files will be retained and the oldest files will be deleted.
The file retention task can be scheduled as a standalone task or as a recurring task. Two instances of the file retention recurring task have been defined in the default configuration: one that can clean up old expensive operation dump files, and another that can clean up old work queue backlog thread dump files. In each case, the recurring task is configured to keep at least the 100 most recent files, and no files less than 30 days old will be removed. While these recurring tasks are defined in the out-of-the-box configuration, they are not part of any recurring task chain and therefore will not actually be invoked unless they are configured as part of a chain.
The Directory Server and Directory Proxy Server now include recurring tasks in the out-of-the-box configuration that can clean up old expensive operation dump log files or work queue backlog thread dump log files if too many of them have collected in the server logs directory. For each type of file, if there are more than 100 of them in the server logs directory, then any of the remaining files that are more than 30 days old are candidates for removal. A recurring task chain will perform this cleanup every day at 12:05 a.m. in the JVM's default time zone. Issues:DS-35652,DS-36559
A header containing a correlation ID is now added to outgoing HTTP servlet responses, allowing HTTP responses to be correlated with log messages across server instances. The name of the correlation ID response header defaults to "Correlation-Id" but may be changed by setting the HTTP Connection Handler's correlation-id-response-header property. By default, the server will generate a globally unique correlation ID automatically, but the correlation-id-request-header configuration property may be used to optionally specify one or more request headers that provide an existing correlation ID value from the requesting client. The correlation ID header can be disabled on a per-HTTP Connection Handler basis using the use-correlation-id-header configuration property.
For Server SDK extensions that have access to the current HttpServletRequest, the correlation ID can be retrieved as a String via the HttpServletRequest's "com.pingidentity.pingdata.correlation_id" attribute. For example: <code>(String) request.getAttribute("com.pingidentity.pingdata.correlation_id");</code> Issue:DS-36209
HTTP Connection Handlers will now raise an alarm during initialization if a context path conflict is detected. Issue:DS-35909
Fixed an issue in which the HTTP Servlet Config Monitor could cause an exception in an HTTP Servlet Extension when attempting to determine its context paths. This caused the status tool and the Administrative Console to potentially omit the HTTP Servlet Extension from the list of active HTTP extensions. Issue:DS-37131
Added a Mock Access Token Validator, which accepts access tokens without validating the authenticity of the tokens using a trusted authorization server or signing certificate. When enabled, a Mock Access Token Validator accepts bearer tokens in the form of a plain text JSON object containing an arbitrary set of claims. Mock Access Token Validators are intended for test or demonstration use only and should never be enabled in production deployments or used to access sensitive data. Issue:DS-36433
Added a time limit retention policy to support removing log files older than a specified age. Issue:DS-37492
To facilitate testing in multiple GC (garbage collection) environments, GC JVM options having been moved to separate Java properties in the java.properties file. The new ".gc-type" suffix will select the GC type to use, and the new ".gc-<GC type>-args" suffix will have the JVM options for that GC type. Issue:DS-6930
Added Change Detectors as a new type of Server SDK extension. A Change Detector can be used to customize the way a Sync Source detects changes to entries in a source server.
As an example, a developer could build a Change Detector that reads messages from a Kafka topic and uses them as the source of changes for an OpenLDAP server configured as the Sync Source.
Initially, Change Detectors are only available for use with the new Generic LDAP Sync Source. Issue:DS-36435
Updated the Sync Pipe monitor entry (cn=Sync Pipe Monitor: [Pipe Name], cn=monitor) to include information about sync operation processing time. This includes the average, the maximum, and a histogram breakdown of the processing times. Issue:DS-37263
A new --topologyFilePath argument has been added to remove-defunct-server, making it possible to remove a defunct server cleanly from the topology using one of the servers in the provided topology file. The topology file may be obtained by running the manage-topology export command. Issues:DS-36023,DS-37686
Updated the server to expose information about the duration of lengthy phases of server startup. The longest phases are logged to the error log and more detail is provided in the "cn=Startup Phase Times,cn=monitor" monitor entry. Starting the server with the --verbose option will show fine-grained timing information for all phases of server startup. Issue:DS-35567
Fixed an issue where an OutOfMemoryException could be encountered during resync. Issue:DS-37601 SF#:00645279
Enhanced LDAP Sync Destinations such as PingDirectory, DSEE, and Generic LDAP so that the IgnoreNoUserModificationRequestControl is applied to requests by default. This allows Sync Sources to write and update operational attributes, such as "createtimestamp" or "modifiersdn." Additional configuration on the destination server may be required to enable modifying specific operational attributes. Issue:DS-37804
All tools will now enforce a minimum heap size requirement. Overriding the heap size for the system, using the --maxHeapSize argument of the dsjavaproperties tool, is only effective if the provided value is greater than the minimum required heap size for the tool. Issue:DS-36834
Added the ability to modify an LDAP operation prior to execution for creates, deletes, modifies, and searches in an LDAP Sync Destination with a plugin. Issue:DS-36242
These were known issues at the time of the release of version 7.0.1.0 of the Data Sync Server:
Servers to be monitored by the PingDataMetrics Server must have an instance name of less than 256 characters. A server's instance name is specified during setup. Issue:DS-36788
These issues were resolved with version 7.0.1.0 of the Data Sync Server:
Added a configuration option to allow a null serverFQDN for the GSSAPI SASL mechanism to allow an unbound SASL server connection. Issue:DS-36642 SF#:00637397
Added the ability to synchronize pre-encoded passwords to the PingOne for Customers Sync Destination.
Updated the keys and values used in the monitoring JMX MBeans to conform with best practices. The keys "type" and "name" are now used in place of "Rdn1" and "Rdn2".
To maintain backwards compatibility with existing monitoring solutions, installations upgrading to this release will retain the old behavior, but they can revert to the default behavior by changing the Global Configuration property jmx-use-legacy-mbean-names to false. Issue:DS-37235
Improved supportability by adding additional logging for the PingOne for Customer Sync Destination to expose all errors from the PingOne API. Issue:DS-37275
Important upgrade considerations for version 7.0.0.0 of the Data Sync Server:
This release introduces significant changes to the way servers in a topology are configured with information about each other. Once a server has been upgraded from a pre-7.0 version to 7.0 or later, reverting to the previous version is not supported. Before beginning the upgrade process, make sure you have read and understood the Administration Guide's chapter "Upgrading the Server".
SCIM 2 error responses, including Config API error responses, now represent the "status" field as a JSON string rather than as a number. Clients written to expect the earlier version format will need to be updated. In particular, clients written using the SCIM 2 SDK for Java should upgrade to version 2.2.0 or higher.
The Administrative Console now uses server information found in the topology registry to populate its server selection control. If the Console is used to manage a legacy server that does not use the topology registry, then the server selection control will not be populated. To manage a different server, the administrator will need to log out of the Console and provide the other server's connection details from the login page.
These features were added for version 7.0.0.0 of the Data Sync Server:
Simplified management tasks related to configuring servers in a large cluster topology or in an automated deployment. Most notably, servers can now be added to a cluster while other servers are offline.
Added management features for SSL/TLS certificates. The default certificates used in inter-server replication can be replaced; validation of client certificates for HTTPS-based services like the SCIM REST API can be configured; and you can reload from the trust store for HTTPS client certificates without restarting the server or the HTTP-based services.
Added support for these operating system versions: Ubuntu LTS 16.04, CentOS 7.4, RedHat Linux 7.4, SUSE Enterprise 12 SP3
These were known issues at the time of the release of version 7.0.0.0 of the Data Sync Server:
Simultaneous cloning multiple PingDirectory Proxy, PingData Sync, and PingData Governance Servers from another server of the same type is not currently possible. To create server instances that are identical to a master server, cloning must be performed one at a time.
These issues were resolved with version 7.0.0.0 of the Data Sync Server:
Support for the IBM JDK has been retired. Issue:DS-35536
Sync Pipe Plugins can now be enabled for specific Sync Classes when creating or editing a Sync Class. As before, any Sync Pipe Plugins enabled on the Sync Pipe will run for all associated Sync Classes. In the case where plugins have been enabled on the Sync Pipe and an associated Sync Class, the plugins enabled on the Sync Pipe will run before any enabled on the Sync Class. Issue:DS-17543
Fixed a class loader issue where Sync Source extensions written using the Server SDK throw a ClassNotFound exception when importing classes not included in the Data Sync Server base classpath. Issue:DS-17347
Updated the JMX connection handler's monitor provider so that when a JMX connection is closed, it is removed from the list of established connections. After a JMX client disconnects, it may take the server a few minutes to detect the closure and update the monitor. Issue:DS-35576
The admin backend and the tool used to manage it, dsframework, have been replaced by the topology registry and dsconfig, respectively. The topology registry is automatically mirrored across all servers in the topology, so administrative information is kept in-sync on all servers at all times. Issues:DS-14281,DS-14282,DS-14283,DS-14284,DS-17197,DS-17366,DS-4570
Added a new manage-certificates tool that can be used to perform a number of functions related to TLS certificate management. Issue:DS-17891
Added a new Monitor Entry for SSL Cipher Suite and Protocol information. It is available under cn=SSL Context,cn=monitor. Issue:DS-35601
Added a missing double-quote to bat/transform-ldif.bat, which prevented the command from being invoked successfully on Windows systems. Issue:DS-35648
Updated the server to include an instance of the Periodic Stats Logger Plugin that is enabled out-of-the-box to aid in diagnosing support issues. The "Historical Stats Logger" plugin will log performance statistics to logs/monitor-history/historical-dsstats.csv every five minutes. This works in concert with the "Monitor History" plugin, which logs the full contents of cn=monitor to logs/monitor-history every five minutes. The tail of this csv file is automatically included in the output generated by collect-support-data. Issue:DS-35581
Fixed a defect where configuring a Directory server on a Windows machine with a space in the home directory pathname would cause server setup to fail. Issue:DS-35583
Added the ability to configure data encryption during setup using a randomly generated key, a key generated from a user-supplied passphrase, or a key obtained from an export of another server's encryption settings database. When setting up multiple instances, providing the same encryption passphrase to each instance will ensure that all instances have the same encryption key.
The encryption-settings tool has also been updated to allow creating encryption settings definitions from a passphrase, to allow providing a description when creating a new encryption settings definition, and to record a create timestamp for new definitions. It is now possible to create ciphers that use the Galois Counter Mode (GCM) cipher mode (for example, using a cipher transformation of "AES/GCM/PKCS5Padding") for authenticated encryption. Definitions created with with just a cipher algorithm but no transformation will now use stronger settings.
The default encryption settings export format now provides stronger encryption. Newer server instances should be able to import encryption settings exported from other servers without issue. When exporting encryption settings for import into older servers, use the new --use-legacy-export-format argument. Issues:DS-15223,DS-35895
The create-systemd-script command now suggests placing the script created in "/etc/systemd/system." Issue:DS-35868
Added an encrypt-file tool that can encrypt and decrypt data with a user-supplied passphrase, an encryption settings definition, or a topology key shared among server instances. It includes support for decrypting the content in encrypted backups, LDIF exports, and log files. Issue:DS-36054
Fixed an issue with compressed logging that could leave some data buffered in memory and not actually written out to disk until the logger is closed. Issue:DS-36070 SF#:00628238
Added support for encrypted logging, using a key generated from an encryption settings definition. Encrypted log files may be decrypted with the encrypt-file tool. Issue:DS-6970
Made a number of improvements to backend backup and restore, and to LDIF export and import:
* Added the ability to encrypt backups and LDIF exports with a key generated from a user-supplied passphrase or with a key generated from an encryption settings definition. Previously, encrypted backups and LDIF exports only used a secret key that was known only to servers within the replication topology. The new options make it easier to restore encrypted backups and import encrypted LDIF files in servers outside of the replication topology. The encrypt-file utility can be used to decrypt encrypted backups and LDIF exports, regardless of how the encryption key was obtained.
* Added the ability to limit the rate at which backups and LDIF exports will be written to disk, which can help avoid performance problems that result from these operations saturating the disk subsystem.
* Added new global configuration properties for automatically encrypting backups and LDIF exports by default, which will be set to true if data encryption is enabled during setup.
* Added new global configuration properties that can specify which encryption settings definitions will be used to obtain the encryption keys for automatically encrypted backups and LDIF exports. If not specified, then the server will use its preferred encryption settings definition, or an internal topology key if no encryption settings definitions are available.
* Added a new configuration property for automatically compressing encrypted LDIF exports.
* Updated the backup tool to add new --promptForEncryptionPassphrase, --encryptionPassphraseFile, and --encryptionSettingsDefinitionID arguments that can be used to specify which key to use for encrypting the backup. Added a new --doNotEncrypt argument that can be used to force a backup to be unencrypted even if automatic encryption is enabled. Added a new --maxMegabytesPerSecond argument that can be used to impose a limit on the rate at which the backup may be written to disk.
* Updated the restore tool to add new --promptForEncryptionPassphrase and --encryptionPassphraseFile arguments that can be used to provide a user-supplied passphrase for use in accessing the contents of an encrypted backup. For backups encrypted with an encryption settings definition or an internal topology key, the server will automatically be able to determine the correct key.
* Updated the export-ldif tool to add new --promptForEncryptionPassphrase, --encryptionPassphraseFile, and --encryptionSettingsDefinitionID arguments that can be used to specify which key to use for encrypting the export. Added a new --doNotEncrypt argument that can be used to force an LDIF export to be unencrypted even if automatic encryption is enabled. Added a new --maxMegabytesPerSecond argument that can be used to impose a limit on the rate at which the LDIF file may be written to disk.
* Updated the import-ldif tool to add new --promptForEncryptionPasshprase and --encryptionPassphraseFile arguments that can be used to provide a user-supplied passphrase for use in accessing the contents of an encrypted LDIF export. The --isEncrypted and --isCompressed arguments are no longer necessary, as the tool can automatically detect encryption and compression (although those arguments are still available to preserve backward] compatibility), and it can automatically identify the correct key for exports encrypted with a key obtained from an encryption settings definition or an internal topology key. Issues:DS-12157,DS-35896 SF#:3628
Updated setup to include key usage, extended key usage, and subject alternative name extensions in the self-signed certificates that it generates. Issues:DS-35727,DS-35728
Implemented invocation logging for several server tools, which will write to logs/tools/tool-invocation.log by default upon startup and shutdown. Some of the information recorded by log entries include the tool's start and completion times, the command-line arguments used to initialize them, and the name of the system account used to launch the tool. To modify this behavior, edit the config/tool-invocation-logging.properties file. Issue:DS-4406
Updated the SCIM Sync Destination to always send credentials preemptively when configured to use HTTP basic authentication. Issue:DS-36198
Updated tools that interact with log or LDIF files to support reading from input files that are compressed and encrypted and writing to compressed and encrypted output files. Issue:DS-36075
Added support for TLS1.2 with STARTLS to connect to an SMTP server. Issue:DS-36093 SF#:00631871
Added the ability to generate administrative alert notifications when a task starts running, when it completes successfully, or when it fails to complete successfully. Also added the ability to send an email message to a specified set of users when a task starts running or completes successfully, which complements the existing ability to send an email message when a task fails to complete successfully or when it completes with any state, regardless of success or failure. Issue:DS-426
Added support for TLS1.2 with STARTLS to connect to SMTP server Issue:DS-36093 SF#:00631871
Provided the means to request that the server dynamically reload the certificate key and trust stores used by all HTTP connection handler instances that provide support for HTTPS. The request can be made using a new reload HTTP connection handler certificates task, the reload-http-connection-handler-certificates tool, or programmatically from a Server SDK extension using the ServerContext#reloadHTTPConnectionHandlerCertificates method. Issue:DS-35990 SF#:00629638
Added the PingOne for Customers Sync Destination for Ping Data Sync. Identities can now be synchronized from on premises identity stores to PingOne for Customers. Issue:DS-36000
The update tool now enforces specification of a new product license when updating to a new major version. The license can be specified using the --licenseKeyFile command-line options, or by copying the license file to the top-level directory of the server package used to perform the update. Request a license key through the Ping Identity licensing website https://www.pingidentity.com/en/account/request-license-key.html, or contact sales@pingidentity.com. Issue:DS-35523
In addition to specifying an exact set of desired cipher suites for the LDAP and HTTP Connection Handlers, administrators can now specify inclusions to, or exclusions from, the set of cipher suites selected by the server. Issue:DS-36088
Added support for recurring tasks, which can be used to automatically invoke certain kinds of administrative tasks based on a specified schedule.
At present, only certain kinds of tasks can be scheduled as recurring tasks. This includes both backups and LDIF exports, each of which provides retention support to limit the amount of disk space that the backups and LDIF files consume. It also includes support for any kind of task in which each instance of the task should use exactly the same values for all of the task-specific attributes. The Server SDK also provides an API for creating custom third-party recurring task implementations. Issue:DS-426
Updated the server to reduce contention when converting between strings and the bytes that comprise those strings. Issue:DS-36328 SF#:626850
Added a sanitize option to the Monitor History Plugin that, if enabled, will redact the small amount of potentially personally identifiable information that could appear in search filters and LDAP DNs within the monitor. This makes it easier to share the monitor history files with the support team in secure environments. Issue:DS-36545
Fixed an issue where the password attribute could be deleted when using the PingData Sync Server with an Active Directory Sync Source. Issue:DS-36466 SF#:00631858
Fixed a compatibility issue with PingFederate when it is used as a SCIM Sync Destination, in which a PingFederate server's SCIM schema response containing the schema for the User resource type could not be parsed. Issue:DS-36326
Increased the default size of the queue used to hold alert notifications so they can be asynchronously processed by a background thread. This makes it less likely that the queue will become full if many alerts are generated in a short period of time, which would cause subsequent attempts to generate alerts to block while the server catches up. Also updated the server to log a message when the queue becomes full so that administrators will be aware of the problem and will have suggestions for addressing it. Issue:DS-36360 SF#:635134
Updated the dsconfig list subcommands to list objects of all complexity levels rather than requiring the --advanced flag to list advanced and expert objects. Issue:DS-16508
These issues were resolved with version 6.2.0.0 of the Data Sync Server:
Added a disabled-alert-type configuration property to the Alert Backend that can be used to suppress specific alert types from being added to the backend. Issue:DS-16906 SF#:3556
The SNMP context name for the server can now be configured using the new context-name property of the SNMP Subagent Plugin. The server instance name remains the default context name when this property is not set. Issue:DS-16405
The server now requires Java version 8. Issue:DS-17019
It is now possible for Sync Destination Server SDK Extensions to ignore a change by calling the setIgnore() method on the provided sync operation. Ignoring a change prevents the change from being included as an applied operation in the statistics that are reported for the Sync Pipe via the Management Console, the bin/status command line utility, and in a raw form under "cn=Sync Pipe Monitor: <pipe name>,cn=monitor". Issue:DS-15051 SF#:2281
Updated a couple of cases where filtered SCIM searches for groups with missing members were not returned. Issue:DS-17078 SF#:00003677,00003683
The Data Sync Server now supports OpenDJ as an external server and as a sync source. Issue:DS-16423
The Data Sync Server now supports generic LDAP servers as sync destinations. Issue:DS-17089
The Data Sync Server now supports Oracle Unified Directory as an external server and as a sync source. Issue:DS-10016
Updated the logic used to select which TLS cipher suites should be enabled by default, and the logic used to prioritize those cipher suites. The selection process has been updated to use the guidelines provided in the OWASP "Transport Layer Protection Cheat Sheet" document.
Some of the changes include:
- The server already preferred cipher suites that support forward secrecy over those that don't. It now prefers DHE over ECDHE, and avoids suites that use non-RSA keys.
- The server already avoided cipher suites that used known-weak cryptographic weaknesses, including null encryption, the RC4 symmetric cipher, and the MD5 digest algorithm. It now also avoids anonymous encryption, the single-DES symmetric cipher, the IDEA symmetric cipher, and any suite using export-level encryption.
- The server now prefers cipher suites that use the Galois/Counter Mode (GCM) over the Cipher Block Chaining (CBC) mode.
- The server now prefers AES-based cipher suites with 256-bit keys over those that use 128-bit keys. For suites with equivalent key sizes, it prefers suites with a stronger message digest algorithm over suites with a weaker digest algorithm (e.g., SHA384 over SHA256 over SHA).
- The server now provides better support for selecting and prioritizing ciphers when running on the IBM JVM. The IBM JVM uses somewhat different naming for its cipher suites than the Oracle implementation, which previously allowed certain desirable suites to not be included in the selected set. Issue:DS-17146
Fixed an issue where incorrect names were displayed in the usage for the start scripts. Issue:DS-16593
Removed the default root password from the out-of-the-box configuration. This password was never actually used because it was replaced by the user-supplied password provided when running setup, and it has been removed for additional security. Issue:DS-17318
Updated the installer to discourage the use of weak root passwords.
When run in interactive mode, setup will display a list of password quality recommendations before prompting for the initial root password, suggesting that it should be at least 12 characters long, should not be contained in a dictionary of English words, and should not be contained in a dictionary of commonly-used passwords. If the proposed password does not meet these constraints, then the user will be given the option of proceeding with the provided weak password or choosing a different password.
When run in non-interactive mode, setup will exit with an error if the proposed initial root password does not satisfy the above constraints, unless the command line also includes the --allowWeakRootUserPassword argument.
In either mode, when a strong initial root password is supplied, setup will also configure the root users' password policy to ensure that subsequent root user passwords will also be required to satisfy these constraints. Issue:DS-2074
Updated the access and audit loggers so that, when logging information about an internal operation that was triggered by an external client request, the log message will include the connection and operation ID for that request. Also updated the error logger so that when logging a message from a thread that is actively processing an operation, the log message will include the connection and operation ID for that operation. Issue:DS-16509 SF#:3536
For Active Directory external servers, it is now possible for the bind-dn property to be a User Principal Name (UPN). Issue:DS-16603
Replaced the ldapsearch and ldapmodify tools with new versions. The new versions are backward-compatible, but offer a number of new features, including better connection handling, better output formatting, better support for bulk operations, support for referrals, support for additional request and response controls, and rate limiting. The ldapsearch tool now offers the ability to output results in JSON, CSV, or tab-delimited text as an alternative to LDIF, and provides support for a number of data transformations. The ldapmodify tool now supports the LDIF control syntax, as well as writing to output and reject files. Issues:DS-15861,DS-15862
Improved error reporting for the manage-extensions tool. Issue:DS-17080
The modifierName and modifyTimestamp attributes are now updated when offline configuration changes are made. Issue:DS-16858
Corrected the port number returned in the error message that is displayed when an administrator is trying to set up a server that is already running. Issue:DS-13721
The server now monitors important certificates used for client and inter-server communication. Certificate information is available in the Administrative Console and in the status tool output. An alarm is raised and alerts are sent when a monitored certificate is 30 days from expiration. Issue:DS-1029
Fixed an issue that could cause a sync pipe to crash due to missing attributes in the changelog. Issue:DS-17531 SF#:00609552
The Administrative Console can be deployed in an external web container, such as Tomcat, using the contents of resource/admin-console.zip, located in the server root. Issue:DS-17544
Added an optional reason parameter for dsconfig changes that will be automatically included in the server's config-audit.log file. Issue:DS-811
Updated the Server SDK to provide methods for obtaining a single LDAP connection or an LDAP connection pool with connections established to a specified LDAP external server defined in the server configuration.
Also updated the server configuration to add support for obscured values. An obscured value is a general-purpose string that is stored in an obscured form in the configuration so that its plaintext value is not readily discernible to anyone looking at the configuration file and so that the value is not displayed in administrative interfaces. The Server SDK provides a method for obtaining the plaintext representation of an obscured value, and this mechanism can be used to store potentially sensitive values in the configuration for use in Server SDK extensions without the need to store those values in the clear. Issue:DS-10694
Addressed an issue where the server would throw a NullPointerException if a com.unboundid.directory.sdk.sync.api.SyncSource implementation did not set a modifiers name in a generated ChangeRecord. Issue:DS-17653
Fixed an issue in the Data Sync Server where attribute matching rules were not correctly applied when synchronizing. Issue:DS-17652 SF#:610262
Sync correlation attributes now support correlating using JSON keys within JSON attribute values. A JSON key may be referenced with syntax <attribute name>.<JSON key> If a JSON correlation attribute is used and the JSON key does not exist for the source or destination entry or the source or destination entry considered does not have valid JSON data then the destination entry considered entry will not be matched. Issue:DS-17668
Constructed attribute mappings now support multivalued source attributes for conditional ("conditional-value-pattern" property) and non-conditional ("value-pattern" property) value patterns. Only one of the source attributes that contribute to a given value pattern may be multivalued. Issue:DS-1045
The Administrative Console is no longer compatible with older versions of the server. Issue:DS-17241
Updated the server to reduce the use of the SHA-1 message digest. The server will now use a 256-bit SHA-2 digest instead of a SHA-1 digest in all of the following cases:
- When hashing or signing a backup. - When signing an LDIF export. - When signing log data. - When generating MACs for an encrypted collect-support-data archive. - When generating unique identifiers for encryption settings definitions. - When determining whether the configuration changed with the server offline.
In all of the above cases, the server includes metadata in the output of the cryptographic processing to indicate the digest or MAC algorithm used for that processing, which ensures that the output remains compatible across server versions. For example, an LDIF export that uses a signature generated with the SHA-2 digest can be successfully imported into older versions of the server.
Also, the fingerprint certificate mapper has been updated so that it can use the 256-bit SHA-2 digest when mapping a client certificate to the corresponding user entry. The previous MD5 and SHA-1 digests remain supported.
Finally, the example enhanced password storage scheme provided with the UnboundID Server SDK has been updated so that it uses the 256-bit SHA-2 digest instead of a SHA-1 digest. Issue:DS-17444
Sync constructed attribute mappings now support extracting JSON fields within JSON attributes by appending '.' and then the JSON field to extract to the attribute name. For example, if the JSON attribute 'ubidEmailJSON' and the 'value' field is to be extracted, then 'ubidEmailJSON.value' could be specified for the attribute name resulting in '{ubidEmailJSON.value}' or '{ubidEmailJSON.value:/regex/replacement/flags}' if a regular expression is to be used. Issue:DS-17689
LDAP changelog-based Sync Sources such as the DSEE Sync Source will no longer fetch source entries if the targetDN in the corresponding changelog entry is not contained in any of the Sync Source's configured base-dn values. This can reduce the load placed on the source directory servers. Issue:DS-17742 SF#:613996
LDAP referral entries are now synchronized as a raw entry rather than trying to follow the referral. Issue:DS-17741 SF#:00613775
Constructed attribute mappings now support modifers which alter the value of referenced source attributes when added to the end of source attribute references. For example, if attribute 'mail' is to be included in a constructed JSON value then modifier 'jsonEscape' could be specified resulting in '{{ "userMail":{mail:jsonEscape} }}' or '{{ "userMail":{mail:/regex/replacement/flags:jsonEscape} }}' if a regular expression is to be used. Note that '{{' expands to '{' and '}}' to '}'. The modifiers are: * jsonEscape Escape text for use in a JSON value. * ldapFilterEscape Escape text for use in an LDAP filter. * lowerCase Convert text to lower case. * trim Remove leading and trailing whitespace. * upperCase Convert text to upper case. Issue:DS-17693
The Data Sync Server now logs an error and continues processing if it encounters a corrupted changelog entry. Previously, the server would continually throw an exception and stop processing subsequent changes. Issue:DS-17711 SF#:613375
The script files used to stop and start the server have been renamed stop-server and start-server. The older scripts are still present but may be removed in a future release of the product. Issue:DS-16789
The Data Sync Server now supports synchronizing particular fields within JSON attributes. The way fields are synchronized as well as the way source and destination values are correlated is controlled by a the "JSON Attribute" configuration object. Issue:DS-17688
Updated the Server SDK's ServerContext to expose a ValueConstructor, which can be used to build String values using a value-pattern template that references attribute values within an Entry. See the Javadoc for the ValueConstructor class with the Server SDK packaging for more information. Issue:DS-17576
The sync server will now try to use the source schema, if available, in cases where the destination schema is unavailable. Issue:DS-17936 SF#:00598940
Critical: Fixed an issue with the sync connect and response timeouts being set with incorrect units of time. Issue:DS-18026 SF#:00616763
Updated the Server SDK so that HTTPServletExtensions can be installed on Data Sync Server instances. Issue:DS-18003
Added additional logging to the Data Sync Server for cases where attribute mapping fails, but does not abort the synchronization operation. Issue:DS-17993
Limited the ACI search on collect support data tool to only pull 100 entries. This will reduce the time the tool takes to run for organizations with a large number of ACIs. Issue:DS-17968
Added additional logging for ignored synchronization operations. Issue:DS-17356
Added additional support to the Data Sync Server for synchronizing Active Directory Groups with more than 1500 members. Issue:DS-17697
Added configuration options for setting the SSL Protocol and/or the SSL Cipher Suites to the HTTPS Connection Handler. Issue:DS-10748 SF#:00003622,614777
Enhanced the HTTPS Connection Handler to send a HTTP Strict Transport Security header by default in all responses. Issue:DS-14650
Updated the default configuration of the File-Based Access Logger (logs/access) so that requests from peer Data Sync instances are no longer suppressed. This simplifies troublehooting connection and health-checking issues between server instances. Issue:DS-18199
Updated PingDirectory, PingDirectoryProxy, PingDataSync, and PingDataGovernance with the capability to run as Windows Services. Issue:DS-4161
Fixed an incompatibility between Java and PKCS12 truststores and keystores that caused an error during Data Sync Server installations. Issue:DS-18169 SF#:00619730
Enhanced Attribute Mapping in the Data Sync Server to allow users to filter out or exclude values from entries in the Sync Destination. Issues:DS-17545,DS-17546,DS-18008
Enhanced the LDAP Sync Destination to allow administrators to configure options for synchronizing user passwords with clear-text, in cases where the LDAP Server does not support pre-encoded password synchronization. Setting password-synchronization-format:clear-text on the LDAP Sync Destination will now enable the clear-text synchronization of passwords. By default, passwords will only be synchronized in the clear over a secure connection. This can be overridden by setting require-secure-connection-for-clear-text-passwords:false on the LDAP Sync Destination. Issue:DS-18136
Updated dsconfig batch mode to operate more efficiently over the WAN by consolidating the number of LDAP searches required to retrieve the full configuration when pre-validating configuration changes. Issue:DS-35495
A license key is required when setting up a server for the first time. Request a license key through the Ping Identity licensing website https://www.pingidentity.com/en/account/request-license-key.html or contact sales@pingidentity.com. Issue:DS-18100
Removed the ability to create custom HTTP trace loggers using the Server SDK. Issue:DS-18188
Added a disabled-alert-type configuration property to the Alert Backend that can be used to suppress specific alert types from being added to the backend. Issue:DS-16906 SF#:3556
The SNMP context name for the server can now be configured using the new context-name property of the SNMP Subagent Plugin. The server instance name remains the default context name when this property is not set. Issue:DS-16405
The server now requires Java version 8. Issue:DS-17019
It is now possible for Sync Destination Server SDK Extensions to ignore a change by calling the setIgnore() method on the provided sync operation. Ignoring a change prevents the change from being included as an applied operation in the statistics that are reported for the Sync Pipe via the Management Console, the bin/status command line utility, and in a raw form under "cn=Sync Pipe Monitor: <pipe name>,cn=monitor". Issue:DS-15051 SF#:2281
Updated a couple of cases where filtered SCIM searches for groups with missing members were not returned. Issue:DS-17078 SF#:00003677,00003683
The Data Sync Server now supports OpenDJ as an external server and as a sync source. Issue:DS-16423
The Data Sync Server now supports generic LDAP servers as sync destinations. Issue:DS-17089
The Data Sync Server now supports Oracle Unified Directory as an external server and as a sync source. Issue:DS-10016
Updated the logic used to select which TLS cipher suites should be enabled by default, and the logic used to prioritize those cipher suites. The selection process has been updated to use the guidelines provided in the OWASP "Transport Layer Protection Cheat Sheet" document.
Some of the changes include:
- The server already preferred cipher suites that support forward secrecy over those that don't. It now prefers DHE over ECDHE, and avoids suites that use non-RSA keys.
- The server already avoided cipher suites that used known-weak cryptographic weaknesses, including null encryption, the RC4 symmetric cipher, and the MD5 digest algorithm. It now also avoids anonymous encryption, the single-DES symmetric cipher, the IDEA symmetric cipher, and any suite using export-level encryption.
- The server now prefers cipher suites that use the Galois/Counter Mode (GCM) over the Cipher Block Chaining (CBC) mode.
- The server now prefers AES-based cipher suites with 256-bit keys over those that use 128-bit keys. For suites with equivalent key sizes, it prefers suites with a stronger message digest algorithm over suites with a weaker digest algorithm (e.g., SHA384 over SHA256 over SHA).
- The server now provides better support for selecting and prioritizing ciphers when running on the IBM JVM. The IBM JVM uses somewhat different naming for its cipher suites than the Oracle implementation, which previously allowed certain desirable suites to not be included in the selected set. Issue:DS-17146
Fixed an issue where incorrect names were displayed in the usage for the start scripts. Issue:DS-16593
Removed the default root password from the out-of-the-box configuration. This password was never actually used because it was replaced by the user-supplied password provided when running setup, and it has been removed for additional security. Issue:DS-17318
Updated the installer to discourage the use of weak root passwords.
When run in interactive mode, setup will display a list of password quality recommendations before prompting for the initial root password, suggesting that it should be at least 12 characters long, should not be contained in a dictionary of English words, and should not be contained in a dictionary of commonly-used passwords. If the proposed password does not meet these constraints, then the user will be given the option of proceeding with the provided weak password or choosing a different password.
When run in non-interactive mode, setup will exit with an error if the proposed initial root password does not satisfy the above constraints, unless the command line also includes the --allowWeakRootUserPassword argument.
In either mode, when a strong initial root password is supplied, setup will also configure the root users' password policy to ensure that subsequent root user passwords will also be required to satisfy these constraints. Issue:DS-2074
Updated the access and audit loggers so that, when logging information about an internal operation that was triggered by an external client request, the log message will include the connection and operation ID for that request. Also updated the error logger so that when logging a message from a thread that is actively processing an operation, the log message will include the connection and operation ID for that operation. Issue:DS-16509 SF#:3536
For Active Directory external servers, it is now possible for the bind-dn property to be a User Principal Name (UPN). Issue:DS-16603
Replaced the ldapsearch and ldapmodify tools with new versions. The new versions are backward-compatible, but offer a number of new features, including better connection handling, better output formatting, better support for bulk operations, support for referrals, support for additional request and response controls, and rate limiting. The ldapsearch tool now offers the ability to output results in JSON, CSV, or tab-delimited text as an alternative to LDIF, and provides support for a number of data transformations. The ldapmodify tool now supports the LDIF control syntax, as well as writing to output and reject files. Issues:DS-15861,DS-15862
Improved error reporting for the manage-extensions tool. Issue:DS-17080
The modifierName and modifyTimestamp attributes are now updated when offline configuration changes are made. Issue:DS-16858
Corrected the port number returned in the error message that is displayed when an administrator is trying to set up a server that is already running. Issue:DS-13721
The server now monitors important certificates used for client and inter-server communication. Certificate information is available in the Administrative Console and in the status tool output. An alarm is raised and alerts are sent when a monitored certificate is 30 days from expiration. Issue:DS-1029
Fixed an issue that could cause a sync pipe to crash due to missing attributes in the changelog. Issue:DS-17531 SF#:00609552
The Administrative Console can be deployed in an external web container, such as Tomcat, using the contents of resource/admin-console.zip, located in the server root. Issue:DS-17544
Added an optional reason parameter for dsconfig changes that will be automatically included in the server's config-audit.log file. Issue:DS-811
Updated the Server SDK to provide methods for obtaining a single LDAP connection or an LDAP connection pool with connections established to a specified LDAP external server defined in the server configuration.
Also updated the server configuration to add support for obscured values. An obscured value is a general-purpose string that is stored in an obscured form in the configuration so that its plaintext value is not readily discernible to anyone looking at the configuration file and so that the value is not displayed in administrative interfaces. The Server SDK provides a method for obtaining the plaintext representation of an obscured value, and this mechanism can be used to store potentially sensitive values in the configuration for use in Server SDK extensions without the need to store those values in the clear. Issue:DS-10694
Addressed an issue where the server would throw a NullPointerException if a com.unboundid.directory.sdk.sync.api.SyncSource implementation did not set a modifiers name in a generated ChangeRecord. Issue:DS-17653
Fixed an issue in the Data Sync Server where attribute matching rules were not correctly applied when synchronizing. Issue:DS-17652 SF#:610262
Sync correlation attributes now support correlating using JSON keys within JSON attribute values. A JSON key may be referenced with syntax <attribute name>.<JSON key> If a JSON correlation attribute is used and the JSON key does not exist for the source or destination entry or the source or destination entry considered does not have valid JSON data then the destination entry considered entry will not be matched. Issue:DS-17668
Constructed attribute mappings now support multivalued source attributes for conditional ("conditional-value-pattern" property) and non-conditional ("value-pattern" property) value patterns. Only one of the source attributes that contribute to a given value pattern may be multivalued. Issue:DS-1045
The Administrative Console is no longer compatible with older versions of the server. Issue:DS-17241
Updated the server to reduce the use of the SHA-1 message digest. The server will now use a 256-bit SHA-2 digest instead of a SHA-1 digest in all of the following cases:
- When hashing or signing a backup. - When signing an LDIF export. - When signing log data. - When generating MACs for an encrypted collect-support-data archive. - When generating unique identifiers for encryption settings definitions. - When determining whether the configuration changed with the server offline.
In all of the above cases, the server includes metadata in the output of the cryptographic processing to indicate the digest or MAC algorithm used for that processing, which ensures that the output remains compatible across server versions. For example, an LDIF export that uses a signature generated with the SHA-2 digest can be successfully imported into older versions of the server.
Also, the fingerprint certificate mapper has been updated so that it can use the 256-bit SHA-2 digest when mapping a client certificate to the corresponding user entry. The previous MD5 and SHA-1 digests remain supported.
Finally, the example enhanced password storage scheme provided with the UnboundID Server SDK has been updated so that it uses the 256-bit SHA-2 digest instead of a SHA-1 digest. Issue:DS-17444
Sync constructed attribute mappings now support extracting JSON fields within JSON attributes by appending '.' and then the JSON field to extract to the attribute name. For example, if the JSON attribute 'ubidEmailJSON' and the 'value' field is to be extracted, then 'ubidEmailJSON.value' could be specified for the attribute name resulting in '{ubidEmailJSON.value}' or '{ubidEmailJSON.value:/regex/replacement/flags}' if a regular expression is to be used. Issue:DS-17689
LDAP changelog-based Sync Sources such as the DSEE Sync Source will no longer fetch source entries if the targetDN in the corresponding changelog entry is not contained in any of the Sync Source's configured base-dn values. This can reduce the load placed on the source directory servers. Issue:DS-17742 SF#:613996
LDAP referral entries are now synchronized as a raw entry rather than trying to follow the referral. Issue:DS-17741 SF#:00613775
Constructed attribute mappings now support modifers which alter the value of referenced source attributes when added to the end of source attribute references. For example, if attribute 'mail' is to be included in a constructed JSON value then modifier 'jsonEscape' could be specified resulting in '{{ "userMail":{mail:jsonEscape} }}' or '{{ "userMail":{mail:/regex/replacement/flags:jsonEscape} }}' if a regular expression is to be used. Note that '{{' expands to '{' and '}}' to '}'. The modifiers are: * jsonEscape Escape text for use in a JSON value. * ldapFilterEscape Escape text for use in an LDAP filter. * lowerCase Convert text to lower case. * trim Remove leading and trailing whitespace. * upperCase Convert text to upper case. Issue:DS-17693
The Data Sync Server now logs an error and continues processing if it encounters a corrupted changelog entry. Previously, the server would continually throw an exception and stop processing subsequent changes. Issue:DS-17711 SF#:613375
The script files used to stop and start the server have been renamed stop-server and start-server. The older scripts are still present but may be removed in a future release of the product. Issue:DS-16789
The Data Sync Server now supports synchronizing particular fields within JSON attributes. The way fields are synchronized as well as the way source and destination values are correlated is controlled by a the "JSON Attribute" configuration object. Issue:DS-17688
Updated the Server SDK's ServerContext to expose a ValueConstructor, which can be used to build String values using a value-pattern template that references attribute values within an Entry. See the Javadoc for the ValueConstructor class with the Server SDK packaging for more information. Issue:DS-17576
The sync server will now try to use the source schema, if available, in cases where the destination schema is unavailable. Issue:DS-17936 SF#:00598940
Critical: Fixed an issue with the sync connect and response timeouts being set with incorrect units of time. Issue:DS-18026 SF#:00616763
Updated the Server SDK so that HTTPServletExtensions can be installed on Data Sync Server instances. Issue:DS-18003
Added additional logging to the Data Sync Server for cases where attribute mapping fails, but does not abort the synchronization operation. Issue:DS-17993
Limited the ACI search on collect support data tool to only pull 100 entries. This will reduce the time the tool takes to run for organizations with a large number of ACIs. Issue:DS-17968
Added additional logging for ignored synchronization operations. Issue:DS-17356
Added additional support to the Data Sync Server for synchronizing Active Directory Groups with more than 1500 members. Issue:DS-17697
Added configuration options for setting the SSL Protocol and/or the SSL Cipher Suites to the HTTPS Connection Handler. Issue:DS-10748 SF#:00003622,614777
Enhanced the HTTPS Connection Handler to send a HTTP Strict Transport Security header by default in all responses. Issue:DS-14650
Updated the default configuration of the File-Based Access Logger (logs/access) so that requests from peer Data Sync instances are no longer suppressed. This simplifies troublehooting connection and health-checking issues between server instances. Issue:DS-18199
Updated PingDirectory, PingDirectoryProxy, PingDataSync, and PingDataGovernance with the capability to run as Windows Services. Issue:DS-4161
Fixed an incompatibility between Java and PKCS12 truststores and keystores that caused an error during Data Sync Server installations. Issue:DS-18169 SF#:00619730
Enhanced Attribute Mapping in the Data Sync Server to allow users to filter out or exclude values from entries in the Sync Destination. Issues:DS-17545,DS-17546,DS-18008
Enhanced the LDAP Sync Destination to allow administrators to configure options for synchronizing user passwords with clear-text, in cases where the LDAP Server does not support pre-encoded password synchronization. Setting password-synchronization-format:clear-text on the LDAP Sync Destination will now enable the clear-text synchronization of passwords. By default, passwords will only be synchronized in the clear over a secure connection. This can be overridden by setting require-secure-connection-for-clear-text-passwords:false on the LDAP Sync Destination. Issue:DS-18136
Updated dsconfig batch mode to operate more efficiently over the WAN by consolidating the number of LDAP searches required to retrieve the full configuration when pre-validating configuration changes. Issue:DS-35495
A license key is required when setting up a server for the first time. Request a license key through the Ping Identity licensing website https://www.pingidentity.com/en/account/request-license-key.html or contact sales@pingidentity.com. Issue:DS-18100
Removed the ability to create custom HTTP trace loggers using the Server SDK. Issue:DS-18188
Important upgrade considerations for version 6.0.0.0 of the Data Sync Server:
Note: The product names have been updated to reflect the UnboundID acquisition by Ping Identity. This is a naming and branding change only; the code base is the same as in prior releases and will continue to be maintained into the future.
If upgrading the server that was running an older version of the JDK, run the dsjavaproperties --initialize command after the software upgrade to compare the settings of the older JDK with requirements for the new server software. Apply any necessary changes to the upgraded server based on previous performance settings.
The 6.0 release makes these changes to supported platforms:
PBKDF2 is now the default encoding for root passwords. This only affects new installations.
In addition to changing the default password storage scheme for root users to PBKDF2, the default password storage scheme for regular users has been changed to salted 256-bit SHA-2.
HTTPS defaults to ON: servers now default to use HTTPS for console and API connections including the SCIM API. This may affect automation scripts and development environments where HTTPS has not been in use before.
Generated user passwords, for example those created by the server during a password reset sequence, are now created as pass-phrases instead of random character strings. This makes them them easier to type and remember. This change will not affect upgrades.
The /config directory file permissions have been changed so that they are only accessible by the server user.
Customers who choose to use the optional encryption algorithms provided by the third-party BouncyCastle library are encouraged to upgrade to BouncyCastle 1.54.
These features were added for version 6.0.0.0 of the Data Sync Server:
The Data Sync Server will now recognize and correctly handle changes from Active Directory sources which include the non-standard "range=n-m" suffixes for multi-valued attributes.
Added the ability to programmatically apply attribute maps in Server Extensions so that specific maps can be applied to a cascading set of changes generated by the extension.
For Java developers whose tools and workflows make use of Maven, the Server SDK jar has been deployed to Maven Central so that a developer can now add the Server SDK as a project dependency by adding a few lines to a project's pom.xml. Also, developers can now generate a Server SDK project that Maven-aware IDEs such as IntelliJ IDEA can package into an extension bundle with no special configuration needed. This benefit extends similarly to continuous integration systems such as Jenkins.
The dsconfig tool provides the ability to search for and quickly navigate to configuration objects and properties in which the name, synopsis, or description matches a provided pattern.
A new rotate-log tool and task have been added, which can be used to trigger rotation of one or more log files.
The Configuration API is now fully supported for all servers. In this release, the API was changed to match SCIM conventions for attribute naming, resource modeling, and the standard HTTP verbs. The UnboundID SCIM 2 SDK (available through GitHub) can now be used with the Configuration API.
All servers have an updated web Administrative Console, which includes:
The new Administrative Console can also be deployed to independent application servers instead of being co-hosted by the servers. This simplifies deployment models and increases separation between data and application layers.
To assist with situations where a very large number of changes may cause disk, memory, and server start time to increase unexpectedly, alerting and gauge features have been added to the Recent Changes Database.
Servers can now trigger events whenever log file rotation occurs. This includes "copy on rotate" and "summarize on rotate" listeners, as well as Server SDK support for creating custom log file rotation listeners.
It is now possible to create, change, and remove root user accounts across the topology using the dsconfig tool and Administrative Console.
These were known issues at the time of the release of version 6.0.0.0 of the Data Sync Server:
When deploying the Administrative Console in Tomcat 8, and accessing the Administrative Console application using Tomcat's Web Application Manager, some browsers (including Safari and Firefox) will generate a path URL that encodes the dash in ubid-console. This results in a path such as http://localhost:8888/ubid%2Dconsole/, which causes session management errors. To workaround this issue, copy and paste the generated link into a browser, and replace the encoded dash with a dash (-) character.
The PermSize and MaxPermSize JVM properties are no longer supported in JDK 8, and will safely be ignored. These properties can be removed by modifying the config/java.properties file and running "bin/dsjavaproperties" while the server is offline.
Security criteria for root passwords with the default configuration will be increased in a future release. This might affect automated installation scripts that currently use less secure passwords. This will not affect existing root accounts.
The dsconfig tool and the Administrative Console enables creating and managing new Root DN users in this release. However, there is a limitation with changing the password of the currently logged in administrator. The ldappasswordmodify command can be used to change the administrator's password by providing the current and new password.
These issues were resolved with version 6.0.0.0 of the Data Sync Server:
Updated the Active Directory Data Sync Source so that it correctly handles the "range" options on attributes returned with the DirSync control. Issue:DS-14765 SF#:00003110
Updated command-line tools based on the LDAP SDK tool APIs to add the following features:
The resync tool now supports sync pipes that use notification mode. Issue:DS-14606
Added a --prettyPrint option to the config-diff tool to make the output more human-readable. Issue:DS-14694
Improved memory utilization when processing entries with very large attributes, to prevent possible data retention in memory. Issue:DS-14878 SF#:00003169
Fixed an issue with the dsjavaproperties tool where java properties for PermSize and MaxPermSize could be added when using JDK 8, which no longer supports these options. Issue:DS-14857 SF#:3187
Updated the bcrypt, crypt, PBKDF2 and scrypt password storage schemes so they can be used to create new instances. Issue:DS-14923
A new API, applyMaps(), is available for Data Sync Server plugins to programmatically apply Sync maps. Issue:DS-14298 SF#:00002971
Updated the Apache commons collections library to address the security vulnerability described by CVE 2015-4852. Issue:DS-14430 SF#:00003216
Fixed a case where attribute syntax configuration changes would not apply to undefined attributes, which rely on default attribute types. Issue:DS-14979
Collect-support-data tool now captures Kerberos config and log information. Issue:DS-13823
Updated the server's support for the Twilio Messaging Service so that it uses the newer "Messages" API when sending SMS messages instead of the older "SMS" API. The older API has been deprecated, and Twilio now imposes a 120-character limit for messages sent via that API. The messages API allows the server to take advantage of the full 160 characters per SMS message. Issue:DS-14749
Fixed an issue where destination attributes with hard-coded values were incorrectly excluded during a resync, when the --excludeSourceAttr option was used. Issue:DS-15087
Server SDK extensions are now built with a Java source version of 1.7 by default. Issue:DS-15015
Changed interactive setup default value for HTTPS enablement. Issue:DS-15221
Updated the initial server configuration to improve security and usability. These changes apply only to new installations and will not be applied when updating an existing installation. Changes include:
Updated the default file permissions for new installations on UNIX-based systems. Files and directories included in the zip file will be only be accessible to their owner (the user that unzipped the file) by default.
Newly-created files and directories will also be assigned permissions that allow them to be accessed only by the account used to run the server. Existing configuration options for setting file permissions (the log-file-permissions and db-directory-permissions properties) will continue to behave as before. The new config/server.umask file will control the default permissions for all other newly-created files and directories. Issues:DS-13571,DS-13860,DS-7505 SF#:2703
It is now possible to include literal curly brackets ('{}'s) in value patterns and conditional value patterns of constructed attribute mappings by doubling them. Literal '{' and '}' are specified by '{{' and '}}' respectively. Issue:DS-15187
Updated the global ACIs that ship with the server to use a separate ACI for each control or extended request to allow by default, rather than grouping all desired controls together in one ACI and all desired extended requests together in a second ACI. This change will only be reflected in new installations, and not when updating an existing deployment. Issue:DS-15417
Addressed an issue where dsconfig incorrectly allowed certain configuration objects to be deleted. Issue:DS-15400
Provided a graphical tool, watch-entry, that is intended to demonstrate replication or synchronization latency by watching an LDAP entry for changes. If the entry changes, then the background of modified attributes will temporarily be red. Attributes can also be directly modified as well. Issue:DS-15437
Updated interactive setup to display default values, and improved the overall layout and appearance. Issues:DS-15361,DS-15363,DS-15434
Added more logging information when initializing web application and servlet extensions in case an extension causes conflicts or delays. Issue:DS-15466
Updated setup to encode the root password with the PBKDF2 password storage scheme instead of SSHA512. Issue:DS-15521
Increase the minimum memory requirements for the server process from 256MB to 384MB to accommodate the Administrative Console. Issue:DS-15571
Updater tool will increase PermSize and MaxPermSize parameters to recommended value to prevent Java JVM pauses. Issue:DS-15522 SF#:00003324
The Data Sync Server now uses the "max-connection-age" configuration property to limit the age of LDAP connections to external servers. Issue:DS-15513 SF#:00003309
Fixed an error that could occur during upgrade when the configuration can not be loaded due to missing custom schema. Issue:DS-15592 SF#:3340
Updated the Groovy Scripting Language version to 2.4.6. Issue:DS-15621
The prepare-endpoint-server CLI included with the Data Sync Server will now correctly set permissions when the Synchronization user already exists. Issue:DS-15670
The Data Services Markup Language (DSML) client and gateway components have been discontinued and are no longer available. Issue:DS-15753
Added a new rotate-log tool to request the rotation of one or more log files. Issue:DS-10464
Improved the error messages produced by the manage-extensions tool when attempting to install invalid extensions. Issue:DS-15412
Improved the error messages and examples for create-rc-script and create-systemd-script by explicitly suggesting the use of sudo so that the scripts can modify protected files. Issue:DS-15178
The former suite of Administrative Console applications, each of which were tied to a particular product (for example the dsconsole.war for the Directory Server) are no longer available, and have been superceded by a new version of the Administrative Console capable of managing any server product. You can choose to access the Administrative Console by hosting it within a server, or by deploying it in an external servlet container. For the former, enable an HTTP Connection Handler and add the Administrative Console Web Application Extension to the handler. For the latter, download and unzip the management-console-[version].zip file, and install the ubid-console.war file according to your container's instructions. Issue:DS-15088
Root DN User configuration entries can now be fully managed through the configuration management interfaces such as dsconfig and the Administrative Console. Issue:DS-15422
Added support for log file rotation listeners, which allow for custom processing whenever a log file is rotated out of service so that the server will no longer write to it. A copy listener (which will copy the rotated log file to an alternate location, optionally compressing it in the process), and a summarize listener (which will invoke the summarize-access-log tool on the rotated log file) are included. The Server SDK also includes an API for creating custom log file rotation listeners. Issue:DS-4235
Replaced the scramble-ldif tool with a more powerful transform-ldif tool with support for a number of additional transformation types. The new transform-ldif tool is backward compatible with the former scramble-ldif tool, and the scramble-ldif shell script and batch file are still included with the server to ensure compatibility with scripts that depend on that tool. Issue:DS-15108
Improved the collect-support-data tool to include information provided by systemd on platforms that support it. Issue:DS-13401
Improved the error messages for create-rc-script and create-systemd-script when the directory in which the script will be created does not exist. Issue:DS-15337
Added the server's process ID to the output of the status tool. Issue:DS-10312
Added a monitor entry for each Server SDK extension. Issue:DS-14548
The Configuration API now returns unquoted, native Javascript values for integer, real number, and boolean properties. Duration and size property values, for example '1 w' or '100 G', continue to be represented as Javascript string types. Issue:DS-15175
Added the ability to create local constants in LDIF template files using the new 'local' keyword. Issue:DS-14213
Addressed a few issues in config-diff. In some situations, config-diff would not generate commands in an order that respected all dependencies. This has been fixed. Most expected warnings are now excluded by default but can be included in the output with the --includeAllWarnings option. The --sourceBindPasswordFile and --targetBindPassword are now applied in conjunction with the --targetConfigGroup and --sourceConfigGroup options. Issues:DS-10466,DS-10765,DS-14479,DS-15318,DS-16154
Added support for setting the request header size in the Jetty http configuration server properties. Issue:DS-12191 SF#:00002580
Updated the restore command so that it can no longer be used to restore a backup of the config backend. The command now points the administrator for safer ways to revert configuration changes, including using config-diff. Issue:DS-14704
Updated the sanitize-log tool to add support for JSON-formatted access and error log files. Issue:DS-16224
Added the ability to search for configuration objects and their properties by name with the dsconfig tool. Issue:DS-979
Added a --dry-run option to dsconfig, which can be used in batch mode to validate the configuration changes in a batch file without applying them. Issue:DS-10946
Tools used to prepare a server for access by another server, such as prepare-external-server, now validate base DN entries before any modifications are performed on the prepared server. Issue:DS-14807
Fixed an issue that prevented the deletion of disabled debug loggers. Issue:DS-15622
Sensitive attribute values can now be redacted in the sync and sync-failed-ops log files. By default, the value of encoded passwords are redacted. The redaction can be configured with the log-redaction-regex property in the global Sync configuration. Issue:DS-16728
These issues were resolved with version 5.2.0.0 of the Data Sync Server:
The setup tool has been updated to use HTTPS for initial configuration. Unsecure HTTP can be enabled post-setup, or by using non-interactive setup. Issue:DS-12182
Addressed cases where some messages may be suppressed in logs and alerts. Issue:DS-12287
Updated the Configuration API output where properties and their values are listed to include those that are undefined. Issue:DS-12123
Added support for running on Oracle Java 8 and OpenJDK 8 platforms. Issue:DS-12483
The Configuration API has been updated to support filtering, sorting, and paging for object list operations. See the Administration guide for usage. Issue:DS-12245
Added logging of all HTTP requests disallowed due to CORS. This should make it easier to debug HTTP 403/Forbidden errors. Issue:DS-12496
The server can now detect an "out of file handles" situation on the operating system, and shut down to prevent running in an unreliable state. Issue:DS-12579 SF#:2655
Update the Detailed HTTP Operation Log Publisher to log the correct return code (404 NOT FOUND) when a request is not handled by defined endpoints. Issue:DS-12576
Fixed an issue caused by a JVM bug where a Sync Pipe becomes blocked after the error, "AckHandler#completed called with a value that's not pending," is reported to the error log. Issue:DS-12708 SF#:2680
Fixed an issue where changes to SMTP External Server configurations did not take effect until after a server restart. Issue:DS-12285
The Sync Source ignore-changes-by-dn configuration parameter now defaults to "cn=Sync User,cn=Root DNs,cn=config," which is the default synchronization user. This is done to prevent changes from being synchronized in a circular manner. Issue:DS-10558
Now when setting the synchronization start point via the "set-startpoint" command it's no longer necessary that global synchronization be turned on. Issue:DS-5425
The resync command has been updated to include the "Recent ops/second" statistic, which displays the operations per second for the last ten second iteration. The "Average ops/second" statistic has also been renamed to "Overall ops/second" to be consistent with other tools. Issue:DS-12782
Improved server locking used by dsconfig in offline batch mode, so that the server lock is held for the entire batch duration, instead of for each invocation. Also, reduced the probability of contention for file locks used by server tools to determine the server status. Issue:DS-12969 SF#:2717
Fixed a rare condition that might cause the logger rotation and retention thread to exit under heavy file system load or a network file system outage. Issue:DS-12880
MakeLDIF templates now have the ability to escape special characters curly braces, angle brackets, and square brackets using a backslash. See config/MakeLDIF/examples-of-all-tags.template for further examples. Issue:DS-12798
Constructed mappings now support mappings that are conditional with the "conditional-value-pattern" dsconfig parameter. Each parameter consists of an LDAP filter and a pattern. The LDAP filter must be matched for the pattern to be used. Issue:DS-11936
Added a gauge to monitor the number of available file descriptors. This Available File Descriptors gauge can detect if a server if running out of file descriptors and degrade the server appropriately. Issue:DS-12727
The setup and initial configuration tools now support offline modes that can be used to bootstrap the server configuration while it is not running. Also, files generated by theses tools are now saved to the server's resource directory. Issues:DS-12704,DS-8794,DS-9652
Addressed an issue where data definition language (DDL) log field mappings for the JDBC error log were not previously documented. Issue:DS-13163
Updated the resync command to display more error messages to the console and to a log file. The location of this log file defaults to <server-root>/logs/tools/resync-errors.log but can be overridden with the --consoleLogFilePath option. Issue:DS-10447
Reduced the memory overhead of debug logging in high throughput environments by sharing logging buffers across multiple threads. Issue:DS-10010
Updated the server's JVM arguments to always log garbage collection information to a rotating set of log files stored within logs/jvm/gc.log.N. The file system usage is limited to 300MB. If the server had previously been configured with VERBOSE_GC, then garbage collection logging information will no longer be logged to logs/server.out. Issue:DS-11522
The following UnboundID product names have changed: - Identity Data Store to Data Store - Identity Proxy to Proxy Server - Identity Data Sync Server to Data Sync Server - Identity Broker to Data Broker Issue:DS-12799
Updated the prepare-external-server tool to suppress output when run with the --quiet option. Issue:DS-13242
When DSEE servers are prepared for synchronization, the nsslapd-changelogmaxentries DSEE configuration property is no longer set to 500,000. This is so that the size of DSEE's Retro changelog is limited entirely by the nsslapd-changelogmaxage configuration property. Issue:DS-2880
Fixed a bug where using the advanced arguments of some tools would result in changing the saved complexity settings for the dsconfig tool. Issue:DS-12897
Added a new search-logs tool. Similar to the command line tool 'grep,' this tool searches across log files to extract lines matching the provided pattern(s). The search-logs tool can handle multi-line log messages, extract log messages within a given time range, and include rotated log files. Issue:DS-3095
Updated the create-systemd-script tool by adding resource limits for available open file descriptors (NOFILE), and shared memory reservations (MEMLOCK). The generated script lists the recommended file descriptors limit and the resource limit setting for enabling large page support. The settings in the create-systemd-script output supersedes prior documentation for setting the number of open file descriptors on non-systemd systems. Issue:DS-13678
Fixed an issue with the resync command, where entries that were already synchronized were falsely reported as being modified. This occurred with Active Directory Sync Destinations. Issue:DS-13718
Fixed an issue where stale state for remote sync pipes could be merged repeatedly with the active Data Sync Server after failover, or when starting and stopping the sync pipes. When a sync pipe is deleted, its saved sync pipe state will now be removed. Issue:DS-13769 SF#:00002859
Updated interactive dsconfig to include an option to toggle between sorting similar properties together or sorting them alphabetically. Issue:DS-1706
The collect-support-data tool now has the option to collect logging information within a specified time range via the '--timeRange' argument. Issue:DS-1261
Improved the server's support for selecting TLS cipher suites. When the server is configured to use a specific set of cipher suites, it will now always validate that all of the configured suites are supported by the JVM. When the server is not configured to use a specific set of cipher suites, it will now customize the set of default suites to prioritize those using strong cryptography (especially those that offer forward secrecy), and exclude suites with known weaknesses. Issues:DS-12681,DS-13475
Updated the alert handler configuration to indicate whether the alert handler should be invoked asynchronously in a background thread rather than by the thread that generated the alert. For alerts generated during the course of processing an operation, invoking potentially time-consuming alert handlers in a background thread can avoid adversely impacting the response time for that operation while still ensuring that administrators are made aware of the issue that arose. Issue:DS-12833
Updated the server to provide support for SMTP connection pooling. When sending an email message, the server will attempt to reuse an existing SMTP connection rather than establishing a new connection for each message. Issue:DS-12833
Fixed an issue where a set-startpoint command could fail when synchronizing from an Active Directory LDS source. Also, when synchronizing changes to an Active Directory LDS instance, the userAccountControl attribute is no longer included when creating new entries. Issue:DS-14061 SF#:00002925
Updated the prepare-* tools to avoid unnecessary confirmation for trust of the prepared server's certificate when the --trustStorePath argument specifies a trust store that establishes trust. Issue:DS-12616
Fixed an issue where an active Sync Server could accept a smaller change number when merging remote state for a changelog Sync Source. Issue:DS-14186 SF#:00002859
Fixed a log publisher defect that would result in an unreadable file when both compression and signing were enabled at the same time. Issue:DS-13552
The ldifsearch command now supports the option "---isCompressed" for LDIF files that have been compressed with gzip. Issue:DS-14140
Added properties to the task backend for limiting the number of log messages retained in task entries, in order to limit the size of the in-memory representation of those entries. All log messages generated by a task will still be recorded in the server error log, even if they are not all retained in the corresponding entry in the task backend. Issue:DS-11067 SF#:2282
These were known issues at the time of the release of version 5.1.5.2 of the Data Sync Server:
Delete operations might not be detected and synchronized when the UnboundID Data Sync Server is configured to only synchronize a subset of attributes from an UnboundID Data Store or Proxy Server. To ensure that delete operations are properly detected and synchronized, add "*" to the changelog-deleted-entry-include-attribute configuration property on the changelog backend in the Data Store.
These issues were resolved with version 5.1.5.2 of the Data Sync Server:
Fixed an issue where a set-startpoint command could fail when synchronizing from an Active Directory LDS source. Also, when synchronizing changes to an Active Directory LDS instance, the userAccountControl attribute is no longer included when creating new entries. Issue:DS-14061 SF#:00002925
These issues were resolved with version 5.1.5.0 of the Data Sync Server:
Fixed an issue with the resync command, where entries that were already synchronized were falsely reported as being modified. This occurred with Active Directory Sync Destinations. Issue:DS-13718
Added a gauge to monitor the number of available file descriptors. This Available File Descriptors gauge can detect if a server if running out of file descriptors and degrade the server appropriately. Issue:DS-12727
Important upgrade considerations for version 5.1.0.0 of the Data Sync Server:
The summarize-config tool is deprecated, and will be removed in future versions of the product. Use the config-diff tool with the "sourceBaseline" argument to list a summary of changes to the local server configuration.
These features were added for version 5.1.0.0 of the Data Sync Server:
Identity Data Sync attribute mappings now support LDAP-filter syntax conditionals (the conditional-value-pattern property), which enables more flexible attribute-to-attribute mapping based on attribute values found in the incoming data.
These issues were resolved with version 5.1.0.0 of the Data Sync Server:
The setup tool has been updated to use HTTPS for initial configuration. Unsecure HTTP can be enabled post-setup, or by using non-interactive setup. Issue:DS-12182
Addressed cases where some messages may be suppressed in logs and alerts. Issue:DS-12287
Updated the Configuration API output where properties and their values are listed to include those that are undefined. Issue:DS-12123
Added support for running on Oracle Java 8 and OpenJDK 8 platforms. Issue:DS-12483
Added logging of all HTTP requests disallowed due to CORS. This should make it easier to debug HTTP 403/Forbidden errors. Issue:DS-12496
Critical: The server can now detect an "out of file handles" situation on the operating system, and shut down to prevent running in an unreliable state. Issue:DS-12579 SF#:2655
Update the Detailed HTTP Operation Log Publisher to log the correct return code (404 NOT FOUND) when a request is not handled by defined endpoints. Issue:DS-12576
The Configuration API has been updated to support filtering, sorting, and paging for object list operations. See the Administration guide for usage. Issue:DS-12245
Fixed an issue caused by a JVM bug where a Sync Pipe becomes blocked after the error, "AckHandler#completed called with a value that's not pending," is reported to the error log. Issue:DS-12708 SF#:2680
Fixed an issue where changes to SMTP External Server configurations did not take effect until after a server restart. Issue:DS-12285
Fixed a rare condition that might cause the logger rotation and retention thread to exit under heavy file system load or a network file system outage. Issue:DS-12880
MakeLDIF templates now have the ability to escape special characters curly braces, angle brackets, and square brackets using a backslash. See config/MakeLDIF/examples-of-all-tags.template for further examples. Issue:DS-12798
Improved server locking used by dsconfig in offline batch mode, so that the server lock is held for the entire batch duration, instead of for each invocation. Also, reduced the probability of contention for file locks used by server tools to determine the server status. Issue:DS-12969 SF#:2717
Constructed mappings now support mappings that are conditional with the "conditional-value-pattern" dsconfig parameter. Each parameter consists of an LDAP filter and a pattern. The LDAP filter must be matched for the pattern to be used. Issue:DS-11936
Updated the resync command to display more error messages to the console and to a log file. The location of this log file defaults to <server-root>/logs/tools/resync-errors.log but can be overridden with the --consoleLogFilePath option. Issue:DS-10447
Reduced the memory overhead of debug logging in high throughput environments by sharing logging buffers across multiple threads. Issue:DS-10010
These issues were resolved with version 5.0.1.0 of the Data Sync Server:
Addressed cases where some messages may be suppressed in logs and alerts. Issue:DS-12287
Updated the Configuration API output where properties and their values are listed to include those that are undefined. Issue:DS-12123
The server can now detect an "out of file handles" situation on the operating system, and shut down to prevent running in an unreliable state. Issue:DS-12579 SF#:2655
Fixed an issue caused by a JVM bug where a Sync Pipe becomes blocked after the error, "AckHandler#completed called with a value that's not pending," is reported to the error log. Issue:DS-12708 SF#:2680
These features were added for version 5.0.0.0 of the Data Sync Server:
Java 7 is now required when setting up a new server or upgrading an existing server.
Enabled support for the SSLv2Hello TLS protocol by default in JVMs that support it. This does not enable support for the insecure SSLv2 protocol, but it can improve compatibility with clients running older versions of Java that may start TLS negotiation with an SSLv2 client hello packet before negotiating to a higher version of the TLS protocol. Support for SSLv2Hello in the initial phase of negotiation does not in any way compromise the strength of the integrity and/or confidentiality protection that is ultimately negotiated between the client and the server.
Added a Monitor History plugin that periodically records cn=monitor to timestamped files to aid in isolating intermittent problems. By default, it logs the full cn=monitor branch every five minutes to compressed files within logs/monitor-history/. Files are deleted automatically, but a sparse set of older files are kept to provide historical perspective on server performance. The collect-support-data tool has also been updated to collect a few of these files to aid in root cause analysis.
Introduced the Configuration HTTP Servlet Extension, which can be used for querying and updating the configuration over a REST API. This feature is currently experimental and is subject to change in the future. Your feedback is welcome.
These issues were resolved with version 5.0.0.0 of the Data Sync Server:
Fixed an issue with the prepare-endpoint-server tool, where the maxChangelogAge argument was not being applied when targeting UnboundID servers. Issue:DS-11487 SF#:2406
Fixed the gauge configuration manager to only re-initialize the gauge that was changed, and not any of the other gauges that did not change. Issue:DS-11472
Fixed the alarm manager to generate alarm-cleared alerts when internal alarms are cleared and the alarm manager's generated-alert-types property has the "alarm" value. Issue:DS-11541 SF#:2421
Fixed the alarm manager to not include the details of the old alarm, (the alarm being cleared), in the "alarm-cleared" alert message. Issue:DS-11546
Fixed the dsconfig tool to suppress all stray output when run in batch mode with the --quiet option. Issue:DS-10460
Fixed an issue in which tools such as dsconfig, status, and dsreplication could not connect to the server over SSL or StartTLS. This occurred when a certificate was accepted with the 'Manually validate' option, while using the interactive LDAP connection menu. Issue:DS-11688
Updated the alarm manager to not generate "alarm-normal" alert when a gauge's condition abates Issue:DS-11637
Reduced the severity of the "unrecognized alert type" message in the error log from SEVERE_WARNING to NOTICE. The message now states that this is expected if the server is reverted to a version prior to the implementation of these alert types. Issue:DS-11453
Removed the "alarm-normal" alert. Issue:DS-11730
Updated the alarm manager to not persist normal alarms. Issue:DS-11719
Updated the ExampleOverloadHandlerPlugin to monitor the alarm backend for delete actions, so that it can react appropriately to abating gauge conditions. Issue:DS-11719
Critical: Disabled support for SSLv3 by default in the LDAP, HTTP, and JMX connection handlers, and for replication communication. The recently-discovered POODLE vulnerability could potentially allow a network attacker to determine the plaintext behind an SSLv3-encrypted session, which would effectively negate the primary benefit of the encryption.
SSLv3 was initially defined in 1996, but was supplanted by the release of the TLSv1 definition in 1999 (and subsequently by TLSv1.1 in 2006 and TLSv1.2 in 2008). These newer TLS protocols are not susceptible to the POODLE vulnerability, and the server has supported them (and preferred them over SSLv3) for many years. The act of disabling SSLv3 by default should not have any adverse effect on clients that support any of the newer TLS protocols. However, if there are any legacy client applications that attempt to communicate securely but do not support the newer TLS protocols, they should be updated to support the newer protocols. In the event that there are known clients that do not support any security protocol newer than SSLv3 and that cannot be immediately updated to support a newer protocol, SSLv3 support can be re-enabled using the newly-introduced allowed-insecure-tls-protocol global configuration property. However, since communication using SSLv3 can no longer be considered secure, it is strongly recommended that every effort be made to update all known clients still using SSLv3.
It is possible to use the server access log to identify LDAP clients that use SSLv3 to communicate with the server. Whenever an LDAP client establishes a secure connection to the server, or whenever a client uses the StartTLS extended operation to secure an existing plaintext connection, the server will generate a SECURITY-NEGOTIATION access log message. The "protocol" element of a SECURITY-NEGOTIATION access log message specifies the name of the security protocol that has been negotiated between the client and the server, and any SECURITY-NEGOTIATION messages with a protocol of "SSLv3" suggest that the associated client is vulnerable to the POODLE attack. In addition, if any connections are terminated for attempting to use the disallowed SSLv3 protocol, the access log message for that disconnect should include a message stating the reason for the termination. Issue:DS-11782
Updated the Web Console so that upon login, the user's old session is always invalidated. Issue:DS-11624
Updated the Web Console to suppress LDAP responses in user messages, such as when the server is unavailable or for authentication failures. Also added a context parameter to exclude stack traces and detailed error messages from appearing in the application's internal error page. Issues:DS-11629,DS-11645
Fixed incorrect property references for trustStorePassword and keyStorePasswordFile in tools.properties that corresponded to the wrong argument names. Issue:DS-11751
Updated the setup tools to enable definition of external server instances that are configured to reject unauthenticated requests. Previously the tools would erroneously indicate these servers were unavailable. Issues:DS-11068,DS-11784,DS-11887
Increased the possible parallelism within a Sync Pipe by skipping past the operation at the head of the incoming queue, if it cannot be processed, because it depends on an uncompleted active operation. This increases the overall throughput of a Sync Pipe when the stream of incoming changes includes many dependent operations, which must be processed in order. Examples of dependent operations include changes to the same entry or changes to entries that have a parent child relationship. With these changes, the Sync Server still guarantees that dependent operations are processed in order. Issue:DS-11947 SF#:00002516
Disabled log rotation during startup to prevent potential problems with rotation dependencies on server components that have not yet been initialized. Issue:DS-10441
Added a gauge to the server to track JVM memory usage and alert if the amount of free memory gets low enough that it could impact server performance. Issue:DS-11993
Updated the server to make it easier to control the order of values in the ssl-protocol and ssl-cipher-suite properties in the LDAP connection handler and crypto manager configuration objects. Issue:DS-12147
Updated the HTTP Connection Handler to return a 404 Not Found response to requests for endpoints not handled by any servlet or web application extensions. Previously the hander would return a 200 OK with no response body. Issues:DS-12120,DS-8368
These features were added for version 4.7.0.0 of the Data Sync Server:
Updated the server to support Alarms. An alarm represents a stateful condition of the server that might indicate a problem, such as low disk space or external server unavailability. The status command line utility and the monitoring page of the web console have been updated to expose the active alarms. Many existing alert types have been updated to be treated as alarms. When the condition associated with an alarm abates, the alarm is cleared.
Added support for Gauges. A gauge examines specific server monitoring data, and raises an alarm when a configured threshold is crossed. The server has out-of-the-box gauges such as CPU Usage and Disk Busy, and new ones can be added through the Gauge Data Source and Gauge configuration object types.
OpenJDK 7 is now supported on Linux.
These were known issues at the time of the release of version 4.7.0.0 of the Data Sync Server:
JDK 6 is currently deprecated and will not be supported in the next major release.
UnboundID products are not supported on JDK 8.
These issues were resolved with version 4.7.0.0 of the Data Sync Server:
Fixed the web console so that attempts to reconnect (after the console is restarted) succeed. Issue:DS-11043
Updated the server preparation tools to use secure communication when setting up a Data Store for access over TLS. Previously the tools may fail when the server is configured to reject insecure requests. Issues:DS-11058,DS-6200
Added a result code tracker that maintains a monitor entry with counts and response times of results. Each result is categorized by operation type, post-response result code, and whether it is a failure or non-failure. Issue:DS-3270
If a Sync Class is configured with the ignore-zero-length-values property set to true, values that are empty after attribute mapping are now ignored. Issue:DS-11293
The Password Sync Agent for Active Directory no longer restricts installation on specific versions of Windows. The admin guide will contain the current list of supported versions, and support for new releases may be requested as needed. Issue:DS-11373 SF#:2384
Fixed a problem that prevented the server from starting if a TLS-enabled connection handler was configured with a certificate nickname that referenced a non-RSA certificate. Issue:DS-10949
These were known issues at the time of the release of version 4.6.0.0 of the Data Sync Server:
UnboundID products, Java SE, and the JVM do not use OpenSSL libraries and are therefore not vulnerable to OpenSSL issues. Oracle has provided a statement on the April 2014 OpenSSL Heartbleed vulnerability at http://www.oracle.com/technetwork/topics/security/opensslheartbleedcve-2014-0160-2188454.html. Issue:DS-10807
These issues were resolved with version 4.6.0.0 of the Data Sync Server:
Updated the validate-file-signature tool to ensure that it will always display a final summary message to indicate whether any warnings or errors were encountered during processing. Issue:DS-10333
Updated the signed logging implementation to better handle any problems that may arise during cryptographic processing. If any such problem is encountered, the server will now include a message with information about the error in the signature block rather than suspending the logger with an exception recorded in the server.out log file. Issue:DS-10310
Fixed an issue in the Periodic Stats Logger, where no logging would occur when suppress-if-idle=true was configured, even when the server was not idle. Issue:DS-10387 SF#:2170
Added a new sanitize-log tool that can be used to remove sensitive information from server log files, including the file-based access log, the operation timing access log, the file-based error log, the file-based sync log, the file-based resync log, and the detailed HTTP operation log.
The sanitization process operates on fields that consist of name-value pairs. The field name and equal sign will always be retained, but in cases where the value may contain sensitive data, that value may either be replaced with the string "---REDACTED---", or it may be tokenized. If the tokenized value is a DN or filter, then attribute names in that DN or filter will be preserved while the values will be replaced with a string consisting of a number inside curly braces. If the tokenized value is not a DN or filter, then the entire value will be replaced with a number inside curly braces. If a string to be tokenized appears multiple times in the log, the same replacement token will be used for each occurrence of that string to make it possible to correlate occurrences of that string without revealing the actual content.
The sanitize-log tool has a default configuration that should be sufficient for many environments, allowing it to tokenize or redact sensitive information while preserving non-sensitive content for use in diagnosing problems or understanding usage patterns. However, this behavior can be customized using command-line arguments by indicating whether to preserve, tokenize, or redact a given log field. Issue:DS-10472
Fixed issues with the JDBC Access Logger that were related to Oracle Thin Client, where column values were "null" and disabling the logger resulted in losing a connection to the server when using the dsconfig command. Issue:DS-10485
Update the server to support pulling virtual attribute values from delete operation changelog entries, which enables using these virtual attributes reliably as destination-correlation-attributes and/or destination-correlation-attributes-on-delete settings Issue:DS-10253 SF#:2150
Fixed an issue so that collect-support-data now generates filename entries correctly. Previously, the tool would hang if the archiving of files following a symbolic link required generating a non-duplicating filename entry. Issue:DS-10582
Enabled the Host System Monitor Provider by default so that system CPU and memory utilization will be reported automatically through the server's monitoring framework. Disk and network monitoring can be enabled by configuring values for the disk-devices and network-devices configuration properties. Issue:DS-10562
The default timeout period for smtp-timeout was changed from none to two minutes to prevent non-responsive mail servers from disrupting administrative functions. Issue:DS-10230
Updated configuration object descriptions and menu items to reflect that Directory Server Enterprise Edition support applies to both Sun and Oracle branded versions of the product. Issue:DS-10449
The setup command no longer saves user-provided key store and trust store passwords in PIN files. Passwords provided during setup are encrypted with the configuration data. If the administrator chooses to use PIN files to supply the passwords, the files are referenced in the server configuration by the key manager and trust manager. Issue:DS-10787
Updated the access logger so that result messages include user-friendly names for result codes in addition to their numeric values. Issue:DS-9946
Updated the Periodic Stats Logger to include an empty value in the output rather than "infinity" in certain circumstances. This avoids problems plotting the output in a spreadsheet. Issue:DS-8842
Updated dsconfig to treat tabs as whitespace in batch files. Issue:DS-10549
Added Metrics Collection Size Limit Retention Policy to the metrics backend to allow up to 2 GB of metric data to be buffered locally, which allows the Metrics Engine to be offline for a longer time without missing collected data. Issue:DS-10156
Removed deprecated "lshal" command from Linux-specific processes performed by the collect-support-data tool and added similar command, "udevadm info --export-db" Issue:DS-10713
Updated the Replication Servers table produced by the dsreplication tool to omit unnecessary "Security" column. Issue:DS-10442
These features were added for version 4.5.1.0 of the Data Sync Server:
The collect-support-data tool now refers to tools.properties for default command-line options.
The collect-support-data tool now supports an option to encrypt the data archive, to ensure protection of customer data while in transit, and an option to reduce the amount of potentially sensitive data that is collected.
Cross-origin Resource sharing (CORS) support is now included for HTTP Servlet Extensions, including the SCIM RESTful APIs.
These were known issues at the time of the release of version 4.5.1.0 of the Data Sync Server:
When the Velocity servlet receives CORS-enabled requests and has a cross-origin policy in effect, it will return multiple Access-Control-* headers with duplicate values. This will cause cross-origin requests issued by web browsers to fail. Issue:DS-10205
These issues were resolved with version 4.5.1.0 of the Data Sync Server:
Update the make-ldif tool to no longer assign the objectClass value of extensibleObject to branch entries. If needed, "objectClass: extensibleObject" can be added explicitly to the branch definition. Issue:DS-8530
Enhance the ignore-zero-length-values configuration option for Sync Class during add operations to not include a destination attribute when the source attribute has only zero length values, or no values at all. Issue:DS-10021 SF#:2105
Enhance the ignore-zero-length-values configuration option for Sync Class during "resync" for add operations to not include a destination attribute when the source attribute has only zero length values, or no values at all. Issue:DS-10046 SF#:2115
The setup tool's --aggressiveJVMTuning and --verboseGC command-line options have been deprecated. Instead, use --jvmTuningParameter AGGRESSIVE and --jvmTuningParameter VERBOSE_GC respectively. Issue:DS-9079
Update the server configuration to use a new default limit for duplicate alert suppression. The previous default imposed a maximum of 100 alerts of the same type per hour. The new default imposes a maximum of 10 alerts of the same type every ten minutes. This is more likely to suppress bursts in which the same alert is repeatedly generated over a short time without interfering with multiple occurrences of alerts of the same type over a longer period of time. Issue:DS-9259
Update the Server SDK convenience method TransactionContext#searchToRawEntry to support parsing java.util.Date column types including subclasses such as java.sql.Timestamp. This enables natively parsing columns with a DATE type defined in Oracle databases. Issue:DS-9995
Update the server to improve the caching behavior for PIN files as used by key and trust manager providers. In the case that the keystore or truststore file has been updated to require a new PIN and the existing PIN file is updated without a configuration change to the associated key or trust manager provider, the server would previously keep trying to use the old PIN. It will now look for and use an updated PIN if a failure is encountered while using the old PIN. Issue:DS-10113 SF#:2123
Update the collect-support-data tool so that it can encrypt the data that is captured to protect it from unauthorized third parties. The encryption key is generated from a passphrase which may be read from a file, interactively provided by the user, or dynamically generated by the tool. This passphrase must be provided to support personnel (ideally over a different communication channel than the encrypted support data archive itself) for them to be able to access the information it contains.
There is also a new option to decrypt an encrypted collect-support-data archive when provided with the encryption passphrase. Issue:DS-10129
Update the collect-support-data tool so that it is possible to configure default values for most arguments in the tools.properties file. Issue:DS-10178
Fix an issue in the Active Directory Password Sync Agent to prevent resending detected changes to the Identity Data Sync Server in situations where the server is not able to process changes because the Sync Destination is unavailable. Issue:DS-10166 SF#:2097
Update the collect-support-data tool to further reduce the possibility of gathering sensitive information. Potentially sensitive data will be replaced with ---REDACTED--- in the output. A new "--securityLevel maximum" option can also be specified that redacts DNs and search filters, which might include personally identifiable information. Issue:DS-10115
These features were added for version 4.5.0.0 of the Data Sync Server:
A new config-diff command line utility can compare two server configurations and produce the difference as a dsconfig batch file. The file can then be used to bring the source configuration in line with the target. Comparisons can be done between live servers or configuration files, and between current or legacy configurations. Run 'config-diff --help' to get more information including example use cases.
These were known issues at the time of the release of version 4.5.0.0 of the Data Sync Server:
Java 1.7 has a synchronization bottleneck in HashMap that severely impacts performance. Use update 1.7u40, if possible, to avoid this issue. Issue:DS-9477
These issues were resolved with version 4.5.0.0 of the Data Sync Server:
Fix a bug in the JDBC Access Logger that could cause incompatibility with some database versions and display a "Cannot commit when autoCommit is enabled" error message. Issue:DS-8750
Update the server startup process so that if no messages have been logged for at least five minutes, the server will generate and log a message about the current phase of startup processing. This can help reassure administrators that the server is still starting and provide information about what phase of startup may be taking so long. Issue:DS-7450
Update java.properties generation so that comments related to alternative JVM tunings are no longer present in the file. In most cases, rather than updating java.properties by hand you should use the dsjavaproperties tool to generate JVM options. Issue:DS-8339
Add an allow-insecure-local-jmx-access option to the global config that will expose JMX data via insecure local JVM connection Issue:DS-4300
Add a new alert handler that can use the Twilio service to deliver administrative alerts via SMS. Long alerts may be either truncated or split into multiple SMS messages. Issue:DS-5587
Update the configuration schema to make the ds-cfg-inherit-default-root-privileges attribute mandatory for object class ds-cfg-root-dn-user which is used to define Root User DNs. When this attribute is not present on Root DN User entries, the effect is for the root user to inherit default privileges. It has been made mandatory to make this behavior more explicit. During an update of the server, root DN user entries that do not explicitly declare a value for this attribute will be updated with a value of 'true'. Issue:DS-8450
Add a WebLogic specific descriptor file for the web console to help with deployment compatibility. Issue:DS-8925 SF#:1915
The trust store password options have been deprecated for most tools that do not require read-write access to a trust store. Issue:DS-8789
Make a number of criteria-related improvements:
- Add Server SDK support for creating custom connection, request, result, search entry, and search reference criteria implementations.
- Update the simple request criteria type to make it possible to consider the search scope in determining whether a search operation matches the criteria.
- Update the simple result criteria type to make it possible to consider the indexed/unindexed status in determining whether a search operation matches the criteria.
- Add a new type of request criteria that may be used to more easily identify operations that target the server root DSE.
- Add a new type of result criteria that may be used to classify operations based on replication assurance requirements and/or whether those requirements were satisfied.
- Add a new allowed-insecure-request-criteria global configuration property that may be used to identify requests that the server should allow even if they are received over an insecure connection and the server is configured to reject insecure requests.
- Add a new allowed-unauthenticated-request-criteria global configuration property that may be used to identify requests that the server should allow even if they are received over an unauthenticated connection and the server is configured to reject unauthenticated requests. Issues:DS-5079,DS-8168,DS-8770
Add a new sign-log configuration property to file-based loggers that may be used to cause the server to digitally sign messages written by that logger. A new validate-file-signature tool may be used to verify signature information in signed log files, as well as LDIF files generated by signed LDIF exports. Issue:DS-8662
Add support for two new extended operations. A list configurations extended operation may be used to obtain information about the configurations that are available within to the server, including the currently-active configuration, the baseline configuration (i.e., the base configuration for an out-of-the-box installation of the current version), and all archived configurations that reflect configuration changes over time. A get configuration extended operation may be used to retrieve a specific configuration from the server. Issue:DS-9149
Fix a bug that displayed a null pointer exception when using the create-sync-pipe-config for Active Directory and choosing to not sync CREATE operations. Issue:DS-9065 SF#:1954
Update setup to fix a bug in which file path options specified as relative to the current directory may cause the server to be configured incorrectly or cause setup failure. Issue:DS-8389
Update the HTTP Connection handler to support configuration for tracking sessions either through HTTP cookies or by URL rewriting. Issues:DS-8639,DS-9128
Fix a bug where Changelog Password Encryption Plugin would throw an exception when processing a password change from an Active Directory Sync source. Issue:DS-9178 SF#:1954
Add special handling when creating users on an Active Directory Sync Destination that will set the userAccountControl attribute to an appropriate value. Attribute Mapping for userAccountControl is now strongly discouraged. Issue:DS-9108
Add fallback system for Active Directory Sync sources that detect deletes that do not have a distinguished name associated with the change. We will now attempt to use the objectGUID as the DN for those sync operations. Issue:DS-9187 SF#:1997
Improve error handling for DN attribute mapper to tolerate invalid DNs by not mapping the invalid DN, instead of not mapping all of the DN values when one is invalid. Issue:DS-9291 SF#:2001
Update the server to provide a degree of sandboxing around Server SDK extensions so that an unexpected exception thrown by an extension will be caught and result in an administrative alert rather than being caught further up in the stack and potentially causing other problems. Issue:DS-9247
In the rare cases where it is necessary to forcefully terminate the JVM from within the server itself, ensure that any files marked for deletion when the JVM shuts down are manually deleted before the JVM is terminated. This can help avoid problems like server shutdown not being detected properly because the server PID file hasn't been removed. Issue:DS-9267
Provide improved schema validation to detect additional cases in which certain misspelled tokens in the definition for a schema token could be silently interpreted as an extra property for that schema element. The server will now log a warning message about these unexpected tokens so that administrators can either correct them or prefix them with "X-" to indicate that they are an extra property provided for informational purposes. Issue:DS-9236
Reduce the time it takes the server to shut down in certain situations. Background threads sometimes missed a signal to wake up and had to wait for their next polling interval to see that a shut down had been requested. Issue:DS-9334
Update the default behavior of all file-based loggers to have include-thread-id=true. This will include a compact thread ID in all log messages. This can make it easier to correlate log messages generated by the same thread within a single log file or across different types of log files. Issue:DS-9352
Fix a bug that could prevent correlation of attributes with binary values. Most notably objectGUID from Active Directory. Issue:DS-9187 SF#:1997
Remove -XX:+UseMembar from the default set of generated JVM properties except on early JVM versions where this setting was required to work around a threading bug in the JVM.
Update the server JVM arguments generated by setup and dsjavaproperties to explicitly define -XX:MaxNewSize and -XX:NewSize for JVM's 1GB in size and larger. Also, add a comment to the generated java.properties file directing the administrator to use dsjavaproperties for making memory-related changes to this file rather than editing it directly. Issue:DS-9227
Add password file arguments to the scripts used to prepare external servers. Issue:DS-9406
Update the make-ldif utility such that first and last names are now generated randomly instead of in sequential alphabetical order. The original ordered behavior can be enabled by using the --orderedNames option. Issue:DS-9504
Update the setup and dsjavaproperties tools to permit maximum heap size values for memory that is not currently available on the host, though the value must still be less than the total amount of memory present on the host. Issue:DS-9111
Update the setup and dsjavapropeties tools to permit JVM heap size values to be as large as the amount of memory present on the system would permit. Issue:DS-9494
Update the Server SDK to provide the ability to run command line utilities within the server process. A ToolExecutor can be retrieved from the ServerContext. Currently, only the config-diff command is supported, but additional commands might be supported in the future. Issue:DS-9537 SF#:00001858
Enhance dsconfig to write to the config audit log when in offline mode. Issue:DS-1495
On Linux, issue a warning on startup and after a JVM pause if the kernel setting vm.swappiness is not 0 as this can cause the server to become unresponsive for several seconds when memory is paged back from disk during a garbage collection. Issue:DS-9070
Automatically record server monitor data at shutdown, as it may be useful for debugging purposes in cases where a problem was experienced within the server that was resolved by a restart. Issue:DS-9777
Improve the performance of certain monitor entry searches that target specific monitor entries by object class. In particular, this includes searches with AND or OR filters, as well as filters that target object classes not defined in the server schema. Issue:DS-9772
Lower the severity from SEVERE to MILD for attribute mapping failures due to missing source attribute. Issue:DS-9858 SF#:2068
These features were added for version 4.1.0.0 of the Data Sync Server:
The UnboundID Identity Broker is the first of a new class of components for consumer and subscriber identity management architectures.
As a stand-alone server, it provides authorization decisions for client applications, provisioning systems, API gateways, and analytical tools in any architecture involving personal, account, or sensitive identity data.
Working together with the UnboundID Identity Data Store and Identity Proxy, the Identity Broker is designed to make high-volume and high-speed authorization decisions based on ever-changing consumer profile and consent data. Functionally, the Identity Broker is both the Policy Decision Point and the OAuth2 provider for externalized authorization. Performance-wise, the Identity Broker can support the request volumes driven by the complex, real-time interactions necessary to support today's consumer-facing mobile, social, and cloud ecosystems.
These issues were resolved with version 4.1.0.0 of the Data Sync Server:
Add support for RPM based installation. Issue:DS-5990
Update the names of the UnboundID-branded products which are now: - Identity Data Store (formerly Directory Server) - Identity Proxy (formerly Proxy Server) - Identity Data Sync Server (formerly Synchronization Server) - Metrics Engine (name unchanged) Issues:DS-7514,DS-7515,DS-7516,DS-7518
Add ability to set the maxHeapSize and listenAddress arguments in a properties file when running setup. Issue:DS-6003
Fix a bug where current heap size was not displayed in error message about being too low. Issue:DS-7292
Fix a bug where under certain error conditions the start server scripts could prompt user to overwrite existing file. Issue:DS-7268
Introduce an "include-thread-id" configuration property on many of the file-based loggers that when enabled adds a threadID field to logging output. This makes it possible to know exactly which thread logged a message, which can simplify correlating errors between log messages and separate logs. This ID can be correlated to a thread name using the cn=JVM Stack Trace,cn=monitor entry. Issue:DS-8212
Change stop-*.bat to attempt a soft server shutdown before terminating the process. Issue:DS-408
Addressed an issue in the monitoring pages of the web console where they incorrectly listed directory server replication related information. Issue:DS-7289
Update the JMX connection handler to infer an appropriate Java type (e.g. Boolean, Long, Float, Date, or String) for JMX attributes from the underlying LDAP attribute type and value. The legacy behavior to return all JMX attributes as String values can be set if desired through the advanced global configuration property 'jmx-value-behavior'. Issue:DS-7635
Add the --noPropertiesFile option to the status command so that it does not fail when the option is provided to collect-support-data. Issue:DS-8390
Update the Synchronization Server to reduce the overhead of retrieving schema information from LDAP servers. Previously, the process of establishing a connection to an LDAP server would always cause the server schema to be retrieved over that connection. Now, schema information is shared across connections in a connection pool so that subsequent connections established within that pool no longer need to retrieve the server schema. This can reduce both the time and bandwidth required to establish connections to an LDAP server. Issue:DS-7597
Update setup to add a masters/peers trust-all argument so that the deployer must explicitly indicate that they trust the master/peer as well as any other masters/peers that are accessed during setup. In addition, if this argument is not specified a prompting trust store manager will be used instead of the previous behavior of using a trust-all manager all the time. If setup is in non-interactive mode and neither the trust-all argument nor the JKS trust store has been specified, and setup is accessing the master/peer over SSL or StartTLS setup will fail. Issue:DS-8381
Updated the UnsupportedOperationExceptions thrown by the Server SDK to include details about the exceptions that were thrown including the method name and the name and/or Java class of the extension that failed to implement the method. Issue:DS-8421 SF#:1898
These features were added for version 4.0.0.0 of the Data Sync Server:
Update the names of the UnboundID-branded products, which are now:
Add support for RPM based installation.
These were known issues at the time of the release of version 4.0.0.0 of the Data Sync Server:
UnboundID RPMs do not support the "--relocate" option. However, the RPMs are relocatable using the "--prefix" option at install time. Issue:DS-7890
These issues were resolved with version 4.0.0.0 of the Data Sync Server:
To comply with the SCIM 1.1 specification, the default scim-resources.xml configuration file now maps SCIM IDs to LDAP entryUUIDs rather than DNs by default. Issue:DS-7478
Change the stop-* tools to behave like other task based tools. These tools require the use of the --task argument to ensure that user knows they are using a server task. These tools also will not use properties files unless you provide the --usePropertiesFile or --propertiesFilePath arguments.
Fix a bug that prevented viewing hidden and complex configuration properties using dsconfig in a non-interactive mode. Issue:DS-7245
Improve the performance of ldifmodify when executed with a large source LDIF file. Issue:DS-7656
Fix a bug in the manage-extension tool that would cause an error when attempting to install an extension if that extension's getExtensionDescription method returned null (which is allowed as per the documentation). Issue:DS-7367
Improve the overall sync throughput out of the box by allowing the Change Detection Thread to pull in a larger set of changes between each change polling interval, which helps keep the sync worker threads from becoming starved. Issue:DS-7330
Fix a bug in the LDAPConnectionHandler where it did not close the NIO Selector on shutdown. On some platforms this caused the underlying socket channel to remain bound, which prevented the server from being able to restart. Issue:DS-7373
These features were added for version 3.6.0.0 of the Data Sync Server:
IPv6 is now a supported deployment option.
64-bit JDK 7 is now a supported deployment option, but 32-bit JDKs are no longer supported.
These issues were resolved with version 3.6.0.0 of the Data Sync Server:
Update the JMX Connection Handler configuration to issue a warning if it is enabled. On some JVMs, enabling this aspect of JMX can lead to long garbage collection pauses. Issue:DS-6832
Fix a problem where the Synchronization Server could log an error message saying "Could not register a JMX bean", which was due to a race condition in the JMX processing code. Issue:DS-6722
Add help text to web console deployment descriptor with JBoss compatibility tips. Issue:DS-6976 SF#:1749
Update the logic that the server and its associated tools use to select the SSL/TLS protocol version for secure communication to provide the best combination of security and compatibility. Also, a new log message type is available that can provide information about the negotiated security protocol, including the selected SSL/TLS protocol version and cipher suite. Issues:DS-6720,DS-6903
Prevent failures for configuration group changes where the parent configuration entry may or may not exist amongst the various servers in the configuration group. Issue:DS-6088
Fix the manage-tasks tool so that it does not use an insecure connection when --useStartTLS is specified, and does not prompt for certificate trust when --trustAll is specified. Issue:DS-5924
Fix an issue where, when using a JDBC Sync Source, the Synchronization Server would search for the destination entry using the "identifiableInfo" from the database change record instead of the constructed source entry DN. Issue:DS-6946
Add the --isCompressed option to the parallel-update tool so that it can read input LDIF files that are gzip compressed. Issue:DS-7237
Fix a problem in the manage-extension tool where it would fail on Windows because it tried to delete some temp files that were currently in use. Issue:DS-6770
Fix a bug in dsconfig that prevented going back when adding a new configuration object inside of an existing one. Issue:DS-7263 SF#:1793
Increase the maximum heap size to 4GB from 1GB for Synchronization Server. Issue:DS-7230
These features were added for version 3.5.1.0 of the Data Sync Server:
The Metrics Engine is a core server product that collects and aggregates key diagnostic, capacity, and usage information from an UnboundID server topology consisting of instrumented Directory Server, Directory Proxy Server, and Synchronization Servers running release 3.5.0.0 and above. Metrics data can be explored and graphed using the included query-metric tool, and the Metrics Engine REST API makes this information available to custom applications and third-party systems. To learn more about the Metrics Engine, please refer to the UnboundID Metrics Engine Administration Guide.
These issues were resolved with version 3.5.1.0 of the Data Sync Server:
Add help text to web console deployment descriptor with JBoss compatibility tips. Issue:DS-6976 SF#:1749
These features were added for version 3.5.0.0 of the Data Sync Server:
The Synchronization Server now supports a generic Sync Source API in the Server SDK, which allows the Synchronization Server to detect changes from different types of endpoints such as the Force.com platform, Google Apps, a flat file, or virtually any other kind of data source. For more information and an example, please refer to the UnboundID Server SDK documentation.
The Synchronization Server now supports a SCIM Sync Destination, which enables the synchronization of user and group entries to the cloud via the SCIM protocol.
Server SDK extension bundles may now be installed and updated using the manage-extension tool. For information about using the tool and building and packaging extensions, please refer to the UnboundID Server SDK documentation.
These were known issues at the time of the release of version 3.5.0.0 of the Data Sync Server:
When using a GSSAPI SASL Mechanism Handler the kerberos-service-principal property is only used to determine the protocol (i.e. "ldap"). The hostname will always be determined using the server-fqdn property. Issue:DS-5053
In failover scenarios, the SCIM Sync Destination may report that HTTP status code 500 was received and was the cause of a failover, even if no response was actually received from the server at all. Currently the SCIM-SDK does not have a mechanism to report local errors such as this. Issue:SCIM-287
These issues were resolved with version 3.5.0.0 of the Data Sync Server:
Fix an issue where multiple server configuration changes would fail if any of the servers were configured with an LDAPS (SSL) connection handler. Issue:DS-5100
Update the Synchronization Server to not look at the deletedEntryAttrs on changelog entries when determining whether to ignore a change based on the modifiersName attribute. Issue:DS-4171
Update the collect-support-data tool to include the equivalent of jstack output for IBM VMs on non-AIX platforms. Issue:MON-5027
Update the Synchronization Server to utilize the destination schema when synchronizing with an LDAP directory. This ensures that the sync engine will treat DN attributes and other types correctly when performing its diff. Issue:DS-3447
Update the Password Sync Agent for Active Directory to support 64-bit Windows operating systems. Separate binaries for x64 and x86 are now provided.
Fix a Validator error which could occur in the SyncPipe if the modification set to apply was empty (which can only happen in notification mode) and there was a failure at the destination endpoint. Issue:DS-5320
Update the Sync Server to use the original source DN when correlating entries at the destination for modify operations. Previously, if the entry had been renamed at the source immediately after the modify, the Sync Server would use the new entry DN to try and locate the destination entry. Issue:DS-5403 SF#:00001603
Add the ability to specify a reason when entering and leaving lockdown mode. This is recorded in the logs and in the alerts that are generated. Issue:DS-5331
Fix a bug that could cause the server to pass the old configuration into the isConfigurationChangeAcceptable method for a number of types of Server SDK extensions. Issue:DS-5597
Update the Server SDK to provide extensions a way to dynamically register their own monitor providers with the server, without requiring any server-side configuration objects. Issue:DS-5271
Change informational message when changelog indexing is not used on a sync pipe to be a debug trace instead. Issue:DS-5604
Add workaround in SSL processing to detect potential buffer underflow or renegotiation even when processing appears to be OK. Issue:DS-5748 SF#:1636
Fix a bug where method level debug tracing could cause extraneous logging from other methods in the same class. Issue:DS-5760 SF#:1636
collect-support-data now excludes binary files unless --includeBinaryFiles is specified. Issue:DS-4260
Fix an issue where DirectoryThreads did not set their context classloader to the one provided by our ClassLoaderProvider. This caused all the threads in the server to use the system classloader by default, which only has access to the classes specified on the classpath (i.e the core server libraries under the /lib directory). This becomes problematic if one of these threads calls into a library that uses Thread.getContextClassLoader() to load a class that is outside of the core server libraries (for example in an extension library). In this case it would use the system classloader and subsequently throw a NoClassDefError. Issue:DS-5876
Fix a bug where peer installs were updating servers of the wrong type from the master server's ADS. Issue:DS-5552
Change prepare-external-server to allow not supplying a trust store password in non-interactive mode, which will force the script to only trust the servers that are already present. Issue:DS-5872
Add additional checking to prepare-endpoint-server to ensure that the Sync User on a DSEE server has the appropriate attributes to prevent usage limitations. Issue:DS-5367 SF#:1598
Fix an unqualified static method invocation in the example ComplexJDBCSyncDestination Groovy script that is shipped with the Synchronization Server. Issue:DS-5299
Add a new property override-local-password to the Pass Through Authentication Plugin so that with the default value of false, it will only attempt the bind remotely if and only if the local bind fails because there is not a local password defined. When set to true, it will attempt the bind remotely if the local bind fails for any reason.
The new override-local-password property changes the default behaviour of the Pass Through Authentication Plugin. To restore the previous behaviour, change the value to true. Issue:DS-5766
Fix a problem where the collect-support-data tool could timeout when connecting over SSL, or prompt the user to verify the server certificate even when the --no-prompt argument was specified. Issue:DS-4823
Fix a bug that caused many command-line tools to output to stderr rather than stdout. Existing scripts that depend on the old behavior may need to be modified in order to continue working correctly. Issues:DS-3610,DS-4195
Update the Synchronization Server to use location-based failover for LDAP and SCIM endpoints. It will prefer to fail over to external servers which are in the same location as itself whenever possible, followed by servers in the preferred failover locations as defined in the configuration. Issues:DS-5016,DS-5412,DS-5413
Fix an issue where the Generic Sync Source and Destination extensions did not properly handle the ABORT_OPERATION result when it was specified in an EndpointException thrown by the extension. Issue:DS-5270
Update the file format used by "dsconfig --batch-file" to support using '\' as a line continuation character. If the last character on a line is a '\', then it will be removed and the following line concatenated on to it. Issue:DS-635
Remove the "Custom" type from the list when creating new objects in dsconfig. This was often confused with the "Third-Party" and "Groovy Scripted" types when users intended to create a Server SDK extension. Issue:DS-5229
Update the create-sync-pipe-config tool so that it will fail immediately and display an error message if there is a problem creating the configuration objects in the Synchronization Server. Issues:DS-4208,DS-5019
Update ldap-diff to use the schema of the target server when comparing entries. This enables comparing entries whose DN's include case-sensitive components. Issues:DS-2748,DS-6197
Modify the ldap-diff tool to add LDAP connection options for SSL, StartTLS, and SASL authentication. Issue:DS-6034
Update the status tool to fix an issue in the tool may fail to connect to the server to retrieve some status information when the --no-prompt option is specified. Issue:DS-5989
Add changelog-reset detection to the Synchronization Server so that it will issue a purge alert if it detects that the source changelog has been deleted or reset. Issue:DS-5280
Update the Server SDK to make it possible to create an internal connection that behaves like an external connection and is subject to its constraints. Issue:DS-5851
Update the Synchronization Server to make sure it merges remote state from all other servers at least once before letting the sync pipes begin. This eliminates a very small window that could only happen at startup, where a Synchronization Server could take over the master status and begin syncing before it has merged any remote state from secondary Synchronization Servers.
Update the Synchronization Server to handle a network partition between redundant instances. This guarantees that the true master will incorporate state from the other master(s) after the network is restored.
Fix a bug in the LDAP changelog where it would not incorporate GetChangelogBatch token information about other replicas in a certain edge-case scenario. Issues:DS-4520,DS-6320,DS-6340
Improve detection of active sync servers in a failover group to ensure that low priority master servers will stop syncing if more than one is ever made master at the same time due to network partition or other system wide failures. Issue:DS-6395 SF#:1710
Fix an issue where out-of-the-box server required more memory than it should have, because of how the DictionaryPasswordValidator stored its word dictionary. The memory usage has been reduced by roughly 35MB. Issue:DS-6040
Added support for two-legged OAuth 2 authentication method for SCIM external servers. Both the "Bearer" and "OAuth" token types are supported. Issue:SCIM-302
Add the ability to constrain a Sync Log Publisher to one or more specific sync pipes. This allows you to create a sync logger that will show only the output for the sync pipe(s) you are interested in. Issue:DS-5344
Updated the server to support hosting of standard web applications using the HTTP Connection Handler. Issue:MON-754
Fix a bug that could result in a standby Sync Server starting a Sync Pipe when it comes online. Issue:DS-6486 SF#:1727
Add lock down mode detection for the sync server that will force the sync engine to stop when entering lock down and start up again when leaving lock down mode. Issue:DS-6573 SF#:1734
Provide an argument to the setup tool to configure the server to automatically include verbose garbage collection output in the server.out log file. Issue:DS-5681
On Linux, the server and its tools now attempt to raise the limit on maximum user processes to 16,383 if the current value reported by ulimit is less than that. This is because Linux counts a thread as a user process, and some recent Linux distributions have a very low default value for max user processes. Issue:DS-6410
Update dsconfig so that inclusion of the --advanced option will list expert-level objects. Issue:DS-6652
Fix an issue where the resync tool did not adequately report the lack of endpoint preparation when the sync pipe was set up against the Directory Proxy Server. Issue:DS-3426
Fix an issue where a Sync Pipe would stop updating its monitor statistics for the source endpoint if it became severely backlogged and was waiting for some in-flight operations to finish before bringing in new changes. Issue:DS-6666 SF#:1740
Improve the prompt that is displayed by command-line tools when establishing a secure connection to a server when no trust manager was specified and the server certificate should not be automatically trusted. The information is formatted more neatly, and the prompt will now include MD5 and SHA-1 versions of the certificate fingerprint and information about the issuer certificate chain if appropriate. There will also be an additional warning if the certificate is self signed. Issue:DS-5127
Update tools that can perform LDAP SASL authentication to add support for the UNBOUNDID-TOTP SASL mechanism that can be used for multifactor authentication. Issue:DS-6676
Fix the prepare-endpoint-server tool to handle the case where ACI values for the Sync User already exist on the synchronization base DN. Issue:DS-6552
Fix a bug in the SCIM Sync Destination where it might try to connect to the SCIM endpoint when the sync pipe is stopped, and also might log a NullPointerException. Issue:DS-6728
Fix an inconsistency in the Sync Pipe statistics where the 'ops-completed-failed-at-resource' stat could be incremented when it shouldn't be. Issue:DS-6464
Update the JMX Connection Handler configuration to issue a warning if it is enabled. On some JVMs, enabling this aspect of JMX can lead to long garbage collection pauses. Issue:DS-6832
Fix a problem where the Synchronization Server could log an error message saying "Could not register a JMX bean", which was due to a race condition in the JMX processing code. Issue:DS-6722
These features were added for version 3.2.0.0 of the Data Sync Server:
AIX is now a supported deployment operating system.
These issues were resolved with version 3.2.0.0 of the Data Sync Server:
Fix an issue where, for an UnboundID Sync Source, the Sync Server could lose some changes during fail over to another sync server if the persistent state had not yet been saved for the first time. This is unlikely to affect a production environment. Issue:3400
Update command-line tools providing support for SASL authentication to add additional properties that may be used in conjunction with the GSSAPI mechanism. This includes the ability to control whether a ticket cache should be allowed and/or required, the ability to specify an alternate location for the ticket cache file, the ability to request that the Kerberos ticket-granting ticket be renewed, and the ability to supply a custom JAAS configuration file rather than using one automatically generated by the tool. Issue:3437
Fix a bug that prevents going back from the type selection when creating a new configuration object in dsconfig. Issue:2913 SF#:1435
Update a number of LDAP command-line tools to provide a new --help-sasl option that can be used to obtain information about the SASL mechanisms that are available for use and the supported options for those mechanisms. In addition, the command-line tool reference has been updated to provide a new page on supported SASL mechanisms and options. Issue:3452
Fix a bug in which dsconfig and other tools may not properly evaluate path-based property values for remotely managed servers. Issue:3439 SF#:00001484
Add a new stat to the Sync Pipe Monitor to expose the current throughput rate, in operations processed per second. This stat is available using the status command on the Synchronization Server as well. Issue:3556
Modify the update tool to handle potential issues migrating the admin-backend.ldif backend file if the ds-create-time attribute is present in the entry cn=all-servers,cn=Server Groups,cn=admin. Issue:3584 SF#:00001501
Update shell scripts used for the server and associated tools so that they will display a warning if it is not possible to set the desired number of file descriptors. Issue:3590
Fixed an issue with the JDBC Sync Source where after a failure it could potentially re-fetch changes that had already been processed. Issue:3276
Fix an issue in the Synchronization Server where alerts were not always generated when an endpoint server transitioned from unavailable to back to available (i.e. came back online). Limit changelog purge alerts from the Synchronization Server to once every ten seconds rather than once per poll of the changelog, and add the name of the Sync Pipe to the alert. Issue:3626
Fix an issue where the Synchronization Server memory usage could spike when a Directory Server goes down behind a Proxy. Issue:3628
Improve error message when extension class can not be loaded. Issue:3245
Fix an issue in the Synchronization Server where JDBC Sync Source and Destination extensions would not correctly initialize their ArgumentParser when running within the resync command. Issue:DS-4169
Update dsconfig to make the list-properties subcommand more visible and more usable. This includes the following changes:
- The list-properties output will now be written to standard output rather than standard error. This makes it easier to process the output with text tools like grep.
- The list-properties subcommand can now be used with the "--offline" argument even if the server is running.
- A new "--complexity" argument has been added that can be used to customize the complexity level of the objects included in the output.
- A new "--includeDescription" argument has been added that can be used to include synopsis and description information in the output.
- The top-level dsconfig help now includes an example demonstrating the use of the list-properties option.
- A docs/config-properties.txt file containing this information is now provided with the server. This information was previously already available in the HTML config reference guide. Issue:DS-2985 SF#:00001413
Make it possible to configure the server to configure the number of file descriptors that it should attempt to use on UNIX-based systems. Previously, the server was hard-coded to try to use 65535 file descriptors. It is now possible to override this default by setting the NUM_FILE_DESCRIPTORS environment variable with the desired number of descriptors to use. Alternately, you can do this by creating a config/num-file-descriptors file with a single line, like:
NUM_FILE_DESCRIPTORS=12345
If an error occurs while attempting to use the desired number of file descriptors, then a message will be written to the terminal, and if the error occurs while starting the server, then a message will be logged to the server's error log. Issue:DS-3590
Add the ability to compress log files as they are written. This can significantly increase the amount of data that can be stored in a given amount of space so that log information can kept for a longer period of time. Because of the inherent problems with mixing compressed and uncompressed data, compression is something that can be enabled only at the time the logger is created, and compression cannot be turned on or off later. Further, because of problems in trying to append to an existing compressed file, if the server encounters an existing log file on startup, it will rotate that file and begin a new one rather than attempting to append to the previous file.
Compression is performed using the standard gzip algorithm, so compressed log files can be accessed using readly-available tools. Further, the summarize-access-log tool has been updated so that it can work directly on compressed log files rather than requiring them to be uncompressed first. However, because it can be useful to have a small amount of uncompressed log data available for troubleshooting purposes, administrators using compressed logging may wish to have a second logger defined that does not use compression and has rotation and retention policies that will minimize the amount of space consumed by those logs while still making them useful for diagnostic purposes without the need to uncompress files before examining them. Issue:DS-2983 SF#:00001410
Critical: Change the default behavior of the Synchronization Server to not lock entries across all Sync Pipes when processing changes.
The Sync Server has a specialized mutex that ensures that changes to the same entry are processed serially. The primary reason for this mutex is to ensure that the server can safely process changes in parallel to achieve high throughput. However, we also use this mutex to ensure that two Sync Pipes don't process the same entry at the same time for deployments that synchronize changes bi-directionally. A consequence of this locking is that if one Sync Pipe is failing (because the destination is unavailable) then it retains the lock on an entry, and when other Sync Pipes try to process changes to that entry they will block that change and all changes that follow it while they wait on the lock.
This change turns off using a shared mutex by default, but adds a new advanced configuration option on the Sync Pipe, shared-mutex-name, that specifies the name of a mutex that is shared by other Sync Pipes. This gives greater control over the locking so that two Sync Pipes that share end points can ensure that two changes to the same logical user are not processed concurrently, while not impacting other Sync Pipes.
See the shared-mutex-name property for more information.
This property is subject to change in a future release. Issue:DS-4202 SF#:1527
Enhance sync setup, when adding to an existing topology with JDBC nodes configured, to check if needed library files are in place before warning that manual copying may be needed. Issue:DS-3194 SF#:1471
Update dsconfig to remove a redundant prompt when a user chose to "Change the value" of an existing property. Issue:DS-2140
Add a new reject-insecure-requests global configuration option that can cause the server to reject all operations except StartTLS extended requests received over insecure connections. This makes it easier to allow clients to use StartTLS without allowing other requests over an insecure connection. Issue:DS-4397
Update server access loggers to add a number of new options:
- An option to include request details in search result entry messages. - An option to include request details in search result reference messages. - An option to include request details in intermediate response messages. - An option to include the names of attributes included in an add request. - An option to include the names of attributes targetd in a modify request. - An option to include the names of attributes included in a search result entry. - An option to include extended search request details, including the size limit, time limit, types only, and alias dereferencing behavior. Issue:DS-4404
Update the server to add a "--lockdownMode" argument which can be used to cause the server to be started in lockdown mode. Issue:DS-1488
Update the server to generate an administrative alert if it detects that a configuration change was made with the server offline (whether by manually editing the configuration file or using dsconfig in offline mode). Issue:DS-4407
Update the server to provide better reporting around the use of third-party extensions. If any such extensions are loaded in the server, then the DNs of their configuration entries will be listed in the thirdPartyExtensionDN attribute of the cn=monitor entry. Further, some extensions are loaded at startup, and a message will be written to the error log with the DNs of all of their configuration entries. Please note that not all extensions are loaded at startup, in particular Sync extensions. Issue:DS-4398
Fix an issue in which terminal focus may be lost during command-line setup just before the Summary step is shown. Issue:DS-4551
Fix an issue in which dsconfig cannot set an unlimited value for an object property that supports an unlimited value. Issue:DS-4173
Update the UnboundID work queue to add a dedicated thread pool that may be used for processing certain administrative operations. This dedicated thread pool may make it possible for an administrator to diagnose and take corrective action in a server even if all "normal" worker threads are tied up processing other operations. By default, eight worker threads will be created for this purpose, but this may be altered via the num-administrative-session-worker-threads property in the work queue configuration.
Some administrative tools like dsconfig, status, collect-support-data, enter-lockdown-mode, and leave-lockdown-mode will automatically attempt to create an administrative session in which all operations they request will be processed in this dedicated pool. Other tools like ldapsearch, ldapmodify, ldapcompare, ldapdelete, ldappasswordmodify, backup, restore, import-ldif, export-ldif, and manage-tasks have a new "--useAdministrativeSession" argument that can be used to request that they attempt to use this dedicated thread pool for operations that they process. Further, the Commercial Edition of the UnboundID LDAP SDK for Java has been updated to provide support for the new start administrative session and end administrative session extended operations that are needed to use this feature, so third-party applications can also take advantage of this capability.
In order to request that operations be processed using the administrative session thread pool, the requester must have the use-admin-session privilege (which is included in the default set of privileges automatically granted to root users). The use of the administrative session thread pool will be recorded in the access log, and a new "using-administrative-session-worker-thread" property has been added to the simple request criteria and can be used to filter operations based on whether they are using this capability. Issue:DS-4401
Fix an issue in which setting the changelog-max-before-after-values property to unlimited causes sync server to indicate all attributes as exceeeding the maximum value. Issue:DS-4549
Update the Periodic Stats Logger to include additional information in the "SyncPipeInfo" category. The new output columns are "Source Is Connected", "Source Connected Server", "Destination Is Connected", "Destination Connected Server", "Source Unretrieved Changes", "Completed Aborted by Plugin", "Completed Failed in Plugin", and "Synchronized Out of Date Changes". Issue:DS-3575
Fix an issue where the Synchronization Server would report null pointer exceptions in the error log if all external servers for a given endpoint became unavailable. Issue:DS-4415
Update the SyncPipe with the ability to specify a "filter-changes-by-user", which allows it to perform access control filtering on the source data before synchronizing. Modified the sync engine to send alerts only if the unavailable source attributes (due to ACI filtering or exceeding the max values) were ones that the Sync Pipe was interested in. Added new stats to track how many sync operations have been filtered by ACIs and how many have exceeded the max before/after values. Updated the create-sync-pipe-config tool to support the new ACI filtering features and enable them on the Sync Pipe if desired. Updated the prepare-endpoint-server tool to support the new ACI filtering features and enable them on the directory server changelog if desired. Issue:DS-4564
Add a new listAllEntries() method to the JDBCSyncSourceAPI which allows resync from a file to include the entryType argument. Note that this breaks compatibility with the previous listAllEntries() which did not include the entryType parameter. Existing code will continue to compile, but the old method will no longer be used by the server and you will be forced to implement the new version when resyncing a database using a source input file. Issue:DS-3542
Update the create-sync-pipe-config tool to check that the source endpoint has the minimum required version before configuring ACI filtering for notifications. Issue:DS-4564
Fix an issue with database synchronization where a null pointer exception could be thrown if the username or password were left blank on the JDBCExternalServer configuration.
Fix an issue where old configuration data may get left in a topology of Sync or Proxy servers after a server is uninstalled or removed from the topology. Issue:DS-4712
Modify the tools to recognize instances of the Sun DSEE 7 Directory Server when deployed as part of the Oracle Identity Management 11g. Issue:DS-4716
Update the Synchronization Server, so that within a polling interval, multiple batches of changes can be retrieved from the source server. This increases throughput without having to specify a small polling interval. Issue:DS-4859
Fix an issue with the "realtime-sync" tool where it would not print specific error data to the screen if a problem occurred while stopping a sync pipe. Issue:DS-4521
Add a new configuration property to the SyncPipe called "include-changes-for-unchanged-attributes". This can be used to override the default behavior of only applying the minimum set of changes to bring the destination entry into sync and instead cause Sync to include modifications for attributes that were changed to the same value they already have. Issue:DS-4963
Add support for the IBM JDK for the GSSAPI SASL bind mechanism handler and when using GSSAPI SASL binds with tools and utilities. Due to restrictions with the IBM JDK, when using tools and utilities and the option "ticketcache" is set, the bind will always fail if the credentials are not found in the specified ticket cache, even if the option "requirecache" option is false. Issue:DS-4749
Update the Synchronization Server to take advantage of changelog indexing on the source endpoint if it is available. This can provide significant performance gains when synchronizing from UnboundID servers, especially if the data is entry-balanced. Issue:DS-4955
Improve the dsframework tool to support multi-valued server propreties. Issue:DS-5040
Fix an issue in the Synchronization Server where a sync pipe in notification mode would not take advantage of virtual attribute information in changelog entries. Issues:DS-5069,DS-5071
These features were added for version 3.1.0.0 of the Data Sync Server:
Update the Synchronization Server to support arbitrary Sync Destination end points via a Server SDK extension. Sync Destination implementations can be written in Java or Groovy.
Add a new "notification" mode to the Synchronization Server. When a Sync Pipe is configured in this mode, it skips fetching the full entry from the source and instead immediately notifies the destination with the contents of the change. The details of the change are completely derived from the source changelog entry. This enables a destination to go through the same sequence of changes as the source.
The Synchronization Server can now be monitored over SNMP.
These issues were resolved with version 3.1.0.0 of the Data Sync Server:
Add new global configuration attribute that allows specifying a SMTP timeout to use for all configured SMTP servers. Issue:2283
Exposed the SyncOperation class where appropriate in the JDBC Sync Source and Destination interfaces for the Server SDK. This is consistent with the other types of sync plugins that have access to the SyncOperation. Issue:2934
Limit collect-support-data to only run against the local server it is ran from. All supported versions of the products have collect-support-data available, and should use that version to do any needed data collection. Issue:2827
Enhance timeout for SMTP External Servers to be used for socket I/O and connection based timeouts. Previously the timeout value applied only to socket I/O. Issue:2939
Added support for Java-based JDBC Sync Source and Destination extensions. Previously these were only available in Groovy. Issue:3003
The update and revert-update tools now respect that -Q/--quiet option which when specified, suppresses console output of messages that are not warnings or errors. In addition, the tools will not solicit input if the -n/--no-prompt option is specified. Issue:3056 SF#:00001432
The dsconfig tool has been fixed to that it does not exit in an error when the root DSE entry is not available. Issue:3122
Add a new type of access logger which can be used to obtain very detailed information about requests and responses and the contexts in which the associated operations have been processed. This is primarily intended for troubleshooting purposes rather than general use, and the content is meant to be human-readable rather than machine-parsable. Further, because the output can be quite verbose, it is recommended that it only be enabled when attempting to diagnose a problem, and that it be used in conjunction with the filtered logging framework so that only potential messages of interest will be captured. Issue:3064
Update tools, such as searchrate, that use --ratePerSecond to not use 100% of one CPU when running at a low rate. The cutoff for this rate depends on the minimum amount of time that a process can sleep, which is operating system dependent.
Fix an issue where status output failed to complete due to stale JDBC connection. Issue:3170
Updated the create-sync-pipe-config tool to support setting up a Sync Pipe in notification mode. It now also supports using pre-existing Sync Sources discovered from the existing configuration. Issue:3024
Changed the --sourceDNsFile argument for the Resync command to be --sourceInputFile. It now supports DNs for resync from LDAP as well as a user-defined format for resync from a database. Issue:2734
Update collect-support-data to collect more system level information (especially on Linux) and validate that any value specified with the --pid option does not match the servers PID, since information about the server process is always collected. Issues:2920,2930,3152,3171,3206
Hide the subtree-view option in the Client Connection Policy configuration in DS and Sync. There is currently no way to create manual subtree views for these products, but this option may be add back for future features as needed. Issue:3125
Add a --missingOnly option to ldap-diff to allow the tool to only report on entries that exist on only one of the servers; entries that exist on both servers but are out-of-sync are ignored. Issue:2918
Update tools which can be used to schedule tasks to add a new "--task" argument that makes it explicit that the tool is intended to run as a task rather than in offline mode. At present, this argument is optional, but we intend to make it required in the future, and if a tool is invoked as a task without this new "--task" argument, then a warning message will be displayed recommending that it be used in the future.
In addition, if the "--task" argument is provided but the tool was not given an appropriate set of other arguments to allow it to connect and/or authenticate to the server, then an error message will be displayed and the tool will exit with an error. This behavior will also be exhibited for other arguments that are only applicable for tools running as tasks, including the "--start", "--dependency", "--failedDependencyAction", "--completionNotify", and "--errorNotify" arguments. Issue:3224
Added support for SNMP monitoring to the Synchronization Server via the SNMP Subagent Plugin. A Sync Pipe MIB has been supplied. Issue:3229
Update the manage-tasks tool so that it can detect cases in which the authenticated user doesn't have permission to access information about tasks in the server and will provide a more useful error message. It would previously always report that there were no tasks in the server, which may not be true and is not very helpful. Issue:2957
Update tools which create scheduled tasks to display a message indicating that killing the tool will not interrupt the task. For tasks that can be interrupted, the tool will also display a manage-tasks command line that can be used to cancel that task. Issue:2954
These issues were resolved with version 3.0.3.0 of the Data Sync Server:
Fix an issue in the Synchronization Server where "the next batch of changes" could not be retrieved. This occurred when synchronizing through a proxy server after a backend directory server had been restarted. Issue:3205 SF#:1472
Add an option to collect-support-data for collecting data from expensive processes. These expensive operations will not be executed by default. Issue:3176
Fix an issue where debug messages logged by a command line tool (when using --enableDebug) might not be flushed to disk before the command exited. Issue:3218
Add a configurable sync backlog threshold to the Sync Server so that alerts will be generated when a sync pipe becomes severely backlogged with unprocessed changes. Another alert will be generated when the backlog goes back below the configured threshold. Add alerting when the Sync Server detects that changes have been missed because they were purged from the source changelog before the sync pipe had a chance to process them. Issue:3199
Add a configuration option to allow the Synchronization Server to synchronize delete and moddn operations even if they are out-of-date with the source server. Issues:3181,3230 SF#:1460,1461
Add a configuration option (allow-destination-renames) to the Sync Class to control whether a rename of an entry (e.g. moddn in LDAP) should be allowed at the destination in the process of synchronizing a modify operation. Issue:3225 SF#:1475
Fix an issue where the sync server could attempt to process source changes it had already seen when an UnboundID or Sun DS sync source failed over from one source server to another. This would happen when the only changes being made on the source servers were ones that did not need to be synchronized (for example if bi-directional sync was configured and the changes in question had been made by the sync server). Issue:3100
Fix a bug that could cause resync from an UnboundID sync source to fail with a null pointer exception. Issue:3275
Fix an issue where in rare circumstances if all endpoint servers were unavailable, the Synchronization Server might not reconnect to them when they become available. Issue:3278
These issues were resolved with version 3.0.2.0 of the Data Sync Server:
Modify the update tool to fix an issue where in some cases the tool would fail to migrate an older configuration, displaying errors related to duplicate LDIF change records. Issues:2942,2962,2967
Add support for Java-based JDBC Sync Source and Destination extensions. Previously these were only available in Groovy. Issue:3003
Fix an issue in the Synchronization Server where the attribute-synchronization-mode 'all-attributes' would handle deleted attributes incorrectly. Issue:3026
Fix a regression with the stop-sync-server command where the port argument was ignored. Issue:2925
The create-sync-pipe-config tool now correctly handles connecting to the SSL port of an LDAP server when specified as part of an endpoint. Issue:2952
Fix an issue where the status command would warn that the port argument was ignored even though the argument was not provided. Issue:3052 SF#:1447
Expose destination-create-only-attr as a Sync Class property to allow certain attributes (such as objectclass) to only be set when an entry is created and not when it is modified. Issue:2947
The command-line tools now use the full terminal width for output on Windows platforms. Issue:1019
Fix a regression where the Synchronization Server could not synchronize with Sun DSEE 5.2p4 instances because this version of DSEE did not allow the filter "(&)" to match any entry contrary to the LDAP specification. Issue:3069 SF#:00001449
The setup tool has been modified to correct an issue in which the presence of the --rootUserDN option, when specified with any of the "Set Up From Peer/Master Server Options", would cause setup to exit with an error. Issue:3084
Increase the recommended default for LDAP changelog retention to be 2 days instead of 2 hours to match real world deployment expectations. Issue:3105
Fix an issue where the Groovy "assert" statement was not handled correctly in Server SDK extensions used to synchronize with a database. Issue:3137
Increase the default value for duplicate error messages (allow 2000 in 5 minutes) and alerts (allow 100 in 1 hour) before they are suppressed. Avoid duplicate suppression for certain types of alerts, such as configuration changes. Ensure that the severity of a duplicate alert summary message matches the severity of the duplicate messages being suppressed.
Address an issue where Server SDK extensions running within a command line tool could cause the process to run out of memory if they logged a high volume of error log messages. Issue:3173
These issues were resolved with version 3.0.1.0 of the Data Sync Server:
Change collect-support-data tool to prompt for missing LDAP connection arguments if needed. Issue:2461
Fix an issue where the Synchronization Server could occasionally report a severe initialization error on startup leaving the Sync Pipes disabled. Issue:2835
The script file for stopping the server on non-Windows operating systems have been modified so that when it is invoked with no arguments, the server is killed using the operating system's kill command, ensuring that the server will have stopped when the script returns. Issue:2821
The remove-defunct-server tool has been enhanced to allow the user to choose to continue processing of topology servers even if one of the servers is down. In non-interactive mode this is accomplished using the --continueOnError option. Issue:2856
Update the server so that some of the specialized access loggers (e.g., failed operations and expensive operations) do not include messages about intermediate responses. Issue:2822
Address an issue with collect-support-data when run on Windows where certain commands that were executed would timeout without reading the full output of the command.
Add a new external server type for configuring SMTP servers. This can be used to provide secure connections and authentication to outgoing mail servers. Issue:1150
The SNMP Master Agent Plugin is no longer exposed as configurable because it is not a supported component. It is only used for test purposes.
Fix a bug in the web console that prevented the creation of configuration objects with a slash character in the name. Issue:2836
Add the ability to log debug statements from server components that are running within the context of a command line tool. This also enables logging from third-party extensions developed with the Server SDK to be captured when run from the context of a command line tool. Issue:2834
Expose the SyncOperation class where appropriate in the JDBC Sync Source and Destination interfaces for the Server SDK. This is consistent with the other types of sync plugins that have access to the SyncOperation. Issue:2934
The dsframework tool has been modified so that whenever a server is registered or updated with port values whose corresponding protocol enablement properties (ldapEnabled, ldapsEnabled) are not present, the tool will automatically set the value of the enablement property to "true". Issue:783
These features were added for version 3.0.0.0 of the Data Sync Server:
Database Synchronization - Support for high-scale, highly-available data synchronization from one endpoint consisting of one of our supported Directory Servers with the other endpoint consisting of a relational database management system (RDBMS). UnboundID officially supports synchronization with Oracle Database 10g and 11g as well as Microsoft SQL Server 2005 and 2008. The architecture, however, does not make any assumptions about the type of database or schema being managed; any database with a Type 4 JDBC driver can be used.
Server SDK - Server-side SDK for extending the functionality of the core server.
Synchronization Through Proxy - Support for Synchronizing to or from an load-balanced or entry-balanced proxy server deployment.
Virtualization Support - Achieved "VMware Ready Status" for all of our server products, which we now support deploying in VMware environments.
These issues were resolved with version 3.0.0.0 of the Data Sync Server:
Update the Synchronization Server installer so that all servers within a topology will use the same IntraSync-User password. Issue:1970
Add a background retry mechanism to the Synchronization Server so that failed operations can be optionally retried after a specified delay. Issue:2040
Add the ability to specify separate destination correlation rules for deleted entries, so that deletes can use a more relaxed set of rules if need be. This can be useful in scenarios where applications delete and then re-add the same entries (with different attributes), for example. Issue:2072
Expose version information for many of the libraries used by the server in both "status --fullVersion" and in the "cn=Version,cn=monitor" entry. It will always include the LDAP SDK version number, and if available may also include any or all of the Berkeley DB JE, JZlib, SNMP4J, SNMP4J Agent, and SNMP4J AgentX library versions strings.
Add a configuration option that may be used to indicate whether the server should shut down in the event that a severe error (e.g., out of memory) is raised within the JVM that indicates it may not be able to continue running properly. Issue:2265
The dsjavaproperties tool now supports options for generating, regenerating, and updating the config/java.properties file. Issue:2280
Fix a potential memory leak in the Synchronization Server which could occur during Sync Source failover if there were a large number of pending changes in the queue at the time of failover. Issue:2169
Fix a bug in the timestamp-naming mechanism used in log file rotation which could cause log files that were manually renamed to still get rotated and eventually deleted if their names were still parsable as the original file name. Issue:1285
Added a safeguard to the LDAP Sync Source so that it will not wait forever for responses to asynchronous changelog searches (particularly with DSEE).
Update the stop script so that the "restart" option will correctly restart the server after a successful shutdown Issue:2329 SF#:1362
Update dsconfig to work correctly in environments with a server-group set. This issue only affected dsconfig when run in a partially interactive mode where some of the configuration arguments were provided on the command line. The user is now prompted whether the configuration change should be applied to the current server or all servers in the group. Issue:2373 SF#:1370
Reduce the maximum timeout value from 10 minutes to 5 minutes for Synchronization Server changelog searches against DSEE, and add some extra checks for connection health. The configured response timeout on the Sync Source is still preferred, but if none is set, then this value will be used as a ceiling. Issue:2383
Address an issue where the Unique Attribute Plugin incorrectly detected conflicts when under heavy. Issue:1873
Web Console displays a communication error alert when editing configurations objects if the server has been disconnected. Issue:2270 SF#:1239
Fix a bug in which the server and tool JVM configurations in java.properties would lack -Xms and/or -Xmx options if the amount of memory specified as the maximum heap size was not available when setup was run. Issue:890
Fix a bug in which setup fails if the 'locks' directory is missing, setup erroneously indicated that the server was running.
Fix a bug that prevented the display in dsconfig and the web console of configuration objects whose name contained a slash character. Issue:2244 SF#:1373
Modify the update tool to disallow the update tool from being used from a package in which setup has been run. Issue:2464
Provide a custom title renderer that escapes configuration object names in the web console. This avoids a theoretical security concern with configuration object names that contain embedded JavaScript. Issue:2454
Fix a bug in the ldapmodify command-line tool that caused it to incorrectly treat a 'referral' result as success. Referrals are still not supported by this tool, but it will now treat them as a special kind of error and will provide a more useful message. Issue:1062
Update the resync tool to fail immediately if no destination servers are available. Issue:1181
Generate a warning message at startup if the server is unable to determine the IP address or hostname of the local system, or if the local system's hostname resolves to a different IP address. These conditions may indicate a problem with the system configuration that could cause certain server components to break or function abnormally. Issue:2318
Change the way that the serverUUID value is generated so that it is based on a combination of the system's primary IP address and the canonical server root path. This can be used to help detect cases in which a new server instance is created by copying the files associated with an existing server instance, which would have previously created two instances with the same serverUUID value. In the event that the stored serverUUID does not match the generated value, a log message will be generated to warn administrators of the change, and the newly-generated UUID will continue to be used. Issue:2470
Remove forced min utilization configuration setting for replication and LDAP change logs. These settings had led to excessive database growth in some circumstance. Issue:2294 SF#:1352
Improve the output of the ldapsearch tool to mention that a password has expired when the bind occurs. Issue:1981 SF#:1227
Modify the updater so that the --ignoreWarnings option can be used to continue with update when there are warnings related to version compatibility issues. This allows an update to be run in a non-interactive environment, such as a script. Issue:2495
The admin alerts list no longer includes alert types that are clearly not applicable to the product. Issue:1738
Update generated command line arguments (such as for dsconfig) to be quoted in a mechanism specific to the operating system where they are generated and to eliminate all escaping with \, which had caused problems when replaying certain commands. This is done with as much portability across systems as possible. Issue:2455
Update the Synchronization Server to connect immediately to the destination server of a Sync Pipe rather than waiting for a change to come through. This enables the server to show in its status that the destination server is connected. Issues:2005,2389,2547
Update the Synchronization Server to send an admin alert when a Sync Pipe fails to start up (because of a configuration error or scripting error) and continue starting other pipes. A new alert type has been added for this condition, called sync-pipe-initialization-error. In the case of such an error, a Sync Pipe may be restarted with the server online via the realtime-sync tool. Issue:2547
Update the realtime-sync tool to read arguments from the config/tools.properties file if present. Issue:2513
Improved status command output to better inform the user of how the local server status was determined, based on the arguments provided. Issue:2487
Update cli documentation to include new commands for updating and reverting a server installation. Issue:2573 SF#:1390
Tools using a scope argument are now correctly documented in the CLI documentation. Issue:2594
Added a new configuration property to the Sync Class which allows you to control as part of a sync operation whether all attributes should be brought into sync on the destination or only those that were affected by the originally modified attributes at the source. The property is called 'attribute-synchronization-mode'.
Several enhancements to the Periodic Stats Logger: all columns in the output can now be turned on/off, many more built-in metrics are available to be logged, and additional custom metrics driven off of cn=monitor entries can be added by creating Custom Logged Status objects. Issue:2039
Add extension points for the Synchronization Server. This includes Sync Pipe Plugins, LDAP Sync Source Plugins, and LDAP Sync Destination Plugin. Issue:2410
The server now issues an alert when it has begun the startup process. Issue:2642
The server now issues an alert when a JVM pause (possibly due to garbage collection) has been detected. Issue:2637
The web console now allows the specification of multiple LDAP servers to be used for authentication and discovery of topology servers. Issue:2466
The web console now supports specification of a server from its login page. Issue:2190
Add an option to display the status for just a specific Sync Pipe in the status command output. This makes the output a lot easier to read if there are multiple Sync Pipes configured. Issue:2606
Update the ldappasswordmodify tool to supply the bind password as the user's current password when making a self-change. This is convenient when making a root user password change so that the current password does not have to be specified twice in the command line arguments. Issue:2525
Provide better descriptions in the MIB for SNMP trap variable bindings. Issue:2508
The file-based loggers now optionally support millisecond level precision. Issue:2603
Added a "invoke-gc-day-of-week" property to the Periodic GC Plugin so that it can be configured to run only on certain days of the week. Issue:2660
Improve output when JVM errors occur in scripts used to set up environment for command line tools. Issue:2172
Update the default JVM arguments to improve garbage collection tuning.
Update dsjavaproperties to validate that all java-home properties specified in config/java.properties reference valid Java installations. Issue:2719
Fix an issue where the alerts backend could write an incomplete LDIF backing file if an error were to occur during the write. Also, if an error in the LDIF file is discovered when the server is started, the alerts backend will now read as much as it can from the file and preserve a copy of the bad file. Issue:2700
Add support for logging intermediate response messages that are returned to the client. Intermediate response logging will be enabled by default, but may be disabled if desired. Issue:2428
Address an issue with the web console where it would not allow read-only configuration properties to be set when an object was initially created. Issue:2730
These issues were resolved with version 2.2.0.0 of the Data Sync Server:
Modify the command-line argument parsers to generate a warning message if an argument value is the same as the short or long form for another argument. This can help prevent users from forgetting to supply a value for an argument which requires one. Issue:944
Update MakeLDIF to add a "<random:timestamp>" tag that can be used to include a randomly-selected date from any time within the last ten years. It is also possible to use "<random:timestamp:min:max>" to specify the desired time range, where min and max should be given in the generalized time format. Issue:1083
Add a new configuration property for alert handlers that makes it possible to filter the types of alerts that should be processed based on the alert severity. By default, all types of alerts will be processed.
Modify the prepare-external-server tool so that it will look for trust store and password files in the default locations when using SSL or StartTLS and the locations of those files are not explicitly provided.
Provide a new alert handler that can be used to execute a specified command whenever an alert is generated within the server. The details of the alert notification will be provided as arguments when executing that command. The arguments will be provided in the following order: the name of the alert type, the OID for the alert type, the alert severity, the fully-qualified name of the Java class that generated the alert, the unique identifier assigned to that alert, and the text of the alert message. The alert handler will ensure that only one instance of the command may be invoked at a time to avoid problems from commands that aren't safe to run concurrently. If multiple alerts are generated concurrently, then they will be queued and the command will be executed sequentially for each of them. Issue:1146
Update the ldapsearch and ldapmodify tools so that in the event that an error response is received from the server, the diagnostic message from that error response will be displayed to the user rather than the generic error message that had previously been used.
Add a new error log alert handler, which makes it possible to control which types of alerts should be logged (based on either the alert severity or specific alert type). Further, the severity of the log message will reflect the severity of the alert notification.
Update the collect-support-data tool to archive information about the upgrade history of the server installation.
Modify the enter-lockdown-mode and leave-lockdown-mode tools to allow them to connect to any local address rather than requiring the request to be sent over the loopback address. Issue:1144
Update the LDAP connection handler to disable TLS renegotiation by default, which can eliminate a vulnerability in which a man-in-the-middle could potentially inject arbitrary cleartext between TLS negotiation and initial data from the client.
Avoid setting the "-XX:ParallelCMSThreads" JVM argument on systems containing a single CPU. This option has been observed to cause the JVM to fail to run properly, particularly in virtualized environments. Issue:1300
Update the system information monitor entry to include information about the system account being used to run the server and a list of all system properties defined in the JVM.
Add a new global configuration option which makes it possible to specify the maximum length of time that the server shutdown process may take before it attempts to interrupt threads which have not yet completed their processing. In most cases, server threads will react to a shutdown in a timely manner and no interrupt is needed.
Fix a bug in the parallel-update tool that could cause operations to be retried even when the --neverRetry argument was provided. Also, when the tool is configured to retry operations, the reject file will now include the result code and diagnostic message received from the last failure after no more progress can be made, rather than providing a generic message.
Fix a bug in the collect-support-data tool that could cause it to make incorrect use of a password file when capturing the output of the status command. Issue:1593
Update the SNMP alert handler so that the traps it creates have a more sensible value for the uptime field. Previously, the uptime value was always zero, but it will now reflect the length of time that the Directory Server has been online.
Improve the process for stopping threads when the server is shutting down, and provide additional debugging information that may be useful if any threads are slow to stop running. Issue:900
Update the ldap-diff tool to take advantage of the stream directory values extended operation when it is available. This can dramatically improve the performance of the tool when attempting to identify the set of all entries in the server. Issue:794
Update the ldap-diff tool to provide support for reading the DNs of all the entries in one or both directories from files instead of obtaining them over LDAP. In directories which do not support the stream directory values extended operation, this may provide a significantly faster way to obtain this information if it is already available in some form.
Fix a bug in the ldap-diff tool that could cause it to report incorrect percent complete values when comparing data sets of more than 20 million entries.
Fix a bug in the upgrade tool that could cause the same warning message multiple times if the version obtained from the server was different from what was expected (e.g., because a server jar file had previously been replaced without using the upgrade tool). Issue:1640
Update the parallel-update tool to add the ability to use the permissive modify request control, which may be used to request that the server ignore attempts to add attribute values which are already present or remove attribute values which are not present.
Update the ldap-diff tool to make it more likely that its output can be replayed without any alteration. The order of operations has been updated so that all deletes are listed first, followed by all modifies, and finally all adds. In addition, all delete operations are ordered such that subordinate entries will always be removed before their ancestors.
Update the scripts used to stop the server to prevent them from falling through to try to stop the server over LDAP if the attempt to kill the process fails or times out, since the attempt to stop the server over LDAP would fail without at least the appropriate authentication credentials, and could potentially be dangerous in some contexts.
Update the system information monitor entry to include information about all environment variables defined in the server process. In addition, it will now attempt to determine and report the process ID of the JVM in which the server is running.
Update the logic for sending an e-mail message from the server so that it will always attempt to determine the fully-qualified name of the system to include in the HELO/EHLO request. In the event that the fully-qualified name cannot be determined, then the IP address of the server will be used rather than using an unqualified name. Issue:1337
Update the server to make it possible to configure the length of time that name-to-IP address mappings may be cached within the server. This may be useful in environments in which the addresses associated with a particular hostname may change frequently. Issue:941
Update the upgrade and revert-upgrade tools to ignore directories that contain backup files. Issue:1143
Update the Directory Server to change the implementation of the show-all-attributes configuration option in the schema and root DSE backends to be more robust, particularly for clients requests explicitly requesting a specific set of attributes. Issue:1590
Updated the logic used to identify previous log files that had been rotated so that only files with names that might have been created by the rotation process will be candidates for removal by the retention policy. Issue:1285
Update the Directory Server to add a search shutdown plugin which can be used to perform a specified internal search when the server is shutting down and have the results of that search written to a specified file. This may be useful, for example, to automatically dump the contents of the monitor backend on shutdown. Issue:1334
Update the server so that when creating a duplicate of an existing configuration object, some key properties may be excluded from the clone so that they must be explicitly configured by the administrator rather than automatically using the same value as the object being duplicated. This can help prevent problems in which a duplicated value was inadvertently used. Issue:1675
Update the setup process so that the server will be configured without an LDAP connection handler if the "--no-prompt" argument is provided without an "--ldapPort" argument. This option is only available for use when using the non-interactive setup mechanism. Issue:1759
Change the behavior of the dsconfig tool when creating a new configuration object so that the user will first be prompted about whether to create a completely new configuration object or clone an existing object. This simplifies the interface and makes it less likely that an administrator will incorrectly attempt to clone an existing object rather than creating a new one. Issue:1747
Update a number of access log retention policies to make them more robust and to fix bugs that could prevent old log files from being removed when the appropriate conditions were met. Over long periods of time, this could potentially cause available disk space to run low and necessitate the manual removal of files to avoid running out of space. Issues:1867,1867
Modify the upgrade process so that schema definitions are always migrated before the configuration. In some rare cases, attempting to migrate the configuration before the schema could lead to failures in the upgrade process. Issue:1812
Update the server to prevent multiple loggers from being configured with the same target log file. Issue:1676
Significantly revise the upgrade tool in an attempt to make it more robust and minimize the amount of work required for performing an upgrade. Issues:1927,1931,2031,2037
Add support for a new search-and-mod-rate command line tool which operates in a manner similar to the searchrate tool but that will also modify any entries returned from the search.
Rename the upgrade tool to be "update", and rename the revert-upgrade tool to be "revert-update".
Update the Directory Server to make the lockdown-mode privilege usable by non-root users. Issue:1109
Update the server so that it includes a patch version number in addition to the existing major, minor, and point version numbers. This can help better distinguish versions with the same major, minor, and point version numbers which differ only based on patches applied.
Update the Directory Server to abort the startup process with an error message if the admin data backend includes a malformed entry. Previously, malformed entries in the admin data backend would be silently ignored. Issue:2049
Update the collect-support-data tool to change the way that the jstack tool is invoked to dramatically reduce the impact that it has on the running process. Issue:2038
Update the export-ldif and verify-index tools so that they can be used against a server whose database files are contained on a read-only filesystem, including a ZFS snapshot. Issue:71
Update the alert backend to be able to handle entries with unrecognized alert types. This is unlikely to occur in normal conditions, but could cause a problem in deployments in which the server was upgraded and subsequently reverted, and an alert was generated in the upgraded server that uses an alert type not defined in the older version. Issue:2126
Change the way that the worker thread percent busy values are calculated in the work queue monitor entry to make them more accurate. Also, add new recent-average-queue-size and current-worker-thread-percent-busy monitor attributes. Issue:1982
Modify the update process to require that the system user performing the update is the same as the system user used to run the server. This will help prevent files from being created or altered during the update process with permissions that would prevent the server from being able to access them when the server is started as the appropriate user. Issue:2158
Modify the update tool to ensure that the documentation is updated for the new release if appropriate. Issue:2178
Update the dsconfig tool and the Web administration console so that they inform the administrator of any administrative action (e.g., disabling and re-enabling the specified component, or restarting the server) that may be required as a result of a configuration change to be made. Issues:211,2132
Update the subject attribute to user attribute certificate mapper to provide support for VeriSign certificates whose subject contained an emailAddress attribute with an unusual encoding. Issue:2177