Ping Identity Data Sync Server Release Notes

The following issues have been resolved with this release of the Data Sync Server:

• Fixed an issue where the server was attempting to connect by an IP address rather than a hostname when DNS lookup was successful. Issue:DS-40366 SF#:00668508

• To support multiple trace loggers, each trace logger now has its own resource key, which is shown in the "Resource" column in the output of "status". This key allows multiple alarms, due to sensitive message types for multiple trace loggers. Issue:DS-37955

• Fixed an issue that stopped new extensions from being installed. Issue:DS-41054 SF#:00677974

• Added a --duration argument to collect-support-data. When used, only the log files covering the specified duration before the current time will be collected. Issue:DS-40771

• Allows users who were migrated from the admin backend to the topology to manage the topology. Migrated users are granted the "manage-topology" privilege if they do not already have it. Issue:DS-39799

• Fixed an issue in Active Directory Sync Source where the persistent state was updated before applying changes, so changes could be missed when stopping the Sync Pipe. Issue:DS-40020

• Fixed a memory leak when performing SCIM queries on the Directory Server. Issue:DS-41206 SF#:00681395

These issues were resolved with version 7.3.0.4 of the Data Sync Server:

• Fixed an issue where some state associated with a JMX connection was not freed after the connection was closed. This led to a slow memory leak in servers that were monitored by an application that created a new JMX connection each polling interval. Issue:DS-40828

These issues were resolved with version 7.3.0.3 of the Data Sync Server:

• Fixed an issue that could cause the server to leak a small amount of memory each time it failed to establish an LDAP connection to another server. Issue:PDSTAGING-840

These issues were resolved with version 7.3.0.1 of the Data Sync Server:

• Updated the Groovy scripting language version to 2.5.7. For a list of changes, visit groovy-lang.org and view the Groovy 2.5 release notes. As of this release, only the core Groovy runtime and the groovy-json module are bundled with the server. To deploy a Groovy-scripted Server SDK extension that requires a Groovy module not bundled with the server, such as groovy-xml or groovy-sql, download the appropriate jar file from groovy-lang.org and place it in the server's "lib/extensions" directory. Issues:DS-39176,DS-39308

• Added a "cn=Server Status Timeline,cn=monitor" monitor entry to track a history of the local server's last 100 status changes and their timestamps. Updated the LDAP external server monitor to include attributes tracking health check state changes for external servers. The new attributes include the number of times a health check transition has occurred, timestamps of the most recent transitions, and messages associated with the most recent transitions. Issue:DS-17278

These features were added for version 7.3.0.0 of the Data Sync Server:

• New features for data encryption in transit and at rest: added support for TLS 1.3, ability to encrypt and automatically decrypt sensitive files such as tools.properties and keystore pin files using the server data encryption keys, and the ability to more easily and securely separate master keys from data encryption keys by protecting the server encryption settings database using either Amazon Key Management Service (AWS KMS) or HashiCorp Vault.

• Added support for Amazon Corretto JDK 8, Windows Server 2019, Red Hat Enterprise Linux 7.6, CentOS 7.6, Amazon Linux 2, and Docker 18.09.0 on Ubuntu 18.04 LTS.

• More configuration flexibility. Brand new JSON Attribute Mapping provides a more natural way to construct JSON-based attributes rather than using Constructed Attribute Mapping. Base64 encoding and decoding options are added to allow for synchronizing binary data. LDAP extensible match and approximate match filters can be used to better select changes that require synchronization. Rate limiting has been added per sync pipe using the max-rate-per-second setting.

• Publish changes to Kafka. Sync pipes in notification mode can now be configured to publish changes in PingDirectory to topics in Kafka. Messages are published to Kafka for additions, deletions, modifications, and for resynchronization. Each message contains the JSON-serialized format of the mapped attributes per LDAP object, before and after modification, if applicable.

These issues were resolved with version 7.3.0.0 of the Data Sync Server:

• HTTP Connection Handlers now accept client-provided correlation IDs by default. To adjust the set of HTTP request headers that may include a correlation ID value, change the HTTP Connection Handler's correlation-id-request-header property. Issue:DS-37617

• Updated the server to enable TLSv1.3 by default on JVMs that support it (Java 11 and higher). Issue:DS-38072

• Updated the server to support encrypting the contents of the PIN files needed to unlock certificate key and trust stores. If data encryption is enabled during setup, then the default PIN files will automatically be encrypted.

  Also, updated the command-line tool framework so that the tools.properties file (which can provide default values for arguments not provided on the command line), and passphrase files (for example, used to hold the bind password) can be encrypted. Issue:DS-38050

• Added support for insignificant configuration archive attributes.

  The configuration archive is a collection of the configurations that have been used by the server at some time. It is updated whenever a change is made to data in the server configuration, and it is very useful for auditing and troubleshooting. However, because the entries that define root users and topology administrators reside in the configuration, changes to those entries will also cause a new addition to the configuration archive. This is true even for changes that affect metadata for those entries, like updates to the password policy state information for one of those users. For example, if last login time tracking is enabled for one of those users (especially with high-precision time stamps), a new configuration may be generated and added to the configuration archive every time that user authenticates to the server. While it is important for this information to be persisted, it is not as important for it to be part of the server's configuration history.

  This update can help avoid the configuration archive from storing information about updates that only affect this kind of account metadata. If a configuration change only modifies an existing entry, and if the only changes to that entry affect insignificant configuration archive attributes, then that change may not be persisted in the server's configuration archive.

  By default, the following attributes are now considered insignificant for the purpose of the configuration archive:

  * ds-auth-delivered-otp * ds-auth-password-reset-token * ds-auth-single-use-token * ds-auth-totp-last-password-used * ds-last-access-time * ds-pwp-auth-failure * ds-pwp-last-login-ip-address * ds-pwp-last-login-time * ds-pwp-password-changed-by-required-time * ds-pwp-reset-time * ds-pwp-retired-password * ds-pwp-warned-time * modifiersName * modifyTimestamp * pwdAccountLockedTime * pwdChangedTime * pwdFailureTime * pwdGraceUseTime * pwdHistory * pwdReset Issue:DS-37959

• Fixed an issue in the installer where the Administrative Console’s trust store type would be incorrectly set if it differed from the key store type. Issue:DS-38085 SF#:648467

• Critical: The following enhancements were made to the topology manager to make it easier to diagnose the connection errors described in PDSTAGING-570:

  - Added monitoring information for all the failed outbound connections (including the time since it's been failing and the last error message seen when the failure occurred) from a server to one of its configured peers and the number of failed outbound connections.

  - Added alarms/alerts for when a server fails to connect to a peer server within a configured grace period. Issues:DS-38334,PDSTAGING-570 SF#:00655578

• Critical: The topology manager will now raise a mirrored-subtree-manager-connection-asymmetry alarm when a server is able to establish outbound connections to its peer servers, but those peer servers are unable to establish connections back to the server within the configured grace period. The alarm is cleared as soon as there is connection symmetry. Issues:DS-38344,PDSTAGING-570 SF#:00655578

• Critical: The dsreplication tool has been fixed to work when the node being used to enable replication is currently out-of-sync with the topology master. Issues:DS-38335,PDSTAGING-570 SF#:00655578

• Fixed an issue that could prevent certain types of initialization failures from appearing in the server error log by default. Issue:DS-38403

• Added a max-rate-per-second configuration property to the Sync Pipe, which can be used to throttle the rate that changes are applied to the destination. Issue:DS-38098

• Added the PingOne for Customers Sync Source to Ping Data Sync. Identities can now be synchronized from PingOne for Customers to on premises identity stores. Issue:DS-38253

• Added 'base64-encode-value' and 'base64-decode-value' properties to direct attribute mappings to facilitate synchronizing binary data. Issue:DS-38460

• Added a cipher stream provider that can be used to protect the contents of the encryption settings database with a key from the Amazon Key Management Service. Issue:DS-15734 SF#:3718

• The response header used for correlation IDs may now be set at the HTTP Servlet Extension level using the correlation-id-response-header configuration property. If set, this property overrides the HTTP Connection Handler's correlation-id-response-header property. Issues:DS-38090,DS-38564,DS-38567

• Updated the PingDataSync Server to support LDAP filters in Sync Classes that include extensible match or approximate match components. The Data Sync Server supports the same matching rules as the PingDirectory Server. Issue:DS-38495 SF#:00654434

• Added a cipher stream provider that can be used to protect the contents of the encryption settings database with a secret passphrase obtained from a HashiCorp Vault instance. Issue:DS-38512

• Added an HTTP servlet extension that can be used to retrieve the server's current availability state. It accepts any GET, POST, or HEAD request sent to a specified endpoint and returns a minimal response whose HTTP status code may be used to determine whether the server considers itself to be AVAILABLE, DEGRADED, or UNAVAILABLE. The status code for each of these states is configurable, and the response may optionally include a JSON object with an "availability-state" field with the name of the current state.

  Two instances of this servlet extension are now available in the default configuration. A request sent to /available-state will return an HTTP status code of 200 (OK) if the server has a state of AVAILABLE, and 503 (Service Unavailable) if the server has a state of DEGRADED or UNAVAILABLE. A request sent to the /available-or-degraded-state will return an HTTP status code of 200 for a state of AVAILABLE or DEGRADED, and 503 for a state of UNAVAILABLE. The former may be useful for load balancers that you only want to have route requests to servers that are fully available. The latter may be useful for orchestration frameworks if you wish to destroy and replace any instance that is completely unavailable. Issue:DS-18060

• Fixed a bug where the startIndex value for SCIM requests would be incorrect if the used LDAPSearch element had more than one baseDN defined in the scim-resources XML file. Issue:DS-38670 SF#:00643950

• Critical: Fixed two issues in which the server could have exposed some clear-text passwords in files on the server file system.

  * When creating an encrypted backup of the alarms, alerts, configuration, encryption settings, schema, tasks, or trust store backends, the password used to generate the encryption key (which may have been obtained from an encryption settings definition) could have been inadvertently written into the backup descriptor. This problem does not affect local DB backends (like userRoot), the LDAP changelog backend, or the replication database.

  * When running certain command-line tools with an argument instructing the tool to read a password from a file, the password contained in that file could have been written into the server's tool invocation log instead of the path to that file. Affected tools include backup, create-initial-config, create-initial-proxy-config, dsreplication, enter-lockdown-mode, export-ldif, import-ldif, ldappasswordmodify, leave-lockdown-mode, manage-tasks, manage-topology, migrate-ldap-schema, parallel-update, prepare-endpoint-server, prepare-external-server, realtime-sync, rebuild-index, re-encode-entries, reload-http-connection-handler-certificates, reload-index, remove-defunct-server, restore, rotate-log, and stop-server. Other tools are not affected. Also note that this only includes passwords contained in files that were provided as command-line arguments; passwords included in the tools.properties file, or in a file referenced from tools.properties, would not have been exposed.

  In each of these cases, the files would have been written with permissions that make their contents only accessible to the system account used to run the server. Further, while administrative passwords may have been exposed in the tool invocation log, neither the passwords for regular users, nor any other data from their entries, should have been affected. We have introduced new automated tests to help ensure that such incidents do not occur in the future.

  We recommend changing any administrative passwords you fear may have been compromised as a result of this issue. If you are concerned that the passphrase for an encryption settings definition may have been exposed, then we recommend creating a new encryption settings definition that is preferred for all subsequent encryption operations, exporting your data to LDIF, and re-importing so that it will be encrypted with the new key. You also may wish to re-encrypt or destroy any existing backups, LDIF exports, or other data encrypted with a compromised key, and you may wish to sanitize or destroy any existing tool invocation log files that may contain clear-text passwords. Issues:DS-38897,DS-38908

• Added a Kafka Sync Destination, which can be used to publish changes from PingDirectory Sync Sources to Kafka topics. Changes are represented as the entry contents before and the entry contents after the change in JSON format. Issue:DS-38100

• Fixed an issue in which backups of the encryption settings database could be encrypted with a key from the encryption settings database. Issue:DS-38550

• Added an indent-ldap-filter tool that can make it easier to visualize the structure and components of a complex search filter. Issue:DS-38849

• Make Fingerprint Certificate Mapper and Subject DN to User Attribute Certificate Mapper disabled by default on fresh installations. This will not affect upgrades from installations where these mappers are enabled. Issue:DS-37839

• Added attribute-comparison-method configuration option to SyncClass to allow syncing with a syntax-based comparison (default mode) or a byte-for-byte comparison method. Issue:DS-38582

• Added support for using Population names both as a default when configuring the PingOne Customer Sync Destination and in attribute mapping. Issue:DS-36269

• Fixed an issue where inter-server bind requests would fail if the cipher used reported a maximum unencrypted block size of 0. Issue:DS-38737 SF#:00658314

• Fixed an issue that would throw an exception when trying to delete an entry containing uncached attributes if the LDAP changelog was enabled and using reversible form. Issue:DS-38957 SF#:00662848

• Added the --skipHostnameCheck command line option to the setup script, which bypasses validation of the provided hostname for the server. Issue:DS-38109

• Updated the ldapdelete command-line tool to improve robustness and add features. Some of the new features include support for client-side subtree delete, deleting entries that match search filters, following referrals, writing failures to a rejects file, rate limiting, and support for a variety of additional controls. Issue:DS-36474

• Addressed an issue in DataSync where LDAP operational attributes (such as isMemberOf) used in a SyncClass include-filter would not be requested from the source LDAP server if the LDAP filter was a nested filter. Issue:DS-39014

• Changed the default value of the HTTP Configuration property include-stack-traces-in-error-pages from 'true' to 'false'. Disabling this property prevents information about exceptions thrown by servlet or web application extensions from being revealed in HTTP error responses. Issue:DS-38864

• Added JSON Attribute Mapping and JSON Attribute Mapping Field configuration objects that simplify constructing destination JSON attribute values from source attributes. Compared to using a Constructed Attribute Mapping to build up JSON, using these configuration objects does not requiring providing the JSON structure (braces, colons, and quotes), does not require escaping string values, and automatically omits fields whose values cannot be constructed. Issue:DS-38833

• Added a set of message types to Trace Log Publishers that records events related to access token validation. Issue:DS-38913

• Removed the version information page from the docs/build-info.txt endpoint. This information is now available in build-info.txt, which is located in the root directory. Issue:DS-39086

• Addressed an issue where a resync command could fail with a SEVERE_ERROR condition when run with multiple passes and the Sync Source was either Active Directory or PingOne for Customers. Issue:DS-39174

• Addressed an issue where a resync operation might not synchronize all users if a PingOne Customer Sync Source was configured with a value for population-to-synchronize. Issue:DS-39177

These issues were resolved with version 7.2.1.0 of the Data Sync Server:

• Critical: The following enhancements were made to the topology manager to make it easier to diagnose the connection errors:

  - Added monitoring information for all the failed outbound connections (including the time since it's been failing and the last error message seen when the failure occurred) from a server to one of its configured peers and the number of failed outbound connections.

  - Added alarms/alerts for when a server fails to connect to a peer server within a configured grace period. Issue:DS-38334 SF#:00655578

• Critical: The topology manager will now raise a mirrored-subtree-manager-connection-asymmetry alarm when a server is able to establish outbound connections to its peer servers, but those peer servers are unable to establish connections back to the server within the configured grace period. The alarm is cleared when connection symmetry is achieved. Issue:DS-38344 SF#:00655578

• Critical: The dsreplication tool has been fixed to work when the node being used to enable replication is currently out-of-sync with the topology master. Issue:DS-38335 SF#:00655578

• Added a max-rate-per-second configuration property to the Sync Pipe, which can be used to throttle the rate that changes are applied to the destination. Issue:DS-38098

• Added the PingOne for Customers Sync Source to Ping Data Sync. Identities can now be synchronized from PingOne for Customers to on premises identity stores. Issue:DS-38253

These features were added for version 7.2.0.0 of the Data Sync Server:

• Introduced a Directory REST API to create, read, update and delete (CRUD) any object in the directory using JSON over HTTP. Compared to the SCIM-based Identity Access API (introduced in 4.0), the Directory REST API offers more capability without the configuration overhead and SCIM protocol limitations. See https://apidocs.pingidentity.com/pingdirectory/directory/v1/api/guide/ for more information.

These issues were resolved with version 7.2.0.0 of the Data Sync Server:

• Added support for an exec task that can invoke commands on the server. There are several safeguards in place to prevent unauthorized users from invoking arbitrary commands on the server system, including a new exec-task privilege and a whitelist file that must be updated to include the absolute paths of the allowed commands. A new schedule-exec-task tool helps create an exec task from the command line, and the LDAP SDK has also been updated to allow interacting with exec tasks programmatically. Issue:DS-35873

• Added support for recurring exec tasks. Issue:DS-35873

• Added support for a delay task, which can be used on its own or as a recurring task. It is primarily intended to be used as a spacer between other tasks, and can sleep for a specified period of time, wait for the server to be idle (that is, there are no outstanding operations and all worker threads are idle), or wait for sets of search criteria to match at least one entry (for example, until a monitor entry indicates that the server is in a desired state). Issue:DS-36510

• Added a new preFetch method to the LDAPSyncSourcePlugin Server SDK extensions. This method allows extension developers to modify the search request or source entry fetch behavior for LDAP Sync Sources. Issue:DS-36434

• Added support for a new file retention task that can identify files in an indicated directory that match a given pattern and remove any matching files that fall outside of the specified retention criteria. You can specify the minimum number of files that should be retained, the minimum age of files that should be retained, the minimum aggregate size of files that should be retained, or any combination thereof. The files that match the pattern will be sorted by timestamp so that if any files are to be removed, the most recent files will be retained and the oldest files will be deleted.

  The file retention task can be scheduled as a standalone task or as a recurring task. Two instances of the file retention recurring task have been defined in the default configuration: one that can clean up old expensive operation dump files, and another that can clean up old work queue backlog thread dump files. In each case, the recurring task is configured to keep at least the 100 most recent files, and no files less than 30 days old will be removed. While these recurring tasks are defined in the out-of-the-box configuration, they are not part of any recurring task chain and therefore will not actually be invoked unless they are configured as part of a chain.

  The PingDirectory Server and PingDirectoryProxy Server now include recurring tasks in the out-of-the-box configuration that can clean up old expensive operation dump log files or work queue backlog thread dump log files if too many of them have collected in the server logs directory. For each type of file, if there are more than 100 of them in the server logs directory, then any of the remaining files that are more than 30 days old are candidates for removal. A recurring task chain will perform this cleanup every day at 12:05 a.m. in the JVM's default time zone. Issues:DS-35652,DS-36559

• A header containing a correlation ID is now added to outgoing HTTP servlet responses, allowing HTTP responses to be correlated with log messages across server instances. The name of the correlation ID response header defaults to "Correlation-Id" but may be changed by setting the HTTP Connection Handler's correlation-id-response-header property. By default, the server will generate a globally unique correlation ID automatically, but the correlation-id-request-header configuration property may be used to optionally specify one or more request headers that provide an existing correlation ID value from the requesting client. The correlation ID header can be disabled on a per-HTTP Connection Handler basis using the use-correlation-id-header configuration property.

  For Server SDK extensions that have access to the current HttpServletRequest, the correlation ID can be retrieved as a String via the HttpServletRequest's "com.pingidentity.pingdata.correlation_id" attribute. For example: <code>(String) request.getAttribute("com.pingidentity.pingdata.correlation_id");</code> Issue:DS-36209

• HTTP Connection Handlers will now raise an alarm during initialization if a context path conflict is detected. Issue:DS-35909

• Fixed an issue in which the HTTP Servlet Config Monitor could cause an exception in an HTTP Servlet Extension when attempting to determine its context paths. This caused the status tool and the Administrative Console to potentially omit the HTTP Servlet Extension from the list of active HTTP extensions. Issue:DS-37131

• Added a Mock Access Token Validator, which accepts access tokens without validating the authenticity of the tokens using a trusted authorization server or signing certificate. When enabled, a Mock Access Token Validator accepts bearer tokens in the form of a plain text JSON object containing an arbitrary set of claims. Mock Access Token Validators are intended for test or demonstration use only and should never be enabled in production deployments or used to access sensitive data. Issue:DS-36433

• Added a time limit retention policy to support removing log files older than a specified age. Issue:DS-37492

• To facilitate testing in multiple GC (garbage collection) environments, GC JVM options having been moved to separate Java properties in the java.properties file. The new ".gc-type" suffix will select the GC type to use, and the new ".gc-<GC type>-args" suffix will have the JVM options for that GC type. Issue:DS-6930

• Added Change Detectors as a new type of Server SDK extension. A Change Detector can be used to customize the way a Sync Source detects changes to entries in a source server.

  As an example, a developer could build a Change Detector that reads messages from a Kafka topic and uses them as the source of changes for an OpenLDAP server configured as the Sync Source.

  Initially, Change Detectors are only available for use with the new Generic LDAP Sync Source. Issue:DS-36435

These issues were resolved with version 7.0.1.3 of the Data Sync Server:

• Fixed a bug where the startIndex value for SCIM requests would be incorrect if the used LDAPSearch element had more than one baseDN defined in the scim-resources XML file. Issue:DS-38670 SF#:00643950

• Critical: Fixed two issues in which the server could have exposed some clear-text passwords in files on the server file system.

  * When creating an encrypted backup of the alarms, alerts, configuration, encryption settings, schema, tasks, or trust store backends, the password used to generate the encryption key (which may have been obtained from an encryption settings definition) could have been inadvertently written into the backup descriptor. This problem does not affect local DB backends (like userRoot), the LDAP changelog backend, or the replication database.

  * When running certain command-line tools with an argument instructing the tool to read a password from a file, the password contained in that file could have been written into the server's tool invocation log instead of the path to that file. Affected tools include backup, create-initial-config, create-initial-proxy-config, dsreplication, enter-lockdown-mode, export-ldif, import-ldif, ldappasswordmodify, leave-lockdown-mode, manage-tasks, manage-topology, migrate-ldap-schema, parallel-update, prepare-endpoint-server, prepare-external-server, realtime-sync, rebuild-index, re-encode-entries, reload-http-connection-handler-certificates, reload-index, remove-defunct-server, restore, rotate-log, and stop-server. Other tools are not affected. Also note that this only includes passwords contained in files that were provided as command-line arguments; passwords included in the tools.properties file, or in a file referenced from tools.properties, would not have been exposed.

  In each of these cases, the files would have been written with permissions that make their contents only accessible to the system account used to run the server. Further, while administrative passwords may have been exposed in the tool invocation log, neither the passwords for regular users, nor any other data from their entries, should have been affected. We have introduced new automated tests to help ensure that such incidents do not occur in the future.

  We recommend changing any administrative passwords you fear may have been compromised as a result of this issue. If you are concerned that the passphrase for an encryption settings definition may have been exposed, then we recommend creating a new encryption settings definition that is preferred for all subsequent encryption operations, exporting your data to LDIF, and re-importing so that it will be encrypted with the new key. You also may wish to re-encrypt or destroy any existing backups, LDIF exports, or other data encrypted with a compromised key, and you may wish to sanitize or destroy any existing tool invocation log files that may contain clear-text passwords. Issues:DS-38897,DS-38908

These issues were resolved with version 7.0.1.2 of the Data Sync Server:

• Enhanced LDAP Sync Destinations such as PingDirectory, DSEE, and Generic LDAP so that the IgnoreNoUserModificationRequestControl is applied to requests by default. This allows Sync Sources to write and update operational attributes, such as "createtimestamp" or "modifiersdn." Additional configuration on the destination server may be required to enable modifying specific operational attributes. Issue:DS-37804

These were known issues at the time of the release of version 7.0.1.0 of the Data Sync Server:

• Servers to be monitored by the PingDataMetrics Server must have an instance name of less than 256 characters. A server's instance name is specified during setup. Issue:DS-36788

These issues were resolved with version 7.0.1.0 of the Data Sync Server:

• Added a configuration option to allow a null serverFQDN for the GSSAPI SASL mechanism to allow an unbound SASL server connection. Issue:DS-36642 SF#:00637397

• Added the ability to synchronize pre-encoded passwords to the PingOne for Customers Sync Destination.

• Updated the keys and values used in the monitoring JMX MBeans to conform with best practices. The keys "type" and "name" are now used in place of "Rdn1" and "Rdn2".

  To maintain backwards compatibility with existing monitoring solutions, installations upgrading to this release will retain the old behavior, but they can revert to the default behavior by changing the Global Configuration property jmx-use-legacy-mbean-names to false. Issue:DS-37235

• Improved supportability by adding additional logging for the PingOne for Customer Sync Destination to expose all errors from the PingOne API. Issue:DS-37275

Important upgrade considerations for version 7.0.0.0 of the Data Sync Server:

• This release introduces significant changes to the way servers in a topology are configured with information about each other. Once a server has been upgraded from a pre-7.0 version to 7.0 or later, reverting to the previous version is not supported. Before beginning the upgrade process, make sure you have read and understood the Administration Guide's chapter "Upgrading the Server".

• SCIM 2 error responses, including Config API error responses, now represent the "status" field as a JSON string rather than as a number. Clients written to expect the earlier version format will need to be updated. In particular, clients written using the SCIM 2 SDK for Java should upgrade to version 2.2.0 or higher.

• The Administrative Console now uses server information found in the topology registry to populate its server selection control. If the Console is used to manage a legacy server that does not use the topology registry, then the server selection control will not be populated. To manage a different server, the administrator will need to log out of the Console and provide the other server's connection details from the login page.

These features were added for version 7.0.0.0 of the Data Sync Server:

• Simplified management tasks related to configuring servers in a large cluster topology or in an automated deployment. Most notably, servers can now be added to a cluster while other servers are offline.

• Added management features for SSL/TLS certificates. The default certificates used in inter-server replication can be replaced; validation of client certificates for HTTPS-based services like the SCIM REST API can be configured; and you can reload from the trust store for HTTPS client certificates without restarting the server or the HTTP-based services.

• Added support for these operating system versions: Ubuntu LTS 16.04, CentOS 7.4, RedHat Linux 7.4, SUSE Enterprise 12 SP3

These were known issues at the time of the release of version 7.0.0.0 of the Data Sync Server:

• Simultaneous cloning multiple PingDirectory Proxy, PingData Sync, and PingDataGovernance Servers from another server of the same type is not currently possible. To create server instances that are identical to a master server, cloning must be performed one at a time.

These issues were resolved with version 7.0.0.0 of the Data Sync Server:

• Support for the IBM JDK has been retired. Issue:DS-35536

• Sync Pipe Plugins can now be enabled for specific Sync Classes when creating or editing a Sync Class. As before, any Sync Pipe Plugins enabled on the Sync Pipe will run for all associated Sync Classes. In the case where plugins have been enabled on the Sync Pipe and an associated Sync Class, the plugins enabled on the Sync Pipe will run before any enabled on the Sync Class. Issue:DS-17543

• Fixed a class loader issue where Sync Source extensions written using the Server SDK throw a ClassNotFound exception when importing classes not included in the Data Sync Server base classpath. Issue:DS-17347

• Updated the JMX connection handler's monitor provider so that when a JMX connection is closed, it is removed from the list of established connections. After a JMX client disconnects, it may take the server a few minutes to detect the closure and update the monitor. Issue:DS-35576

• The admin backend and the tool used to manage it, dsframework, have been replaced by the topology registry and dsconfig, respectively. The topology registry is automatically mirrored across all servers in the topology, so administrative information is kept in-sync on all servers at all times. Issues:DS-14281,DS-14282,DS-14283,DS-14284,DS-17197,DS-17366,DS-4570

• Added a new manage-certificates tool that can be used to perform a number of functions related to TLS certificate management. Issue:DS-17891

• Added a new Monitor Entry for SSL Cipher Suite and Protocol information. It is available under cn=SSL Context,cn=monitor. Issue:DS-35601

• Added a missing double-quote to bat/transform-ldif.bat, which prevented the command from being invoked successfully on Windows systems. Issue:DS-35648

• Updated the server to include an instance of the Periodic Stats Logger Plugin that is enabled out-of-the-box to aid in diagnosing support issues. The "Historical Stats Logger" plugin will log performance statistics to logs/monitor-history/historical-dsstats.csv every five minutes. This works in concert with the "Monitor History" plugin, which logs the full contents of cn=monitor to logs/monitor-history every five minutes. The tail of this csv file is automatically included in the output generated by collect-support-data. Issue:DS-35581

• Fixed a defect where configuring a Directory server on a Windows machine with a space in the home directory pathname would cause server setup to fail. Issue:DS-35583

• Added the ability to configure data encryption during setup using a randomly generated key, a key generated from a user-supplied passphrase, or a key obtained from an export of another server's encryption settings database. When setting up multiple instances, providing the same encryption passphrase to each instance will ensure that all instances have the same encryption key.

  The encryption-settings tool has also been updated to allow creating encryption settings definitions from a passphrase, to allow providing a description when creating a new encryption settings definition, and to record a create timestamp for new definitions. It is now possible to create ciphers that use the Galois Counter Mode (GCM) cipher mode (for example, using a cipher transformation of "AES/GCM/PKCS5Padding") for authenticated encryption. Definitions created with with just a cipher algorithm but no transformation will now use stronger settings.

  The default encryption settings export format now provides stronger encryption. Newer server instances should be able to import encryption settings exported from other servers without issue. When exporting encryption settings for import into older servers, use the new --use-legacy-export-format argument. Issues:DS-15223,DS-35895

• The create-systemd-script command now suggests placing the script created in "/etc/systemd/system." Issue:DS-35868

• Added an encrypt-file tool that can encrypt and decrypt data with a user-supplied passphrase, an encryption settings definition, or a topology key shared among server instances. It includes support for decrypting the content in encrypted backups, LDIF exports, and log files. Issue:DS-36054

• Fixed an issue with compressed logging that could leave some data buffered in memory and not actually written out to disk until the logger is closed. Issue:DS-36070 SF#:00628238

• Added support for encrypted logging, using a key generated from an encryption settings definition. Encrypted log files may be decrypted with the encrypt-file tool. Issue:DS-6970

• Made a number of improvements to backend backup and restore, and to LDIF export and import:

  * Added the ability to encrypt backups and LDIF exports with a key generated from a user-supplied passphrase or with a key generated from an encryption settings definition. Previously, encrypted backups and LDIF exports only used a secret key that was known only to servers within the replication topology. The new options make it easier to restore encrypted backups and import encrypted LDIF files in servers outside of the replication topology. The encrypt-file utility can be used to decrypt encrypted backups and LDIF exports, regardless of how the encryption key was obtained.

  * Added the ability to limit the rate at which backups and LDIF exports will be written to disk, which can help avoid performance problems that result from these operations saturating the disk subsystem.

  * Added new global configuration properties for automatically encrypting backups and LDIF exports by default, which will be set to true if data encryption is enabled during setup.

  * Added new global configuration properties that can specify which encryption settings definitions will be used to obtain the encryption keys for automatically encrypted backups and LDIF exports. If not specified, then the server will use its preferred encryption settings definition, or an internal topology key if no encryption settings definitions are available.

  * Added a new configuration property for automatically compressing encrypted LDIF exports.

  * Updated the backup tool to add new --promptForEncryptionPassphrase, --encryptionPassphraseFile, and --encryptionSettingsDefinitionID arguments that can be used to specify which key to use for encrypting the backup. Added a new --doNotEncrypt argument that can be used to force a backup to be unencrypted even if automatic encryption is enabled. Added a new --maxMegabytesPerSecond argument that can be used to impose a limit on the rate at which the backup may be written to disk.

  * Updated the restore tool to add new --promptForEncryptionPassphrase and --encryptionPassphraseFile arguments that can be used to provide a user-supplied passphrase for use in accessing the contents of an encrypted backup. For backups encrypted with an encryption settings definition or an internal topology key, the server will automatically be able to determine the correct key.

  * Updated the export-ldif tool to add new --promptForEncryptionPassphrase, --encryptionPassphraseFile, and --encryptionSettingsDefinitionID arguments that can be used to specify which key to use for encrypting the export. Added a new --doNotEncrypt argument that can be used to force an LDIF export to be unencrypted even if automatic encryption is enabled. Added a new --maxMegabytesPerSecond argument that can be used to impose a limit on the rate at which the LDIF file may be written to disk.

  * Updated the import-ldif tool to add new --promptForEncryptionPasshprase and --encryptionPassphraseFile arguments that can be used to provide a user-supplied passphrase for use in accessing the contents of an encrypted LDIF export. The --isEncrypted and --isCompressed arguments are no longer necessary, as the tool can automatically detect encryption and compression (although those arguments are still available to preserve backward] compatibility), and it can automatically identify the correct key for exports encrypted with a key obtained from an encryption settings definition or an internal topology key. Issues:DS-12157,DS-35896 SF#:3628

• Updated setup to include key usage, extended key usage, and subject alternative name extensions in the self-signed certificates that it generates. Issues:DS-35727,DS-35728

• Implemented invocation logging for several server tools, which will write to logs/tools/tool-invocation.log by default upon startup and shutdown. Some of the information recorded by log entries include the tool's start and completion times, the command-line arguments used to initialize them, and the name of the system account used to launch the tool. To modify this behavior, edit the config/tool-invocation-logging.properties file. Issue:DS-4406

• Updated the SCIM Sync Destination to always send credentials preemptively when configured to use HTTP basic authentication. Issue:DS-36198

• Updated tools that interact with log or LDIF files to support reading from input files that are compressed and encrypted and writing to compressed and encrypted output files. Issue:DS-36075

• Added support for TLS1.2 with STARTLS to connect to an SMTP server. Issue:DS-36093 SF#:00631871

• Added the ability to generate administrative alert notifications when a task starts running, when it completes successfully, or when it fails to complete successfully. Also added the ability to send an email message to a specified set of users when a task starts running or completes successfully, which complements the existing ability to send an email message when a task fails to complete successfully or when it completes with any state, regardless of success or failure. Issue:DS-426

• Added support for TLS1.2 with STARTLS to connect to SMTP server Issue:DS-36093 SF#:00631871

• Provided the means to request that the server dynamically reload the certificate key and trust stores used by all HTTP connection handler instances that provide support for HTTPS. The request can be made using a new reload HTTP connection handler certificates task, the reload-http-connection-handler-certificates tool, or programmatically from a Server SDK extension using the ServerContext#reloadHTTPConnectionHandlerCertificates method. Issue:DS-35990 SF#:00629638

• Added the PingOne for Customers Sync Destination for Ping Data Sync. Identities can now be synchronized from on premises identity stores to PingOne for Customers. Issue:DS-36000

• The update tool now enforces specification of a new product license when updating to a new major version. The license can be specified using the --licenseKeyFile command-line options, or by copying the license file to the top-level directory of the server package used to perform the update. Request a license key through the Ping Identity licensing website https://www.pingidentity.com/en/account/request-license-key.html, or contact sales@pingidentity.com. Issue:DS-35523

• In addition to specifying an exact set of desired cipher suites for the LDAP and HTTP Connection Handlers, administrators can now specify inclusions to, or exclusions from, the set of cipher suites selected by the server. Issue:DS-36088

• Added support for recurring tasks, which can be used to automatically invoke certain kinds of administrative tasks based on a specified schedule.

  At present, only certain kinds of tasks can be scheduled as recurring tasks. This includes both backups and LDIF exports, each of which provides retention support to limit the amount of disk space that the backups and LDIF files consume. It also includes support for any kind of task in which each instance of the task should use exactly the same values for all of the task-specific attributes. The Server SDK also provides an API for creating custom third-party recurring task implementations. Issue:DS-426

• Updated the server to reduce contention when converting between strings and the bytes that comprise those strings. Issue:DS-36328 SF#:626850

• Added a sanitize option to the Monitor History Plugin that, if enabled, will redact the small amount of potentially personally identifiable information that could appear in search filters and LDAP DNs within the monitor. This makes it easier to share the monitor history files with the support team in secure environments. Issue:DS-36545

• Fixed an issue where the password attribute could be deleted when using the PingData Sync Server with an Active Directory Sync Source. Issue:DS-36466 SF#:00631858

• Fixed a compatibility issue with PingFederate when it is used as a SCIM Sync Destination, in which a PingFederate server's SCIM schema response containing the schema for the User resource type could not be parsed. Issue:DS-36326

• Increased the default size of the queue used to hold alert notifications so they can be asynchronously processed by a background thread. This makes it less likely that the queue will become full if many alerts are generated in a short period of time, which would cause subsequent attempts to generate alerts to block while the server catches up. Also updated the server to log a message when the queue becomes full so that administrators will be aware of the problem and will have suggestions for addressing it. Issue:DS-36360 SF#:635134

• Updated the dsconfig list subcommands to list objects of all complexity levels rather than requiring the --advanced flag to list advanced and expert objects. Issue:DS-16508

These issues were resolved with version 6.2.0.0 of the Data Sync Server:

• Added a disabled-alert-type configuration property to the Alert Backend that can be used to suppress specific alert types from being added to the backend. Issue:DS-16906 SF#:3556

• The SNMP context name for the server can now be configured using the new context-name property of the SNMP Subagent Plugin. The server instance name remains the default context name when this property is not set. Issue:DS-16405

• The server now requires Java version 8. Issue:DS-17019

• It is now possible for Sync Destination Server SDK Extensions to ignore a change by calling the setIgnore() method on the provided sync operation. Ignoring a change prevents the change from being included as an applied operation in the statistics that are reported for the Sync Pipe via the Management Console, the bin/status command line utility, and in a raw form under "cn=Sync Pipe Monitor: <pipe name>,cn=monitor". Issue:DS-15051 SF#:2281

• Updated a couple of cases where filtered SCIM searches for groups with missing members were not returned. Issue:DS-17078 SF#:00003677,00003683

• The Data Sync Server now supports OpenDJ as an external server and as a sync source. Issue:DS-16423

• The Data Sync Server now supports generic LDAP servers as sync destinations. Issue:DS-17089

• The Data Sync Server now supports Oracle Unified Directory as an external server and as a sync source. Issue:DS-10016

• Updated the logic used to select which TLS cipher suites should be enabled by default, and the logic used to prioritize those cipher suites. The selection process has been updated to use the guidelines provided in the OWASP "Transport Layer Protection Cheat Sheet" document.

  Some of the changes include:

  - The server already preferred cipher suites that support forward secrecy over those that don't. It now prefers DHE over ECDHE, and avoids suites that use non-RSA keys.

  - The server already avoided cipher suites that used known-weak cryptographic weaknesses, including null encryption, the RC4 symmetric cipher, and the MD5 digest algorithm. It now also avoids anonymous encryption, the single-DES symmetric cipher, the IDEA symmetric cipher, and any suite using export-level encryption.

  - The server now prefers cipher suites that use the Galois/Counter Mode (GCM) over the Cipher Block Chaining (CBC) mode.

  - The server now prefers AES-based cipher suites with 256-bit keys over those that use 128-bit keys. For suites with equivalent key sizes, it prefers suites with a stronger message digest algorithm over suites with a weaker digest algorithm (e.g., SHA384 over SHA256 over SHA).

  - The server now provides better support for selecting and prioritizing ciphers when running on the IBM JVM. The IBM JVM uses somewhat different naming for its cipher suites than the Oracle implementation, which previously allowed certain desirable suites to not be included in the selected set. Issue:DS-17146

• Fixed an issue where incorrect names were displayed in the usage for the start scripts. Issue:DS-16593

• Removed the default root password from the out-of-the-box configuration. This password was never actually used because it was replaced by the user-supplied password provided when running setup, and it has been removed for additional security. Issue:DS-17318

• Updated the installer to discourage the use of weak root passwords.

  When run in interactive mode, setup will display a list of password quality recommendations before prompting for the initial root password, suggesting that it should be at least 12 characters long, should not be contained in a dictionary of English words, and should not be contained in a dictionary of commonly-used passwords. If the proposed password does not meet these constraints, then the user will be given the option of proceeding with the provided weak password or choosing a different password.

  When run in non-interactive mode, setup will exit with an error if the proposed initial root password does not satisfy the above constraints, unless the command line also includes the --allowWeakRootUserPassword argument.

  In either mode, when a strong initial root password is supplied, setup will also configure the root users' password policy to ensure that subsequent root user passwords will also be required to satisfy these constraints. Issue:DS-2074

• Updated the access and audit loggers so that, when logging information about an internal operation that was triggered by an external client request, the log message will include the connection and operation ID for that request. Also updated the error logger so that when logging a message from a thread that is actively processing an operation, the log message will include the connection and operation ID for that operation. Issue:DS-16509 SF#:3536

• For Active Directory external servers, it is now possible for the bind-dn property to be a User Principal Name (UPN). Issue:DS-16603

• Replaced the ldapsearch and ldapmodify tools with new versions. The new versions are backward-compatible, but offer a number of new features, including better connection handling, better output formatting, better support for bulk operations, support for referrals, support for additional request and response controls, and rate limiting. The ldapsearch tool now offers the ability to output results in JSON, CSV, or tab-delimited text as an alternative to LDIF, and provides support for a number of data transformations. The ldapmodify tool now supports the LDIF control syntax, as well as writing to output and reject files. Issues:DS-15861,DS-15862

• Improved error reporting for the manage-extensions tool. Issue:DS-17080

• The modifierName and modifyTimestamp attributes are now updated when offline configuration changes are made. Issue:DS-16858

• Corrected the port number returned in the error message that is displayed when an administrator is trying to set up a server that is already running. Issue:DS-13721

• The server now monitors important certificates used for client and inter-server communication. Certificate information is available in the Administrative Console and in the status tool output. An alarm is raised and alerts are sent when a monitored certificate is 30 days from expiration. Issue:DS-1029

• Fixed an issue that could cause a sync pipe to crash due to missing attributes in the changelog. Issue:DS-17531 SF#:00609552

• The Administrative Console can be deployed in an external web container, such as Tomcat, using the contents of resource/admin-console.zip, located in the server root. Issue:DS-17544

• Added an optional reason parameter for dsconfig changes that will be automatically included in the server's config-audit.log file. Issue:DS-811

• Updated the Server SDK to provide methods for obtaining a single LDAP connection or an LDAP connection pool with connections established to a specified LDAP external server defined in the server configuration.

  Also updated the server configuration to add support for obscured values. An obscured value is a general-purpose string that is stored in an obscured form in the configuration so that its plaintext value is not readily discernible to anyone looking at the configuration file and so that the value is not displayed in administrative interfaces. The Server SDK provides a method for obtaining the plaintext representation of an obscured value, and this mechanism can be used to store potentially sensitive values in the configuration for use in Server SDK extensions without the need to store those values in the clear. Issue:DS-10694

• Addressed an issue where the server would throw a NullPointerException if a com.unboundid.directory.sdk.sync.api.SyncSource implementation did not set a modifiers name in a generated ChangeRecord. Issue:DS-17653

• Fixed an issue in the Data Sync Server where attribute matching rules were not correctly applied when synchronizing. Issue:DS-17652 SF#:610262

• Sync correlation attributes now support correlating using JSON keys within JSON attribute values. A JSON key may be referenced with syntax <attribute name>.<JSON key> If a JSON correlation attribute is used and the JSON key does not exist for the source or destination entry or the source or destination entry considered does not have valid JSON data then the destination entry considered entry will not be matched. Issue:DS-17668

• Constructed attribute mappings now support multivalued source attributes for conditional ("conditional-value-pattern" property) and non-conditional ("value-pattern" property) value patterns. Only one of the source attributes that contribute to a given value pattern may be multivalued. Issue:DS-1045

• The Administrative Console is no longer compatible with older versions of the server. Issue:DS-17241

• Updated the server to reduce the use of the SHA-1 message digest. The server will now use a 256-bit SHA-2 digest instead of a SHA-1 digest in all of the following cases:

  - When hashing or signing a backup. - When signing an LDIF export. - When signing log data. - When generating MACs for an encrypted collect-support-data archive. - When generating unique identifiers for encryption settings definitions. - When determining whether the configuration changed with the server offline.

  In all of the above cases, the server includes metadata in the output of the cryptographic processing to indicate the digest or MAC algorithm used for that processing, which ensures that the output remains compatible across server versions. For example, an LDIF export that uses a signature generated with the SHA-2 digest can be successfully imported into older versions of the server.

  Also, the fingerprint certificate mapper has been updated so that it can use the 256-bit SHA-2 digest when mapping a client certificate to the corresponding user entry. The previous MD5 and SHA-1 digests remain supported.

  Finally, the example enhanced password storage scheme provided with the UnboundID Server SDK has been updated so that it uses the 256-bit SHA-2 digest instead of a SHA-1 digest. Issue:DS-17444

• Sync constructed attribute mappings now support extracting JSON fields within JSON attributes by appending '.' and then the JSON field to extract to the attribute name. For example, if the JSON attribute 'ubidEmailJSON' and the 'value' field is to be extracted, then 'ubidEmailJSON.value' could be specified for the attribute name resulting in '{ubidEmailJSON.value}' or '{ubidEmailJSON.value:/regex/replacement/flags}' if a regular expression is to be used. Issue:DS-17689

• LDAP changelog-based Sync Sources such as the DSEE Sync Source will no longer fetch source entries if the targetDN in the corresponding changelog entry is not contained in any of the Sync Source's configured base-dn values. This can reduce the load placed on the source directory servers. Issue:DS-17742 SF#:613996

• LDAP referral entries are now synchronized as a raw entry rather than trying to follow the referral. Issue:DS-17741 SF#:00613775

• Constructed attribute mappings now support modifers which alter the value of referenced source attributes when added to the end of source attribute references. For example, if attribute 'mail' is to be included in a constructed JSON value then modifier 'jsonEscape' could be specified resulting in '{{ "userMail":{mail:jsonEscape} }}' or '{{ "userMail":{mail:/regex/replacement/flags:jsonEscape} }}' if a regular expression is to be used. Note that '{{' expands to '{' and '}}' to '}'. The modifiers are: * jsonEscape Escape text for use in a JSON value. * ldapFilterEscape Escape text for use in an LDAP filter. * lowerCase Convert text to lower case. * trim Remove leading and trailing whitespace. * upperCase Convert text to upper case. Issue:DS-17693

• The Data Sync Server now logs an error and continues processing if it encounters a corrupted changelog entry. Previously, the server would continually throw an exception and stop processing subsequent changes. Issue:DS-17711 SF#:613375

• The script files used to stop and start the server have been renamed stop-server and start-server. The older scripts are still present but may be removed in a future release of the product. Issue:DS-16789

• The Data Sync Server now supports synchronizing particular fields within JSON attributes. The way fields are synchronized as well as the way source and destination values are correlated is controlled by a the "JSON Attribute" configuration object. Issue:DS-17688

• Updated the Server SDK's ServerContext to expose a ValueConstructor, which can be used to build String values using a value-pattern template that references attribute values within an Entry. See the Javadoc for the ValueConstructor class with the Server SDK packaging for more information. Issue:DS-17576

• The sync server will now try to use the source schema, if available, in cases where the destination schema is unavailable. Issue:DS-17936 SF#:00598940

• Critical: Fixed an issue with the sync connect and response timeouts being set with incorrect units of time. Issue:DS-18026 SF#:00616763

• Updated the Server SDK so that HTTPServletExtensions can be installed on Data Sync Server instances. Issue:DS-18003

• Added additional logging to the Data Sync Server for cases where attribute mapping fails, but does not abort the synchronization operation. Issue:DS-17993

• Limited the ACI search on collect support data tool to only pull 100 entries. This will reduce the time the tool takes to run for organizations with a large number of ACIs. Issue:DS-17968

• Added additional logging for ignored synchronization operations. Issue:DS-17356

• Added additional support to the Data Sync Server for synchronizing Active Directory Groups with more than 1500 members. Issue:DS-17697

• Added configuration options for setting the SSL Protocol and/or the SSL Cipher Suites to the HTTPS Connection Handler. Issue:DS-10748 SF#:00003622,614777

• Enhanced the HTTPS Connection Handler to send a HTTP Strict Transport Security header by default in all responses. Issue:DS-14650

• Updated the default configuration of the File-Based Access Logger (logs/access) so that requests from peer Data Sync instances are no longer suppressed. This simplifies troublehooting connection and health-checking issues between server instances. Issue:DS-18199

• Updated PingDirectory, PingDirectoryProxy, PingDataSync, and PingDataGovernance with the capability to run as Windows Services. Issue:DS-4161

• Fixed an incompatibility between Java and PKCS12 truststores and keystores that caused an error during Data Sync Server installations. Issue:DS-18169 SF#:00619730

• Enhanced Attribute Mapping in the Data Sync Server to allow users to filter out or exclude values from entries in the Sync Destination. Issues:DS-17545,DS-17546,DS-18008

• Enhanced the LDAP Sync Destination to allow administrators to configure options for synchronizing user passwords with clear-text, in cases where the LDAP Server does not support pre-encoded password synchronization. Setting password-synchronization-format:clear-text on the LDAP Sync Destination will now enable the clear-text synchronization of passwords. By default, passwords will only be synchronized in the clear over a secure connection. This can be overridden by setting require-secure-connection-for-clear-text-passwords:false on the LDAP Sync Destination. Issue:DS-18136

• Updated dsconfig batch mode to operate more efficiently over the WAN by consolidating the number of LDAP searches required to retrieve the full configuration when pre-validating configuration changes. Issue:DS-35495

• A license key is required when setting up a server for the first time. Request a license key through the Ping Identity licensing website https://www.pingidentity.com/en/account/request-license-key.html or contact sales@pingidentity.com. Issue:DS-18100

• Removed the ability to create custom HTTP trace loggers using the Server SDK. Issue:DS-18188

• Added a disabled-alert-type configuration property to the Alert Backend that can be used to suppress specific alert types from being added to the backend. Issue:DS-16906 SF#:3556

• The SNMP context name for the server can now be configured using the new context-name property of the SNMP Subagent Plugin. The server instance name remains the default context name when this property is not set. Issue:DS-16405

• The server now requires Java version 8. Issue:DS-17019

• It is now possible for Sync Destination Server SDK Extensions to ignore a change by calling the setIgnore() method on the provided sync operation. Ignoring a change prevents the change from being included as an applied operation in the statistics that are reported for the Sync Pipe via the Management Console, the bin/status command line utility, and in a raw form under "cn=Sync Pipe Monitor: <pipe name>,cn=monitor". Issue:DS-15051 SF#:2281

• Updated a couple of cases where filtered SCIM searches for groups with missing members were not returned. Issue:DS-17078 SF#:00003677,00003683

• The Data Sync Server now supports OpenDJ as an external server and as a sync source. Issue:DS-16423

• The Data Sync Server now supports generic LDAP servers as sync destinations. Issue:DS-17089

• The Data Sync Server now supports Oracle Unified Directory as an external server and as a sync source. Issue:DS-10016

• Updated the logic used to select which TLS cipher suites should be enabled by default, and the logic used to prioritize those cipher suites. The selection process has been updated to use the guidelines provided in the OWASP "Transport Layer Protection Cheat Sheet" document.

  Some of the changes include:

  - The server already preferred cipher suites that support forward secrecy over those that don't. It now prefers DHE over ECDHE, and avoids suites that use non-RSA keys.

  - The server already avoided cipher suites that used known-weak cryptographic weaknesses, including null encryption, the RC4 symmetric cipher, and the MD5 digest algorithm. It now also avoids anonymous encryption, the single-DES symmetric cipher, the IDEA symmetric cipher, and any suite using export-level encryption.

  - The server now prefers cipher suites that use the Galois/Counter Mode (GCM) over the Cipher Block Chaining (CBC) mode.

  - The server now prefers AES-based cipher suites with 256-bit keys over those that use 128-bit keys. For suites with equivalent key sizes, it prefers suites with a stronger message digest algorithm over suites with a weaker digest algorithm (e.g., SHA384 over SHA256 over SHA).

  - The server now provides better support for selecting and prioritizing ciphers when running on the IBM JVM. The IBM JVM uses somewhat different naming for its cipher suites than the Oracle implementation, which previously allowed certain desirable suites to not be included in the selected set. Issue:DS-17146

• Fixed an issue where incorrect names were displayed in the usage for the start scripts. Issue:DS-16593

• Removed the default root password from the out-of-the-box configuration. This password was never actually used because it was replaced by the user-supplied password provided when running setup, and it has been removed for additional security. Issue:DS-17318

• Updated the installer to discourage the use of weak root passwords.

  When run in interactive mode, setup will display a list of password quality recommendations before prompting for the initial root password, suggesting that it should be at least 12 characters long, should not be contained in a dictionary of English words, and should not be contained in a dictionary of commonly-used passwords. If the proposed password does not meet these constraints, then the user will be given the option of proceeding with the provided weak password or choosing a different password.

  When run in non-interactive mode, setup will exit with an error if the proposed initial root password does not satisfy the above constraints, unless the command line also includes the --allowWeakRootUserPassword argument.

  In either mode, when a strong initial root password is supplied, setup will also configure the root users' password policy to ensure that subsequent root user passwords will also be required to satisfy these constraints. Issue:DS-2074

• Updated the access and audit loggers so that, when logging information about an internal operation that was triggered by an external client request, the log message will include the connection and operation ID for that request. Also updated the error logger so that when logging a message from a thread that is actively processing an operation, the log message will include the connection and operation ID for that operation. Issue:DS-16509 SF#:3536

• For Active Directory external servers, it is now possible for the bind-dn property to be a User Principal Name (UPN). Issue:DS-16603

• Replaced the ldapsearch and ldapmodify tools with new versions. The new versions are backward-compatible, but offer a number of new features, including better connection handling, better output formatting, better support for bulk operations, support for referrals, support for additional request and response controls, and rate limiting. The ldapsearch tool now offers the ability to output results in JSON, CSV, or tab-delimited text as an alternative to LDIF, and provides support for a number of data transformations. The ldapmodify tool now supports the LDIF control syntax, as well as writing to output and reject files. Issues:DS-15861,DS-15862

• Improved error reporting for the manage-extensions tool. Issue:DS-17080

• The modifierName and modifyTimestamp attributes are now updated when offline configuration changes are made. Issue:DS-16858

• Corrected the port number returned in the error message that is displayed when an administrator is trying to set up a server that is already running. Issue:DS-13721

• The server now monitors important certificates used for client and inter-server communication. Certificate information is available in the Administrative Console and in the status tool output. An alarm is raised and alerts are sent when a monitored certificate is 30 days from expiration. Issue:DS-1029

• Fixed an issue that could cause a sync pipe to crash due to missing attributes in the changelog. Issue:DS-17531 SF#:00609552

• The Administrative Console can be deployed in an external web container, such as Tomcat, using the contents of resource/admin-console.zip, located in the server root. Issue:DS-17544

• Added an optional reason parameter for dsconfig changes that will be automatically included in the server's config-audit.log file. Issue:DS-811

• Updated the Server SDK to provide methods for obtaining a single LDAP connection or an LDAP connection pool with connections established to a specified LDAP external server defined in the server configuration.

  Also updated the server configuration to add support for obscured values. An obscured value is a general-purpose string that is stored in an obscured form in the configuration so that its plaintext value is not readily discernible to anyone looking at the configuration file and so that the value is not displayed in administrative interfaces. The Server SDK provides a method for obtaining the plaintext representation of an obscured value, and this mechanism can be used to store potentially sensitive values in the configuration for use in Server SDK extensions without the need to store those values in the clear. Issue:DS-10694

• Addressed an issue where the server would throw a NullPointerException if a com.unboundid.directory.sdk.sync.api.SyncSource implementation did not set a modifiers name in a generated ChangeRecord. Issue:DS-17653

• Fixed an issue in the Data Sync Server where attribute matching rules were not correctly applied when synchronizing. Issue:DS-17652 SF#:610262

• Sync correlation attributes now support correlating using JSON keys within JSON attribute values. A JSON key may be referenced with syntax <attribute name>.<JSON key> If a JSON correlation attribute is used and the JSON key does not exist for the source or destination entry or the source or destination entry considered does not have valid JSON data then the destination entry considered entry will not be matched. Issue:DS-17668

• Constructed attribute mappings now support multivalued source attributes for conditional ("conditional-value-pattern" property) and non-conditional ("value-pattern" property) value patterns. Only one of the source attributes that contribute to a given value pattern may be multivalued. Issue:DS-1045

• The Administrative Console is no longer compatible with older versions of the server. Issue:DS-17241

• Updated the server to reduce the use of the SHA-1 message digest. The server will now use a 256-bit SHA-2 digest instead of a SHA-1 digest in all of the following cases:

  - When hashing or signing a backup. - When signing an LDIF export. - When signing log data. - When generating MACs for an encrypted collect-support-data archive. - When generating unique identifiers for encryption settings definitions. - When determining whether the configuration changed with the server offline.

  In all of the above cases, the server includes metadata in the output of the cryptographic processing to indicate the digest or MAC algorithm used for that processing, which ensures that the output remains compatible across server versions. For example, an LDIF export that uses a signature generated with the SHA-2 digest can be successfully imported into older versions of the server.

  Also, the fingerprint certificate mapper has been updated so that it can use the 256-bit SHA-2 digest when mapping a client certificate to the corresponding user entry. The previous MD5 and SHA-1 digests remain supported.

  Finally, the example enhanced password storage scheme provided with the UnboundID Server SDK has been updated so that it uses the 256-bit SHA-2 digest instead of a SHA-1 digest. Issue:DS-17444

• Sync constructed attribute mappings now support extracting JSON fields within JSON attributes by appending '.' and then the JSON field to extract to the attribute name. For example, if the JSON attribute 'ubidEmailJSON' and the 'value' field is to be extracted, then 'ubidEmailJSON.value' could be specified for the attribute name resulting in '{ubidEmailJSON.value}' or '{ubidEmailJSON.value:/regex/replacement/flags}' if a regular expression is to be used. Issue:DS-17689

• LDAP changelog-based Sync Sources such as the DSEE Sync Source will no longer fetch source entries if the targetDN in the corresponding changelog entry is not contained in any of the Sync Source's configured base-dn values. This can reduce the load placed on the source directory servers. Issue:DS-17742 SF#:613996

• LDAP referral entries are now synchronized as a raw entry rather than trying to follow the referral. Issue:DS-17741 SF#:00613775

• Constructed attribute mappings now support modifers which alter the value of referenced source attributes when added to the end of source attribute references. For example, if attribute 'mail' is to be included in a constructed JSON value then modifier 'jsonEscape' could be specified resulting in '{{ "userMail":{mail:jsonEscape} }}' or '{{ "userMail":{mail:/regex/replacement/flags:jsonEscape} }}' if a regular expression is to be used. Note that '{{' expands to '{' and '}}' to '}'. The modifiers are: * jsonEscape Escape text for use in a JSON value. * ldapFilterEscape Escape text for use in an LDAP filter. * lowerCase Convert text to lower case. * trim Remove leading and trailing whitespace. * upperCase Convert text to upper case. Issue:DS-17693

• The Data Sync Server now logs an error and continues processing if it encounters a corrupted changelog entry. Previously, the server would continually throw an exception and stop processing subsequent changes. Issue:DS-17711 SF#:613375

• The script files used to stop and start the server have been renamed stop-server and start-server. The older scripts are still present but may be removed in a future release of the product. Issue:DS-16789

• The Data Sync Server now supports synchronizing particular fields within JSON attributes. The way fields are synchronized as well as the way source and destination values are correlated is controlled by a the "JSON Attribute" configuration object. Issue:DS-17688

• Updated the Server SDK's ServerContext to expose a ValueConstructor, which can be used to build String values using a value-pattern template that references attribute values within an Entry. See the Javadoc for the ValueConstructor class with the Server SDK packaging for more information. Issue:DS-17576

• The sync server will now try to use the source schema, if available, in cases where the destination schema is unavailable. Issue:DS-17936 SF#:00598940

• Critical: Fixed an issue with the sync connect and response timeouts being set with incorrect units of time. Issue:DS-18026 SF#:00616763

• Updated the Server SDK so that HTTPServletExtensions can be installed on Data Sync Server instances. Issue:DS-18003

• Added additional logging to the Data Sync Server for cases where attribute mapping fails, but does not abort the synchronization operation. Issue:DS-17993

• Limited the ACI search on collect support data tool to only pull 100 entries. This will reduce the time the tool takes to run for organizations with a large number of ACIs. Issue:DS-17968

• Added additional logging for ignored synchronization operations. Issue:DS-17356

• Added additional support to the Data Sync Server for synchronizing Active Directory Groups with more than 1500 members. Issue:DS-17697

• Added configuration options for setting the SSL Protocol and/or the SSL Cipher Suites to the HTTPS Connection Handler. Issue:DS-10748 SF#:00003622,614777

• Enhanced the HTTPS Connection Handler to send a HTTP Strict Transport Security header by default in all responses. Issue:DS-14650

• Updated the default configuration of the File-Based Access Logger (logs/access) so that requests from peer Data Sync instances are no longer suppressed. This simplifies troublehooting connection and health-checking issues between server instances. Issue:DS-18199

• Updated PingDirectory, PingDirectoryProxy, PingDataSync, and PingDataGovernance with the capability to run as Windows Services. Issue:DS-4161

• Fixed an incompatibility between Java and PKCS12 truststores and keystores that caused an error during Data Sync Server installations. Issue:DS-18169 SF#:00619730

• Enhanced Attribute Mapping in the Data Sync Server to allow users to filter out or exclude values from entries in the Sync Destination. Issues:DS-17545,DS-17546,DS-18008

• Enhanced the LDAP Sync Destination to allow administrators to configure options for synchronizing user passwords with clear-text, in cases where the LDAP Server does not support pre-encoded password synchronization. Setting password-synchronization-format:clear-text on the LDAP Sync Destination will now enable the clear-text synchronization of passwords. By default, passwords will only be synchronized in the clear over a secure connection. This can be overridden by setting require-secure-connection-for-clear-text-passwords:false on the LDAP Sync Destination. Issue:DS-18136

• Updated dsconfig batch mode to operate more efficiently over the WAN by consolidating the number of LDAP searches required to retrieve the full configuration when pre-validating configuration changes. Issue:DS-35495

• A license key is required when setting up a server for the first time. Request a license key through the Ping Identity licensing website https://www.pingidentity.com/en/account/request-license-key.html or contact sales@pingidentity.com. Issue:DS-18100

• Removed the ability to create custom HTTP trace loggers using the Server SDK. Issue:DS-18188

Important upgrade considerations for version 6.0.0.0 of the Data Sync Server:

• Note: The product names have been updated to reflect the UnboundID acquisition by Ping Identity. This is a naming and branding change only; the code base is the same as in prior releases and will continue to be maintained into the future.

• If upgrading the server that was running an older version of the JDK, run the dsjavaproperties --initialize command after the software upgrade to compare the settings of the older JDK with requirements for the new server software. Apply any necessary changes to the upgraded server based on previous performance settings.

• The 6.0 release makes these changes to supported platforms:

  • CentOS 7.2 and RedHat 7.2 are now supported operating system versions.
  • SUSE 11 SP4 is now supported; SUSE 11 SP2 support has been retired.
  • Linux KVM and VMWare ESXi 6.x have been added as supported virtual machine environments.
  • Deprecation of JDKs 7.x. Customers are strongly advised to use JDK 8 with this release. A later release of the platform will remove support for JDK versions 7.x. At that time customers will be required to upgrade to JDK 8.x when upgrading servers. This will apply to all of our JDK flavors (OpenJDK, OracleJDK, IBM JDK) on all platforms.
  • WildFly 9.x (renamed from JBoss) is now a supported application server for all web applications, including the Administrative Console and sample applications. Support for JBoss 7.x has been retired. Tomcat support remains unchanged in this release.

• PBKDF2 is now the default encoding for root passwords. This only affects new installations.

• In addition to changing the default password storage scheme for root users to PBKDF2, the default password storage scheme for regular users has been changed to salted 256-bit SHA-2.

• HTTPS defaults to ON: servers now default to use HTTPS for console and API connections including the SCIM API. This may affect automation scripts and development environments where HTTPS has not been in use before.

• Generated user passwords, for example those created by the server during a password reset sequence, are now created as pass-phrases instead of random character strings. This makes them them easier to type and remember. This change will not affect upgrades.

• The /config directory file permissions have been changed so that they are only accessible by the server user.

• Customers who choose to use the optional encryption algorithms provided by the third-party BouncyCastle library are encouraged to upgrade to BouncyCastle 1.54.

These features were added for version 6.0.0.0 of the Data Sync Server:

• The Data Sync Server will now recognize and correctly handle changes from Active Directory sources which include the non-standard "range=n-m" suffixes for multi-valued attributes.

• Added the ability to programmatically apply attribute maps in Server Extensions so that specific maps can be applied to a cascading set of changes generated by the extension.

• For Java developers whose tools and workflows make use of Maven, the Server SDK jar has been deployed to Maven Central so that a developer can now add the Server SDK as a project dependency by adding a few lines to a project's pom.xml. Also, developers can now generate a Server SDK project that Maven-aware IDEs such as IntelliJ IDEA can package into an extension bundle with no special configuration needed. This benefit extends similarly to continuous integration systems such as Jenkins.

• The dsconfig tool provides the ability to search for and quickly navigate to configuration objects and properties in which the name, synopsis, or description matches a provided pattern.

• A new rotate-log tool and task have been added, which can be used to trigger rotation of one or more log files.

• The Configuration API is now fully supported for all servers. In this release, the API was changed to match SCIM conventions for attribute naming, resource modeling, and the standard HTTP verbs. The UnboundID SCIM 2 SDK (available through GitHub) can now be used with the Configuration API.

• All servers have an updated web Administrative Console, which includes:

  • New layouts for operational statistics, processing time, queues, all monitors, and the list of installed extensions.
  • Alert and alarm displays, summarizing the data in cn=alerts and cn=alarms and based on the configured gauges. Plus, filtering and searching for these.
  • A new LDAP Schema Editor for importing schema files, validity checking, creation and editing of object classes and attribute types. The editor also supports viewing of the attribute syntaxes, inheritance, and indexes that exist for each attribute and the dependencies between object classes and attributes.

• The new Administrative Console can also be deployed to independent application servers instead of being co-hosted by the servers. This simplifies deployment models and increases separation between data and application layers.

• To assist with situations where a very large number of changes may cause disk, memory, and server start time to increase unexpectedly, alerting and gauge features have been added to the Recent Changes Database.

• Servers can now trigger events whenever log file rotation occurs. This includes "copy on rotate" and "summarize on rotate" listeners, as well as Server SDK support for creating custom log file rotation listeners.

• It is now possible to create, change, and remove root user accounts across the topology using the dsconfig tool and Administrative Console.

These were known issues at the time of the release of version 6.0.0.0 of the Data Sync Server:

• When deploying the Administrative Console in Tomcat 8, and accessing the Administrative Console application using Tomcat's Web Application Manager, some browsers (including Safari and Firefox) will generate a path URL that encodes the dash in ubid-console. This results in a path such as http://localhost:8888/ubid%2Dconsole/, which causes session management errors. To workaround this issue, copy and paste the generated link into a browser, and replace the encoded dash with a dash (-) character.

• The PermSize and MaxPermSize JVM properties are no longer supported in JDK 8, and will safely be ignored. These properties can be removed by modifying the config/java.properties file and running "bin/dsjavaproperties" while the server is offline.

• Security criteria for root passwords with the default configuration will be increased in a future release. This might affect automated installation scripts that currently use less secure passwords. This will not affect existing root accounts.

• The dsconfig tool and the Administrative Console enables creating and managing new Root DN users in this release. However, there is a limitation with changing the password of the currently logged in administrator. The ldappasswordmodify command can be used to change the administrator's password by providing the current and new password.

These issues were resolved with version 6.0.0.0 of the Data Sync Server:

• Updated the Active Directory Data Sync Source so that it correctly handles the "range" options on attributes returned with the DirSync control. Issue:DS-14765 SF#:00003110

• Updated command-line tools based on the LDAP SDK tool APIs to add the following features:

  • Tools can obtain default values for any arguments not provided on the command line from a properties file. If it exists, the server's config/tools.properties file will be used by default. Command-line arguments can be used to specify an alternate properties file or to indicate that no properties file should be used.

  • Tools can be launched in an interactive mode, in which the user is prompted for arguments used to establish and authenticate the connection, and for any other required arguments. The user can then use an interactive menu to specify values for any remaining arguments.
  Issue:DS-13700

• The resync tool now supports sync pipes that use notification mode. Issue:DS-14606

• Added a --prettyPrint option to the config-diff tool to make the output more human-readable. Issue:DS-14694

• Improved memory utilization when processing entries with very large attributes, to prevent possible data retention in memory. Issue:DS-14878 SF#:00003169

• Fixed an issue with the dsjavaproperties tool where java properties for PermSize and MaxPermSize could be added when using JDK 8, which no longer supports these options. Issue:DS-14857 SF#:3187

• Updated the bcrypt, crypt, PBKDF2 and scrypt password storage schemes so they can be used to create new instances. Issue:DS-14923

• A new API, applyMaps(), is available for Data Sync Server plugins to programmatically apply Sync maps. Issue:DS-14298 SF#:00002971

• Updated the Apache commons collections library to address the security vulnerability described by CVE 2015-4852. Issue:DS-14430 SF#:00003216

• Fixed a case where attribute syntax configuration changes would not apply to undefined attributes, which rely on default attribute types. Issue:DS-14979

• Collect-support-data tool now captures Kerberos config and log information. Issue:DS-13823

• Updated the server's support for the Twilio Messaging Service so that it uses the newer "Messages" API when sending SMS messages instead of the older "SMS" API. The older API has been deprecated, and Twilio now imposes a 120-character limit for messages sent via that API. The messages API allows the server to take advantage of the full 160 characters per SMS message. Issue:DS-14749

• Fixed an issue where destination attributes with hard-coded values were incorrectly excluded during a resync, when the --excludeSourceAttr option was used. Issue:DS-15087

• Server SDK extensions are now built with a Java source version of 1.7 by default. Issue:DS-15015

• Changed interactive setup default value for HTTPS enablement. Issue:DS-15221

• Updated the initial server configuration to improve security and usability. These changes apply only to new installations and will not be applied when updating an existing installation. Changes include:

  • Updated the default password policy to use a default password storage scheme that uses a salted 256-bit SHA-2 digest rather than a salted SHA-1 digest.

  • Updated the root password policy to use a default password storage scheme of PBKDF2 rather than salted 512-bit SHA-2.

  • Updated the secure password policy to use a default password storage scheme of PBKDF2 rather than a CRYPT variant that uses multiple rounds of 256-bit SHA-2.

  • Updated the password policy import plugin so that it will attempt to use the default password policy to select the password storage scheme(s) to use for entries that do not explicitly specify a password policy. The plugin will also fall back to using a salted 256-bit SHA-2 scheme instead of a salted SHA-1 scheme.

  • A number of weaker password storage schemes have been disabled by default, including base64, clear, unsalted MD5, salted MD5, 3DES, RC4, and unsalted SHA-1.

  • The default password policy has been updated to use a password generator that generates very strong yet memorable passphrases rather than a shorter and less-memorable string of randomly-selected characters.

  • Many of the server loggers have been updated to include additional log elements by default, including the instance name, requester DN, requester IP address, and request controls.

  • The exact match identity mapper has been updated to look at the mail attribute in addition to the uid attribute. When targeting a user with an authentication ID value (as when using SASL authentication or the proxied authorization v2 request control), it is now possible to specify an email address as an alternative to a user ID.

  • The UNBOUNDID-TOTP SASL mechanism handler has been updated to prevent TOTP password reuse by default.

  • Added new request criteria that make it possible to identify requests that target the root DSE or the subschema subentry. The global configuration has been updated so that requests targeting these entries will be in the default exceptions lists if the server is configured to reject insecure or unauthenticated requests.

  • Updated the template that setup generates for creating sample data to use a more logical and user-friendly numeric range. When the user requests N entries, setup would previously number the entries 0 through N-1 (for example, if the user requested 1000 entries, they would be numbered 0 through 999). It is logical for a user to expect them to be numbered 1 through 1000, but this change could break things that expecting to find an entry numbered with zero. To address this, if the user requests the server be populated with sample data, setup will create one more entry than actually requested so the numbering will go from 0 to N.
  Issues:DS-15183,DS-15220,DS-9407

• Updated the default file permissions for new installations on UNIX-based systems. Files and directories included in the zip file will be only be accessible to their owner (the user that unzipped the file) by default.

  Newly-created files and directories will also be assigned permissions that allow them to be accessed only by the account used to run the server. Existing configuration options for setting file permissions (the log-file-permissions and db-directory-permissions properties) will continue to behave as before. The new config/server.umask file will control the default permissions for all other newly-created files and directories. Issues:DS-13571,DS-13860,DS-7505 SF#:2703

• It is now possible to include literal curly brackets ('{}'s) in value patterns and conditional value patterns of constructed attribute mappings by doubling them. Literal '{' and '}' are specified by '{{' and '}}' respectively. Issue:DS-15187

• Updated the global ACIs that ship with the server to use a separate ACI for each control or extended request to allow by default, rather than grouping all desired controls together in one ACI and all desired extended requests together in a second ACI. This change will only be reflected in new installations, and not when updating an existing deployment. Issue:DS-15417

• Addressed an issue where dsconfig incorrectly allowed certain configuration objects to be deleted. Issue:DS-15400

• Provided a graphical tool, watch-entry, that is intended to demonstrate replication or synchronization latency by watching an LDAP entry for changes. If the entry changes, then the background of modified attributes will temporarily be red. Attributes can also be directly modified as well. Issue:DS-15437

• Updated interactive setup to display default values, and improved the overall layout and appearance. Issues:DS-15361,DS-15363,DS-15434

• Added more logging information when initializing web application and servlet extensions in case an extension causes conflicts or delays. Issue:DS-15466

• Updated setup to encode the root password with the PBKDF2 password storage scheme instead of SSHA512. Issue:DS-15521

• Increase the minimum memory requirements for the server process from 256MB to 384MB to accommodate the Administrative Console. Issue:DS-15571

• Updater tool will increase PermSize and MaxPermSize parameters to recommended value to prevent Java JVM pauses. Issue:DS-15522 SF#:00003324

• The Data Sync Server now uses the "max-connection-age" configuration property to limit the age of LDAP connections to external servers. Issue:DS-15513 SF#:00003309

• Fixed an error that could occur during upgrade when the configuration can not be loaded due to missing custom schema. Issue:DS-15592 SF#:3340

• Updated the Groovy Scripting Language version to 2.4.6. Issue:DS-15621

• The prepare-endpoint-server CLI included with the Data Sync Server will now correctly set permissions when the Synchronization user already exists. Issue:DS-15670

• The Data Services Markup Language (DSML) client and gateway components have been discontinued and are no longer available. Issue:DS-15753

• Added a new rotate-log tool to request the rotation of one or more log files. Issue:DS-10464

• Improved the error messages produced by the manage-extensions tool when attempting to install invalid extensions. Issue:DS-15412

• Improved the error messages and examples for create-rc-script and create-systemd-script by explicitly suggesting the use of sudo so that the scripts can modify protected files. Issue:DS-15178

• The former suite of Administrative Console applications, each of which were tied to a particular product (for example the dsconsole.war for the PingDirectory Server) are no longer available, and have been superceded by a new version of the Administrative Console capable of managing any server product. You can choose to access the Administrative Console by hosting it within a server, or by deploying it in an external servlet container. For the former, enable an HTTP Connection Handler and add the Administrative Console Web Application Extension to the handler. For the latter, download and unzip the management-console-[version].zip file, and install the ubid-console.war file according to your container's instructions. Issue:DS-15088

• Root DN User configuration entries can now be fully managed through the configuration management interfaces such as dsconfig and the Administrative Console. Issue:DS-15422

• Added support for log file rotation listeners, which allow for custom processing whenever a log file is rotated out of service so that the server will no longer write to it. A copy listener (which will copy the rotated log file to an alternate location, optionally compressing it in the process), and a summarize listener (which will invoke the summarize-access-log tool on the rotated log file) are included. The Server SDK also includes an API for creating custom log file rotation listeners. Issue:DS-4235

• Replaced the scramble-ldif tool with a more powerful transform-ldif tool with support for a number of additional transformation types. The new transform-ldif tool is backward compatible with the former scramble-ldif tool, and the scramble-ldif shell script and batch file are still included with the server to ensure compatibility with scripts that depend on that tool. Issue:DS-15108

• Improved the collect-support-data tool to include information provided by systemd on platforms that support it. Issue:DS-13401

• Improved the error messages for create-rc-script and create-systemd-script when the directory in which the script will be created does not exist. Issue:DS-15337

• Added the server's process ID to the output of the status tool. Issue:DS-10312

• Added a monitor entry for each Server SDK extension. Issue:DS-14548

• The Configuration API now returns unquoted, native Javascript values for integer, real number, and boolean properties. Duration and size property values, for example '1 w' or '100 G', continue to be represented as Javascript string types. Issue:DS-15175

• Added the ability to create local constants in LDIF template files using the new 'local' keyword. Issue:DS-14213

• Addressed a few issues in config-diff. In some situations, config-diff would not generate commands in an order that respected all dependencies. This has been fixed. Most expected warnings are now excluded by default but can be included in the output with the --includeAllWarnings option. The --sourceBindPasswordFile and --targetBindPassword are now applied in conjunction with the --targetConfigGroup and --sourceConfigGroup options. Issues:DS-10466,DS-10765,DS-14479,DS-15318,DS-16154

• Added support for setting the request header size in the Jetty http configuration server properties. Issue:DS-12191 SF#:00002580

• Updated the restore command so that it can no longer be used to restore a backup of the config backend. The command now points the administrator for safer ways to revert configuration changes, including using config-diff. Issue:DS-14704

• Updated the sanitize-log tool to add support for JSON-formatted access and error log files. Issue:DS-16224

• Added the ability to search for configuration objects and their properties by name with the dsconfig tool. Issue:DS-979

• Added a --dry-run option to dsconfig, which can be used in batch mode to validate the configuration changes in a batch file without applying them. Issue:DS-10946

• Tools used to prepare a server for access by another server, such as prepare-external-server, now validate base DN entries before any modifications are performed on the prepared server. Issue:DS-14807

• Fixed an issue that prevented the deletion of disabled debug loggers. Issue:DS-15622

• Sensitive attribute values can now be redacted in the sync and sync-failed-ops log files. By default, the value of encoded passwords are redacted. The redaction can be configured with the log-redaction-regex property in the global Sync configuration. Issue:DS-16728

These issues were resolved with version 5.2.0.0 of the Data Sync Server:

• The setup tool has been updated to use HTTPS for initial configuration. Unsecure HTTP can be enabled post-setup, or by using non-interactive setup. Issue:DS-12182

• Addressed cases where some messages may be suppressed in logs and alerts. Issue:DS-12287

• Updated the Configuration API output where properties and their values are listed to include those that are undefined. Issue:DS-12123

• Added support for running on Oracle Java 8 and OpenJDK 8 platforms. Issue:DS-12483

• The Configuration API has been updated to support filtering, sorting, and paging for object list operations. See the Administration guide for usage. Issue:DS-12245

• Added logging of all HTTP requests disallowed due to CORS. This should make it easier to debug HTTP 403/Forbidden errors. Issue:DS-12496

• The server can now detect an "out of file handles" situation on the operating system, and shut down to prevent running in an unreliable state. Issue:DS-12579 SF#:2655

• Update the Detailed HTTP Operation Log Publisher to log the correct return code (404 NOT FOUND) when a request is not handled by defined endpoints. Issue:DS-12576

• Fixed an issue caused by a JVM bug where a Sync Pipe becomes blocked after the error, "AckHandler#completed called with a value that's not pending," is reported to the error log. Issue:DS-12708 SF#:2680

• Fixed an issue where changes to SMTP External Server configurations did not take effect until after a server restart. Issue:DS-12285

• The Sync Source ignore-changes-by-dn configuration parameter now defaults to "cn=Sync User,cn=Root DNs,cn=config," which is the default synchronization user. This is done to prevent changes from being synchronized in a circular manner. Issue:DS-10558

• Now when setting the synchronization start point via the "set-startpoint" command it's no longer necessary that global synchronization be turned on. Issue:DS-5425

• The resync command has been updated to include the "Recent ops/second" statistic, which displays the operations per second for the last ten second iteration. The "Average ops/second" statistic has also been renamed to "Overall ops/second" to be consistent with other tools. Issue:DS-12782

• Improved server locking used by dsconfig in offline batch mode, so that the server lock is held for the entire batch duration, instead of for each invocation. Also, reduced the probability of contention for file locks used by server tools to determine the server status. Issue:DS-12969 SF#:2717

• Fixed a rare condition that might cause the logger rotation and retention thread to exit under heavy file system load or a network file system outage. Issue:DS-12880

• MakeLDIF templates now have the ability to escape special characters curly braces, angle brackets, and square brackets using a backslash. See config/MakeLDIF/examples-of-all-tags.template for further examples. Issue:DS-12798

• Constructed mappings now support mappings that are conditional with the "conditional-value-pattern" dsconfig parameter. Each parameter consists of an LDAP filter and a pattern. The LDAP filter must be matched for the pattern to be used. Issue:DS-11936

• Added a gauge to monitor the number of available file descriptors. This Available File Descriptors gauge can detect if a server if running out of file descriptors and degrade the server appropriately. Issue:DS-12727

• The setup and initial configuration tools now support offline modes that can be used to bootstrap the server configuration while it is not running. Also, files generated by theses tools are now saved to the server's resource directory. Issues:DS-12704,DS-8794,DS-9652

• Addressed an issue where data definition language (DDL) log field mappings for the JDBC error log were not previously documented. Issue:DS-13163

• Updated the resync command to display more error messages to the console and to a log file. The location of this log file defaults to <server-root>/logs/tools/resync-errors.log but can be overridden with the --consoleLogFilePath option. Issue:DS-10447

• Reduced the memory overhead of debug logging in high throughput environments by sharing logging buffers across multiple threads. Issue:DS-10010

• Updated the server's JVM arguments to always log garbage collection information to a rotating set of log files stored within logs/jvm/gc.log.N. The file system usage is limited to 300MB. If the server had previously been configured with VERBOSE_GC, then garbage collection logging information will no longer be logged to logs/server.out. Issue:DS-11522

• The following UnboundID product names have changed: - Identity Data Store to Data Store - Identity Proxy to Proxy Server - Identity Data Sync Server to Data Sync Server - Identity Broker to Data Broker Issue:DS-12799

• Updated the prepare-external-server tool to suppress output when run with the --quiet option. Issue:DS-13242

• When DSEE servers are prepared for synchronization, the nsslapd-changelogmaxentries DSEE configuration property is no longer set to 500,000. This is so that the size of DSEE's Retro changelog is limited entirely by the nsslapd-changelogmaxage configuration property. Issue:DS-2880

• Fixed a bug where using the advanced arguments of some tools would result in changing the saved complexity settings for the dsconfig tool. Issue:DS-12897

• Added a new search-logs tool. Similar to the command line tool 'grep,' this tool searches across log files to extract lines matching the provided pattern(s). The search-logs tool can handle multi-line log messages, extract log messages within a given time range, and include rotated log files. Issue:DS-3095

• Updated the create-systemd-script tool by adding resource limits for available open file descriptors (NOFILE), and shared memory reservations (MEMLOCK). The generated script lists the recommended file descriptors limit and the resource limit setting for enabling large page support. The settings in the create-systemd-script output supersedes prior documentation for setting the number of open file descriptors on non-systemd systems. Issue:DS-13678

• Fixed an issue with the resync command, where entries that were already synchronized were falsely reported as being modified. This occurred with Active Directory Sync Destinations. Issue:DS-13718

• Fixed an issue where stale state for remote sync pipes could be merged repeatedly with the active Data Sync Server after failover, or when starting and stopping the sync pipes. When a sync pipe is deleted, its saved sync pipe state will now be removed. Issue:DS-13769 SF#:00002859

• Updated interactive dsconfig to include an option to toggle between sorting similar properties together or sorting them alphabetically. Issue:DS-1706

• The collect-support-data tool now has the option to collect logging information within a specified time range via the '--timeRange' argument. Issue:DS-1261

• Improved the server's support for selecting TLS cipher suites. When the server is configured to use a specific set of cipher suites, it will now always validate that all of the configured suites are supported by the JVM. When the server is not configured to use a specific set of cipher suites, it will now customize the set of default suites to prioritize those using strong cryptography (especially those that offer forward secrecy), and exclude suites with known weaknesses. Issues:DS-12681,DS-13475

• Updated the alert handler configuration to indicate whether the alert handler should be invoked asynchronously in a background thread rather than by the thread that generated the alert. For alerts generated during the course of processing an operation, invoking potentially time-consuming alert handlers in a background thread can avoid adversely impacting the response time for that operation while still ensuring that administrators are made aware of the issue that arose. Issue:DS-12833

• Updated the server to provide support for SMTP connection pooling. When sending an email message, the server will attempt to reuse an existing SMTP connection rather than establishing a new connection for each message. Issue:DS-12833

• Fixed an issue where a set-startpoint command could fail when synchronizing from an Active Directory LDS source. Also, when synchronizing changes to an Active Directory LDS instance, the userAccountControl attribute is no longer included when creating new entries. Issue:DS-14061 SF#:00002925

• Updated the prepare-* tools to avoid unnecessary confirmation for trust of the prepared server's certificate when the --trustStorePath argument specifies a trust store that establishes trust. Issue:DS-12616

• Fixed an issue where an active Sync Server could accept a smaller change number when merging remote state for a changelog Sync Source. Issue:DS-14186 SF#:00002859

• Fixed a log publisher defect that would result in an unreadable file when both compression and signing were enabled at the same time. Issue:DS-13552

• The ldifsearch command now supports the option "---isCompressed" for LDIF files that have been compressed with gzip. Issue:DS-14140

• Added properties to the task backend for limiting the number of log messages retained in task entries, in order to limit the size of the in-memory representation of those entries. All log messages generated by a task will still be recorded in the server error log, even if they are not all retained in the corresponding entry in the task backend. Issue:DS-11067 SF#:2282

These were known issues at the time of the release of version 5.1.5.2 of the Data Sync Server:

• Delete operations might not be detected and synchronized when the UnboundID Data Sync Server is configured to only synchronize a subset of attributes from an UnboundID Data Store or Proxy Server. To ensure that delete operations are properly detected and synchronized, add "*" to the changelog-deleted-entry-include-attribute configuration property on the changelog backend in the Data Store.

These issues were resolved with version 5.1.5.2 of the Data Sync Server:

• Fixed an issue where a set-startpoint command could fail when synchronizing from an Active Directory LDS source. Also, when synchronizing changes to an Active Directory LDS instance, the userAccountControl attribute is no longer included when creating new entries. Issue:DS-14061 SF#:00002925

These issues were resolved with version 5.1.5.0 of the Data Sync Server:

• Fixed an issue with the resync command, where entries that were already synchronized were falsely reported as being modified. This occurred with Active Directory Sync Destinations. Issue:DS-13718

• Added a gauge to monitor the number of available file descriptors. This Available File Descriptors gauge can detect if a server if running out of file descriptors and degrade the server appropriately. Issue:DS-12727

Important upgrade considerations for version 5.1.0.0 of the Data Sync Server:

• The summarize-config tool is deprecated, and will be removed in future versions of the product. Use the config-diff tool with the "sourceBaseline" argument to list a summary of changes to the local server configuration.

These features were added for version 5.1.0.0 of the Data Sync Server:

• Identity Data Sync attribute mappings now support LDAP-filter syntax conditionals (the conditional-value-pattern property), which enables more flexible attribute-to-attribute mapping based on attribute values found in the incoming data.

These issues were resolved with version 5.1.0.0 of the Data Sync Server:

• The setup tool has been updated to use HTTPS for initial configuration. Unsecure HTTP can be enabled post-setup, or by using non-interactive setup. Issue:DS-12182

• Addressed cases where some messages may be suppressed in logs and alerts. Issue:DS-12287

• Updated the Configuration API output where properties and their values are listed to include those that are undefined. Issue:DS-12123

• Added support for running on Oracle Java 8 and OpenJDK 8 platforms. Issue:DS-12483

• Added logging of all HTTP requests disallowed due to CORS. This should make it easier to debug HTTP 403/Forbidden errors. Issue:DS-12496

• Critical: The server can now detect an "out of file handles" situation on the operating system, and shut down to prevent running in an unreliable state. Issue:DS-12579 SF#:2655

• Update the Detailed HTTP Operation Log Publisher to log the correct return code (404 NOT FOUND) when a request is not handled by defined endpoints. Issue:DS-12576

• The Configuration API has been updated to support filtering, sorting, and paging for object list operations. See the Administration guide for usage. Issue:DS-12245

• Fixed an issue caused by a JVM bug where a Sync Pipe becomes blocked after the error, "AckHandler#completed called with a value that's not pending," is reported to the error log. Issue:DS-12708 SF#:2680

• Fixed an issue where changes to SMTP External Server configurations did not take effect until after a server restart. Issue:DS-12285

• Fixed a rare condition that might cause the logger rotation and retention thread to exit under heavy file system load or a network file system outage. Issue:DS-12880

• MakeLDIF templates now have the ability to escape special characters curly braces, angle brackets, and square brackets using a backslash. See config/MakeLDIF/examples-of-all-tags.template for further examples. Issue:DS-12798

• Improved server locking used by dsconfig in offline batch mode, so that the server lock is held for the entire batch duration, instead of for each invocation. Also, reduced the probability of contention for file locks used by server tools to determine the server status. Issue:DS-12969 SF#:2717

• Constructed mappings now support mappings that are conditional with the "conditional-value-pattern" dsconfig parameter. Each parameter consists of an LDAP filter and a pattern. The LDAP filter must be matched for the pattern to be used. Issue:DS-11936

• Updated the resync command to display more error messages to the console and to a log file. The location of this log file defaults to <server-root>/logs/tools/resync-errors.log but can be overridden with the --consoleLogFilePath option. Issue:DS-10447

• Reduced the memory overhead of debug logging in high throughput environments by sharing logging buffers across multiple threads. Issue:DS-10010

These issues were resolved with version 5.0.1.0 of the Data Sync Server:

• Addressed cases where some messages may be suppressed in logs and alerts. Issue:DS-12287

• Updated the Configuration API output where properties and their values are listed to include those that are undefined. Issue:DS-12123

• The server can now detect an "out of file handles" situation on the operating system, and shut down to prevent running in an unreliable state. Issue:DS-12579 SF#:2655

• Fixed an issue caused by a JVM bug where a Sync Pipe becomes blocked after the error, "AckHandler#completed called with a value that's not pending," is reported to the error log. Issue:DS-12708 SF#:2680

These features were added for version 5.0.0.0 of the Data Sync Server:

• Java 7 is now required when setting up a new server or upgrading an existing server.

• Enabled support for the SSLv2Hello TLS protocol by default in JVMs that support it. This does not enable support for the insecure SSLv2 protocol, but it can improve compatibility with clients running older versions of Java that may start TLS negotiation with an SSLv2 client hello packet before negotiating to a higher version of the TLS protocol. Support for SSLv2Hello in the initial phase of negotiation does not in any way compromise the strength of the integrity and/or confidentiality protection that is ultimately negotiated between the client and the server.

• Added a Monitor History plugin that periodically records cn=monitor to timestamped files to aid in isolating intermittent problems. By default, it logs the full cn=monitor branch every five minutes to compressed files within logs/monitor-history/. Files are deleted automatically, but a sparse set of older files are kept to provide historical perspective on server performance. The collect-support-data tool has also been updated to collect a few of these files to aid in root cause analysis.

• Introduced the Configuration HTTP Servlet Extension, which can be used for querying and updating the configuration over a REST API. This feature is currently experimental and is subject to change in the future. Your feedback is welcome.

These issues were resolved with version 5.0.0.0 of the Data Sync Server:

• Fixed an issue with the prepare-endpoint-server tool, where the maxChangelogAge argument was not being applied when targeting UnboundID servers. Issue:DS-11487 SF#:2406

• Fixed the gauge configuration manager to only re-initialize the gauge that was changed, and not any of the other gauges that did not change. Issue:DS-11472

• Fixed the alarm manager to generate alarm-cleared alerts when internal alarms are cleared and the alarm manager's generated-alert-types property has the "alarm" value. Issue:DS-11541 SF#:2421

• Fixed the alarm manager to not include the details of the old alarm, (the alarm being cleared), in the "alarm-cleared" alert message. Issue:DS-11546

• Fixed the dsconfig tool to suppress all stray output when run in batch mode with the --quiet option. Issue:DS-10460

• Fixed an issue in which tools such as dsconfig, status, and dsreplication could not connect to the server over SSL or StartTLS. This occurred when a certificate was accepted with the 'Manually validate' option, while using the interactive LDAP connection menu. Issue:DS-11688

• Updated the alarm manager to not generate "alarm-normal" alert when a gauge's condition abates Issue:DS-11637

• Reduced the severity of the "unrecognized alert type" message in the error log from SEVERE_WARNING to NOTICE. The message now states that this is expected if the server is reverted to a version prior to the implementation of these alert types. Issue:DS-11453

• Removed the "alarm-normal" alert. Issue:DS-11730

• Updated the alarm manager to not persist normal alarms. Issue:DS-11719

• Updated the ExampleOverloadHandlerPlugin to monitor the alarm backend for delete actions, so that it can react appropriately to abating gauge conditions. Issue:DS-11719

• Critical: Disabled support for SSLv3 by default in the LDAP, HTTP, and JMX connection handlers, and for replication communication. The recently-discovered POODLE vulnerability could potentially allow a network attacker to determine the plaintext behind an SSLv3-encrypted session, which would effectively negate the primary benefit of the encryption.

  SSLv3 was initially defined in 1996, but was supplanted by the release of the TLSv1 definition in 1999 (and subsequently by TLSv1.1 in 2006 and TLSv1.2 in 2008). These newer TLS protocols are not susceptible to the POODLE vulnerability, and the server has supported them (and preferred them over SSLv3) for many years. The act of disabling SSLv3 by default should not have any adverse effect on clients that support any of the newer TLS protocols. However, if there are any legacy client applications that attempt to communicate securely but do not support the newer TLS protocols, they should be updated to support the newer protocols. In the event that there are known clients that do not support any security protocol newer than SSLv3 and that cannot be immediately updated to support a newer protocol, SSLv3 support can be re-enabled using the newly-introduced allowed-insecure-tls-protocol global configuration property. However, since communication using SSLv3 can no longer be considered secure, it is strongly recommended that every effort be made to update all known clients still using SSLv3.

  It is possible to use the server access log to identify LDAP clients that use SSLv3 to communicate with the server. Whenever an LDAP client establishes a secure connection to the server, or whenever a client uses the StartTLS extended operation to secure an existing plaintext connection, the server will generate a SECURITY-NEGOTIATION access log message. The "protocol" element of a SECURITY-NEGOTIATION access log message specifies the name of the security protocol that has been negotiated between the client and the server, and any SECURITY-NEGOTIATION messages with a protocol of "SSLv3" suggest that the associated client is vulnerable to the POODLE attack. In addition, if any connections are terminated for attempting to use the disallowed SSLv3 protocol, the access log message for that disconnect should include a message stating the reason for the termination. Issue:DS-11782

• Updated the Web Console so that upon login, the user's old session is always invalidated. Issue:DS-11624

• Updated the Web Console to suppress LDAP responses in user messages, such as when the server is unavailable or for authentication failures. Also added a context parameter to exclude stack traces and detailed error messages from appearing in the application's internal error page. Issues:DS-11629,DS-11645

• Fixed incorrect property references for trustStorePassword and keyStorePasswordFile in tools.properties that corresponded to the wrong argument names. Issue:DS-11751

• Updated the setup tools to enable definition of external server instances that are configured to reject unauthenticated requests. Previously the tools would erroneously indicate these servers were unavailable. Issues:DS-11068,DS-11784,DS-11887

• Increased the possible parallelism within a Sync Pipe by skipping past the operation at the head of the incoming queue, if it cannot be processed, because it depends on an uncompleted active operation. This increases the overall throughput of a Sync Pipe when the stream of incoming changes includes many dependent operations, which must be processed in order. Examples of dependent operations include changes to the same entry or changes to entries that have a parent child relationship. With these changes, the Sync Server still guarantees that dependent operations are processed in order. Issue:DS-11947 SF#:00002516

• Disabled log rotation during startup to prevent potential problems with rotation dependencies on server components that have not yet been initialized. Issue:DS-10441

• Added a gauge to the server to track JVM memory usage and alert if the amount of free memory gets low enough that it could impact server performance. Issue:DS-11993

• Updated the server to make it easier to control the order of values in the ssl-protocol and ssl-cipher-suite properties in the LDAP connection handler and crypto manager configuration objects. Issue:DS-12147

• Updated the HTTP Connection Handler to return a 404 Not Found response to requests for endpoints not handled by any servlet or web application extensions. Previously the hander would return a 200 OK with no response body. Issues:DS-12120,DS-8368

These features were added for version 4.7.0.0 of the Data Sync Server:

• Updated the server to support Alarms. An alarm represents a stateful condition of the server that might indicate a problem, such as low disk space or external server unavailability. The status command line utility and the monitoring page of the web console have been updated to expose the active alarms. Many existing alert types have been updated to be treated as alarms. When the condition associated with an alarm abates, the alarm is cleared.

• Added support for Gauges. A gauge examines specific server monitoring data, and raises an alarm when a configured threshold is crossed. The server has out-of-the-box gauges such as CPU Usage and Disk Busy, and new ones can be added through the Gauge Data Source and Gauge configuration object types.

• OpenJDK 7 is now supported on Linux.

These were known issues at the time of the release of version 4.7.0.0 of the Data Sync Server:

• JDK 6 is currently deprecated and will not be supported in the next major release.

• UnboundID products are not supported on JDK 8.

These issues were resolved with version 4.7.0.0 of the Data Sync Server:

• Fixed the web console so that attempts to reconnect (after the console is restarted) succeed. Issue:DS-11043

• Updated the server preparation tools to use secure communication when setting up a Data Store for access over TLS. Previously the tools may fail when the server is configured to reject insecure requests. Issues:DS-11058,DS-6200

• Added a result code tracker that maintains a monitor entry with counts and response times of results. Each result is categorized by operation type, post-response result code, and whether it is a failure or non-failure. Issue:DS-3270

• If a Sync Class is configured with the ignore-zero-length-values property set to true, values that are empty after attribute mapping are now ignored. Issue:DS-11293

• The Password Sync Agent for Active Directory no longer restricts installation on specific versions of Windows. The admin guide will contain the current list of supported versions, and support for new releases may be requested as needed. Issue:DS-11373 SF#:2384

• Fixed a problem that prevented the server from starting if a TLS-enabled connection handler was configured with a certificate nickname that referenced a non-RSA certificate. Issue:DS-10949

These were known issues at the time of the release of version 4.6.0.0 of the Data Sync Server:

• UnboundID products, Java SE, and the JVM do not use OpenSSL libraries and are therefore not vulnerable to OpenSSL issues. Oracle has provided a statement on the April 2014 OpenSSL Heartbleed vulnerability at http://www.oracle.com/technetwork/topics/security/opensslheartbleedcve-2014-0160-2188454.html. Issue:DS-10807

These issues were resolved with version 4.6.0.0 of the Data Sync Server:

• Updated the validate-file-signature tool to ensure that it will always display a final summary message to indicate whether any warnings or errors were encountered during processing. Issue:DS-10333

• Updated the signed logging implementation to better handle any problems that may arise during cryptographic processing. If any such problem is encountered, the server will now include a message with information about the error in the signature block rather than suspending the logger with an exception recorded in the server.out log file. Issue:DS-10310

• Fixed an issue in the Periodic Stats Logger, where no logging would occur when suppress-if-idle=true was configured, even when the server was not idle. Issue:DS-10387 SF#:2170

• Added a new sanitize-log tool that can be used to remove sensitive information from server log files, including the file-based access log, the operation timing access log, the file-based error log, the file-based sync log, the file-based resync log, and the detailed HTTP operation log.

  The sanitization process operates on fields that consist of name-value pairs. The field name and equal sign will always be retained, but in cases where the value may contain sensitive data, that value may either be replaced with the string "---REDACTED---", or it may be tokenized. If the tokenized value is a DN or filter, then attribute names in that DN or filter will be preserved while the values will be replaced with a string consisting of a number inside curly braces. If the tokenized value is not a DN or filter, then the entire value will be replaced with a number inside curly braces. If a string to be tokenized appears multiple times in the log, the same replacement token will be used for each occurrence of that string to make it possible to correlate occurrences of that string without revealing the actual content.

  The sanitize-log tool has a default configuration that should be sufficient for many environments, allowing it to tokenize or redact sensitive information while preserving non-sensitive content for use in diagnosing problems or understanding usage patterns. However, this behavior can be customized using command-line arguments by indicating whether to preserve, tokenize, or redact a given log field. Issue:DS-10472

• Fixed issues with the JDBC Access Logger that were related to Oracle Thin Client, where column values were "null" and disabling the logger resulted in losing a connection to the server when using the dsconfig command. Issue:DS-10485

• Update the server to support pulling virtual attribute values from delete operation changelog entries, which enables using these virtual attributes reliably as destination-correlation-attributes and/or destination-correlation-attributes-on-delete settings Issue:DS-10253 SF#:2150

• Fixed an issue so that collect-support-data now generates filename entries correctly. Previously, the tool would hang if the archiving of files following a symbolic link required generating a non-duplicating filename entry. Issue:DS-10582

• Enabled the Host System Monitor Provider by default so that system CPU and memory utilization will be reported automatically through the server's monitoring framework. Disk and network monitoring can be enabled by configuring values for the disk-devices and network-devices configuration properties. Issue:DS-10562

• The default timeout period for smtp-timeout was changed from none to two minutes to prevent non-responsive mail servers from disrupting administrative functions. Issue:DS-10230

• Updated configuration object descriptions and menu items to reflect that PingDirectory Server Enterprise Edition support applies to both Sun and Oracle branded versions of the product. Issue:DS-10449

• The setup command no longer saves user-provided key store and trust store passwords in PIN files. Passwords provided during setup are encrypted with the configuration data. If the administrator chooses to use PIN files to supply the passwords, the files are referenced in the server configuration by the key manager and trust manager. Issue:DS-10787

• Updated the access logger so that result messages include user-friendly names for result codes in addition to their numeric values. Issue:DS-9946

• Updated the Periodic Stats Logger to include an empty value in the output rather than "infinity" in certain circumstances. This avoids problems plotting the output in a spreadsheet. Issue:DS-8842

• Updated dsconfig to treat tabs as whitespace in batch files. Issue:DS-10549

• Added Metrics Collection Size Limit Retention Policy to the metrics backend to allow up to 2 GB of metric data to be buffered locally, which allows the Metrics Engine to be offline for a longer time without missing collected data. Issue:DS-10156

• Removed deprecated "lshal" command from Linux-specific processes performed by the collect-support-data tool and added similar command, "udevadm info --export-db" Issue:DS-10713

• Updated the Replication Servers table produced by the dsreplication tool to omit unnecessary "Security" column. Issue:DS-10442

These features were added for version 4.5.1.0 of the Data Sync Server:

• The collect-support-data tool now refers to tools.properties for default command-line options.

• The collect-support-data tool now supports an option to encrypt the data archive, to ensure protection of customer data while in transit, and an option to reduce the amount of potentially sensitive data that is collected.

• Cross-origin Resource sharing (CORS) support is now included for HTTP Servlet Extensions, including the SCIM RESTful APIs.

These were known issues at the time of the release of version 4.5.1.0 of the Data Sync Server:

• When the Velocity servlet receives CORS-enabled requests and has a cross-origin policy in effect, it will return multiple Access-Control-* headers with duplicate values. This will cause cross-origin requests issued by web browsers to fail. Issue:DS-10205

These issues were resolved with version 4.5.1.0 of the Data Sync Server:

• Update the make-ldif tool to no longer assign the objectClass value of extensibleObject to branch entries. If needed, "objectClass: extensibleObject" can be added explicitly to the branch definition. Issue:DS-8530

• Enhance the ignore-zero-length-values configuration option for Sync Class during add operations to not include a destination attribute when the source attribute has only zero length values, or no values at all. Issue:DS-10021 SF#:2105

• Enhance the ignore-zero-length-values configuration option for Sync Class during "resync" for add operations to not include a destination attribute when the source attribute has only zero length values, or no values at all. Issue:DS-10046 SF#:2115

• The setup tool's --aggressiveJVMTuning and --verboseGC command-line options have been deprecated. Instead, use --jvmTuningParameter AGGRESSIVE and --jvmTuningParameter VERBOSE_GC respectively. Issue:DS-9079

• Update the server configuration to use a new default limit for duplicate alert suppression. The previous default imposed a maximum of 100 alerts of the same type per hour. The new default imposes a maximum of 10 alerts of the same type every ten minutes. This is more likely to suppress bursts in which the same alert is repeatedly generated over a short time without interfering with multiple occurrences of alerts of the same type over a longer period of time. Issue:DS-9259

• Update the Server SDK convenience method TransactionContext#searchToRawEntry to support parsing java.util.Date column types including subclasses such as java.sql.Timestamp. This enables natively parsing columns with a DATE type defined in Oracle databases. Issue:DS-9995

• Update the server to improve the caching behavior for PIN files as used by key and trust manager providers. In the case that the keystore or truststore file has been updated to require a new PIN and the existing PIN file is updated without a configuration change to the associated key or trust manager provider, the server would previously keep trying to use the old PIN. It will now look for and use an updated PIN if a failure is encountered while using the old PIN. Issue:DS-10113 SF#:2123

• Update the collect-support-data tool so that it can encrypt the data that is captured to protect it from unauthorized third parties. The encryption key is generated from a passphrase which may be read from a file, interactively provided by the user, or dynamically generated by the tool. This passphrase must be provided to support personnel (ideally over a different communication channel than the encrypted support data archive itself) for them to be able to access the information it contains.

  There is also a new option to decrypt an encrypted collect-support-data archive when provided with the encryption passphrase. Issue:DS-10129

• Update the collect-support-data tool so that it is possible to configure default values for most arguments in the tools.properties file. Issue:DS-10178

• Fix an issue in the Active Directory Password Sync Agent to prevent resending detected changes to the Identity Data Sync Server in situations where the server is not able to process changes because the Sync Destination is unavailable. Issue:DS-10166 SF#:2097

• Update the collect-support-data tool to further reduce the possibility of gathering sensitive information. Potentially sensitive data will be replaced with ---REDACTED--- in the output. A new "--securityLevel maximum" option can also be specified that redacts DNs and search filters, which might include personally identifiable information. Issue:DS-10115

These features were added for version 4.5.0.0 of the Data Sync Server:

• A new config-diff command line utility can compare two server configurations and produce the difference as a dsconfig batch file. The file can then be used to bring the source configuration in line with the target. Comparisons can be done between live servers or configuration files, and between current or legacy configurations. Run 'config-diff --help' to get more information including example use cases.

These were known issues at the time of the release of version 4.5.0.0 of the Data Sync Server:

• Java 1.7 has a synchronization bottleneck in HashMap that severely impacts performance. Use update 1.7u40, if possible, to avoid this issue. Issue:DS-9477

These issues were resolved with version 4.5.0.0 of the Data Sync Server:

• Fix a bug in the JDBC Access Logger that could cause incompatibility with some database versions and display a "Cannot commit when autoCommit is enabled" error message. Issue:DS-8750

• Update the server startup process so that if no messages have been logged for at least five minutes, the server will generate and log a message about the current phase of startup processing. This can help reassure administrators that the server is still starting and provide information about what phase of startup may be taking so long. Issue:DS-7450

• Update java.properties generation so that comments related to alternative JVM tunings are no longer present in the file. In most cases, rather than updating java.properties by hand you should use the dsjavaproperties tool to generate JVM options. Issue:DS-8339

• Add an allow-insecure-local-jmx-access option to the global config that will expose JMX data via insecure local JVM connection Issue:DS-4300

• Add a new alert handler that can use the Twilio service to deliver administrative alerts via SMS. Long alerts may be either truncated or split into multiple SMS messages. Issue:DS-5587

• Update the configuration schema to make the ds-cfg-inherit-default-root-privileges attribute mandatory for object class ds-cfg-root-dn-user which is used to define Root User DNs. When this attribute is not present on Root DN User entries, the effect is for the root user to inherit default privileges. It has been made mandatory to make this behavior more explicit. During an update of the server, root DN user entries that do not explicitly declare a value for this attribute will be updated with a value of 'true'. Issue:DS-8450

• Add a WebLogic specific descriptor file for the web console to help with deployment compatibility. Issue:DS-8925 SF#:1915

• The trust store password options have been deprecated for most tools that do not require read-write access to a trust store. Issue:DS-8789

• Make a number of criteria-related improvements:

  - Add Server SDK support for creating custom connection, request, result, search entry, and search reference criteria implementations.

  - Update the simple request criteria type to make it possible to consider the search scope in determining whether a search operation matches the criteria.

  - Update the simple result criteria type to make it possible to consider the indexed/unindexed status in determining whether a search operation matches the criteria.

  - Add a new type of request criteria that may be used to more easily identify operations that target the server root DSE.

  - Add a new type of result criteria that may be used to classify operations based on replication assurance requirements and/or whether those requirements were satisfied.

  - Add a new allowed-insecure-request-criteria global configuration property that may be used to identify requests that the server should allow even if they are received over an insecure connection and the server is configured to reject insecure requests.

  - Add a new allowed-unauthenticated-request-criteria global configuration property that may be used to identify requests that the server should allow even if they are received over an unauthenticated connection and the server is configured to reject unauthenticated requests. Issues:DS-5079,DS-8168,DS-8770

• Add a new sign-log configuration property to file-based loggers that may be used to cause the server to digitally sign messages written by that logger. A new validate-file-signature tool may be used to verify signature information in signed log files, as well as LDIF files generated by signed LDIF exports. Issue:DS-8662

• Add support for two new extended operations. A list configurations extended operation may be used to obtain information about the configurations that are available within to the server, including the currently-active configuration, the baseline configuration (i.e., the base configuration for an out-of-the-box installation of the current version), and all archived configurations that reflect configuration changes over time. A get configuration extended operation may be used to retrieve a specific configuration from the server. Issue:DS-9149

• Fix a bug that displayed a null pointer exception when using the create-sync-pipe-config for Active Directory and choosing to not sync CREATE operations. Issue:DS-9065 SF#:1954

• Update setup to fix a bug in which file path options specified as relative to the current directory may cause the server to be configured incorrectly or cause setup failure. Issue:DS-8389

• Update the HTTP Connection handler to support configuration for tracking sessions either through HTTP cookies or by URL rewriting. Issues:DS-8639,DS-9128

• Fix a bug where Changelog Password Encryption Plugin would throw an exception when processing a password change from an Active Directory Sync source. Issue:DS-9178 SF#:1954

• Add special handling when creating users on an Active Directory Sync Destination that will set the userAccountControl attribute to an appropriate value. Attribute Mapping for userAccountControl is now strongly discouraged. Issue:DS-9108

• Add fallback system for Active Directory Sync sources that detect deletes that do not have a distinguished name associated with the change. We will now attempt to use the objectGUID as the DN for those sync operations. Issue:DS-9187 SF#:1997

• Improve error handling for DN attribute mapper to tolerate invalid DNs by not mapping the invalid DN, instead of not mapping all of the DN values when one is invalid. Issue:DS-9291 SF#:2001

• Update the server to provide a degree of sandboxing around Server SDK extensions so that an unexpected exception thrown by an extension will be caught and result in an administrative alert rather than being caught further up in the stack and potentially causing other problems. Issue:DS-9247

• In the rare cases where it is necessary to forcefully terminate the JVM from within the server itself, ensure that any files marked for deletion when the JVM shuts down are manually deleted before the JVM is terminated. This can help avoid problems like server shutdown not being detected properly because the server PID file hasn't been removed. Issue:DS-9267

• Provide improved schema validation to detect additional cases in which certain misspelled tokens in the definition for a schema token could be silently interpreted as an extra property for that schema element. The server will now log a warning message about these unexpected tokens so that administrators can either correct them or prefix them with "X-" to indicate that they are an extra property provided for informational purposes. Issue:DS-9236

• Reduce the time it takes the server to shut down in certain situations. Background threads sometimes missed a signal to wake up and had to wait for their next polling interval to see that a shut down had been requested. Issue:DS-9334

• Update the default behavior of all file-based loggers to have include-thread-id=true. This will include a compact thread ID in all log messages. This can make it easier to correlate log messages generated by the same thread within a single log file or across different types of log files. Issue:DS-9352

• Fix a bug that could prevent correlation of attributes with binary values. Most notably objectGUID from Active Directory. Issue:DS-9187 SF#:1997

• Remove -XX:+UseMembar from the default set of generated JVM properties except on early JVM versions where this setting was required to work around a threading bug in the JVM.

• Update the server JVM arguments generated by setup and dsjavaproperties to explicitly define -XX:MaxNewSize and -XX:NewSize for JVM's 1GB in size and larger. Also, add a comment to the generated java.properties file directing the administrator to use dsjavaproperties for making memory-related changes to this file rather than editing it directly. Issue:DS-9227

• Add password file arguments to the scripts used to prepare external servers. Issue:DS-9406

• Update the make-ldif utility such that first and last names are now generated randomly instead of in sequential alphabetical order. The original ordered behavior can be enabled by using the --orderedNames option. Issue:DS-9504

• Update the setup and dsjavaproperties tools to permit maximum heap size values for memory that is not currently available on the host, though the value must still be less than the total amount of memory present on the host. Issue:DS-9111

• Update the setup and dsjavapropeties tools to permit JVM heap size values to be as large as the amount of memory present on the system would permit. Issue:DS-9494

• Update the Server SDK to provide the ability to run command line utilities within the server process. A ToolExecutor can be retrieved from the ServerContext. Currently, only the config-diff command is supported, but additional commands might be supported in the future. Issue:DS-9537 SF#:00001858

• Enhance dsconfig to write to the config audit log when in offline mode. Issue:DS-1495

• On Linux, issue a warning on startup and after a JVM pause if the kernel setting vm.swappiness is not 0 as this can cause the server to become unresponsive for several seconds when memory is paged back from disk during a garbage collection. Issue:DS-9070

• Automatically record server monitor data at shutdown, as it may be useful for debugging purposes in cases where a problem was experienced within the server that was resolved by a restart. Issue:DS-9777

• Improve the performance of certain monitor entry searches that target specific monitor entries by object class. In particular, this includes searches with AND or OR filters, as well as filters that target object classes not defined in the server schema. Issue:DS-9772

• Lower the severity from SEVERE to MILD for attribute mapping failures due to missing source attribute. Issue:DS-9858 SF#:2068

These features were added for version 4.1.0.0 of the Data Sync Server:

• The UnboundID Identity Broker is the first of a new class of components for consumer and subscriber identity management architectures.

  As a stand-alone server, it provides authorization decisions for client applications, provisioning systems, API gateways, and analytical tools in any architecture involving personal, account, or sensitive identity data.

  Working together with the UnboundID Identity Data Store and Identity Proxy, the Identity Broker is designed to make high-volume and high-speed authorization decisions based on ever-changing consumer profile and consent data. Functionally, the Identity Broker is both the Policy Decision Point and the OAuth2 provider for externalized authorization. Performance-wise, the Identity Broker can support the request volumes driven by the complex, real-time interactions necessary to support today's consumer-facing mobile, social, and cloud ecosystems.

These issues were resolved with version 4.1.0.0 of the Data Sync Server:

• Add support for RPM based installation. Issue:DS-5990

• Update the names of the UnboundID-branded products which are now: - Identity Data Store (formerly PingDirectory Server) - Identity Proxy (formerly Proxy Server) - Identity Data Sync Server (formerly Synchronization Server) - Metrics Engine (name unchanged) Issues:DS-7514,DS-7515,DS-7516,DS-7518

• Add ability to set the maxHeapSize and listenAddress arguments in a properties file when running setup. Issue:DS-6003

• Fix a bug where current heap size was not displayed in error message about being too low. Issue:DS-7292

• Fix a bug where under certain error conditions the start server scripts could prompt user to overwrite existing file. Issue:DS-7268

• Introduce an "include-thread-id" configuration property on many of the file-based loggers that when enabled adds a threadID field to logging output. This makes it possible to know exactly which thread logged a message, which can simplify correlating errors between log messages and separate logs. This ID can be correlated to a thread name using the cn=JVM Stack Trace,cn=monitor entry. Issue:DS-8212

• Change stop-*.bat to attempt a soft server shutdown before terminating the process. Issue:DS-408

• Addressed an issue in the monitoring pages of the web console where they incorrectly listed directory server replication related information. Issue:DS-7289

• Update the JMX connection handler to infer an appropriate Java type (e.g. Boolean, Long, Float, Date, or String) for JMX attributes from the underlying LDAP attribute type and value. The legacy behavior to return all JMX attributes as String values can be set if desired through the advanced global configuration property 'jmx-value-behavior'. Issue:DS-7635

• Add the --noPropertiesFile option to the status command so that it does not fail when the option is provided to collect-support-data. Issue:DS-8390

• Update the Synchronization Server to reduce the overhead of retrieving schema information from LDAP servers. Previously, the process of establishing a connection to an LDAP server would always cause the server schema to be retrieved over that connection. Now, schema information is shared across connections in a connection pool so that subsequent connections established within that pool no longer need to retrieve the server schema. This can reduce both the time and bandwidth required to establish connections to an LDAP server. Issue:DS-7597

• Update setup to add a masters/peers trust-all argument so that the deployer must explicitly indicate that they trust the master/peer as well as any other masters/peers that are accessed during setup. In addition, if this argument is not specified a prompting trust store manager will be used instead of the previous behavior of using a trust-all manager all the time. If setup is in non-interactive mode and neither the trust-all argument nor the JKS trust store has been specified, and setup is accessing the master/peer over SSL or StartTLS setup will fail. Issue:DS-8381

• Updated the UnsupportedOperationExceptions thrown by the Server SDK to include details about the exceptions that were thrown including the method name and the name and/or Java class of the extension that failed to implement the method. Issue:DS-8421 SF#:1898

These features were added for version 4.0.0.0 of the Data Sync Server:

• Update the names of the UnboundID-branded products, which are now:

  • Identity Data Store (formerly PingDirectory Server)
  • Identity Proxy (formerly Proxy Server)
  • Identity Data Sync (formerly Synchronization Server)
  • Metrics Engine (name unchanged)

• Add support for RPM based installation.

These were known issues at the time of the release of version 4.0.0.0 of the Data Sync Server:

• UnboundID RPMs do not support the "--relocate" option. However, the RPMs are relocatable using the "--prefix" option at install time. Issue:DS-7890

These issues were resolved with version 4.0.0.0 of the Data Sync Server:

• To comply with the SCIM 1.1 specification, the default scim-resources.xml configuration file now maps SCIM IDs to LDAP entryUUIDs rather than DNs by default. Issue:DS-7478

• Change the stop-* tools to behave like other task based tools. These tools require the use of the --task argument to ensure that user knows they are using a server task. These tools also will not use properties files unless you provide the --usePropertiesFile or --propertiesFilePath arguments.

• Fix a bug that prevented viewing hidden and complex configuration properties using dsconfig in a non-interactive mode. Issue:DS-7245

• Improve the performance of ldifmodify when executed with a large source LDIF file. Issue:DS-7656

• Fix a bug in the manage-extension tool that would cause an error when attempting to install an extension if that extension's getExtensionDescription method returned null (which is allowed as per the documentation). Issue:DS-7367

• Improve the overall sync throughput out of the box by allowing the Change Detection Thread to pull in a larger set of changes between each change polling interval, which helps keep the sync worker threads from becoming starved. Issue:DS-7330

• Fix a bug in the LDAPConnectionHandler where it did not close the NIO Selector on shutdown. On some platforms this caused the underlying socket channel to remain bound, which prevented the server from being able to restart. Issue:DS-7373

These features were added for version 3.6.0.0 of the Data Sync Server:

• IPv6 is now a supported deployment option.

• 64-bit JDK 7 is now a supported deployment option, but 32-bit JDKs are no longer supported.

These issues were resolved with version 3.6.0.0 of the Data Sync Server:

• Update the JMX Connection Handler configuration to issue a warning if it is enabled. On some JVMs, enabling this aspect of JMX can lead to long garbage collection pauses. Issue:DS-6832

• Fix a problem where the Synchronization Server could log an error message saying "Could not register a JMX bean", which was due to a race condition in the JMX processing code. Issue:DS-6722

• Add help text to web console deployment descriptor with JBoss compatibility tips. Issue:DS-6976 SF#:1749

• Update the logic that the server and its associated tools use to select the SSL/TLS protocol version for secure communication to provide the best combination of security and compatibility. Also, a new log message type is available that can provide information about the negotiated security protocol, including the selected SSL/TLS protocol version and cipher suite. Issues:DS-6720,DS-6903

• Prevent failures for configuration group changes where the parent configuration entry may or may not exist amongst the various servers in the configuration group. Issue:DS-6088

• Fix the manage-tasks tool so that it does not use an insecure connection when --useStartTLS is specified, and does not prompt for certificate trust when --trustAll is specified. Issue:DS-5924

• Fix an issue where, when using a JDBC Sync Source, the Synchronization Server would search for the destination entry using the "identifiableInfo" from the database change record instead of the constructed source entry DN. Issue:DS-6946

• Add the --isCompressed option to the parallel-update tool so that it can read input LDIF files that are gzip compressed. Issue:DS-7237

• Fix a problem in the manage-extension tool where it would fail on Windows because it tried to delete some temp files that were currently in use. Issue:DS-6770

• Fix a bug in dsconfig that prevented going back when adding a new configuration object inside of an existing one. Issue:DS-7263 SF#:1793

• Increase the maximum heap size to 4GB from 1GB for Synchronization Server. Issue:DS-7230

These features were added for version 3.5.1.0 of the Data Sync Server:

• The Metrics Engine is a core server product that collects and aggregates key diagnostic, capacity, and usage information from an UnboundID server topology consisting of instrumented PingDirectory Server, PingDirectoryProxy Server, and Synchronization Servers running release 3.5.0.0 and above. Metrics data can be explored and graphed using the included query-metric tool, and the Metrics Engine REST API makes this information available to custom applications and third-party systems. To learn more about the Metrics Engine, please refer to the UnboundID Metrics Engine Administration Guide.

These issues were resolved with version 3.5.1.0 of the Data Sync Server:

• Add help text to web console deployment descriptor with JBoss compatibility tips. Issue:DS-6976 SF#:1749

These features were added for version 3.5.0.0 of the Data Sync Server:

• The Synchronization Server now supports a generic Sync Source API in the Server SDK, which allows the Synchronization Server to detect changes from different types of endpoints such as the Force.com platform, Google Apps, a flat file, or virtually any other kind of data source. For more information and an example, please refer to the UnboundID Server SDK documentation.

• The Synchronization Server now supports a SCIM Sync Destination, which enables the synchronization of user and group entries to the cloud via the SCIM protocol.

• Server SDK extension bundles may now be installed and updated using the manage-extension tool. For information about using the tool and building and packaging extensions, please refer to the UnboundID Server SDK documentation.

These were known issues at the time of the release of version 3.5.0.0 of the Data Sync Server:

• When using a GSSAPI SASL Mechanism Handler the kerberos-service-principal property is only used to determine the protocol (i.e. "ldap"). The hostname will always be determined using the server-fqdn property. Issue:DS-5053

• In failover scenarios, the SCIM Sync Destination may report that HTTP status code 500 was received and was the cause of a failover, even if no response was actually received from the server at all. Currently the SCIM-SDK does not have a mechanism to report local errors such as this. Issue:SCIM-287

These issues were resolved with version 3.5.0.0 of the Data Sync Server:

• Fix an issue where multiple server configuration changes would fail if any of the servers were configured with an LDAPS (SSL) connection handler. Issue:DS-5100

• Update the Synchronization Server to not look at the deletedEntryAttrs on changelog entries when determining whether to ignore a change based on the modifiersName attribute. Issue:DS-4171

• Update the collect-support-data tool to include the equivalent of jstack output for IBM VMs on non-AIX platforms. Issue:MON-5027

• Update the Synchronization Server to utilize the destination schema when synchronizing with an LDAP directory. This ensures that the sync engine will treat DN attributes and other types correctly when performing its diff. Issue:DS-3447

• Update the Password Sync Agent for Active Directory to support 64-bit Windows operating systems. Separate binaries for x64 and x86 are now provided.

• Fix a Validator error which could occur in the SyncPipe if the modification set to apply was empty (which can only happen in notification mode) and there was a failure at the destination endpoint. Issue:DS-5320

• Update the Sync Server to use the original source DN when correlating entries at the destination for modify operations. Previously, if the entry had been renamed at the source immediately after the modify, the Sync Server would use the new entry DN to try and locate the destination entry. Issue:DS-5403 SF#:00001603

• Add the ability to specify a reason when entering and leaving lockdown mode. This is recorded in the logs and in the alerts that are generated. Issue:DS-5331

• Fix a bug that could cause the server to pass the old configuration into the isConfigurationChangeAcceptable method for a number of types of Server SDK extensions. Issue:DS-5597

• Update the Server SDK to provide extensions a way to dynamically register their own monitor providers with the server, without requiring any server-side configuration objects. Issue:DS-5271

• Change informational message when changelog indexing is not used on a sync pipe to be a debug trace instead. Issue:DS-5604

• Add workaround in SSL processing to detect potential buffer underflow or renegotiation even when processing appears to be OK. Issue:DS-5748 SF#:1636

• Fix a bug where method level debug tracing could cause extraneous logging from other methods in the same class. Issue:DS-5760 SF#:1636

• collect-support-data now excludes binary files unless --includeBinaryFiles is specified. Issue:DS-4260

• Fix an issue where DirectoryThreads did not set their context classloader to the one provided by our ClassLoaderProvider. This caused all the threads in the server to use the system classloader by default, which only has access to the classes specified on the classpath (i.e the core server libraries under the /lib directory). This becomes problematic if one of these threads calls into a library that uses Thread.getContextClassLoader() to load a class that is outside of the core server libraries (for example in an extension library). In this case it would use the system classloader and subsequently throw a NoClassDefError. Issue:DS-5876

• Fix a bug where peer installs were updating servers of the wrong type from the master server's ADS. Issue:DS-5552

• Change prepare-external-server to allow not supplying a trust store password in non-interactive mode, which will force the script to only trust the servers that are already present. Issue:DS-5872

• Add additional checking to prepare-endpoint-server to ensure that the Sync User on a DSEE server has the appropriate attributes to prevent usage limitations. Issue:DS-5367 SF#:1598

• Fix an unqualified static method invocation in the example ComplexJDBCSyncDestination Groovy script that is shipped with the Synchronization Server. Issue:DS-5299

• Add a new property override-local-password to the Pass Through Authentication Plugin so that with the default value of false, it will only attempt the bind remotely if and only if the local bind fails because there is not a local password defined. When set to true, it will attempt the bind remotely if the local bind fails for any reason.

  The new override-local-password property changes the default behaviour of the Pass Through Authentication Plugin. To restore the previous behaviour, change the value to true. Issue:DS-5766

• Fix a problem where the collect-support-data tool could timeout when connecting over SSL, or prompt the user to verify the server certificate even when the --no-prompt argument was specified. Issue:DS-4823

• Fix a bug that caused many command-line tools to output to stderr rather than stdout. Existing scripts that depend on the old behavior may need to be modified in order to continue working correctly. Issues:DS-3610,DS-4195

• Update the Synchronization Server to use location-based failover for LDAP and SCIM endpoints. It will prefer to fail over to external servers which are in the same location as itself whenever possible, followed by servers in the preferred failover locations as defined in the configuration. Issues:DS-5016,DS-5412,DS-5413

• Fix an issue where the Generic Sync Source and Destination extensions did not properly handle the ABORT_OPERATION result when it was specified in an EndpointException thrown by the extension. Issue:DS-5270

• Update the file format used by "dsconfig --batch-file" to support using '\' as a line continuation character. If the last character on a line is a '\', then it will be removed and the following line concatenated on to it. Issue:DS-635

• Remove the "Custom" type from the list when creating new objects in dsconfig. This was often confused with the "Third-Party" and "Groovy Scripted" types when users intended to create a Server SDK extension. Issue:DS-5229

• Update the create-sync-pipe-config tool so that it will fail immediately and display an error message if there is a problem creating the configuration objects in the Synchronization Server. Issues:DS-4208,DS-5019

• Update ldap-diff to use the schema of the target server when comparing entries. This enables comparing entries whose DN's include case-sensitive components. Issues:DS-2748,DS-6197

• Modify the ldap-diff tool to add LDAP connection options for SSL, StartTLS, and SASL authentication. Issue:DS-6034

• Update the status tool to fix an issue in the tool may fail to connect to the server to retrieve some status information when the --no-prompt option is specified. Issue:DS-5989

• Add changelog-reset detection to the Synchronization Server so that it will issue a purge alert if it detects that the source changelog has been deleted or reset. Issue:DS-5280

• Update the Server SDK to make it possible to create an internal connection that behaves like an external connection and is subject to its constraints. Issue:DS-5851

• Update the Synchronization Server to make sure it merges remote state from all other servers at least once before letting the sync pipes begin. This eliminates a very small window that could only happen at startup, where a Synchronization Server could take over the master status and begin syncing before it has merged any remote state from secondary Synchronization Servers.

  Update the Synchronization Server to handle a network partition between redundant instances. This guarantees that the true master will incorporate state from the other master(s) after the network is restored.

  Fix a bug in the LDAP changelog where it would not incorporate GetChangelogBatch token information about other replicas in a certain edge-case scenario. Issues:DS-4520,DS-6320,DS-6340

• Improve detection of active sync servers in a failover group to ensure that low priority master servers will stop syncing if more than one is ever made master at the same time due to network partition or other system wide failures. Issue:DS-6395 SF#:1710

• Fix an issue where out-of-the-box server required more memory than it should have, because of how the DictionaryPasswordValidator stored its word dictionary. The memory usage has been reduced by roughly 35MB. Issue:DS-6040

• Added support for two-legged OAuth 2 authentication method for SCIM external servers. Both the "Bearer" and "OAuth" token types are supported. Issue:SCIM-302

• Add the ability to constrain a Sync Log Publisher to one or more specific sync pipes. This allows you to create a sync logger that will show only the output for the sync pipe(s) you are interested in. Issue:DS-5344

• Updated the server to support hosting of standard web applications using the HTTP Connection Handler. Issue:MON-754

• Fix a bug that could result in a standby Sync Server starting a Sync Pipe when it comes online. Issue:DS-6486 SF#:1727

• Add lock down mode detection for the sync server that will force the sync engine to stop when entering lock down and start up again when leaving lock down mode. Issue:DS-6573 SF#:1734

• Provide an argument to the setup tool to configure the server to automatically include verbose garbage collection output in the server.out log file. Issue:DS-5681

• On Linux, the server and its tools now attempt to raise the limit on maximum user processes to 16,383 if the current value reported by ulimit is less than that. This is because Linux counts a thread as a user process, and some recent Linux distributions have a very low default value for max user processes. Issue:DS-6410

• Update dsconfig so that inclusion of the --advanced option will list expert-level objects. Issue:DS-6652

• Fix an issue where the resync tool did not adequately report the lack of endpoint preparation when the sync pipe was set up against the PingDirectoryProxy Server. Issue:DS-3426

• Fix an issue where a Sync Pipe would stop updating its monitor statistics for the source endpoint if it became severely backlogged and was waiting for some in-flight operations to finish before bringing in new changes. Issue:DS-6666 SF#:1740

• Improve the prompt that is displayed by command-line tools when establishing a secure connection to a server when no trust manager was specified and the server certificate should not be automatically trusted. The information is formatted more neatly, and the prompt will now include MD5 and SHA-1 versions of the certificate fingerprint and information about the issuer certificate chain if appropriate. There will also be an additional warning if the certificate is self signed. Issue:DS-5127

• Update tools that can perform LDAP SASL authentication to add support for the UNBOUNDID-TOTP SASL mechanism that can be used for multifactor authentication. Issue:DS-6676

• Fix the prepare-endpoint-server tool to handle the case where ACI values for the Sync User already exist on the synchronization base DN. Issue:DS-6552

• Fix a bug in the SCIM Sync Destination where it might try to connect to the SCIM endpoint when the sync pipe is stopped, and also might log a NullPointerException. Issue:DS-6728

• Fix an inconsistency in the Sync Pipe statistics where the 'ops-completed-failed-at-resource' stat could be incremented when it shouldn't be. Issue:DS-6464

• Update the JMX Connection Handler configuration to issue a warning if it is enabled. On some JVMs, enabling this aspect of JMX can lead to long garbage collection pauses. Issue:DS-6832

• Fix a problem where the Synchronization Server could log an error message saying "Could not register a JMX bean", which was due to a race condition in the JMX processing code. Issue:DS-6722

These features were added for version 3.2.0.0 of the Data Sync Server:

• AIX is now a supported deployment operating system.

These issues were resolved with version 3.2.0.0 of the Data Sync Server:

• Fix an issue where, for an UnboundID Sync Source, the Sync Server could lose some changes during fail over to another sync server if the persistent state had not yet been saved for the first time. This is unlikely to affect a production environment. Issue:3400

• Update command-line tools providing support for SASL authentication to add additional properties that may be used in conjunction with the GSSAPI mechanism. This includes the ability to control whether a ticket cache should be allowed and/or required, the ability to specify an alternate location for the ticket cache file, the ability to request that the Kerberos ticket-granting ticket be renewed, and the ability to supply a custom JAAS configuration file rather than using one automatically generated by the tool. Issue:3437

• Fix a bug that prevents going back from the type selection when creating a new configuration object in dsconfig. Issue:2913 SF#:1435

• Update a number of LDAP command-line tools to provide a new --help-sasl option that can be used to obtain information about the SASL mechanisms that are available for use and the supported options for those mechanisms. In addition, the command-line tool reference has been updated to provide a new page on supported SASL mechanisms and options. Issue:3452

• Fix a bug in which dsconfig and other tools may not properly evaluate path-based property values for remotely managed servers. Issue:3439 SF#:00001484

• Add a new stat to the Sync Pipe Monitor to expose the current throughput rate, in operations processed per second. This stat is available using the status command on the Synchronization Server as well. Issue:3556

• Modify the update tool to handle potential issues migrating the admin-backend.ldif backend file if the ds-create-time attribute is present in the entry cn=all-servers,cn=Server Groups,cn=admin. Issue:3584 SF#:00001501

• Update shell scripts used for the server and associated tools so that they will display a warning if it is not possible to set the desired number of file descriptors. Issue:3590

• Fixed an issue with the JDBC Sync Source where after a failure it could potentially re-fetch changes that had already been processed. Issue:3276

• Fix an issue in the Synchronization Server where alerts were not always generated when an endpoint server transitioned from unavailable to back to available (i.e. came back online). Limit changelog purge alerts from the Synchronization Server to once every ten seconds rather than once per poll of the changelog, and add the name of the Sync Pipe to the alert. Issue:3626

• Fix an issue where the Synchronization Server memory usage could spike when a PingDirectory Server goes down behind a Proxy. Issue:3628

• Improve error message when extension class can not be loaded. Issue:3245

• Fix an issue in the Synchronization Server where JDBC Sync Source and Destination extensions would not correctly initialize their ArgumentParser when running within the resync command. Issue:DS-4169

• Update dsconfig to make the list-properties subcommand more visible and more usable. This includes the following changes:

  - The list-properties output will now be written to standard output rather than standard error. This makes it easier to process the output with text tools like grep.

  - The list-properties subcommand can now be used with the "--offline" argument even if the server is running.

  - A new "--complexity" argument has been added that can be used to customize the complexity level of the objects included in the output.

  - A new "--includeDescription" argument has been added that can be used to include synopsis and description information in the output.

  - The top-level dsconfig help now includes an example demonstrating the use of the list-properties option.

  - A docs/config-properties.txt file containing this information is now provided with the server. This information was previously already available in the HTML config reference guide. Issue:DS-2985 SF#:00001413

• Make it possible to configure the server to configure the number of file descriptors that it should attempt to use on UNIX-based systems. Previously, the server was hard-coded to try to use 65535 file descriptors. It is now possible to override this default by setting the NUM_FILE_DESCRIPTORS environment variable with the desired number of descriptors to use. Alternately, you can do this by creating a config/num-file-descriptors file with a single line, like:

  NUM_FILE_DESCRIPTORS=12345

  If an error occurs while attempting to use the desired number of file descriptors, then a message will be written to the terminal, and if the error occurs while starting the server, then a message will be logged to the server's error log. Issue:DS-3590

• Add the ability to compress log files as they are written. This can significantly increase the amount of data that can be stored in a given amount of space so that log information can kept for a longer period of time. Because of the inherent problems with mixing compressed and uncompressed data, compression is something that can be enabled only at the time the logger is created, and compression cannot be turned on or off later. Further, because of problems in trying to append to an existing compressed file, if the server encounters an existing log file on startup, it will rotate that file and begin a new one rather than attempting to append to the previous file.

  Compression is performed using the standard gzip algorithm, so compressed log files can be accessed using readly-available tools. Further, the summarize-access-log tool has been updated so that it can work directly on compressed log files rather than requiring them to be uncompressed first. However, because it can be useful to have a small amount of uncompressed log data available for troubleshooting purposes, administrators using compressed logging may wish to have a second logger defined that does not use compression and has rotation and retention policies that will minimize the amount of space consumed by those logs while still making them useful for diagnostic purposes without the need to uncompress files before examining them. Issue:DS-2983 SF#:00001410

• Critical: Change the default behavior of the Synchronization Server to not lock entries across all Sync Pipes when processing changes.

  The Sync Server has a specialized mutex that ensures that changes to the same entry are processed serially. The primary reason for this mutex is to ensure that the server can safely process changes in parallel to achieve high throughput. However, we also use this mutex to ensure that two Sync Pipes don't process the same entry at the same time for deployments that synchronize changes bi-directionally. A consequence of this locking is that if one Sync Pipe is failing (because the destination is unavailable) then it retains the lock on an entry, and when other Sync Pipes try to process changes to that entry they will block that change and all changes that follow it while they wait on the lock.

  This change turns off using a shared mutex by default, but adds a new advanced configuration option on the Sync Pipe, shared-mutex-name, that specifies the name of a mutex that is shared by other Sync Pipes. This gives greater control over the locking so that two Sync Pipes that share end points can ensure that two changes to the same logical user are not processed concurrently, while not impacting other Sync Pipes.

  See the shared-mutex-name property for more information.

  This property is subject to change in a future release. Issue:DS-4202 SF#:1527

• Enhance sync setup, when adding to an existing topology with JDBC nodes configured, to check if needed library files are in place before warning that manual copying may be needed. Issue:DS-3194 SF#:1471

• Update dsconfig to remove a redundant prompt when a user chose to "Change the value" of an existing property. Issue:DS-2140

• Add a new reject-insecure-requests global configuration option that can cause the server to reject all operations except StartTLS extended requests received over insecure connections. This makes it easier to allow clients to use StartTLS without allowing other requests over an insecure connection. Issue:DS-4397

• Update server access loggers to add a number of new options:

  - An option to include request details in search result entry messages. - An option to include request details in search result reference messages. - An option to include request details in intermediate response messages. - An option to include the names of attributes included in an add request. - An option to include the names of attributes targetd in a modify request. - An option to include the names of attributes included in a search result entry. - An option to include extended search request details, including the size limit, time limit, types only, and alias dereferencing behavior. Issue:DS-4404

• Update the server to add a "--lockdownMode" argument which can be used to cause the server to be started in lockdown mode. Issue:DS-1488

• Update the server to generate an administrative alert if it detects that a configuration change was made with the server offline (whether by manually editing the configuration file or using dsconfig in offline mode). Issue:DS-4407

• Update the server to provide better reporting around the use of third-party extensions. If any such extensions are loaded in the server, then the DNs of their configuration entries will be listed in the thirdPartyExtensionDN attribute of the cn=monitor entry. Further, some extensions are loaded at startup, and a message will be written to the error log with the DNs of all of their configuration entries. Please note that not all extensions are loaded at startup, in particular Sync extensions. Issue:DS-4398

• Fix an issue in which terminal focus may be lost during command-line setup just before the Summary step is shown. Issue:DS-4551

• Fix an issue in which dsconfig cannot set an unlimited value for an object property that supports an unlimited value. Issue:DS-4173

• Update the UnboundID work queue to add a dedicated thread pool that may be used for processing certain administrative operations. This dedicated thread pool may make it possible for an administrator to diagnose and take corrective action in a server even if all "normal" worker threads are tied up processing other operations. By default, eight worker threads will be created for this purpose, but this may be altered via the num-administrative-session-worker-threads property in the work queue configuration.

  Some administrative tools like dsconfig, status, collect-support-data, enter-lockdown-mode, and leave-lockdown-mode will automatically attempt to create an administrative session in which all operations they request will be processed in this dedicated pool. Other tools like ldapsearch, ldapmodify, ldapcompare, ldapdelete, ldappasswordmodify, backup, restore, import-ldif, export-ldif, and manage-tasks have a new "--useAdministrativeSession" argument that can be used to request that they attempt to use this dedicated thread pool for operations that they process. Further, the Commercial Edition of the UnboundID LDAP SDK for Java has been updated to provide support for the new start administrative session and end administrative session extended operations that are needed to use this feature, so third-party applications can also take advantage of this capability.

  In order to request that operations be processed using the administrative session thread pool, the requester must have the use-admin-session privilege (which is included in the default set of privileges automatically granted to root users). The use of the administrative session thread pool will be recorded in the access log, and a new "using-administrative-session-worker-thread" property has been added to the simple request criteria and can be used to filter operations based on whether they are using this capability. Issue:DS-4401

• Fix an issue in which setting the changelog-max-before-after-values property to unlimited causes sync server to indicate all attributes as exceeeding the maximum value. Issue:DS-4549

• Update the Periodic Stats Logger to include additional information in the "SyncPipeInfo" category. The new output columns are "Source Is Connected", "Source Connected Server", "Destination Is Connected", "Destination Connected Server", "Source Unretrieved Changes", "Completed Aborted by Plugin", "Completed Failed in Plugin", and "Synchronized Out of Date Changes". Issue:DS-3575

• Fix an issue where the Synchronization Server would report null pointer exceptions in the error log if all external servers for a given endpoint became unavailable. Issue:DS-4415

• Update the SyncPipe with the ability to specify a "filter-changes-by-user", which allows it to perform access control filtering on the source data before synchronizing. Modified the sync engine to send alerts only if the unavailable source attributes (due to ACI filtering or exceeding the max values) were ones that the Sync Pipe was interested in. Added new stats to track how many sync operations have been filtered by ACIs and how many have exceeded the max before/after values. Updated the create-sync-pipe-config tool to support the new ACI filtering features and enable them on the Sync Pipe if desired. Updated the prepare-endpoint-server tool to support the new ACI filtering features and enable them on the directory server changelog if desired. Issue:DS-4564

• Add a new listAllEntries() method to the JDBCSyncSourceAPI which allows resync from a file to include the entryType argument. Note that this breaks compatibility with the previous listAllEntries() which did not include the entryType parameter. Existing code will continue to compile, but the old method will no longer be used by the server and you will be forced to implement the new version when resyncing a database using a source input file. Issue:DS-3542

• Update the create-sync-pipe-config tool to check that the source endpoint has the minimum required version before configuring ACI filtering for notifications. Issue:DS-4564

• Fix an issue with database synchronization where a null pointer exception could be thrown if the username or password were left blank on the JDBCExternalServer configuration.

• Fix an issue where old configuration data may get left in a topology of Sync or Proxy servers after a server is uninstalled or removed from the topology. Issue:DS-4712

• Modify the tools to recognize instances of the Sun DSEE 7 Directory Server when deployed as part of the Oracle Identity Management 11g. Issue:DS-4716

• Update the Synchronization Server, so that within a polling interval, multiple batches of changes can be retrieved from the source server. This increases throughput without having to specify a small polling interval. Issue:DS-4859

• Fix an issue with the "realtime-sync" tool where it would not print specific error data to the screen if a problem occurred while stopping a sync pipe. Issue:DS-4521

• Add a new configuration property to the SyncPipe called "include-changes-for-unchanged-attributes". This can be used to override the default behavior of only applying the minimum set of changes to bring the destination entry into sync and instead cause Sync to include modifications for attributes that were changed to the same value they already have. Issue:DS-4963

• Add support for the IBM JDK for the GSSAPI SASL bind mechanism handler and when using GSSAPI SASL binds with tools and utilities. Due to restrictions with the IBM JDK, when using tools and utilities and the option "ticketcache" is set, the bind will always fail if the credentials are not found in the specified ticket cache, even if the option "requirecache" option is false. Issue:DS-4749

• Update the Synchronization Server to take advantage of changelog indexing on the source endpoint if it is available. This can provide significant performance gains when synchronizing from UnboundID servers, especially if the data is entry-balanced. Issue:DS-4955

• Improve the dsframework tool to support multi-valued server propreties. Issue:DS-5040

• Fix an issue in the Synchronization Server where a sync pipe in notification mode would not take advantage of virtual attribute information in changelog entries. Issues:DS-5069,DS-5071

These features were added for version 3.1.0.0 of the Data Sync Server:

• Update the Synchronization Server to support arbitrary Sync Destination end points via a Server SDK extension. Sync Destination implementations can be written in Java or Groovy.

• Add a new "notification" mode to the Synchronization Server. When a Sync Pipe is configured in this mode, it skips fetching the full entry from the source and instead immediately notifies the destination with the contents of the change. The details of the change are completely derived from the source changelog entry. This enables a destination to go through the same sequence of changes as the source.

• The Synchronization Server can now be monitored over SNMP.

These issues were resolved with version 3.1.0.0 of the Data Sync Server:

• Add new global configuration attribute that allows specifying a SMTP timeout to use for all configured SMTP servers. Issue:2283

• Exposed the SyncOperation class where appropriate in the JDBC Sync Source and Destination interfaces for the Server SDK. This is consistent with the other types of sync plugins that have access to the SyncOperation. Issue:2934

• Limit collect-support-data to only run against the local server it is ran from. All supported versions of the products have collect-support-data available, and should use that version to do any needed data collection. Issue:2827

• Enhance timeout for SMTP External Servers to be used for socket I/O and connection based timeouts. Previously the timeout value applied only to socket I/O. Issue:2939

• Added support for Java-based JDBC Sync Source and Destination extensions. Previously these were only available in Groovy. Issue:3003

• The update and revert-update tools now respect that -Q/--quiet option which when specified, suppresses console output of messages that are not warnings or errors. In addition, the tools will not solicit input if the -n/--no-prompt option is specified. Issue:3056 SF#:00001432

• The dsconfig tool has been fixed to that it does not exit in an error when the root DSE entry is not available. Issue:3122

• Add a new type of access logger which can be used to obtain very detailed information about requests and responses and the contexts in which the associated operations have been processed. This is primarily intended for troubleshooting purposes rather than general use, and the content is meant to be human-readable rather than machine-parsable. Further, because the output can be quite verbose, it is recommended that it only be enabled when attempting to diagnose a problem, and that it be used in conjunction with the filtered logging framework so that only potential messages of interest will be captured. Issue:3064

• Update tools, such as searchrate, that use --ratePerSecond to not use 100% of one CPU when running at a low rate. The cutoff for this rate depends on the minimum amount of time that a process can sleep, which is operating system dependent.

• Fix an issue where status output failed to complete due to stale JDBC connection. Issue:3170

• Updated the create-sync-pipe-config tool to support setting up a Sync Pipe in notification mode. It now also supports using pre-existing Sync Sources discovered from the existing configuration. Issue:3024

• Changed the --sourceDNsFile argument for the Resync command to be --sourceInputFile. It now supports DNs for resync from LDAP as well as a user-defined format for resync from a database. Issue:2734

• Update collect-support-data to collect more system level information (especially on Linux) and validate that any value specified with the --pid option does not match the servers PID, since information about the server process is always collected. Issues:2920,2930,3152,3171,3206

• Hide the subtree-view option in the Client Connection Policy configuration in DS and Sync. There is currently no way to create manual subtree views for these products, but this option may be add back for future features as needed. Issue:3125

• Add a --missingOnly option to ldap-diff to allow the tool to only report on entries that exist on only one of the servers; entries that exist on both servers but are out-of-sync are ignored. Issue:2918

• Update tools which can be used to schedule tasks to add a new "--task" argument that makes it explicit that the tool is intended to run as a task rather than in offline mode. At present, this argument is optional, but we intend to make it required in the future, and if a tool is invoked as a task without this new "--task" argument, then a warning message will be displayed recommending that it be used in the future.

  In addition, if the "--task" argument is provided but the tool was not given an appropriate set of other arguments to allow it to connect and/or authenticate to the server, then an error message will be displayed and the tool will exit with an error. This behavior will also be exhibited for other arguments that are only applicable for tools running as tasks, including the "--start", "--dependency", "--failedDependencyAction", "--completionNotify", and "--errorNotify" arguments. Issue:3224

• Added support for SNMP monitoring to the Synchronization Server via the SNMP Subagent Plugin. A Sync Pipe MIB has been supplied. Issue:3229

• Update the manage-tasks tool so that it can detect cases in which the authenticated user doesn't have permission to access information about tasks in the server and will provide a more useful error message. It would previously always report that there were no tasks in the server, which may not be true and is not very helpful. Issue:2957

• Update tools which create scheduled tasks to display a message indicating that killing the tool will not interrupt the task. For tasks that can be interrupted, the tool will also display a manage-tasks command line that can be used to cancel that task. Issue:2954

These issues were resolved with version 3.0.3.0 of the Data Sync Server:

• Fix an issue in the Synchronization Server where "the next batch of changes" could not be retrieved. This occurred when synchronizing through a proxy server after a backend directory server had been restarted. Issue:3205 SF#:1472

• Add an option to collect-support-data for collecting data from expensive processes. These expensive operations will not be executed by default. Issue:3176

• Fix an issue where debug messages logged by a command line tool (when using --enableDebug) might not be flushed to disk before the command exited. Issue:3218

• Add a configurable sync backlog threshold to the Sync Server so that alerts will be generated when a sync pipe becomes severely backlogged with unprocessed changes. Another alert will be generated when the backlog goes back below the configured threshold. Add alerting when the Sync Server detects that changes have been missed because they were purged from the source changelog before the sync pipe had a chance to process them. Issue:3199

• Add a configuration option to allow the Synchronization Server to synchronize delete and moddn operations even if they are out-of-date with the source server. Issues:3181,3230 SF#:1460,1461

• Add a configuration option (allow-destination-renames) to the Sync Class to control whether a rename of an entry (e.g. moddn in LDAP) should be allowed at the destination in the process of synchronizing a modify operation. Issue:3225 SF#:1475

• Fix an issue where the sync server could attempt to process source changes it had already seen when an UnboundID or Sun DS sync source failed over from one source server to another. This would happen when the only changes being made on the source servers were ones that did not need to be synchronized (for example if bi-directional sync was configured and the changes in question had been made by the sync server). Issue:3100

• Fix a bug that could cause resync from an UnboundID sync source to fail with a null pointer exception. Issue:3275

• Fix an issue where in rare circumstances if all endpoint servers were unavailable, the Synchronization Server might not reconnect to them when they become available. Issue:3278

These issues were resolved with version 3.0.2.0 of the Data Sync Server:

• Modify the update tool to fix an issue where in some cases the tool would fail to migrate an older configuration, displaying errors related to duplicate LDIF change records. Issues:2942,2962,2967

• Add support for Java-based JDBC Sync Source and Destination extensions. Previously these were only available in Groovy. Issue:3003

• Fix an issue in the Synchronization Server where the attribute-synchronization-mode 'all-attributes' would handle deleted attributes incorrectly. Issue:3026

• Fix a regression with the stop-sync-server command where the port argument was ignored. Issue:2925

• The create-sync-pipe-config tool now correctly handles connecting to the SSL port of an LDAP server when specified as part of an endpoint. Issue:2952

• Fix an issue where the status command would warn that the port argument was ignored even though the argument was not provided. Issue:3052 SF#:1447

• Expose destination-create-only-attr as a Sync Class property to allow certain attributes (such as objectclass) to only be set when an entry is created and not when it is modified. Issue:2947

• The command-line tools now use the full terminal width for output on Windows platforms. Issue:1019

• Fix a regression where the Synchronization Server could not synchronize with Sun DSEE 5.2p4 instances because this version of DSEE did not allow the filter "(&)" to match any entry contrary to the LDAP specification. Issue:3069 SF#:00001449

• The setup tool has been modified to correct an issue in which the presence of the --rootUserDN option, when specified with any of the "Set Up From Peer/Master Server Options", would cause setup to exit with an error. Issue:3084

• Increase the recommended default for LDAP changelog retention to be 2 days instead of 2 hours to match real world deployment expectations. Issue:3105

• Fix an issue where the Groovy "assert" statement was not handled correctly in Server SDK extensions used to synchronize with a database. Issue:3137

• Increase the default value for duplicate error messages (allow 2000 in 5 minutes) and alerts (allow 100 in 1 hour) before they are suppressed. Avoid duplicate suppression for certain types of alerts, such as configuration changes. Ensure that the severity of a duplicate alert summary message matches the severity of the duplicate messages being suppressed.

• Address an issue where Server SDK extensions running within a command line tool could cause the process to run out of memory if they logged a high volume of error log messages. Issue:3173

These issues were resolved with version 3.0.1.0 of the Data Sync Server:

• Change collect-support-data tool to prompt for missing LDAP connection arguments if needed. Issue:2461

• Fix an issue where the Synchronization Server could occasionally report a severe initialization error on startup leaving the Sync Pipes disabled. Issue:2835

• The script file for stopping the server on non-Windows operating systems have been modified so that when it is invoked with no arguments, the server is killed using the operating system's kill command, ensuring that the server will have stopped when the script returns. Issue:2821

• The remove-defunct-server tool has been enhanced to allow the user to choose to continue processing of topology servers even if one of the servers is down. In non-interactive mode this is accomplished using the --continueOnError option. Issue:2856

• Update the server so that some of the specialized access loggers (e.g., failed operations and expensive operations) do not include messages about intermediate responses. Issue:2822

• Address an issue with collect-support-data when run on Windows where certain commands that were executed would timeout without reading the full output of the command.

• Add a new external server type for configuring SMTP servers. This can be used to provide secure connections and authentication to outgoing mail servers. Issue:1150

• The SNMP Master Agent Plugin is no longer exposed as configurable because it is not a supported component. It is only used for test purposes.

• Fix a bug in the web console that prevented the creation of configuration objects with a slash character in the name. Issue:2836

• Add the ability to log debug statements from server components that are running within the context of a command line tool. This also enables logging from third-party extensions developed with the Server SDK to be captured when run from the context of a command line tool. Issue:2834

• Expose the SyncOperation class where appropriate in the JDBC Sync Source and Destination interfaces for the Server SDK. This is consistent with the other types of sync plugins that have access to the SyncOperation. Issue:2934

• The dsframework tool has been modified so that whenever a server is registered or updated with port values whose corresponding protocol enablement properties (ldapEnabled, ldapsEnabled) are not present, the tool will automatically set the value of the enablement property to "true". Issue:783

These features were added for version 3.0.0.0 of the Data Sync Server:

• Database Synchronization - Support for high-scale, highly-available data synchronization from one endpoint consisting of one of our supported PingDirectory Servers with the other endpoint consisting of a relational database management system (RDBMS). UnboundID officially supports synchronization with Oracle Database 10g and 11g as well as Microsoft SQL Server 2005 and 2008. The architecture, however, does not make any assumptions about the type of database or schema being managed; any database with a Type 4 JDBC driver can be used.

• Server SDK - Server-side SDK for extending the functionality of the core server.

• Synchronization Through Proxy - Support for Synchronizing to or from an load-balanced or entry-balanced proxy server deployment.

• Virtualization Support - Achieved "VMware Ready Status" for all of our server products, which we now support deploying in VMware environments.

These issues were resolved with version 3.0.0.0 of the Data Sync Server:

• Update the Synchronization Server installer so that all servers within a topology will use the same IntraSync-User password. Issue:1970

• Add a background retry mechanism to the Synchronization Server so that failed operations can be optionally retried after a specified delay. Issue:2040

• Add the ability to specify separate destination correlation rules for deleted entries, so that deletes can use a more relaxed set of rules if need be. This can be useful in scenarios where applications delete and then re-add the same entries (with different attributes), for example. Issue:2072

• Expose version information for many of the libraries used by the server in both "status --fullVersion" and in the "cn=Version,cn=monitor" entry. It will always include the LDAP SDK version number, and if available may also include any or all of the Berkeley DB JE, JZlib, SNMP4J, SNMP4J Agent, and SNMP4J AgentX library versions strings.

• Add a configuration option that may be used to indicate whether the server should shut down in the event that a severe error (e.g., out of memory) is raised within the JVM that indicates it may not be able to continue running properly. Issue:2265

• The dsjavaproperties tool now supports options for generating, regenerating, and updating the config/java.properties file. Issue:2280

• Fix a potential memory leak in the Synchronization Server which could occur during Sync Source failover if there were a large number of pending changes in the queue at the time of failover. Issue:2169

• Fix a bug in the timestamp-naming mechanism used in log file rotation which could cause log files that were manually renamed to still get rotated and eventually deleted if their names were still parsable as the original file name. Issue:1285

• Added a safeguard to the LDAP Sync Source so that it will not wait forever for responses to asynchronous changelog searches (particularly with DSEE).

• Update the stop script so that the "restart" option will correctly restart the server after a successful shutdown Issue:2329 SF#:1362

• Update dsconfig to work correctly in environments with a server-group set. This issue only affected dsconfig when run in a partially interactive mode where some of the configuration arguments were provided on the command line. The user is now prompted whether the configuration change should be applied to the current server or all servers in the group. Issue:2373 SF#:1370

• Reduce the maximum timeout value from 10 minutes to 5 minutes for Synchronization Server changelog searches against DSEE, and add some extra checks for connection health. The configured response timeout on the Sync Source is still preferred, but if none is set, then this value will be used as a ceiling. Issue:2383

• Address an issue where the Unique Attribute Plugin incorrectly detected conflicts when under heavy. Issue:1873

• Web Console displays a communication error alert when editing configurations objects if the server has been disconnected. Issue:2270 SF#:1239

• Fix a bug in which the server and tool JVM configurations in java.properties would lack -Xms and/or -Xmx options if the amount of memory specified as the maximum heap size was not available when setup was run. Issue:890

• Fix a bug in which setup fails if the 'locks' directory is missing, setup erroneously indicated that the server was running.

• Fix a bug that prevented the display in dsconfig and the web console of configuration objects whose name contained a slash character. Issue:2244 SF#:1373

• Modify the update tool to disallow the update tool from being used from a package in which setup has been run. Issue:2464

• Provide a custom title renderer that escapes configuration object names in the web console. This avoids a theoretical security concern with configuration object names that contain embedded JavaScript. Issue:2454

• Fix a bug in the ldapmodify command-line tool that caused it to incorrectly treat a 'referral' result as success. Referrals are still not supported by this tool, but it will now treat them as a special kind of error and will provide a more useful message. Issue:1062

• Update the resync tool to fail immediately if no destination servers are available. Issue:1181

• Generate a warning message at startup if the server is unable to determine the IP address or hostname of the local system, or if the local system's hostname resolves to a different IP address. These conditions may indicate a problem with the system configuration that could cause certain server components to break or function abnormally. Issue:2318

• Change the way that the serverUUID value is generated so that it is based on a combination of the system's primary IP address and the canonical server root path. This can be used to help detect cases in which a new server instance is created by copying the files associated with an existing server instance, which would have previously created two instances with the same serverUUID value. In the event that the stored serverUUID does not match the generated value, a log message will be generated to warn administrators of the change, and the newly-generated UUID will continue to be used. Issue:2470

• Remove forced min utilization configuration setting for replication and LDAP change logs. These settings had led to excessive database growth in some circumstance. Issue:2294 SF#:1352

• Improve the output of the ldapsearch tool to mention that a password has expired when the bind occurs. Issue:1981 SF#:1227

• Modify the updater so that the --ignoreWarnings option can be used to continue with update when there are warnings related to version compatibility issues. This allows an update to be run in a non-interactive environment, such as a script. Issue:2495

• The admin alerts list no longer includes alert types that are clearly not applicable to the product. Issue:1738

• Update generated command line arguments (such as for dsconfig) to be quoted in a mechanism specific to the operating system where they are generated and to eliminate all escaping with \, which had caused problems when replaying certain commands. This is done with as much portability across systems as possible. Issue:2455

• Update the Synchronization Server to connect immediately to the destination server of a Sync Pipe rather than waiting for a change to come through. This enables the server to show in its status that the destination server is connected. Issues:2005,2389,2547

• Update the Synchronization Server to send an admin alert when a Sync Pipe fails to start up (because of a configuration error or scripting error) and continue starting other pipes. A new alert type has been added for this condition, called sync-pipe-initialization-error. In the case of such an error, a Sync Pipe may be restarted with the server online via the realtime-sync tool. Issue:2547

• Update the realtime-sync tool to read arguments from the config/tools.properties file if present. Issue:2513

• Improved status command output to better inform the user of how the local server status was determined, based on the arguments provided. Issue:2487

• Update cli documentation to include new commands for updating and reverting a server installation. Issue:2573 SF#:1390

• Tools using a scope argument are now correctly documented in the CLI documentation. Issue:2594

• Added a new configuration property to the Sync Class which allows you to control as part of a sync operation whether all attributes should be brought into sync on the destination or only those that were affected by the originally modified attributes at the source. The property is called 'attribute-synchronization-mode'.

• Several enhancements to the Periodic Stats Logger: all columns in the output can now be turned on/off, many more built-in metrics are available to be logged, and additional custom metrics driven off of cn=monitor entries can be added by creating Custom Logged Status objects. Issue:2039

• Add extension points for the Synchronization Server. This includes Sync Pipe Plugins, LDAP Sync Source Plugins, and LDAP Sync Destination Plugin. Issue:2410

• The server now issues an alert when it has begun the startup process. Issue:2642

• The server now issues an alert when a JVM pause (possibly due to garbage collection) has been detected. Issue:2637

• The web console now allows the specification of multiple LDAP servers to be used for authentication and discovery of topology servers. Issue:2466

• The web console now supports specification of a server from its login page. Issue:2190

• Add an option to display the status for just a specific Sync Pipe in the status command output. This makes the output a lot easier to read if there are multiple Sync Pipes configured. Issue:2606

• Update the ldappasswordmodify tool to supply the bind password as the user's current password when making a self-change. This is convenient when making a root user password change so that the current password does not have to be specified twice in the command line arguments. Issue:2525

• Provide better descriptions in the MIB for SNMP trap variable bindings. Issue:2508

• The file-based loggers now optionally support millisecond level precision. Issue:2603

• Added a "invoke-gc-day-of-week" property to the Periodic GC Plugin so that it can be configured to run only on certain days of the week. Issue:2660

• Improve output when JVM errors occur in scripts used to set up environment for command line tools. Issue:2172

• Update the default JVM arguments to improve garbage collection tuning.

• Update dsjavaproperties to validate that all java-home properties specified in config/java.properties reference valid Java installations. Issue:2719

• Fix an issue where the alerts backend could write an incomplete LDIF backing file if an error were to occur during the write. Also, if an error in the LDIF file is discovered when the server is started, the alerts backend will now read as much as it can from the file and preserve a copy of the bad file. Issue:2700

• Add support for logging intermediate response messages that are returned to the client. Intermediate response logging will be enabled by default, but may be disabled if desired. Issue:2428

• Address an issue with the web console where it would not allow read-only configuration properties to be set when an object was initially created. Issue:2730

These issues were resolved with version 2.2.0.0 of the Data Sync Server:

• Modify the command-line argument parsers to generate a warning message if an argument value is the same as the short or long form for another argument. This can help prevent users from forgetting to supply a value for an argument which requires one. Issue:944

• Update MakeLDIF to add a "<random:timestamp>" tag that can be used to include a randomly-selected date from any time within the last ten years. It is also possible to use "<random:timestamp:min:max>" to specify the desired time range, where min and max should be given in the generalized time format. Issue:1083

• Add a new configuration property for alert handlers that makes it possible to filter the types of alerts that should be processed based on the alert severity. By default, all types of alerts will be processed.

• Modify the prepare-external-server tool so that it will look for trust store and password files in the default locations when using SSL or StartTLS and the locations of those files are not explicitly provided.

• Provide a new alert handler that can be used to execute a specified command whenever an alert is generated within the server. The details of the alert notification will be provided as arguments when executing that command. The arguments will be provided in the following order: the name of the alert type, the OID for the alert type, the alert severity, the fully-qualified name of the Java class that generated the alert, the unique identifier assigned to that alert, and the text of the alert message. The alert handler will ensure that only one instance of the command may be invoked at a time to avoid problems from commands that aren't safe to run concurrently. If multiple alerts are generated concurrently, then they will be queued and the command will be executed sequentially for each of them. Issue:1146

• Update the ldapsearch and ldapmodify tools so that in the event that an error response is received from the server, the diagnostic message from that error response will be displayed to the user rather than the generic error message that had previously been used.

• Add a new error log alert handler, which makes it possible to control which types of alerts should be logged (based on either the alert severity or specific alert type). Further, the severity of the log message will reflect the severity of the alert notification.

• Update the collect-support-data tool to archive information about the upgrade history of the server installation.

• Modify the enter-lockdown-mode and leave-lockdown-mode tools to allow them to connect to any local address rather than requiring the request to be sent over the loopback address. Issue:1144

• Update the LDAP connection handler to disable TLS renegotiation by default, which can eliminate a vulnerability in which a man-in-the-middle could potentially inject arbitrary cleartext between TLS negotiation and initial data from the client.

• Avoid setting the "-XX:ParallelCMSThreads" JVM argument on systems containing a single CPU. This option has been observed to cause the JVM to fail to run properly, particularly in virtualized environments. Issue:1300

• Update the system information monitor entry to include information about the system account being used to run the server and a list of all system properties defined in the JVM.

• Add a new global configuration option which makes it possible to specify the maximum length of time that the server shutdown process may take before it attempts to interrupt threads which have not yet completed their processing. In most cases, server threads will react to a shutdown in a timely manner and no interrupt is needed.

• Fix a bug in the parallel-update tool that could cause operations to be retried even when the --neverRetry argument was provided. Also, when the tool is configured to retry operations, the reject file will now include the result code and diagnostic message received from the last failure after no more progress can be made, rather than providing a generic message.

• Fix a bug in the collect-support-data tool that could cause it to make incorrect use of a password file when capturing the output of the status command. Issue:1593

• Update the SNMP alert handler so that the traps it creates have a more sensible value for the uptime field. Previously, the uptime value was always zero, but it will now reflect the length of time that the PingDirectory Server has been online.

• Improve the process for stopping threads when the server is shutting down, and provide additional debugging information that may be useful if any threads are slow to stop running. Issue:900

• Update the ldap-diff tool to take advantage of the stream directory values extended operation when it is available. This can dramatically improve the performance of the tool when attempting to identify the set of all entries in the server. Issue:794

• Update the ldap-diff tool to provide support for reading the DNs of all the entries in one or both directories from files instead of obtaining them over LDAP. In directories which do not support the stream directory values extended operation, this may provide a significantly faster way to obtain this information if it is already available in some form.

• Fix a bug in the ldap-diff tool that could cause it to report incorrect percent complete values when comparing data sets of more than 20 million entries.

• Fix a bug in the upgrade tool that could cause the same warning message multiple times if the version obtained from the server was different from what was expected (e.g., because a server jar file had previously been replaced without using the upgrade tool). Issue:1640

• Update the parallel-update tool to add the ability to use the permissive modify request control, which may be used to request that the server ignore attempts to add attribute values which are already present or remove attribute values which are not present.

• Update the ldap-diff tool to make it more likely that its output can be replayed without any alteration. The order of operations has been updated so that all deletes are listed first, followed by all modifies, and finally all adds. In addition, all delete operations are ordered such that subordinate entries will always be removed before their ancestors.

• Update the scripts used to stop the server to prevent them from falling through to try to stop the server over LDAP if the attempt to kill the process fails or times out, since the attempt to stop the server over LDAP would fail without at least the appropriate authentication credentials, and could potentially be dangerous in some contexts.

• Update the system information monitor entry to include information about all environment variables defined in the server process. In addition, it will now attempt to determine and report the process ID of the JVM in which the server is running.

• Update the logic for sending an e-mail message from the server so that it will always attempt to determine the fully-qualified name of the system to include in the HELO/EHLO request. In the event that the fully-qualified name cannot be determined, then the IP address of the server will be used rather than using an unqualified name. Issue:1337

• Update the server to make it possible to configure the length of time that name-to-IP address mappings may be cached within the server. This may be useful in environments in which the addresses associated with a particular hostname may change frequently. Issue:941

• Update the upgrade and revert-upgrade tools to ignore directories that contain backup files. Issue:1143

• Update the PingDirectory Server to change the implementation of the show-all-attributes configuration option in the schema and root DSE backends to be more robust, particularly for clients requests explicitly requesting a specific set of attributes. Issue:1590

• Updated the logic used to identify previous log files that had been rotated so that only files with names that might have been created by the rotation process will be candidates for removal by the retention policy. Issue:1285

• Update the PingDirectory Server to add a search shutdown plugin which can be used to perform a specified internal search when the server is shutting down and have the results of that search written to a specified file. This may be useful, for example, to automatically dump the contents of the monitor backend on shutdown. Issue:1334

• Update the server so that when creating a duplicate of an existing configuration object, some key properties may be excluded from the clone so that they must be explicitly configured by the administrator rather than automatically using the same value as the object being duplicated. This can help prevent problems in which a duplicated value was inadvertently used. Issue:1675

• Update the setup process so that the server will be configured without an LDAP connection handler if the "--no-prompt" argument is provided without an "--ldapPort" argument. This option is only available for use when using the non-interactive setup mechanism. Issue:1759

• Change the behavior of the dsconfig tool when creating a new configuration object so that the user will first be prompted about whether to create a completely new configuration object or clone an existing object. This simplifies the interface and makes it less likely that an administrator will incorrectly attempt to clone an existing object rather than creating a new one. Issue:1747

• Update a number of access log retention policies to make them more robust and to fix bugs that could prevent old log files from being removed when the appropriate conditions were met. Over long periods of time, this could potentially cause available disk space to run low and necessitate the manual removal of files to avoid running out of space. Issues:1867,1867

• Modify the upgrade process so that schema definitions are always migrated before the configuration. In some rare cases, attempting to migrate the configuration before the schema could lead to failures in the upgrade process. Issue:1812

• Update the server to prevent multiple loggers from being configured with the same target log file. Issue:1676

• Significantly revise the upgrade tool in an attempt to make it more robust and minimize the amount of work required for performing an upgrade. Issues:1927,1931,2031,2037

• Add support for a new search-and-mod-rate command line tool which operates in a manner similar to the searchrate tool but that will also modify any entries returned from the search.

• Rename the upgrade tool to be "update", and rename the revert-upgrade tool to be "revert-update".

• Update the PingDirectory Server to make the lockdown-mode privilege usable by non-root users. Issue:1109

• Update the server so that it includes a patch version number in addition to the existing major, minor, and point version numbers. This can help better distinguish versions with the same major, minor, and point version numbers which differ only based on patches applied.

• Update the PingDirectory Server to abort the startup process with an error message if the admin data backend includes a malformed entry. Previously, malformed entries in the admin data backend would be silently ignored. Issue:2049

• Update the collect-support-data tool to change the way that the jstack tool is invoked to dramatically reduce the impact that it has on the running process. Issue:2038

• Update the export-ldif and verify-index tools so that they can be used against a server whose database files are contained on a read-only file system, including a ZFS snapshot. Issue:71

• Update the alert backend to be able to handle entries with unrecognized alert types. This is unlikely to occur in normal conditions, but could cause a problem in deployments in which the server was upgraded and subsequently reverted, and an alert was generated in the upgraded server that uses an alert type not defined in the older version. Issue:2126

• Change the way that the worker thread percent busy values are calculated in the work queue monitor entry to make them more accurate. Also, add new recent-average-queue-size and current-worker-thread-percent-busy monitor attributes. Issue:1982

• Modify the update process to require that the system user performing the update is the same as the system user used to run the server. This will help prevent files from being created or altered during the update process with permissions that would prevent the server from being able to access them when the server is started as the appropriate user. Issue:2158

• Modify the update tool to ensure that the documentation is updated for the new release if appropriate. Issue:2178

• Update the dsconfig tool and the Web administration console so that they inform the administrator of any administrative action (e.g., disabling and re-enabling the specified component, or restarting the server) that may be required as a result of a configuration change to be made. Issues:211,2132

• Update the subject attribute to user attribute certificate mapper to provide support for VeriSign certificates whose subject contained an emailAddress attribute with an unusual encoding. Issue:2177