A Sync Pipe defines how data is synchronized from a Sync Source to a Sync Destination.
↓Relations from This Component
↓Relations to This Component
↓Properties
↓dsconfig Usage
The following components have a direct composition relation from Sync Pipes:
The following components have a direct aggregation relation from Sync Pipes:
The following components have a direct aggregation relation to Sync Pipes:
The properties supported by this managed object are as follows:
| Description | A description for this Sync Pipe |
| Default Value | None |
| Allowed Values | A string |
| Multi-Valued | No |
| Required | No |
| Admin Action Required | None. Modification requires no further action |
| Description | Indicates whether the synchronization for the Sync Pipe is started. If a Sync Pipe is not started, then synchronization does not occur from the Sync Source to the Sync Destination. |
| Default Value | false |
| Allowed Values | true false |
| Multi-Valued | No |
| Required | Yes |
| Admin Action Required | None. Modification requires no further action |
| Description | Specifies whether this Sync Pipe will operate in standard synchronization mode or as a notification provider. A Sync Pipe can either determine the changes necessary to bring a destination entry into sync and make those changes, or it can simply notify the destination endpoint that the change has happened and provide it with the details of the change. This property determines which of these two modes the Sync Pipe will operate in. |
| Default Value | standard |
| Allowed Values | standard - In this mode, the Sync Pipe will fetch the full source and destination entries, do an attribute comparison, and make any changes to bring the destination entry into sync. notification - In this mode, the Sync Pipe will directly forward the contents of a source changelog entry on to the destination endpoint without fetching the source or destination entries or performing any diff. The destination will receive the full sequence of changes, even if they represent changes that are no longer valid at the source, for example modifying an entry that no longer exists. Filtering rules, attribute maps, and DN maps are still applied in this mode. In both standard and notification modes, the Sync Pipe allows most changes to be processed in parallel, producing a higher overall throughput. It guarantees that changes to the same entry will be processed sequentially and parent entries will be processed before child entries. If changes must be applied strictly in order, then the num-worker-threads property should be set to 1. Note that this will limit the maximum throughput of the Sync Pipe, especially with a slow or remote destination endpoint. |
| Multi-Valued | No |
| Required | No |
| Admin Action Required | None. Modification requires no further action |
| Description | Specifies the source of the synchronization changes. |
| Default Value | None |
| Allowed Values | The DN of any Sync Source. |
| Multi-Valued | No |
| Required | Yes |
| Admin Action Required | None. Modification requires no further action |
| Description | Specifies the destination of the synchronization changes. |
| Default Value | None |
| Allowed Values | The DN of any Sync Destination. |
| Multi-Valued | No |
| Required | Yes |
| Admin Action Required | None. Modification requires no further action |
change-detection-polling-interval
| Description | The amount of time to wait between polls of the Sync Source for changes. Using a larger value will increase the synchronization latency. Using a smaller value may slightly increase the load on the source server. |
| Default Value | 500 ms |
| Allowed Values | A duration. Lower limit is 0 milliseconds. |
| Multi-Valued | No |
| Required | Yes |
| Admin Action Required | None. Modification requires no further action |
| Description | Specifies the number of worker threads that should be used to process synchronization operations. This value controls how many threads concurrently synchronize changes from the Sync Source to the Sync Destination. Using a large value can increase throughput, but it also increases the resources allocated at the Sync Source and Sync Destination. For example, using 20 worker threads will open up to 20 concurrent LDAP connections to both the Sync Source Server and the Sync Destination Server. When synchronizing with a relational database, the number of JDBC connections is configurable, but care should be taken that the number of worker threads does not exceed the number of available connections; otherwise thread starvation may occur. If there is a large network delay to the Sync Source or Sync Destination, then using a larger value can improve throughput. |
| Default Value | 20 |
| Allowed Values | An integer value. Lower limit is 1. Upper limit is 1000 . |
| Multi-Valued | No |
| Required | Yes |
| Admin Action Required | None. Modification requires no further action |
include-changes-for-unchanged-attributes (Advanced Property)
| Description | The default setting for this attribute is correct for all typical use cases. It should only be changed at the suggestion of Ping Identity. This indicates whether the Data Sync Server should process modifications which change an attribute to the same value it already has. By default, these types of changes are filtered out because there is no net change. Normally, the Data Sync Server will only apply the minimum changes necessary to bring the destination endpoint into sync. This means that if an attribute was modified to have the same value that it already has, this modification would not be present in resulting changeset applied at the destination. If it is desirable to keep these "same-to-same" attribute value changes in the modification set, then this option can be set to true. In this case, the destination is guaranteed to receive changes to all destination attributes that are dependent on the source attributes that were changed. This option may be useful if the destination endpoint is interested in receiving a modification set which more closely represents the original change(s) applied at the source. Note that this setting may override the replace-all-attr-values setting on the Sync Class in some cases, because it will cause modifications with the REPLACE modification type to come through, even if the Sync Class is configured to use the ADD/DELETE modification types. |
| Default Value | false |
| Allowed Values | true false |
| Multi-Valued | No |
| Required | No |
| Admin Action Required | None. Modification requires no further action |
changes-queue-size (Advanced Property)
| Description | Specifies the size of the changes queue. This value controls the number of elements that can be contained in the changes queue, which is responsible for holding changes that are to be written to a Sync Destination. If a Sync Pipe is consistently running into out-of-memory issues, this value can be reduced to curb the amount of objects that will be kept in memory at once. This is particularly useful for Sync Pipes that are responsible for syncing many large groups. Setting this value extremely low can lessen the number of changes that can be processed in parallel, causing the average throughput of the Sync Pipe to be reduced. |
| Default Value | 5000 |
| Allowed Values | An integer value. Lower limit is 10. Upper limit is 50000 . |
| Multi-Valued | No |
| Required | No |
| Admin Action Required | The Sync Pipe must be disabled and re-enabled for changes to this setting to take effect. In order for this modification to take effect, the component must be restarted, either by disabling and re-enabling it, or by restarting the server |
max-operation-attempts (Advanced Property)
| Description | The maximum number of times a worker thread should attempt to process a synchronization operation before putting it in the background retry queue or failing the operation altogether. A sync operation that fails may or may not trigger a failover to a different source or destination server (depending on the failover settings) before retrying, but the maximum tries across all servers will not exceed this value. The exception to this rule is if the failure is due to a connection error. Connection errors trigger an immediate failover and do not count against the total number of tries for the sync operation; they get unlimited retries. For a directory server, the following LDAP result codes are considered a connection error:
For HTTP connections (such as with the SCIM Sync Destination), the following HTTP status codes are considered a connection error:
|
| Default Value | 5 |
| Allowed Values | An integer value. Lower limit is 1. |
| Multi-Valued | No |
| Required | Yes |
| Admin Action Required | None. Modification requires no further action |
retry-backoff-initial-wait (Advanced Property)
| Description | The initial amount of time to wait before retrying an operation for the first time. Whenever an operation fails, such as connecting to a server or searching for an entry, the operation is retried with a (possibly) increasing amount of backoff. This attribute defines the duration to wait before retrying the operation for the first time. This attribute works in conjunction with retry-backoff-max-wait, retry-backoff-increase-by, and retry-backoff-percentage-increase. |
| Default Value | 100 ms |
| Allowed Values | A duration. Lower limit is 0 milliseconds. |
| Multi-Valued | No |
| Required | Yes |
| Admin Action Required | None. Modification requires no further action |
retry-backoff-max-wait (Advanced Property)
| Description | The maximum amount of time to wait between operation retry attempts. Whenever an operation fails, such as connecting to a server or searching for an entry, the operation is retried with a (possibly) increasing amount of backoff. This attribute defines the maximum duration to wait between consecutive retry attempts. This attribute works in conjunction with retry-backoff-initial-wait, retry-backoff-increase-by, and retry-backoff-percentage-increase. |
| Default Value | 10 s |
| Allowed Values | A duration. Lower limit is 0 milliseconds. |
| Multi-Valued | No |
| Required | Yes |
| Admin Action Required | None. Modification requires no further action |
retry-backoff-increase-by (Advanced Property)
| Description | This specifies the amount of time to increase the backoff by between consecutive retry attempts. Whenever an operation fails, such as connecting to a server or searching for an entry, the operation is retried with a (possibly) increasing amount of backoff. This attribute defines the amount of time to increase the backoff by between retry attempts. This attribute works in conjunction with retry-backoff-initial-wait, retry-backoff-max-wait, and retry-backoff-percentage-increase. |
| Default Value | 0 s |
| Allowed Values | A duration. Lower limit is 0 milliseconds. |
| Multi-Valued | No |
| Required | Yes |
| Admin Action Required | None. Modification requires no further action |
retry-backoff-percentage-increase (Advanced Property)
| Description | Between consecutive retry attempts, the percentage that the wait duration should be increased. Whenever an operation fails, such as connecting to a server or searching for an entry, the operation is retried with a (possibly) increasing amount of backoff. This attribute defines the percentage to increase the current backoff value by between retry attempts. For example, a value of 100 will double the backoff each time. This attribute works in conjunction with retry-backoff-initial-wait, retry-backoff-max-wait, and retry-backoff-increase-by. |
| Default Value | 100 |
| Allowed Values | An integer value. Lower limit is 0. |
| Multi-Valued | No |
| Required | Yes |
| Admin Action Required | None. Modification requires no further action |
max-rate-per-second (Advanced Property)
| Description | Specifies the maximum number of synchronization operations to perform per second. This can be used to reduce the chance of overwhelming the destination server. This setting applies to both realtime synchronization and resync. |
| Default Value | Operation processing will not be throttled. |
| Allowed Values | An integer value. Lower limit is 1. |
| Multi-Valued | No |
| Required | No |
| Admin Action Required | None. Modification requires no further action |
max-failed-op-background-retries (Advanced Property)
| Description | The number of times a failed operation should be retried in the background, after a specified delay. This can be set to zero if there should be no delayed retries. Note that this applies after the max-operation-attempts have been exceeded. Whenever an operation fails, such as searching for or modifying an entry, the operation is retried according to the max-operation-attempts and retry-backoff-xxx properties. If the maximum attempts are reached and the operation has still not succeeded, it can be added to a delayed-retry queue which will retry the operation in the background after a specified delay. This attribute works in conjunction with the failed-op-background-retry-delay attribute. |
| Default Value | 0 |
| Allowed Values | An integer value. Lower limit is 0. Upper limit is 1000 . |
| Multi-Valued | No |
| Required | Yes |
| Admin Action Required | None. Modification requires no further action |
failed-op-background-retry-delay (Advanced Property)
| Description | Specifies how long to wait between consecutive retries of an operation that is in the delayed-retry queue. Note that this applies after the max-operation-attempts have been exceeded. Whenever an operation fails, such as searching for or modifying an entry, the operation is retried according to the max-operation-attempts and retry-backoff-xxx properties. If the maximum attempts are reached and the operation has still not succeeded, it can be added to a delayed-retry queue which will retry the operation in the background after a delay (specified by this property). This attribute works in conjunction with the max-failed-op-background-retries attribute. |
| Default Value | 1 m |
| Allowed Values | A duration. Lower limit is 0 milliseconds. |
| Multi-Valued | No |
| Required | Yes |
| Admin Action Required | None. Modification requires no further action |
| Description | Specifies sync pipe plugins that should be applied to all operations that are synchronized by this Sync Pipe. If multiple plugins are provided, then they will be invoked in the order they are specified. Sync Classes defined on this Sync Pipe can also define a list of sync pipe plugins. If defined, that list of plugins will be invoked in the order they are specified after any plugins defined in this list. |
| Default Value | None |
| Allowed Values | The DN of any Sync Pipe Plugin. |
| Multi-Valued | Yes |
| Required | No |
| Admin Action Required | None. Modification requires no further action |
shared-mutex-name (Advanced Property)
| Description | Specifies the name of a mutex that is shared by other Sync Pipes to ensure that two changes to the same logical user are not processed concurrently. When the num-worker-threads has a value larger than one (such as the default of 20), the Data Sync Server allows most changes to be processed in parallel, producing a higher overall throughput. It uses a special mutex to guarantee that changes to the same entry will be processed sequentially and parent entries will be processed before child entries. In some environments, such as when synchronizing changes bi-directionally between the same two end points, it's desirable to ensure that changes to the same entry are not processed by two Sync Pipes concurrently. In this case, any Sync Pipes that include the same mutex name in this property will share the same mutex, and changes to the same entry will only be processed by one Sync Pipe at a time. Care should be taken when setting this property since errors on one Sync Pipe can have an impact on Sync Pipes that share the same mutex. For example, if this Sync Pipe is processing a change to user A, and it is not able to apply the change because the destination is down, then it will continue to retry this change until the destination is available. During this time, it holds a lock on user A. If another Sync Pipe shares a mutex with this Sync Pipe, then when it encounters a change to user A, it will block waiting for the lock on user A to be released, and all subsequent changes on the Sync Pipe will stall. This is typically not a problem when two Sync Pipes share the same two end points--that is, one end point is the source for one Sync Pipe and the destination for the other one (and vice versa). Sharing a mutex across Sync Pipes that do not share end points should be avoided. Leaving this property unset will ensure that worker threads within this Sync Pipe will not coordinate with worker threads in any other Sync Pipe. The create-sync-pipe-config tool will automatically set this property when configuring bi-directional sync, and the mutex will be named "Mutex for {source-to-dest-pipe-name} and {dest-to-source-pipe-name}". |
| Default Value | Changes to the same entry are serialized within this Sync Pipe but not with others. |
| Allowed Values | A string |
| Multi-Valued | No |
| Required | No |
| Admin Action Required | None. Modification requires no further action |
filter-changes-by-user (Advanced Property)
| Description | Specifies the DN of a user for which the source directory server will apply access control filtering to the synchronized data before it is processed (only applicable for notification mode). A Sync Pipe in notification mode can optionally perform access control filtering on the changelog data as it comes back from the source directory server. If specified, this property controls the bind DN for which ACIs should be applied. By default, no ACI processing is performed. The details of the access control filtering are as follows. For each changelog entry that would be returned by a poll of the changelog:
|
| Default Value | Changelog entries are not subject to access control filtering. |
| Allowed Values | A valid DN. |
| Multi-Valued | No |
| Required | No |
| Admin Action Required | None. Modification requires no further action |
To list the configured Sync Pipes:
dsconfig list-sync-pipes
[--property {propertyName}] ...
To view the configuration for an existing Sync Pipe:
dsconfig get-sync-pipe-prop
--pipe-name {name}
[--tab-delimited]
[--script-friendly]
[--property {propertyName}] ...
To update the configuration for an existing Sync Pipe:
dsconfig set-sync-pipe-prop
--pipe-name {name}
(--set|--add|--remove) {propertyName}:{propertyValue}
[(--set|--add|--remove) {propertyName}:{propertyValue}] ...
To create a new Sync Pipe:
dsconfig create-sync-pipe
--pipe-name {name}
--set sync-source:{propertyValue}
--set sync-destination:{propertyValue}
[--set {propertyName}:{propertyValue}] ...
To delete an existing Sync Pipe:
dsconfig delete-sync-pipe
--pipe-name {name}