resync

Description

A resync operation can be used to resynchronize a Sync Destination with the contents of the Sync Pipe's corresponding Sync Source.

The resync command operates on a single Sync Pipe at a time by retrieving entries from the Sync Source in bulk, comparing them with the corresponding entries at the Sync Destination, and if the entries are out-of-sync, acting according to the Synchronization Server configuration and command-line options. If the --dry-run option is specified, then resync only logs what is out-of-sync. Otherwise, if the entry is missing, then it is created, and if attributes are out-of-sync, then they are updated. Details of the resync operation are written to a log with a configurable level of verbosity.

The resync command can optionally take in a file which contains a specific list of source DNs to synchronize. With this option, base-level searches will be issued for each DN in the file, rather than a bulk subtree search for a batch of entries. This functionality allows for very fine-grained control of which entries get synchronized, and also gives the ability to retry resync for entries that failed on a previous attempt. On very large data sets, this can be a practical way to avoid several long-running resync jobs. Any entries that fail to synchronize are written to the file specified in the --failedDNsFile argument, which can then be passed in as the --sourceInputFile argument on a subsequent run. The 'sync-failed-dns' file in the logs directory, which contains the DNs of all entries that have failed to synchronize (even after retries), can also be used as an input file for resync.

Resync can also run against a RDBMS environment (i.e. if the source of the specified Sync Pipe is a JDBC Sync Source). In this case you must either specify the --entryType argument to identify which database entry type to resync, or the --sourceInputFile argument to identify a file containing data to pass to the extension. The Server SDK extension is responsible for listing out all the database entries of that type and fetching them into LDAP entries, which are then processed based on the rules of the matching Sync Class and the command-line options. Certain options such as --baseDN and --sourceSearchFilter, are not applicable for this type of resync and will have no effect. If a database is the destination of the Sync Pipe, then resync will behave just like it does with a normal directory-to-directory Sync Pipe, and all options may be used.

Entries synchronized from the Sync Source undergo the same filtering and mapping as during real-time synchronization. Entries are synchronized according to the settings in their matching Sync Class. The Sync Class controls what attribute and DN maps are used and whether entries at the destination are created or modified (this can be overridden with the --synchronizeCreates and --synchronizeModifies command-line options). If an entry does not match any Sync Class, then it will be dropped.

The resync command-line options give fine-grained control over what is synchronized. Specific Sync Classes can be included or excluded from synchronization. The set of source entries to be compared can be restricted using the --sourceSearchFilter option. Also, the set of attributes that are synchronized can be controlled by including or excluding both source and destination attributes from synchronization. Note: A destination attribute might depend on multiple source attributes (e.g., the 'initials' destination attribute might depend on both 'givenname' and 'sn' from the source), and a single source attribute might be used to construct the value of multiple destination attributes (e.g. 'givenname' at the source might be used to construct a value for both 'cn' and 'initials' at the destination). Including and excluding source attributes from resync can affect both of these scenarios. If a destination attribute depends on multiple source attributes, then all of those attributes must be included for synchronization (implicitly or explicitly) for that attribute to be updated.

The resync command is designed to function well in an environment where both the Sync Source and Sync Destination are actively being updated by other clients including the Synchronization Server synchronizing changes in real-time. There are a few options that are especially useful in these scenarios. When a value greater than one is provided with the --numPasses option, then resync rechecks out-of-sync entries multiple times to account for entries that are temporarily out-of-sync due to synchronization delays. The --dry-run option allows the administrator to preview any changes that would be applied by resync without updating the Sync Destination. Finally, the --ratePerSecond option can be used to limit the impact of the resync command on the end servers. By default, resync attempts to resynchronize the Sync Source and Sync Destination as quickly as possible, but this can have a performance impact on the servers. Explicitly setting the maximum resync rate of entries compared per second can limit this impact.

The resync command serves many purposes in a deployment.

When initially configuring a Sync Pipe with an empty Sync Destination, resync can be used to populate the Sync Destination with entries equivalent to the entries in the Sync Source before starting real-time synchronization. For very large deployments, it can be faster to export entries from the source into an LDIF file, run the translate-ldif tool to translate and filter the entries into the destination format, and then import the resulting LDIF file into the destination server.

Likewise, when initially configuring a Sync Pipe with a Sync Destination that already contains data, resync can be used to bring the Sync Destination in-sync with the Sync Source before starting real-time synchronization.

Running resync on a scheduled basis, for example nightly, can take the place of real-time synchronization in deployments where a synchronization delay is tolerable.

Running resync with the --dry-run option can be used to validate that real-time synchronization is behaving as expected. When coupled with the "--numPasses 3" option, for example, resync even avoids reporting entries that are only temporarily out-of-sync due to real-time synchronization delays.

If in an established deployment, the Data Sync Server is configured to only synchronize a subset of attributes from the Sync Source, then if that list of attributes is ever expanded, resync can be used to synchronize existing values of the newly added attributes in bulk.

Examples

resync --pipe-name "Some Sync Pipe"

resync --pipe-name "Some Sync Pipe" --includeSyncClass Users \
     --sourceSearchFilter "(createTimeStamp>=200503311200-0600)" \
     --synchronizeCreates true --synchronizeModifies false

resync --pipe-name "Some Sync Pipe" --sourceSearchFilter "(uid=some-test-user)" \
     --dry-run --logFilePath resync-debug.log --logLevel debug

resync --pipe-name "Some Sync Pipe" --includeSourceAttr givenname \
     --includeSourceAttr sn --excludeDestinationAttr mail

resync --pipe-name "Some Sync Pipe" --sourceInputFile sourceDNs.txt

resync --pipe-name "Some Sync Pipe" --numPasses 3

resync --pipe-name "Some Sync Pipe" --sourceInputFile sourceSQL.txt \
     --entryType ACCOUNTS

Arguments

-V
--version

-H
--help

--help-debug

-p {sync-pipe-name}
--pipe-name {sync-pipe-name}

--includeSyncClass {sync-class-name}

--excludeSyncClass {sync-class-name}

--baseDN {base-dn}

--entryType {database-entry-type}

--sourceSearchFilter {source-ldap-filter}

--sourceSearchSCIMFilter {source-scim-filter}

--synchronizeModifies {true|false}

--synchronizeCreates {true|false}

--includeSourceAttr {attribute}

--excludeSourceAttr {attribute}

--includeDestinationAttr {attribute}

--excludeDestinationAttr {attribute}

--dry-run

--numPasses {num-passes}

--secondsBetweenPass {seconds}

-Q
--quiet

-r {comparisons-per-second}
--ratePerSecond {comparisons-per-second}

--ratePerSecondFile {target-rate-file}

-t {num-threads}
--numThreads {num-threads}

-o {resync-log-file}
--logFilePath {resync-log-file}

-L {log-level}
--logLevel {log-level}

--sourceInputFile {source-input-file}

-F {failed-dn-file}
--failedDNsFile {failed-dn-file}

--consoleLogFilePath {consoleLogFilePath}