Data Governance Server Documentation Index
Configuration Reference Home

File Retention Recurring Task

Note: this component is designated "advanced", which means that objects of this type are not expected to be created or altered in most environments. If you believe that such a change is necessary, you may want to contact support in order to understand the potential impact of that change.

File Retention Recurring Tasks can be used to remove files in a specified directory that match a given pattern, excluding files that match count-based, time-based, or space-based retention criteria.

If any files are to be removed, then the oldest files will be removed before the youngest file. If the filename pattern includes a "${timestamp}" element, then that timestamp will be used to identify the file's age. If the filename pattern does not include a timestamp, then the file's age will be determined using the file's creation time if that is available, or the last modified time if the creation time is not available. If it is not possible to determine a file's age, then that file will not be removed.
If multiple files have the same age, then lexicographic ordering will be used to differentiate between them. However, this lexicographic ordering is only applicable if no retain-file-age is configured, or for files that are older than that age. If there are multiple files with the same age, but that age is younger than the retain-file-age value, then those files will be retained.

Parent Component
Properties
dsconfig Usage

Parent Component

The File Retention Recurring Task component inherits from the Recurring Task

Properties

The properties supported by this managed object are as follows:


Basic Properties: Advanced Properties:
↓ description  None
↓ cancel-on-task-dependency-failure
↓ email-on-start
↓ email-on-success
↓ email-on-failure
↓ alert-on-start
↓ alert-on-success
↓ alert-on-failure
↓ target-directory
↓ filename-pattern
↓ timestamp-format
↓ retain-file-count
↓ retain-file-age
↓ retain-aggregate-file-size

Basic Properties

description

Description
A description for this Recurring Task
Default Value
None
Allowed Values
A string
Multi-Valued
No
Required
No
Admin Action Required
None. Modification requires no further action

cancel-on-task-dependency-failure

Description
Indicates whether an instance of this Recurring Task should be canceled if the task immediately before it in the recurring task chain fails to complete successfully (including if it is canceled by an administrator before it starts or while it is running). If this is true, then the instance of this Recurring Task will also be considered a failure (with a state of "canceled before starting"), and if that instance is itself a dependency for any other Recurring Task instance, then that dependent task's cancel-on-task-dependency-failure property will be used to determine whether it should be canceled or preserved.
Default Value
false
Allowed Values
true
false
Multi-Valued
No
Required
No
Admin Action Required
None. Modification requires no further action

email-on-start

Description
The email addresses to which a message should be sent whenever an instance of this Recurring Task starts running. If this option is used, then at least one smtp-server must be configured in the global configuration.
Default Value
No email message will be sent when instances of this Recurring Task start running.
Allowed Values
A string
Multi-Valued
Yes
Required
No
Admin Action Required
None. Modification requires no further action

email-on-success

Description
The email addresses to which a message should be sent whenever an instance of this Recurring Task completes successfully. If this option is used, then at least one smtp-server must be configured in the global configuration.
Default Value
No email message will be sent when instances of this Recurring Task finish running.
Allowed Values
A string
Multi-Valued
Yes
Required
No
Admin Action Required
None. Modification requires no further action

email-on-failure

Description
The email addresses to which a message should be sent if an instance of this Recurring Task fails to complete successfully. If this option is used, then at least one smtp-server must be configured in the global configuration.
Default Value
No email message will be sent when instances of this Recurring Task fail to complete successfully.
Allowed Values
A string
Multi-Valued
Yes
Required
No
Admin Action Required
None. Modification requires no further action

alert-on-start

Description
Indicates whether the server should generate an administrative alert whenever an instance of this Recurring Task starts running.
Default Value
false
Allowed Values
true
false
Multi-Valued
No
Required
No
Admin Action Required
None. Modification requires no further action

alert-on-success

Description
Indicates whether the server should generate an administrative alert whenever an instance of this Recurring Task completes successfully.
Default Value
false
Allowed Values
true
false
Multi-Valued
No
Required
No
Admin Action Required
None. Modification requires no further action

alert-on-failure

Description
Indicates whether the server should generate an administrative alert whenever an instance of this Recurring Task fails to complete successfully.
Default Value
true
Allowed Values
true
false
Multi-Valued
No
Required
No
Admin Action Required
None. Modification requires no further action

target-directory

Description
The path to the directory containing the files to examine. The directory must exist. The provided path may be relative or absolute. Relative paths will bew relative to the server root.
Default Value
None
Allowed Values
A filesystem path
Multi-Valued
No
Required
Yes
Admin Action Required
None. Modification requires no further action

filename-pattern

Description
A pattern that specifies the names of the files to examine. The pattern may contain zero or more asterisks as wildcards, where each wildcard matches zero or more characters. It may also contain at most one occurrence of the special string "${timestamp}", which will match a timestamp with the format specified using the timestamp-format property. All other characters in the pattern will be treated literally. For example, the pattern "config-${timestamp}*.gz" could be used to match archived configuration files. The pattern "expensive-operation-dump-${timestamp}.log" could be used to match expensive operation dumps. The pattern "work-queue-backlog-thread-dump-${timestamp}.log" could be used to match work queue backlog thread dumps.
Default Value
None
Allowed Values
A string
Multi-Valued
No
Required
Yes
Admin Action Required
None. Modification requires no further action

timestamp-format

Description
The format to use for the timestamp represented by the "${timestamp}" token in the filename pattern.
Default Value
None
Allowed Values
generalized-time-utc-with-milliseconds - Indicates that the timestamp is in the generalized time format with millisecond-level precision, in the UTC time zone indicated with a trailing 'Z' (for example, '20180102123456.789Z' represents January 2, 2018 at 12:34:56.789 p.m. UTC).

generalized-time-utc-with-seconds - Indicates that the timestamp is in the generalized time format with second-level precision, in the UTC time zone indicated with a trailing 'Z' (for example, '20180102123456Z' represents January 2, 2018 at 12:34:56 p.m. UTC).

generalized-time-utc-with-minutes - Indicates that the timestamp is in the generalized time format with minute-level precision, in the UTC time zone indicated with a trailing 'Z' (for example, '201801021234Z' represents January 2, 2018 at 12:34 p.m. UTC).

local-time-with-milliseconds - Indicates that the timestamp is numeric with millisecond-level precision and no time zone indicator (for example, '20180102123456.789' represents January 2, 2018 at 12:34:56.789 p.m. in the local time zone).

local-time-with-seconds - Indicates that the timestamp is numeric with second-level precision and no time zone indicator (for example, '20180102123456' represents January 2, 2018 at 12:34:56 p.m. in the local time zone).

local-time-with-minutes - Indicates that the timestamp is numeric with minute-level precision and no time zone indicator (for example, '201801021234' represents January 2, 2018 at 12:34 p.m. in the local time zone).

local-date - Indicates that the timestamp is numeric with day-level precision (for example, '20180102' represents January 2, 2018).
Multi-Valued
No
Required
Yes
Admin Action Required
None. Modification requires no further action

retain-file-count

Description
The minimum number of files matching the pattern that will be retained.
If a value is specified for this property and the target directory has more than that number of files that match the pattern, then at least the specified number of the most recent files will be retained, with older files being candidates for removal. A value of zero indicates that all files matching the pattern are candidates for removal.
If a value is specified for this property, and either or both of the retain-file-age and retain-aggregate-file-size properties are also specified, then a file matching the filename pattern will only be a candidate for removal if it does not match any of the retention criteria. That is, the task may retain more than the specified number of files if they are newer than the retain-file-age value or if the aggregate size is less than the retain-aggregate-file-size value.
Default Value
None
Allowed Values
An integer value. Lower limit is 0.
Multi-Valued
No
Required
No
Admin Action Required
None. Modification requires no further action

retain-file-age

Description
The minimum age of files matching the pattern that will be retained. If a value is specified for this property, then any file whose age (as determined by the timestamp contained in the filename, the file creation time, or the last modified time, in that order) is less than or equal to this value will be retained, with files older than that age being candidates for removal. If for any reason it is not possible to determine the age of a file, it will not be a candidate for removal.
If a value is specified for this property, and either or both of the retain-file-count and retain-aggregate-file-size properties are also specified, then a file matching the filename pattern will only be a candidate for removal if it does not match any of the retention criteria. That is, the task may retain files older than the specified age if the number of files within the retain age is less than the retain-file-count value, or if their aggregate size is less than the retain-aggregate-file-size value.
Default Value
None
Allowed Values
A duration. Lower limit is 1 milliseconds.
Multi-Valued
No
Required
No
Admin Action Required
None. Modification requires no further action

retain-aggregate-file-size

Description
The minimum aggregate size of files that will be retained. The size should be specified as an integer followed by a unit that is one of "b" or "bytes", "kb" or "kilobytes", "mb" or "megabytes", "gb" or "gigabytes", or "tb" or "terabytes". For example, a value of "1 gb" indicates that at least one gigabyte of files should be retained. If a value is specified for this property, then the sizes of the files, from youngest to oldest (with age determined by the timestamp contained in the filename, the file creation time, or the last modified time, in that order), will be added together until their total size meets or exceeds the retain file size. Any additional older files that remain will be candidates for removal.
If a value is specified for this property, and either or both of the retain-file-count and retain-file-age properties are also specified, then a file matching the filename pattern will only be a candidate for removal if it does not match any of the retention criteria. That is, the aggregate size of the files that are retained may be substantially greater than the specified size if the number of files within the retention size is less than the retain-file-count value, or if their age is less than the retain-file-age value.
Default Value
None
Allowed Values
A positive integer representing a size. Lower limit is 1048576.
Multi-Valued
No
Required
No
Admin Action Required
None. Modification requires no further action


dsconfig Usage

To list the configured Recurring Tasks:

dsconfig list-recurring-tasks
     [--property {propertyName}] ...

To view the configuration for an existing Recurring Task:

dsconfig get-recurring-task-prop
     --task-name {name}
     [--tab-delimited]
     [--script-friendly]
     [--property {propertyName}] ...

To update the configuration for an existing Recurring Task:

dsconfig set-recurring-task-prop
     --task-name {name}
     (--set|--add|--remove) {propertyName}:{propertyValue}
     [(--set|--add|--remove) {propertyName}:{propertyValue}] ...

To create a new File Retention Recurring Task:

dsconfig create-recurring-task
     --task-name {name}
     --type file-retention
     --set target-directory:{propertyValue}
     --set filename-pattern:{propertyValue}
     --set timestamp-format:{propertyValue}
     [--set {propertyName}:{propertyValue}] ...

To delete an existing Recurring Task:

dsconfig delete-recurring-task
     --task-name {name}