CedarBackup3.actions package¶

Cedar Backup actions.

This package code related to the offical Cedar Backup actions (collect, stage, store, purge, rebuild, and validate).

The action modules consist of mostly “glue” code that uses other lower-level functionality to actually implement a backup. There is one module for each high-level backup action, plus a module that provides shared constants.

All of the public action function implement the Cedar Backup Extension Architecture Interface, i.e. the same interface that extensions implement.

author:	Kenneth J. Pronovici <pronovic@ieee.org>

Submodules¶

CedarBackup3.actions.collect module¶

Implements the standard ‘collect’ action. :author: Kenneth J. Pronovici <pronovic@ieee.org>

CedarBackup3.actions.collect.executeCollect(configPath, options, config)[source]¶

Executes the collect backup action.

Note: When the collect action is complete, we will write a collect indicator to the collect directory, so it’s obvious that the collect action has completed. The stage process uses this indicator to decide whether a peer is ready to be staged.

Parameters:	configPath (String representing a path on disk) – Path to configuration file on disk options (Options object) – Program command-line options config (Config object) – Program configuration
Raises:	`ValueError` – Under many generic error conditions `TarError` – If there is a problem creating a tar file

CedarBackup3.actions.constants module¶

Provides common constants used by standard actions. :author: Kenneth J. Pronovici <pronovic@ieee.org>

CedarBackup3.actions.initialize module¶

Implements the standard ‘initialize’ action. :author: Kenneth J. Pronovici <pronovic@ieee.org>

CedarBackup3.actions.initialize.executeInitialize(configPath, options, config)[source]¶

Executes the initialize action.

The initialize action initializes the media currently in the writer device so that Cedar Backup can recognize it later. This is an optional step; it’s only required if checkMedia is set on the store configuration.

Parameters:	configPath (String representing a path on disk) – Path to configuration file on disk options (Options object) – Program command-line options config (Config object) – Program configuration

CedarBackup3.actions.purge module¶

Implements the standard ‘purge’ action. :author: Kenneth J. Pronovici <pronovic@ieee.org>

CedarBackup3.actions.purge.executePurge(configPath, options, config)[source]¶

Executes the purge backup action.

For each configured directory, we create a purge item list, remove from the list anything that’s younger than the configured retain days value, and then purge from the filesystem what’s left.

Parameters:	configPath (String representing a path on disk) – Path to configuration file on disk options (Options object) – Program command-line options config (Config object) – Program configuration
Raises:	`ValueError` – Under many generic error conditions

CedarBackup3.actions.rebuild module¶

Implements the standard ‘rebuild’ action. :author: Kenneth J. Pronovici <pronovic@ieee.org>

CedarBackup3.actions.rebuild.executeRebuild(configPath, options, config)[source]¶

Executes the rebuild backup action.

This function exists mainly to recreate a disc that has been “trashed” due to media or hardware problems. Note that the “stage complete” indicator isn’t checked for this action.

Note that the rebuild action and the store action are very similar. The main difference is that while store only stores a single day’s staging directory, the rebuild action operates on multiple staging directories.

Parameters:	configPath (String representing a path on disk) – Path to configuration file on disk options (Options object) – Program command-line options config (Config object) – Program configuration
Raises:	`ValueError` – Under many generic error conditions `IOError` – If there are problems reading or writing files

CedarBackup3.actions.stage module¶

Implements the standard ‘stage’ action. :author: Kenneth J. Pronovici <pronovic@ieee.org>

CedarBackup3.actions.stage.executeStage(configPath, options, config)[source]¶

Executes the stage backup action.

Note: The daily directory is derived once and then we stick with it, just in case a backup happens to span midnite.

Note: As portions of the stage action is complete, we will write various indicator files so that it’s obvious what actions have been completed. Each peer gets a stage indicator in its collect directory, and then the master gets a stage indicator in its daily staging directory. The store process uses the master’s stage indicator to decide whether a directory is ready to be stored. Currently, nothing uses the indicator at each peer, and it exists for reference only.

Parameters:	configPath (String representing a path on disk) – Path to configuration file on disk options (Options object) – Program command-line options config (Config object) – Program configuration
Raises:	`ValueError` – Under many generic error conditions `IOError` – If there are problems reading or writing files

CedarBackup3.actions.store module¶

Implements the standard ‘store’ action. :author: Kenneth J. Pronovici <pronovic@ieee.org> :author: Dmitry Rutsky <rutsky@inbox.ru>

CedarBackup3.actions.store.consistencyCheck(config, stagingDirs)[source]¶

Runs a consistency check against media in the backup device.

It seems that sometimes, it’s possible to create a corrupted multisession disc (i.e. one that cannot be read) although no errors were encountered while writing the disc. This consistency check makes sure that the data read from disc matches the data that was used to create the disc.

The function mounts the device at a temporary mount point in the working directory, and then compares the indicated staging directories in the staging directory and on the media. The comparison is done via functionality in filesystem.py.

If no exceptions are thrown, there were no problems with the consistency check. A positive confirmation of “no problems” is also written to the log with info priority.

@warning: The implementation of this function is very UNIX-specific.

Parameters:	config – Config object stagingDirs – Dictionary mapping directory path to date suffix
Raises:	`ValueError` – If the two directories are not equivalent `IOError` – If there is a problem working with the media

CedarBackup3.actions.store.executeStore(configPath, options, config)[source]¶

Executes the store backup action.

Note: The rebuild action and the store action are very similar. The main difference is that while store only stores a single day’s staging directory, the rebuild action operates on multiple staging directories.

Note: When the store action is complete, we will write a store indicator to the daily staging directory we used, so it’s obvious that the store action has completed.

Parameters:	configPath (String representing a path on disk) – Path to configuration file on disk options (Options object) – Program command-line options config (Config object) – Program configuration
Raises:	`ValueError` – Under many generic error conditions `IOError` – If there are problems reading or writing files

CedarBackup3.actions.store.writeImage(config, newDisc, stagingDirs)[source]¶

Builds and writes an ISO image containing the indicated stage directories.

The generated image will contain each of the staging directories listed in stagingDirs. The directories will be placed into the image at the root by date, so staging directory /opt/stage/2005/02/10 will be placed into the disc at /2005/02/10.

Note: This function is implemented in terms of writeImageBlankSafe. The newDisc flag is passed in for both rebuildMedia and todayIsStart.

Parameters:	config – Config object newDisc – Indicates whether the disc should be re-initialized stagingDirs – Dictionary mapping directory path to date suffix
Raises:	`ValueError` – Under many generic error conditions `IOError` – If there is a problem writing the image to disc

CedarBackup3.actions.store.writeImageBlankSafe(config, rebuildMedia, todayIsStart, blankBehavior, stagingDirs)[source]¶

Builds and writes an ISO image containing the indicated stage directories.

The generated image will contain each of the staging directories listed in stagingDirs. The directories will be placed into the image at the root by date, so staging directory /opt/stage/2005/02/10 will be placed into the disc at /2005/02/10. The media will always be written with a media label specific to Cedar Backup.

This function is similar to writeImage, but tries to implement a smarter blanking strategy.

First, the media is always blanked if the rebuildMedia flag is true. Then, if rebuildMedia is false, blanking behavior and todayIsStart come into effect:

If no blanking behavior is specified, and it is the start of the week,
the disc will be blanked

If blanking behavior is specified, and either the blank mode is "daily"
or the blank mode is "weekly" and it is the start of the week, then
the disc will be blanked if it looks like the weekly backup will not
fit onto the media.

Otherwise, the disc will not be blanked

How do we decide whether the weekly backup will fit onto the media? That is what the blanking factor is used for. The following formula is used:

will backup fit? = (bytes available / (1 + bytes required) <= blankFactor

The blanking factor will vary from setup to setup, and will probably require some experimentation to get it right.

Parameters:	config – Config object rebuildMedia – Indicates whether media should be rebuilt todayIsStart – Indicates whether today is the starting day of the week blankBehavior – Blank behavior from configuration, or `None` to use default behavior stagingDirs – Dictionary mapping directory path to date suffix
Raises:	`ValueError` – Under many generic error conditions `IOError` – If there is a problem writing the image to disc

CedarBackup3.actions.store.writeStoreIndicator(config, stagingDirs)[source]¶

Writes a store indicator file into staging directories.

The store indicator is written into each of the staging directories when either a store or rebuild action has written the staging directory to disc.

Parameters:	config – Config object stagingDirs – Dictionary mapping directory path to date suffix

CedarBackup3.actions.util module¶

Implements action-related utilities :author: Kenneth J. Pronovici <pronovic@ieee.org>

CedarBackup3.actions.util.buildMediaLabel()[source]¶: Builds a media label to be used on Cedar Backup media. :returns: Media label as a string

CedarBackup3.actions.util.checkMediaState(storeConfig)[source]¶

Checks state of the media in the backup device to confirm whether it has been initialized for use with Cedar Backup.

We can tell whether the media has been initialized by looking at its media label. If the media label starts with MEDIA_LABEL_PREFIX, then it has been initialized.

The check varies depending on whether the media is rewritable or not. For non-rewritable media, we also accept a None media label, since this kind of media cannot safely be initialized.

Parameters:	storeConfig – Store configuration
Raises:	`ValueError` – If media is not initialized

CedarBackup3.actions.util.createWriter(config)[source]¶

Creates a writer object based on current configuration.

This function creates and returns a writer based on configuration. This is done to abstract action functionality from knowing what kind of writer is in use. Since all writers implement the same interface, there’s no need for actions to care which one they’re working with.

Currently, the cdwriter and dvdwriter device types are allowed. An exception will be raised if any other device type is used.

This function also checks to make sure that the device isn’t mounted before creating a writer object for it. Experience shows that sometimes if the device is mounted, we have problems with the backup. We may as well do the check here first, before instantiating the writer.

Parameters:	config – Config object
Returns:	Writer that can be used to write a directory to some media
Raises:	`ValueError` – If there is a problem getting the writer `IOError` – If there is a problem creating the writer object

CedarBackup3.actions.util.findDailyDirs(stagingDir, indicatorFile)[source]¶

Returns a list of all daily staging directories that do not contain the indicated indicator file.

Parameters:	stagingDir – Configured staging directory (config.targetDir)
Returns:	List of absolute paths to daily staging directories

CedarBackup3.actions.util.getBackupFiles(targetDir)[source]¶

Gets a list of backup files in a target directory.

Files that match INDICATOR_PATTERN (i.e. "cback.store", "cback.stage", etc.) are assumed to be indicator files and are ignored.

Parameters:	targetDir – Directory to look in
Returns:	List of backup files in the directory
Raises:	`ValueError` – If the target directory does not exist

CedarBackup3.actions.util.initializeMediaState(config)[source]¶

Initializes state of the media in the backup device so Cedar Backup can recognize it.

This is done by writing an mostly-empty image (it contains a “Cedar Backup” directory) to the media with a known media label.

Note: Only rewritable media (CD-RW, DVD+RW) can be initialized. It doesn’t make any sense to initialize media that cannot be rewritten (CD-R, DVD+R), since Cedar Backup would then not be able to use that media for a backup.

Parameters:	config – Cedar Backup configuration
Raises:	`ValueError` – If media could not be initialized `ValueError` – If the configured media type is not rewritable

CedarBackup3.actions.util.writeIndicatorFile(targetDir, indicatorFile, backupUser, backupGroup)[source]¶

Writes an indicator file into a target directory. :param targetDir: Target directory in which to write indicator :param indicatorFile: Name of the indicator file :param backupUser: User that indicator file should be owned by :param backupGroup: Group that indicator file should be owned by

Raises:	`IOException` – If there is a problem writing the indicator file

CedarBackup3.actions.validate module¶

Implements the standard ‘validate’ action. :author: Kenneth J. Pronovici <pronovic@ieee.org>

CedarBackup3.actions.validate.executeValidate(configPath, options, config)[source]¶

Executes the validate action.

This action validates each of the individual sections in the config file. This is a “runtime” validation. The config file itself is already valid in a structural sense, so what we check here that is that we can actually use the configuration without any problems.

There’s a separate validation function for each of the configuration sections. Each validation function returns a true/false indication for whether configuration was valid, and then logs any configuration problems it finds. This way, one pass over configuration indicates most or all of the obvious problems, rather than finding just one problem at a time.

Any reported problems will be logged at the ERROR level normally, or at the INFO level if the quiet flag is enabled.

Parameters:	configPath (String representing a path on disk) – Path to configuration file on disk options (Options object) – Program command-line options config (Config object) – Program configuration
Raises:	`ValueError` – If some configuration value is invalid