org.jets3t.apps.synchronize

Class Synchronize

java.lang.Object

org.jets3t.apps.synchronize.Synchronize

public class Synchronize
extends java.lang.Object

Console application to synchronize the local file system with a storage service. For more information and help please see the Synchronize Guide.

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	APPLICATION_DESCRIPTION String provided to service as User-Agent description.
protected static int	REPORT_LEVEL_ACTIONS
protected static int	REPORT_LEVEL_ALL
protected static int	REPORT_LEVEL_DIFFERENCES
protected static int	REPORT_LEVEL_NONE

Constructor Summary

Constructors

Constructor and Description
Synchronize(StorageService service, boolean doAction, boolean isQuiet, boolean isNoProgress, boolean isForce, boolean isKeepFiles, boolean isNoDelete, boolean isMoveEnabled, boolean isBatchMode, boolean isGzipEnabled, boolean isEncryptionEnabled, int reportLevel, Jets3tProperties properties) Constructs the application with a pre-initialised service and the user-specified options.

Method Summary

Modifier and Type	Method and Description
protected org.jets3t.apps.synchronize.Synchronize.ComparisonResult	compareLocalAndRemoteFiles(FileComparerResults mergedDiscrepancyResults, java.lang.String bucketName, java.lang.String rootObjectPath, java.lang.String priorLastKey, java.util.Map<java.lang.String,java.lang.String> objectKeyToFilepathMap, BytesProgressWatcher md5GenerationProgressWatcher)
static void	main(java.lang.String[] args) Runs this application from the console, accepts and checks command-line parameters and runs an upload or download operation when all the necessary parameters are provided.
void	restoreToLocalDirectory(java.util.Map<java.lang.String,java.lang.String> objectKeyToFilepathMap, java.lang.String rootObjectPath, java.io.File localDirectory, StorageBucket bucket, BytesProgressWatcher md5GenerationProgressWatcher) Copies the contents of a root path in service to the local file system.
void	run(java.lang.String servicePath, java.io.File[] files, java.lang.String actionCommand, java.lang.String cryptoPassword, java.lang.String aclString, java.lang.String providerId) Runs the application, performing the action specified on the given service and local directory paths.
void	run(java.lang.String servicePath, java.util.List<java.io.File> files, java.lang.String actionCommand, java.lang.String cryptoPassword, java.lang.String aclString, java.lang.String providerId) Runs the application, performing the action specified on the given service and local directory paths.
void	uploadLocalDirectory(java.util.Map<java.lang.String,java.lang.String> objectKeyToFilepathMap, StorageBucket bucket, java.lang.String rootObjectPath, java.lang.String aclString, BytesProgressWatcher md5GenerationProgressWatcher) Copies the contents of a local directory to a service, storing them in the given root path.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

APPLICATION_DESCRIPTION

public static final java.lang.String APPLICATION_DESCRIPTION

String provided to service as User-Agent description.

See Also:

Constant Field Values

REPORT_LEVEL_NONE

protected static final int REPORT_LEVEL_NONE

See Also:

Constant Field Values

REPORT_LEVEL_ACTIONS

protected static final int REPORT_LEVEL_ACTIONS

See Also:

Constant Field Values

REPORT_LEVEL_DIFFERENCES

protected static final int REPORT_LEVEL_DIFFERENCES

See Also:

Constant Field Values

REPORT_LEVEL_ALL

protected static final int REPORT_LEVEL_ALL

See Also:

Constant Field Values

Constructor Detail

Synchronize

public Synchronize(StorageService service,
                   boolean doAction,
                   boolean isQuiet,
                   boolean isNoProgress,
                   boolean isForce,
                   boolean isKeepFiles,
                   boolean isNoDelete,
                   boolean isMoveEnabled,
                   boolean isBatchMode,
                   boolean isGzipEnabled,
                   boolean isEncryptionEnabled,
                   int reportLevel,
                   Jets3tProperties properties)

Constructs the application with a pre-initialised service and the user-specified options.

Parameters:

service - a pre-initialised service (including Provider credentials)

doAction - Files will only be transferred if true.

isQuiet - Report will only include summary of actions if true.

isNoProgress - Upload/download progress updates will not be printed.

isForce - Files will be overwritten when unchanged if true.

isKeepFiles - Files will not be replaced/deleted if true.

isMoveEnabled - If true, items will be moved rather than just copied. Files will be deleted after they have been uploaded, and objects will be deleted after they have been downloaded.

isBatchMode - If true, uploads or downloads will proceed in batches rather than all at once. This mode is useful for large buckets where listing all the objects and their details at once may consume a large amount of time and memory.

isNoDelete - Files will not be deleted if true, but may be replaced.

isGzipEnabled - Files will be gzipped prior to upload if true.

isEncryptionEnabled - Files will be encrypted prior to upload if true.

reportLevel - The level or amount of reporting to perform. The default value is REPORT_LEVEL_ALL.

properties - The configuration properties that will be used by this instance.

Method Detail

compareLocalAndRemoteFiles

protected org.jets3t.apps.synchronize.Synchronize.ComparisonResult compareLocalAndRemoteFiles(FileComparerResults mergedDiscrepancyResults,
                                                                                              java.lang.String bucketName,
                                                                                              java.lang.String rootObjectPath,
                                                                                              java.lang.String priorLastKey,
                                                                                              java.util.Map<java.lang.String,java.lang.String> objectKeyToFilepathMap,
                                                                                              BytesProgressWatcher md5GenerationProgressWatcher)
                                                                                       throws ServiceException,
                                                                                              java.security.NoSuchAlgorithmException,
                                                                                              java.io.FileNotFoundException,
                                                                                              java.io.IOException,
                                                                                              java.text.ParseException

Throws:

ServiceException

java.security.NoSuchAlgorithmException

java.io.FileNotFoundException

java.io.IOException

java.text.ParseException

uploadLocalDirectory

public void uploadLocalDirectory(java.util.Map<java.lang.String,java.lang.String> objectKeyToFilepathMap,
                                 StorageBucket bucket,
                                 java.lang.String rootObjectPath,
                                 java.lang.String aclString,
                                 BytesProgressWatcher md5GenerationProgressWatcher)
                          throws java.lang.Exception

Copies the contents of a local directory to a service, storing them in the given root path.

A set of comparisons is used to determine exactly how the local files differ from the contents of the service location, and files are transferred based on these comparisons and options set by the user.

The following object properties are set when a file is uploaded:

• The object's key name
• Content-Length: The size of the uploaded file. This will be 0 for directories, and will differ from the original file if gzip or encryption options are set.
• Content-Type: Mimetypes.MIMETYPE_BINARY_OCTET_STREAM for directories, otherwise a mimetype determined by Mimetypes.getMimetype(java.lang.String) unless the gzip option is set, in which case the Content-Type is set to application/x-gzip.

The following jets3t-specific metadata items are also set:

• The local file's last-modified date, as Constants.METADATA_JETS3T_LOCAL_FILE_DATE
• An MD5 hash of file data, as StorageObject.METADATA_HEADER_HASH_MD5

Parameters:

objectKeyToFilepathMap - map of '/'-delimited object key names to local file absolute paths

bucket - the bucket to put the objects in (will be created if necessary)

rootObjectPath - the root path where objects are put (will be created if necessary)

aclString - the ACL to apply to the uploaded object

md5GenerationProgressWatcher - a class that reports on the progress of this method

Throws:

java.lang.Exception

restoreToLocalDirectory

public void restoreToLocalDirectory(java.util.Map<java.lang.String,java.lang.String> objectKeyToFilepathMap,
                                    java.lang.String rootObjectPath,
                                    java.io.File localDirectory,
                                    StorageBucket bucket,
                                    BytesProgressWatcher md5GenerationProgressWatcher)
                             throws java.lang.Exception

Copies the contents of a root path in service to the local file system.

A set of comparisons is used to determine exactly how the service objects differ from the local target, and files are transferred based on these comparisons and options set by the user.

If an object is gzipped (according to its Content-Type) and the gzip option is set, the object is inflated. If an object is encrypted (according to the metadata item Constants.METADATA_JETS3T_CRYPTO_ALGORITHM) and the crypt option is set, the object is decrypted. If encrypted and/or gzipped objects are restored without the corresponding option being set, the user will be responsible for inflating or decrypting the data.

Note: If a file was backed-up with both encryption and gzip options it cannot be restored with only the gzip option set, as files are gzipped prior to being encrypted and cannot be inflated without first being decrypted.

Parameters:

objectKeyToFilepathMap - map of '/'-delimited object key names to local file absolute paths

rootObjectPath - the root path in service where backed-up objects were stored

localDirectory - the directory to which the objects will be restored

bucket - the bucket into which files were backed up

md5GenerationProgressWatcher - a class that reports on the progress of this method

Throws:

java.lang.Exception

run

public void run(java.lang.String servicePath,
                java.io.File[] files,
                java.lang.String actionCommand,
                java.lang.String cryptoPassword,
                java.lang.String aclString,
                java.lang.String providerId)
         throws java.lang.Exception

Runs the application, performing the action specified on the given service and local directory paths.

Parameters:

servicePath - the path in service (including the bucket name) to which files are backed-up, or from which files are restored.

files - an array of one or more File objects for Uploads, or a single target directory for Downloads.

actionCommand - the action to perform, UP(load) or DOWN(load)

cryptoPassword - if non-null, an EncryptionUtil object is created with the provided password to encrypt or decrypt files.

aclString - the ACL to apply to the uploaded object

providerId - service provider name: "S3" or "GS"

Throws:

java.lang.Exception

run

public void run(java.lang.String servicePath,
                java.util.List<java.io.File> files,
                java.lang.String actionCommand,
                java.lang.String cryptoPassword,
                java.lang.String aclString,
                java.lang.String providerId)
         throws java.lang.Exception

Runs the application, performing the action specified on the given service and local directory paths.

Parameters:

servicePath - the path in service (including the bucket name) to which files are backed-up, or from which files are restored.

files - a set of one or more of File objects for Uploads, or a single target directory for Downloads.

actionCommand - the action to perform, UP(load) or DOWN(load)

cryptoPassword - if non-null, an EncryptionUtil object is created with the provided password to encrypt or decrypt files.

aclString - the ACL to apply to the uploaded object

providerId - service provider name: "S3" or "GS"

Throws:

java.lang.Exception

main

public static void main(java.lang.String[] args)
                 throws java.lang.Exception

Runs this application from the console, accepts and checks command-line parameters and runs an upload or download operation when all the necessary parameters are provided.

Parameters:

args -

Throws:

java.lang.Exception