1. No category

Backup and Restore
Backup and Restore
Overview / CQ / Adobe Experience Manager 5.6.1 / CRX (embedded in AEM 5.6.1) / Administering /
There are two ways to back up and restore CRX repository content:
• You can create an external backup of the repository and store it in a safe location. If the repository
breaks down, you can restore it to the previous state.
• You can create internal versions of the repository content. These versions are stored in the repository
along with the content, so you can quickly restore nodes and trees you have changed or deleted.
GENERAL
The approach described here applies for system backup and recovery.
If you need to backup and/or recover a small amount of content, which is lost, a recovery of the system is not
necessarily required:
• Either you can fetch the data from another system via a package
• or you restore the backup on a temporary system, create a content package and deploy it on the system,
where this content is missing.
For details, see Package Backup below.
TIMING
Do not run backup in parallel with the datastore garbage collection, as it might harm the results of both
processes.
Generally it is recommended to run the Tar Optimizer after the backup has been performed.
Offline Backup
You can always do an offline backup. This requires a downtime of CRX, but can be quite efficient in terms of
required time compared to an online backup.
In most cases you will use a filesystem snapshot to create a read-only copy of the storage at that time. To
create a offline backup perform these steps:
• stop the application
• make a snapshot backup
• start the application
As the snapshot backup usually takes only a few seconds, the entire downtime is less than a few minutes.
NOTE
Instead of stopping CRX, you can also use calls to JMX MBeans to prevent CRX from writing
to the disk, which reduces the time CRX is unavailable to seconds. This is described under
Filesystem Snapshot Backup below.
Online Backup
This backup method creates a backup of the entire repository, including any applications deployed under
it, such as CQ5. The backup includes content, version history, configuration, software, hotfixes, custom
applications, log files, search indexes, and so on. If you are using clustering and if the shared folder is a
subdirectory of crx-quickstart (either physically, or using a softlink), the shared directory is also backed up.
You can restore the entire repository (and any applications) at a later point.
This method operates as a hot or online backup so it can be performed while the repository is running.
Therefore the repository is usable while the backup is running. This method works for the default, TarPMbased, CRX instances.
When creating a backup, you have the following options:
© 2012 Adobe Systems Incorporated.
All rights reserved.
Page 1
Created on 2014-11-09
Backup and Restore
• Backing up to a directory using CRX's integrated backup tool.
• Backing up to a directory using a filesystem snapshot
In any case, the backup creates an image (or snapshot) of the repository. Then the systems backup agent
should take care to actually transfer this image to a dedicated backup system (tape drive).
CAUTION
The online backup only backs up the file system. If you store the repository content and/or the
repository files in a database, that database needs to backed up separately.
The online backup only backs up the file system. If you store the repository content and/or
the repository files in a database, that database needs to backed up separately.
CRX ONLINE BACKUP
An online backup of your repository lets you create, download, and delete backup files. It is a "hot" or
"online" backup feature, so can be executed while the repository is being used normally in the read-write
mode.
When starting a backup you can specify a Target Path and/or a Delay.
Target Path
The backup files are usually saved in the parent folder of the folder holidng the quickstart jar file (.jar). For
example, if you have the CRX jar file located under /InstallationKits/CRX, then the backup will be generated
under /InstallationKits. You can also specify a target to a location of your choice.
If the TargetPath is a directory, the image of the repository is created in this directory. If the same directory
is used multiple times (or always) to storing backup,
• modified files in the repository are modified accordingly in the TargetPath
• deleted files in the repository are deleted in the TargetPath
• created files in the repository are created in the TargetPath
NOTE
If TargetPath is set to filename with the extension .zip, the repository is backupped to a
temporary directory and then the content of this temporary directory is compressed and stored in
the ZIP file.
This approach is discouraged, because
• it requires additional disk storage during the backup process (temporary directory plus the zip
file)
• the compression process is done by the CRX and might influence its performance.
• It delays the backup process.
• Up to Java 1.6 Java is only able to create ZIP files up to a size of 4 gigabytes.
If you need to create a ZIP as backup format, you should backup to a directory and then use a
compression program to create the zip file.
Delay
Indicates a time delay (in milliseconds), so that repository performance is not affected.
By default, the CRX backup runs at full speed. You can slow down creating an online backup, so that it does
not slow down other tasks.
A delay of 1 millisecond typically results in 10% CPU usage, and a delay of 10 milliseconds usually results in
less than 3% CPU usage. The total delay in seconds can be estimated as follows:
Repository size in MB * delay in milliseconds / 2 (if the zip option is used)
or / 4 (when backing up to a directory).
That means a backup to a directory of a 200 MB repository with 1 ms delay increases the backup time by
about 50 seconds.
© 2012 Adobe Systems Incorporated.
All rights reserved.
Page 2
Created on 2014-11-09
Backup and Restore
You can access backups (made to both zip files and target directories) from the file
system. From here you can use any kind of backup tool (full backup or incremental
backup).
You can also back up and restore tar files.
NOTE
See How CRX Online Backup Works for internal details of the process.
To create a backup:
1.
Log in to CRX as the administrator (admin).
2.
Select Backup from the Welcome screen:
3.
The backup console will open. Specify the Target Path and Delay as required.
NOTE
The backup console is also available using:
http://<hostname>:<port-number>/libs/granite/backup/content/admin.html
4.
Click Start, a progress bar will indicate the progress of the backup.
NOTE
You can Cancel a running backup at any time.
5.
When the backup is complete the zip files are listed in the left pane of the console.
© 2012 Adobe Systems Incorporated.
All rights reserved.
Page 3
Created on 2014-11-09
Backup and Restore
NOTE
Backup files that are no longer needed can be removed using the console. Select the
backup file in the left pane, then click Delete.
NOTE
If you have backed up to a directory: after the backup process is finished CRX will not
write to the target directory.
AUTOMATING CRX ONLINE BACKUP
If possible, the online backup should be run when there is little load on the system, for example in the
morning. By default the Tar PM optimization runs between 2 am and 5 am, which also slows down the
system, that means a good time to run the online backup is 5 am.
Backups can be automated using the wget or curl HTTP clients. The following show examples of how to
automate backup by using curl.
Backing up to the default Target Directory
CAUTION
In the following example various parameters in the curl command might need to be configured for
your instance; for example, the hostname (localhost), port (4502), admin password (xyz) and file
name (backup.zip).
curl -u admin:admin -X POST http://localhost:4502/system/console/jmx/
com.adobe.granite:type=Repository/op/startBackup/java.lang.String?target=backup.zip
The backup file/directory is created on the server in the parent folder of the folder containing the crxquickstart folder (the same as if you were creating the backup using the browser). For example, if you
have installed CRX in the directory /InstallationKits/crx-quickstart/, then the backup is created in the /
InstallationKits directory.
The curl command returns immediately, so you must monitor this directory to see when the zip file is ready.
While the backup is being created a temp directory (with the name based on that of the final zip file) can be
seen, at the end this will be zipped. For example:
• name of resulting zip file: backup.zip
• name of temporary directory: backup.f4d5.temp
Backing up to a non-default Target Directory
Usually the backup file/directory is created on the server in the parent folder of the folder containing the crxquickstart folder.
© 2012 Adobe Systems Incorporated.
All rights reserved.
Page 4
Created on 2014-11-09
Backup and Restore
If you want to save your backup (of either sort) to a different location you can set an absolute path to the
target parameter in the curl command.
For example, to generate backupJune.zip in the directory /Backups/2012:
curl -u admin:admin -X POST http://localhost:4502/system/console/jmx/
com.adobe.granite:type=Repository/op/startBackup/java.lang.String?target=/Backups/2012/
backupJune.zip"
CAUTION
When using a different application server (such as JBoss), the online backup may not work as
expected, because the target directory is not writable. In this case, please contact Support.
If the file data store has been configured outside the main repository, it is not included
in the backup. This will reduce the size of the online backup and the size of the backup
zip file. However, the data store needs to be backed up as well. Because files in the file
data store directory are immutable, they can be backed up incrementally (potentially using
rsync) or after running the online backup.
Do not run the data store backup and garbage collection concurrently.
NOTE
A backup can also be triggered using the MBeans provided by CRX.
A backup can also be triggered using the MBeans provided by CRX.
FILESYSTEM SNAPSHOT BACKUP
With CRX 2.3 it is possible to prevent writing to disk via a JMX call. While this is used internally by the CRX
backup tool, it can also be used by an external process.
The process described here is specially suited for large repositories.
NOTE
If you want to use this backup approach, your system must support filesystem snapshots. E.g. for
Linux this means your filesystems should be placed on a logical volume.
1.
Via JMX call the operation blockRepositoryWrites on the MBean com.adobe.granite:type=Repository.
This will prevent CRX from writing to disk. Instead all write requests are blocked.
2.
Do a snapshot of the filesystem CRX is deployed on.
3.
Via JMX call the operation unblockRepositoryWrites on the MBean
com.adobe.granite:type=Repository.
The blocked write requests will continue to work.
4.
Mount the filesystem snapshot, do a backup from there, unmount the snapshot.
CAUTION
© 2012 Adobe Systems Incorporated.
All rights reserved.
Page 5
Created on 2014-11-09
Backup and Restore
To call unblockRepositoryWrites, you need to use a locally attached JMX client.
The JMX console is inappropriate in this case, as its availability is not guaranteed after
blockRepositoryWrites has been called.
BACKING UP THE DATA STORE SEPARATELY
If the file data store has been configured outside the main repository, it is not included in the backup. This
will reduce the size of the online backup and the backup directory. However, the data store needs to be
backed up as well. Because files in the file data store directory are immutable, they can be backed up
incrementally (potentially using rsync) or after running the online backup.
NOTE
Do not run the data store backup and garbage collection concurrently.
If a shared directory should be included in the backup (usually yes, as it contains the
data store), then the shared directory needs to be a subdirectory of the backup source
directory (installDir). This is the case in the default installation. If this is not the case in your
installation, then one solution is to create a soft link to the shared directory from within the
backup source directory.
HOW CRX ONLINE BACKUP WORKS
CRX Online Backup is comprised of a series of internal actions to ensure the integrity of the data being
backed up and the backup file(s) being created. These are listed below for those interested:
1. The online backup uses the following algorithm:
2. When creating a zip file, the first step is to create or locate the target directory.
1. If backing up to a zip file, a temporary directory is created. The directory name starts with backup.
and ends with .temp; for example backup.f4d3.temp.
2. If backing up to a directory, the name specified in the target path is used. An existing directory can
be used, otherwise a new directory will be created.
An empty file named backupInProgress.txt is created in the target directory when the backup starts.
This file is deleted when the backup is finished.
3. All files are copied from the source directory to the target directory (or temporary directory when creating
a zip file). The progress bar indicator of this sub-process is between 0% - 70% when creating a zip file,
or 0% - 100% if no zip file is created.
4. If the backup is being made to a pre-existing directory, then "old" files in the target directory are deleted.
Old files are files that do not exist in the source directory.
5. The files are copied to the target directory in four stages.
1. In the first copy stage (progress indicator 0% - 63% when creating a zip file or 0% - 90% if no zip file
is created), all files are copied concurrently while the repository is running normally.
2. In the second copy stage (progress indicator 63% - 66.5% when creating a zip file or 90% - 95% if
no zip file is created) only files that were created or modified in the source directory since the first
copy stage was started are copied. Depending on the activity of the repository, this might range from
no files at all, up to a significant number of files (because the first file copy stage usually takes a lot
of time).
3. In the third copy stage (progress indicator 66.5% - 68.6% when creating a zip file or 95% - 98% if no
zip file is created) only files that were created or modified in the source directory since the second
copy stage was started are copied. Depending on the activity of the repository, there might be no
files to copyl, or a very small number of files (because the second file copy stage is usually fast).
4. File copy stages one to three are all done concurrently while the repository is running. The fourth
and last file copy stage will first lock repository write operations (write operations are paused; they do
© 2012 Adobe Systems Incorporated.
All rights reserved.
Page 6
Created on 2014-11-09
Backup and Restore
not throw an exception, but will wait). Only files that were created or modified in the source directory
since the third copy stage was started are copied. Depending on the activity of the repository, there
might be no files to copy, or a very, very small number of files (because the second file copy stage
usually is very fast). After that, repository access continues. Progress indicator 68.6% - 70% when
creating a zip file or 98% - 100% if no zip file is created.
6. Depending on the target:
1. If a zip file was specified, this is now created from the temporary directory. Progress indicator 70% 100%. The temporary directory is then deleted.
2. If the target was a directory, the empty file named backupInProgress.txt is deleted to indicate that the
backup is finished.
Restoring the Backup
NOTE
This kind of restore restores the complete repository including the application, all content, logfiles,
etc.
To restore the backup from a backup:
1.
Restore a backup image on the system. if you backup the datastore separately, make sure, that the
datastore is restored as well to the correct location.
In case you have created a backup as a zip file, unpack this zip file using:
jar -xvf backupJune.zip
2.
3.
On Unix systems, the "x"-bit of the following scripts are not preserved by the zip file:
• server/start
• server/stop
• server/serverctl
You have to adjust these manually after restoring the backup.
Now the repository is ready to use. You can start it now using the regular start scripts.
•
Large ZIP (> 4GB) files created with the backup mechanism may fail to open with
some ZIP tools. In that case, use the Java jar tool as documented.
• Using this method, you can restore only the entire repository. If you need to
restore a single node or tree, you have to restore the entire backup in a separate
location, and then copy the node or tree over to your current repository.
On Unix systems, the "x"-bit of the following scripts are not preserved by the zip file:
• server/start
• server/stop
• server/serverctl
You have to adjust these manually after restoring the backup.
As your repository grows the size can start to impact the length of time required to make a
backup. This in turn can impact either downtime or performance of the application.
There are various options available for you to consider when making backups of a large
repository:
A snapshot backup involves taking a read-only copy of a storage device at a given
moment. They are designed to be instantaneous, or as close as possible.
To make a snapshot backup of your application (for example, CRX or CQ) you need to:
© 2012 Adobe Systems Incorporated.
All rights reserved.
Page 7
Created on 2014-11-09
Backup and Restore
• stop the application
• make a snapshot backup
• start the application
As the snapshot backup usually takes only a few seconds, the entire downtime is less than
a few minutes; and if you are running a cluster, you only need to stop and backup one
cluster node so there is no downtime.
The online backup can write to a target directory instead of a zip file. With this you can
achieve both full and incremental backups.
Once the online backup is finished, you can backup this target directory using either a
snapshot or a classical incremental backup tool.
You can also configure a time delay when performing online backups to minimize the
impact the backup has on the performance of CRX (or CQ).
Storing your data store outside the repository allows it to be backed up separately. It also
allows you to take incremental backups of the data store which will be quicker than a full
backup (though sporadic full backups are recommended to provide a base point should a
restore ever be required).
The datastore directory can be backed up at runtime, after the repository; datastore
garbage
collection must not be run until the backup is finished.
This approach has some unique features:
• additional disk consumption is low. A snapshot only consumes space if data on the original data is
changed.
• Snapshots are very fast and efficient, therefor the required time to create a snapshot is low (much lower
than copying the data), as only metadata are duplicated.
Package Backup
To back up and restore content, you can use one of the following:
• Package Manager, which uses the Content Package format to back up and restore content. The
Package Manager provides more flexibility in defining and managing packages.
• Content Zipper, which uses the CRX Package, XML Sys View Package, XML Doc View, or ZIP format to
back up content.You restore content in these package formats with the Content Loader. See Creating a
Backup using the Content Zipper and Restoring a Backup using the Content Loader.
For details on the features and tradeoffs of each of these individual content package formats, see Importing
and Exporting Content in the User Guide.
SCOPE OF BACKUP
When you back up nodes using either the Package Manager or the Content Zipper, CRX saves the following
information:
• The CRX repository content below the tree you have selected.
• The Node type definitions that are used for the content you back up.
• The Namespace definitions that are used for the content you back up.
When backing up, CRX loses the following information:
• The version history.
CREATING A BACKUP USING THE CONTENT ZIPPER
To create the backup using the Content Zipper:
1. Lock the top node of the tree you want to back up or a parent node of that node.
2. In the Content Zipper, type the path of the tree you want to back up. For a format, click CRX package.
© 2012 Adobe Systems Incorporated.
All rights reserved.
Page 8
Created on 2014-11-09
Backup and Restore
3. Click Submit Query. Your Web browser now offers the package file as a download. Save the download
on your computer.
4. Unlock the node again.
The file you have downloaded contains the current version of the tree you have exported, including the node
type and namespace definitions, but without the version history.
NOTE
The CRX package file is Adobe’s proprietary file format for CRX node information. It is optimized
for a small file size and optimal performance. If you prefer a standard XML file for further
processing, click XML sys view in step 2. If you use the file only for archiving, use the CRX
package format. Importing and Exporting Content describes the various file formats and their
uses.
RESTORING A BACKUP USING THE CONTENT LOADER
To restore the backup using the Content Loader:
1. Lock the node you want to restore. You can still modify the node and the nodes below it, but others
cannot.
2. In the Content Loader, load the CRX package that you want to restore.
3. Unlock the node again.
NOTE
You cannot restore the versioning history using the previous steps. CRX allows you to save the
version history, but it does not currently support restoring it.
© 2012 Adobe Systems Incorporated.
All rights reserved.
Page 9
Created on 2014-11-09