Backup creation is slow due to large file sizes


Enter Your Query:
Use '%' for wildcards and quotes for "exact phrases"


Top Level » DirectAdmin » Backup & Restore Process

Backup creation is slow due to large file sizesLast Modified: Jan 17, 2014, 5:53 pm
If your backups end up being very large and take a long time to create, you have a few options to reduce the load on your system:

1) Do backups less regularly.  Doing backups only once per week on the slowest day of the week will reduce the load for other days.

2) Only backup some users one day, other users another day, etc..   Spreading out the backup load for all accounts over the span of a week (or longer) will reduce your system load.

3) You can disable some areas of the backups if you don't need them, or want to use other backup methods (like rsync) for those areas.
Skip IMAP Data
Skip Uebimiau Data
Skip Databases
Skip the /home/user/domains directory.  Rsync would be a good replacement for this.
Skip the home.tar.gz (other files in the User's home)


4) Relating to #3 above, newer versions of DA have options as to which areas you'd like to backup directly in the backup options.

Admin Level -> Admin Backup/Transfer -> Create Backup

"Step 4: What" of the backup creation has options so you can pick what areas are to be included in the backup.

If Users have an over-sized /home/username/domains directory, you can use this technique to skip the "Domains Directory" option in the backup, and use rsync to transfer the Domains Directory.  Related guide:
http://help.directadmin.com/item.php?id=500


5) A more complex option is to move data under the User's home directory into a folder that won't be backed up.  A list of these directories can be found here.  For example, say they have a path with a lot of static images that don't need regular backups:

/home/user/domains/domain.com/public_html/images

This concept would be to move the images into a skipped folder, eg:

cd /home/user
mkdir var
chown user:user var
cd var
cp -Rp ../domains/domain.com/public_html/images .

which will now give you /home/user/var/images, which won't be included in the backup.

Next, move the current folder out of the way, and then link to it:

cd /home/user/domains/domain.com/public_html
mv images images.old
ln -s ../../../var/images .

and test to ensure it's working.  Once satisfied it's working, you can delete the images.old directory.

6) Some OS's support a program called ionice.  This will cap the rate of disk access done by any binary.
See this thread for information on using ionice to slow down the dataskq and it's child (tar) processes.

To cap just the tar binary with ionice, see this feature:
http://www.directadmin.com/features.php?id=1423
 
Related Helpfiles
Limiting Reseller and User Level Backup to only be able to run at specific times
Which folders are skipped from a DirectAdmin backup?
I want to spread out the backup process over a longer period of time to lower the load on the box.
I want to skip a lot of the User data when creating backups
Pulling data from a remote directory with rsync
Backups take too long to create. Use rsync for /home

© 2018 JBMC Software, Suite 173  3-11 Bellerose Drive, St Albert, AB  T8N 1P7  Canada.  Mon-Fri 9AM-5PM MST