DockStar Backup
The most basic backup is to make a clone of the USB-stick. That can be done with ddrescue.
In order to have a convenient data backup I wanted to have something similar to Time Machine from Apple Mac OS X.
The program Back in time did a good job in the past on my Ubuntu 12.04 server, so I wanted to use it also on the Dockstar server in console mode with Debian 7 (wheezy). Unfortunately this is not very much documented. Therefore I try to document this console mode.
There is no parameter editor for the console. The config file was setup with the GUI under Ubuntu 12.04 and copied to the Debian installation.
Clone USB-stick
If you have just one free USB port, copy from the source USB-stick to a file on the hard disk, then from the file to the target USB-stick.
The file system is ext2, because of less disk access than ext4.
If the program ddrescue and gparted is not jet installed, install it with:
$ sudo apt-get install gddrescue $ sudo apt-get install gparted
Then make sure which USB-sticks are plugged in, with gparted, or:
sudo lshw -C disk
Check the source drive partition for any errors with gparted, or:
# Partition must be unmounted for the check! $ umount /dev/sdb1 $ sudo e2fsck -fv /dev/sdb1 # for example /dev/sdb1 # option -v = verbose, tell about work in progress # option -f = force to check, even if the partition looks clean.
The target USB-stick must be equal in size of the source drive, or larger. Then you clone with:
# Partitions must be mounted, mounting is done by pulling and plugging the USB-stick. # source Partition is for example: /dev/sdb1 # target Partition is for example: /dev/sdc1 $ sudo ddrescue -vfb 1M /dev/sdb1 /dev/sdc1 ddrescue.log # option -v = verbose, tell about work in progress # option -f = force to overwrite, in case of a drive as target # option -b 1M = use 1 MB chunks as block size
The swap partition (primary partition, 256 MB) should be created next, with gparted.
Back in Time Installation
Because "Back in time" is not in the Debian repository, you have to install it from the source code (Python based).
Download backintime_1.0.10_orig.tar.gz for Debian wheezy 2012-05-21
- Install the program:
# take care about prerequisites $ sudo apt-get install rsync make gettext # unpack the archive $ tar xzf backintime_1.0.10_orig.tar.gz $ cd backintime-1.0.10/common # Create Makefile $ ./configure # Install the user part of backintime $ make # install the root parts of backintime $ sudo make install # read the man page $ man backintime # check for version $ backintime -v 1.0.10 # This is hard coded in file: common/config.py # The manual file for the 'config' file must be downloaded, see Links # copy in place $ sudo cp backintime-config.1 /usr/share/man/man1/ $ sudo gzip -f /usr/share/man/man1/backintime-config.1 # Read content $ man backintime-config
Backup Media
The easiest Backup Media is an USB-Stick of 4 GB size or more. The procedure to integrate it in Back in Time is:
# plug in USB-Stick # format to ext4 file system $ sudo mkfs.ext4 /dev/sdb1 # test for file system check (none was set) $ sudo tune2fs -l /dev/sdb1 ... Mount count: 0 Maximum mount count: -1 ... Check interval: 0 (<none>) # setup file system check, every 101 mounts or every 91 days, whatever comes first. $ sudo tune2fs -c 101 -i 91 /dev/sdb1 $ sudo tune2fs -l /dev/sdb1 ... Mount count: 0 Maximum mount count: 101 Last checked: Thu Jan 31 12:08:14 2013 Check interval: 7862400 (3 months, 1 day) Next check after: Thu May 2 13:08:14 2013 # set a label name $ sudo e2label /dev/sdb1 SAVE # make a directory $ sudo mkdir /media/SAVE # mount partition $ sudo mount /dev/sdb1 /media/SAVE # check for mounted $ mount ... /dev/sdb1 on /media/SAVE type ext4 (rw,relatime,user_xattr,barrier=1,data=ordered) # in order to mount at boot time edit /etc/fstab # append the line: /dev/sdb1 /media/SAVE ext4 rw,relatime,user_xattr,barrier=1,data=ordered # check for content $ ls /media/SAVE/ lost+found # test with a reboot $ sudo shutdown -r now $ mount ... /dev/sdb1 on /media/SAVE type ext4 (rw,relatime,user_xattr,barrier=1,data=ordered)
Back in Time use
Only user data and configuration data are saved (/etc, /home and /var). In case not only the /home/user folder is saved, Back in Time must be run with root rights.
# check for total data amount $ du -BK /etc 2108K /etc $ du -BK /home 158292K /home $ du -BK /var 233268K /var # The configuration file is in either (user/root): ~/.config/backintime/config or for root /root/.config/backintime/config # Do the first snapshot $ sudo backintime -b (run with root rights) Back In Time Version: 1.0.10 Back In Time comes with ABSOLUTELY NO WARRANTY. This is free software, and you are welcome to redistribute it under certain conditions; type `backintime --license' for details. INFO: Lock INFO: on process begins INFO: Profile_id: 1 INFO: Call rsync to take the snapshot INFO: Command "rsync -rtDH --links --no-p --no-g --no-o --delete --delete-excluded -v --chmod=Du+wx --exclude="/media/SAVE" --exclude="/root/.local/share/backintime" --include="/etc/" --include="/home/" --include="/var/" --exclude=".gvfs" --exclude=".cache*" --exclude="[Cc]ache*" --exclude=".thumbnails*" --exclude="[Tt]rash*" --exclude="*.backup*" --exclude="*~" --exclude="/root/Ubuntu One" --exclude=".dropbox*" --include="/etc/**" --include="/home/**" --include="/var/**" --exclude="*" / "/media/SAVE/backintime/FADS11/root/1/new_snapshot/backup/" 2>&1" returns 0 INFO: Save config file INFO: Command "cp /root/.config/backintime/config /media/SAVE/backintime/FADS11/root/1/new_snapshot/backup/.." returns 0 INFO: Save permissions INFO: Create info file INFO: Command "chmod -R a-w "/media/SAVE/backintime/FADS11/root/1/20130131-155214-750/backup"" returns 0 INFO: Remove backups older than: 20030131-000000 INFO: [smart remove] considered: ['20130131-155214-750'] INFO: [smart remove] There is only one snapshots, so keep it INFO: Keep min free disk space: 1024 Mb INFO: Unlock $ du -BK /media/SAVE ... 222648K /media/SAVE/
At last setup the root crontab, as explained in the next chapter.
In case of a media change you get a WARNING and no backup:
user.warn backintime (root): WARNING: Can't find snapshots folder ! # to fix it, make the directory $ sudo mkdir -p /media/SAVE/backintime/<hostname>/root/1
root crontab
In order to let Back in time do the job automatically, it can be done with cron. Following you will see how to setup cron with root rights.
# Edit root crontab table
$ sudo crontab -e
#Back In Time system entry, this was be edited by the GUI:
#m h d m wd
0 0 * * * nice -n 19 ionice -c2 -n7 /usr/bin/backintime --backup-job >/dev/null 2>&1
# explanations
0 0 : at midnight
nice -n 19 : lowest priority
ionice -c2 -n7 : class "best effort", low priority
--backup-job
take a snapshot (if needed) depending on schedule rules (used for cron jobs)
>/dev/null : no output
2>&1 : error output mapped to output
# check for setup
$ sudo crontab -l
...
# m h dom mon dow command
# run backintime at midnight
0 0 * * * nice -n 19 ionice -c2 -n7 /usr/bin/backintime --backup-job >/dev/null 2>&1
config file
The settings were read out from the Back in Time GUI:
General:
Where to save snapshots: /media/SAVE
Schedule: every day, 00:00 (midnight)
Include:
/etc
/home
/var
Excludes:
.gvfs
.cache*
.thumbnails*
.trash*
*backup*
~*
/root/UbuntuOne
.dropbox*
Auto-remove:
older than: 10 years
if free space is less than: 1 GB
smart remove:
keep all snapshots for at least: 2 days
keep one snapshot per day for at least: 7 days
keep one snapshot per week for at least: 4 weeks
keep one snapshot per month for at least: 24 month
keep one snapshot per year for all years
Don't remove named snapshots
Options:
Enable notification
Backup files on restore
Log level: All
Expert options:
Run 'nice' as cron job
Run 'ionice' as cron job This is the produced config file, done with the GUI under Ubuntu 12.04. Adopt as needed.
Delete the exclude [Cc]ache*, otherwise the moinmoin draft file in wiki/data/cache/wikiconfig/drafts/ will not be saved.
$ cat /root/.config/backintime/config gnome.last_path=/root gnome.main_window.height=532 gnome.main_window.hpaned1=200 gnome.main_window.hpaned2=200 gnome.main_window.width=782 gnome.main_window.x=121 gnome.main_window.y=34 gnome.show_hidden_files=false # #0-manual,10-hourly,20-daily,30-weekly,40-monthly profile1.snapshots.automatic_backup_mode=20 profile1.snapshots.automatic_backup_time=0 profile1.snapshots.backup_on_restore.enabled=true profile1.snapshots.continue_on_errors=false profile1.snapshots.copy_links=false profile1.snapshots.copy_unsafe_links=false profile1.snapshots.cron.ionice=true profile1.snapshots.cron.nice=true profile1.snapshots.dont_remove_named_snapshots=true profile1.snapshots.exclude.1.value=.gvfs profile1.snapshots.exclude.2.value=.cache* profile1.snapshots.exclude.3.value=.thumbnails* profile1.snapshots.exclude.4.value=[Tt]rash* profile1.snapshots.exclude.5.value=*.backup* profile1.snapshots.exclude.6.value=*~ profile1.snapshots.exclude.7.value=/root/Ubuntu One profile1.snapshots.exclude.8.value=.dropbox* profile1.snapshots.exclude.size=8 profile1.snapshots.include.1.type=0 profile1.snapshots.include.1.value=/etc profile1.snapshots.include.2.type=0 profile1.snapshots.include.2.value=/home profile1.snapshots.include.3.type=0 profile1.snapshots.include.3.value=/var profile1.snapshots.include.size=3 profile1.snapshots.log_level=3 profile1.snapshots.min_free_space.enabled=true # 10=Mb, 20=Gb profile1.snapshots.min_free_space.unit=20 profile1.snapshots.min_free_space.value=1 profile1.snapshots.no_on_battery=false profile1.snapshots.notify.enabled=true profile1.snapshots.path=/media/SAVE profile1.snapshots.path.auto=true profile1.snapshots.path.host=rudiswiki.de profile1.snapshots.path.profile=1 profile1.snapshots.path.user=root profile1.snapshots.preserve_acl=false profile1.snapshots.preserve_xattr=false profile1.snapshots.remove_old_snapshots.enabled=true # 20=days,30=weeks,80=years profile1.snapshots.remove_old_snapshots.unit=80 profile1.snapshots.remove_old_snapshots.value=10 profile1.snapshots.smart_remove=true profile1.snapshots.smart_remove.keep_all=2 profile1.snapshots.smart_remove.keep_one_per_day=7 profile1.snapshots.smart_remove.keep_one_per_month=24 profile1.snapshots.smart_remove.keep_one_per_week=4 profile1.snapshots.use_checksum=false profile1.snapshots.user_backup.ionice=false profiles.version=1
user id
The user id is stored in file /etc/group.
If you want to know your user id type:
$ id uid=1001(rudi) gid=1001(rudi) ...
To change your user id number to number xxxx, you can do (see also: man usermod):
$ sudo usermod -u xxxx rudi # works in the "home" folder only!
To see, which users are logged in:
$ w 16:57:58 up 6:51, 1 user, load average: 0,11, 0,04, 0,05 USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT rudi pts/0 rudis-mac-pro.fr 10:07 0.00s 0.27s 0.01s w
To see, which open files belong to whom:
$ lsof COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME ... lsof 2317 rudi cwd DIR 8,1 4096 39965 /home/rudi ...
To go through all files (not only home directory), check if they have the old uid and change it to the new one:
$ sudo find / -uid 1001 -exec chown -R rudi {} \;If you want to see the uid number of a file you can use the command find:
$ find heizung -type f -printf "%h/%f %U\n" ... heizung/hd131115.txt 1001
Backup File Tree
In order to have a full backup of the file tree for creating an new USB-Stick in case of a hardware failure, I developed the following shell script backup_FADS90.sh.
The restore procedure is documented in the shell script, as an example. You need to adopt it to your system.
First line in the shell script is #!/bin/sh
# file: backup_FADS90.sh
# Info: https://help.ubuntu.com/community/BackupYourSystem/TAR
# Info: http://www.gnu.org/software/tar/manual/tar.html
# 2013-11-17 RudolfReuter
#
# restore to a new USB-stick, e.g. (suit to your system):
# $ cd <backup folder>
# $ sudo mount /dev/sdb1 /mnt
# $ sudo tar xvpfa FADS90-2013-11-17.tar.gz -C /mnt/
# option "-C <folder>" change to folder before extracting
# Make sure only root can run our script
if [ "$(id -u)" != "0" ]; then
echo "This script must be run as root" 1>&2
exit 1
fi
# Are we on the root of our file system?
if [ ! -d "./boot" ]; then
echo "Error: This file must be on the root (/) level (Dockstar)" 1>&2
fi
DEST="/media/SAVE/FADS90-$(date +%Y-%m-%d).tar.gz"
EXCLUDE="
--exclude=*.pyc
--exclude=./mnt/*
--exclude=./media/*
--exclude=./proc/*
--exclude=./sys/*
--exclude=./lost+found/*
--exclude=./tmp/*
--exclude=./var/cache/apt/apt-file/*
--exclude=./var/cache/apt/archives/*
--exclude=./var/lib/apt/lists/*dists*
--exclude=./var/run/*
--exclude=./var/tmp/*
--exclude=./var/log/*
--exclude=./.Trash*
"
# option "--one-file-system" excludes /dev, /media, /mnt, /proc, /run, /sys
# option "-a" enables automatic compressed format recognition based on the archive suffix
# option "-c" create archiv
# option "-f <file name>" output to file
# option "-p" preserves all permissions
# option "-v" verbose messages
START=$(date +%s)
# for debugging the shell script
set -x
tar cvpfa ${DEST} ${EXCLUDE} --one-file-system .
FINISH=$(date +%s)
echo "total time: $(( ($FINISH-$START) / 60 )) minutes, $(( ($FINISH-$START) % 60 )) seconds"
Backup logread
In order to avoid too much USB-Stick writes, the system log is written in a circular buffer in RAM. In case of an error reboot the log is lost. To habe at least some history, you can save on a daily base the log file, with the following shell script logread_bak.sh.
First line in the shell script is #!/bin/sh
# file: logread_bak.sh # save daily the Linux log (logread) to an external USB-Stick # "logread" uses a circular RAM buffer # run the script per (user) cron at 6 o'clock every day # edit cron parameter: # $ crontab -e # m h d m wd # 0 6 * * * /home/rudi/Install/logread_bak.sh >/dev/null 2>&1 USB_STICK="/media/SAVE/log" logread > $USB_STICK/logread-$(date '+%Y-%m-%d_%T').log
Links
List of pages in this category:
-- RudolfReuter 2013-01-30 19:03:41
Go back to CategoryDockStar or FrontPage ; KontaktEmail (ContactEmail)