DockStar Backup

The most basic backup is to make a clone of the USB-stick. That can be done with ddrescue.

In order to have a convenient data backup I wanted to have something similar to Time Machine from Apple Mac OS X.

The program Back in time did a good job in the past on my Ubuntu 12.04 server, so I wanted to use it also on the Dockstar server in console mode with Debian 7 (wheezy). Unfortunately this is not very much documented. Therefore I try to document this console mode.

There is no parameter editor for the console. The config file was setup with the GUI under Ubuntu 12.04 and copied to the Debian installation.

Clone USB-stick

If you have just one free USB port, copy from the source USB-stick to a file on the hard disk, then from the file to the target USB-stick.

The file system is ext2, because of less disk access than ext4.

If the program ddrescue and gparted is not jet installed, install it with:

$ sudo apt-get install gddrescue
$ sudo apt-get install gparted

Then make sure which USB-sticks are plugged in, with gparted, or:

sudo lshw -C disk

Check the source drive partition for any errors with gparted, or:

# Partition must be unmounted for the check!
$ umount /dev/sdb1
$ sudo e2fsck -fv /dev/sdb1   # for example /dev/sdb1
# option -v = verbose, tell about work in progress
# option -f = force to check, even if the partition looks clean.

The target USB-stick must be equal in size of the source drive, or larger. Then you clone with:

# Partitions must be mounted, mounting is done by pulling and plugging the USB-stick.
# source Partition is for example: /dev/sdb1
# target Partition is for example: /dev/sdc1

$ sudo ddrescue -vfb 1M /dev/sdb1 /dev/sdc1 ddrescue.log

# option -v = verbose, tell about work in progress
# option -f = force to overwrite, in case of a drive as target
# option -b 1M = use 1 MB chunks as block size

The swap partition (primary partition, 256 MB) should be created next, with gparted.

Back in Time Installation

Because "Back in time" is not in the Debian repository, you have to install it from the source code (Python based).

# take care about prerequisites
$ sudo apt-get install rsync make gettext

# unpack the archive
$ tar xzf backintime_1.0.10_orig.tar.gz

$ cd backintime-1.0.10/common
# Create Makefile
$ ./configure
# Install the user part of backintime
$ make

# install the root parts of backintime
$ sudo make install

# read the man page
$ man backintime

# check for version
$ backintime -v
1.0.10
# This is hard coded in file: common/config.py 

# The manual file for the 'config' file must be downloaded, see Links
# copy in place
$ sudo cp backintime-config.1 /usr/share/man/man1/
$ sudo gzip -f /usr/share/man/man1/backintime-config.1

# Read content
$ man backintime-config

Backup Media

The easiest Backup Media is an USB-Stick of 4 GB size or more. The procedure to integrate it in Back in Time is:

# plug in USB-Stick
# format to ext4 file system
$ sudo mkfs.ext4 /dev/sdb1

# test for file system check (none was set)
$ sudo tune2fs -l /dev/sdb1
...
Mount count:              0
Maximum mount count:      -1
...
Check interval:           0 (<none>)

# setup file system check, every 101 mounts or every 91 days, whatever comes first.
$ sudo tune2fs -c 101 -i 91 /dev/sdb1

$ sudo tune2fs -l /dev/sdb1
...
Mount count:              0
Maximum mount count:      101
Last checked:             Thu Jan 31 12:08:14 2013
Check interval:           7862400 (3 months, 1 day)
Next check after:         Thu May  2 13:08:14 2013

# set a label name
$ sudo e2label /dev/sdb1 SAVE

# make a directory
$ sudo mkdir /media/SAVE

# mount partition
$ sudo mount /dev/sdb1 /media/SAVE

# check for mounted
$ mount
...
/dev/sdb1 on /media/SAVE type ext4 (rw,relatime,user_xattr,barrier=1,data=ordered)

# in order to mount at boot time edit /etc/fstab
# append the line:
/dev/sdb1 /media/SAVE  ext4 rw,relatime,user_xattr,barrier=1,data=ordered

# check for content
$ ls /media/SAVE/
lost+found

# test with a reboot
$ sudo shutdown -r now

$ mount
...
/dev/sdb1 on /media/SAVE type ext4 (rw,relatime,user_xattr,barrier=1,data=ordered)

Back in Time use

Only user data and configuration data are saved (/etc, /home and /var). In case not only the /home/user folder is saved, Back in Time must be run with root rights.

# check for total data amount
$ du -BK /etc
2108K   /etc

$ du -BK /home
158292K /home

$ du -BK /var
233268K /var

# The configuration file is in either (user/root):
~/.config/backintime/config
or for root
/root/.config/backintime/config

# Do the first snapshot
$ sudo backintime -b  (run with root rights)

Back In Time
Version: 1.0.10

Back In Time comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to redistribute it
under certain conditions; type `backintime --license' for details.

INFO: Lock
INFO: on process begins
INFO: Profile_id: 1
INFO: Call rsync to take the snapshot
INFO: Command "rsync -rtDH --links --no-p --no-g --no-o  --delete --delete-excluded  -v  --chmod=Du+wx  --exclude="/media/SAVE" --exclude="/root/.local/share/backintime" --include="/etc/" --include="/home/" --include="/var/" --exclude=".gvfs" --exclude=".cache*" --exclude="[Cc]ache*" --exclude=".thumbnails*" --exclude="[Tt]rash*" --exclude="*.backup*" --exclude="*~" --exclude="/root/Ubuntu One" --exclude=".dropbox*" --include="/etc/**" --include="/home/**" --include="/var/**" --exclude="*" / "/media/SAVE/backintime/FADS11/root/1/new_snapshot/backup/" 2>&1" returns 0
INFO: Save config file
INFO: Command "cp /root/.config/backintime/config /media/SAVE/backintime/FADS11/root/1/new_snapshot/backup/.." returns 0
INFO: Save permissions
INFO: Create info file
INFO: Command "chmod -R a-w "/media/SAVE/backintime/FADS11/root/1/20130131-155214-750/backup"" returns 0
INFO: Remove backups older than: 20030131-000000
INFO: [smart remove] considered: ['20130131-155214-750']
INFO: [smart remove] There is only one snapshots, so keep it
INFO: Keep min free disk space: 1024 Mb
INFO: Unlock

$ du -BK /media/SAVE
...
222648K /media/SAVE/

At last setup the root crontab, as explained in the next chapter.

In case of a media change you get a WARNING and no backup:

user.warn backintime (root): WARNING: Can't find snapshots folder !

# to fix it, make the directory
$ sudo mkdir -p /media/SAVE/backintime/<hostname>/root/1

root crontab

In order to let Back in time do the job automatically, it can be done with cron. Following you will see how to setup cron with root rights.

# Edit root crontab table
$ sudo crontab -e
#Back In Time system entry, this was be edited by the GUI:
#m h d m wd
0 0 * * * nice -n 19 ionice -c2 -n7 /usr/bin/backintime --backup-job >/dev/null 2>&1

# explanations
  0 0            : at midnight
  nice -n 19     : lowest priority
  ionice -c2 -n7 : class "best effort", low priority

  --backup-job
    take a snapshot (if needed) depending on schedule rules (used for cron jobs)

  >/dev/null     : no output
  2>&1           : error output mapped to output

# check for setup
$ sudo crontab -l
...
# m h  dom mon dow   command
# run backintime at midnight
0 0 * * * nice -n 19 ionice -c2 -n7 /usr/bin/backintime --backup-job >/dev/null 2>&1

config file

The settings were read out from the Back in Time GUI:

General:
    Where to save snapshots: /media/SAVE
    Schedule: every day, 00:00  (midnight)

Include:
    /etc
    /home
    /var

Excludes:
    .gvfs
    .cache*
    .thumbnails*
    .trash*
    *backup*
    ~*
    /root/UbuntuOne
    .dropbox*

Auto-remove:
    older than: 10 years
    if free space is less than: 1 GB
    smart remove:
        keep all snapshots for at least: 2 days
        keep one snapshot per day for at least: 7 days
        keep one snapshot per week for at least: 4 weeks
        keep one snapshot per month for at least: 24 month
        keep one snapshot per year for all years
    Don't remove named snapshots

Options:
    Enable notification
    Backup files on restore
    Log level: All

Expert options:
    Run 'nice' as cron job
    Run 'ionice' as cron job 

This is the produced config file, done with the GUI under Ubuntu 12.04. Adopt as needed.

$ cat /root/.config/backintime/config
gnome.last_path=/root
gnome.main_window.height=532
gnome.main_window.hpaned1=200
gnome.main_window.hpaned2=200
gnome.main_window.width=782
gnome.main_window.x=121
gnome.main_window.y=34
gnome.show_hidden_files=false
#
#0-manual,10-hourly,20-daily,30-weekly,40-monthly
profile1.snapshots.automatic_backup_mode=20
profile1.snapshots.automatic_backup_time=0
profile1.snapshots.backup_on_restore.enabled=true
profile1.snapshots.continue_on_errors=false
profile1.snapshots.copy_links=false
profile1.snapshots.copy_unsafe_links=false
profile1.snapshots.cron.ionice=true
profile1.snapshots.cron.nice=true
profile1.snapshots.dont_remove_named_snapshots=true
profile1.snapshots.exclude.1.value=.gvfs
profile1.snapshots.exclude.2.value=.cache*
profile1.snapshots.exclude.3.value=.thumbnails*
profile1.snapshots.exclude.4.value=[Tt]rash*
profile1.snapshots.exclude.5.value=*.backup*
profile1.snapshots.exclude.6.value=*~
profile1.snapshots.exclude.7.value=/root/Ubuntu One
profile1.snapshots.exclude.8.value=.dropbox*
profile1.snapshots.exclude.size=8
profile1.snapshots.include.1.type=0
profile1.snapshots.include.1.value=/etc
profile1.snapshots.include.2.type=0
profile1.snapshots.include.2.value=/home
profile1.snapshots.include.3.type=0
profile1.snapshots.include.3.value=/var
profile1.snapshots.include.size=3
profile1.snapshots.log_level=3
profile1.snapshots.min_free_space.enabled=true
# 10=Mb, 20=Gb
profile1.snapshots.min_free_space.unit=20
profile1.snapshots.min_free_space.value=1
profile1.snapshots.no_on_battery=false
profile1.snapshots.notify.enabled=true
profile1.snapshots.path=/media/SAVE
profile1.snapshots.path.auto=true
profile1.snapshots.path.host=rudiswiki.de
profile1.snapshots.path.profile=1
profile1.snapshots.path.user=root
profile1.snapshots.preserve_acl=false
profile1.snapshots.preserve_xattr=false
profile1.snapshots.remove_old_snapshots.enabled=true
# 20=days,30=weeks,80=years
profile1.snapshots.remove_old_snapshots.unit=80
profile1.snapshots.remove_old_snapshots.value=10
profile1.snapshots.smart_remove=true
profile1.snapshots.smart_remove.keep_all=2
profile1.snapshots.smart_remove.keep_one_per_day=7
profile1.snapshots.smart_remove.keep_one_per_month=24
profile1.snapshots.smart_remove.keep_one_per_week=4
profile1.snapshots.use_checksum=false
profile1.snapshots.user_backup.ionice=false
profiles.version=1

user id

The user id is stored in file /etc/group.

If you want to know your user id type:

$ id
uid=1001(rudi) gid=1001(rudi) ...

To change your user id number to number xxxx, you can do (see also: man usermod):

$ sudo usermod -u xxxx rudi
# works in the "home" folder only!

To see, which users are logged in:

$ w
 16:57:58 up  6:51,  1 user,  load average: 0,11, 0,04, 0,05
USER     TTY      FROM             LOGIN@   IDLE   JCPU   PCPU WHAT
rudi     pts/0    rudis-mac-pro.fr 10:07    0.00s  0.27s  0.01s w

To see, which open files belong to whom:

$ lsof
COMMAND    PID     USER   FD      TYPE DEVICE SIZE/OFF  NODE NAME
...
lsof      2317     rudi  cwd       DIR    8,1     4096 39965 /home/rudi
...

To go through all files (not only home directory), check if they have the old uid and change it to the new one:

$ sudo find / -uid 1001 -exec chown -R rudi {} \;

If you want to see the uid number of a file you can use the command find:

$ find heizung -type f  -printf "%h/%f %U\n"
...
heizung/hd131115.txt 1001

Backup File Tree

In order to have a full backup of the file tree for creating an new USB-Stick in case of a hardware failure, I developed the following shell script backup_FADS90.sh.

The restore procedure is documented in the shell script, as an example. You need to adopt it to your system.

First line in the shell script is #!/bin/sh

# file: backup_FADS90.sh
# Info: https://help.ubuntu.com/community/BackupYourSystem/TAR
# Info: http://www.gnu.org/software/tar/manual/tar.html
# 2013-11-17 RudolfReuter
#
# restore to a new USB-stick, e.g. (suit to your system):
# $ cd <backup folder>
# $ sudo mount /dev/sdb1 /mnt
# $ sudo tar xvpfa FADS90-2013-11-17.tar.gz -C /mnt/
# option "-C <folder>" change to folder before extracting

# Make sure only root can run our script
if [ "$(id -u)" != "0" ]; then
   echo "This script must be run as root" 1>&2
   exit 1
fi

# Are we on the root of our file system?
if [ ! -d "./boot" ]; then
  echo "Error: This file must be on the root (/) level (Dockstar)" 1>&2
fi

DEST="/media/SAVE/FADS90-$(date +%Y-%m-%d).tar.gz"

EXCLUDE="
    --exclude=*.pyc
    --exclude=./mnt/*
    --exclude=./media/*
    --exclude=./proc/*
    --exclude=./sys/*
    --exclude=./lost+found/*
    --exclude=./tmp/*
    --exclude=./var/cache/apt/apt-file/*
    --exclude=./var/cache/apt/archives/*
    --exclude=./var/lib/apt/lists/*dists*
    --exclude=./var/run/*
    --exclude=./var/tmp/*
    --exclude=./var/log/*
    --exclude=./.Trash*
"
# option "--one-file-system" excludes /dev, /media, /mnt, /proc, /run, /sys
# option "-a" enables automatic compressed format recognition based on the archive suffix
# option "-c" create archiv
# option "-f <file name>" output to file
# option "-p" preserves all permissions
# option "-v" verbose messages

START=$(date +%s)
# for debugging the shell script
set -x

tar cvpfa ${DEST} ${EXCLUDE} --one-file-system .

FINISH=$(date +%s)
echo "total time: $(( ($FINISH-$START) / 60 )) minutes, $(( ($FINISH-$START) % 60 )) seconds"

Backup logread

In order to avoid too much USB-Stick writes, the system log is written in a circular buffer in RAM. In case of an error reboot the log is lost. To habe at least some history, you can save on a daily base the log file, with the following shell script logread_bak.sh.

First line in the shell script is #!/bin/sh

# file: logread_bak.sh
# save daily the Linux log (logread) to an external USB-Stick
# "logread" uses a circular RAM buffer
# run the script per (user) cron at 6 o'clock every day
# edit cron parameter:
# $ crontab -e
# m h d m wd
# 0 6 * * * /home/rudi/Install/logread_bak.sh >/dev/null 2>&1

USB_STICK="/media/SAVE/log"
logread > $USB_STICK/logread-$(date '+%Y-%m-%d_%T').log 

List of pages in this category:

-- RudolfReuter 2013-01-30 19:03:41


Go back to CategoryDockStar or FrontPage ; KontaktEmail (ContactEmail)

DockStarBackup (last edited 2015-02-27 20:20:07 by RudolfReuter)