User:Rogerhc/Backup trail

How I backup my wikigogy stuff. Don't copy me; I don't know what I am doing! Instead see Manual:Backup

tar
This grabs everything except the data base, and most importantly it contains /images

tar -cpvf wikigogy.org.tar wikigogy.org/

Copy that tar file to a safe drive at home.

mysqldump
This grabs the database

I use: mysqldump -p roger_wgdb > roger_wgdb.`date +%Y-%m-%d`.sql

Use --skip-opt ? I don't know what that does. Not using it right now.

Then copy ~/BU/wikigogy.____-__-__.sql to another box for safekeeping with scp or, as I do below, with rsync.
 * See http://meta.wikimedia.org/wiki/Database_dump_and_restore_examples
 * man mysqldump
 * I have not investigated mixing in incremental dumps yet but eventually I will need to, to save space.

mysql
To revert to a saved version: mysql -p roger_wgdb < file.sql

cronjob
I have not set this up yet but it could run the mysqldump automatically say once a day.

Safekeeping box side
Do a fresh mysqldump on the server side first, see above.

Then do rsync, below, on the safekeeping box to move that fresh mysqldump to safekeeping.

I initiate rsync from the safekeeping box (at home) instead of from the server (at the data center) simply because my safekeeping box does not have a static ip address and thus I don't know how to access it from the server, but I can access the server from my desktop box easily because the server does have a static ip address.

scp
This on home box, without those spaces in s c p:

s c p roger@wikigogy.org:roger_wgdb.2010-01-12.sql.

or use rsync.

rsync
Run my ~/BU-Dreamhost-rsync.sh

It is a shell script containing the following bash commands (which run rsync on the safekeeping box to copy stuff from the server):

Maybe this?:

rsync -av --delete --stats --progress m2_cn@wikigogy.org:/home/m2_cn /home/roger/BU-Dreamhost-rsync/home..m2_cn/

cp -R /home/roger/BU-Dreamhost-rsync/ /home/roger/BU-Dreamhost-rsync.`date+%Y-%m-%d`


 * The last line cans and dates the stuff so it wont get overwritten next time, in case I need to revert to a previous date to recover from some screw up or vandalism on the server.

Thumb drive
rsync -a --delete --stats --progress /home/roger/a /media/danube

Note re trailing slashes on directories, when rsync'ing directories

'src/dir -> dest' okay.

'src/dir/ -> dest/dir' okay, contents of src/dir/ go into dest/dir. You may like a different dest/dir-name.

'src/dir/ -> dest/dir/' okay, does same as above. ??However, --delete will not remove first level dest/dirs here.??

'src/dir/ -> dest' oops! dumps contents of dir/ into dest/ without dir/! Messy! Okay if you want it.

'src/dir/ -> dest/' oops! same as above.

'src/dir -> dest/dir' oops! duplicates /dir within /dir (ie dest/dir/dir)! Messy! Okay if you want it.

'src/dir -> dest/dir/' oops! same as above.

'''Sashes after dir names? Here's what 'man rsync' USAGE section says:

rsync -avz foo:src/bar /data/tmp This would recursively transfer all files from the directory src/bar  on  the  machine  foo  into the /data/tmp/bar directory on the local machine. The files are  transferred  in  "archive"  mode, which ensures that symbolic links, devices,  attributes, permissions, ownerships, etc. are preserved in the transfer. Additionally, compression will be  used to reduce  the size of data portions of the trans‐ fer. rsync -avz foo:src/bar/ /data/tmp A trailing slash on the source changes this behav‐ ior to avoid creating an additional directory level at the destination. You can think of a trailing / on a  source as meaning "copy the contents of this       directory" as opposed to  "copy  the  directory  by       name", but in both cases the attributes of the con‐ taining directory are transferred to the containing directory on the destination. In other words, each of the following commands copies the files in  the same way, including their setting of the attributes of /dest/foo:

More about rsync and slashes -> http://defindit.com/readme_files/rsync_backup.html

'cp' command however does _not_ seem to have that last feature, the trailing slash on source dir to prevent recreating it in destination dir, per my tests on 2010-10-17

Note group and owner might only be preserved if rsync is run as sudo or super user. See -g and -o below.


 * --stats: This tells rsync to print a verbose set of statistics on the file transfer, allowing you  to  tell  how effective rsync’s delta-transfer algorithm is for your data.


 * --delete: This tells rsync to delete extraneous files from the receiving side (ones that aren’t  on  the  sending side),  but only for the directories that are being synchronized.  You must have asked rsync to send the whole directory (e.g. "dir" or "dir/") without using a  wildcard  for  the  directory’s  contents  (e.g. "dir/*")  since the wildcard is expanded by the shell and rsync thus gets a request to transfer individual files, not the files’ parent directory.  Files that are excluded from the transfer are also excluded from being deleted unless you use the --delete-excluded option or mark the rules as only matching on the sending side (see the include/exclude modifiers in the FILTER RULES section).


 * --progress: This option tells rsync to print information showing the progress of the transfer. This gives  a  bored user something to watch.  Implies --verbose if it wasn’t already specified.


 * -a: Archive mode, equivalent to -rlptgoD


 * -r, --recursive: This tells rsync to copy directories recursively. See also --dirs (-d).


 * -l, --links: When symlinks are encountered, recreate the symlink on the destination.


 * -p, --perms: Preserve permissions


 * -t, --times: Preserve modification times


 * -g, --group: This option causes rsync to set the group of the destination file to be the same as the source file. If the receiving program is not running as the super-user (or if --no-super  was  specified),  only  groups that the invoking user on the receiving side is a member of will be preserved.  Without this option, the group is set to the default group of the invoking user on the receiving side.  The preservation of group information will associate matching names by default, but  may  fall  back  to using the ID number in some circumstances (see also the --numeric-ids option for a full discussion).


 * -o, --owner : This option causes rsync to set the owner of the destination file to be the same as the source file, but only if the receiving rsync is being run as the super-user  (see  also  the  --super  and  --fake-super options).   Without  his option, the owner of new and/or transferred files are set to the invoking user on the receiving side.  The preservation of ownership will associate matching names by default, but may fall back to  using  the ID number in some circumstances (see also the --numeric-ids option for a full discussion).


 * -D: --devices --specials


 * --devices: This option  causes rsync to transfer character and block device files to the remote system to recreate these devices.  This option has no effect if the receiving rsync is not run as the super-user (see  also the --super and --fake-super options).


 * --specials: This option causes rsync to transfer special files such as named sockets and fifos.

cronjob
I have not set this up yet but it could run the rsync automatically say once a day. man cron man cronjob