Automated backup of an external drive on Linux

Posted on by austinmarton

The following steps set up a backup for a particular USB drive every time it is inserted. Very useful for avoiding the perils of working directly from a flash drive on multiple computers.
 

Create the backup script that uses duplicity:

/usr/local/bin/usb_drive_backup.sh

#!/bin/bash

DRIVE_NAME=USBDRIVE
BACKUP_DIR=/home/username/BACKUPS/USBDriveBackup

sleep 4

duplicity incremental /media/username/USBDRIVE file://$BACKUP_DIR --no-encryption >> $BACKUP_DIR/log.txt
if [ $? -eq 0 ]; then
    echo "## Backup complete `date`" >> $BACKUP_DIR/log.txt
    echo
fi

/usr/local/bin/usb-backup-in_udev

#!/bin/bash

/usr/local/bin/usb_drive_backup.sh &

 
 
Set up the udev rule to run the backup script every time the drive is inserted:

Find the device ID

$ lsusb
Bus 002 Device 002: ID 0930:1400 Toshiba Corp. Memory Stick 2GB
...

/etc/udev/rules.d/00-usb-backup.rules

ACTION=="add", ATTRS{idVendor}=="0930", ATTRS{idProduct}=="1400", ENV{XAUTHORITY}="/home/username/.Xauthority", ENV{DISPLAY}=":0", OWNER="username", RUN+="/usr/local/bin/usb-backup-in_udev"

 
 
Restoring the backup:

duplicity restore file:///home/username/BACKUPS/USBDriveBackup ./RestoredFiles/ --no-encryption

Corrupt USB Flash Drive Data Recovery on Mac OS X Mavericks

Posted on by austinmarton

An oldy but a goody…

  • Identify the device node
    Remove the drive from the computer then run
    $ ls /dev/rdisk*
    Insert the drive and re-run the command, for me a new device /dev/rdisk3 shows up
  • Checkout what might be wrong
    $ sudo dmesg | tail
    ...
    disk3s1: I/O error.
    0 [Level 3] [Facility com.apple.system.fs] [ErrType IO] [ErrNo 5] [IOType Write] [PBlkNum 14988] [LBlkNum 0]
    0 [Level 3] [Facility com.apple.system.fs] [DevNode /dev/disk3s1] [MountPt /Volumes/Lexar]
  • Make an image of the disk
    $ dd bs=64k if=/dev/rdisk3 of=./recovered.dmg conv=noerror,sync
    478816+0 records in
    478816+0 records out
    31379685376 bytes transferred in 1278.080407 secs (24552200 bytes/sec)
  • Try mount the created image in Finder (unlikely)
  • Install foremost for data carving
    $ brew install foremost
    ==> Downloading http://foremost.sourceforge.net/pkg/foremost-1.5.7.tar.gz
    ######################################################################## 100.0%
    ==> make mac
    🍺 /usr/local/Cellar/foremost/1.5.7: 5 files, 84K, built in 29 seconds
  • Run foremost on the image
    $ foremost -i ./recovered.dmg -o ./recovered
  • Browse the glorious recovered data!

Using git as a subversion interface – git-svn

Posted on by austinmarton

Git is my favourite source control system but unfortunately some companies still use historic subversion repos that we have to put up with. This is my basic workflow when dealing with svn.

Importing the repo

Cloning the svn repo to git:
git svn clone https://address/to/svn/repo

For a very large repo this periodically failed with:
error: git-svn died of signal 13
To continue from the same spot just run:
git svn fetch

This happened quite a few times when cloning from a slow server that took all day.

Note that git-svn doesn’t recursively clone svn externals (nested repositories). There are a few complex solutions on stackoverflow, haven’t picked my favourite yet.

Pulling changes

Instead of a git pull I do:

git stash; git svn fetch; git svn rebase; git stash pop

Commiting changes

For simple projects I think a major advantage of git is the ability to easily queue commits from the command line. I use my normal process of the interactive prompt to add files and a commit message:

git add -i
git commit -m "Message"

Then instead of a git push I do:

git stash; git svn dcommit; git svn rebase; git stash pop

Bash – Loops using pipe and read

Posted on by austinmarton

Perform some operation for every line in the output of a command, e.g. add .txt to every file in a directory:

ls /var/* | while read fname; do cp $fname $fname.txt; done

Instead of:

for fname in `ls /var/*`; do cp $fname $fname.txt; done

Neat trick I actually picked up in a job interview. Easier to read if the original command is long as it avoids the backticks, and bash is all about readability, right?? 🙂

Bash String Manipulation

Posted on by austinmarton

Bash has some convenient string manipulation functions. I often use substring removal which is very handy for trimming suffixes and prefixes from things like file names. These operations all evaluate to a new string and don’t modify the original variable.

Suffix removal
Remove the longest match from end:
    output=${string%%suffix}
Remove the shortest match from end:
    output=${string%suffix}

Prefix removal
Remove the longest match from front:
    output=${string##prefix}
Remove the shortest match from front:
    output=${string#prefix}

Great reference for more string operations: http://tldp.org/LDP/abs/html/string-manipulation.html

Mozilla Game On: Hackable Javascript + HTML5 Game

Posted on by austinmarton

I have submitted my entry to the Mozilla Game On competition under the ‘hackable games’ category. It can be played online now at: http://austinmarton.github.com/phys-ed/

PhysEd

I developed a basic platformer written from scratch in Javascript using HTML5 canvas and some jQuery. It has a ‘mod mode’ in which the user can view some of what is going on behind the scenes and change some of the physical parameters. Of course I ran out of time in the end but the competition was a great push to at least get a playable prototype out there.

It is released under GPLv3 so please check out or even fork the source code on github!

VirtualBox driver (vboxdrv) missing after Ubuntu 12.04 upgrade

Posted on by austinmarton

After upgrading to 12.04 I got the following error when trying to start VirtualBox:

The VirtualBox Linux kernel driver (vboxdrv) is either not loaded or there is a permission problem with /dev/vboxdrv. Please reinstall the kernel module by executing

‘/etc/init.d/vboxdrv setup’

as root. If it is available in your distribution, you should install the DKMS package first. This package keeps track of Linux kernel changes and recompiles the vboxdrv kernel module if necessary.

The script mentioned did not exist because the vboxdrv was not compiled.
# sudo /etc/init.d/vboxdrv setup
sudo: /etc/init.d/vboxdrv: command not found

If you reinstall virtualbox-dkms it will recompile the driver for your kernel version, provided you have the correct kernel headers.
# sudo apt-get remove dkms virtualbox-dkms
# uname -a
Linux pc786-ubu 3.2.0-33-generic #52-Ubuntu SMP Thu Oct 18 16:29:15 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
# sudo apt-get install linux-headers-3.2.0-33
# sudo apt-get install dkms virtualbox-dkms