Personal Backup Scripting Example

I’m using Personal Backup by Dr. Rathlev as my backup tool. It’s name is a bit misleading, because you could think that it is only allowed to use it as a private person. But no – its current license says it may be used also by any club, organization or even private companys for free.

It is easy to create backup tasks with Personal Backup. But a backup task does not run automatically. And I have to say, it is not easy to create backups that run automatically with this program. The UI of the part of the program which deals with creating automatically running backup tasks is near incomprehensible – even when you read the docs.

What compensates this deficit is that Personal Backup has a perfect command line interface with a good docmentation. So it is very well suited for being scripted.

I control my backups with some tclkit scripts. As you might know, I do use tclkit for many tasks.

The script below creates a full backup in every uneven month (Jan, Mar, May, …) and in between it creates an incremental backup daily. It needs a fitting Personal Backup task file Backup-Task-All.buj.

To trigger the script daily, I use the Windows Task Planner.

The directory structure of the created backups looks like this:

G:/Backup/2014/BD01F   # A full backup created on March 1st in 2014
G:/Backup/2014/BD22I   # An incremental backup created on March 22nd
.....
G:/Backup/2014/BD21I   # An incremental backup from September 21st 
#
#  This script steers the backups that I do with Pesonal Backup.
#
# To adapt the script, just set the following variables taskName,
# rootdir and personalBackupExe to your liking. 

# The name of the backup task in PB
set taskName Backup-Task-All

# The name of the target root directory.  This must be the same as 
# in the Backup-Task mentioned above.
set rootdir G:/Backup

# Full path to the Personal Backup executable
set personalBackupExe "C:/Program Files/Personal Backup 5/Persbackup.exe"

#console show

puts "BackupSteer.tcl  (c) A.J.W 2014"

set rootdrive [string tolower [string range $rootdir 0 0]]
set seconds [clock seconds]
set yearMonth [clock format $seconds -format {%Y/%m}]
# the name of the basedir of the backup of today. E.g  G:/Backup/2014/09
set basedir [file join $rootdir $yearMonth]

puts "    basedir=$basedir"
puts "    taskName=$taskName"

# Return 1 if it's an even month, 0 if not.  
proc isEvenMonth { seconds } {
    set month [clock format $seconds -format {%m}]
    if { [string range $month 0 0] == 0 } {
        set month [string range $month 1 1]
    }
    set even [expr $month % 2 == 0] 
    return $even 
}

# Return a list of all connected drives. 
proc drives {} {
    foreach drive [list a b c d e f g h i j k l m n o p q r s t u v x y z] {
        if {[catch {file stat ${drive}: dummy}] == 0} {
            lappend drives $drive
        }
    }
    return $drives
}

proc doBackup {} {
    global  taskName  seconds yearMonth basedir  personalBackupExe 

    set relativeDirForFullBackup BD01F
    set pathForFullBackup [file join $basedir $relativeDirForFullBackup]
    puts "    pathForFullBackup=$pathForFullBackup"

    # if full backup for this month already exists or we have an even month
    if { [file isdirectory $pathForFullBackup] || [isEvenMonth $seconds] } {
        # only do an incremental backup
        set mode incr
        set day [clock format $seconds -format {%d}]
        set relDir BD$day
        append relDir I
        set pathForIncrBackup [file join $basedir $relDir]
        puts "    pathForIncrBackup=$pathForIncrBackup"

        if [file isdirectory $pathForIncrBackup] {
            set mode none
        }

    } else {
        # do a full  backup
        set mode full
        set relDir $relativeDirForFullBackup
    }

    puts "    mode=$mode"
    puts "    relDir=$relDir"

    if { $mode != "none" } {
        puts "    Running backup like this:"
        puts "    $personalBackupExe $taskName /force /hide /mode:$mode /prompt:delay  /backupdir:$relDir & "
        # If I do not add the & at the end, Persbackup hangs. 
        exec $personalBackupExe $taskName /force /hide /mode:$mode /prompt:delay  /backupdir:$relDir &
        puts "     Backup running." 
    } else {
        puts "    Nothing to do."
    }
}

if { [lsearch [drives] $rootdrive] == -1 } {
    bell; bell; bell
    set a "The daily backup is about to be done. "
    append a "Please connect the backup drive, make it "
    append a  "available as network drive $rootdrive: and click Ok."

    message .m  -textvariable a -width 250
    # strange, the width of the button is in another dimension than that of the message. 
    button .hello  -text "Ok" -command { 
            if { [lsearch [drives] $rootdrive] != -1 } { doBackup; exit } else { bell }
        } -default active -width 15

    bind .  {.hello invoke}

    pack  .m .hello -padx 5 -pady 5

    wm deiconify .
} else {
    doBackup
    exit
}

How to Change the Encoding of Filenames, Directory Names and File Content From ISO 8859-1 to UTF-8

8859-1 Encoded Filenames, Directory Names And Content

I have got a lot of Perl-scripts, and text data files on an older Linux system. All this stuff together makes a management system which I still need. Now I want to bring this management system to a current Linux system, because I suspect that the old one maybe will stop to work some day.

The oldish Linux system uses 8859-1 as encoding for filenames, directory names and content of text files.

The current Linux uses UTF-8 as encoding for filenames, directory names and content of text files.

Of course, as the system is in German, many files, filenames and directory names contain Umlauts like ÄÖÜäöü.

To work with the system on the current Linux, all must be converted from ISO 8859-1 encoding into UTF-8 encoding, including file- and directory names.

Side Note: Bash Glob Problem

Otherwise, such simple things as a bash glob do not work any more. Example: I have two files Maus and Möhre on the old 8859-1 system in a directory.

Now I bring these files to the new system and try to list them with ls.

> ls
Maus M?hre
>
> ls M*
Maus

Unbelievable. But true.

Use Iconv to Change Encoding

The iconv tool is available on most Linux systems, including my new one. With iconv, you can convert text from any encoding known in the world to any other.

As I have to change the encoding not only of the file contents but also of the filenames and directories, I have written a bash script to do all the transformation.

The script creates a backup .bak.8859-1 of every file it transcodes and it creates an empty .eonc file (eonc for encoding of name changed) for every file or directory as a marker that its name has been changed. We need to set markers for the already transcoded names, otherwise a second run of the script would damage things.

The script transcodes all filenames, directory names, file content in and below the current directory from ISO 8859-1 to UTF-8. Don’t run it in / 😉

#! /bin/bash

eoncExtension=.eonc

function iconvContent {
  local tmpf=tmp.`basename $1`
  if [[ -s $1 && ! -e $1.bak.8859-1 ]] ; then 
    echo "iconvContent $PWD $1"
    cp -p $1 $1.bak.8859-1
    iconv -f ISO_8859-1 -t UTF8 $1 -o $tmpf

    # Iconv does no preserve file attributes. 
    # chmod --reference  ... does not work on my Linux version. 
    cat $tmpf > $1      # this preserves the file attributes of $1
    rm $tmpf
  fi 
}

# Change encoding of a file- or directory name.
# But only if the encoding has not already been changed.     
function iconvName {
  if [[ -e $1 && ! -f $1$eoncExtension ]] ; then 
    local target=$(echo $1 |  iconv -f ISO_8859-1 -t UTF8)
    if [[ $1 != $target ]] ; then
      echo "iconvName $PWD $1  --> $target"
      mv $1 $target

      # Create a .eonc file as a marker that the encoding is 
      # already changed.
      touch $target$eoncExtension      
    fi
  fi
}              

# Check if the file is not one of those that I've created myself 
# as backup or marker or temp file.
function isSelfCreatedFile {
  local bak='.*\.bak.*'
  local eonc='.*\.eonc'
  local tmpre=".*tmp\..*"
  local f=$1
  local self=1
  if [[ $f =~ $bak || $f =~ $eonc || $f =~ $tmpre ]] ; then   
    self=0
  fi
  return $self
}

function iconvNamesInDir {
  for f in `find . -maxdepth 1`  ; do
    if [ -e $f ] ; then 
      if ! isSelfCreatedFile $f ; then   
        iconvName $f
      fi
    fi
  done
}

function iconvContentInDir {
  for f in `find . -maxdepth 1 -type f`  ; do
    if [  -s $f ] ; then   
      if ! isSelfCreatedFile $f ; then 
        iconvContent $f
      fi
    fi
  done
}

function iconvCurrentDir {
  echo "iconvCurrentDir  $PWD"

  iconvNamesInDir
  iconvContentInDir  

  for f in `find . -maxdepth 1 -type d`  ; do
    if [[ ! $f == . ]] ; then 
      local dir=$PWD
      cd $f
      iconvCurrentDir
      cd $dir
    fi
  done
}

if [ "$1" == "run" ] ;  then 
  iconvCurrentDir
else 
  echo "USAGE: iconv-multi run"
  echo "  This script changes the encoding of all filenames, "
  echo "  directory names and all file content in and below the "
  echo "  current directory from ISO 8859-1 to UTF-8."
  echo "  Running it twice does not hurt because the script "
  echo "  creates backup and marker files to detect which "
  echo "  files already have been encoded."
  echo "  (c) 2014 by Andreas Wicker."
fi