One solution is to protect each and every document using a password. Many programs have such a capability built in.  For many one, two or three person organisations this solution could work; the people would password protect every file using a phrase that is shared amongst colleagues. As the number of employees increase, guaranteeing that everyone is obeying the rules makes this solution one that is too problematic. Besides certain file types cannot be password protected.

The script I am sharing is one that addresses this problem. It makes use of the commercial product WinRar to archive an entire directory (including subdirectories) into a RAR file. The RAR file name is user definable and is placed in a folder under C:\RSB. The RAR archive is password protected using a password passed to the script. The script is called rsb.cmd.

Custom Search

The example below would archive everything starting from D:\Personal Docs\Articles to an archive called C:\RSB\Documents. The password used to encrypt the archive is 123456.

rsb Documents “D:\Personal Docs\Articles” 123456

If you have another folder you would like to archive, simply call the command above with a different archive name and a different directory. Using a different password is up to you.

Below is the script to perform this task:

@echo off
:: This script archives a directory and all its contents with a
:: password for storage in online backup service. It adds
:: recovery information to the archive thereby increasing the
:: chance of it being opened up if the archive is damaged.
:: This script compresses files thereby reducing the storage
:: requirements as well as upload times.
:: Written by Alan C. Bonnici (email chribonn@gmail.com) 2010/05

set r_Version=1.0

rem This script takes three parameters:
rem  1. The name of the archive
rem  2. The directory (and its sub-directories) that are to be archived
rem  3. The archive password
rem The archive will be placed into a directory called RSB. Your
rem online backup program should backup all files in this
rem directory

rem All 3 parameters are mandatory
if [%1]==[] GOTO :Error
if [%2]==[] GOTO :Error
if [%3]==[] GOTO :Error

set r_Archive=%1
call :DeQuote r_Archive

set r_Dir=%2
call :DeQuote r_Dir

if EXIST C:\RSB\NUL GOTO :DirExists
md C:\RSB

:DirExists

echo The contents of this archive are intended only for the person or entity to whom they belong and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. > "%TEMP%\comment.rsb"

if EXIST "%TEMP%\output.rsb" del /q "%TEMP%\output.rsb" > NUL

"%PROGRAMFILES%\winrar\winrar.exe" u -as -av -cfg- -ow -r -rr10p -inul -ilog"%TEMP%\output.rsb" -t -hp%3 -z"%TEMP%\comment.rsb" -- "C:\RSB\%r_Archive%" "%r_Dir%"
set r_Err=%ERRORLEVEL%
if %r_Err%==0 GOTO :EndCmd

rem An error occurred. Dump the file if it exists
if EXIST "%TEMP%\output.rsb" type "%TEMP%\output.rsb"

echo.

if %r_Err%==1 echo Warning. Non fatal error(s) occurred.
if %r_Err%==1 GOTO :EndCmd

if %r_Err%==2 echo Error. A fatal error occurred.
if %r_Err%==2 GOTO :EndCmd

if %r_Err%==3 echo Error. CRC error occurred when unpacking.
if %r_Err%==3 GOTO :EndCmd

if %r_Err%==4 echo Error. Attempt to modify a locked archive.
if %r_Err%==4 GOTO :EndCmd

if %r_Err%==5 echo Error. Write error.
if %r_Err%==5 GOTO :EndCmd

if %r_Err%==6 echo Error. File open error.
if %r_Err%==6 GOTO :EndCmd

if %r_Err%==7 echo Error. Wrong command line option.
if %r_Err%==7 GOTO :EndCmd

if %r_Err%==8 echo Error. Not enough memory.
if %r_Err%==8 GOTO :EndCmd

if %r_Err%==9 echo Error. File create error.
if %r_Err%==9 GOTO :EndCmd

if %r_Err%==255 echo Error. You aborted the process
if %r_Err%==255 GOTO :EndCmd

rem Undefined error.
echo Error. Undefined error %r_Err%

goto :EndCmd

:EndCmd
rem Clean up
if EXIST "%TEMP%\comment.rsb" del /q "%TEMP%\comment.rsb" > NUL
if EXIST "%TEMP%\output.rsb" del /q "%TEMP%\output.rsb" > NUL
set r_Archive=
set r_Dir=
set r_Err=
set r_Version=

goto :EOF

:Error
echo This script takes three values:
echo    1. The name of the archive
echo    2. The directory (and its sub directories) that are to be archived
echo    3. The archive password
echo The archive will be placed into a directory called RSB (it will be created if it does not exist).
echo Your online backup program should backup all files in this directory.
echo RSB Documents "C:\Users\ACBonnici\Documents" Pa$$w0rd
goto :EOF

   :: Removes the outer set of double quotes from a variable.
   :: Written by Frank P. Westlake, 2001.09.22, 2001.09.24
   :: Modified by Simon Sheppard 2002.06.09

   :: Usage as a function within a script:
   ::   CALL :DeQuote VariableName
   ::
   :: Calling as a function from another batch file:
   ::   CALL DeQuote.cmd VariableName
   ::
   :: If the first and last characters of the variable contents are double
   :: quotes then they will be removed. This function preserves cases such as
   ::   Set Height=5'6" and Set Symbols="!@#
   ::
   :: If a variable is quoted twice and has delimiters then you will
   :: need to run the function twice to remove both sets.
   ::   Set var=""Two Quotes;And,Delimiters=Fails""
   ::
   :: If the variable name itself contains spaces the routine will fail
   :: e.g. %v_my_variable% rather than %my variable%

   :DeQuote
   SET DeQuote.Variable=%1
   CALL Set DeQuote.Contents=%%%DeQuote.Variable%%%
   Echo.%DeQuote.Contents%|FindStr/brv ""^">NUL:&&Goto :EOF
   Echo.%DeQuote.Contents%|FindStr/erv ""^">NUL:&&Goto :EOF

   Set DeQuote.Contents=####%DeQuote.Contents%####
   Set DeQuote.Contents=%DeQuote.Contents:####"=%
   Set DeQuote.Contents=%DeQuote.Contents:"####=%
   Set %DeQuote.Variable%=%DeQuote.Contents%

   Set DeQuote.Variable=
   Set DeQuote.Contents=
   Goto :EOF

If you would like to download this script rather than copy and paste it from this article point your browser to http://www.RemoteStorageBackup.com/downloads/RSBArticleCode.rar. What remains is to set your online backup program to backup everything in the c:\RSB directory. Don’t forget to periodically test that everything is working well.

Next time I’ll delve into the code and explain what it does and how it works. This will allow you to customise it to your needs.

If you have any observations or questions send an email to chribonn@gmail.com.

Last 3 posts by chribonn

• When Green Is Bad - July 5th, 2010
• What your backup solution should be and do - June 27th, 2010
• Backing up, Restoring and Tweaking Sage Pastel Evolution - April 5th, 2010

Sage Pastel Evolution is an Accounting / Stock / Point of Sale program for medium to large sized companies. The program can support a large number of concurrent users and an unlimited number of companies. It uses Microsoft SQL Server as its underlying database. Small setups (up to 10 users) can use Microsoft SQL Server Express with larger shops necessitating the use of the standard edition of SQL Server.

In order to backup Sage Pastel Evolution one needs to backup the Microsoft SQL Server database as well as the appropriate registry entries. In this article I will not discuss how to backup Microsoft SQL Server Express or Standard edition—email me on chribonn@gmail.com if you are interested in a write-up on how to backup these products.

The Windows Registry

A word of warning about handling the registry: unlike many programs, the registry program does not have an undo or an undelete option and changes go into effect immediately. If you delete a file by mistake you can pull it out of limbo by visiting the Recycle Bin. If you screw up a document, pressing undo will take you back one action at a time until you reach the stage before the oops or, if necessary, you can abandon all changes done during that session simply by not saving the updated document. None of this exists in the registry so be cautious. 

Custom Search

How Sage Pastel Evolution uses the Windows registry

The Sage Pastel Evolution program is a thick client that makes use of windows registry to store customisations each user makes to the program. Module settings, what information is shown, how it is formatted and the order of columns are all examples of the type of information that one can find in the registry. This is the approach Sage used to give each user his own personalised environment. If a single Windows login is used by multiple sage users, each user can have his own tweaks.

Storing the settings of each user within a computer’s registry rather than as a record within the Microsoft SQL database is a debatable design decision especially since database technology constitutes a core component of this product. My reasoning is that if a user wants column A to come before column B and column C being hidden, he would want that on each and every computer he uses. A person who uses more than one computer must set the customisations on each computer. When a new account is created, there is no option to copy settings from an existing user to the new one. And, if anyone of the client computers fails, the settings have to be reprogrammed. This article will explain how to tackle each one of these scenarios.

Backing up the Registry

Sage keeps all program settings in the registry hive HKCU\Software\Softline. Backing up this registry key ensures that all customisations are available if the computer fails.

1. Run the program regedit either from the command prompt, by clicking on the Run option found in the Start Menu (this option is not visible by default in Windows 7) or by typing in the program name in the Search Program Files and options box that is situated at the bottom on the Menu that comes up when you click the Start Menu button (this is not available in Windows XP).
2. Navigate to the HKCU\Software\Softline hive. Select the option File -> Export…from regedit (1).
3. A dialog box comes up. Select the folder and specify the file name you want to call this registry entry (2). As shown in the diagram above, the selected radio button of the group Export rangeshould refer to the Softline registry branch (3).
4. Backup this file and update it whenever settings change.

You have to repeat the process above for each Sage Pastel Evolution user who logs into the computer.

Restoring the registry

You may need to restore the registry file you backed up either because a computer failed and had to be reinitialised or to transfer settings from one computer to another.

Because this program retains information in the HKCU part of the registry, the user must be logged into the computer before his registry settings are imported.

After the user has logged in, execute the program regedit.exe. This time round choose the option Import… from the File menu (1). Navigate to the folder where you had previously placed your backup and open the file (2). A final dialog box will confirm that the registry was updated with the contents of that file.

Changing Agents

This section is highly technical and should only be performed by people who understand and are comfortable tweaking the Windows registry. An error could, at the very least, end up making this program stop functioning.

Each Sage Pastel Evolution user is known as an agent. A user who logs into this program using the user name Chris would have an entry in the registry starting at HKCU\Software\Softline\Evolution\Agents\Chris. Assume that you would like to copy Chris’s agent setting to those belonging to Mary so that Mary does not have to make the changes already done by Chris. Use the following steps:

1. Ask Chris to log into his computer and using regedit export the registry hive HKCU\Software\Softline\Evolution\Agents\Chris.
2. Open the exported registry file within a text editor such as notepad and change all instances of [HKEY_CURRENT_USER\Software\Softline\Evolution\Agents\Chris to [HKEY_CURRENT_USER\Software\Softline\Evolution\Agents\Mary. Save the modified registry file.
3. Ask Mary to log into Windows with her credentials and install the program on her PC if it is not already functional.
4. Test that she can log into the program.
5. Import the modified registry file created in point (2) into Mary’s Sage.

Conclusion

If you have any comments or observations or would like to request or share information on how to backup a particular program, send me an email on chribonn@gmail.com.

Last 3 posts by chribonn

• When Green Is Bad - July 5th, 2010
• What your backup solution should be and do - June 27th, 2010
• Securing your online backup archives - May 12th, 2010

In my entry titled “How to backup your Microsoft Office 2007 Outlook in its entirety (Windows XP, Windows Vista and Windows 7)” I described how to backup the Microsoft Outlook 2007 data file together with that part of the Windows registry that stores within it the email accounts from where Outlook retrieves emails or via which emails are sent out. Today I discuss how to use that backed up information to either recover a failed computer or to transfer the settings from one computer to another. The former may happen if the hard disk fails or as a result of a malware attack while the latter would apply if you are upgrading your computer or operating system.

On your freshly formatted computer you need to install Microsoft Outlook. Accept all the defaults. After the installer completes, run Microsoft Outlook for the first time. You will be asked a number of questions. Enter anything that will allow you to proceed to the end of this process. Do not enter valid email account details because Outlook will attempt to download mail after you complete the process and since this is not the database you will be using, you do not want any messages to end up here (this can be fixed but why create a problem that will need fixing).

1. Restore the files you had previously backed up to their original location. If you followed my previous article, the location would be %USERPROFILE%\Documents if you’re using Windows 7 and %USERPROFILE%\My Documents for Windows XP.

If you were using an online backup solution, you could have had a process that backed up this directory multiple times a day thereby guaranteeing that not even one email was lost when disaster struck.

2. Delete the files created by Microsoft Outlook when you first ran it. As we did when backing up, we need to find out the default location where Microsoft Outlook places the pst database. The same steps you performed during the backup stage must be repeated:

1. With Outlook running, right click on the Personal Folders entry (1) and choose the option Properties for ‘Personal Folders’ from the short cut menu that comes up.
2. A dialog box titled Outlook Today – (Personal Folders) Properties comes up. Click on the Advanced… button (2).
3. This brings up yet another dialog box titled Personal Folders. The Filename: entry (3) depicts the location of where your Outlook data file currently resides.
4. Copy the directory location from the Filename: entry and paste it into Windows Explorer.
5. Click OK on all the dialog boxes you have just opened and shut down Outlook.
6. Delete the contents of this folder.

 3. Click on the Microsoft Outlook 2007 icon to run the program. Since the pst data file is no longer there, Outlook prompts you that it cannot find this file and asks that you select the location where this file can be found. Move to the directory you have just restored and select the appropriate database. Outlook will memorize the new location.

A  message box comes up to tell you that Outlook must be restarted. Press the OK button.

4. Run Outlook again and confirm that all your emails, contacts and calendar entries are present. Although everything should be there, the email account settings are missing. This is because these are not stored within the pst file but are held in the registry.

If you did not follow the article about how to backup this information, you will have to manually setup each email account. Search your mail provider’s website for information about how to go about it. If, on the other hand, you retained a backup of this chunk of the registry then read on.

 5. Close Microsoft Outlook.

 6. Restore the Registry entries by double clicking on the registry file found in the directory you restored earlier. You will be asked to confirm the operation (1). You need to choose the Yes button. A warning comes up (2) to tell you that mucking up with the registry is dangerous. This is because one can easily render a computer an electronic vegetable if one deletes, adds or modifies incorrect entries to the registry—check out the backup part of this work for more about the Windows registry. Press the Yes button for this dialog box. A third dialog box (3) will tell you that the entries have been successfully added.

7. The next time you run Outlook you will get a prompt for the password associated with each email account managed by this program. Type in the password. If you do not wish to have to enter the password every time you run Outlook, click the appropriate checkbox to save the password.

8. Confirm that everything is working as it did before.

End of part 2 – Conclusion

You do not appreciate something until that something is no longer there. Emails, contacts and appointments are three of the most important things in the electronic world we live in. A backup solution that is easy to setup and forget about while being equally easy to restore when the need arises is the ideal candidate to ensure that you do not ever lose one bit of data.

Your comments and opinions are welcome. Is there a program you would like to backup but do not know how to go about it? Did I miss something out in my article on how to backup and restore of Microsoft Outlook 2007? Leave a comment or email me on chribonn@gmail.com.

Last 3 posts by chribonn

• When Green Is Bad - July 5th, 2010
• What your backup solution should be and do - June 27th, 2010
• Securing your online backup archives - May 12th, 2010

Backing up the data file

The folk who decided on the default location of the Outlook data file chose a location that is difficult to figure out.  Also when installing Outlook (or when running it for the first time) there is no way to specify a different folder.

Changing the default location

The first thing I am going to demonstrate in this tutorial is how to move your existing Outlook files to a more humane location under the Windows 7 Documents (or Windows XP My Documents) folders.

Open Outlook which I am assuming you are already running.

1. Right click on the Personal Folders entry (1) and choose the option Properties for ‘Personal Folders’ from the short cut menu that comes up. If your folder has a different name, Personal Folders will be called differently.
2. A dialog box titled Outlook Today – (Personal Folders) Properties comes up. Click on the Advanced… button (2).
3. This brings up yet another dialog box titled Personal Folders. The Filename: entry (3) depicts the location of where your Outlook data file currently resides.
4. Copy the directory location from the Filename: entry and paste it into a Windows Explorer window.
5. Click OK on all the dialog boxes you have just opened and shut down Outlook.
6. Move the folder that contains the Outlook files to a more appropriate location. I would suggest under %USERPROFILE%\Documents if you’re using Windows 7 and %USERPROFILE%\My Documents if you’re still happily using Windows XP.

Click on the Microsoft Outlook 2007 icon to run the program. Since you have moved the original location of the data file, you will get a message box telling you that Outlook can no longer find the pst file in its original location (1).

Clicking OK on the dialog box brings up a file selection screen that allows you to specify the new location where you had shifted your Outlook data files. Move to the new location, click on the pst file and hit the Open Button (2).

Another message box comes up to tell you that Outlook must be restarted. OK that.

The second time onwards Microsoft Outlook 2007 will automatically find your files no questions asked. And the directory where all your important emails reside is now in an easy to find location in your Documents folder together with all your other important stuff.

Backing up the registry

Microsoft Outlook retains your email account settings within the registry. This means that when the time comes to reinstall your computer, unless you have saved your registry Outlook settings you will have to re-enter all the email accounts you process using Outlook 2007. If you happen to have a single Gmail account this might not be that much of a problem (provided you still remember how to do it) but if, like me, you have almost a dozen email accounts flowing into your Outlook remembering each mailbox’s parameters can be time consuming.

A word of warning about handling the registry: unlike many programs, the registry program does not have an undo or an undelete option and changes go into effect immediately. If you delete a file by mistake you can pull it out of limbo by visiting the Recycle Bin. If you screw up a document, pressing undo will take you back one action at a time until you reach the stage before the oops or, if necessary, you can abandon all changes done during that session simply by not saving the updated document. None of this exists in the registry so be cautious.

In Windows 7, type regedit in the Search Programs and Files to run the registry program. With Windows XP, type the same command in the Run… box. Both options can be accessed from the Start Menu. Windows 7 will bring up a dialog box asking you to confirm your action. You should say yes in order to run the program.

Navigate to the registry key HKEY_CURRENT_USER\Software\Microsoft\Windows NT\CurrentVersion\Windows Messaging Subsystem\Profiles (1). Each profile you have defined will appear under this key. In my case there is one profile called Outlook.

Right click on Outlook and choose the option Export from the menu that comes up. I suggest that you place the registry snippet you are saving in the same location as the Outlook data files. This ensures that all your Microsoft Outlook related data is in one easy-to-backup folder.

End of part 1 – Conclusion

By backing up the folder to an online backup service you can rest assured that whenever something happens to your computer all your emails and the mailbox settings can be retrieved with ease. If you add a new email account or remove one you no longer use, you should export the registry settings again.

When backing up your Outlook data files, Outlook should be shut down.

Although I have not mentioned Windows Vista, the procedure for Windows 7 is almost identical to that of Windows Vista.

Next time I will describe how to use the information you have saved to restore your Microsoft Outlook 2007 in case of corruption. I’ll also throw in how you can use this backed up data when upgrading your computer from Windows XP to Windows 7.

We all appreciate how important emails are to us. Back them up regularly.

Last 3 posts by chribonn

• When Green Is Bad - July 5th, 2010
• What your backup solution should be and do - June 27th, 2010
• Securing your online backup archives - May 12th, 2010

Addendum

RSync: The first time a file is transfered

RSync: What happens when the source file changes.

In this environment, when one discusses online backups, the critical issue is to try to limit as much as possible the amount of data that has to be uploaded from the person’s computer (the source) to that person’s archive vault (the destination).   Many people would consider online backups only if the lengthy wait periods are few and far between.

On a regularly backed up system, the state of the source compared to the destination can be as follows:

• New file on source that does not exist on the destination – the file must be copied over to the archive vault;
• File on source that no longer exists on the destination – the file must be removed from the archive vault;
• File on source and destination are identical – no need to do anything;
• File on source is different from that on destination.

On a system that is regularly backed up, one would normally find few files that are new or that have been deleted. The majority of files would either have not changed or would have been altered.  As I will demonstrate shortly, the Rsync algorithm for transferring data from source to destination is greatly suited for situations in which a file has been adjusted.  There are two ways a system can handle altered files, resending the adjusted files in its entirety or simply transferring the changed pieces. The Rsync algorithm does the latter.

The Rsync algorithm was developed by Andrew Tridgell and Paul Mackerras. Taking a data backup using the Rsync method results in ultra fast and efficient backups. Imagine a database, worksheet or word processing document in which the author only changed one record, cell or paragraph.  Take your PIM database; on a daily basis you receive new emails, delete junk and old mail, setup appointments and have the system remove expired ones.  Rsync backups will transmit a few megabytes of changes rather than let you wait until all modified files (that probably run into gigabytes) are uploaded.

The Rsync algorithm

The file on the archive server (the destination file that needs to be updated) is split into a number of blocks of equal size (the last block could be an exception). For each block a signature is generated.

The signature consists of a quick-to-compute 32 bit checksum (I’ll explain the rolling capability later on) as well as a 128 bit checksum. In Rsync, the MD5 (Message digest) algorithm is now being utilised.

The block number as well as the associated signatures for each block is transmitted to the source computer. The source computer takes the 32 bit low computational checksum of each block and generates a simple 16 bit hash value.

Simplistically speaking, the source end will transmit to the destination how to reconstruct the file using either literal data transmitted from the source to the destination or by telling the destination to utilise a block already present at its end.

At the source, a block of identical size as those computed at the destination is analysed and the low overhead 32 bit checksum is generated. Its 16 bit hash is compared to those computed from the destination 32 bit checksums. If no match is found than the code that deals with a nonexistent block kicks in.

If the hashes match, the 128 bit checksum is computed on the source block and this is compared to the 128 bit blocks received from the destination that have the same hashing function (there may be more than one) as the source. If no match is found, the code that deals with a nonexistent block kicks in.

If a destination block is matched, the source sends an instruction to the destination to copy the block having a particular index to the new output file. The position at the source is advanced forward the length of the block and the process loops.

The logic behind the nonexistent block is as follows:

1. It transmits the character at the beginning of the block to the destination to append to the file that is being reconstructed;
2. Advances the block by one character and repeats the process.

Point 2 explains where the rolling 32 bit algorithm comes into play. By having a rolling 32 bit algorithm, the computational overhead necessary to calculate the new CRC is minimised further since all that is necessary would be to subtract the value of the byte of the previous start of block and add the value of the byte at the new (shifted) end of block. The alternative would have to sum up the range again and this consumes more computing cycles.

The performance increase between a copy-all backup and one using Rsync can be more than 110 times. All data transmitted by Rsync is actually compressed and is encrypted using SSL. For online backups the Rsync method is the best way to guarantee that backups are fast, secure, and non-intrusive and actually get done.

In a future article I’ll give some practical examples to explain how Rsync deals with different file situations, namely adding new data at various positions within a file, deleting data from different positions within a file and altering data. For more in depth information on Rsync, you might want to visit the Rsync home page at http://Rsync.samba.org/ as well as spend a couple of evenings reading through Andrew Tridgell’s paper Efficient Algorithms for Sorting and Synchronization.