Low cost Data Protection concepts for Virtualized environments

February 7th, 2010

Given the widespread success of virtualization, many SMBs rely exclusively on the inbuilt features as their main data protection plan and which may appear adequate at face value! One would need to see the overall picture! Virtual server based tools such as, snapshots and export/import utilities that are found in the major products, help organizations protect their data to a certain degree! It is not enough to backup data, being a clone or a file to the same virtual server. In fact, major manufacturers and their fellow partners provide a vast range of backup solutions that fulfill all requirements of a proper backup policy. With SMBs, the deciding factor may be costs! Do they have the budget to purchase such applications? However, for the unlucky IT techies that never manage to get their superiors buying these expensive products can still develop a similar mechanism. Using some basic scripting and inexpensive hardware they can build backup solutions similar to these expensive applications as they too, make use of the inbuilt or native tools found in the major virtualization products.

Basic setup  Basic setup

Virtual machines (VMs) are copied (cloned) to a remote computer (storage location) through the export functionality of the virtual server over a Gbit network connection. This can be performed overnight on a weekly or daily basis (depending on the function of your data). VMs can either be left in their exported format or successively imported locally to the computer as to verify their integrity. Additionally, the remote computer can be mounted as a local storage to the virtual server using the inbuilt NFS or iSCSI functionality. All you need is a good reference of the scripting commands available for your environment and the virtual server admin client application. The main cost of this setup is a computer with adequate storage space; say, two or three 1 TB SATA hard drives in a RAID configuration.

Advanced setup

Advanced setup

This setup can perform the same tasks as the basic one plus more! Exported VMs to the central location can be successively imported to a second virtual server. This requires that the second server has sufficient free space to host the imported VMs (clones). In the event of a failure on server 1, the recovery time of any exported-imported VM is minimal as you only need to start the imported clone on server 2. The only additional task required would be, the restore operation of any data (at file level) that has changed since the last clone was done. For file data backups one would use the native backup utilities found in the major Operating Systems such as, NT backup for MS windows and Tar for Linux, running with scheduled jobs. Also, a faster alternate solution would be to connect both virtual servers as repositories of each other through NFS or iSCSI.

The suggested setups have some limitations and these are:

  • Backups are done at an image level (clones) and not at file or folder level but this can be overcome through the free reliable solutions incorporated in the main operating systems.
  • You need to shutdown the VM during an export procedure and hence, the backup process should happen after office hours but this would not be an option if the resource is required to be running 24/7. One would need to explore the snapshot functionality to overcome this setback.

Conclusion: – With what I call a high spec PC and a fast local network (Gbit NICs & switches – quite standard nowadays), the suggestions presented above can offer SMBs a backup and recovery solution without spending thousands of dollars. The same concept can applied between remote sites over a good Internet connection to provide a disaster recovery site. In the articles to come, I will be explaining some basic scripts that perform similar tasks in a typical virtual server environment.

Last 3 posts by George

How to backup your Microsoft Office 2007 Outlook in its entirety

January 27th, 2010

Many people use Microsoft Outlook to download mail from their POP3 or IMAP server. While some may only link their Outlook to one particular email, many today have multiple email account all pouring into the same data file. Backing up Microsoft Outlook consists essentially of backing up two different components; the pst file that stores messages (or in the case of IMAP a cached version of the server) together with the registry entry that stores all the different email accounts that are being processed by Outlook.

Backing up the data file

The folk who decided on the default location of the Outlook data file chose a location that is difficult to figure out.  Also when installing Outlook (or when running it for the first time) there is no way to specify a different folder.

Changing the default location

The first thing I am going to demonstrate in this tutorial is how to move your existing Outlook files to a more humane location under the Windows 7 Documents (or Windows XP My Documents) folders.

BackupOff2007-1

Open Outlook which I am assuming you are already running.

  1. Right click on the Personal Folders entry (1) and choose the option Properties for ‘Personal Folders’ from the short cut menu that comes up. If your folder has a different name, Personal Folders will be called differently.
  2. A dialog box titled Outlook Today – (Personal Folders) Properties comes up. Click on the Advanced… button (2).
  3. This brings up yet another dialog box titled Personal Folders. The Filename: entry (3) depicts the location of where your Outlook data file currently resides.
  4. Copy the directory location from the Filename: entry and paste it into a Windows Explorer.
  5. Click OK on all the dialog boxes you have just opened and shut down Outlook.
  6. BackupOff2007-2Move the folder that contains the Outlook files to a more appropriate location. I would suggest under %USERPROFILE%\Documents if you’re using Windows 7 and %USERPROFILE%\My Documents if you’re still happily using Windows XP.

BackupOff2007-3

Click on the Microsoft Outlook 2007 icon to run the program. Since you have moved the original location of the data file, you will get a message box telling you that Outlook can no longer find the pst file in its original location (1).

Clicking OK on the dialog box brings up a file selection screen that allows you to specify the new location where you had shifted your Outlook data files. Move to the new location, click on the pst file and hit the Open Button (2).

Another message box comes up to tell you that Outlook must be restarted. OK that.

The second time onwards Microsoft Outlook 2007 will automatically find your files no questions asked. And the directory where with all your important emails is now in an easy to find location in your Documents folder together with all your other important stuff.

Backing up the registry

Microsoft Outlook retains your email account settings within the registry. This means that when the time comes to reinstall your computer, unless you have saved your registry Outlook settings you will have to re-enter all the email accounts you process using Outlook 2007. If you happen to have a single Gmail account this might not be that much of a problem (provided you still remember how to do it) but if, like me, you have almost a dozen email accounts flowing into your Outlook remembering each mailbox’s parameters can be time consuming.

A word of warning about handling the registry: unlike many programs, the registry program does not have an undo or an undelete option and changes go into effect immediately. If you delete a file by mistake you can pull it out of limbo by visiting the Recycle Bin. If you screw up a document, pressing undo will take you back one action at a time until you reach the stage before the oops or, if necessary, you can abandon all changes done during that session simply by not saving the updated document. None of this exists in the registry so be cautious.

In Windows 7, type regedit in the Search Programs and Files to run the registry program. With Windows XP, type the same command in the Run… box. Both options can be accessed from the Start Menu. Windows 7 will bring up a dialog box asking you to confirm your action. You should say yes in order to run the program.

BackupOff2007-4

Navigate to the registry key HKEY_CURRENT_USER\Software\Microsoft\Windows NT\CurrentVersion\Windows Messaging Subsystem\Profiles (1). Each profile you have defined will appear under this key. In my case there is one profile called Outlook.

Right click on Outlook and choose the option Export from the menu that comes up. I suggest that you place the registry snippet you are saving in the same location as the Outlook data files. This ensures that all your Microsoft Outlook related data is in one easy-to-backup folder.

BackupOff2007-5

End of part 1 – Conclusion

By backing up the folder to an online backup service you can rest assured that whenever something happens to your computer all your emails and the mailbox settings can be retrieved with ease. If you add a new email account or remove one you no longer use, you should export the registry settings again.

When backing up your Outlook data files, Outlook should be shut down.

Although I have not mentioned Windows Vista, the procedure for Windows 7 is almost identical to that of Windows Vista.

Next time I will describe how to use the information you have saved to restore your Microsoft Outlook 2007 in case of corruption. I’ll also throw in how you can use this backed up data when upgrading your computer from Windows XP to Windows 7.

We all appreciate how important emails are to us. Back them up regularly.

Last 3 posts by Alan C

Total Cost of Ownership of Data Backups

January 19th, 2010

moneyAre online backup and recovery solutions cheaper than the counterpart traditional solutions? Before I deal with this argument I would like to point out a few points: – online backups provide an offsite disaster recovery solution, you can access your data from anywhere given that you have an internet connection and additionally, you will be enjoying the expertise and the scalability of big vendors.

Online backups offer cheaper costs per GB for the same functionality because you only pay for what you use. :) The costs include the storage used, bandwidth consumed and other related services. Data security is based on the latest encryption algorithms and adequate auditing features would place the end-user’s mind at rest!

A typical SMB that generates a full backup of say, 600GB of data would need an initial expense consisting of an entry level LTO-3 storage drive and an entry level server plus a couple of LTO-3 tape cartridges. Such setup would permit the full backup to be completed in about three hours and the need of one IT staff to be present to swop the tapes if compressing the data is not good enough to fit all data on one tape.

The Initial capital or better the total purchase cost for a similar tape solution would be about $3600. Therefore, the initial cost per GB would be of $6 per GB. If we assume that the above mentioned setup would run for a period of three whole years without entering into additional expenses such as, maintenance and repair costs (which is highly unlikely), then the cost per GB would be much lower. However, one would need to include the IT staff, space requirements and electricity expenses into the costs equation.

On the other hand, an online backup solution requires a good and stable internet connection, especially with regards to upload speeds. Various service providers provide services to move the initial large volumes of data through other means and hence, at cheaper rates. However, if an SMB had to implement this solution, it would cost him/her about $360 to transfer the initial 600GB of data which would yield a cost of 0.60c per GB. As already noted, an adequate internet connection is a must since an internet connection with an upload speed of 2Mbps would take a month to upload a full data backup of about 600GB. This may not be practical; however, alternate arrangements can be made with the provider to shift the initial data!

Conclusion: – An online backup solution has a lower TCO (Total Cost of Ownership) when compared to an internal solution.

Last 3 posts by George

Typical Backup Destination Media for Home Users

January 11th, 2010

Internal hard drives: Hard drives are relatively inexpensive and efficient but they are prone to viruses or software corruption due to other software running on the system. You can’t store it in a location separate from your computer. You have to install it in another computer in case you have problems with the computer and you need to recover the backed up data.

External hard drives: An external hard drive can easily be attached to your computer using a USB port. External hard drives can be stored in a location that’s separate from your computer, which can help protect your backup. You need to connect it with your computer every time a data backup is done and is usually slower than internal hard drives.

Writeable CDs or DVDs: CDs and DVDs are relatively inexpensive and are easy to find in stores. Practically, all recent computers have CD/DVD burners installed. You can store the CDs or DVDs in a location that’s separate from your computer. Depending on how much data you have, it might take several CDs or DVDs to hold your backup while CDs or DVDs can become corrupted over time.

USB flash drives: USB flash drives are relatively inexpensive and can hold a fair amount of data. You can store a flash drive in a location that’s separate from your computer. It could fill up quickly as typical sizes are still small.

Network locations: If you have two or more computers connected together, a shared folder or drive on one computer can be a convenient place to save your backup. Ideally, the network share is protected with a username and a password while backup time depend on the network devices’ transfer speeds.

For the most flexibility, my recommendation is to save your backup to an external hard drive :)

Last 3 posts by George

Wish You and Your Dearest a No Data Loss Year

December 31st, 2009

I want to take the opportunity to wish you and your dearest the very best for the forthcoming year. Some places are already referring to the newly hung calendar while others have a few more hours to go before their 2009 almanac would have served its purpose.

Health, good fortune and prosperity are the common form of wishes we bestow upon one another. And since I do not want to break with tradition I would like to bestow them onto you. Yet with all of these there is little we can do to determine the outcome. What destiny has in stock for each and every one of us will be revealed at the appropriate time. Some of the wishes can be even considered to be dangerous. For example I would never suggest to a friend to take on gambling so as to increase her prospects of getting some good fortune.

Yet I can safely wish you a year without any loss of data. The simple trick is to backup your data regularly, ideally in a manner that is automated and does not necessitate you having to remember to do it. If the process is automated there is a great chance that you will succeed in not losing any critical information if your computer takes all the data with it when it dies out.

Last 3 posts by Alan C

Are online backups for your computer a safe idea?

December 23rd, 2009

Are cars safe? Are computers safe? Is buying over the internet safe? These are some of the questions people who pose the question above might have asked a hundred, thirty and fifteen years ago. Today, many of us use cars, computers and regularly effect payments over the internet without much thought. When talking about online backups, the simple answer to the question being asked is yes.

Loony_Bin

I have, over the past year and a bit discussed this topic with my students. My students are normal people taking an office productivity computer course. Their computer is shared by at least one other member of the family. They come from all walks of life and a considerable portion of them have been the victim of some sort of data loss. The loss may have been as a result of a virus, an incompetent relative trying to tinker something on the computer or a hard disk failure. All of them reminisce about lost photos, assignments and other important electronic material once held on the computer.

Outside the dominion of what it is one should or should not do, the fact of the matter is that backing up to CDs, DVDs or other media is a non starter with the absolute majority of commoners. Some may take up the hobby right after a disaster but as soon as the memory of the event starts fading into the background so does the backup regime.
With the internet being consistently stable and fast the possibility of being able to backup your important stuff to an offsite server is bliss. Imagine a solution that is easy to setup and intuitively selects your important folders automatically. When you open the program it gives you a Windows Explorer type interface with the important folders already selected but lets you customize the backup list. When you hit the Backup button, the selected files and folders are copied in the background to a fast, safe and secure server. If the software cannot backup a file because it is currently being used, it prompts you about the matter and tries again later. Once a backup has been completed the programs sits quietly in the background monitoring changed files which it will update automatically. The program would keep deleted and prior versions of modified files for a specified number of days just in case you need to revert back to them.

If something happens to your computer and you lose everything, you simply install the program again and after typing in your username and password, the program automatically downloads saved backup settings so that you do not have to recreate them. This time round, when you hit the Restore button, the Windows Explorer type window comes up and you simply transfer the files from the remote server to your local computer. Similar reasoning would apply if you delete an individual file or need to revert to an earlier version of a document.

There are people who are rightfully concerned about their privacy. Most office productivity products allow you to password protect documents making them unreadable to anyone without the password. Certain file types such as images and audio files cannot be password protected at the file level although there are free programs that will create password protected archives. 7-Zip and Truecrypt are two products I have used frequently for this purpose.

Last 3 posts by Alan C

Is my backed up data complete?

December 15th, 2009

Wizard1How many computer users rely completely on application wizards?  I have nothing against application wizards, in fact, I do recommend users to make use of these when they are using new applications and lack experience with that application. However, wizards most often set certain parameters to default values rather than adding an extra screen or step to allow the user to verify some advanced parameters! Wizards are tools with a sole purpose of making your life easier and hence, would assume many assumptions! Hopefully, safe assumptions rather than deceiving ones! But, is this ok with backup applications? Without debating whether this is correct or not, I would prefer to share with you the most important parameters to look for when performing a data backup:

  1. The application or wizard allows you to set verification of a backup job – tick the verify check box
  2. Enable logging features – make sure you know the log files location
  3. The application allows you to encrypt backed up data with a password if your destination location is a remote storage host or removable media such as, pen drives – dual password entry text box
  4. Enable job status features such as, completion reports – check for failures or warnings

If these options are not presented to you in the application wizard then go and look for them before you start the job. Although, the verification mechanism was more intended to verify data integrity when backing up to tape media, it is still a useful check for all kind of media.

Another important task that I recommend you to perform from time to time is a restore test operation. Select a previous backup set and perform a restore operation to an alternate location (if you restore to the original location, remember you would overwrite your recent files with older ones). After a test restore operation check that the restored files are not corrupted such as, opening a word document or spreadsheet and verify that the data is complete and accessible.

Last 3 posts by George

The Computer’s First Backup

December 9th, 2009

As soon as computers came into existence they started breaking down; and breakdown they did. The early computers used vacuum tubes and would break down roughly every other day. The absolute majority of problems related to failed vacuum tubes although with so many mechanical parts, there were other breakdowns that would need addressing albeit comparatively much less frequently.

card

Punched cards also known as IBM or Hollerith cards were the medium used for input and output. Different codes would be represented by the presence and absence of holes in predefined positions. Programmers would have their program punched into a stack of cards. They would submit the program deck, followed by cards that held the data on which the program worked. The operator would load these cards into the card reader (together with other jobs). The operator would use a special language called the JCL (Job Control Language) to make the computer read the program into the computer’s memory and execute thereafter. The JCL instructions themselves would be entered on punched cards. Data used by the program would be loaded in a similar manner. Output from the program would be to paper if it needed to be human readable or to new punched cards if the output would have to be processed again. Today’s equivalent to punched cards is files stored on the hard disk.

A punched card was a stiff piece of paper. They came in a variety of colours allowing colour coding of different sections of a deck such as JCL, programs and data.  Punched cards did not originate with the computer industry; they were first used in the 19th century for controlling textile looms and for operating fairground organs and related instruments.

The punched card cannot be thought of a backup in the modern sense of the word. Important programs and data cards could be duplicated just in case anything happened to the working set.  The punched cards would be stored offsite and being made of stiff material proved to be quite durable.

We’ve come a long way in the world of backups, the technologies may have changed but the purpose and scope have remained fixed.

Merry Christmas to one and all.

Last 3 posts by Alan C

System protection in Windows 7

December 7th, 2009

Last 3 posts by George


System protection in Windows 7 is a very important mechanism that enables the saving of information related to system files and settings, saving previous versions of files that were modified, and creates restore points every 7 days and prior to significant systems changes. System protection is automatically enabled for the system drive only (Windows 7 installation location) but can be manually enabled for other drives given that they are formatted with the NTFS file system.

Important Note - a drive needs to have system protection enabled in order to be included in a restore point and in order to have the previous copies of files and folders functionality!

You launch the System Protection box from the control panel: (Click Start\Control Panel)

  1. In the Control Panel Click System and Security
  2. In the System and Security window click System
  3. In the System window click System Protection – now you have the System Properties window open with the System Protection tab selected

system properties

  4.   Select the drive you want to set and click Configure
  5.   After you make the required settings click OK

Required settings:

Restore system settings and previous versions of files

This is the default setting for the system drive or volume. Do not change this default setting, since restore points here hold system information and user files that are present in the user profile. Disk Space Usage depends on how much space is used for user files. If your files reside in this drive or volume then 20% to 40% should be ok. If the drive is set to hold just system files then 10% should be enough.

Only restore previous versions of files

You normally use this option on drives or volumes that contain user files only such as, a second disk drive for data storage. A value of 40% of Disk space usage would give you a large number of previous versions of your files.

Turn off system protection

This would be the setting for external backup drives or USB disks that hold backup file sets. The main problem with USB drives is that they are normally formatted using the FAT file system. Remember that System Protection works only with NTFS formatted drives!

Last but crucial point is that old previous versions of files that were overwritten due to a low Disk Space Usage setting will not become available by increasing the Disk Space Usage percentage! Then, you need to revert to a restore process from a backup set!

Restoring files in Windows 7

December 1st, 2009

Using the Backup and Restore console to restore files is very easy :) You can restore files to their original location or to a different location. There is another file recovery option and this is the restore from a shadow copy! Shadow copies are copies of files and folders that Windows 7 automatically saves when it creates a restore point. This feature may give you more flexibility when restoring files but such explanation is found in the Shadow Copies article.

To restore a file or folder follow these simple steps:

  1. Open the Backup and Restore console – type backup in the search box Start/Search programs and files and click the Backup & Restore Icon
  2. Click Restore My Files – assuming one or more backups have been done previously
  3.  The Restore Files window shows the latest backup as default but you can click on the Choose A different Date to select an older backup
  4. Click the Browse for Files button – click Browse for Folders button to restore a complete folder
  5. Search for the file you wish to restore and click the Add File button – repeat for more files or folders (use the CTRL button to add multiple files)

Restore a file

    6.  Click the Next button
    7.  Specify the restore location
    8.  Click Restore – make sure when overwriting current files with older ones, that the older file is the one you want!

Finally, click Finish :)