Will This Work To Make Backups Easy?

The other day, I received an email from Berkeley about helping out again by donating CPU time to BOINC -- that's open-source software for volunteer computing and grid computing. People who need massive computing power can use BOINC to run their software using volunteer's computers. These programs have to do with mathematics, astrophysics, physics, biochemistry, climate, aerospace engineering, astronomy, cryptography, chemistry, drug discovery, epidemiology, cognitive science, factorization of large integers, artificial intelligence, seismology, environmental research, molecular biology, biology, computer science, astrobiology, biology, molecular biology, chemical engineering, nanotechnology, genetic linkage analysis, life science, software testing, medical relearch, humanitarian research, and evolution. Here's more info: http://boinc.berkeley.edu/

What started this was years ago I volunteered CPU cycles to SETI, the search for extraterrestrial intelligence. Yes, millions of computers are processing data from "the beyond," looking for intelligent communications.

If you keep your computer running 24/7, I highly recommend donating your CPU idle time to one or more of the projects listed on the Boinc subdomain listed above.

But that's not the original reason for this post. It's what brought this post about, though. Part of helping through BOINC includes software that uses virtual machine software from Oracle -- open source, free software called virtualbox (https://virtualbox.org). I did not know this software existed until the email from Berkeley arrived.

I know enough about what I'm writing below to be dangerous. I've worked through several problems with what I'm going to tell you, but I'm no expert here. That's why I hope you'll add to this post with your learned comments.

You may have heard of (or used) VMware, which is a commercial product. And Microsoft has it's own VM software. If you have, skip down a bit to get to my reason for bringing all of this up.

"VM" stands for virtual machine. VM software allows you to create virtual machines that run on your computer. These machines operate like a separate computer within your computer. I use Windows 7, but with virtualbox, I also use Ubuntu, Windows XP, DOS, and a copy of my Windows 7 that's on my "host" computer. These VM's are "guests" by the way. People who test software often use virtual machines for their testing. If the software crashes the VM so it won't boot -- if the software contains a virus and totally wrecks the VM, it's no big deal. They just delete the virtual machine and create a new one with a fresh installation of the operating system. All the while, their "real" computer (the host in VM terms) is untouched, unchanged, left alone, working as it always has, etc. This is similar to using a sandbox, except it runs completely independently of the host machine's OS. Programs run by sharing the CPU and system memory. VM disk drives are virtual drives that exist as a file on host machine hard disks. Virtualbox allows you to create a drive that's dynamic in size, so it's only as large as the virtual machine needs for storage of OS, programs and/or data.

Here's where I'm going: why not install your music creation software on a VM running a copy of the operating system you use on your computer? After the installation, you have a virtual computer that you can tweak just for your music software. And you'll have your exact setup that's easily saved and backed up. As an example of what I'm saying, take Sonar, which I use (mostly). To properly back it up as I like it, with all the settings, file locations, VST's, registry settings, etc., I have to back up my entire system -- something that takes many hours when I do it. To save just the OS and a complete Sonar installation, I'd have to save all sorts of directories located all over my  system. And what if I leave something out? And there's no way for me to check a complete backup to see if it will work. I'd need an exact duplicate of my physical system to test that.

Each hard disk on a virtual machine is a single file. That means that I can save my whole Sonar setup and the OS by copying this one file to a backup drive -- or the cloud -- or wherever.

I'm in the process of testing this thought out. There are problems that have come up. I'm not sure how well a Virtualbox Windows 7 guest is going to work with my USB digital audio interface. Will I be able to use ASIO drivers? I don't know yet. I have been able to get the Win 7 VM to use DirectSound to play the sound through the USB audio interface. And I got Ubuntu audio to work by turning off the enable audio setting in that VM. Somehow that still routes the audio through the USB audio interface. 

I'm sure that at least a few of you reading this have some experience trying something like this. I'd love to hear how it worked out for you. And if this starts some of you on the journey, feel free to share your experience.

Backups are a huge headache. Where do you store a huge backup of a whole system (say you have 4 TB system)? Compression ain't gonna help that much.

Some things I've learned:

1. Disk2vhd (http://technet.microsoft.com/en-us/sysinternals/ee656415.aspx) will create a clone of a disk drive as a vhd file, which is the Microsoft version of Virtualbox's vdi (virtual disk) file. Virtualbox will mount a vhd directly (it also handles many other types). Where this came in handy is that I had a USB drive with multitrack files on it. Instead of making that USB drive available to a Win 7 VM with Sonar, I cloned the USB drive to a vhd so I could work with the data as a second virtual drive. That saves having the USB drive connected to the system. I have enough wires running already. And the drive is completely cloned (byte for byte) as far as I can tell. Backing up the drive will be a fairly quick copy of the vhd file to a backup drive or the cloud. I've read where people have had problems with vhd bootable drives. I've only tried it with data drives.
2. I haven't found a free solution to cloning a vhd to a physical drive.
3. Windows 7 works well in a VM. It's interesting to see how much I have tweaked Windows 7 over the years of use. I've only updated the VM version with updates that were required by the software I've tested. Since I don't plan to use the VM to connect to the internet, I'm not worried about firewalls, anti-virus software and the like. So, the VM runs faster than the host, even with less physical memory and one or two cores being dedicated to it.
4. CloneVDI is a free tool for working with vdi files outside of Virtualbox. It will save many virtual drive types as a vdi. Virtualbox has added some of the benefits of CloneVDI, but it still does some things faster and easier. https://forums.virtualbox.org/viewtopic.php?f=6&t=22422 

I'll post further information about this as I come upon it. I'll be trying out some different music creation software to see if it plays well in a Windows 7 VM.


Happy 2014 and may the best things come your way this year!

EDIT (18 JAN 2014) **********
I've been in "experiment land" for the past week and have learned quite a bit.

1. As far as I can tell from my own trials and hundreds of searches, presently there's no way to connect a MIDI device to a VM (Virtualbox or VMware). My USB MIDI interface doesn't appear in the USB settings of a VM.
2. Same for the USB digital audio interface, which does appear in the VM USB settings but does not directly appear as an audio device in the VM audio settings.
3. Virtualbox has an audio driver that gives the host machine access to the USB digital audio interface through DirectSound.
4. No ASIO drivers are seen by the VM.
5. After discovering the above, I thought that maybe the best way to use one machine for everything is to have all the "interface intensive" software (video, audio, MIDI, etc.) reside on the host and put all the other (non/no interface intensive) software in a VM. Then backup the host, but exclude the VM's from the backup. They can be backed up separately.
6. With free extensions to Virtualbox, it supports dual monitors, among other things.
7. Maybe someone reading this has the knowledge to make MIDI work? and even the USB digital audio interface?