Hey All,
This site has been such a great resource that I just wanted to give back. This write up is my experience with building a Custom Unitrends Recovery 943 as well as my solution to eSATA Archiving. I compare my build to the 943 directly because they both sit on the same hardware platform....SuperMicro's Storage Server with 36 Drive Bays. Although I am comparing the two, they are not exactly the same as the Recovery 943 has more power and memory, it's running separate volumes for the OS and Database which is not possible for UEB as well as including active monitoring and some other things. Here are my build specs.
Hardware:
Super Micro Storage Server 6047-E1R36N
2 x Intel E5-2620
128GB RAM
25 x 4TB Hitachi Ultrastar 7200RPM Enterprise SATA Drives
LSI SAS2108 RAID Controller w/ BBU (comes standard with the server)
Sandisk Cruzer Fit 4GB
Software:
VMware ESXi 5.1U2
Unitrends UEB 7.3
Windows Server
Background:
I was running Unitrends 7.2 on a Dell Tower running Windows Server 2012 Hyper-V and a Promise iSCSI SAN for backup storage. I initially purchased a Synology RS3412RPxs with 10 x 4TB Enterprise SATA Drives. That was a disaster. iSCSI and even NFS was unbearably slow on the Synology regardless of the Server connected to it or the hypervisor. I tried both VMware and Hyper-V. I compared it to our 2 bay Synology NAS running iSCSI and that was way faster. In the end, we returned it after working with their support team for months. I got so frustrated with iSCSI that I decided to go the local storage route. Don't get me wrong...I love iSCSI and we have 2 SANs in our environment and I know how to manage them but using iSCSI with Unitrends didn't work for me. Even now with the Promise and Hyper-V, it disconnects and takes down the UEB VM....although I hear that's an issue with Hyper-V most likely.
Server Hardware Configuration:
System was ordered through Aspen Systems with a 3 year 24x7 4Hour Onsite Warranty. They did their component and system tests to make sure everything is working and shipped it to me. Took roughly two weeks but once received and Racked, I went ahead and configured the RAID array. The hard drives are configured in a 24 x 4TB RAID10 (OBR10 all the way!) with 1 hot spare. Initialization for the 48TB RAID10 array took about 18 hours. The reason I went with RAID10 is because of the large amount of storage that I needed. I could have gone with RAID6 (which is what they use on the Recovery 943) but I wanted the array to be able to rebuild faster in case of a drive failure. RAID 6 would take longer to rebuild than RAID 10 especially for the large 4TB drives. I also wanted to give as much IO performance to the server since Incremental Forever backups are very disk intensive. I purchased a large amount of RAM to give UEB as much as possible while allowing me to possibly use this server as an emergency ESX host in case of failure of my other hosts. I also plugged in the IPMI port and configured the IP address and it works really well which I will discuss below. Lastly, I plugged in the 4GB Cruzer fit into the rear USB port.
Server Software Configuration:
I installed ESXi 5.1U2 on the Cruzer fit. I went back and forth between installing ESXi locally or a USB flash device. I read all the posts here and out in the interweb. At the end, I decided on the USB Flash install. The main reason is I only have 1 storage array. If there was a storage failure, I could still boot the system and see what's going on. Plus upgrading ESXi will be done on a separate storage device and not on the OBR10 where the UEB VM will be sitting. In case something does go horribly wrong with ESXi...only the Cruzer Fit will be affected. I know I may sound a little paranoid but better safe than sorry. I also have another Cruzer fit as a backup. I was actually home on the weekend when I installed ESXi. The install CD had actually ejected (external slim DVD ROM) and it wouldn't go back in. Luckily the IPMI allows you to load an ISO through the browser as a virtual CD ROM drive. I was able to install ESXi that way and saved me a trip back in the office. The install did not go without a hitch. I read beforehand that sometimes ESXi will not boot when installed on a USB Flash drive. All I had to do was go into the boot options of the ESXi install and run the command "runweasel formatwithmbr". It basically formats the flash drive with MBR instead of GPT which is the default. ESXi installed with no issues after that. One thing you'll have to configure post installation of ESXi is the location of the scratch partition. Make sure it's not pointing to the USB flash drive as it will wear it our faster. Point it to a local VMFS location when one becomes available. I then went ahead and deployed the UEB package from Unitrends. One thing to note here is that the current UEB VM Template is running Virtual HW version 7. This limits the number of vCPUs that can be configured for UEB (which is running Cent OS). The max is 8 but I purchased 2 x 6 Core CPUs.....which in a way is a waste since it can only utilize 8 and this VM host is mainly dedicated to UEB. I spoke to Unitrends and they plan on certifying HW version 8 for use with UEB in the future. After the deployment of UEB, it goes through the configuration wizard which was straight forward. One thing to watch out for is that the time setup never seems to work properly for me. I choose -8 Los Angeles but it's always wrong. I have deployed UEB all the way from 7.0 to 7.3 dozens of times and this has always been a problem for me. Probably something I'm doing wrong but haven't had a chance to ping Unitrends Support on. Just set the time manually. As for configuring the storage, I just created 2TB Thick Lazy Zeroed VM Disks and added it to the UEB VM and using the expand storage. At the moment I am provisioning about 25TB of backup storage to UEB with retention set at 30 days min/120 days max. The beauty of it is I don't have to commit all 48TB to UEB. I can provision the VM Disks and expand the storage in UEB as needed. Right now I am backing up 9 Physicals, 3 VM hosts, and 3 SQL Servers. My backup schedules are Masters with Differentials and 1 server with Incremental Forever. The reason for the FULL DIFF is that I have the space and it's faster to restore than Incremental Forever. From previous conversations with many Unitrends techs, I get the impression that FULL/DIFFs are faster and better (less load) than Incremental forever. The only reason I have 1 incremental forever is because that server is huge....4TB+. A weekly master would take too long to finish each week.
I'd love to hear from everyone and see how you guys setup your backups, why and how well has it worked for you including when restoring.
Hardware Monitoring:
Unitrends provides active monitoring with their appliances which is a great selling point for their hardware. I wanted that monitoring for my build but it's obviously not possible through the UEB product. The main thing that needs to be monitored is the hard drives. The server components itself can be monitored by any ESXi compatible monitoring software but the health of the RAID is not transparent through ESXi. I was able to find information on the interweb about setting up a Windows VM to run LSI's MegaRAID software and manage the host's LSI RAID card from the guest VM. In my case I'm using Windows Server 2008 R2 but it can be done with Windows 7 or 8 if you want to go on the cheap. With this setup, you can configure health alerts as well as manage the RAID array without having to resort to MegaCLI or the BIOS setup. Instructions are in the links below. It works great for me.
http://www.tinkertry.com/lsi92xx-health-under-esxi-51/
http://www.tinkertry.com/esxi-5-1-can-run-lsi-megaraid-in-a-vm/
Archiving to eSATA:
Another feature of the hardware appliance is the ability to archive to eSATA using either their eSATA hardware or an eSATA dock and plugging it in the back. Unitrends just recently added the ability to archive to eSata via VMware's Pass-through feature (see below for a link to the KB article). I'm passing through the onboard Intel SATA controller and I'm using an eSATA to SATA cable. Just make sure you configure the SATA ports in the Server BIOS to be hot pluggable. I configured 2 drives and it's being recognized as a Recovery Archive target and created a 2 drive archive set.
http://support.unitrends.com/ikm/questions.php?questionid=1470
http://support.unitrends.com/ikm/questions.php?questionid=1554
I am however having an issue with the Archiving Schedule screen seeing my eSATA as a target. The Archive Now screen can see it but the schedule can't. For now I'm manually kicking off Archives.
Another option for me is to follow the docs above and get an LSI 9211-4i card which has been tested by Unitrends and buy their Four Drive RXDA 1U eSATA enclosure and use that for archiving. The plan is to do an offsite 4 week archive set rotation.
That pretty much it. If I do any other customization or changes worth sharing, I will update the thread. Thanks!