Recovering Data from a Failed Synology NAS
by Ganesh T S on August 22, 2014 6:00 AM EST
It was bound to happen. After 4+ years of running multiple NAS units 24x7, I finally ended up in a situation that brought my data availability to a complete halt. Even though I perform RAID rebuild as part of every NAS evaluation, I have never had the necessity to do one in the course of regular usage. On two occasions (once with a Seagate Barracuda 1 TB drive in a Netgear NV+ v2 and another time with a Samsung Spinpoint 1 TB drive in a QNAP TS-659 Pro II), the NAS UI complained about increasing reallocated sector counts on the drive and I promptly backed up the data and reinitialized the units with new drives.
Failure Symptoms
I woke up last Saturday morning to incessant beeping from the recently commissioned Synology DS414j. All four HDD lights were blinking furiously and the status light was glowing orange. The unit's web UI was inaccessible. Left with no other option, I powered down the unit with a long press of the front panel power button and restarted it. This time around, the web UI was accessible, but I was presented with the dreaded message that there were no hard drives in the unit.
The NAS, as per its intended usage scenario, had been only very lightly loaded in terms of network and disk traffic. However, in the interest of full disclosure, I have to note that the unit had been used againt Synology's directions with reference to hot-swapping during the review process. The unit doesn't support hot-swap, but we tested it out and found that it worked. However, the drives that were used for long term testing were never hot-swapped.
Data Availability at Stake
In my original DS414j review, I had indicated its suitability as a backup NAS. After prolonged usage, it was re-purposed slightly. The Cloud Station and related packages were uninstalled as they simply refused to let the disks go to sleep. However, I created a shared folder for storing data and mapped it on a Windows 8.1 VM in the QNAP TS-451 NAS (that is currently under evaluation). By configuring that shared folder as the local path for QSync (QNAP's Dropbox-like package), I intended to get any data uploaded to the DS414j's shared folder backed up in real time to the QNAP AT-TS-451's QSync folder (and vice-versa). The net result was that I was expecting data to be backed up irrespective of whether I uploaded it to the TS-451 or the DS414j. Almost all the data I was storing on the NAS units at that time was being generated by benchmark runs for various reviews in progress.
My first task after seeing the 'hard disk not present' message on the DS414j web page was to ensure that my data backup was up to date on the QNAP TS-451. I had copied over some results to the DS414j on Friday afternoon, but, to my consternation, I found that QSync had failed me. The updates that had occurred in the mapped Samba share hadn't reflected properly on to the QSync folder in the TS-451 (the last version seemed to be from Thursday night, which leads me to suspect that QSync wasn't doing real-time monitoring / updates, or, it was not recognizing updates made to a monitored folder from another machine). In any case, I had apparently lost a day's work (machine time, mainly) worth of data.
Facebook
Twitter
RSS
55 Comments
View All Comments
Lerianis - Friday, September 5, 2014 - link
Links to the articles supporting that please.YoshoMasaki - Monday, August 25, 2014 - link
Hi there, I was wondering if you would recommend "Windows Server 2012 R2 Essentials with Update x64" as a home server for backups? I can get this for free through Dreamspark (https://www.dreamspark.com/Product/Product.aspx?pr... but I have never used WHS before and I'm a little intimidated by it. Reading "Windows Server 2012 R2 Essentials ... continues to have the requirement that it must be an Active Directory domain controller and that it must be the root of the forest and domain" (Source here: http://winsupersite.com/windows-server-2012/window... makes me think I'm in over my head, but I REALLY get lost when folks here talk Linux/Unix file systems and custom RAID stuff with a half-dozen drives. I'm a Windows guy for 20+ years now so I think I can learn it but I wonder if it'd be worth it. Thanks for your input.YoshoMasaki - Monday, August 25, 2014 - link
Post above ate my links ... please remove the parenthesis from the end or click here:https://www.dreamspark.com/Product/Product.aspx?pr...
http://winsupersite.com/windows-server-2012/window...
fatbong - Monday, August 25, 2014 - link
Completely agree. I wish there was a NAS available which used NTFS and simple disk mirroring. It would make data recovery extremely easy, if the NAS were to suffer hardware failure. I have an aging Buffalo Linkstation Quad, and hardware failure worries me. Is there any NAS out there which uses NTFS ? And no, I dont want to build/buy a server. I want an appliance.Stylex - Thursday, August 28, 2014 - link
Yeah, I migrated from WHS 2003 to Win8 with DriveBender, similar to Drivepool. Love that if it goes sideways all my stuff is NTFS. I don't have time or stress levels to deal with linux command line stuff to get it working again.BD2003 - Friday, August 22, 2014 - link
I recently decided to drop my home NAS (synology ds212j), since I no longer have multiple PCs...and getting that data off was a nightmare. It had a backup drive that was formatted in ext4, since synology didn't support incremental backups to NTFS.Because of the way it stored the incremental backup, it was basically useless for reading directly through an ext driver for windows. I had to completely wipe the backup drive and reformat it in NTFS to make a one time backup, and cross my fingers that I didn't lose a drive during the damn near 24 hour process (thanks to the hyper fragmented NAS drives, barely adequate NAS CPU and USB 2.0.) Then I had to pull the drives, reformat them, and pray the backup worked. Then transfer everything back. This process literally took days.
If it was a windows based box, I could have just pulled the drives, dropped them in the PC and been done with it in 5 minutes, without even rebooting. I probably would have never even dropped the NAS, since I could upgrade it without having to migrate anything.
Basically the entire experience put me off of ever using a Linux based NAS ever again. Between the file system incompatibility and the potential for RAID array failure....it's just not worth it. My data has never felt so unsafe than during that process.
dabotsonline - Friday, August 22, 2014 - link
"In the end, I decided to go with a portable installed system, which, unlike a persistent install, can be upgraded / updated without issues. I used a Corsair Voyager GT USB 3.0 128 GB thumb drive to create a 'Ubuntu-to-go' portable installation in which I installed mdadm and lvm2 manually."Even though it wasn't specified in the Synology FAQ, wouldn't a portable installation of Parted Magic or SystemRescueCd work OK?
Christobevii3 - Friday, August 22, 2014 - link
From my experience with my synology 413j for a while I've learned a few things:First: Turn on the smart disk check to run weekly, otherwise if you never restart the device you won't have a hint of failure coming.
Second: 5TB usb's are cheap, setup a backup task to these.
Third: UPS. Always run a ups. A $50 apc is enough and will hold the device up for a while and allow it to shutdown properly in a power outage.
Last: If you have a disk failure it locks up the device. You will probably be able to detect which disk it is by pulling one at a time. When you get the proper one it will be accessible and you know which disk to replace.
jbm - Friday, August 22, 2014 - link
Very interesting article - have often asked myself what to do if my NAS ever should die a sudden death, because it's a bit old (Thecus 4200) and I probably would neither manage to buy another one nor want to (not because it is bad, just because I'd rather switch to something newer). More articles like this, please!Nogami - Saturday, August 23, 2014 - link
I'd be curious what exactly was the component that died - I've had yet another capacitor failure in the last week which took out an old LGA 775 motherboard (though it happily enabled an upgrade to an i7-4790K).All in all, the vast majority of hardware failures I've had in the last 5 years has been due to capacitor death, usually in power supplies, causing general flakyness, and eventually becoming terminal. I'm curious if that was the case here as well.