NAS Data Recovery from a RAID 1 Buffalo Linkstation

NAS data recovery Ireland

NAS devices have never been so popular. They are compact, relatively easy to administer and can often perform the same storage functions of a traditional server.

There are numerous brands of NAS widely available in the Irish market to cater for most storage requirements and budgets. Some of these manufacturers include Drobo, LaCie, Buffalo, Iomega, ReadyNAS (from Netgear), Seagate, Western Digital, Qnap, ZyXel, Synology and G-Technology. All of these manufacturers have a wide variety of models available with different capacities, I/O specs, RAID levels and file systems.

One popular file system commonly used in NAS servers is the XFS file system (developed by Silicon Graphics International in 1993). It is a powerful, fast and scalable file system which can handle a whopping 8 exabytes of data. It is not surprising therefore that organisations such as the CERN research laboratory and NASA’s supercomputing division use XFS for many of their projects requiring high-capacity data storage. One of the main reasons for the growing popularity of XFS is its speed. It is significantly faster than EXT3. When deploying any operations which utilise sequential buffered writes – it will be faster than EXT4. How does it do this? XFS cleverly deploys Allocation Groups to allow multiple I / O requests. Moreover, XFS has another trick up its sleeve. It uses a feature known as Direct I/O – this means data can be transferred from the file system directly to RAM space – obviating the need for a cache or processor request. Clever stuff – but the dexterity of the XFS file system does not end there. XFS does not just offer great IOPS rates; it also uses file parsing. A lot of files will contain a large number of zeros. XFS – hating to see wasted space – will put metadata in place of these zeros. When the file is accessed – it will revert to its original size. In addition, XFS uses online defragmentation – data fragments are converted into continuous blocks on-the-fly. Both of these attributes of XFS means space allocation on XFS-formatted drives is used very efficiently.

So taking into account its great storage capacity, speed and efficient parsing – you might be thinking “if Carlsberg did file systems…” it would be XFS? Maybe, but XFS does have some drawbacks. For example, XFS does not handle sudden power loss very well. Whilst it is a journalised file system, the journaling system is designed to increase performance and not offer redundancy. If there is a power failure on an NAS or server running XFS – your data will probably be irreversibly lost. (hence, a UPS will be indispensable when using XPS). Moreover, like any file system XFS can go corrupt.

Take for instance a case we were dealing with last week. A multinational medical device company from Limerick contacted us. In one of their research laboratories, their Buffalo Linkstation NAS device became inaccessible. Their IT administrators removed the hard drives (two Seagate Barracuda 2TB – model ST2000DM001) and slaved them onto a Linux system. They ran the “xfs_repair” command. This is a fairly standard repair command for XFS which, unlike, “fsck” is not invoked automatically upon system start-up. However, this operation was continually aborting for them. Errors were being returned to them about primary and superblock corruption. Not to be defeated so easily they unmounted the volume again and ran a “xfs_repair –n” command which performs a more thorough check of the file system. Alas, this operation was also aborting for them also.

They sent the Buffalo NAS box to our Dublin lab. We removed its drives and performed diagnostics on them. Drive 0 had several thousand bad sectors on it. This explained why “xfs_repair” and “xfs_repair –n” commands was unable to complete. As always, in order to maintain the integrity of the data we imaged the drives using a hardware imager designed for data recovery purposes. This means we can perform data recovery using a copy of the volume rather than using the original drives themselves. Once imaging had completed, we then examined the inode maps on the volume. We removed all duplicate blocks. We cleared the lost and found directory. Then we rebuilt the volume’s trees and headers. Any disconnected inodes which we found – we moved to the lost and found folder. The volume was then mounted. We then had a workable volume with which we could rebuild their RAID array with. From our examination of the volume’s stripe size, RAID header and parity – we determined they were using RAID 1. Their array was rebuilt. The rebuilt volume was then mounted and we saw what looked like a valid structure and folder directory. We then invited our client to login to our systems remotely to view and verify their retrieved files. The client was delighted. Every file they needed had been successfully recovered. Without this data, their R&D team would have had to replicate months of work. Their recovered data was extracted onto a high-capacity external USB hard drive and along with their Buffalo NAS box and original drives was dispatched back to Limerick. The lesson: even the best file systems can go corrupt if the underlying hardware starts to fail. NAS devices are not bulletproof. The data stored on them should be backed up onto another drive. Better still, many NAS manufacturers like QNAP and Synology have options in their software to backup directly to the Cloud.