« Back to all recent discussions

NAS310 - RAID1 failed, volume down, how do I recover data?

steststest Posts: 9  Junior Member
edited July 29 in Questions
I've been running an NSA310 for a long time with just a 1TB hard drive inside, and now I am trying to add a second 1TB ESATA drive for a RAID1 setup. Not long after I clicked the migrate button to start, a the youngest in the house managed to grab the ESATA cable and it pulled out of the NAS. Now my volume is down, even though all it should have been doing was copying data to the other drive.

When I run the scan volume utility in the web client, I get the following results:
e2fsck 1.41.14 (22-Dec-2010)
The filesystem size (according to the superblock) is 244061472 blocks
The physical size of the device is 244061454 blocks
Either the superblock or the partition table is likely to be corrupt!
Abort? no
/dev/md0 contains a file system with errors check forced.
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/md0: 399705/61022208 files (2.5% non-contiguous) 206351709/244061472 blocks

How do I recover my many years' worth of data?


#NAS_Jul_2019
«1

Comments

  • MijzelfMijzelf Posts: 707  Heroic Warrior Member
    Can you open the Telnet backdoor, login over telnet, and post the output of

    cat /proc/mdstat
    cat /proc/mounts
    cat /proc/partitions
    su
    mdadm --examine /dev/sda2

  • steststest Posts: 9  Junior Member
    Let's see if this can get around the HTML filter...

    ~ $ cat /proc/mdstat
    Personalities : [linear] [raid0] [raid1]
    md0 : active raid1 sda2[0]
          976245816 blocks super 1.0 [2/1] [U_]
         
    unused devices: [lessthan]none[greaterthan]
    ~ $ cat /proc/mounts
    rootfs / rootfs rw 0 0
    /proc /proc proc rw,relatime 0 0
    /sys /sys sysfs rw,relatime 0 0
    none /proc/bus/usb usbfs rw,relatime 0 0
    devpts /dev/pts devpts rw,relatime,mode=600 0 0
    /dev/mtdblock6 /zyxel/mnt/nand yaffs2 ro,relatime 0 0
    /dev/sda1 /zyxel/mnt/sysdisk ext2 ro,relatime,errors=continue 0 0
    /dev/loop0 /ram_bin ext2 ro,relatime,errors=continue 0 0
    /dev/loop0 /usr ext2 ro,relatime,errors=continue 0 0
    /dev/loop0 /lib/security ext2 ro,relatime,errors=continue 0 0
    /dev/loop0 /lib/modules ext2 ro,relatime,errors=continue 0 0
    /dev/ram0 /tmp/tmpfs tmpfs rw,relatime,size=5120k 0 0
    /dev/ram0 /usr/local/etc tmpfs rw,relatime,size=5120k 0 0
    /dev/ram0 /usr/local/var tmpfs rw,relatime,size=5120k 0 0
    /dev/mtdblock4 /etc/zyxel yaffs2 rw,relatime 0 0
    /dev/mtdblock4 /usr/local/apache/web_framework/data/config yaffs2 rw,relatime 0 0
    ~ $ cat /proc/partitions
    major minor  #blocks  name

       7        0     139264 loop0
       8        0  976762584 sda
       8        1     514048 sda1
       8        2  976245952 sda2
       8       16  976762584 sdb
       8       17     514048 sdb1
       8       18  976245952 sdb2
      31        0       1024 mtdblock0
      31        1        512 mtdblock1
      31        2        512 mtdblock2
      31        3        512 mtdblock3
      31        4      10240 mtdblock4
      31        5      10240 mtdblock5
      31        6      48896 mtdblock6
      31        7      10240 mtdblock7
      31        8      48896 mtdblock8
       9        0  976245816 md0
    ~ $ su
    Password:


    BusyBox v1.17.2 (2016-03-11 17:11:16 CST) built-in shell (ash)
    Enter 'help' for a list of built-in commands.

    ~ # mdadm --examine /dev/sda2
    /dev/sda2:
              Magic : a92b4efc
            Version : 1.0
        Feature Map : 0x0
         Array UUID : 898a84cb:95d74d20:df7e7952:01ede6f0
               Name : nsa310:0  (local to host nsa310)
      Creation Time : Thu Jul 18 18:27:46 2019
         Raid Level : raid1
       Raid Devices : 2

     Avail Dev Size : 976245816 (931.02 GiB 999.68 GB)
         Array Size : 976245816 (931.02 GiB 999.68 GB)
       Super Offset : 1952491888 sectors
              State : clean
        Device UUID : 77f39edb:6c997257:d8007d77:af1cd38a

        Update Time : Tue Jul 30 20:47:29 2019
           Checksum : 88ee3ef8 - correct
             Events : 150


       Device Role : Active device 0
       Array State : A. ('A' == active, '.' == missing)

  • steststest Posts: 9  Junior Member
    edited duplicate comment to not take up space, since I can't delete.
  • MijzelfMijzelf Posts: 707  Heroic Warrior Member
    For some reason the filesystem on the raid array is bigger than the array itself. I think that is caused because it was a linear array, and now raid1, and the raid1 header needs some extra space.
    Fortunately the header is a version 1.0 header, which means it's at the end of the partition. So by acessing the filesystem on the partition itself, we get some extra space to play with.
    The partition is in use by the raid array, so to access it, we first have to stop the array.
    The procedure:
    stop raid array
    repair filesystem
    shrink filesystem
    start raid array
    grow filesystem on raid array

    In commands:
    su
    mdadm --stop /dev/md0
    e2fsck -f /dev/sda2
    resize2fs /dev/sda2 920G
    mdadm --assemble /dev/md0 /dev/sda2 --run
    resize2fs /dev/md0
    If any of this commands fail, stop and post the results.

    It is possible that e2fsck is actually called e2fsck.new.  You can check that beforehand by simply executing it without arguments.
  • steststest Posts: 9  Junior Member
    I ran this and thought e2fsck's result wasn't a failure but now that I'm thinking about it, I'm a bit worried.
     ~ # mdadm --stop /dev/md0mdadm: stopped /dev/md0
    ~ # e2fsck.new -f /dev/sda2
    e2fsck 1.41.14 (22-Dec-2010)
    Pass 1: Checking inodes, blocks, and sizes
    Pass 2: Checking directory structure
    Pass 3: Checking directory connectivity
    Pass 4: Checking reference counts
    Pass 5: Checking group summary information
    /dev/sda2: 399705/61022208 files (2.5% non-contiguous), 206351709/244061472 blocks
    ~ # resize2fs /dev/sda2 920G
    resize2fs 1.41.14 (22-Dec-2010)
    Resizing the filesystem on /dev/sda2 to 241172480 (4k) blocks.
    The filesystem on /dev/sda2 is now 241172480 blocks long.

    ~ # mdadm --assemble /dev/md0 /dev/sda2 --run
    mdadm: /dev/md0 has been started with 1 drive (out of 2).
    ~ # resize2fs /dev/md0
    resize2fs 1.41.14 (22-Dec-2010)
    Resizing the filesystem on /dev/md0 to 244061454 (4k) blocks.
    The filesystem on /dev/md0 is now 244061454 blocks long.
    The browser GUI is now showing the volume as Degraded instead of Down, and it still shows as "84.30% (772.56 GB) Used", but I'm still unable to access anything stored on the drive.

    I did have another separate issue come up that having the ESATA drive would be useful for. At this point, I'd be happy with either the RAID 1 functioning as intended, or just having access to my files on the internal disk with no RAID like I did before (and I could then use the ESATA elsewhere). Thanks for your continuing help in either direction.
  • MijzelfMijzelf Posts: 707  Heroic Warrior Member
    Hm. Your files should be visible by now. Have you tried to reboot the box?
  • steststest Posts: 9  Junior Member
    I thought I had, and I tried it again and am able to see a handful of files. I can see all the shares that were on it before; most are empty, and three of them have some small files visible.

    In the web GUI, I see the volume still degraded, with options to scan (with optional "Auto File Repair") or Repair the volume (as well as edit or delete, which I'm sure I don't want). Should these repair operations be safe to perform?
  • steststest Posts: 9  Junior Member
    I just now stopped and thought about the note on the volume page of the web gui:
    Note:
    When internal disk becomes defective while in RAID1 mode, the NSA will be in "uninitilized" state.
    You can bring the NSA to its normal state by using the external disk as the new internal disk.
    After login WEB GUI, you can repair the degraded RAID1 by another external disk.
    Does this mean to say that the only way to recover usage is by the repair button, letting it copy everything over to the ESATA drive to reconstruct the RAID1? I'm very apprehensive about just trying things on my own now, with all my data at stake.
  • MijzelfMijzelf Posts: 707  Heroic Warrior Member
    First, don't panic. According to e2fsck you have 399705 files, occupying 206351709*4k, which is a bit more than 800GB. e2fsck isn't complaining about anything, so the files are just there. The only problem is to find them back.
    Do you know an (exact) filename of one of the missing files? Then login on the telnet backdoor, and execute
    cd /i-data/md0
    find . | grep <filename>
    where <filename> is your file (When containing spaces, put it in quotes). It should show up, showing where it is.
    You can also get a complete listing by only running 'find .'

    Does this mean to say that the only way to recover usage is by the repair button, letting it copy everything over to the ESATA drive to reconstruct the RAID1?
    No. A degraded array can simply be used. It's only not redundant. This story is about a crashed internal disk, apparently the external is not visible then. You have to put the external disk inside to get a normal, degraded volume.






  • steststest Posts: 9  Junior Member
    I don't remember any specific filenames, but I tried a bunch of file extensions with nothing promising.
    ~ $ cd /i-data/md0/
    /etc/zyxel/storage/sysvol $ find . | grep apk
    /etc/zyxel/storage/sysvol $ find . | grep jpg
    /etc/zyxel/storage/sysvol $ find . | grep jpeg
    /etc/zyxel/storage/sysvol $ find . | grep png
    /etc/zyxel/storage/sysvol $ find . | grep txt
    ./.zyxel/storage.txt
    /etc/zyxel/storage/sysvol $ find . | grep mpg
    /etc/zyxel/storage/sysvol $ find . | grep mpeg
    /etc/zyxel/storage/sysvol $ find . | grep mov
    /etc/zyxel/storage/sysvol $ find . | grep mkv
    /etc/zyxel/storage/sysvol $ find . | grep mp4
    /etc/zyxel/storage/sysvol $ find . | grep exe

  • MijzelfMijzelf Posts: 707  Heroic Warrior Member
    $ cd /i-data/md0/
    /etc/zyxel/storage/sysvol $
    That looks strange. Normally /i-data/md0 is a symlink to /etc/zyxel/storage/sysvol, which is a symlink to the data volume in /i-data/, and so I expect it to end in /i-data/<something>, and not in /etc/zyxel/storage/sysvol.

    Possibly that symlink is damaged. Can you post the output of
    ls -l /etc/zyxel/storage/
    ls -l /i-data/


  • steststest Posts: 9  Junior Member
    ~ $ ls -l /etc/zyxel/storage/
    -rw-rw-rw-    1 root     root           433 Apr 25  2017 extuuid.table
    -rw-rw-rw-    1 root     root            44 Jun 23  2012 mduuid.table
    drwxrwxrwx    1 root     root          2048 Jul 29  2013 sysvol
    -rw-rw-rw-    1 root     root            29 Apr 24  2017 usbcopy.table
    -rw-rw-rw-    1 root     root            44 Apr 24  2017 usbzync.table
    ~ $ ls -l /i-data/
    drwxrwxrwx   17 root     root          4096 Aug  1 19:10 898a84cb
    lrwxrwxrwx    1 root     root            25 Aug  4 19:15 md0 -> /etc/zyxel/storage/sysvol


  • MijzelfMijzelf Posts: 707  Heroic Warrior Member
    That is strange. sysvol is a directory, and it is created in 2013! How can it ever have worked?

    Anyway, let's rename it, and create a symlink

    su
    mv
    /etc/zyxel/storage/sysvol
    /etc/zyxel/storage/sysvol.old
    ln -s /i-data/
    898a84cb /etc/zyxel/storage/sysvol
    reboot





  • steststest Posts: 9  Junior Member
    2013 is a reasonable estimate for when I started using this drive in the NAS, it might have been even earlier. The 310 is pretty old hardware lol. If that was a deciding factor, should I still follow those steps?
  • MijzelfMijzelf Posts: 707  Heroic Warrior Member
    Yes. If it doesn't lead to anything, the directory can be changed back. But As far as I remember even my NSA220 from 2008 had a symlink for sysvol.
Sign In or Register to comment.