Dell EMC VNX storage failure: how to recover data and services safely

End of support: where you stand

Dell no longer provides standard support for any VNX model. That means no new software releases, no original parts under the manufacturer's service and no root-cause analysis of failures. The last VNX2 software releases date from 2022.

From Dell's hardware and software end-of-service-life notifications.

In practice these arrays are kept alive today by third-party maintenance companies and second-hand parts. That can safely bridge a few months, but not years. The array often holds data for systems that must be available around the clock, and for organisations covered by NIS2, hardware without security updates is also a compliance problem.

The first hour: protect what still works

Most data lost in storage failures is lost not to the failure itself but to rushed repair attempts. So we start by securing the current state:

1. Freeze changesNobody pulls drives, reboots processors or changes configuration until it is clear what happened. One person coordinates the work and records every step.
2. Assess the impactWhich servers and systems have lost access to data and which are still running, even if slower. Whether hosts have paths to both processors (SPA and SPB).
3. Collect logsSP Collect from both processors (each takes 10 to 15 minutes and may briefly slow the array). On unified arrays, also nas_checkup and support material collection on the Control Station.
4. Save the configurationA full capture of the physical and logical configuration: RAID groups, pools, LUNs, storage groups. In a worst-case recovery this is the only map of what was where.
5. Rescue the data that is still availableIf the array is running degraded, move or copy the most important virtual machines and databases to other storage straight away. Also check when each system last had a successful backup.

Diagnosis with naviseccli

Unisphere, the VNX graphical console, runs as a Java applet. Modern browsers and current Java versions no longer run it, so during a failure it is easy to lose time fighting the console. Dell provides VNX Launcher (the Unisphere Client for Windows) with its own bundled Java, but the most reliable tool is the naviseccli command line.

# security file, so you do not pass the password with every command
naviseccli -AddUserSecurity -scope 0 -user admin

# list of faults: drives, enclosures, power supplies, processors, batteries
naviseccli -h 10.0.0.10 faults -list

# component status: processors, standby power (SPS or BBU), LCC cards
naviseccli -h 10.0.0.10 getcrus -all

# state of pools and RAID groups
naviseccli -h 10.0.0.10 storagepool -list -state -status -prcntFull
naviseccli -h 10.0.0.10 getrg 0 -state -disks -lunlist

# state of a specific drive, e.g. system drive 0_0_0
naviseccli -h 10.0.0.10 getdisk 0_0_0 -state

# owner of a RAID group LUN: current and default processor
naviseccli -h 10.0.0.10 getlun 12 -owner -default -trespass -state

# write cache state (VNX2)
naviseccli -h 10.0.0.10 cache -sp -info -state

# latest event log entries from a given processor
naviseccli -h 10.0.0.10 getlog -100

We collect logs from both processors, then download them from the array and capture the configuration:

naviseccli -h 10.0.0.10 spcollect
naviseccli -h 10.0.0.11 spcollect
# after 10 to 15 minutes
naviseccli -h 10.0.0.10 managefiles -list
naviseccli -h 10.0.0.10 managefiles -retrieve

naviseccli -h 10.0.0.10 arrayconfig -capture -output vnx-config.xml

What we look for: which drives are Faulted, Removed or rebuilding, whether pools and RAID groups are degraded or offline, whether the write cache is enabled, whether standby power is healthy and whether LUNs are running on their default processor. LUNs trespassed to the other processor usually point to a path problem or a faulty processor.

Common failures and how to handle them

A single drive failure

This is the most common and least dangerous case: the RAID group keeps working and the array starts rebuilding onto a spare. On VNX1 the spares are drives designated for that role. On VNX2 (MCx architecture) any free, unbound drive of a compatible type can become a spare. After a drive is removed, the array waits 5 minutes and logs changes: if the same drive comes back in time, a quick partial rebuild is enough. After 5 minutes a full rebuild starts on another drive.

After the failed drive is replaced, VNX2 does not copy the data back automatically: the new drive becomes a free spare. If the original layout matters, the copytodisk command does the copy, but it cannot be aborted and may take a long time, so we schedule it outside working hours. Dell recommends one spare per 30 drives; check the policy with hotsparepolicy -list.

Two drives in one RAID 5 group

A RAID 5 group survives one drive failure, not two. In a pool, data is spread across many private RAID groups, so the failure of one of them usually takes the LUNs of the whole pool offline. This is the moment when it is easiest to make things worse: do not pull more drives, do not delete or recreate RAID groups and LUNs (unbind destroys data) and do not update the array software. First establish whether the second drive really failed or just lost its connection, for example through an LCC card or a cable. If the data matters and there is no backup, bring in a company specialising in storage data recovery before touching anything. In parallel, start restoring from backups onto other storage.

System (vault) drives

On VNX2 the system drives are the first four drives of the array, 0_0_0 to 0_0_3. Each has about 300 GB reserved for the array software and system data, and the write cache is dumped to them on power loss. A single failed system drive does not stop the array, but with several failed at once the array disables the write cache, which noticeably slows every system. Replace system drives one at a time, following Dell's procedure, and never pull them "to see what happens".

Processor or standby power failure

When one storage processor (SP) stops working, the other takes over its LUNs, provided hosts have paths to both processors and the correct failover mode. The array runs slower, but it runs. The worst thing you can do at that point is reboot the surviving processor. A failure of standby power (an SPS or the built-in BBU battery) can disable the write cache: data is safe, but performance drops.

After the processor is repaired, LUNs can be moved back to their default owner with trespass mine. Dell's documentation warns, however, that doing this with active I/O can cause I/O errors, so we do it in a maintenance window.

Unified (file) arrays

Models with file services (NFS, SMB) add Data Movers and a Control Station. The Control Station must be running for a Data Mover to fail over to its standby. On a system with a single Control Station, its failure means that file shares will not fail over on their own if another fault follows.

What not to do

• Do not pull several drives, or any drive from a group that is already degraded. Every pulled drive means less redundancy.
• Do not reboot both processors, and reboot a single one only when the other's standby power is healthy. The rebootSP -both command refuses to run while the write cache is enabled, and there is a reason for that.
• Do not reseat the LCC card or SAS cable of the first disk enclosure while its processor is running. On models with a separate processor enclosure (VNX5700, VNX7500, VNX8000) the processor then loses access to the system drives and reboots, and according to Dell this can lead to both processors rebooting at once and data becoming unavailable. Shut that processor down first.
• Do not use unbind and do not recreate RAID groups or pools. That is not a repair, it is deleting data.
• Do not change hosts' failover mode without a plan. A wrong value cuts off access for the whole storage group.
• Do not update the array software during a failure, and do not start repairs without SP Collects from both processors.

Restoring services and data

Once the array is stable, we restore services in a fixed order, from the foundations up to the applications:

1. Array: drives rebuilt or rebuilding, pools and RAID groups online, write cache enabled.
2. Paths: LUNs on their default processors, hosts seeing all paths (rescan in VMware, Hyper-V or on physical servers).
3. Virtualisation: datastores mounted, virtual machines started from domain controllers and DNS, through databases, to applications.
4. Consistency: file system and database consistency checks, comparison with the latest backups.
5. Backups: jobs resumed and an immediate full backup of everything that was saved.

If some data cannot be recovered from the array, we do not wait for the hardware repair: we restore it from backups onto other storage. Finally, we write down the cause, the course of events and the lessons learned. Without that, the same failure will come back.

Planning the replacement

A failure is a good moment to close the chapter on an unsupported array. There are three routes:

• A new Dell array. PowerStore can import LUNs directly from VNX2, either without downtime (with a host plug-in and multipathing) or without installing anything on the hosts. File import handles virtual Data Movers (VDMs) from VNX2 with NFS or SMB shares. For VNX1, import needs checking against the support matrix.
• A different array or storage built into the virtualisation cluster (for example Ceph in Proxmox VE or Storage Spaces Direct). Data is then moved with the hypervisor's tools or by restoring from backup.
• Migration together with a change of hypervisor. If you are leaving VMware anyway, it is worth combining the two projects. We cover this in our guide Proxmox VE or Hyper-V.

• An inventory of LUNs, shares and hosts with capacity, performance and owner.
• Measured real workload (IOPS, throughput, latency) before sizing new hardware.
• Backups of every system on the array, and a tested restore, before migration.
• A chosen migration method and maintenance windows for each system.
• A rollback plan: the VNX stays connected and untouched until everything is confirmed working on the new platform.
• Secure erasure of the old array's drives before disposal.

Frequently asked questions

Sources

• Dell: end-of-life documents
• EMC VNX Series 5.33: CLI Reference for Block (naviseccli)
• Dell KB 000029126: replacing a vault drive on VNX2, spares and rebuilds
• Dell KB 000065874: SP reboot during LCC replacement
• Dell KB 000019534: collecting SP Collects
• Dell KB 000019852: VNX Launcher (Unisphere Client)
• Dell PowerStore: importing external storage

As of 6 November 2026. The procedures cover typical cases; for a specific failure, the array logs and the documentation for that model take precedence.