Lucene search

K
veeamVeeam softwareVEEAM:KB4573
HistoryApr 18, 2024 - 12:00 a.m.

Backup fails with: "The system cannot find the file specified." or "The device is not ready."

2024-04-1800:00:00
Veeam software
www.veeam.com
4
backup failure
missing files
shadow copy

AI Score

7

Confidence

High


Version Applicability

This article does not list specific versions of Veeam products because the underlying issue stems from a problem with the Microsoft Volume Shadow Copy Service and is independent of any Veeam product versions. As such, the issue can affect environments regardless of the Veeam software version in use.

Challenge

A backup job for a Windows machine being processed using Veeam Agent for Microsoft Windows fails during the disk read operation with either of the following errors:

  • The system cannot find the file specified.
    

    Asynchronous read operation failed
    Failed to upload disk.
    Agent failed to process method {DataTransfer.SyncDisk}.

  • The device is not ready.
    

    Failed to schedule asynchronous read operation
    Failed to upload disk.
    Agent failed to process method {DataTransfer.SyncDisk}.

_
Example job report:_

emerror

Cause

These errors occur either because:

  • The shadow copy that Veeam Agent for Microsoft Windows was reading from has been unexpectedly removed.

or

  • The shadow copy that Veeam Agent for Microsoft Windows requested the VSS subsystem create was unable to be created.

Veeam Agent for Microsoft Windows issues requests to the Microsoft Software Shadow Copy Provider to create a volume shadow copy during a backup. If the shadow copy cannot be created or is unexpectedly removed during the backup operation, the backup job will fail.

Solution

Since these issues stem from the Microsoft Volume Shadow Copy Service, it is essential to examine the System Event logs for β€˜volsnap’ events to pinpoint the root cause within your environment. Typically, the most effective resolution for such issues involves redirecting shadow copy storage to a volume that has ample free space and a high number of available IOPS.

Example Error

Example volsnap Error

Event Filter

Event Filter Example

General Troubleshooting Advice

The β€˜volsnap’ error will provide details about which drive is impacted and the underlying issue.

For example:

  • The most common Volsnap error reported by customers is:

    Event ID: 25
    

    Description: The shadow copies of volume X: were deleted because the shadow copy storage could not grow in time.Β  Consider reducing the IO load on the system or choose a shadow copy storage volume that is not being shadow copied.

As the error indicates, the shadow copy was deleted when the shadow copy could not grow. Since Veeam Agent for Microsoft Windows was reading from that shadow copy, the backup immediately fails with,β€œThe system cannot find the file specified. Asynchronous read operation failed.” The underlying cause, as explained in the error, is that the disk’s IO load was too high. The solution is to reduce the disk IO load, implement faster storage, or redirect shadow copies to a volume with less IO load. Customers most commonly select redirection of shadow copies, as it requires no infrastructure changes.

  • The next most common Volsnap error reported by customers is:

    Event ID: 36
    

    Description: The shadow copies of volume X: were aborted because the shadow copy storage could not grow due to a user imposed limit.

Like Event ID:25 above, the shadow copy could not grow and instead was deleted. This again causes the Veeam Agent for Microsoft Windows backup job to fail with the error, **β€œThe system cannot find the file specified. Asynchronous read operation failed.”**In this case, the resolution is to review the shadow copy storage limitations or redirect shadow copies to a volume with more storage space.

Additional examples of Volsnap errors can be found in the More Information section of this article.

Manage Shadow Copy Settings

The most common solution for almost all issues is redirecting shadow copy storage to a volume with sufficient free space and plenty of available IOPS.

Click a section below to expand and view instructions.

Configure Shadow Copies Using the Shadow Copies Utility (Server OS Only)

Shadow Copies Utility (Server OS Only)
  1. Right-click any volume and choose **Configure Shadow Copies. ****Note:**On older server operating systems, this may instead be a tab in Properties.
  2. Select the volume mentioned in the volsnap error.
  3. Click Settings… 4. Review and update theStorage area settings.
    * If the volsnap error that occurred implies that there is an issue with the storage, try assigning the shadow copy storage to a drive that is expected to have less IO load.
    * If the volsnap error mentions a β€œuser defined limit” or β€œuser imposed limit”, update the Maximum size setting to large value or set no limit.

servershadows


Volume Shadow Copy Service supports a volume size of up to 64 TB, larger volumes cannot be selected as storage area. Manual changes may cause software malfunction.

Configure Shadow Copies Using the Command Line (Desktop and Server OS)

**Note:**All commands below must be run from an Administrative Command Prompt.

View Existing Shadow Storage Associations

Run the following command to view existing shadow storage associations.
If no shadow storage exists for any volume, the command response will be "No items found that satisfy the query."

vssadmin list shadowstorage

Copy

Add Shadow Storage Associations

If no association is listed for the volume, use the following command to change the location of the shadow storage area:

vssadmin add shadowstorage /for=X: /on=Z: /maxsize=##GB

Copy

For example, to redirect shadows from D: to E:

vssadmin add shadowstorage /for=D: /on=E: /maxsize=50GB
Resize Shadow Storage Association

If a shadow storage association already exists, use the following command to change the size shadow storage area:

vssadmin resize shadowstorage /for=X: /on=Z: /maxsize=###GB

Copy

For example, to resize the shadow storage in place for a redirection from from D: to E:

vssadmin resize shadowstorage /for=D: /on=E: /maxsize=200GB

If no value is specified for /maxsize, there is no limit placed on the amount of storage space that can be used.
Otherwise, a value of 15-20% of the source volume's size is generally recommended for busy servers.

More Information

Common Volsnap Errors Reported by Customers

  • Event ID: 5 – Description: The shadow copy of volume E: could not be created due to insufficient non-paged memory pool for a bitmap structure.
  • Event ID: 8– Description: The flush and hold writes operation on volume D: timed out while waiting for a release writes command.
  • Event ID: 9– Description: The flush and hold writes operation on volume E: timed out while waiting for file system cleanup.
  • Event ID: 14– Description: The shadow copies of volume E: were aborted because of an IO failure on volume R:.

In this example, the shadow copies for E: were redirected to R:, but the underlying storage for R: was failing.

  • Event ID: 16– Description: The shadow copies of volume R: were aborted because volume , which contains shadow copy storage for this shadow copy, was force dismounted.

In this example, the volume where the shadow copies were being stored for R: was dismounted during the backup operation (i.e., drive failure).

  • Event ID: 20– Description: The shadow copies of volume S: were aborted because of a failed free space computation.

  • Event ID: 24– Description: There was insufficient disk space on volume C: to grow the shadow copy storage for shadow copies of C:. As a result of this failure all shadow copies of volume C: are at risk of being deleted.

  • Event ID: 25 – Description: The shadow copies of volume T: were deleted because the shadow copy storage could not grow in time. Consider reducing the IO load on the system or choose a shadow copy storage volume that is not being shadow copied.

In this example, the T: drive was where the exchange transaction logs were stored and the IO load on the volume was too high to grow the shadow copy, resulting in the need to redirect shadow copies to a different volume.

  • Event ID: 27 – Description: The shadow copies of volume L: were aborted during detection because a critical control file could not be opened.
  • Event ID: 28 – Description: The shadow copy of volume C: could not be created due to a failure in creating the necessary on disk structures.
  • Event ID: 32 – Description: The shadow copies of volume H: were aborted because the shadow copy storage volume was not present.
  • Event ID: 33 – Description: The oldest shadow copy of volume C: was deleted to keep disk space usage for shadow copies of volume C: below the user defined limit.
  • Event ID: 35 – Description: The shadow copies of volume C: were aborted because the shadow copy storage failed to grow.
  • Event ID: 36 – Description: The shadow copies of volume F: were aborted because the shadow copy storage could not grow due to a user imposed limit.
  • Event ID: 39 – Description: When preparing a new volume shadow copy for volume C:, the shadow copy storage on volume C: could not be located in non-critical space. Consider using a shadow copy storage volume that does not have any shadow copies.
  • Event ID: 41 – Description: When preparing a new volume shadow copy for volume C:, the shadow copy storage on volume C: did not have sufficiently large contiguous blocks. Consider deleting unnecessary files on the shadow copy storage volume or use a different shadow copy storage volume.

To submit feedback regarding this article, please click this link: Send Article Feedback
To report a typo on this page, highlight the typo with your mouse and press CTRL + Enter.

AI Score

7

Confidence

High