Initialize Amazon EBS volumes - Amazon EBS

Initialize Amazon EBS volumes

Empty EBS volumes receive their maximum performance the moment that they are created and do not require initialization (formerly known as pre-warming).

For volumes, of any volume type, that were created from snapshots, the storage blocks must be pulled down from Amazon S3 and written to the volume before you can access them. This preliminary action takes time and can cause a significant increase in the latency of I/O operations the first time each block is accessed. Volume performance is achieved after all blocks have been downloaded and written to the volume.

Important

While initializing Provisioned IOPS SSD volumes that were created from snapshots, the performance of the volume may drop below 50 percent of its expected level, which causes the volume to display a warning state in the I/O Performance status check. This is expected, and you can ignore the warning state on Provisioned IOPS SSD volumes while you are initializing them. For more information, see Amazon EBS volume status checks.

For most applications, amortizing the initialization cost over the lifetime of the volume is acceptable. To avoid this initial performance hit in a production environment, you can use one of the following options:

  • Force the immediate initialization of the entire volume. For more information, see Linux instances (Linux instances) or Windows instances (Windows instances).

  • Enable fast snapshot restore on a snapshot to ensure that the EBS volumes created from it are fully-initialized at creation and instantly deliver all of their provisioned performance. For more information, see Amazon EBS fast snapshot restore.

To initialize a volume created from a snapshot on Linux

  1. Attach the newly-restored volume to your Linux instance.

  2. Use the lsblk command to list the block devices on your instance.

    $ lsblk
NAME  MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
xvdf  202:80   0  30G  0 disk
xvda1 202:1    0   8G  0 disk /

    Here you can see that the new volume, /dev/xvdf, is attached, but not mounted (because there is no path listed under the MOUNTPOINT column).

  3. Use the dd or fio utilities to read all of the blocks on the device. The dd command is installed by default on Linux systems, but fio is considerably faster because it allows multi-threaded reads.

    Note

    This step may take several minutes up to several hours, depending on your EC2 instance bandwidth, the IOPS provisioned for the volume, and the size of the volume.

    [dd] The if (input file) parameter should be set to the drive you wish to initialize. The of (output file) parameter should be set to the Linux null virtual device, /dev/null. The bs parameter sets the block size of the read operation; for optimal performance, this should be set to 1 MB.

    Important

    Incorrect use of dd can easily destroy a volume's data. Be sure to follow precisely the example command below. Only the if=/dev/xvdf parameter will vary depending on the name of the device you are reading.

    $ sudo dd if=/dev/xvdf of=/dev/null bs=1M status=progress

    [fio] If you have fio installed on your system, use the following command to initialize your volume. The --filename (input file) parameter should be set to the drive you wish to initialize.

    $ sudo fio --filename=/dev/xvdf --rw=read --bs=1M --iodepth=32 --ioengine=libaio --direct=1 --name=volume-initialize

    To install fio on Amazon Linux, use the following command:

    sudo yum install -y fio

    To install fio on Ubuntu, use the following command:

    sudo apt-get install -y fio

    When the operation is finished, you will see a report of the read operation. Your volume is now ready for use. For more information, see Make an Amazon EBS volume available for use.

Before using either tool, gather information about the disks on your system as follows:

To gather information about the system disks

  1. Use the wmic command to list the available disks on your system:

    wmic diskdrive get size,deviceid

    The following is example output:

    DeviceID            Size
\\.\PHYSICALDRIVE2  80517265920
\\.\PHYSICALDRIVE1  80517265920
\\.\PHYSICALDRIVE0  128849011200
\\.\PHYSICALDRIVE3  107372805120

  2. Identify the disk to initialize using dd or fio. The C: drive is on \\.\PHYSICALDRIVE0. You can use the diskmgmt.msc utility to compare drive letters to disk drive numbers if you are not sure which drive number to use.

Use the dd utility

Complete the following procedures to install and use dd to initialize a volume.

Important considerations

  • Initializing a volume takes from several minutes up to several hours, depending on your EC2 instance bandwidth, the IOPS provisioned for the volume, and the size of the volume.

  • Incorrect use of dd can easily destroy a volume's data. Be sure to follow this procedure precisely.

To install dd for Windows

The dd for Windows program provides a similar experience to the dd program that is commonly available for Linux and Unix systems, and it enables you to initialize Amazon EBS volumes that have been created from snapshots. The most recent beta versions support the /dev/null virtual device. If you install an earlier version, you can use the nul virtual device instead. Full documentation is available at http://www.chrysocome.net/dd.

  1. Download the most recent binary version of dd for Windows from http://www.chrysocome.net/dd.

  2. (Optional) Create a folder for command line utilities that is easy to locate and remember, such as C:\bin. If you already have a designated folder for command line utilities, you can use that folder instead in the following step.

  3. Unzip the binary package and copy the dd.exe file to your command line utilities folder (for example, C:\bin).

  4. Add the command line utilities folder to your Path environment variable so you can run the programs in that folder from anywhere.

    1. Choose Start, open the context (right-click) menu for Computer, and then choose Properties.

    2. Choose Advanced system settings, Environment Variables.

    3. For System Variables, select the variable Path and choose Edit.

    4. For Variable value, append a semicolon and the location of your command line utility folder (;C:\bin\) to the end of the existing value.

    5. Choose OK to close the Edit System Variable window.

  5. Open a new command prompt window. The previous step doesn't update the environment variables in your current command prompt windows. The command prompt windows that you open now that you completed the previous step are updated.

To initialize a volume using dd for Windows

Run the following command to read all blocks on the specified device (and send the output to the /dev/null virtual device). This command safely initializes your existing data.

dd if=\\.\PHYSICALDRIVEn of=/dev/null bs=1M --progress --size

You might get an error if dd attempts to read beyond the end of the volume. You can safely ignore this error.

If you used an earlier version of the dd command, it does not support the /dev/null device. Instead, you can use the nul device as follows.

dd if=\\.\PHYSICALDRIVEn of=nul bs=1M --progress --size
Use the fio utility

Complete the following procedures to install and use fio to initialize a volume.

To install fio for Windows

The fio for Windows program provides a similar experience to the fio program that is commonly available for Linux and Unix systems, and it allows you to initialize Amazon EBS volumes created from snapshots. For more information, see https://github.com/axboe/fio.

  1. Download the fio MSI installer by expanding Assets for the latest release and selecting the MSI installer.

  2. Install fio.

To initialize a volume using fio for Windows

  1. Run a command similar to the following to initialize a volume:

    fio --filename=\\.\PHYSICALDRIVEn  --rw=read --bs=128k --iodepth=32 --direct=1 --name=volume-initialize

  2. When the operation completes, you are ready to use your new volume. For more information, see Make an Amazon EBS volume available for use.