SSD Over-provisioning using hdparm

From Thomas-Krenn-Wiki
Jump to navigation Jump to search

Solid-State Drives (SSDs) have a so-called spare area, a data area that is not directly visible to the operating system. The SSD uses this spare area for wear leveling, among other things. Increasing the size of the spare area (over-provisioning) can increase performance and the life cycle of an SSD. This article will show how to increase the size of the spare area of a SATA SSD using hdparm under Linux by enabling the host protected area (ATA8-ACS SET MAX ADDRESS). The net usable capacity of the SSD will be reduced by doing so. The spare area section of the article about solid-state drives presents alternative methods for over-provisioning and other information.

Preparation

Before increasing the size of the spare area, all SSD blocks must be deleted. Only in this manner can the SSD controller actually use the hidden data areas for wear leveling.

All SSD blocks will have been deleted, when:

Enabling the Host Protected Area

Warning: Only enable the Host Protected Area (HPA) on SSDs that do not contain any data. Otherwise, there is the threat of data loss.

Under normal circumstances, the host protected area is disabled. Thus, the SSD, in this example an Intel 320 Series 160 GB SSD, displays its normal net capacity (displayed in groups of 512-byte sectors). 

root@ubuntu-10-10:~# hdparm -N /dev/sdb

/dev/sdb:
 max sectors   = 312581808/312581808, HPA is disabled
root@ubuntu-10-10:~#

These 312,581,808 sectors multiplied by 512 correspond to exactly 160,041,885,696 bytes, thus the normal net capacity of this SSD, precisely.

hdparm -Np enables the host protected area. In doing this, the number of visible remaining sectors is specified after the –Np flag (without a space between). The --yes-i-know-what-i-am-doing flag and device name are required as additional parameters. 

root@ubuntu-10-10:~# hdparm -Np281323627 /dev/sdb

/dev/sdb:
 setting max visible sectors to 281323627 (permanent)
Use of -Nnnnnn is VERY DANGEROUS.
You have requested reducing the apparent size of the drive.
This is a BAD idea, and can easily destroy all of the drive's contents.
Please supply the --yes-i-know-what-i-am-doing flag if you really want this.
Program aborted.
root@ubuntu-10-10:~# hdparm -Np281323627 --yes-i-know-what-i-am-doing /dev/sdb

/dev/sdb:
 setting max visible sectors to 281323627 (permanent)
 max sectors   = 281323627/312581808, HPA is enabled

root@ubuntu-10-10:~#

Calling hdparam –N again confirms that the HPA has been unlocked. 

root@ubuntu-10-10:~# hdparm -N /dev/sdb

/dev/sdb:
 max sectors   = 281323627/312581808, HPA is enabled
root@ubuntu-10-10:~#

HPA Unlocked Problem

Many Linux distribution files enable the libata parameter, ignore_hpa.[1] By doing so, the HPA will be disabled when the computer is next started or hot-plugging (for eSATA), which can lead to many problems.[2]

Whether ignore_hpa has been enabled or not can be determined by using the following command. 

cat /sys/module/libata/parameters/ignore_hpa

If the HPA will automatically be disabled as soon as an SSD with an HPA has been connected, "1" will be returned. The following example displays the /var/log/messages file for Ubuntu 10.10. The SSD has been connected to it using eSATA. The fourth line shows the disabling of the HPA (HPA unlocked). 

Jun  1 10:07:17 ubuntu-10-10 kernel: [  116.644752] ata6: hard resetting link
Jun  1 10:07:18 ubuntu-10-10 kernel: [  117.583941] ata6: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
Jun  1 10:07:18 ubuntu-10-10 kernel: [  117.584645] ata6.00: ACPI cmd f5/00:00:00:00:00:a0 (SECURITY FREEZE LOCK) filtered out
Jun  1 10:07:18 ubuntu-10-10 kernel: [  117.606303] ata6.00: HPA unlocked: 281323627 -> 312581808, native 312581808
Jun  1 10:07:18 ubuntu-10-10 kernel: [  117.606310] ata6.00: ATA-8: INTEL SSDSA2CW160G3, 4PC10302, max UDMA/133
Jun  1 10:07:18 ubuntu-10-10 kernel: [  117.606314] ata6.00: 312581808 sectors, multi 1: LBA48 NCQ (depth 31/32)
Jun  1 10:07:18 ubuntu-10-10 kernel: [  117.607145] ata6.00: ACPI cmd f5/00:00:00:00:00:a0 (SECURITY FREEZE LOCK) filtered out
Jun  1 10:07:18 ubuntu-10-10 kernel: [  117.607576] ata6.00: configured for UDMA/133
Jun  1 10:07:18 ubuntu-10-10 kernel: [  117.608514] ata6.00: configured for UDMA/133
Jun  1 10:07:18 ubuntu-10-10 kernel: [  117.608522] ata6: EH complete
Jun  1 10:07:18 ubuntu-10-10 kernel: [  117.608699] scsi 5:0:0:0: Direct-Access     ATA      INTEL SSDSA2CW16 4PC1 PQ: 0 ANSI: 5
Jun  1 10:07:18 ubuntu-10-10 kernel: [  117.608953] sd 5:0:0:0: Attached scsi generic sg2 type 0
Jun  1 10:07:18 ubuntu-10-10 kernel: [  117.608992] sd 5:0:0:0: [sdb] 312581808 512-byte logical blocks: (160 GB/149 GiB)
Jun  1 10:07:18 ubuntu-10-10 kernel: [  117.609079] sd 5:0:0:0: [sdb] Write Protect is off
Jun  1 10:07:18 ubuntu-10-10 kernel: [  117.609128] sd 5:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Jun  1 10:07:18 ubuntu-10-10 kernel: [  117.609373]  sdb:
Jun  1 10:07:18 ubuntu-10-10 kernel: [  117.610090] sd 5:0:0:0: [sdb] Attached SCSI disk

Affected Linux Systems

Many Linux distribution files will be affected by this behavior, including:

  • Ubuntu 10.10

A patch from Tejun Heo (with on-demand HPA unlocking) will resolve this problem.[3][4] Newer Linux kernels and Linux distribution files have already integrated this patch, including:

  • Ubuntu 11.04

The use of the HPA (HPA detected) will work with Ubuntu 11.04. 

root@ubuntu:~# tail -f /var/log/kern.log
Jun  1 08:54:32 ubuntu kernel: [  213.433902] ata7: exception Emask 0x10 SAct 0x0 SErr 0x4050000 action 0xe frozen
Jun  1 08:54:32 ubuntu kernel: [  213.433909] ata7: irq_stat 0x00400040, connection status changed
Jun  1 08:54:32 ubuntu kernel: [  213.433914] ata7: SError: { PHYRdyChg CommWake DevExch }
Jun  1 08:54:32 ubuntu kernel: [  213.433926] ata7: hard resetting link
Jun  1 08:54:32 ubuntu kernel: [  214.182197] ata7: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
Jun  1 08:54:32 ubuntu kernel: [  214.182631] ata7.00: HPA detected: current 281323627, native 312581808
Jun  1 08:54:32 ubuntu kernel: [  214.182640] ata7.00: ATA-8: INTEL SSDSA2CW160G3, 4PC10302, max UDMA/133
Jun  1 08:54:32 ubuntu kernel: [  214.182645] ata7.00: 281323627 sectors, multi 1: LBA48 NCQ (depth 31/32)
Jun  1 08:54:32 ubuntu kernel: [  214.183140] ata7.00: configured for UDMA/133
Jun  1 08:54:32 ubuntu kernel: [  214.202174] ata7: EH complete
Jun  1 08:54:32 ubuntu kernel: [  214.202317] scsi 6:0:0:0: Direct-Access     ATA      INTEL SSDSA2CW16 4PC1 PQ: 0 ANSI: 5
Jun  1 08:54:32 ubuntu kernel: [  214.202601] sd 6:0:0:0: [sdd] 281323627 512-byte logical blocks: (144 GB/134 GiB)
Jun  1 08:54:32 ubuntu kernel: [  214.202679] sd 6:0:0:0: Attached scsi generic sg4 type 0
Jun  1 08:54:32 ubuntu kernel: [  214.202761] sd 6:0:0:0: [sdd] Write Protect is off
Jun  1 08:54:32 ubuntu kernel: [  214.202768] sd 6:0:0:0: [sdd] Mode Sense: 00 3a 00 00
Jun  1 08:54:32 ubuntu kernel: [  214.202823] sd 6:0:0:0: [sdd] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Jun  1 08:54:32 ubuntu kernel: [  214.203555]  sdd:
Jun  1 08:54:32 ubuntu kernel: [  214.204000] sd 6:0:0:0: [sdd] Attached SCSI disk
^C
root@ubuntu:~# cat /proc/partitions
major minor  #blocks  name

   7        0    1258132 loop0
   8        0  976762584 sda
   8        1   19530752 sda1
   8        2          1 sda2
   8        5    1998848 sda5
   8        6  878905344 sda6
   8       48  140661813 sdd
root@ubuntu:~# fdisk -l /dev/sdd

Disk /dev/sdd: 144.0 GB, 144037697024 bytes
255 heads, 63 sectors/track, 17511 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xafd4e895

   Device Boot      Start         End      Blocks   Id  System
root@ubuntu:~#

Automatically Preventing HPA Disabling for Affected Linux Systems

Temporarily

ignore_hpa can be temporarily disabled through the sysfs file system. This temporary change can be helpful, if an SSD with an HPA has been quickly connected to a computer. For SSDs that were already connected when the computer was started, a permanent solution will be required. The temporary setting will remain valid until the next reboot. 

root@ubuntu-10-10:~# cd /sys/module/libata/parameters
root@ubuntu-10-10:/sys/module/libata/parameters# cat ignore_hpa 
1
root@ubuntu-10-10:/sys/module/libata/parameters# echo 0 > ignore_hpa 
root@ubuntu-10-10:/sys/module/libata/parameters# cat ignore_hpa 
0
root@ubuntu-10-10:/sys/module/libata/parameters#

The HPA will now be used correctly (HPA detected: current 281323627, native 312581808): 

Jun  1 11:20:13 ubuntu-10-10 kernel: [   70.824850] ata6: hard resetting link
Jun  1 11:20:14 ubuntu-10-10 kernel: [   71.764776] ata6: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
Jun  1 11:20:14 ubuntu-10-10 kernel: [   71.765573] ata6.00: ACPI cmd f5/00:00:00:00:00:a0 (SECURITY FREEZE LOCK) filtered out
Jun  1 11:20:14 ubuntu-10-10 kernel: [   71.765980] ata6.00: HPA detected: current 281323627, native 312581808
Jun  1 11:20:14 ubuntu-10-10 kernel: [   71.765989] ata6.00: ATA-8: INTEL SSDSA2CW160G3, 4PC10302, max UDMA/133
Jun  1 11:20:14 ubuntu-10-10 kernel: [   71.765993] ata6.00: 281323627 sectors, multi 1: LBA48 NCQ (depth 31/32)
Jun  1 11:20:14 ubuntu-10-10 kernel: [   71.766845] ata6.00: ACPI cmd f5/00:00:00:00:00:a0 (SECURITY FREEZE LOCK) filtered out
Jun  1 11:20:14 ubuntu-10-10 kernel: [   71.767284] ata6.00: configured for UDMA/133
Jun  1 11:20:14 ubuntu-10-10 kernel: [   71.768148] ata6.00: configured for UDMA/133
Jun  1 11:20:14 ubuntu-10-10 kernel: [   71.768157] ata6: EH complete
Jun  1 11:20:14 ubuntu-10-10 kernel: [   71.768374] scsi 5:0:0:0: Direct-Access     ATA      INTEL SSDSA2CW16 4PC1 PQ: 0 ANSI: 5
Jun  1 11:20:14 ubuntu-10-10 kernel: [   71.768663] sd 5:0:0:0: [sdb] 281323627 512-byte logical blocks: (144 GB/134 GiB)
Jun  1 11:20:14 ubuntu-10-10 kernel: [   71.768679] sd 5:0:0:0: Attached scsi generic sg2 type 0
Jun  1 11:20:14 ubuntu-10-10 kernel: [   71.768772] sd 5:0:0:0: [sdb] Write Protect is off
Jun  1 11:20:14 ubuntu-10-10 kernel: [   71.768807] sd 5:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Jun  1 11:20:14 ubuntu-10-10 kernel: [   71.769104]  sdb:
Jun  1 11:20:14 ubuntu-10-10 kernel: [   71.770897] sd 5:0:0:0: [sdb] Attached SCSI disk

Permanently

Depending on whether libdata has been compiled into the kernel or exists as a module, ignore_hpa can be permanently disabled as follows.

For libdata in the kernel:[5]

  • Supplement the libata.ignore_hpa=0 kernel flag in the boot loader configuration file

For libdata as a module:[6]

  • bash -c 'echo options libata ignore_hpa=0 > /etc/modprobe.d/libata.conf' (as root or via sudo)

References

  1. ↑ (PATCH 1/3) ide: add "ignore_hpa" module parameter
  2. ↑ Bug #380138 in linux (Ubuntu): “Do NOT disable HPA by default -> leads to data loss” (bugs.launchpad.net)
  3. ↑ (PATCH 8/8) libata: implement on-demand HPA unlocking (Linux Kernel Mailing List)
  4. ↑ libata: implement on-demand HPA unlocking (kerneltrap.org)
  5. ↑ Bug #380138 in linux (Ubuntu): “Do NOT disable HPA by default -> leads to data loss” (bugs.launchpad.net), Comment 13
  6. ↑ Bug #380138 in linux (Ubuntu): “Do NOT disable HPA by default -> leads to data loss” (bugs.launchpad.net), Comment 15


Foto Werner Fischer.jpg

Author: Werner Fischer

Werner Fischer, working in the Knowledge Transfer team at Thomas-Krenn, completed his studies of Computer and Media Security at FH Hagenberg in Austria. He is a regular speaker at many conferences like LinuxTag, OSMC, OSDC, LinuxCon, and author for various IT magazines. In his spare time he enjoys playing the piano and training for a good result at the annual Linz marathon relay.


Related articles

M.2 and mSATA SSD Support of Thomas-Krenn Servers
Show article
Perform a SSD Secure Erase
Show article
Western Digital SN640 firmware updates R1110021 and R1410004
Show article