SSD Over-provisioning using hdparm
Solid-State Drives (SSDs) have a so-called spare area, a data area that is not directly visible to the operating system. The SSD uses this spare area for wear leveling, among other things. Increasing the size of the spare area (over-provisioning) can increase performance and the life cycle of an SSD. This article will show how to increase the size of the spare area of a SATA SSD using hdparm under Linux by enabling the host protected area (ATA8-ACS SET MAX ADDRESS
). The net usable capacity of the SSD will be reduced by doing so. The spare area section of the article about solid-state drives presents alternative methods for over-provisioning and other information.
Preparation
Before increasing the size of the spare area, all SSD blocks must be deleted. Only in this manner can the SSD controller actually use the hidden data areas for wear leveling.
All SSD blocks will have been deleted, when:
- the SSD is in its delivery state or
- a secure erase or manual TRIM (depending on the specific SSD) has been performed
Enabling the Host Protected Area
Warning: Only enable the Host Protected Area (HPA) on SSDs that do not contain any data. Otherwise, there is the threat of data loss.
Under normal circumstances, the host protected area is disabled. Thus, the SSD, in this example an Intel 320 Series 160 GB SSD, displays its normal net capacity (displayed in groups of 512-byte sectors).
root@ubuntu-10-10:~# hdparm -N /dev/sdb /dev/sdb: max sectors = 312581808/312581808, HPA is disabled root@ubuntu-10-10:~#
These 312,581,808 sectors multiplied by 512 correspond to exactly 160,041,885,696 bytes, thus the normal net capacity of this SSD, precisely.
hdparm -Np enables the host protected area. In doing this, the number of visible remaining sectors is specified after the –Np flag (without a space between). The --yes-i-know-what-i-am-doing flag and device name are required as additional parameters.
root@ubuntu-10-10:~# hdparm -Np281323627 /dev/sdb /dev/sdb: setting max visible sectors to 281323627 (permanent) Use of -Nnnnnn is VERY DANGEROUS. You have requested reducing the apparent size of the drive. This is a BAD idea, and can easily destroy all of the drive's contents. Please supply the --yes-i-know-what-i-am-doing flag if you really want this. Program aborted. root@ubuntu-10-10:~# hdparm -Np281323627 --yes-i-know-what-i-am-doing /dev/sdb /dev/sdb: setting max visible sectors to 281323627 (permanent) max sectors = 281323627/312581808, HPA is enabled root@ubuntu-10-10:~#
Calling hdparam –N again confirms that the HPA has been unlocked.
root@ubuntu-10-10:~# hdparm -N /dev/sdb /dev/sdb: max sectors = 281323627/312581808, HPA is enabled root@ubuntu-10-10:~#
HPA Unlocked Problem
Many Linux distribution files enable the libata parameter, ignore_hpa.[1] By doing so, the HPA will be disabled when the computer is next started or hot-plugging (for eSATA), which can lead to many problems.[2]
Whether ignore_hpa has been enabled or not can be determined by using the following command.
cat /sys/module/libata/parameters/ignore_hpa
If the HPA will automatically be disabled as soon as an SSD with an HPA has been connected, "1" will be returned. The following example displays the /var/log/messages file for Ubuntu 10.10. The SSD has been connected to it using eSATA. The fourth line shows the disabling of the HPA (HPA unlocked).
Jun 1 10:07:17 ubuntu-10-10 kernel: [ 116.644752] ata6: hard resetting link Jun 1 10:07:18 ubuntu-10-10 kernel: [ 117.583941] ata6: SATA link up 3.0 Gbps (SStatus 123 SControl 300) Jun 1 10:07:18 ubuntu-10-10 kernel: [ 117.584645] ata6.00: ACPI cmd f5/00:00:00:00:00:a0 (SECURITY FREEZE LOCK) filtered out Jun 1 10:07:18 ubuntu-10-10 kernel: [ 117.606303] ata6.00: HPA unlocked: 281323627 -> 312581808, native 312581808 Jun 1 10:07:18 ubuntu-10-10 kernel: [ 117.606310] ata6.00: ATA-8: INTEL SSDSA2CW160G3, 4PC10302, max UDMA/133 Jun 1 10:07:18 ubuntu-10-10 kernel: [ 117.606314] ata6.00: 312581808 sectors, multi 1: LBA48 NCQ (depth 31/32) Jun 1 10:07:18 ubuntu-10-10 kernel: [ 117.607145] ata6.00: ACPI cmd f5/00:00:00:00:00:a0 (SECURITY FREEZE LOCK) filtered out Jun 1 10:07:18 ubuntu-10-10 kernel: [ 117.607576] ata6.00: configured for UDMA/133 Jun 1 10:07:18 ubuntu-10-10 kernel: [ 117.608514] ata6.00: configured for UDMA/133 Jun 1 10:07:18 ubuntu-10-10 kernel: [ 117.608522] ata6: EH complete Jun 1 10:07:18 ubuntu-10-10 kernel: [ 117.608699] scsi 5:0:0:0: Direct-Access ATA INTEL SSDSA2CW16 4PC1 PQ: 0 ANSI: 5 Jun 1 10:07:18 ubuntu-10-10 kernel: [ 117.608953] sd 5:0:0:0: Attached scsi generic sg2 type 0 Jun 1 10:07:18 ubuntu-10-10 kernel: [ 117.608992] sd 5:0:0:0: [sdb] 312581808 512-byte logical blocks: (160 GB/149 GiB) Jun 1 10:07:18 ubuntu-10-10 kernel: [ 117.609079] sd 5:0:0:0: [sdb] Write Protect is off Jun 1 10:07:18 ubuntu-10-10 kernel: [ 117.609128] sd 5:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA Jun 1 10:07:18 ubuntu-10-10 kernel: [ 117.609373] sdb: Jun 1 10:07:18 ubuntu-10-10 kernel: [ 117.610090] sd 5:0:0:0: [sdb] Attached SCSI disk
Affected Linux Systems
Many Linux distribution files will be affected by this behavior, including:
- Ubuntu 10.10
A patch from Tejun Heo (with on-demand HPA unlocking) will resolve this problem.[3][4] Newer Linux kernels and Linux distribution files have already integrated this patch, including:
- Ubuntu 11.04
The use of the HPA (HPA detected) will work with Ubuntu 11.04.
root@ubuntu:~# tail -f /var/log/kern.log Jun 1 08:54:32 ubuntu kernel: [ 213.433902] ata7: exception Emask 0x10 SAct 0x0 SErr 0x4050000 action 0xe frozen Jun 1 08:54:32 ubuntu kernel: [ 213.433909] ata7: irq_stat 0x00400040, connection status changed Jun 1 08:54:32 ubuntu kernel: [ 213.433914] ata7: SError: { PHYRdyChg CommWake DevExch } Jun 1 08:54:32 ubuntu kernel: [ 213.433926] ata7: hard resetting link Jun 1 08:54:32 ubuntu kernel: [ 214.182197] ata7: SATA link up 3.0 Gbps (SStatus 123 SControl 300) Jun 1 08:54:32 ubuntu kernel: [ 214.182631] ata7.00: HPA detected: current 281323627, native 312581808 Jun 1 08:54:32 ubuntu kernel: [ 214.182640] ata7.00: ATA-8: INTEL SSDSA2CW160G3, 4PC10302, max UDMA/133 Jun 1 08:54:32 ubuntu kernel: [ 214.182645] ata7.00: 281323627 sectors, multi 1: LBA48 NCQ (depth 31/32) Jun 1 08:54:32 ubuntu kernel: [ 214.183140] ata7.00: configured for UDMA/133 Jun 1 08:54:32 ubuntu kernel: [ 214.202174] ata7: EH complete Jun 1 08:54:32 ubuntu kernel: [ 214.202317] scsi 6:0:0:0: Direct-Access ATA INTEL SSDSA2CW16 4PC1 PQ: 0 ANSI: 5 Jun 1 08:54:32 ubuntu kernel: [ 214.202601] sd 6:0:0:0: [sdd] 281323627 512-byte logical blocks: (144 GB/134 GiB) Jun 1 08:54:32 ubuntu kernel: [ 214.202679] sd 6:0:0:0: Attached scsi generic sg4 type 0 Jun 1 08:54:32 ubuntu kernel: [ 214.202761] sd 6:0:0:0: [sdd] Write Protect is off Jun 1 08:54:32 ubuntu kernel: [ 214.202768] sd 6:0:0:0: [sdd] Mode Sense: 00 3a 00 00 Jun 1 08:54:32 ubuntu kernel: [ 214.202823] sd 6:0:0:0: [sdd] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA Jun 1 08:54:32 ubuntu kernel: [ 214.203555] sdd: Jun 1 08:54:32 ubuntu kernel: [ 214.204000] sd 6:0:0:0: [sdd] Attached SCSI disk ^C root@ubuntu:~# cat /proc/partitions major minor #blocks name 7 0 1258132 loop0 8 0 976762584 sda 8 1 19530752 sda1 8 2 1 sda2 8 5 1998848 sda5 8 6 878905344 sda6 8 48 140661813 sdd root@ubuntu:~# fdisk -l /dev/sdd Disk /dev/sdd: 144.0 GB, 144037697024 bytes 255 heads, 63 sectors/track, 17511 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0xafd4e895 Device Boot Start End Blocks Id System root@ubuntu:~#
Automatically Preventing HPA Disabling for Affected Linux Systems
Temporarily
ignore_hpa can be temporarily disabled through the sysfs file system. This temporary change can be helpful, if an SSD with an HPA has been quickly connected to a computer. For SSDs that were already connected when the computer was started, a permanent solution will be required. The temporary setting will remain valid until the next reboot.
root@ubuntu-10-10:~# cd /sys/module/libata/parameters root@ubuntu-10-10:/sys/module/libata/parameters# cat ignore_hpa 1 root@ubuntu-10-10:/sys/module/libata/parameters# echo 0 > ignore_hpa root@ubuntu-10-10:/sys/module/libata/parameters# cat ignore_hpa 0 root@ubuntu-10-10:/sys/module/libata/parameters#
The HPA will now be used correctly (HPA detected: current 281323627, native 312581808):
Jun 1 11:20:13 ubuntu-10-10 kernel: [ 70.824850] ata6: hard resetting link Jun 1 11:20:14 ubuntu-10-10 kernel: [ 71.764776] ata6: SATA link up 3.0 Gbps (SStatus 123 SControl 300) Jun 1 11:20:14 ubuntu-10-10 kernel: [ 71.765573] ata6.00: ACPI cmd f5/00:00:00:00:00:a0 (SECURITY FREEZE LOCK) filtered out Jun 1 11:20:14 ubuntu-10-10 kernel: [ 71.765980] ata6.00: HPA detected: current 281323627, native 312581808 Jun 1 11:20:14 ubuntu-10-10 kernel: [ 71.765989] ata6.00: ATA-8: INTEL SSDSA2CW160G3, 4PC10302, max UDMA/133 Jun 1 11:20:14 ubuntu-10-10 kernel: [ 71.765993] ata6.00: 281323627 sectors, multi 1: LBA48 NCQ (depth 31/32) Jun 1 11:20:14 ubuntu-10-10 kernel: [ 71.766845] ata6.00: ACPI cmd f5/00:00:00:00:00:a0 (SECURITY FREEZE LOCK) filtered out Jun 1 11:20:14 ubuntu-10-10 kernel: [ 71.767284] ata6.00: configured for UDMA/133 Jun 1 11:20:14 ubuntu-10-10 kernel: [ 71.768148] ata6.00: configured for UDMA/133 Jun 1 11:20:14 ubuntu-10-10 kernel: [ 71.768157] ata6: EH complete Jun 1 11:20:14 ubuntu-10-10 kernel: [ 71.768374] scsi 5:0:0:0: Direct-Access ATA INTEL SSDSA2CW16 4PC1 PQ: 0 ANSI: 5 Jun 1 11:20:14 ubuntu-10-10 kernel: [ 71.768663] sd 5:0:0:0: [sdb] 281323627 512-byte logical blocks: (144 GB/134 GiB) Jun 1 11:20:14 ubuntu-10-10 kernel: [ 71.768679] sd 5:0:0:0: Attached scsi generic sg2 type 0 Jun 1 11:20:14 ubuntu-10-10 kernel: [ 71.768772] sd 5:0:0:0: [sdb] Write Protect is off Jun 1 11:20:14 ubuntu-10-10 kernel: [ 71.768807] sd 5:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA Jun 1 11:20:14 ubuntu-10-10 kernel: [ 71.769104] sdb: Jun 1 11:20:14 ubuntu-10-10 kernel: [ 71.770897] sd 5:0:0:0: [sdb] Attached SCSI disk
Permanently
Depending on whether libdata has been compiled into the kernel or exists as a module, ignore_hpa can be permanently disabled as follows.
For libdata in the kernel:[5]
- Supplement the
libata.ignore_hpa=0
kernel flag in the boot loader configuration file
For libdata as a module:[6]
bash -c 'echo options libata ignore_hpa=0 > /etc/modprobe.d/libata.conf'
(as root or via sudo)
References
- ↑ (PATCH 1/3) ide: add "ignore_hpa" module parameter
- ↑ Bug #380138 in linux (Ubuntu): “Do NOT disable HPA by default -> leads to data loss” (bugs.launchpad.net)
- ↑ (PATCH 8/8) libata: implement on-demand HPA unlocking (Linux Kernel Mailing List)
- ↑ libata: implement on-demand HPA unlocking (kerneltrap.org)
- ↑ Bug #380138 in linux (Ubuntu): “Do NOT disable HPA by default -> leads to data loss” (bugs.launchpad.net), Comment 13
- ↑ Bug #380138 in linux (Ubuntu): “Do NOT disable HPA by default -> leads to data loss” (bugs.launchpad.net), Comment 15
Author: Werner Fischer Werner Fischer, working in the Knowledge Transfer team at Thomas-Krenn, completed his studies of Computer and Media Security at FH Hagenberg in Austria. He is a regular speaker at many conferences like LinuxTag, OSMC, OSDC, LinuxCon, and author for various IT magazines. In his spare time he enjoys playing the piano and training for a good result at the annual Linz marathon relay.
|