T ECH NI C AL W HIT E P A P ERNimble Storage Security Technical Note:SmartSecure Software-Based Encryption

Document /2015Revision1. (author)Draft release (Bill Roth)Draft 2 (Bill Roth)Draft 3 (Bill Roth)Published version 1(Bill Roth)Published version 2 (Bill Roth)THIS BEST PRACTICE GUIDE IS FOR INFORMATIONAL PURPOSES ONLY, AND MAY CONTAINTYPOGRAPHICAL ERRORS AND TECHNICAL INACCURACIES. THE CONTENT IS PROVIDED ASIS, WITHOUT EXPRESS OR IMPLIED WARRANTIES OF ANY KIND.Nimble Storage: All rights reserved. Reproduction of this material in any manner whatsoever without theexpress written permission of Nimble Storage is strictly prohibited.NIMBLE STORAGE W HITE PAPER2

Table of ContentsINTRODUCTION . 5Audience . 5Assumptions . 5OVERVIEW . 5Terminology. 6Implementation . 6DEPLOYMENT GUIDELINES . 7Enabling Encryption . 7Default Setting .10Scope .11New Volume .11System Startup Mode .12Replication .15Clones .16Role Based Administration Privileges .16Alerts .16LIMITATIONS .17Existing Volumes .17Disabling Encryption .18Data Shredding .18GROUP MERGE .18REFERENCES .20SUMMARY .20NIMBLE STORAGE W HITE PAPER3

List of FiguresFigure 1: Administration-Security-Encryption . 7Figure 2: Data Encryption . 8Figure 3: Passphrase. 8Figure 4: Warning Dialog . 9Figure 5: Testing Passphrase Recovery . 9Figure 6: Passphrase Settings Saved .10Figure 7: Failed to Authenticate.10Figure 8: Default Setting .11Figure 9: Scope .11Figure 10: Create Volume with Encryption Enabled and Force Default Setting.12Figure 11: Create Volume with Encryption Enabled and Allow Override Setting .12Figure 12: System Startup Mode .13Figure 13: Encryption Not Active Message .13Figure 14: Enter Passphrase .13Figure 15: Encrypted Volume Offline.14Figure 16: Encryption Key Inactive .14Figure 17: Enable Master Key .15Figure 18: Nimble Connection Manager .15Figure 19: Alert – Encryption Deactivated .17Figure 20: Alert – Configuration Altered .17Figure 21: Delete Master Key Failure .18Figure 22: Delete Master Key Success .18Figure 23: Group Merge Error .19Figure 24: Enter Passphrase .19NIMBLE STORAGE W HITE PAPER4

IntroductionAudienceNimble Storage and security administrators are encouraged to read this document. Therecommendations presented set out to assist in deploying a supported, successful, and reliable solution.Assumptions General knowledge and familiarity with Nimble Storage, the Nimble Storage user interface, and basicsetup tasks. An understanding of the encryption and security requirements for a given product deployment.OverviewThe Nimble Storage SmartSecure software based encryption feature is available for arrays runningNimble OS version 2.3 or higher. Data encryption uses the AES-256-XTS cipher for cryptographic protection of data on block orientedstorage devices. Performance optimized, the implementation leverages the Intel AES-NI instruction set on later modelCS series arrays. Data compression occurs prior to encryption, preserving capacity savings. Is selectively deployable on a volume-by-volume or array group basis. Provides two modes of operation: oSecure system startup mode, where a passphrase must be entered after an array restart.oAvailable system startup mode, where a passphrase does not need to be entered after anarray restart.Support includes:oNimble Storage Scale Out configurations with multiple arrays in a group.oNimble Storage volume collection cloning and volume collection replication.Validated to FIPS (Federal Information Processing Standards) 140-2 level 1 certificationThe Nimble Storage SmartSecure software based encryption feature ensures secrecy of data within thearray by encrypting volumes using the AES-256-XTS cipher. This protects against the theft of the arrayitself, or against theft of components within the array such as a disk drive. The feature is implemented insoftware within the Nimble operating system, and takes advantage of the Intel AES-NI instruction set onlater model CS series arrays. The implementation also incorporates flexibility, where only specific datavolumes may employ encryption, or an entire array group is encrypted. Additional flexibility is providedwithin the implementation with secure and available system startup modes, where after an array restartthe passphrase is either required or not required before volumes with encrypted data are accessible.NIMBLE STORAGE W HITE PAPER5

TerminologyPortions of the encryption terminology used in this document may introduce new or unfamiliar conceptsfor some readers. At a high level, this subsection defines encryption related terminology to assist incomprehension of subsequent content. Cipher - An algorithm for performing encryption or decryption. For example, AES (AdvancedEncryption Standard) is a cipher. Additional examples of ciphers include but are not limited to; DES(Data Encryption Standard), RC5 (Rivest Cipher 5), and Blowfish. AES-256-KeyWrap – An algorithm that provides security to protect encryption keys within the contextof a key management architecture. For example, AES-256-KeyWrap is used for secure transmissionof encryption keys over a network connection. AES-256-XTS – A block cipher based disk encryption scheme that makes use of two different keys of256 bits each resulting in a combined 512 bit key. OpenSSL – An open source implementation of the SSL (Secure Sockets Layer) and TLS (TransportLayer Security) protocols designed to provide secure communications over a network connection. SHA-256 – A cryptographic hash function, SHA-256 (Secure Hash Algorithm-256) is used todetermine data integrity by comparing a known hash value to the computed hash value of a given file. Passphrase – Similar to a password, a passphrase is used to control access to encrypted dataresiding on a Nimble Storage array that has the encryption feature enabled. A passphrase mayconsist of 8 to 64 printable ASCII characters. The passphrase is used to encrypt the master key. Master Key – A 256 bit encryption key generated by seeding the OpenSSL random numbergenerator. The master encryption key is used to encrypt or decrypt all other encryption keys. Volume Key – A 256 bit encryption key generated by seeding the OpenSSL random numbergenerator. New encrypted volumes, or volumes cloned from snapshots of encrypted volumes areassigned a new volume encryption key. Key Table – A table structure internal to the Nimble Storage array where all keys are encrypted withthe master key using the AES-256-KeyWrap algorithm.ImplementationThe master encryption key plays an important role as it is used to encrypt or decrypt all other encryptionkeys used within the Nimble Storage SmartSecure software based encryption feature. The passphrasealso plays an important role as it is used to encrypt the master key. Taking a deeper look at the masterencryption key, it is important to understand how the master encryption key value is generated, protectedfrom unauthorized access, and recovered after an array restart.Master encryption key generation occurs when the Nimble Storage encryption feature is enabled. Atinitialization time, the user must input a passphrase consisting of 8 to 64 printable ASCII characters. Thepassphrase is used to generate a SHA-256 hash. The master encryption key generation process seedsthe OpenSSL random number generator using 256 bits of pseudo-random data output from“/dev/urandom”. The resulting master encryption key is then encrypted with AES-256-KeyWrap using thepassphrase hash.The passphrase is never stored within the Nimble Storage array. It is the responsibility of the arrayadministrative team to keep track of the passphrase. The master encryption key is stored in an encryptedNIMBLE STORAGE W HITE PAPER6

state in the Key Table, a Postgres table internal to the Nimble Operating System. Pieces of the masterkey can exist in array RAM allocated to certain processes to allow key access during normal operationsor during a failover event.Deployment GuidelinesPlease be sure to read the “Nimble OS Release Notes” document associated with the running version ofthe Nimble Storage operating system. The “Nimble OS Release Notes” document is available fordownload on the Nimble Storage InfoSight web portal.Enabling EncryptionBy default encryption is disabled. The administrator role privilege set is required to enable the dataencryption feature. To enable encryption click the “Administration” menu item and then select “Security”from the pull down menu. Select “Encryption” from the available options.Figure 1: Administration-Security-EncryptionThe “Encryption” dialog display will appear.NIMBLE STORAGE W HITE PAPER7

Figure 2: Data EncryptionEntering a passphrase and clicking the “Save” button will enable data encryption.The passphrase may consist of 8 to 64 printable ASCII characters. Selecting the checkbox ”Show typing”,allows the user to see the passphrase characters that are being typed.Figure 3: PassphraseWhen the “Save” button is clicked a warning dialog window will appear.NIMBLE STORAGE W HITE PAPER8

Figure 4: Warning DialogThe warning dialog should be taken seriously. Clicking the “I accept” button enables encryption. The“Cancel” button allows the user to leave the encryption feature in an uninitialized state.Record the passphrase and retain it in a secure location as determined by site procedures. It is theresponsibility of the user to maintain the passphrase forever. Regardless of the encryption settingsselected for use, access to the passphrase will be required at some point in the future.The passphrase is not stored within the array. The passphrase is not transmitted to Nimble Storagetechnical support by means of the AutoSupport process. The array does not generate copies of thepassphrase in Email Alerts, SNMP, or Syslog.Important note: Some passphrase storage utilities may not be capable of accommodating up to 64 ASCIIcharacters. If you plan on using a passphrase storage utility, it is strongly recommended to test the abilityto recover the passphrase from the utility prior to creating encrypted volumes. For instance, attempt tomodify the existing passphrase. In the “Modify Passphrase” dialog window enter the existing passphrasein the “Current”, “New”, and “Confirm” fields. Click the “Modify Passphrase” button to continue.Figure 5: Testing Passphrase RecoveryIf passphrase retrieval was successful, a message indicating “Passphrase settings saved” will be posted.NIMBLE STORAGE W HITE PAPER9

Figure 6: Passphrase Settings SavedIn the event that passphrase retrieval was not successful, an error dialog will appear indicating thatchanging the passphrase failed.Figure 7: Failed to AuthenticateIf changing the passphrase failed indicating that passphrase retrieval was not successful, see the sectiontitled, “Disabling Encryption” later in this document. Disabling encryption will return the encryption featureto an uninitialized state. Starting over from an uninitialized state allows a new passphrase to be set.Outside the scope of testing passphrase retrieval, the existing passphrase can be changed based on siterequirements using the same technique outlined in this section. When the passphrase is changed, themaster key is decrypted using the current passphrase and is then re-encrypted using the newpassphrase.The implications associated with a lost or forgotten passphrase should be well understood: If the passphrase is lost and the array has been configured to use the “Secure” system startupmode, power cycling the array or rebooting the array will place all encrypted volumes into an offlinestate. If the passphrase is lost, the data in these volumes becomes permanently inaccessible.Consider changing the system startup mode to “Available” to mitigate this issue should a power cycleor reboot event occur. Plan to copy or migrate data on encrypted volumes to new unencryptedvolumes. The encryption feature will need to be set to an uninitialized state in order to create a newpassphrase. If the passphrase is lost there is no ability to modify the passphrase to a known value. In order tomodify the passphrase, the current passphrase must be provided. Plan to copy or migrate data onencrypted volumes to new unencrypted volumes. The encryption feature will need to be set to anuninitialized state in order to create a new passphrase.Default SettingThe “Default Setting”, “Enable encryption on newly created volumes (Cipher: AES-256-XTS)”, defines thedefault encryption setting that will be used on all newly created volumes.NIMBLE STORAGE W HITE PAPER10

Figure 8: Default SettingThe default value for the “Default Setting” is enabled (check marked).ScopeThe “Scope” setting is used to either enforce the “Default Setting” or to allow overriding the “DefaultSetting”.Figure 9: ScopeWhen selected, the “Force the default setting to be applied to all new volumes in the group” radio buttonenforces the previously mentioned “Default Setting” where encryption is either enabled or disabled on allnewly created volumes. The “Allow overriding the default setting on a per-volume basis” setting allowsencryption to be selectively enabled or disabled on a new volume when it is created.The following table clarifies product behavior based on the “Default Setting” selection and the nScopeAllow OverrideForce DefaultSettingAllow OverrideForce DefaultSettingResultEncryption is enabled by default. Encryption can be disabled atvolume creation time.Encryption is enabled by default and cannot be disabled at volumecreation time.Encryption is disabled by default. Encryption can be enabled atvolume creation time.Encryption is disabled by default and cannot be enabled at volumecreation time.Table 1: Default Setting / Scope MatrixNew VolumeData encryption is either enabled or disabled on a volume only at volume creation time. An existingvolume that is not encrypted cannot be altered to enable encryption. Similarly, a volume that is encryptedcannot be altered to disable encryption. Each new encrypted volume gets a new volume key assigned toit at creation time.NIMBLE STORAGE W HITE PAPER11

When creating a new volume, the “Data Encryption” property will either be enabled or disabled based onthe value of the “Default Setting” parameter. The ability to enable or disable encryption at volume creationtime is dependent on the “Scope” parameter setting.Figure 10: Create Volume with Encryption Enabled and Force Default SettingThe example shown above has the “Default Setting” enabled to enable encryption on new volumes bydefault. The “Scope” setting parameter has been set to “Force the default setting to be applied to all newvolumes”. As a result, the “Data Encryption” parameter is enabled and cannot be altered.Figure 11: Create Volume with Encryption Enabled and Allow Override SettingThe example shown above has the “Default Setting” enabled to enable encryption on new volumes bydefault. The “Scope” setting parameter has been set to “Allow overriding the default setting on a pervolume basis”. The “Data Encryption” parameter is enabled and can be altered.System Startup ModeIn the event of an array restart, powering on the array for instance, master encryption key access will bedifferent based on the “System Startup Mode” configuration setting. By default, the available systemstartup mode is enabled. This does not negate the requirement to maintain the passphrase forever. In theNIMBLE STORAGE W HITE PAPER12

available system startup mode an array restart or powering on the array will result in all encryptedvolumes being set to an online state. Known exceptions to this are: Controller upgrades – If controllers are being swapped the passphrase must be entered to enableaccess to encrypted volumes NVRAM loss – In the rare scenario of NVRAM loss, which includes component failure or completebattery discharge, the passphrase must be entered to enable access to encrypted volumesWhen secure system startup mode is enabled, the user must input the passphrase so that the systemcan decrypt the master key. There is no access to encrypted volumes on the array until the passphrasehas been entered.Figure 12: System Startup ModeAfter an array restart in secure system startup mode, a message will indicate that encryption is not active.Clicking the highlighted “Enter passphrase” portion of the message allows the passphrase to be entered.Figure 13: Encryption Not Active MessageAfter clicking the “Enter Passphrase” portion of the message, a dialog window will appear where thepassphrase can be entered.Figure 14: Enter PassphraseAlternatively, the passphrase can be entered by means of the command line interface (CLI) using thesyntax, “encryptkey --enable master”, which will prompt the user for the passphrase.NIMBLE STORAGE W HITE PAPER13

“System Startup Mode” selection should be carefully considered. If secure system startup mode isenabled, all encrypted volumes are in an offline state following an array restart or power on event. Afterentering the passphrase encrypted volumes enter an online state on the array, however these volumesmay be in a disconnected state on the host(s) they are normally connected to.Figure 15: Encrypted Volume OfflineShown above is an example of an encrypted volume in an offline state. “System Startup Mode” was set toenable secure mode and the array was rebooted. Note that an alert message is present indicating thatencryption is not active. Moving the cursor over the offline volume provides additional detail, indicatingthat the encryption key is not active.Figure 16: Encryption Key InactiveEntering the passphrase changes the state of encrypted volumes to online from the perspective of thearray.NIMBLE STORAGE W HITE PAPER14

Figure 17: Enable Master KeyLooking at the Nimble Connection Manager (NCM) on the host using this volume, the encrypted volumeis no longer connected to the host.Figure 18: Nimble Connection ManagerIn the example above, the encrypted volume requires manual reconnection to the host following a securesystem startup mode reboot of the array. The effect of using the secure system startup mode may varydependent on the connection type, (Fiber Channel or iSCSI), as well as the host operating system typeand version.ReplicationReplication of encrypted volumes requires that data encryption is enabled on both replication partners.Data blocks are replicated in their compressed and encrypted state. Volume keys for encrypted volumesbeing replicated are securely transmitted to the partner array by first encrypting them with AES-256KeyWrap. The wrapping key is generated using a secure SSL transaction that is authenticated using thepartner arrays shared secret.Volume collection replication of encrypted volumes is administered the same way in which unencryptedvolume collection replication is administered. A volume collection may contain both encrypted andunencrypted volumes. The replica volume(s) maintain their original encrypted or unencrypted property.NIMBLE STORAGE W HITE PAPER15

ClonesWhen encrypted volumes are cloned, the new cloned volume is also encrypted. A new volume key isgenerated for the cloned volume. Cloned volumes have access to their ancestors’ volume key in order toread shared data blocks. New data blocks written to an encrypted cloned volume are encrypted using thenew volume key.Role Based Administration PrivilegesDifferent administrative capabilities are available for the data encryption feature based on the role of theuser. The following table defines the capabilities available for the “Administrator”, “Power User”,“Operator”, and “Guest” roles:RoleView ter AdministratorPower UserOperatorGuestTable 2: Administrative PrivilegesImportant notes: The “Administrator” role has full privileges for the SmartSecure software based encryption feature.This includes the ability to delete the master key. The “Power User” and “Operator” roles can enter the passphrase after a system restart in securesystem startup mode. The “Guest” role has no privileges whatsoever.AlertsA variety of alerts are generated automatically when the data encryption feature is enabled, theencryption configuration is altered, or after a system restart in the secure system startup mode.NIMBLE STORAGE W HITE PAPER16

Figure 19: Alert – Encryption DeactivatedFigure 20: Alert – Configuration AlteredLimitationsExisting VolumesThere is no ability to change an unencrypted volume to an encrypted volume. Similarly, there is no abilityto change an encrypted volume to an unencrypted volume. The encrypted state of a volume, encrypted orunencrypted, is only configurable at volume creation time.One strategy that may be applicable to changing the encrypted state of data is to copy the existingvolume to a newly created volume with the desired encryption state. The copy operation is performedusing a host system that has both the old and new volumes connected to it. For example, copy anunencrypted volume to a new encrypted volume. It may also be possible to use data migration tools torelocate data from a source volume to a destination volume, VMware vMotion for example.NIMBLE STORAGE W HITE PAPER17

Disabling EncryptionOnce data encryption is enabled, it can only be disabled by means of the command line interface. Allexisting encrypted volumes must be deleted before encryption can be disabled. The command,“encryptkey –master delete” deletes the master encryption key and places the encryption feature into anuninitialized state.Figure 21: Delete Master Key FailureIf disabling encryption is a business requirement, encrypted volumes containing data that is consideredvaluable or important should be copied to unencrypted volumes. The tasks necessary to copy data froman encrypted volume to an unencrypted volume will vary based on the file system and data type. Forexample, it may be possible to migrate guests within a VMware datastore with vMotion, it may also bepossible to copy data by mounting encrypted and unencrypted volumes to a host and manually initiating acopy operation. Alternatively, encrypted volumes that do not contain valuable or important data can be setto an offline state and deleted prior to disabling the data encryption feature.Figure 22: Delete Master Key SuccessData ShreddingWhen an encrypted volume is set offline and deleted, the corresponding volume key is marked inactive.The Nimble operating system will not permit access to inactive keys. Although an inactive volume keymay still be present in the key table, it is stored encrypted by the master key with the AES-256-KeyWrapalgorithm. In effect the data associated with the deleted volume is not accessible. Over time inactive keysin the key table will be removed.Group MergeThis section looks at the behavior of the data encryption feature when moving pools or volumes betweengroups. Because encryption can be enabled at the group or volume level, and because the settings ingroups can be different, it is important to understand what will happen when a pool or volume leaves oneNIMBLE STORAGE W HITE PAPER18

group and joins a different group. Four possible merge scenarios have been outlined to assist inunderstanding what will occur in each use case. In each use case, “Group A” is the source group and“Group B” is the destination group. Use case 1: Group A (Encryption Enabled) is merged into Group B (Encryption Disabled)oThis use case is not supported. Group B must have encryption enabled prior to adding GroupA.Figure 23: Group Merge Error Use case 2: Group A (Encryption Disabled) is merged into Group B (Encryption Enabled)oPools/volumes on Group A with encryption disabled remain unencrypted after they aremerged into Group B.oPools/volumes on Group B with encryption enabled

The Nimble Storage SmartSecure software based encryption feature ensures secrecy of data within the array by encrypting volumes using the AES-256-XTS cipher. This protects against the theft of the array itself, or against theft of components within the