Encryption at Rest¶

Introduction¶

ScyllaDB protects your sensitive data with data-at-rest encryption. It protects the privacy of your user’s data, reduces the risk of data breaches, and helps meet regulatory requirements. In particular, it provides an additional level of protection for your data persisted in storage or its backups.

When ScyllaDB’s Encryption at Rest is used together with Encryption in Transit (Node to Node and Client to Node), you benefit from end to end data encryption.

About Encryption at Rest¶

The following can be encrypted:

• ScyllaDB persistent tables (SSTables)

• System level data, such as:

  • Commit logs

  • Batches

  • hints logs

  • KMIP Password (part of scylla.yaml)

Encryption at Rest works at table level granularity, so you can choose to encrypt only sensitive tables. For both system and table data, you can use different algorithms that are supported by OpenSSL in a file block encryption scheme.

Encryption Key Types¶

Two types of encryption keys are available: System Keys and Table Keys.

Key Providers¶

When encrypting the system tables or SSTables, you need to state which provider is holding your keys. You can use the following options:

Cipher Algorithms¶

The following cipher_algorithims are available for use with ScyllaDB using OpenSSL. Note that the default algorithm (AES/CBC/PKCS5Padding with key strength 128 ) is recommended.

Create Encryption Keys¶

Depending on your key provider, you will either have the option of allowing ScyllaDB to generate an encryption key, or you will have to provide one:

• KMIP Key Provider - you don’t need to generate any key yourself

• KMS Key Provider - you must generate a key yourself in AWS

• Replicated Key Provider - you must generate a system key yourself

• Local Key Provider - If you do not generate your own secret key, ScyllaDB will create one for you

Set the KMIP Host¶

If you are using KMIP to encrypt tables or system information, add the KMIP server information to the scylla.yaml configuration file.

1. Edit the scylla.yaml file located in /etc/scylla/ and add the following in KMIP host(s) section:

   #
   # kmip_hosts:
   #   <name>:
   #       hosts: <address1[:port]> [, <address2[:port]>...]
   #       certificate: <identifying certificate> (optional)
   #       keyfile: <identifying key> (optional; it is required if "certificate" is set)
   #       truststore: <truststore for SSL connection> (optional)
   #       certficate_revocation_list: <CRL file> (optional)
   #       priority_string: <kmip tls priority string>
   #       username: <login> (optional>
   #       password: <password> (optional)
   #       max_command_retries: <int> (optional; default 3)
   #       key_cache_expiry: <key cache expiry period>
   #       key_cache_refresh: <key cache refresh/prune period>
   #   <name>:
   

   Where:

   • <name> - The cluster name.

   • hosts - The list of hosts specified by IP and port for the KMIP server. The KMIP connection management only supports failover, so all requests go through a single KMIP server. There is no load balancing, as currently no KMIP servers support read replication or other strategies for availability. Hosts are tried in the order they appear, and the next one in the list is tried if the previous one fails. The default number of retries is three, but you can customize it with “max_command_retries”.

   • certificate - The name of the certificate and path used to identify yourself to the KMIP server.

   • keyfile - The name of the key used to identify yourself to the KMIP server. It is generated together with the certificate.

   • truststore - The location and key for the truststore to present to the KMIP server.

   • certficate_revocation_list - The path to a PEM-encoded certificate revocation list (CRL) - a list of issued certificates that have been revoked before their expiration date.

   • priority_string - The KMIP TLS priority string.

   • username - The KMIP server user name.

   • password - The KMIP server password.

   • max_command_retries - The number of attempts to connect to the KMIP server before trying the next host in the list.

   • key_cache_expiry - Key cache expiry period, after which keys will be re-requested from server. Default is 600s.

   • key_cache_refresh - Key cache refresh period - the frequency at which cache is checked for expired entries. Default is 1200s.

2. Save the file.

3. Drain the node with nodetool drain

4. Restart the scylla-server service.

sudo systemctl restart scylla-server

docker exec -it some-scylla supervisorctl restart scylla

Set the KMS Host¶

If you are using AWS KMS to encrypt tables or system information, add the KMS information to the scylla.yaml configuration file.

1. Edit the scylla.yaml file located in /etc/scylla/ to add the following in KMS host(s) section:

    kms_hosts:
      <name>:
          endpoint: http(s)://<host>(:port) (optional if `aws_region` is specified)
          aws_region: <aws region> (optional if `endpoint` is specified)
          aws_access_key_id: <aws access key id> (optional)
          aws_secret_access_key: <aws secret access key> (optional)
          aws_profile: <aws credentials profile to use> (optional)
          aws_use_ec2_credentials: (bool : default false)
          aws_use_ec2_region: (bool : default false)
          aws_assume_role_arn: <arn of aws role to assume before call> (optional)
          master_key: <named KMS key for encrypting data keys> (required)
          certificate: <identifying certificate> (optional)
          keyfile: <identifying key> (optional)
          truststore: <truststore for SSL connection> (optional)
          priority_string: <KMS TLS priority string> (optional)
          key_cache_expiry: <key cache expiry period>
          key_cache_refresh: <key cache refresh/prune period>
   #   <name>:
   

   Where:

   • <name> - The name to identify the KMS host. You have to provide this name to encrypt a new or existing table.

   • endpoint - The explicit KMS host endpoint. If not provided, aws_region is used for connection.

   • aws_region - An AWS region. If not provided, endpoint is used for connection.

   • aws_access_key_id - AWS access key used for authentication. If not specified, the provider reads it from your AWS credentials.

   • aws_secret_access_key - AWS secret access key used for authentication. If not specified, the provider reads it from your AWS credentials.

   • aws_profile - AWS profile to use if reading credentials from file

   • aws_use_ec2_credentials - If true, KMS queries will use the credentials provided by ec2 instance role metadata as initial access key.

   • aws_use_ec2_region - If true, KMS queries will use the AWS region indicated by ec2 instance metadata.

   • aws_assume_role_arn - If set, any KMS query will first attempt to assume this role.

   • master_key - The ID or alias of your AWS KMS key. The key must be generated with an appropriate access policy so that the AWS user has permissions to read the key and encrypt data using that key. This parameter is required.

   • certificate - The name of the certificate and the path used to identify yourself to the KMS server.

   • keyfile - The name of the key for the certificate. It is generated together with the certificate.

   • truststore - The location and key for the truststore to present to the KMS server.

   • priority_string - The KMS TLS priority string.

   • key_cache_expiry - Key cache expiry period, after which keys will be re-requested from server. Default is 600s.

   • key_cache_refresh - Key cache refresh period - the frequency at which cache is checked for expired entries. Default is 1200s.

   Note

   Note that either endpoint, aws_region or aws_use_ec2_region must be set (one of them is required for connection).

   Example:

   kms_hosts:
     my-kms1:
         aws_use_ec2_credentials: true
         aws_use_ec2_region: true
         master_key: myorg/MyKey
   

2. Save the file.

3. Drain the node with nodetool drain

4. Restart the scylla-server service.

sudo systemctl restart scylla-server

docker exec -it some-scylla supervisorctl restart scylla

Set the GCP Host¶

If you are using Google GCP KMS to encrypt tables or system information, add the GCP information to the scylla.yaml configuration file.

1. Edit the scylla.yaml file located in /etc/scylla/ to add the following in KMS host(s) section:

    gcp_hosts:
      <name>:
          gcp_project_id: <gcp project>
          gcp_location: <gcp location>
          gcp_credentials_file: <(service) account json key file - authentication>
          gcp_impersonate_service_account: <service account to impersonate>
          master_key: <keyring>/<keyname> - named GCP key for encrypting data keys (required)
          certificate: <identifying certificate> (optional)
          keyfile: <identifying key> (optional)
          truststore: <truststore for SSL connection> (optional)
          priority_string: <KMS TLS priority string> (optional)
          key_cache_expiry: <key cache expiry period>
          key_cache_refresh: <key cache refresh/prune period>
   #   <name>:
   

   Where:

   • <name> - The name to identify the GCP host. You have to provide this name to encrypt a new or existing table.

   • gcp_project_id - The GCP project from which to retrieve key information.

   • gcp_location - A GCP project location.

   • gcp_credentials_file - GCP credentials file used for authentication. If not specified, the provider reads it from your GCP credentials.

   • gcp_impersonate_service_account - An optional service account to impersonate when issuing key query calls.

   • master_key - The <keyring>/<keyname> of your GCP KMS key. The key must be generated with an appropriate access policy so that the AWS user has permissions to read the key and encrypt data using that key. This parameter is required.

   • certificate - The name of the certificate and the path used to identify yourself to the KMS server.

   • keyfile - The name of the key for the certificate. It is generated together with the certificate.

   • truststore - The location and key for the truststore to present to the KMS server.

   • priority_string - The KMS TLS priority string.

   • key_cache_expiry - Key cache expiry period, after which keys will be re-requested from server. Default is 600s.

   • key_cache_refresh - Key cache refresh period - the frequency at which cache is checked for expired entries. Default is 1200s.

   Example:

   gcp_hosts:
     my-gcp1:
         gcp_project_id: myproject
         gcp_location: global
         master_key: mykeyring/mykey
   

2. Save the file.

3. Drain the node with nodetool drain

4. Restart the scylla-server service.

sudo systemctl restart scylla-server

docker exec -it some-scylla supervisorctl restart scylla

Encrypt Tables¶

ScyllaDB allows you to enable or disable default encryption of tables. When enabled, tables will be encrypted by default using the configuration provided for the user_info_encryption option in the scylla.yaml file.

You can override the default configuration when you CREATE TABLE or ALTER TABLE with scylla_encryption_options. See Encrypt a Single Table for details.

Before you Begin

Ensure you have an encryption key available:

• If you are using AWS KMS, set the KMS Host.

• If you are using KMIP, set the KMIP Host.

• If you are using Google GCP KMS, set the GCP Host.

• If you want to create your own key, follow the procedure in Create Encryption Keys.

• If you do not create your own key, use the following procedure for ScyllaDB to create a key for you (the default location /etc/scylla/data_encryption_keys may cause permission issues; the following example creates a key in the directory /etc/scylla/encryption_keys):

  sudo mkdir -p /etc/scylla/encryption_keys
  sudo chown -R scylla:scylla /etc/scylla/encryption_keys
  sudo chmod -R 700 /etc/scylla/encryption_keys
  

Procedure

Edit the scylla.yaml file located in /etc/scylla/ and configure the user_info_encryption option:

user_info_encryption:
 enabled: <true|false>
 cipher_algorithm: <hashing algorithm to create the key>
 secret_key_strength: <length of the key>
 key_provider: <your key provider>
 secret_key_file: <key file>
 kmip_host: <your kmip_host>
 kms_host: <your kms_host>
 gcp_host: <your gcp_host>

Where:

• enabled - Enables or disables default table encryption. Required.

• cipher_algorithm - One of the cipher algorithms. If not provided, the default will be used.

• secret_key_strength - The length of the key in bytes ( determined by the cipher algorithms you choose). If not provided, the default will be used.

• key_provider - The name of the key provider. See Key Providers. Required.

• secret_key_file - The location of the key created by ScyllaDB (by default /etc/scylla/data_encryption_keys). Required if you use a ScyllaDB-generated key.

• kmip_host - The name of your kmip_host group. Required if you use KMIP.

• kms_host - The name of your kms_host group. Required if you use KMS.

• gcp_host - The name of your gcp_host group. Required if you use GCP.

Example

user_info_encryption:
 enabled: true
 cipher_algorithm: AES
 secret_key_strength: 128
 key_provider: LocalFileSystemKeyProviderFactory
 secret_key_file: scylla /etc/scylla/encryption_keys

Examples for KMS:

In the following example, the master_key configured for kms_host will be used.

user_info_encryption:
 enabled: true
 key_provider: KmsKeyProviderFactory
 kms_host: my-kms1

You can specify a different master_key than the one configured for kms_host:

user_info_encryption:
 enabled: true
 key_provider: KmsKeyProviderFactory
 kms_host: my-kms1
 master_key: myorg/SomeOtherKey

Encrypt a Single Table¶

This procedure demonstrates how to encrypt a new table.

Before you Begin

• Make sure to Set the KMIP Host if you are using KMIP, or the the KMS Host if you are using AWS KMS.

• If you want to make your own key, use the procedure in Create Encryption Keys and skip to step 3. If you do not create your own key, ScyllaDB will create one for you in the secret_key_file path. If you are not creating your own key, start with step 1.

Procedure

1. By default, the encryption key is located in the /etc/scylla/ directory, and the file is named data_encryption_keys. If you want to save the key in a different directory, create one. This example will create encryption keys in a different directory (/etc/scylla/encryption_keys, for example), which ensures that the owner of this directory is scylla and not another user.

   Note

   Using the default location results in a known permission issue (scylladb/scylla-tools-java#94), so it is recommended to use another location as described in the example.

   sudo mkdir -p /etc/scylla/encryption_keys
   sudo chown -R scylla:scylla /etc/scylla/encryption_keys
   sudo chmod -R 700 /etc/scylla/encryption_keys
   

2. Create the keyspace if it doesn’t exist.

3. Create the table using the CREATE TABLE CQL statement, adding any additional options. To encrypt the table, use the options for encryption below, remembering to set the secret_key_file <path> to the same directory you created in step 1.

   CREATE TABLE <keyspace>.<table_name> (...<columns>...) WITH
     scylla_encryption_options = {
       'cipher_algorithm' : <hash>,
       'secret_key_strength' : <len>,
       'key_provider': <provider>,
       'secret_key_file': <path>
     }
   ;
   

   Where:

   • cipher_algorithm - The hashing algorithm which is to be used to create the key. See Cipher Algorithms for more information.

   • secret_key_strength - The length of the key in bytes. This is determined by the cipher you choose. See Cipher Algorithms for more information.

   • key_provider is the name or type of key provider. Refer to Key Providers for more information.

   • secret_key_file - the location that ScyllaDB will store the key it creates (if one does not exist in this location) or the location of the key. By default the location is /etc/scylla/data_encryption_keys.

   Example:

   Continuing the example from above, this command will instruct ScyllaDB to encrypt the table and will save the key in the location created in step 1.

   CREATE TABLE data.atrest (pk text primary key, c0 int) WITH
     scylla_encryption_options = {
       'cipher_algorithm' : 'AES/ECB/PKCS5Padding',
       'secret_key_strength' : 128,
       'key_provider': 'LocalFileSystemKeyProviderFactory',
       'secret_key_file': '/etc/scylla/encryption_keys/data_encryption_keys'
     }
   ;
   

   Example for KMS:

   CREATE TABLE myks.mytable (...<columns>...) WITH
     scylla_encryption_options = {
       'cipher_algorithm' :  'AES/CBC/PKCS5Padding',
       'secret_key_strength' : 128,
       'key_provider': 'KmsKeyProviderFactory',
       'kms_host': 'my-kms1'
     }
   ;
   

   You can skip cipher_algorithm and secret_key_strength (the defaults will be used):

   CREATE TABLE myks.mytable (...<columns>...) WITH
     scylla_encryption_options = {
       'key_provider': 'KmsKeyProviderFactory',
       'kms_host': 'my-kms1'
     }
   ;
   

   You can specify a different master key than the one configured for kms_host in the scylla.yaml file:

   CREATE TABLE myks.mytable (...<columns>...) WITH
     scylla_encryption_options = {
       'key_provider': 'KmsKeyProviderFactory',
       'kms_host': 'my-kms1',
       'master_key':'myorg/SomeOtherKey'
     }
   ;
   

4. From this point, every new SSTable created for the atrest table is encrypted, using the data_encryption_keys key located in /etc/scylla/encryption_keys/. This table will remain encrypted with this key until you either change the key, change the key properties, or disable encryption.

5. To ensure all SSTables for this table on every node are encrypted, run the Nodetool upgradesstables command. If not, the SSTables remain unencrypted until they are compacted or flushed from MemTables.

   For Example:

   nodetool upgradesstables data atrest
   

6. Your SSTables are encrypted. If you want to change the key at any point, use the Update Encryption Properties of Existing Tables procedure. Always keep your key in a safe location known to you. Do not lose it. See When a Key is Lost.

ALTER TABLE ks.test WITH
  scylla_encryption_options = {
     'cipher_algorithm' : 'AES/ECB/PKCS5Padding',
     'secret_key_strength' : 128,
     'key_provider': 'LocalFileSystemKeyProviderFactory',
     'secret_key_file': '/etc/scylla/encryption_keys/data_encryption_keys'
  }
;

ALTER TABLE ks.test WITH
  scylla_encryption_options = {
     'cipher_algorithm' : 'AES/ECB/PKCS5Padding',
     'secret_key_strength' : 192,
     'key_provider': 'LocalFileSystemKeyProviderFactory',
     'secret_key_file': '/etc/scylla/encryption_keys/data_encryption_keys'
  }
;

ALTER TABLE ks.test WITH
   scylla_encryption_options =  { 'key_provider' : 'none’ };

Encrypt System Resources¶

System encryption is applied to semi-transient on-disk data, such as commit logs, batch logs, and hinted handoff data. This ensures that all temporarily stored data is encrypted until fully persisted to final SSTable on disk. Once this encryption is enabled, it is used for all system data.

Procedure

1. Edit the scylla.yaml file - located in /etc/scylla/scylla.yaml and add the following:

   system_info_encryption:
      enabled: <true|false>
      key_provider: (optional) <key provider type>
      system_key_directory: <path to location of system key>
   

   Where:

   • enabled can be true or false. True is enabled; false is disabled.

   • key_provider is the name or type of key provider. Refer to Key Providers for more information.

   • cipher_algorithm is one of the supported Cipher Algorithms.

   • secret_key_file is the name of the key file containing the secret key (key.pem, for example)

   Example:

   system_info_encryption:
      enabled: True
      cipher_algorithm: AES
      secret_key_strength: 128
      key_provider: LocalFileSystemKeyProviderFactory
      secret_key_file: /path/to/systemKey.pem
   

   Example for KMIP:

   system_info_encryption:
      enabled: True
      cipher_algorithm: AES
      secret_key_strength: 128
      key_provider: KmipKeyProviderFactory
      kmip_host:  yourkmipServerIP.com
   

   Where kmip_host is the address for your KMIP server.

   Example for KMS:

   system_info_encryption:
      enabled: True
      cipher_algorithm: AES/CBC/PKCS5Padding
      secret_key_strength: 128
      key_provider: KmsKeyProviderFactory
      kms_host: myScylla
   

   Where kms_host is the unique name of the KMS host specified in the scylla.yaml file.

   Example for GCP:

   system_info_encryption:
      enabled: True
      cipher_algorithm: AES/CBC/PKCS5Padding
      secret_key_strength: 128
      key_provider: GcpKeyProviderFactory
      gcp_host: myScylla
   

   Where gcp_host is the unique name of the GCP host specified in the scylla.yaml file.

2. Do not close the yaml file. Change the system key directory location according to your settings.

   • system_key_directory is the location of the system key you created in Create Encryption Keys.

   system_key_directory: /etc/scylla/encryption_keys/system_keys
   

3. Save the file.

4. Drain the node with nodetool drain

5. Restart the scylla-server service.

   Supported OSDocker

   sudo systemctl restart scylla-server
   

When a Key is Lost¶

It is crucial to back up all of your encryption keys in a secure way. Keep a copy of all keys in a secure location. In the event that you do lose a key, your data encrypted with that key will be unreadable.

Additional Resources¶

• nodetool upgradesstables

• CREATE TABLE parameters