ScyllaDB SStable¶

Schema file¶

The schema file should contain all definitions needed to interpret data belonging to the table.

Example schema.cql:

CREATE KEYSPACE ks WITH replication = {'class': 'NetworkTopologyStrategy', 'mydc1': 1, 'mydc2': 4};

CREATE TYPE ks.mytype (
    f1 int,
    f2 text
);

CREATE TABLE ks.cf (
    pk int,
    ck text,
    v1 int,
    v2 mytype,
    PRIMARY KEY (pk, ck)
);

Note:

• In addition to the table itself, the definition also has to includes any user defined types the table uses.

• The keyspace definition is optional, if missing one will be auto-generated.

• The schema file doesn’t have to be called schema.cql, this is just the default name. Any file name is supported (with any extension).

• The schema file is allowed to contain a single CREATE TABLE statement. If the table is a view, the schema file can contain a CREATE TABLE and a CREATE INDEX or CREATE MATERIALIZED VIEW statements.

INSERT INTO system_schema.dropped_columns (
    keyspace_name,
    table_name,
    column_name,
    dropped_time,
    type
) VALUES (
    'ks',
    'cf',
    'v1',
    1631011979170675,
    'int'
);

CREATE TABLE ks.cf (pk int PRIMARY KEY, v2 int);

System schema¶

If the examined table is a system table – it belongs to one of the system keyspaces (system, system_schema, system_distributed or system_distributed_everywhere) – the tool can be instructed to use the known built-in definition of said table. This is possible with the --system-schema flag. Example:

scylla sstable dump-data --system-schema --keyspace system --table local ./path/to/md-123456-big-Data.db

Schema tables¶

Load the schema from ScyllaDB’s schema tables, located on the disk. The tool has to be able to locate ScyllaDB data directory for this to work. The path to the data directory can be provided the following ways:

• Deduced from the SSTable’s path. This only works if the SSTable itself is located in the ScyllaDB’s data directory.

• Provided directly via --scylla-data-dir parameter.

• Provided indirectly via --scylla-yaml-file parameter. The path to the data directory is obtained from the configuration.

• Provided indirectly via the SCYLLA_HOME environment variable. The scylla.yaml file is expected to be found at the SCYLLA_HOME/conf/scylla.yaml path.

• Provided indirectly via the SCYLLA_CONF environment variable. The scylla.yaml file is expected to be found at the SCYLLA_CONF/scylla.yaml path.

In all except the first case, the keyspace and table names have to be provided via the --keyspace and --table parameters respectively.

Common problems:

• If the schema tables are read while the ScyllaDB node is running, ScyllaDB might compact an SStable that the tool is trying to read, resulting in failure to load the schema. The simplest solution is to retry the operation. To avoid the problem, disable autocompation on the system_schema keyspace, or stop the node. This is intrusive and might not be possible, in this case use another schema source.

• The schema of the SSTable could be still in memtables, not yet flushed to disk. In this case, simply flush the system_schema keyspace memtables and retry the operation.

SStable schema¶

Each SStable contains a schema in its Statistics component. This schema is incomplete: it doesn’t contain names for primary key columns and contains no schema options. That said, this schema is good enough for most operations and is always available.

dump-data¶

Dumps the content of the data component (the component that contains the data-proper of the SStable). This operation might produce a huge amount of output. In general, the human-readable output will be larger than the binary file.

It is possible to filter the data to print via the --partitions or --partitions-file options. Both expect partition key values in the hexdump format.

Supports both a text and JSON output. The text output uses the built-in ScyllaDB printers, which are also used when logging mutation-related data structures.

The schema of the JSON output is the following:

$ROOT := $NON_MERGED_ROOT | $MERGED_ROOT

$NON_MERGED_ROOT := { "$sstable_path": $SSTABLE, ... } // without --merge

$MERGED_ROOT := { "anonymous": $SSTABLE } // with --merge

$SSTABLE := [$PARTITION, ...]

$PARTITION := {
    "key": {
        "token": String,
        "raw": String, // hexadecimal representation of the raw binary
        "value": String
    },
    "tombstone: $TOMBSTONE, // optional
    "static_row": $COLUMNS, // optional
    "clustering_elements": [
        $CLUSTERING_ROW | $RANGE_TOMBSTONE_CHANGE,
        ...
    ]
}

$TOMBSTONE := {
    "timestamp": Int64,
    "deletion_time": String // YYYY-MM-DD HH:MM:SS
}

$COLUMNS := {
    "$column_name": $REGULAR_CELL | $COUNTER_SHARDS_CELL | $COUNTER_UPDATE_CELL | $FROZEN_COLLECTION | $COLLECTION,
    ...
}

$REGULAR_CELL := {
    "is_live": Bool, // is the cell live or not
    "type": "regular",
    "timestamp": Int64,
    "ttl": String, // gc_clock::duration - optional
    "expiry": String, // YYYY-MM-DD HH:MM:SS - optional
    "value": String // only if is_live == true
}

$COUNTER_SHARDS_CELL := {
    "is_live": true,
    "type": "counter-shards",
    "timestamp": Int64,
    "value": [$COUNTER_SHARD, ...]
}

$COUNTER_SHARD := {
    "id": String, // UUID
    "value": Int64,
    "clock": Int64
}

$COUNTER_UPDATE_CELL := {
    "is_live": true,
    "type": "counter-update",
    "timestamp": Int64,
    "value": Int64
}

$FROZEN_COLLECTION is the same as a $REGULAR_CELL, with type = "frozen-collection".

$COLLECTION := {
    "type": "collection",
    "tombstone": $TOMBSTONE, // optional
    "cells": [
        {
            "key": String,
            "value": $REGULAR_CELL
        },
        ...
    ]
}

$CLUSTERING_ROW := {
    "type": "clustering-row",
    "key": {
        "raw": String, // hexadecimal representation of the raw binary
        "value": String
    },
    "tombstone": $TOMBSTONE, // optional
    "shadowable_tombstone": $TOMBSTONE, // optional
    "marker": { // optional
        "timestamp": Int64,
        "ttl": String, // gc_clock::duration
        "expiry": String // YYYY-MM-DD HH:MM:SS
    },
    "columns": $COLUMNS
}

$RANGE_TOMBSTONE_CHANGE := {
    "type": "range-tombstone-change",
    "key": { // optional
        "raw": String, // hexadecimal representation of the raw binary
        "value": String
    },
    "weight": Int, // -1 or 1
    "tombstone": $TOMBSTONE
}

dump-index¶

Dumps the content of the index component. It the partition-index of the data component, which is effectively a list of all the partitions in the SStable, with their starting position in the data component and, optionally, a promoted index. The promoted index contains a sampled index of the clustering rows in the partition. Positions (both that of partition and that of rows) are valid for uncompressed data.

The content is dumped in JSON, using the following schema:

$ROOT := { "$sstable_path": $SSTABLE, ... }

$SSTABLE := [$INDEX_ENTRY, ...]

$INDEX_ENTRY := {
    "key": {
        "raw": String, // hexadecimal representation of the raw binary
        "value": String
    },
    "pos": Uint64
}

dump-compression-info¶

Dumps the content of the compression-info component. It contains compression parameters and maps positions into the uncompressed data to that into compressed data. Note that compression happens over chunks with configurable size, so to get data at a position in the middle of a compressed chunk, the entire chunk has to be decompressed.

The content is dumped in JSON, using the following schema:

$ROOT := { "$sstable_path": $SSTABLE, ... }

$SSTABLE := {
    "name": String,
    "options": {
        "$option_name": String,
        ...
    },
    "chunk_len": Uint,
    "data_len": Uint64,
    "offsets": [Uint64, ...]
}

dump-summary¶

Dumps the content of the summary component. The summary is a sampled index of the content of the index-component (an index of the index). The sampling rate is chosen such that this file is small enough to be kept in memory even for very large SStables.

The content is dumped in JSON, using the following schema:

$ROOT := { "$sstable_path": $SSTABLE, ... }

$SSTABLE := {
    "header": {
        "min_index_interval": Uint64,
        "size": Uint64,
        "memory_size": Uint64,
        "sampling_level": Uint64,
        "size_at_full_sampling": Uint64
    },
    "positions": [Uint64, ...],
    "entries": [$SUMMARY_ENTRY, ...],
    "first_key": $KEY,
    "last_key": $KEY
}

$SUMMARY_ENTRY := {
    "key": $DECORATED_KEY,
    "position": Uint64
}

$DECORATED_KEY := {
    "token": String,
    "raw": String, // hexadecimal representation of the raw binary
    "value": String
}

$KEY := {
    "raw": String, // hexadecimal representation of the raw binary
    "value": String
}

dump-statistics¶

Dumps the content of the statistics component. It contains various metadata about the data component. In the SStable 3 format, this component is critical for parsing the data component.

The content is dumped in JSON, using the following schema:

$ROOT := { "$sstable_path": $SSTABLE, ... }

$SSTABLE := {
    "offsets": {
        "$metadata": Uint,
        ...
    },
    "validation": $VALIDATION_METADATA,
    "compaction": $COMPACTION_METADATA,
    "stats": $STATS_METADATA,
    "serialization_header": $SERIALIZATION_HEADER // >= MC only
}

$VALIDATION_METADATA := {
    "partitioner": String,
    "filter_chance": Double
}

$COMPACTION_METADATA := {
    "ancestors": [Uint, ...], // < MC only
    "cardinality": [Uint, ...]
}

$STATS_METADATA := {
    "estimated_partition_size": $ESTIMATED_HISTOGRAM,
    "estimated_cells_count": $ESTIMATED_HISTOGRAM,
    "position": $REPLAY_POSITION,
    "min_timestamp": Int64,
    "max_timestamp": Int64,
    "min_local_deletion_time": Int64, // >= MC only
    "max_local_deletion_time": Int64,
    "min_ttl": Int64, // >= MC only
    "max_ttl": Int64, // >= MC only
    "compression_ratio": Double,
    "estimated_tombstone_drop_time": $STREAMING_HISTOGRAM,
    "sstable_level": Uint,
    "repaired_at": Uint64,
    "min_column_names": [Uint, ...],
    "max_column_names": [Uint, ...],
    "has_legacy_counter_shards": Bool,
    "columns_count": Int64, // >= MC only
    "rows_count": Int64, // >= MC only
    "commitlog_lower_bound": $REPLAY_POSITION, // >= MC only
    "commitlog_intervals": [$COMMITLOG_INTERVAL, ...] // >= MC only
}

$ESTIMATED_HISTOGRAM := [$ESTIMATED_HISTOGRAM_BUCKET, ...]

$ESTIMATED_HISTOGRAM_BUCKET := {
    "offset": Int64,
    "value": Int64
}

$STREAMING_HISTOGRAM := {
    "$key": Uint64,
    ...
}

$REPLAY_POSITION := {
    "id": Uint64,
    "pos": Uint
}

$COMMITLOG_INTERVAL := {
    "start": $REPLAY_POSITION,
    "end": $REPLAY_POSITION
}

$SERIALIZATION_HEADER_METADATA := {
    "min_timestamp_base": Uint64,
    "min_local_deletion_time_base": Uint64,
    "min_ttl_base": Uint64",
    "pk_type_name": String,
    "clustering_key_types_names": [String, ...],
    "static_columns": [$COLUMN_DESC, ...],
    "regular_columns": [$COLUMN_DESC, ...],
}

$COLUMN_DESC := {
    "name": String,
    "type_name": String
}

dump-scylla-metadata¶

Dumps the content of the scylla-metadata component. Contains ScyllaDB-specific metadata about the data component. This component won’t be present in SStables produced by Apache Cassandra.

The content is dumped in JSON, using the following schema:

$ROOT := { "$sstable_path": $SSTABLE, ... }

$SSTABLE := {
    "sharding": [$SHARDING_METADATA, ...],
    "features": $FEATURES_METADATA,
    "extension_attributes": { "$key": String, ...}
    "run_identifier": String, // UUID
    "large_data_stats": {"$key": $LARGE_DATA_STATS_METADATA, ...}
    "sstable_origin": String
    "scylla_build_id": String
    "scylla_version": String
    "ext_timestamp_stats": {"$key": int64, ...}
    "sstable_identifier": String, // UUID
}

$SHARDING_METADATA := {
    "left": {
        "exclusive": Bool,
        "token": String
    },
    "right": {
        "exclusive": Bool,
        "token": String
    }
}

$FEATURES_METADATA := {
    "mask": Uint64,
    "features": [String, ...]
}

$LARGE_DATA_STATS_METADATA := {
    "max_value": Uint64,
    "threshold": Uint64,
    "above_threshold": Uint
}

validate¶

Validates the content of the sstable on the mutation-fragment level, see sstable content for more details. Any parsing errors will also be detected, but after successful parsing the validation will happen on the fragment level. The following things are validated:

• Partitions are ordered in strictly monotonic ascending order.

• Fragments are correctly ordered.

• Clustering elements are ordered according in a strictly increasing clustering order as defined by the schema.

• All range deletions are properly terminated with a corresponding end bound.

• The stream ends with a partition-end fragment.

Any errors found will be logged with error level to stderr.

The validation result is dumped in JSON, using the following schema:

$ROOT := { "$sstable_path": $RESULT }

$RESULT := {
    "errors": Uint64,
    "valid": Bool,
}

scrub¶

Rewrites the SStable, skipping or fixing corrupt parts. Not all kinds of corruption can be skipped or fixed by scrub. It is limited to ordering issues on the partition, row, or mutation-fragment level. See sstable content for more details.

Scrub has several modes:

• abort - Aborts the scrub as soon as any error is found (recognized or not). This mode is only included for the sake of completeness. We recommend using the validate mode so that all errors are reported.

• skip - Skips over any corruptions found, thus omitting them from the output. Note that this mode can result in omitting more than is strictly necessary, but it guarantees that all detectable corruptions will be omitted.

• segregate - Fixes partition/row/mutation-fragment out-of-order errors by segregating the output into as many SStables as required so that the content of each output SStable is properly ordered.

• validate - Validates the content of the SStable, reporting any corruptions found. Writes no output SStables. In this mode, scrub has the same outcome as the validate operation - and the validate operation is recommended over scrub.

Output SStables are written to the directory specified via --output-directory. They will be written with the BIG format and the highest supported SStable format, with generations chosen by scylla-sstable. Generations are chosen such that they are unique among the SStables written by the current scrub.

The output directory must be empty; otherwise, scylla-sstable will abort scrub. You can allow writing to a non-empty directory by setting the --unsafe-accept-nonempty-output-dir command line flag. Note that scrub will be aborted if an SStable cannot be written because its generation clashes with a pre-existing SStable in the output directory.

validate-checksums¶

There are two kinds of checksums for SStable data files:

• The digest (full checksum), stored in the Digest.crc32 file. It is calculated over the entire content of Data.db.

• The per-chunk checksum. For uncompressed SStables, it is stored in CRC.db; for compressed SStables, it is stored inline after each compressed chunk in Data.db.

During normal reads, ScyllaDB validates the per-chunk checksum for compressed SStables. The digest and the per-chunk checksum of uncompressed SStables are currently not checked on any code paths.

This operation reads the entire Data.db and validates both kinds of checksums against the data. Errors found are logged to stderr. The output contains an object for each SStable that indicates if the SStable has checksums (false only for uncompressed SStables for which CRC.db is not present in TOC.txt), if the SStable has a digest, and if the SStable matches all checksums and the digest. If no digest is available, validation will proceed using only the per-chunk checksums.

The content is dumped in JSON, using the following schema:

$ROOT := { "$sstable_path": $RESULT, ... }

$RESULT := {
    "has_checksums": Bool,
    "has_digest": Bool,
    "valid": Bool, // optional
}

decompress¶

Decompress Data.db if compressed (no-op if not compressed). The decompressed data is written to Data.db.decompressed. For example, for the SStable:

md-12311-big-Data.db

the output will be:

md-12311-big-Data.db.decompressed

write¶

Writes an SStable based on a JSON representation of the content. The JSON representation has to have the same schema as that of a single SStable from the output of the dump-data operation (corresponding to the $SSTABLE symbol). The easiest way to get started with writing your own SStable is to dump an existing SStable, modify the JSON then invoke this operation with the result. You can feed the output of dump-data to write by filtering the output of the former with jq .sstables[]:

scylla sstable dump-data --system-schema system_schema.columns /path/to/me-14-big-Data.db | jq .sstables[] > input.json
scylla sstable write --system-schema system_schema.columns --input-file ./input.json --generation 0
scylla sstable dump-data --system-schema system_schema.columns ./me-0-big-Data.db | jq .sstables[] > dump.json

At the end of the above, input.json and dump.json will have the same content.

Note that write doesn’t yet support all the features of the ScyllaDB storage engine. The following are not supported:

• Counters.

• Non-strictly atomic cells, including frozen multi-cell types like collections, tuples, and UDTs.

Parsing uses a streaming JSON parser, it is safe to pass in input files of any size.

The output SStable will use the BIG format, the highest supported SStable format, and the specified generation (--generation). By default, it is placed in the local directory, which can be changed with --output-dir. If the output SStable clashes with an existing SStable, the write will fail.

The output is validated before being written to the disk. The validation done here is similar to that done by the validate operation. The level of validation can be changed with the --validation-level flag. Possible validation-levels are:

• partition_region - Only checks fragment types, e.g., that a partition-end is followed by partition-start or EOS.

• token - In addition, checks the token order of partitions.

• partition_key - Full check on partition ordering.

• clustering_key - In addition, checks clustering element ordering.

Note that levels are cumulative - each contains all the checks of the previous levels, too. By default, the strictest level is used. This can be relaxed, for example, if you want to produce intentionally corrupt SStables for tests.

shard-of¶

Print out the shards which own the specified SSTables.

The content is dumped in JSON, using the following schema when --vnodes command option is specified:

$ROOT := { "$sstable_path": $SHARD_IDS, ... }

$SHARD_IDS := [$SHARD_ID, ...]

$SHARD_ID := Uint

script¶

Reads the SStable(s) and passes the resulting fragment stream to the script specified by –script-file. Currently, only scripts written in Lua are supported. It is possible to pass command line arguments to the script, via the --script-arg command line flag. The format of this argument is the following:

--script-arg $key1=$value1:$key2=$value2

Alternatively, you can provide each key-value pair via a separate --script-arg:

--script-arg $key1=$value1 --script-arg $key2=$value2

Command line arguments will be received by the consume_stream_start() API method.

attribute_name (type) - description

method_name(arg1_type, arg2_type...) (return_type) - description