Module solana_runtime::snapshot_utils

Modules

• archive_format 🔒
• snapshot_storage_rebuilder 🔒
  Provides interfaces for rebuilding snapshot storages

Structs

• BankFromArchiveTimings
• BankFromDirTimings
• BankSnapshotInfo
  Information about a bank snapshot. Namely the slot of the bank, the path to the snapshot, and the type of the snapshot.
• SnapshotRootPaths 🔒
  Helper type when rebuilding from snapshots. Designed to handle when rebuilding from just a full snapshot, or from both a full snapshot and an incremental snapshot.
• StorageAndNextAppendVecId 🔒
  Helper type for passing around account storage map and next append vec id for reconstructing accounts from a snapshot
• UnarchivedSnapshot 🔒
  Helper type to bundle up the results from unarchive_snapshot()
• UnpackedSnapshotsDirAndVersion 🔒
  Helper type for passing around the unpacked snapshots dir and the snapshot version together
• VerifySlotDeltasStructuralInfo 🔒
  Computed information from verify_slot_deltas_structural(), that may be reused/useful later.

Enums

• ArchiveFormat
  The different archive formats used for snapshots
• BankSnapshotType
  Bank snapshots traditionally had their accounts hash calculated prior to serialization. Since the hash calculation takes a long time, an optimization has been put in to offload the accounts hash calculation. The bank serialization format has not changed, so we need another way to identify if a bank snapshot contains the calculated accounts hash or not.
• ParseError
• SnapshotError
• SnapshotFrom
  When constructing a bank a snapshot, traditionally the snapshot was from a snapshot archive. Now, the snapshot can be from a snapshot directory, or from a snapshot archive. This is the flag to indicate which.
• SnapshotNewFromDirError
• SnapshotVersion
• VerifyBank
  allow tests to specify what happened to the serialized format
• VerifySlotDeltasError
  Errors that can happen in verify_slot_deltas()

Constants

• BANK_SNAPSHOT_PRE_FILENAME_EXTENSION
• DEFAULT_ARCHIVE_COMPRESSION
• DEFAULT_FULL_SNAPSHOT_ARCHIVE_INTERVAL_SLOTS
• DEFAULT_INCREMENTAL_SNAPSHOT_ARCHIVE_INTERVAL_SLOTS
• DEFAULT_MAX_FULL_SNAPSHOT_ARCHIVES_TO_RETAIN
• DEFAULT_MAX_INCREMENTAL_SNAPSHOT_ARCHIVES_TO_RETAIN
• FULL_SNAPSHOT_ARCHIVE_FILENAME_REGEX
• INCREMENTAL_SNAPSHOT_ARCHIVE_FILENAME_REGEX
• MAX_BANK_SNAPSHOTS_TO_RETAIN
• MAX_SNAPSHOT_DATA_FILE_SIZE 🔒
• MAX_SNAPSHOT_VERSION_FILE_SIZE 🔒
• PARALLEL_UNTAR_READERS_DEFAULT 🔒
• SNAPSHOT_ACCOUNTS_HARDLINKS
• SNAPSHOT_ARCHIVE_DOWNLOAD_DIR
• SNAPSHOT_STATE_COMPLETE_FILENAME
• SNAPSHOT_STATUS_CACHE_FILENAME
• SNAPSHOT_VERSION_FILENAME
• SUPPORTED_ARCHIVE_COMPRESSION
• TAR_BZIP2_EXTENSION
• TAR_EXTENSION
• TAR_GZIP_EXTENSION
• TAR_LZ4_EXTENSION
• TAR_ZSTD_EXTENSION
• TMP_BANK_SNAPSHOT_PREFIX 🔒
• TMP_SNAPSHOT_ARCHIVE_PREFIX
• VERSION_STRING_V1_2_0 🔒

Functions

• __stability 🔒
• __stability 🔒
• __static_ref_initialize 🔒
• __static_ref_initialize 🔒
• add_bank_snapshot
  Serialize a bank to a snapshot
• archive_snapshot_package
  Make a snapshot archive out of the snapshot package
• bank_fields_from_snapshot_archives
  Utility for parsing out bank specific information from a snapshot archive. This utility can be used to parse out bank specific information like the leader schedule, epoch schedule, etc.
• bank_fields_from_snapshots 🔒
• bank_from_latest_snapshot_archives
  Rebuild bank from snapshot archives. This function searches full_snapshot_archives_dir and incremental_snapshot_archives_dir for the highest full snapshot and highest corresponding incremental snapshot, then rebuilds the bank.
• bank_from_latest_snapshot_dir
  follow the prototype of fn bank_from_latest_snapshot_archives, implement the from_dir case
• bank_from_snapshot_archives
  Rebuild bank from snapshot archives. Handles either just a full snapshot, or both a full snapshot and an incremental snapshot.
• bank_from_snapshot_dir
  Build bank from a snapshot (a snapshot directory, not a snapshot archive)
• bank_to_full_snapshot_archive
  Convenience function to create a full snapshot archive out of any Bank, regardless of state. The Bank will be frozen during the process. This is only called from ledger-tool or tests. Warping is a special case as well.
• bank_to_incremental_snapshot_archive
  Convenience function to create an incremental snapshot archive out of any Bank, regardless of state. The Bank will be frozen during the process. This is only called from ledger-tool or tests. Warping is a special case as well.
• build_full_snapshot_archive_path
  Build the full snapshot archive path from its components: the snapshot archives directory, the snapshot slot, the accounts hash, and the archive format.
• build_incremental_snapshot_archive_path
  Build the incremental snapshot archive path from its components: the snapshot archives directory, the snapshot base slot, the snapshot slot, the accounts hash, and the archive format.
• build_snapshot_archives_remote_dir
• build_storage_from_snapshot_dir 🔒
  Perform the common tasks when deserialize a snapshot. Handles reading snapshot file, reading the version file, and then returning those fields plus the rebuilt storage
• check_are_snapshots_compatible 🔒
  Check if an incremental snapshot is compatible with a full snapshot. This is done by checking if the incremental snapshot’s base slot is the same as the full snapshot’s slot.
• check_deserialize_file_consumed 🔒
  After running the deserializer function, perform common checks to ensure the snapshot archive files were consumed correctly.
• clean_orphaned_account_snapshot_dirs
  The account snapshot directories under <account_path>/snapshot/ contain account files hardlinked from <account_path>/run taken at snapshot time. They are referenced by the symlinks from the bank snapshot dir snapshot//accounts_hardlinks/. We observed that sometimes the bank snapshot dir could be deleted but the account snapshot directories were left behind, possibly by some manual operations or some legacy code not using the symlinks to clean up the acccount snapshot hardlink directories. This function cleans up any account snapshot directories that are no longer referenced by the bank snapshot dirs, to ensure proper snapshot operations.
• create_accounts_run_and_snapshot_dirs
  To allow generating a bank snapshot directory with full state information, we need to hardlink account appendvec files from the runtime operation directory to a snapshot hardlink directory. This is to create the run/ and snapshot sub directories for an account_path provided by the user. These two sub directories are on the same file system partition to allow hard-linking.
• create_all_accounts_run_and_snapshot_dirs
  For all account_paths, create the run/ and snapshot/ sub directories. If an account_path directory does not exist, create it. It returns (account_run_paths, account_snapshot_paths) or error
• create_snapshot_data_file_stream 🔒
  Before running the deserializer function, perform common operations on the snapshot archive files, such as checking the file size and opening the file into a stream.
• create_snapshot_meta_files_for_unarchived_snapshot 🔒
  BankSnapshotInfo::new_from_dir() requires a few meta files to accept a snapshot dir as a valid one. A dir unpacked from an archive lacks these files. Fill them here to allow new_from_dir() checks to pass. These checks are not needed for unpacked dirs, but it is not clean to add another flag to new_from_dir() to skip them.
• create_tmp_accounts_dir_for_tests
• delete_contents_of_path 🔒
  Delete the files and subdirectories in a directory. This is useful if the process does not have permission to delete the top level directory it might be able to delete the contents of that directory.
• deserialize_snapshot_data_file
• deserialize_snapshot_data_files 🔒
• deserialize_snapshot_data_files_capped 🔒
• deserialize_status_cache 🔒
• do_get_highest_bank_snapshot 🔒
• get_account_path_from_appendvec_path 🔒
  Return account path from the appendvec path after checking its format.
• get_bank_snapshots
  Get the bank snapshots in a directory
• get_bank_snapshots_dir 🔒
• get_bank_snapshots_post
  Get the bank snapshots in a directory
• get_bank_snapshots_pre
  Get the bank snapshots in a directory
• get_full_snapshot_archives
  Get a list of the full snapshot archives from a directory
• get_highest_bank_snapshot
  Get the bank snapshot with the highest slot in a directory
• get_highest_bank_snapshot_post
  Get the bank snapshot with the highest slot in a directory
• get_highest_bank_snapshot_pre
  Get the bank snapshot with the highest slot in a directory
• get_highest_full_snapshot_archive_info
  Get the path (and metadata) for the full snapshot archive with the highest slot in a directory
• get_highest_full_snapshot_archive_slot
  Get the highest slot of the full snapshot archives in a directory
• get_highest_incremental_snapshot_archive_info
  Get the path for the incremental snapshot archive with the highest slot, for a given full snapshot slot, in a directory
• get_highest_incremental_snapshot_archive_slot
  Get the highest slot of the incremental snapshot archives in a directory, for a given full snapshot slot
• get_incremental_snapshot_archives
  Get a list of the incremental snapshot archives from a directory
• get_io_error 🔒
• get_snapshot_accounts_hardlink_dir 🔒
  From an appendvec path, derive the snapshot hardlink path. If the corresponding snapshot hardlink directory does not exist, create it.
• get_snapshot_archives 🔒
  Walk down the snapshot archive to collect snapshot archive file info
• get_snapshot_file_name 🔒
• get_snapshot_storages
  Get the snapshot storages for this bank
• get_storages_to_serialize 🔒
  serializing needs Vec<Vec<Arc>>, but data structure at runtime is Vec<Arc> translates to what we need
• hard_link_storages_to_snapshot 🔒
  Hard-link the files from accounts/ to snapshot/<bank_slot>/accounts/ This keeps the appendvec files alive and with the bank snapshot. The slot and id in the file names are also updated in case its file is a recycled one with inconsistent slot and id.
• move_and_async_delete_path
  Delete directories/files asynchronously to avoid blocking on it. Fist, in sync context, rename the original path to *_deleted, then spawn a thread to delete the renamed path. If the process is killed and the deleting process is not done, the leftover path will be deleted in the next process life, so there is no file space leaking.
• package_and_archive_full_snapshot
  Helper function to hold shared code to package, process, and archive full snapshots
• package_and_archive_incremental_snapshot
  Helper function to hold shared code to package, process, and archive incremental snapshots
• parse_full_snapshot_archive_filename 🔒
  Parse a full snapshot archive filename into its Slot, Hash, and Archive Format
• parse_incremental_snapshot_archive_filename 🔒
  Parse an incremental snapshot archive filename into its base Slot, actual Slot, Hash, and Archive Format
• path_to_file_name_str
  Get the &str from a &Path
• purge_old_bank_snapshots
  Remove outdated bank snapshots
• purge_old_snapshot_archives
• rebuild_bank_from_snapshot 🔒
• rebuild_bank_from_unarchived_snapshots 🔒
• remove_bank_snapshot
  Remove the snapshot directory for this slot
• remove_tmp_snapshot_archives
  If the validator halts in the middle of archive_snapshot_package(), the temporary staging directory won’t be cleaned up. Call this function to clean them up.
• serialize_snapshot_data_file
• serialize_snapshot_data_file_capped 🔒
• serialize_status_cache 🔒
• should_take_full_snapshot
• should_take_incremental_snapshot
• snapshot_version_from_file 🔒
  Reads the snapshot_version from a file. Before opening the file, its size is compared to MAX_SNAPSHOT_VERSION_FILE_SIZE. If the size exceeds this threshold, it is not opened and an error is returned.
• spawn_unpack_snapshot_thread 🔒
  Spawns a thread for unpacking a snapshot
• streaming_snapshot_dir_files 🔒
  Streams snapshot dir files across channel Follow the flow of streaming_unarchive_snapshot(), but handle the from_dir case.
• streaming_unarchive_snapshot 🔒
  Streams unpacked files across channel
• unarchive_snapshot 🔒
  Perform the common tasks when unarchiving a snapshot. Handles creating the temporary directories, untaring, reading the version file, and then returning those fields plus the rebuilt storage
• unpack_snapshot_local 🔒
• untar_snapshot_create_shared_buffer 🔒
• untar_snapshot_in 🔒
• verify_and_unarchive_snapshots 🔒
• verify_bank_against_expected_slot_hash 🔒
  Check to make sure the deserialized bank’s slot and hash matches the snapshot archive’s slot and hash
• verify_slot_deltas 🔒
  Verify that the snapshot’s slot deltas are not corrupt/invalid
• verify_slot_deltas_structural 🔒
  Verify that the snapshot’s slot deltas are not corrupt/invalid These checks are simple/structural
• verify_slot_deltas_with_history 🔒
  Verify that the snapshot’s slot deltas are not corrupt/invalid These checks use the slot history for verification
• verify_snapshot_archive
• verify_unpacked_snapshots_dir_and_version 🔒
• write_snapshot_version_file
  Write the snapshot version as a file into the bank snapshot directory

Type Definitions

• Result