SCR-Dev
legacy

Concepts

  • Overview
    • Intro to the SCR API
    • Jobs, allocations, and runs
    • Group, store, and redundancy descriptors
    • Control, cache, and prefix directories
    • Example of SCR files and directories
    • Scalable checkpoint
    • Scalable restart
    • Catastrophic failures
    • Fetch, flush, and scavenge
    • Configuration parameters
    • Global variables and portability
  • File paths
    • Overview
    • Common functions
      • Creating and freeing path objects
      • Querying paths and converting them to character string
      • Combining paths
      • Slicing paths
      • Other path manipulation
  • Hash
    • Overview
    • Common functions
      • Hash basics
      • Accessing and iterating over hash elements
      • Key/value convenience functions
      • Specifying multiple keys with format functions
      • Sorting hash keys
      • Listing hash keys
      • Packing and unpacking hashes
      • Hash files
      • Sending and receiving hashes
    • Debugging
    • Binary format
      • Packed hash
      • File format
  • Filemap
    • Overview
    • Example filemap hash
    • Common functions
      • Allocating, freeing, merging, and clearing filemaps
      • Adding and removing data
      • Query functions
      • List functions
      • Iterator functions
      • Dataset descriptors
      • File meta data
      • Redundancy descriptors
      • Tags
      • Accessing a filemap file
  • Datasets
    • Overview
    • Example dataset hash
    • Common functions
      • Allocating and freeing dataset objects
      • Setting, getting, and checking field values
  • Meta data
    • Overview
    • Example meta data hash
    • Common functions
      • Allocating, freeing, and copying meta data objects
      • Setting, getting, and checking field values
  • Group descriptors
    • Overview
    • Group descriptor struct
    • Example group descriptor configuration file entries
    • Common functions
      • Creating and freeing the group descriptors array
      • Lookup group descriptor by name
  • Store descriptors
    • Overview
    • Store descriptor struct
    • Example store descriptor configuration file entries
    • Common functions
      • Creating and freeing the store descriptors array
      • Lookup store descriptor by name
      • Create and delete directories on storage device
  • Redundancy descriptors
    • Overview
    • Redundancy descriptor struct
      • Extra state for PARTNER
      • Extra state for XOR
    • Example redundancy descriptor hash
    • Example redundancy descriptor configuration file entries
    • Common functions
      • Initializing and freeing redundancy descriptors
      • Redundancy descriptor array
      • Converting between structs and hashes
      • Interacting with filemaps
  • Redundancy schemes
    • SINGLE
    • PARTNER
  • XOR
    • XOR algorithm
    • XOR file
    • XOR rebuild
  • Containers
  • Scavenge
    • Rank filemap file
    • Scanning files
    • Inspecting files
    • Rebuilding files
    • Scan hash

Files

  • Index file
  • Summary file
  • Rank2file map
  • Filemap files
  • Flush file
  • Halt file
  • Nodes file
  • Transfer file

Program Flow

  • Perl modules
    • scripts/common/scr_hostlist.pm
    • scripts/common/scr_param.pm.in
  • Utilities
    • scripts/common/scr_glob_hosts.in
    • src/scr_flush_file.c
    • scripts/common/scr_list_dir.in
    • scripts/TLCC/scr_list_down_nodes.in
    • scripts/common/scr_check_node.in
    • scripts/common/scr_prefix.in
  • Launch a run
    • scripts/TLCC/scr_run.in
    • scripts/common/scr_test_runtime.in
    • scripts/common/scr_prerun.in
    • src/scr_retries_halt.c
  • SCR_Init
    • scr_groupdescs_create
    • scr_storedescs_create
    • scr_reddescs_create
    • scr_scatter_filemaps
    • scr_cache_rebuild
    • scr_distribute_datasets
    • scr_distribute_reddescs
    • scr_distribute_files
    • scr_reddesc_recover
    • scr_reddesc_recover_xor_attempt
    • scr_reddesc_recover_xor
    • scr_fetch_sync
  • SCR_Need_checkpoint
  • SCR_Start_checkpoint
  • SCR_Route_file
  • SCR_Complete_checkpoint
    • scr_reddesc_apply_partner
    • scr_reddesc_apply_xor
  • SCR_Finalize
  • Flush
    • scr_flush_sync
    • scr_flush_prepare
    • scr_flush_identify
    • scr_flush_identify_dirs
    • scr_flush_identify_containers
    • scr_flush_create_dirs
    • scr_flush_data
    • scr_flush_files_list
    • scr_flush_complete
    • scr_flush_summary
    • scr_flush_summary_map
  • Scavenge
    • scripts/common/scr_postrun.in
    • scripts/TLCC/scr_scavenge.in
    • src/scr_copy.c
    • src/scr_index.c
    • index_add_dir
    • scr_summary_build
    • scr_scan_files
    • scr_scan_file
    • scr_inspect_scan
    • scr_rebuild_scan

Testing

  • Testing SCR on a New Systems
    • Restart in Place
    • Node Health Checks
  • Bamboo Test Suite
    • Bamboo Test Plan Overview
SCR-Dev
  • Docs »
  • Search
  • Edit on GitHub

© Copyright 2017, SCR Revision 68414920.

Built with Sphinx using a theme provided by Read the Docs.