User Guide

This guide is the manual for the command-line programs rms-pdsfile installs: what each one builds or checks inside a PDS holdings tree, every option it takes, and what its output means. Read Concepts first if the words volume, bundle, category and shelf are not already familiar, and Installation and Setup for the install, the expected directory layout and the environment.

• Concepts
  • PDS3 and PDS4
  • Categories
  • Published data and derived products
  • Why the derived products exist
  • The order in which they must be built
  • Tasks
  • Unit set versions
  • The volume information file
• Installation and Setup
  • Supported Python versions
  • Installing
  • One program needs the dev extra
  • Environment variables
  • The holdings tree layout
  • The log tree
  • How to read the examples in this guide
• The Maintenance Tools: the Command Line They Share
  • Invocation
  • The task flags
  • The unit argument
  • --log and --quiet
  • Where the logs go
  • Exit statuses
  • Reading the log
• pdschecksums
  • What it writes
  • What a path may name
  • Options
  • Building a manifest
  • Checking a volume against its manifest
  • Superseded manifests are kept
• pds4checksums
  • What it writes
  • What a path may name
  • Options
  • Building a manifest
  • Two differences from the PDS3 program that are visible in the output
  • Checking a bundle against its manifest
  • Superseded manifests are kept
• pdsarchives
  • What it writes
  • What a path may name
  • Validation compares metadata, not contents
  • Writing an archive
  • Checking an archive
  • Task by task
  • Exit status
• pds4archives
  • How many archives cover a target is looked up, not derived
  • What it writes
  • Validation compares metadata, not contents
  • Two differences from the PDS3 program that are visible in the output
  • Where the logs go
  • Exit status
• pdsinfoshelf
  • It reads the checksum file rather than hashing
  • What it writes
  • What a path may name
  • --archives
  • Building a shelf
  • Modification times are local
  • Checking a shelf
  • Exit status
  • Superseded shelves are kept
• pds4infoshelf
  • It reads the checksum file rather than hashing
  • What it writes
  • What a path may name
  • --archives
  • Building a shelf
  • Two differences from the PDS3 program that are visible in the output
  • Modification times are local
  • Exit status
  • Superseded shelves are kept
• pdslinkshelf
  • What it writes
  • What counts as a link
  • What a path may name
  • Checking a shelf
  • Log file naming
  • Exit status
  • Superseded shelves are kept
• pds4linkshelf
  • What it writes
  • What counts as a link
  • What a path may name
  • Checking a shelf
  • Log file naming
  • Exit status
  • Superseded shelves are kept
• pdsindexshelf
  • What a path may name
  • What it writes
  • The log path is one level deeper too
  • Checking a shelf
  • Nothing versions an index shelf
  • Exit status
• pds4indexshelf
  • What differs from the PDS3 program
  • What it writes
  • Where the logs go
  • Nothing versions an index shelf
  • Exit status
• pdsdependency
  • What a path may name
  • What gets checked is decided by the path, not the contents
  • Running it
  • “Steps required”
  • Where the logs go
  • Exit status
  • Called from elsewhere
• re_validate
  • What it runs, and what it does not
  • Two modes
  • Options
  • An interactive run
  • Batch mode’s schedule
  • Where the logs go
  • Exit status
• crlf
  • Options
  • The four verdicts
  • Reporting
  • The summary line
  • Exit status
  • Rejected abbreviations
  • Where the name comes from
• show_opus_products
  • It needs the dev extra
  • Both holdings roots are required
  • Options
  • Absolute and logical paths
  • The default table
  • The other three forms
  • Exit status
• The Shell Scripts
  • Setting up a holdings tree
  • Copying between two holdings trees
  • Rebuilding a volume set’s metadata products
  • Syncing between drives
• Appendix: File Formats
  • Checksum files: *_md5.txt
  • Shelf files: .pickle and its .py sidecar
  • Archive files: .tar.gz
  • The volume information file: _volinfo/<unit set>.txt
  • Log files