Skip to content
  • Yanqin Jin's avatar
    Support for single-primary, multi-secondary instances (#4899) · 9358178e
    Yanqin Jin authored
    Summary:
    This PR allows RocksDB to run in single-primary, multi-secondary process mode.
    The writer is a regular RocksDB (e.g. an `DBImpl`) instance playing the role of a primary.
    Multiple `DBImplSecondary` processes (secondaries) share the same set of SST files, MANIFEST, WAL files with the primary. Secondaries tail the MANIFEST of the primary and apply updates to their own in-memory state of the file system, e.g. `VersionStorageInfo`.
    
    This PR has several components:
    1. (Originally in #4745). Add a `PathNotFound` subcode to `IOError` to denote the failure when a secondary tries to open a file which has been deleted by the primary.
    
    2. (Similar to #4602). Add `FragmentBufferedReader` to handle partially-read, trailing record at the end of a log from where future read can continue.
    
    3. (Originally in #4710 and #4820). Add implementation of the secondary, i.e. `DBImplSecondary`.
    3.1 Tail the primary's MANIFEST during recovery.
    3.2 Tail the primary's MANIFEST during normal processing by calling `ReadAndApply`.
    3.3 Tailing WAL will be in a future PR.
    
    4. Add an example in 'examples/multi_processes_example.cc' to demonstrate the usage of secondary RocksDB instance in a multi-process setting. Instructions to run the example can be found at the beginning of the source code.
    Pull Request resolved: https://github.com/facebook/rocksdb/pull/4899
    
    Differential Revision: D14510945
    
    Pulled By: riversand963
    
    fbshipit-source-id: 4ac1c5693e6012ad23f7b4b42d3c374fecbe8886
    9358178e