1. 31 Aug, 2017 1 commit
  2. 30 Aug, 2017 4 commits
  3. 29 Aug, 2017 3 commits
  4. 27 Aug, 2017 1 commit
  5. 26 Aug, 2017 3 commits
    • Maysam Yabandeh's avatar
      WriteAtPrepare: Efficient read from snapshot list · fbfa3e7a
      Maysam Yabandeh authored
      Summary:
      Divide the old snapshots to two lists: a few that fit into a cached array and the rest in a vector, which is expected to be empty in normal cases. The former is to optimize concurrent reads from snapshots without requiring locks. It is done by an array of std::atomic, from which std::memory_order_acquire reads are compiled to simple read instructions in most of the x86_64 architectures.
      Closes https://github.com/facebook/rocksdb/pull/2758
      
      Differential Revision: D5660504
      
      Pulled By: maysamyabandeh
      
      fbshipit-source-id: 524fcf9a8e7f90a92324536456912a99aaa6740c
      fbfa3e7a
    • Maysam Yabandeh's avatar
      Blog post for FlushWAL · b01f426f
      Maysam Yabandeh authored
      Summary: Closes https://github.com/facebook/rocksdb/pull/2790
      
      Differential Revision: D5711609
      
      Pulled By: maysamyabandeh
      
      fbshipit-source-id: ea103dac013c0a6a031834541ad67e7d95a80fe8
      b01f426f
    • Yi Wu's avatar
      make blob file close synchronous · 503db684
      Yi Wu authored
      Summary:
      Fixing flaky blob_db_test.
      
      To close a blob file, blob db used to add a CloseSeqWrite job to the background thread to close it. Changing file close to be synchronous in order to simplify logic, and fix flaky blob_db_test.
      Closes https://github.com/facebook/rocksdb/pull/2787
      
      Differential Revision: D5699387
      
      Pulled By: yiwu-arbug
      
      fbshipit-source-id: dd07a945cd435cd3808fce7ee4ea57817409474a
      503db684
  6. 25 Aug, 2017 3 commits
    • Yi Wu's avatar
      Allow DB reopen with reduced options.num_levels · 3c840d1a
      Yi Wu authored
      Summary:
      Allow user to reduce number of levels in LSM by issue a full CompactRange() and put the result in a lower level, and then reopen DB with reduced options.num_levels. Previous this will fail on reopen on when recovery replaying the previous MANIFEST and found a historical file was on a higher level than the new options.num_levels. The workaround was after CompactRange(), reopen the DB with old num_levels, which will create a new MANIFEST, and then reopen the DB again with new num_levels.
      
      This patch relax the check of levels during recovery. It allows DB to open if there was a historical file on level > options.num_levels, but was also deleted.
      Closes https://github.com/facebook/rocksdb/pull/2740
      
      Differential Revision: D5629354
      
      Pulled By: yiwu-arbug
      
      fbshipit-source-id: 545903f6b36b6083e8cbaf777176aef2f488021d
      3c840d1a
    • Yi Wu's avatar
      Fix DropColumnFamily data race · 92bfd6c5
      Yi Wu authored
      Summary:
      It should hold db mutex while accessing max_total_in_memory_state_.
      Closes https://github.com/facebook/rocksdb/pull/2784
      
      Differential Revision: D5696536
      
      Pulled By: yiwu-arbug
      
      fbshipit-source-id: 45430634d7fe11909b38e42e5f169f618681c4ee
      92bfd6c5
    • Maysam Yabandeh's avatar
      Pinnableslice examples and blog post · 7fdf735d
      Maysam Yabandeh authored
      Summary: Closes https://github.com/facebook/rocksdb/pull/2788
      
      Differential Revision: D5700189
      
      Pulled By: maysamyabandeh
      
      fbshipit-source-id: 6f043e652093ff904e52f6d35190855781b87673
      7fdf735d
  7. 24 Aug, 2017 9 commits
  8. 23 Aug, 2017 3 commits
  9. 22 Aug, 2017 3 commits
  10. 21 Aug, 2017 2 commits
    • yiwu-arbug's avatar
      Blob db create a snapshot before every read · 5b68b114
      yiwu-arbug authored
      Summary:
      If GC kicks in between
      
      * A Get() reads index entry from base db.
      * The Get() read from a blob file
      
      The GC can delete the corresponding blob file, making the key not found. Fortunately we have existing logic to avoid deleting a blob file if it is referenced by a snapshot. So the fix is to explicitly create a snapshot before reading index entry from base db.
      Closes https://github.com/facebook/rocksdb/pull/2754
      
      Differential Revision: D5655956
      
      Pulled By: yiwu-arbug
      
      fbshipit-source-id: e4ccbc51331362542e7343175bbcbdea5830f544
      5b68b114
    • yiwu-arbug's avatar
      GC the oldest file when out of space · 4624ae52
      yiwu-arbug authored
      Summary:
      When out of space, blob db should GC the oldest file. The current implementation GC the newest one instead. Fixing it.
      Closes https://github.com/facebook/rocksdb/pull/2757
      
      Differential Revision: D5657611
      
      Pulled By: yiwu-arbug
      
      fbshipit-source-id: 56c30a4c52e6ab04551dda8c5c46006d4070b28d
      4624ae52
  11. 20 Aug, 2017 1 commit
  12. 19 Aug, 2017 4 commits
  13. 18 Aug, 2017 3 commits
    • Archit Mishra's avatar
      Added mechanism to track deadlock chain · bddd5d36
      Archit Mishra authored
      Summary:
      Changes:
      * extended the wait_txn_map to track additional information
      * designed circular buffer to store n latest deadlocks' information
      * added test coverage to verify the additional information tracked is accurately stored in the buffer
      Closes https://github.com/facebook/rocksdb/pull/2630
      
      Differential Revision: D5478025
      
      Pulled By: armishra
      
      fbshipit-source-id: 2b138de7b5a73f5ca554fc3ff8220a3be49f39e7
      bddd5d36
    • yiwu-arbug's avatar
      fix db_stress uint64_t to int32 cast · c1384a70
      yiwu-arbug authored
      Summary:
      Clang complain about an cast from uint64_t to int32 in db_stress. Fixing it.
      Closes https://github.com/facebook/rocksdb/pull/2755
      
      Differential Revision: D5655947
      
      Pulled By: yiwu-arbug
      
      fbshipit-source-id: cfac10e796e0adfef4727090b50975b0d6e2c9be
      c1384a70
    • yiwu-arbug's avatar
      Fix blob db crash during calculating write amp · 29877ec7
      yiwu-arbug authored
      Summary:
      On initial call to BlobDBImpl::WaStats() `all_periods_write_` would be empty, so it will crash when we call pop_front() at line 1627. Apparently it is mean to pop only when `all_periods_write_.size() > kWriteAmplificationStatsPeriods`.
      
      The whole write amp calculation doesn't seems to be correct and it is not being exposed. Will work on it later.
      
      Test Plan
      Change kWriteAmplificationStatsPeriodMillisecs to 1000 (1 second) and run db_bench --use_blob_db for 5 minutes.
      Closes https://github.com/facebook/rocksdb/pull/2751
      
      Differential Revision: D5648269
      
      Pulled By: yiwu-arbug
      
      fbshipit-source-id: b843d9a09bb5f9e1b713d101ec7b87e54b5115a4
      29877ec7