1. 01 Jun, 2019 4 commits
  2. 31 May, 2019 14 commits
    • Yanqin Jin's avatar
      Fix compilation error in LITE mode (#5391) · 83f7a8ee
      Yanqin Jin authored
      Add macro ROCKSDB_LITE to fix compilation.
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/5391
      Differential Revision: D15574522
      Pulled By: riversand963
      fbshipit-source-id: 95aea83c5d9b2bf98a3ba0ef9167b63c9be2988b
    • Zhongyi Xie's avatar
      move LevelCompactionPicker to a separate file (#5369) · ab8f6c01
      Zhongyi Xie authored
      In order to improve code readability, this PR moves LevelCompactionBuilder and LevelCompactionPicker to compaction_picker_level.h and .cc
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/5369
      Differential Revision: D15540172
      Pulled By: miasantreble
      fbshipit-source-id: c1a578b93f127cd63661b53f32b356e6edd349af
    • Sagar Vemuri's avatar
      Reorder DBImpl's private section (#5385) · ff9d2868
      Sagar Vemuri authored
      The methods and fields in the private section of DBImpl were all intermingled, making it hard to figure out where the fields/methods start and where they end. I cleaned up the code a little so that all the type declaration are at the beginning, followed by methods, and all the data fields are at the end. This follows
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/5385
      Differential Revision: D15566978
      Pulled By: sagar0
      fbshipit-source-id: 4618a7d819ad4e2d7cc9ae1af2c59f400140bb1b
    • Yanqin Jin's avatar
      Fix WAL replay by skipping old write batches (#5170) · b9f59006
      Yanqin Jin authored
      1. Fix a bug in WAL replay in which write batches with old sequence numbers are mistakenly inserted into memtables.
      2. Add support for benchmarking secondary instance to db_bench_tool.
      With changes made in this PR, we can start benchmarking secondary instance
      using two processes. It is also possible to vary the frequency at which the
      secondary instance tries to catch up with the primary. The info log of the
      secondary can be found in a directory whose path can be specified with
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/5170
      Differential Revision: D15564608
      Pulled By: riversand963
      fbshipit-source-id: ce97688ed3d33f69d3a0b9266ebbbbf887aa0ec8
    • Siying Dong's avatar
      Move some memory related files from util/ to memory/ (#5382) · 8843129e
      Siying Dong authored
      Move arena, allocator, and memory tools under util to a separate memory/ directory.
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/5382
      Differential Revision: D15564655
      Pulled By: siying
      fbshipit-source-id: 9cd6b5d0d3d52b39606e19221fa154596e5852a5
    • Yanqin Jin's avatar
      Add class-level comments to version-related classes (#5348) · f1302eba
      Yanqin Jin authored
      As title.
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/5348
      Differential Revision: D15564595
      Pulled By: riversand963
      fbshipit-source-id: dd45aa86a70e0343c2e9ef702fad165163f548e6
    • Sagar Vemuri's avatar
      Fix flaky DBTest2.PresetCompressionDict test (#5378) · 1b59a490
      Sagar Vemuri authored
      Fix flaky DBTest2.PresetCompressionDict test.
      This PR fixes two issues with the test:
      1. Replaces `GetSstFiles` with `TotalSize`, which is based on `DB::GetColumnFamilyMetaData` so that only the size of the live SST files is taken into consideration when computing the total size of all sst files. Earlier, with `GetSstFiles`, even obsolete files were getting picked up.
      1. In ZSTD compression, it is sometimes possible that using a trained dictionary is not better than using an untrained one. Using a trained dictionary performs well in 99% of the cases, but still in the remaining ~1% of the cases (out of 10000 runs) using an untrained dictionary gets better compression results.
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/5378
      Differential Revision: D15559100
      Pulled By: sagar0
      fbshipit-source-id: c35adbf13871f520a2cec48f8bad9ff27ff7a0b4
    • Vijay Nadimpalli's avatar
      Organizing rocksdb/table directory by format · 50e47079
      Vijay Nadimpalli authored
      Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/5373
      Differential Revision: D15559425
      Pulled By: vjnadimpalli
      fbshipit-source-id: 5d6d6d615582bedd96a4b879bb25d429a6de8b55
    • Sagar Vemuri's avatar
      Fix env_options_for_read spelling in CompactionJob · e6298626
      Sagar Vemuri authored
      Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/5380
      Differential Revision: D15563386
      Pulled By: sagar0
      fbshipit-source-id: 8b26aef47cfc40ff8016daf815582f21cdd40df2
    • Levi Tamasi's avatar
      Move the index readers out of the block cache (#5298) · 1e355842
      Levi Tamasi authored
      Currently, when the block cache is used for index blocks as well, it is
      not really the index block that is stored in the cache but an
      IndexReader object. Since this object is not pure data (it has, for
      instance, pointers that might dangle), it's not really sharable. To
      avoid the issues around this, the current code uses a dummy unique cache
      key for each TableReader to store the IndexReader, and erases the
      IndexReader entry when the TableReader is closed. Instead of doing this,
      the new code moves the IndexReader out of the cache altogether. In
      particular, instead of the TableReader owning, or caching/pinning the
      IndexReader based on the customer's settings, the TableReader
      unconditionally owns the IndexReader, which in turn owns/caches/pins
      the index block (which is itself sharable and thus can be safely put in
      the cache without any hacks).
      Note: the change has two side effects:
      1) Partitions of partitioned indexes no longer affect the read
      amplification statistics.
      2) Eviction statistics for index blocks are temporarily broken. We plan to fix
      this in a separate phase.
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/5298
      Differential Revision: D15303203
      Pulled By: ltamasi
      fbshipit-source-id: 935a69ba59d87d5e44f42e2310619b790c366e47
    • anand76's avatar
      Fix reopen voting logic in db_stress when using MultiGet (#5374) · bd44ec20
      anand76 authored
      When the --reopen option is non-zero, the DB is reopened after every ops_per_thread/(reopen+1) ops, with the check being done after every op. With MultiGet, we might do multiple ops in one iteration, which broke the logic that checked when to synchronize among the threads and reopen the DB. This PR fixes that logic.
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/5374
      Differential Revision: D15559780
      Pulled By: anand1976
      fbshipit-source-id: ee6563a68045df7f367eca3cbc2500d3e26359ef
    • Siying Dong's avatar
      Move test related files under util/ to test_util/ (#5377) · e9e0101c
      Siying Dong authored
      There are too many types of files under util/. Some test related files don't belong to there or just are just loosely related. Mo
      ve them to a new directory test_util/, so that util/ is cleaner.
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/5377
      Differential Revision: D15551366
      Pulled By: siying
      fbshipit-source-id: 0f5c8653832354ef8caa31749c0143815d719e2c
    • anand76's avatar
      Increase Trash/DB size ratio in DBSSTTest.RateLimitedWALDelete (#5366) · a984040f
      anand76 authored
      By increasing the ratio, we ensure that all files go through background deletion and eliminate flakiness due to timing of deletions.
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/5366
      Differential Revision: D15549992
      Pulled By: anand1976
      fbshipit-source-id: d137375cd791fc1a802841412755d6e2b8fd7688
    • Zhongyi Xie's avatar
      Fix FIFO dynamic options sanitization (#5367) · 87fe4bca
      Zhongyi Xie authored
      When dynamically setting options, we check the option type info and skip options that are marked deprecated. However this check is only done at top level, which results in bugs where SetOptions will corrupt option values and cause unexpected system behavior iff a deprecated second level option is set dynamically.
      For exmaple, the following call:
      was from pre 6.0 release when `ttl` was part of `compaction_options_fifo`. Now that it got moved out of `compaction_options_fifo`, this call will incorrectly set `compaction_options_fifo.max_table_files_size` to 731 (as `max_table_files_size` is the first one in `OptionsHelper::fifo_compaction_options_type_info` struct) and cause files to gett evicted much faster than expected.
      This PR adds verification to second level options like `compaction_options_fifo.ttl` or `compaction_options_fifo.max_table_files_size` when set dynamically, and filter out those marked as deprecated.
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/5367
      Differential Revision: D15530998
      Pulled By: miasantreble
      fbshipit-source-id: 818258be5c3abe09cd82d62f3c083572d70fecdd
  3. 30 May, 2019 1 commit
  4. 29 May, 2019 4 commits
    • Maysam Yabandeh's avatar
      WritePrepared: skip_concurrency_control option (#5330) · eab4f49a
      Maysam Yabandeh authored
      This enables the user to set TransactionDBOptions::skip_concurrency_control so the standard `DB::Write(const WriteOptions& opts, WriteBatch* updates)` would skip the concurrency control. This would give higher throughput to the users who know their use case doesn't need concurrency control.
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/5330
      Differential Revision: D15525932
      Pulled By: maysamyabandeh
      fbshipit-source-id: 68421ac1ba34f549a4a8de9ce4c2dccf6fb4b06b
    • Maysam Yabandeh's avatar
      WritePrepared: disableWAL in commit without prepare (#5327) · f5576c33
      Maysam Yabandeh authored
      When committing a transaction without prepare, WritePrepared simply writes the batch to db and add the commit entry to CommitCache. When two_write_queues=true, following the rule of committing only from 2nd write queue, the first write, writes the batch and the only thing the 2nd write does is to write the commit entry to CommitCache. Currently the write batch in 2nd write is set to an empty LogData entry, while the write to the WAL could simply be entirely disabled.
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/5327
      Differential Revision: D15424546
      Pulled By: maysamyabandeh
      fbshipit-source-id: 3d9ea3922d5196984c584d62a3ed57e1f7ca7b9f
    • Siying Dong's avatar
      Add comments in compaction_picker.h · 4d0c3b1f
      Siying Dong authored
      Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/5357
      Differential Revision: D15522825
      Pulled By: siying
      fbshipit-source-id: d775386b9d10c7179f5d3af2c821ed213abfacdf
    • Yanqin Jin's avatar
      Fix a clang analyze error (#5365) · b5e4ee2e
      Yanqin Jin authored
      The analyzer thinks max_allowed_ space can be 0. In that case, free_space will
      be assigned as free_space. It fails to realize that the function call
      GetFreeSpace actually sets the free_space variable properly, which is possibly
      due to lack of inter-function call analysis.
      Pull Request resolved: https://github.com/facebook/rocksdb/pull/5365
      Differential Revision: D15521790
      Pulled By: riversand963
      fbshipit-source-id: 839d0a285a1c8773a28a385f0c3be4bb7fbe32cb
  5. 25 May, 2019 15 commits
  6. 24 May, 2019 2 commits