1. 16 Mar, 2017 1 commit
    • Islam AbdelRahman's avatar
      Add macros to include file name and line number during Logging · e1916368
      Islam AbdelRahman authored
      Summary:
      current logging
      ```
      2017/03/14-14:20:30.393432 7fedde9f5700 (Original Log Time 2017/03/14-14:20:30.393414) [default] Level summary: base level 1 max bytes base 268435456 files[1 0 0 0 0 0 0] max score 0.25
      2017/03/14-14:20:30.393438 7fedde9f5700 [JOB 2] Try to delete WAL files size 61417909, prev total WAL file size 73820858, number of live WAL files 2.
      2017/03/14-14:20:30.393464 7fedde9f5700 [DEBUG] [JOB 2] Delete /dev/shm/old_logging//MANIFEST-000001 type=3 #1 -- OK
      2017/03/14-14:20:30.393472 7fedde9f5700 [DEBUG] [JOB 2] Delete /dev/shm/old_logging//000003.log type=0 #3 -- OK
      2017/03/14-14:20:31.427103 7fedd49f1700 [default] New memtable created with log file: #9. Immutable memtables: 0.
      2017/03/14-14:20:31.427179 7fedde9f5700 [JOB 3] Syncing log #6
      2017/03/14-14:20:31.427190 7fedde9f5700 (Original Log Time 2017/03/14-14:20:31.427170) Calling FlushMemTableToOutputFile with column family [default], flush slots available 1, compaction slots allowed 1, compaction slots scheduled 1
      2017/03/14-14:20:31.
      Closes https://github.com/facebook/rocksdb/pull/1990
      
      Differential Revision: D4708695
      
      Pulled By: IslamAbdelRahman
      
      fbshipit-source-id: cb8968f
      e1916368
  2. 15 Mar, 2017 2 commits
  3. 14 Mar, 2017 5 commits
    • Maysam Yabandeh's avatar
      Pinnableslice (2nd attempt) · 11526252
      Maysam Yabandeh authored
      Summary:
      PinnableSlice
      
          Summary:
          Currently the point lookup values are copied to a string provided by the
          user. This incures an extra memcpy cost. This patch allows doing point lookup
          via a PinnableSlice which pins the source memory location (instead of
          copying their content) and releases them after the content is consumed
          by the user. The old API of Get(string) is translated to the new API
          underneath.
      
          Here is the summary for improvements:
      
          value 100 byte: 1.8% regular, 1.2% merge values
          value 1k byte: 11.5% regular, 7.5% merge values
          value 10k byte: 26% regular, 29.9% merge values
          The improvement for merge could be more if we extend this approach to
          pin the merge output and delay the full merge operation until the user
          actually needs it. We have put that for future work.
      
          PS:
          Sometimes we observe a small decrease in performance when switching from
          t5452014 to this patch but with the old Get(string) API. The d
      Closes https://github.com/facebook/rocksdb/pull/1756
      
      Differential Revision: D4391738
      
      Pulled By: maysamyabandeh
      
      fbshipit-source-id: 6f3edd3
      11526252
    • Aaron Gao's avatar
      update history.md for fixing the bug that skips keys · e5bd8def
      Aaron Gao authored
      Summary: Closes https://github.com/facebook/rocksdb/pull/1986
      
      Differential Revision: D4699152
      
      Pulled By: siying
      
      fbshipit-source-id: b18c32c
      e5bd8def
    • Sagar Vemuri's avatar
      Add a new SstFileWriter constructor without explicit comparator · 1ffbdfd9
      Sagar Vemuri authored
      Summary:
      The comparator param in SstFileWriter constructor is redundant as it already exists as a field in options. So the current SstFileWriter constructor should be deprecated in favor of a new one which does not take a comparator.
      Note that the jni/java apis have not been touched yet.
      Closes https://github.com/facebook/rocksdb/pull/1978
      
      Differential Revision: D4685629
      
      Pulled By: sagar0
      
      fbshipit-source-id: 372ce96
      1ffbdfd9
    • Reid Horuff's avatar
      Add ability to search for key prefix in sst_dump tool · ebd5639b
      Reid Horuff authored
      Summary:
      Add the flag --prefix to the sst_dump tool
      This flag is similar to, and exclusive from, the --from flag.
      
      --prefix=0x00FF will return all rows prefixed with 0x00FF.
      The --to flag may also be specified and will work as expected.
      
      These changes were used to help in debugging the power cycle corruption issue and theses changes were tested by scanning through a udb.
      Closes https://github.com/facebook/rocksdb/pull/1984
      
      Differential Revision: D4691814
      
      Pulled By: reidHoruff
      
      fbshipit-source-id: 027f261
      ebd5639b
    • Maysam Yabandeh's avatar
      Fix some bugs in MockEnv · e6725e8c
      Maysam Yabandeh authored
      Summary:
      Fixing some bugs in MockEnv so it be actually used.
      Closes https://github.com/facebook/rocksdb/pull/1914
      
      Differential Revision: D4609923
      
      Pulled By: maysamyabandeh
      
      fbshipit-source-id: ca25735
      e6725e8c
  4. 11 Mar, 2017 1 commit
  5. 10 Mar, 2017 2 commits
  6. 09 Mar, 2017 6 commits
  7. 08 Mar, 2017 4 commits
  8. 07 Mar, 2017 3 commits
    • Matteo Merli's avatar
      Expose DB::DeleteRange and WriteBath::DeleteRange in Java · 5f65dc87
      Matteo Merli authored
      Summary:
      Added JNI wrapper from `DeleteRange` methods
      Closes https://github.com/facebook/rocksdb/pull/1951
      
      Differential Revision: D4657746
      
      Pulled By: yiwu-arbug
      
      fbshipit-source-id: 3fc7ab8
      5f65dc87
    • Reid Horuff's avatar
      Set logs as getting flushed before releasing lock, race condition fix · 58b12dfe
      Reid Horuff authored
      Summary:
      Relating to #1903:
      
      In MaybeFlushColumnFamilies() we want to modify the 'getting_flushed' flag before releasing the db mutex when SwitchMemtable() is called.
      
      The following 2 actions need to be atomic in MaybeFlushColumnFamilies()
      - getting_flushed is false on oldest log
      - we determine that all CFs can be flushed to successfully release oldest log
      - we set getting_flushed = true on the oldest log.
      -------
      - getting_flushed is false on oldest log
      - we determine that all CFs can NOT be flushed to successfully release oldest log
      - we set unable_to_flush_oldest_log_ = true on the oldest log.
      
      #### In the 2pc case:
      
      T1 enters function but is unable to flush all CFs to release log
      T1 sets unable_to_flush_oldest_log_ = true
      T1 begins flushing all CFs possible
      
      T2 enters function but is unable to flush all CFs to release log
      T2 sees unable_to_flush_oldes_log_ has been set so exits
      
      T3 enters function and will be able to flush all CFs to release oldest log
      T3 sets getting_flushed = true on oldes
      Closes https://github.com/facebook/rocksdb/pull/1909
      
      Differential Revision: D4646235
      
      Pulled By: reidHoruff
      
      fbshipit-source-id: c8d0447
      58b12dfe
    • Siying Dong's avatar
      Move db_test and external_sst_file_test out of Travis's MAC OS run · f8a4ea02
      Siying Dong authored
      Summary:
      After we have db_basic_test and external_sst_file_basic_test, we don't need to run db_test and external_sst_file_test in Travis's MAC OS run anymore. Move it out.
      Closes https://github.com/facebook/rocksdb/pull/1940
      
      Differential Revision: D4659361
      
      Pulled By: siying
      
      fbshipit-source-id: e64e291
      f8a4ea02
  9. 06 Mar, 2017 1 commit
    • Maysam Yabandeh's avatar
      Fix a bug in tests in options operator= · 534581a3
      Maysam Yabandeh authored
      Summary:
      Note: Using the default operator= is an unsafe approach for Options since it destructs shared_ptr in
      the same order of their creation, in contrast to destructors which
      destructs them in the opposite order of creation. One particular problme is
      that the cache destructor might invoke callback functions that use Option
      members such as statistics. To work around this problem, we manually call
      destructor of table_facotry which eventually clears the block cache.
      Closes https://github.com/facebook/rocksdb/pull/1950
      
      Differential Revision: D4655473
      
      Pulled By: maysamyabandeh
      
      fbshipit-source-id: 6c4bbff
      534581a3
  10. 04 Mar, 2017 3 commits
  11. 03 Mar, 2017 6 commits
  12. 01 Mar, 2017 4 commits
  13. 28 Feb, 2017 2 commits
    • Islam AbdelRahman's avatar
      Move advanced column family options to advanced_options.h · 08864df2
      Islam AbdelRahman authored
      Summary:
      For the sake of making our options simpler, we should keep options.h as simple as possible and move more advanced/less common options to advaned_options.h
      
      I started with ColumnFamilyOptions and also did some re-ordering
      
      I have moved all ColumnFamilyOptions to advanced_options.h and only left these options in options.h
      
      ```
      const Comparator* comparator = BytewiseComparator();
      std::shared_ptr<MergeOperator> merge_operator = nullptr;
      const CompactionFilter* compaction_filter = nullptr;
      std::shared_ptr<CompactionFilterFactory> compaction_filter_factory = nullptr;
      size_t write_buffer_size = 64 << 20;
      CompressionType compression;
      int level0_file_num_compaction_trigger = 4;
      bool disable_auto_compactions = false;
      ```
      Please feel free to comment on specific options if you think they should be advanced or should not be
      Closes https://github.com/facebook/rocksdb/pull/1847
      
      Differential Revision: D4519996
      
      Pulled By: IslamAbdelRahman
      
      fbshipit-source-id: abebd9a
      08864df2
    • Peter (Stig) Edwards's avatar
      Get unique_ptr to use delete[] for char[] in DumpMallocStats · 2ca2059f
      Peter (Stig) Edwards authored
      Summary:
      Avoid mismatched free() / delete / delete [] in DumpMallocStats
      Closes https://github.com/facebook/rocksdb/pull/1927
      
      Differential Revision: D4622045
      
      Pulled By: siying
      
      fbshipit-source-id: 1131b30
      2ca2059f