1. 19 Jul, 2017 1 commit
  2. 16 Jul, 2017 1 commit
  3. 28 Apr, 2017 1 commit
  4. 04 Apr, 2017 1 commit
  5. 16 Mar, 2017 1 commit
    • Islam AbdelRahman's avatar
      Add macros to include file name and line number during Logging · e1916368
      Islam AbdelRahman authored
      current logging
      2017/03/14-14:20:30.393432 7fedde9f5700 (Original Log Time 2017/03/14-14:20:30.393414) [default] Level summary: base level 1 max bytes base 268435456 files[1 0 0 0 0 0 0] max score 0.25
      2017/03/14-14:20:30.393438 7fedde9f5700 [JOB 2] Try to delete WAL files size 61417909, prev total WAL file size 73820858, number of live WAL files 2.
      2017/03/14-14:20:30.393464 7fedde9f5700 [DEBUG] [JOB 2] Delete /dev/shm/old_logging//MANIFEST-000001 type=3 #1 -- OK
      2017/03/14-14:20:30.393472 7fedde9f5700 [DEBUG] [JOB 2] Delete /dev/shm/old_logging//000003.log type=0 #3 -- OK
      2017/03/14-14:20:31.427103 7fedd49f1700 [default] New memtable created with log file: #9. Immutable memtables: 0.
      2017/03/14-14:20:31.427179 7fedde9f5700 [JOB 3] Syncing log #6
      2017/03/14-14:20:31.427190 7fedde9f5700 (Original Log Time 2017/03/14-14:20:31.427170) Calling FlushMemTableToOutputFile with column family [default], flush slots available 1, compaction slots allowed 1, compaction slots scheduled 1
      Closes https://github.com/facebook/rocksdb/pull/1990
      Differential Revision: D4708695
      Pulled By: IslamAbdelRahman
      fbshipit-source-id: cb8968f
  6. 10 Dec, 2016 1 commit
  7. 24 Sep, 2016 1 commit
    • Yi Wu's avatar
      Split DBOptions into ImmutableDBOptions and MutableDBOptions · 9ed928e7
      Yi Wu authored
      Summary: Use ImmutableDBOptions/MutableDBOptions internally and DBOptions only for user-facing APIs. MutableDBOptions is barely a placeholder for now. I'll start to move options to MutableDBOptions in following diffs.
      Test Plan:
        make all check
      Reviewers: yhchiang, IslamAbdelRahman, sdong
      Reviewed By: sdong
      Subscribers: andrewkr, dhruba, leveldb
      Differential Revision: https://reviews.facebook.net/D64065
  8. 10 Jun, 2016 1 commit
    • Wanning Jiang's avatar
      Backup Options · 56887f6c
      Wanning Jiang authored
      Summary: Backup options file to private directory
      Test Plan:
      backupable_db_test.cc, BackupOptions
      	   Modify DB options by calling OpenDB for 3 times. Check the latest options file is in the right place. Also check no redundent files are backuped.
      Reviewers: andrewkr
      Reviewed By: andrewkr
      Subscribers: leveldb, dhruba, andrewkr
      Differential Revision: https://reviews.facebook.net/D59373
  9. 10 Feb, 2016 1 commit
  10. 20 Jun, 2015 1 commit
    • Venkatesh Radhakrishnan's avatar
      Add wal files to Checkpoint for multiple column families. · 04251e1e
      Venkatesh Radhakrishnan authored
      When there are multiple column families, the flush in
      GetLiveFiles is not atomic, so that there are entries in the wal files
      which are needed to get a consisten RocksDB. We now add the log files to
      the checkpoint.
      Test Plan:
      CheckpointCF - This test forces more data to be written to
      the other column families after the flush of the first column family but
      before the second.
      Reviewers: igor, yhchiang, IslamAbdelRahman, anthony, kradhakrishnan, sdong
      Reviewed By: sdong
      Subscribers: dhruba, leveldb
      Differential Revision: https://reviews.facebook.net/D40323
  11. 20 Mar, 2015 1 commit
    • Igor Canadi's avatar
      Don't delete files when column family is dropped · b088c83e
      Igor Canadi authored
      To understand the bug read t5943287 and check out the new test in column_family_test (ReadDroppedColumnFamily), iter 0.
      RocksDB contract allowes you to read a drop column family as long as there is a live reference. However, since our iteration ignores dropped column families, AddLiveFiles() didn't mark files of a dropped column families as live. So we deleted them.
      In this patch I no longer ignore dropped column families in the iteration. I think this behavior was confusing and it also led to this bug. Now if an iterator client wants to ignore dropped column families, he needs to do it explicitly.
      Test Plan: Added a new unit test that is failing on master. Unit test succeeds now.
      Reviewers: sdong, rven, yhchiang
      Reviewed By: yhchiang
      Subscribers: dhruba, leveldb
      Differential Revision: https://reviews.facebook.net/D32535
  12. 13 Feb, 2015 1 commit
    • Igor Canadi's avatar
      Introduce job_id for flush and compaction · e7ea51a8
      Igor Canadi authored
      It would be good to assing background job their IDs. Two benefits:
      1) makes LOGs more readable
      2) I might use it in my EventLogger, which will try to make our LOG easier to read/query/visualize
      Test Plan: ran rocksdb, read the LOG
      Reviewers: sdong, rven, yhchiang
      Reviewed By: yhchiang
      Subscribers: dhruba, leveldb
      Differential Revision: https://reviews.facebook.net/D31617
  13. 05 Feb, 2015 1 commit
    • Yueh-Hsuan Chiang's avatar
      Add a counter for collecting the wait time on db mutex. · 181191a1
      Yueh-Hsuan Chiang authored
      Add a counter for collecting the wait time on db mutex.
      Also add MutexWrapper and CondVarWrapper for measuring wait time.
      Test Plan:
      export ROCKSDB_TESTS=MutexWaitStats
      verify stats output using db_bench
      make clean
      make release
      ./db_bench --statistics=1 --benchmarks=fillseq,readwhilewriting --num=10000 --threads=10
      Sample output:
          rocksdb.db.mutex.wait.micros COUNT : 7546866
      Reviewers: MarkCallaghan, rven, sdong, igor
      Reviewed By: igor
      Subscribers: dhruba, leveldb
      Differential Revision: https://reviews.facebook.net/D32787
  14. 21 Nov, 2014 1 commit
  15. 15 Nov, 2014 3 commits
  16. 05 Nov, 2014 1 commit
  17. 30 Oct, 2014 2 commits
    • Igor Canadi's avatar
      WalManager · 63590548
      Igor Canadi authored
      Summary: Decoupling code that deals with archived log files outside of DBImpl. That will make this code easier to reason about and test. It will also make the code easier to improve, because an improver doesn't have to understand DBImpl code in entirety.
      Test Plan: added test
      Reviewers: ljin, yhchiang, rven, sdong
      Reviewed By: sdong
      Subscribers: dhruba, leveldb
      Differential Revision: https://reviews.facebook.net/D27873
    • Yueh-Hsuan Chiang's avatar
      Apply InfoLogLevel to the logs in db/db_filesnapshot.cc · 7b3a618f
      Yueh-Hsuan Chiang authored
      Summary: Apply InfoLogLevel to the logs in db/db_filesnapshot.cc
      Test Plan: make
      Reviewers: ljin, sdong, rven, igor
      Reviewed By: igor
      Subscribers: dhruba, leveldb
      Differential Revision: https://reviews.facebook.net/D27813
  18. 29 Oct, 2014 1 commit
    • Igor Canadi's avatar
      FlushProcess · a39e931e
      Igor Canadi authored
      Abstract out FlushProcess and take it out of DBImpl.
      This also includes taking DeletionState outside of DBImpl.
      Currently this diff is only doing the refactoring. Future work includes:
      1. Decoupling flush_process.cc, make it depend on less state
      2. Write flush_process_test, which will mock out everything that FlushProcess depends on and test it in isolation
      Test Plan: make check
      Reviewers: rven, yhchiang, sdong, ljin
      Reviewed By: ljin
      Subscribers: dhruba, leveldb
      Differential Revision: https://reviews.facebook.net/D27561
  19. 06 Sep, 2014 1 commit
  20. 05 Sep, 2014 1 commit
  21. 27 Aug, 2014 1 commit
  22. 03 Jul, 2014 1 commit
  23. 21 May, 2014 1 commit
  24. 16 Apr, 2014 1 commit
    • Igor Canadi's avatar
      RocksDBLite · 588bca20
      Igor Canadi authored
      Introducing RocksDBLite! Removes all the non-essential features and reduces the binary size. This effort should help our adoption on mobile.
      Binary size when compiling for IOS (`TARGET_OS=IOS m static_lib`) is down to 9MB from 15MB (without stripping)
      Test Plan: compiles :)
      Reviewers: dhruba, haobo, ljin, sdong, yhchiang
      Reviewed By: yhchiang
      CC: leveldb
      Differential Revision: https://reviews.facebook.net/D17835
  25. 08 Apr, 2014 1 commit
  26. 04 Apr, 2014 1 commit
  27. 03 Apr, 2014 1 commit
    • Haobo Xu's avatar
      [RocksDB] Fix a race condition in GetSortedWalFiles · 48bc0c6a
      Haobo Xu authored
      Summary: This patch fixed a race condition where a log file is moved to archived dir in the middle of GetSortedWalFiles. Without the fix, the log file would be missed in the result, which leads to transaction log iterator gap. A test utility SyncPoint is added to help reproducing the race condition.
      Test Plan: TransactionLogIteratorRace; make check
      Reviewers: dhruba, ljin
      Reviewed By: dhruba
      CC: leveldb
      Differential Revision: https://reviews.facebook.net/D17121
  28. 26 Feb, 2014 1 commit
    • Igor Canadi's avatar
      [CF] Adaptation of GetLiveFiles for CF · dc277f0a
      Igor Canadi authored
      Summary: Even if user flushes the memtables before getting live files, we still can't guarantee that new data didn't come in (to already-flushed memtables). If we want backups to provide consistent view of the database, we still need to get WAL files.
      Test Plan: backupable_db_test
      Reviewers: dhruba
      CC: leveldb
      Differential Revision: https://reviews.facebook.net/D16299
  29. 15 Feb, 2014 1 commit
    • Igor Canadi's avatar
      [CF] DB test to run on non-default column family · c67d48c8
      Igor Canadi authored
      This is a huge diff and it was hectic, but the idea is actually quite simple. Every operation (Put, Get, etc.) done on default column family in DBTest is now forwarded to non-default ("pikachu"). The good news is that we had zero test failures! Column families look stable so far.
      One interesting test that I adapted for column families is MultiThreadedTest. I replaced every Put() with a WriteBatch writing to all column families concurrently. Every Put in the write batch contains unique_id. Instead of Get() I do a multiget across all column families with the same key. If atomicity holds, I expect to see the same unique_id in all column families.
      Test Plan: This is a test!
      Reviewers: dhruba, haobo, kailiu, sdong
      CC: leveldb
      Differential Revision: https://reviews.facebook.net/D16149
  30. 13 Feb, 2014 1 commit
    • Igor Canadi's avatar
      [CF] Rethinking ColumnFamilyHandle and fix to dropping column families · b06840aa
      Igor Canadi authored
      The change to the public behavior:
      * When opening a DB or creating new column family client gets a ColumnFamilyHandle.
      * As long as column family handle is alive, client can do whatever he wants with it, even drop it
      * Dropped column family can still be read from (using the column family handle)
      * Added a new call CloseColumnFamily(). Client has to close all column families that he has opened before deleting the DB
      * As soon as column family is closed, any calls to DB using that column family handle will fail (also any outstanding calls)
      * Ref-counting ColumnFamilyData
      * New thread-safety for ColumnFamilySet
      * Dropped column families are now completely dropped and their memory cleaned-up
      Test Plan: added some tests to column_family_test
      Reviewers: dhruba, haobo, kailiu, sdong
      CC: leveldb
      Differential Revision: https://reviews.facebook.net/D16101
  31. 30 Jan, 2014 1 commit
  32. 28 Jan, 2014 1 commit
  33. 16 Jan, 2014 1 commit
    • Igor Canadi's avatar
      Move functions from VersionSet to Version · 2f4eda78
      Igor Canadi authored
      There were some functions in VersionSet that had no reason to be there instead of Version. Moving them to Version will make column families implementation easier.
      The functions moved are:
      * NumLevelBytes
      * LevelSummary
      * LevelFileSummary
      * MaxNextLevelOverlappingBytes
      * AddLiveFiles (previously AddLiveFilesCurrentVersion())
      * NeedSlowdownForNumLevel0Files
      The diff continues on (and depends on) D15171
      Test Plan: make check
      Reviewers: dhruba, haobo, kailiu, sdong, emayanke
      Reviewed By: sdong
      CC: leveldb
      Differential Revision: https://reviews.facebook.net/D15183
  34. 02 Jan, 2014 1 commit
    • Igor Canadi's avatar
      Support multi-threaded DisableFileDeletions() and EnableFileDeletions() · b60c14f6
      Igor Canadi authored
      We don't want two threads to clash if they concurrently call DisableFileDeletions() and EnableFileDeletions(). I'm adding a counter that will enable file deletions only after all DisableFileDeletions() calls have been negated with EnableFileDeletions().
      However, we also don't want to break the old behavior, so I added a parameter force to EnableFileDeletions(). If force is true, we will still enable file deletions after every call to EnableFileDeletions(), which is what is happening now.
      Test Plan: make check
      Reviewers: dhruba, haobo, sanketh
      Reviewed By: dhruba
      CC: leveldb
      Differential Revision: https://reviews.facebook.net/D14781
  35. 09 Nov, 2013 1 commit
    • Igor Canadi's avatar
      Speed up FindObsoleteFiles · 1510339e
      Igor Canadi authored
      Here's one solution we discussed on speeding up FindObsoleteFiles. Keep a set of all files in DBImpl and update the set every time we create a file. I probably missed few other spots where we create a file.
      It might speed things up a bit, but makes code uglier. I don't really like it.
      Much better approach would be to abstract all file handling to a separate class. Think of it as layer between DBImpl and Env. Having a separate class deal with file namings and deletion would benefit both code cleanliness (especially with huge DBImpl) and speed things up. It will take a huge effort to do this, though.
      Let's discuss offline today.
      Test Plan: Ran ./db_stress, verified that files are getting deleted
      Reviewers: dhruba, haobo, kailiu, emayanke
      Reviewed By: dhruba
      Differential Revision: https://reviews.facebook.net/D13827
  36. 25 Oct, 2013 1 commit
    • Mayank Agarwal's avatar
      Unify DeleteFile and DeleteWalFiles · 56305221
      Mayank Agarwal authored
      This is to simplify rocksdb public APIs and improve the code quality.
      Created an additional parameter to ParseFileName for log sub type and improved the code for deleting a wal file.
      Wrote exhaustive unit-tests in delete_file_test
      Unification of other redundant APIs can be taken up in a separate diff
      Test Plan: Expanded delete_file test
      Reviewers: dhruba, haobo, kailiu, sdong
      Reviewed By: dhruba
      CC: leveldb
      Differential Revision: https://reviews.facebook.net/D13647
  37. 17 Oct, 2013 1 commit