1. 20 Mar, 2015 2 commits
    • Igor Canadi's avatar
      Don't delete files when column family is dropped · b088c83e
      Igor Canadi authored
      Summary:
      To understand the bug read t5943287 and check out the new test in column_family_test (ReadDroppedColumnFamily), iter 0.
      
      RocksDB contract allowes you to read a drop column family as long as there is a live reference. However, since our iteration ignores dropped column families, AddLiveFiles() didn't mark files of a dropped column families as live. So we deleted them.
      
      In this patch I no longer ignore dropped column families in the iteration. I think this behavior was confusing and it also led to this bug. Now if an iterator client wants to ignore dropped column families, he needs to do it explicitly.
      
      Test Plan: Added a new unit test that is failing on master. Unit test succeeds now.
      
      Reviewers: sdong, rven, yhchiang
      
      Reviewed By: yhchiang
      
      Subscribers: dhruba, leveldb
      
      Differential Revision: https://reviews.facebook.net/D32535
      b088c83e
    • Igor Sugak's avatar
      rocksdb: initial util/testharness clean up · 17ae3fcb
      Igor Sugak authored
      Summary: Deleted some redundant code. More comming.
      
      Test Plan:
      ```lang=bash
      % USE_CLANG=1 make check
      ```
      
      Reviewers: meyering, sdong, igor
      
      Reviewed By: igor
      
      Subscribers: dhruba, leveldb
      
      Differential Revision: https://reviews.facebook.net/D35463
      17ae3fcb
  2. 19 Mar, 2015 14 commits
  3. 18 Mar, 2015 5 commits
    • agiardullo's avatar
      Create an abstract interface for write batches · 81345b90
      agiardullo authored
      Summary: WriteBatch and WriteBatchWithIndex now both inherit from a common abstract base class.  This makes it easier to write code that is agnostic toward the implementation of the particular write batch.  In particular, I plan on utilizing this abstraction to allow transactions to support using either implementation of a write batch.
      
      Test Plan: modified existing WriteBatchWithIndex tests to test new functions.  Running all tests.
      
      Reviewers: igor, rven, yhchiang, sdong
      
      Reviewed By: sdong
      
      Subscribers: dhruba, leveldb
      
      Differential Revision: https://reviews.facebook.net/D34017
      81345b90
    • Igor Canadi's avatar
      Only run db_test in Travis · 46214df4
      Igor Canadi authored
      46214df4
    • Igor Canadi's avatar
      Deprecate removeScanCountLimit in NewLRUCache · c88ff4ca
      Igor Canadi authored
      Summary: It is no longer used by the implementation, so we should also remove it from the public API.
      
      Test Plan: make check
      
      Reviewers: sdong
      
      Reviewed By: sdong
      
      Subscribers: dhruba, leveldb
      
      Differential Revision: https://reviews.facebook.net/D34971
      c88ff4ca
    • Igor Sugak's avatar
      rocksdb: switch to gtest · b4b69e4f
      Igor Sugak authored
      Summary:
      Our existing test notation is very similar to what is used in gtest. It makes it easy to adopt what is different.
      In this diff I modify existing [[ https://code.google.com/p/googletest/wiki/Primer#Test_Fixtures:_Using_the_Same_Data_Configuration_for_Multiple_Te | test fixture ]] classes to inherit from `testing::Test`. Also for unit tests that use fixture class, `TEST` is replaced with `TEST_F` as required in gtest.
      
      There are several custom `main` functions in our existing tests. To make this transition easier, I modify all `main` functions to fallow gtest notation. But eventually we can remove them and use implementation of `main` that gtest provides.
      
      ```lang=bash
      % cat ~/transform
      #!/bin/sh
      files=$(git ls-files '*test\.cc')
      for file in $files
      do
        if grep -q "rocksdb::test::RunAllTests()" $file
        then
          if grep -Eq '^class \w+Test {' $file
          then
            perl -pi -e 's/^(class \w+Test) {/${1}: public testing::Test {/g' $file
            perl -pi -e 's/^(TEST)/${1}_F/g' $file
          fi
          perl -pi -e 's/(int main.*\{)/${1}::testing::InitGoogleTest(&argc, argv);/g' $file
          perl -pi -e 's/rocksdb::test::RunAllTests/RUN_ALL_TESTS/g' $file
        fi
      done
      % sh ~/transform
      % make format
      ```
      
      Second iteration of this diff contains only scripted changes.
      
      Third iteration contains manual changes to fix last errors and make it compilable.
      
      Test Plan:
      Build and notice no errors.
      ```lang=bash
      % USE_CLANG=1 make check -j55
      ```
      Tests are still testing.
      
      Reviewers: meyering, sdong, rven, igor
      
      Reviewed By: igor
      
      Subscribers: dhruba, leveldb
      
      Differential Revision: https://reviews.facebook.net/D35157
      b4b69e4f
    • Igor Canadi's avatar
      Merge pull request #540 from dalgaaf/wip-da-fix-elif · 413e3527
      Igor Canadi authored
      Fix '#elif with no expression'
      413e3527
  4. 17 Mar, 2015 11 commits
  5. 16 Mar, 2015 2 commits
  6. 15 Mar, 2015 6 commits
    • Igor Canadi's avatar
      Make RecordIn/RecordOut human readable · c6967a1a
      Igor Canadi authored
      Summary: I had hard time understanding these big numbers. Here's how the output looks like now: https://gist.github.com/igorcanadi/4c39c17685049584a992
      
      Test Plan: db_bench
      
      Reviewers: sdong, MarkCallaghan
      
      Reviewed By: MarkCallaghan
      
      Subscribers: dhruba, leveldb
      
      Differential Revision: https://reviews.facebook.net/D35073
      c6967a1a
    • Mark Callaghan's avatar
      Stop printing per-level stall times. · c8da6703
      Mark Callaghan authored
      Summary:
      Per-level stall times are the suggested stall time, not the actual stall time so this change stops printing them
      both in the per-level output lines and in the summary. Also changed output for total stall time to include units
      in all cases. The new output looks like:
      Level   Files   Size(MB) Score Read(GB)  Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) Stall(cnt)    RecordIn   RecordDrop
      ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
        L0     4/1          7   0.8      0.0     0.0      0.0       0.6      0.6       0.0   0.0      0.0     12.9        50       352    0.141        882            0            0
        L1     5/0          9   0.9      0.0     0.0      0.0       0.0      0.0       0.6   0.0      0.0      0.0         0         0    0.000          0            0            0
        L2    54/0         99   1.0      0.0     0.0      0.0       0.0      0.0       0.6   0.0      0.0      0.0         0         0    0.000          0            0            0
        L3   289/0        527   0.5      0.0     0.0      0.0       0.0      0.0       0.5   0.0      0.0      0.0         0         0    0.000          0            0            0
       Sum   352/1        642   0.0      0.0     0.0      0.0       0.6      0.6       1.7   1.0      0.0     12.9        50       352    0.141        882            0            0
       Int     0/0          0   0.0      0.0     0.0      0.0       0.0      0.0       0.0   1.0      0.0     15.5         0         3    0.118          7            0            0
      Flush(GB): accumulative 0.627, interval 0.005
      Stalls(count): 0 level0_slowdown, 0 level0_numfiles, 882 memtable_compaction, 0 leveln_slowdown_soft, 0 leveln_slowdown_hard
      
      Task ID: #6493861
      
      Blame Rev:
      
      Test Plan:
      run db_bench, look at output
      
      Revert Plan:
      
      Database Impact:
      
      Memcache Impact:
      
      Other Notes:
      
      EImportant:
      
      - begin *PUBLIC* platform impact section -
      Bugzilla: #
      - end platform impact -
      
      Reviewers: igor
      
      Reviewed By: igor
      
      Subscribers: dhruba
      
      Differential Revision: https://reviews.facebook.net/D35085
      c8da6703
    • fyrz's avatar
      [RocksJava] OptimizeFiltersForHits · 04778a94
      fyrz authored
      Summary: Added optimize_filters_for_hits option.
      
      Test Plan:
      make clean jclean rocksdbjava jtest
      mvn -f rocksjni.pom package
      
      Reviewers: adamretter, yhchiang, ankgup87
      
      Subscribers: dhruba
      
      Differential Revision: https://reviews.facebook.net/D35013
      04778a94
    • fyrz's avatar
      RocksJava - JNI Logger callback · 57f2a00c
      fyrz authored
      Summary:
      Within this commit a new AbstractLogger was introduced
      which allows to handle log messages at an application level.
      
      Log messages are passed up to Java using a JNI callback.
      
      This allows a Java-Developer to use common Java APIs for log
      messages e.g. SLF4J, LOG4J, etc. Within this commit no new
      dependencies were introduced, which keeps the RocksDB API clean
      and doesn`t force a developer to use a predefined high-level Java API.
      
      Another feature is to dynamically set a custom loggers verbosity at
      runtime using its public method `setInfoLogLevel` and to retrieve
      the currently active level using the `infoLogLevel` method.
      
      Test Plan:
      make clean jclean rocksdbjava jtest
      mvn -f rocksjni.pom package
      
      Reviewers: adamretter, ankgup87, yhchiang
      
      Subscribers: dhruba
      
      Differential Revision: https://reviews.facebook.net/D34755
      57f2a00c
    • fyrz's avatar
      [RocksJava] Change log level at runtime in custom logger · 814627af
      fyrz authored
      This commit introduces the possibility to change the log level
      of a used custom logger at runtime.
      814627af
    • fyrz's avatar
      [RocksJava] Logging JNI callback · a3bd4142
      fyrz authored
      Within this commit a new AbstractLogger was introduced
      which pushes info log messages all the way up to Java.
      a3bd4142