1. 10 Feb, 2016 5 commits
    • Baraa Hamodi's avatar
      21e95811
    • Andrew Kryczka's avatar
      Env function for bulk metadata retrieval · 59b3ee65
      Andrew Kryczka authored
      Summary:
      Added this new function, which returns filename, size, and modified
      timestamp for each file in the provided directory. The default implementation
      retrieves the metadata sequentially using existing functions. In the next diff
      I'll make HdfsEnv override this function to use libhdfs's bulk get function.
      
      This won't work on windows due to the path separator.
      
      Test Plan:
      new unit test
      
        $ ./env_test --gtest_filter=EnvPosixTest.ConsistentChildrenMetadata
      
      Reviewers: yhchiang, sdong
      
      Reviewed By: sdong
      
      Subscribers: IslamAbdelRahman, dhruba, leveldb
      
      Differential Revision: https://reviews.facebook.net/D53781
      59b3ee65
    • Yueh-Hsuan Chiang's avatar
      Allows Get and MultiGet to read directly from SST files. · 4a8cbf4e
      Yueh-Hsuan Chiang authored
      Summary:
      Add kSstFileTier to ReadTier, which allows Get and MultiGet to
      read only directly from SST files and skip mem-tables.
      
          kSstFileTier = 0x2      // data in SST files.
                                // Note that this ReadTier currently only supports
                                // Get and MultiGet and does not support iterators.
      
      Test Plan: add new test in db_test.
      
      Reviewers: anthony, IslamAbdelRahman, rven, kradhakrishnan, sdong
      
      Reviewed By: sdong
      
      Subscribers: igor, dhruba, leveldb
      
      Differential Revision: https://reviews.facebook.net/D53511
      4a8cbf4e
    • agiardullo's avatar
      Transaction::UndoGetForUpdate · fe93bf9b
      agiardullo authored
      Summary: MyRocks wants to be able to un-lock a key that was just locked by GetForUpdate().  To do this safely, I am now keeping track of the number of reads(for update) and writes for each key in a transaction.  UndoGetForUpdate() will only unlock a key if it hasn't been written and the read count reaches 0.
      
      Test Plan: more unit tests
      
      Reviewers: igor, rven, yhchiang, spetrunia, sdong
      
      Reviewed By: spetrunia, sdong
      
      Subscribers: spetrunia, dhruba, leveldb
      
      Differential Revision: https://reviews.facebook.net/D47043
      fe93bf9b
    • sdong's avatar
      crash_test: cover concurrent memtable insert in default crash test · 2608219c
      sdong authored
      Summary: Default crash test uses prefix hash memtable, which is not compatible to concurrent memtable. Allow prefix test run with skip list and use skip list memtable when concurrent insert is used.
      
      Test Plan: Run "python -u tools/db_crashtest.py whitebox" and watch sometimes skip list is used.
      
      Reviewers: anthony, yhchiang, kradhakrishnan, andrewkr, IslamAbdelRahman
      
      Reviewed By: IslamAbdelRahman
      
      Subscribers: leveldb, dhruba
      
      Differential Revision: https://reviews.facebook.net/D53907
      2608219c
  2. 06 Feb, 2016 6 commits
    • sdong's avatar
      Fix LITE db_test build broken by previous commit · a76e9093
      sdong authored
      Summary: Previous commit introduces a test that is not supported in LITE. Fix it.
      
      Test Plan: Build the test with ROCKSDB_LITE.
      
      Reviewers: kradhakrishnan, IslamAbdelRahman, anthony, yhchiang, andrewkr
      
      Subscribers: leveldb, dhruba
      
      Differential Revision: https://reviews.facebook.net/D53901
      a76e9093
    • sdong's avatar
      Explictly fail when memtable doesn't support concurrent insert · b1887c5d
      sdong authored
      Summary: If users turn on concurrent insert but the memtable doesn't support it, they might see unexcepted crash. Fix it by explicitly fail.
      
      Test Plan:
      Run different setting of stress_test and make sure it fails correctly.
      Will add a unit test too.
      
      Reviewers: anthony, kradhakrishnan, IslamAbdelRahman, yhchiang, andrewkr, ngbronson
      
      Reviewed By: ngbronson
      
      Subscribers: leveldb, dhruba
      
      Differential Revision: https://reviews.facebook.net/D53895
      b1887c5d
    • Gunnar Kudrjavets's avatar
      Add option to run fillseq with WAL enabled in addition to WAL disabled · 8ed34387
      Gunnar Kudrjavets authored
      Summary: This set of changes is part of the work to introduce benchmark for universal style compaction in RocksDB. It's conceptually separate from the compaction work, so sending it out as a separate diff to get it out of the way.
      
      Test Plan:
        - Run `./tools/run_flash_bench.sh`.
        - Look at the contents of `report.txt` and `report2.txt` to make sure that data is reported and attributed correctly.
        - During `db_bench` execution time make sure that the correct flags are passed to `--disable_wal` depending on the benchmark being executed.
      
      Reviewers: MarkCallaghan
      
      Reviewed By: MarkCallaghan
      
      Subscribers: dhruba, leveldb
      
      Differential Revision: https://reviews.facebook.net/D53865
      8ed34387
    • sdong's avatar
      Update version to 4.5 · 73a9b0f4
      sdong authored
      Summary: Time to cut branch for release 4.5. Change the versions.
      
      Test Plan: Not needed
      
      Reviewers: IslamAbdelRahman, yhchiang, kradhakrishnan, andrewkr, anthony
      
      Reviewed By: anthony
      
      Subscribers: leveldb, dhruba
      
      Differential Revision: https://reviews.facebook.net/D53883
      73a9b0f4
    • reid horuff's avatar
      Improve perf of Pessimistic Transaction expirations (and optimistic transactions) · 6f71d3b6
      reid horuff authored
      Summary:
      copy from task 8196669:
      
      1) Optimistic transactions do not support batching writes from different threads.
      2) Pessimistic transactions do not support batching writes if an expiration time is set.
      
      In these 2 cases, we currently do not do any write batching in DBImpl::WriteImpl() because there is a WriteCallback that could decide at the last minute to abort the write.  But we could support batching write operations with callbacks if we make sure to process the callbacks correctly.
      
      To do this, we would first need to modify write_thread.cc to stop preventing writes with callbacks from being batched together.  Then we would need to change DBImpl::WriteImpl() to call all WriteCallback's in a batch, only write the batches that succeed, and correctly set the state of each batch's WriteThread::Writer.
      
      Test Plan: Added test WriteWithCallbackTest to write_callback_test.cc which creates multiple client threads and verifies that writes are batched and executed properly.
      
      Reviewers: hermanlee4, anthony, ngbronson
      
      Subscribers: leveldb, dhruba
      
      Differential Revision: https://reviews.facebook.net/D52863
      6f71d3b6
    • Islam AbdelRahman's avatar
      Add BlockBasedTableOptions::index_block_restart_interval · 8e6172bc
      Islam AbdelRahman authored
      Summary: Add a new option to BlockBasedTableOptions that will allow us to change the restart interval for the index block
      
      Test Plan: unit tests
      
      Reviewers: yhchiang, anthony, andrewkr, sdong
      
      Reviewed By: sdong
      
      Subscribers: march, dhruba
      
      Differential Revision: https://reviews.facebook.net/D53721
      8e6172bc
  3. 05 Feb, 2016 2 commits
  4. 04 Feb, 2016 10 commits
  5. 03 Feb, 2016 6 commits
    • Andrew Kryczka's avatar
      Eliminate duplicated property constants · 284aa613
      Andrew Kryczka authored
      Summary:
      Before this diff, there were duplicated constants to refer to properties (user-
      facing API had strings and InternalStats had an enum). I noticed these were
      inconsistent in terms of which constants are provided, names of constants, and
      documentation of constants. Overall it seemed annoying/error-prone to maintain
      these duplicated constants.
      
      So, this diff gets rid of InternalStats's constants and replaces them with a map
      keyed on the user-facing constant. The value in that map contains a function
      pointer to get the property value, so we don't need to do string matching while
      holding db->mutex_. This approach has a side benefit of making many small
      handler functions rather than a giant switch-statement.
      
      Test Plan: db_properties_test passes, running "make commit-prereq -j32"
      
      Reviewers: sdong, yhchiang, kradhakrishnan, IslamAbdelRahman, rven, anthony
      
      Reviewed By: anthony
      
      Subscribers: dhruba, leveldb
      
      Differential Revision: https://reviews.facebook.net/D53253
      284aa613
    • Gabriela Jacques da Silva's avatar
    • Gabriela Jacques da Silva's avatar
      Removing data race from expirable transactions · 0c2bd5cb
      Gabriela Jacques da Silva authored
      Summary:
      Doing inline checking of transaction expiration instead of
      using a callback.
      
      Test Plan: To be added
      
      Reviewers: anthony
      
      Reviewed By: anthony
      
      Subscribers: leveldb, dhruba
      
      Differential Revision: https://reviews.facebook.net/D53673
      0c2bd5cb
    • Nathan Bronson's avatar
      disable kConcurrentSkipList multithreaded test · 5fcd1ba3
      Nathan Bronson authored
      Summary: Disable test that is intermittently failing
      
      Test Plan: unit tests
      
      Reviewers: igor, andrewkr, sdong
      
      Reviewed By: sdong
      
      Subscribers: dhruba
      
      Differential Revision: https://reviews.facebook.net/D53715
      5fcd1ba3
    • Andrew Kryczka's avatar
      Generate tags for *.c files · 466c2c1b
      Andrew Kryczka authored
      Summary:
      db/c_test.c uses the functions in db/c.cc. If we have tags generated
      for one but not the other, it's easy to make mistakes like updating a function
      signature and missing a call site.
      
      Test Plan:
        $ make tags
      
      in vim:
      
        :cscope find s rocksdb_options_set_compression_options
        ...
        3    325  db/c_test.c <<main>>
                  rocksdb_options_set_compression_options(options, -14, -1, 0);
      
      Reviewers: sdong, yhchiang, IslamAbdelRahman
      
      Reviewed By: IslamAbdelRahman
      
      Subscribers: dhruba, leveldb
      
      Differential Revision: https://reviews.facebook.net/D53685
      466c2c1b
    • Siying Dong's avatar
      Merge pull request #960 from koldat/masterFixes2 · 70c068c9
      Siying Dong authored
      Making use of GetSystemTimePreciseAsFileTime dynamic
      70c068c9
  6. 02 Feb, 2016 11 commits