1. 11 Aug, 2016 6 commits
    • sdong's avatar
      Increase RocksDB version · 5370f44a
      sdong authored
      Summary: RocksDB 4.11 has been branched out. Now need to move to 4.12.
      
      Test Plan: N/A
      
      Reviewers: kradhakrishnan, andrewkr, IslamAbdelRahman
      
      Reviewed By: IslamAbdelRahman
      
      Subscribers: leveldb, andrewkr, dhruba
      
      Differential Revision: https://reviews.facebook.net/D61845
      5370f44a
    • sdong's avatar
      read_options.background_purge_on_iterator_cleanup to cover forward iterator... · 56dd0341
      sdong authored
      read_options.background_purge_on_iterator_cleanup to cover forward iterator and log file closing too.
      
      Summary: With read_options.background_purge_on_iterator_cleanup=true, File deletion and closing can still happen in forward iterator, or WAL file closing. Cover those cases too.
      
      Test Plan: I am adding unit tests.
      
      Reviewers: andrewkr, IslamAbdelRahman, yiwu
      
      Reviewed By: yiwu
      
      Subscribers: leveldb, andrewkr, dhruba
      
      Differential Revision: https://reviews.facebook.net/D61503
      56dd0341
    • Islam AbdelRahman's avatar
      UniversalCompaction should ignore sorted runs being compacted (when compacting for file num) · ccecf3f4
      Islam AbdelRahman authored
      Summary:
      If we have total number of sorted runs greater than level0_file_num_compaction_trigger, Universal compaction will always issue a compaction
      even if the number of sorted runs that are not being compacted is less than level0_file_num_compaction_trigger.
      
      This diff changes this behaviour to relay on the `number of sorted runs not being compacted` instead of `total number of sorted runs`
      
      Test Plan: New unit test
      
      Reviewers: sdong
      
      Reviewed By: sdong
      
      Subscribers: andrewkr, dhruba
      
      Differential Revision: https://reviews.facebook.net/D61533
      ccecf3f4
    • Andrew Kryczka's avatar
      Remove non-gtest from parallelized tests · 1b0069ce
      Andrew Kryczka authored
      Summary:
      compact_on_deletion_collector_test does not support --gtest_list_tests
      since it isn't gtest, so the full program would run for the target
      gen_parallel_tests. This caused gen_parallel_tests to take 8+ minutes for tsan
      and prevented compact_on_deletion_collector_test from running during check_0
      since no t/run-* script could be generated.
      
      Test Plan:
      run make check, verify generating t/run-* scripts is fast and
      ./compact_on_deletion_collector_test is now run
      
      Reviewers: IslamAbdelRahman, wanning, lightmark, sdong
      
      Reviewed By: sdong
      
      Subscribers: andrewkr, dhruba, leveldb
      
      Differential Revision: https://reviews.facebook.net/D61695
      1b0069ce
    • sdong's avatar
      Change HISTORY.md for release 4.11 · 638c49f2
      sdong authored
      Summary:
      Need to change HISTORY.md for 4.11.
      4.10 was not updated either. Update it together.
      
      Test Plan: Not needed.
      
      Reviewers: kradhakrishnan, andrewkr, IslamAbdelRahman
      
      Reviewed By: IslamAbdelRahman
      
      Subscribers: leveldb, andrewkr, dhruba
      
      Differential Revision: https://reviews.facebook.net/D61665
      638c49f2
    • Jay's avatar
      fix vs generator (#1269) · 6b8e9c68
      Jay authored
      6b8e9c68
  2. 10 Aug, 2016 3 commits
  3. 09 Aug, 2016 2 commits
  4. 07 Aug, 2016 1 commit
  5. 06 Aug, 2016 8 commits
  6. 05 Aug, 2016 3 commits
  7. 04 Aug, 2016 4 commits
    • omegaga's avatar
      Fix flaky test `ObsoleteFiles` · c3a4bea5
      omegaga authored
      Summary: The test `ObsoleteFiles` failed occasionally on slow device. This problem appears on Travis CI several times. The reason is that we did not wait until compaction jobs are finished in the test, while in slower device the background jobs take longer time to finish.
      
      Test Plan: Pass existing tests.
      
      Reviewers: yiwu, sdong
      
      Reviewed By: sdong
      
      Subscribers: andrewkr, dhruba, leveldb
      
      Differential Revision: https://reviews.facebook.net/D61479
      c3a4bea5
    • omegaga's avatar
      Fix failed test · 8234faab
      omegaga authored
      Summary: Fix failed persistent cache test introduced by recent PR to support cross-platform build.
      
      Test Plan: Run persistent_cache_test (on multiple platforms)
      
      Reviewers: sdong
      
      Reviewed By: sdong
      
      Subscribers: andrewkr, dhruba, leveldb
      
      Differential Revision: https://reviews.facebook.net/D61467
      8234faab
    • Islam AbdelRahman's avatar
      Remove deprecated LEVELDB_PLATFORM_POSIX · 4990c0d1
      Islam AbdelRahman authored
      Summary:
      We have replaced LEVELDB_PLATFORM_POSIX with ROCKSDB_PLATFORM_POSIX in our code
      replace it in Makefile
      
      Test Plan: make check
      
      Reviewers: sdong, yiwu, andrewkr, lightmark
      
      Reviewed By: lightmark
      
      Subscribers: andrewkr, dhruba
      
      Differential Revision: https://reviews.facebook.net/D61365
      4990c0d1
    • omegaga's avatar
      Fix clang on macOS · 7323e4c8
      omegaga authored
      Summary: Fix problem when casting `size_t` to `uint64_t`. Avoid usage of `size_t` when possible.
      
      Test Plan: Pass related test on Travis CI.
      
      Reviewers: sdong
      
      Reviewed By: sdong
      
      Subscribers: andrewkr, dhruba, leveldb
      
      Differential Revision: https://reviews.facebook.net/D61443
      7323e4c8
  8. 03 Aug, 2016 6 commits
  9. 02 Aug, 2016 6 commits
    • Islam AbdelRahman's avatar
      Fix parallel tests `make check -j` · 0155c73d
      Islam AbdelRahman authored
      Summary:
      parallel tests are broken because gnu_parallel is reading deprecated options from `/etc/parallel/config`
      Fix this by passing `--plain` to ignore `/etc/parallel/config`
      
      Test Plan: make check -j64
      
      Reviewers: kradhakrishnan, sdong, andrewkr, yiwu, arahut
      
      Reviewed By: arahut
      
      Subscribers: andrewkr, dhruba
      
      Differential Revision: https://reviews.facebook.net/D61359
      0155c73d
    • krad's avatar
      Fix to enable running CI jobs locally · c49ea68c
      krad authored
      Summary:
      We may not have permission on all /dev/shm to fix the sticky bit.
      Making the sticky bit fix advisory.
      
      Test Plan: Run CI job locally
      
      Subscribers: andrewkr, dhruba, leveldb
      
      Differential Revision: https://reviews.facebook.net/D61371
      c49ea68c
    • krad's avatar
      Build break fix · 726c2f7e
      krad authored
      Summary:
      Fixing build break on Mac
      
      (1) uint64_t fix
      (2) O_DIRECT works only for Linux
      
      Test Plan: Build and test on Mac and Unix
      
      Subscribers: andrewkr, dhruba, leveldb
      
      Differential Revision: https://reviews.facebook.net/D61353
      726c2f7e
    • omegaga's avatar
      Experiments on column-aware encodings · d51dc96a
      omegaga authored
      Summary:
      Experiments on column-aware encodings. Supported features: 1) extract data blocks from SST file and encode with specified encodings; 2) Decode encoded data back into row format; 3) Directly extract data blocks and write in row format (without prefix encoding); 4) Get column distribution statistics for column format; 5) Dump data blocks separated by columns in human-readable format.
      
      There is still on-going work on this diff. More refactoring is necessary.
      
      Test Plan: Wrote tests in `column_aware_encoding_test.cc`. More tests should be added.
      
      Reviewers: sdong
      
      Reviewed By: sdong
      
      Subscribers: arahut, andrewkr, dhruba
      
      Differential Revision: https://reviews.facebook.net/D60027
      d51dc96a
    • krad's avatar
      Persistent Read Cache (part 6) Block Cache Tier Implementation · c116b478
      krad authored
      Summary:
      The patch is a continuation of part 5. It glues the abstraction for
      file layout and metadata, and flush out the implementation of the API. It
      adds unit tests for the implementation.
      
      Test Plan: Run unit tests
      
      Subscribers: andrewkr, dhruba, leveldb
      
      Differential Revision: https://reviews.facebook.net/D57549
      c116b478
    • omegaga's avatar
      Write a benchmark to emulate time series data · 64046e58
      omegaga authored
      Summary: Add a benchmark to `db_bench`. In this benchmark, a write thread will populate time series data in the format of 'id | timestamp', and multiple read threads will randomly retrieve all data from one id at a time.
      
      Test Plan: Run the benchmark: `num=134217728;bpl=536870912;mb=67108864;overlap=10;mcz=2;del=300000000;levels=6;ctrig=4;delay=8;stop=12;wbn=3;mbc=20;wbs=134217728;dds=0;sync=0;t=32;vs=800;bs=4096;cs=17179869184;of=500000;wps=0;si=10000000; kir=100000; dir=/data/users/jhli/test/; ./db_bench --benchmarks=timeseries --disable_seek_compaction=1 --mmap_read=0 --statistics=1 --histogram=1 --num=$num --threads=$t --value_size=$vs --block_size=$bs --cache_size=$cs --bloom_bits=10 --cache_numshardbits=6 --open_files=$of --verify_checksum=1 --db=$dir --sync=$sync --disable_wal=0 --compression_type=none --stats_interval=$si --compression_ratio=1 --disable_data_sync=$dds --write_buffer_size=$wbs --target_file_size_base=$mb --max_write_buffer_number=$wbn --max_background_compactions=$mbc --level0_file_num_compaction_trigger=$ctrig --level0_slowdown_writes_trigger=$delay --level0_stop_writes_trigger=$stop --num_levels=$levels --delete_obsolete_files_period_micros=$del --min_level_to_compress=$mcz --max_grandparent_overlap_factor=$overlap --stats_per_interval=1 --max_bytes_for_level_base=$bpl --use_existing_db=0 --key_id_range=$kir`
      
      Reviewers: andrewkr, sdong
      
      Reviewed By: sdong
      
      Subscribers: lgalanis, andrewkr, dhruba, leveldb
      
      Differential Revision: https://reviews.facebook.net/D60651
      64046e58
  10. 30 Jul, 2016 1 commit