Running WiredTiger workloads

Setup

Get the latest version of WiredTiger develop branch here:

git clone git@github.com:wiredtiger/wiredtiger.git -b develop

Build it as described here

To enable chunk cache on disk pass -DENABLE_MEMKIND=1 to cmake.

Running a Workload

Enter build directory:

cd  build/bench/wtperf/

Run a workload with a config file

./wtperf -O <config file>

This will run the workload in the current directory. If you want the database to be created elsewhere, you use the -h option, for example:

 ./wtperf -O <config file> -h /mnt/ssd/<user>/WT_TEST

There are many config files in the <wiredtiger>/bench/wtperf/runners directory. For example, bench/wtperf/runners/evict-btree.wtperf creates a BTree database and will run a read-only workload on it.

Increasing Workload DB Size

To create a larger database, you can change the icount variable in the configuration file. E.g., to create a 20GB database, just multiply the number in icount by 10:

-icount=10000000
+icount=100000000

Pre-Populate Database

It is useful to create a database first and then run on it many times over, so you don’t waste time creating it every time.

1. Run evict-btree.wtperf workload as above
2. Change the run_time=120 to something small, e.g., run_time=10.

run_time=120 is the runtime of the benchmark in seconds. You don’t care about running the benchmark at the time you are only populating the database, so you can simply set it to a very small number.

Once your database is created, you run evict-btree as follows:

1. Comment out the line with populate_threads=1 from the file. That way the database will not be populated.
2. Set create=false in the config
3. When you launch the benchmark specify with the -h argument the directory where the database was created.

Suppose you ran the following command to create the database:

./wtperf -O evict-btree.wtperf -h /mnt/ssd/john/WT_TEST

Then you created a modified file evict-btree-workload.wtperf, which looks like the original evict-btree.wtperf, but without the populate_threads=1 line. Run that workload like this:

./wtperf -O evict-btree-workload.wtperf -h /mnt/ssd/<user>/WT_TEST

so it knows where to find the database.

To enable the chunk cache:

To enable the chunk cache modify the conn_config line:conn_config="cache_size=50M,eviction=(threads_max=8)". This config line specifies that the in-memory cache is 50M (increase it for a larger database). To add the chunk cache, do the following:

conn_config="cache_size=50M,eviction=(threads_max=8),chunk_cache=(enabled=1,capacity=50GB,chunk_size=2GB,type=FILE,device_path=/mnt/ssd/<user>/CACHE),verbose=[chunkcache]"

This will enable the chunk cache that will sit on the ssd in the specified path. Or you can also specify the chunk cache to be in type=DRAM — not relevant for experiments, just for testing.