# Development/Benchmarks

This tool needs to be reactivated.

The latest version of this tool can be obtained from darcs clone --lazy http://hub.darcs.net/kowey/darcs-benchmark. (an older version is available on HackageDB).

Currently graph generation is broken, it looks like the Google API we relied on is defunct.

# About the repositories use by darcs-benchmarks

repo patches files darcs dir total size
GHC 20942 1298 143 MB 165 MB
darcs 7920 504 41 MB 45 MB
tahoe-lafs 4232 458 65 MB 65 MB
tabular 55 16 0.34 MB 0.39 MB

TODO: the (pre-git) Agda Darcs repository was as big as GHC’s.

# General notes

Unless otherwise indicated, each benchmark was run with a hot disk cache. Before timing a benchmark, we would run it once first, without timing, to warm up the cache. See below for some results with a forced cold cache.

The number of runs for each test satisfies the following constraints:

• at least 3 runs
• at least 10 seconds worth of runs
• at most 100 runs

We’re currently investigating long term storage of results so that we can use more than 100 runs, and also the use of the criterion package for a more principle approach.

# Other tools for benchmarks

• bench a commandline benchmarking tool based on criterion, with HTML reposts generation

# Improvements needed (help wanted!)

• obliterate

Note that these are all very small jobs…

• Make it possible to toggle profiling from the config file
• Make it possible to run just the latest version of darcs (and then compare with stored performance numbers for older darcs)
• Implement a timeout mechanism to kill darcs after some fixed amount of time
• Determine which graph-producing library to use (need something fairly portable, easy to install)
• Write code to produce shootout-like visualisation of results (maybe hsparklines to produce at-a-glance-overview)
• Verbose mode so that we can get at the .prof files for fine grained performance data.