Manage and Repeat Experiments

It has been painful that when doing experiments I forget saving the parameters and lose track of them when the results come out. OK, the truth is that I’m lazy and I change the parameters in code and hope that my brain can remember the difference. Well it can’t and when I want to repeat any analysis, things start to bite me back. So I searched a bit online and see if there are some good strategies out there to manage the experiments. And luckily I did find some excellent post:

http://stackoverflow.com/questions/6437213/strategies-for-repeating-large-chunk-of-analysis/6550914#6550914

http://stackoverflow.com/questions/7979609/automatic-documentation-of-datasets

In these two posts, one answer mentioned that he uses JSON files to save parameters for different experiments and when reproduction is needed, he can simply import them. Quoting from the answer: “Everything in between is just code that runs with a given parametrization, but the code shouldn’t really change much, should it?”

Since I’m using R recently, I wrote a short script that help a user create a list of parameters and export them to a JSON file. It is kind of raw but  I hope someone will find it useful. It doesn’t have to be R. You can write your own scripts in a language you prefer.

Code on github:

https://github.com/kiribatu/Kiribatu-R-Toolkit/blob/master/docs/parameter_configuration.md

Enhanced by Zemanta
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s