analysis:nsb2015:week0
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
analysis:nsb2015:week0 [2018/07/07 10:19] 127.0.0.1 external edit |
analysis:nsb2015:week0 [2023/02/02 14:29] (current) mvdm |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| - | ~~DISCUSSION~~ | ||
| - | |||
| ==== Introduction: Principles of careful data analysis ==== | ==== Introduction: Principles of careful data analysis ==== | ||
| Line 104: | Line 102: | ||
| * Don't hard-code the locations of any files. Follow the [[http://ctnsrv.uwaterloo.ca/vandermeerlab/doku.php?id=analysis:course-w16:week2#data_files_overview|database format and file naming conventions]] so that it is sufficient to specify the root folder where the data are located. | * Don't hard-code the locations of any files. Follow the [[http://ctnsrv.uwaterloo.ca/vandermeerlab/doku.php?id=analysis:course-w16:week2#data_files_overview|database format and file naming conventions]] so that it is sufficient to specify the root folder where the data are located. | ||
| * Be explicit about what version numbers of various pieces of software you used to generate the results. Taken to the limit, this means also specifying the exact operating system version and shared libraries -- an issue best addressed by including an image or virtual machine (see e.g. [[http://www.russpoldrack.org/2015_12_01_archive.html|this blogpost]] by Russ Poldrack for discussion). A nice way to handle this with respect to code on %%GitHub%% is to create a [[https://help.github.com/articles/creating-releases/|release]] for a publication (essentially an easily linked to snapshot of the code on the repository). | * Be explicit about what version numbers of various pieces of software you used to generate the results. Taken to the limit, this means also specifying the exact operating system version and shared libraries -- an issue best addressed by including an image or virtual machine (see e.g. [[http://www.russpoldrack.org/2015_12_01_archive.html|this blogpost]] by Russ Poldrack for discussion). A nice way to handle this with respect to code on %%GitHub%% is to create a [[https://help.github.com/articles/creating-releases/|release]] for a publication (essentially an easily linked to snapshot of the code on the repository). | ||
| + | |||
| === 5. Be safe === | === 5. Be safe === | ||
| Line 122: | Line 121: | ||
| * [[https://en.wikipedia.org/wiki/Resampling_(statistics)|Resampling]] (aka bootstrapping, shuffling, permutation testing): generating synthetic data sets based on some known distribution, usually to compare to actual data. | * [[https://en.wikipedia.org/wiki/Resampling_(statistics)|Resampling]] (aka bootstrapping, shuffling, permutation testing): generating synthetic data sets based on some known distribution, usually to compare to actual data. | ||
| * Model comparison: the process of determining which model best describes the data. | * Model comparison: the process of determining which model best describes the data. | ||
| + | |||
| === 7. Test on synthetic data === | === 7. Test on synthetic data === | ||
| Analysis pipelines can get complicated quickly, such that it can be difficult to track down where things may be going wrong. A great tool to verify the integrity of single analysis steps, as well as entire workflows, is to test on data you generate, such that you know what the answer should be. For instance, if you input Poisson (random) spike data with a constant firing rate, totally independent of your experimental conditions, it better not be the case that your analysis reports a significant difference! | Analysis pipelines can get complicated quickly, such that it can be difficult to track down where things may be going wrong. A great tool to verify the integrity of single analysis steps, as well as entire workflows, is to test on data you generate, such that you know what the answer should be. For instance, if you input Poisson (random) spike data with a constant firing rate, totally independent of your experimental conditions, it better not be the case that your analysis reports a significant difference! | ||
analysis/nsb2015/week0.1530973191.txt.gz ยท Last modified: 2018/07/07 10:19 by 127.0.0.1
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
