User Tools

Site Tools


analysis:course:week1

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
analysis:course:week1 [2013/09/16 11:18]
mvdm [Read up on the data preprocessing pipeline]
analysis:course:week1 [2014/06/22 14:55]
mvdm
Line 13: Line 13:
 Resources: Resources:
   * Must-read: [[http://​www.ploscompbiol.org/​article/​info%3Adoi%2F10.1371%2Fjournal.pcbi.1000424|Noble,​ A Quick Guide to Organizing Computational Biology Projects]] ([[http://​www.ploscompbiol.org/​article/​fetchObject.action?​uri=info%3Adoi%2F10.1371%2Fjournal.pcbi.1000424&​representation=PDF|direct link to pdf]]) (yes, yours is one)   * Must-read: [[http://​www.ploscompbiol.org/​article/​info%3Adoi%2F10.1371%2Fjournal.pcbi.1000424|Noble,​ A Quick Guide to Organizing Computational Biology Projects]] ([[http://​www.ploscompbiol.org/​article/​fetchObject.action?​uri=info%3Adoi%2F10.1371%2Fjournal.pcbi.1000424&​representation=PDF|direct link to pdf]]) (yes, yours is one)
-  * Optional: [[http://​www.youtube.com/​GitHubGuides|Introduction to version control with GitHub]] ([[http://​git-scm.com/​book/​en/​Getting-Started-Git-Basics|more detailed doc pages]], [[http://​stackoverflow.com/​questions/​tagged/​git|git tagged questions on StackOverflow]])+  * Optional: [[http://​www.youtube.com/​GitHubGuides|Introduction to version control with GitHub]] ([[http://​git-scm.com/​book/​en/​Getting-Started-Git-Basics|more detailed doc pages]], [[http://​stackoverflow.com/​questions/​tagged/​git|git tagged questions on StackOverflow]], [[https://​github.s3.amazonaws.com/​media/​progit.en.pdf|Pro Git manual]], surprisingly readable)
   * Optional: MATLAB documentation on [[http://​www.mathworks.com/​help/​matlab/​matlab_env/​understanding-file-locations-in-matlab.html|File Locations]] and [[http://​www.mathworks.com/​help/​matlab/​matlab_env/​what-is-the-matlab-search-path.html#​br8ch8o|Search Paths]]   * Optional: MATLAB documentation on [[http://​www.mathworks.com/​help/​matlab/​matlab_env/​understanding-file-locations-in-matlab.html|File Locations]] and [[http://​www.mathworks.com/​help/​matlab/​matlab_env/​what-is-the-matlab-search-path.html#​br8ch8o|Search Paths]]
  
Line 31: Line 31:
 If you don't already have a %%GitHub%% account, go to [[http://​www.github.com|GitHub]] and sign up.  If you don't already have a %%GitHub%% account, go to [[http://​www.github.com|GitHub]] and sign up. 
  
-Download and install the GUI client of your choice; for Windows, I recommend [[http://​windows.github.com/​|GitHub Windows]] to get started.+Download and install the GUI client of your choice; for Windows, I recommend [[http://​windows.github.com/​|GitHub Windows]] to get started. ​For installing Git and setting up GitHub on various operating systems, see [[https://​help.github.com/​articles/​set-up-git|GitHub:​ Set Up Git]]
  
 Configure your client. For %%GitHub%% Windows, you'll first need to sign in with your account, then click Tools > Options. Set the "​Default Storage Directory"​ to something reasonable: on lab computers, this should be something on the D: drive (for example, %%D:​\My_Documents\GitHub\%%). Also check that your username and e-mail address look ok (I am ''​mvdm''​). Configure your client. For %%GitHub%% Windows, you'll first need to sign in with your account, then click Tools > Options. Set the "​Default Storage Directory"​ to something reasonable: on lab computers, this should be something on the D: drive (for example, %%D:​\My_Documents\GitHub\%%). Also check that your username and e-mail address look ok (I am ''​mvdm''​).
Line 68: Line 68:
 Use a FTP client such as [[https://​filezilla-project.org/​|Filezilla]] or ''​WinSCP''​ to connect to the lab FTP server, ''​mvdmlab-nas1''​ (129.97.62.84). Configure your FTP client to require "​explicit FTP over TLS" and use ''​BIOL680''​ as username and password. In the ''​BIOL680''​ folder, download the folder ''​R016-2012-10-08''​. A good place to put this folder is in ''​D:​\data\promoted\R016\''​. (In general you want to keep your data separate from your code; for instance, multiple analysis projects may use the same data, so you don't want to duplicate it.) Use a FTP client such as [[https://​filezilla-project.org/​|Filezilla]] or ''​WinSCP''​ to connect to the lab FTP server, ''​mvdmlab-nas1''​ (129.97.62.84). Configure your FTP client to require "​explicit FTP over TLS" and use ''​BIOL680''​ as username and password. In the ''​BIOL680''​ folder, download the folder ''​R016-2012-10-08''​. A good place to put this folder is in ''​D:​\data\promoted\R016\''​. (In general you want to keep your data separate from your code; for instance, multiple analysis projects may use the same data, so you don't want to duplicate it.)
  
-You will have to be on campus to connect. If you still cannot log in to the server, send me your IP address and I will temporarily enable access for you.+Correct FileZilla configuration is the following:​ 
 + 
 +{{ :​analysis:​course:​ftp_config.png?​600 |}} 
 + 
 +You will have to be on campus to connect. If you still cannot log in to the server, send me your IP address and I will temporarily enable access for you. IF it still does not work, get the .zip {{:​analysis:​course:​r016-2012-10-08.zip|here}}.
  
 === Verify things are working === === Verify things are working ===
  
-As explained in the {{:​noble09.pdf|Noble (2009)}} ​paper, create a folder with today'​s date; I do this within a folder called ''​daily''​ so to keep things manageable. Create a ''​sandbox.m''​ file in it and use [[http://​blogs.mathworks.com/​videos/​2011/​07/​26/​starting-in-matlab-cell-mode-scripts/​|Cell Mode]] to check that you can load a data file from the data folder you grabbed (because the loader function is in your path):+As explained in the Noble paper, create a folder with today'​s date; I do this within a folder called ''​daily''​ so to keep things manageable. Create a ''​sandbox.m''​ file in it and use [[http://​blogs.mathworks.com/​videos/​2011/​07/​26/​starting-in-matlab-cell-mode-scripts/​|Cell Mode]] to check that you can load a data file from the data folder you grabbed (because the loader function is in your path):
  
 <code matlab> <code matlab>
Line 85: Line 89:
 plot(tvec(1:​nSamples),​raw_LFP(1:​nSamples));​ plot(tvec(1:​nSamples),​raw_LFP(1:​nSamples));​
 </​code>​ </​code>​
 +
 +You should replace the comment above with a ''​cd''​ command to change directory to where your data is located. Do not place the data in your code folder!
  
 If you get no errors and see a nice neural signal, save your ''​sandbox.m''​ script. Commit and sync to your %%GitHub%% fork. If you do get errors, verify that your path is set up correctly (you can type ''​path''​ to get a listing; it should have the various vandermeerlab folders in it. If not, go back to the Setting up MATLAB steps.) If you get no errors and see a nice neural signal, save your ''​sandbox.m''​ script. Commit and sync to your %%GitHub%% fork. If you do get errors, verify that your path is set up correctly (you can type ''​path''​ to get a listing; it should have the various vandermeerlab folders in it. If not, go back to the Setting up MATLAB steps.)
 +
 +=== For Mac/OS X users ===
 +
 +If you are running Matlab on OS X (and possibly Linux), the above ''​sandbox.m''​ code will probably fail. This is most likely because the ''​vandermeerlab''​ codebase downloaded from GitHub calls on low-level Windows functions. The following steps have worked for someone using OS X 10.8, with Matlab R2013a:
 +  * Head over to the [[http://​neuralynx.com/​research_software/​file_converters_and_utilities]|Neuralynx website]].
 +  * Download the **//​Neuralynx to Matlab Import for Linux and Mac OS X//** package ([[http://​neuralynx.com/​software/​Nlx2Mat_relDec11.tar|direct link]]).
 +  * Extract the archive you have downloaded into a folder, and add that folder to your path shortcut (See [[computing:​matlabsetup|Setting up MATLAB]])
 +  * Navigate to the ''​extracted folder/​binaries/'',​ find the file ''​Nlx2MatCSC_v3.mexmaci'',​ and rename it to ''​Nlx2MatCSC.mexmaci''​ (removing ''​_v3''​)
 +  * Again, make sure this folder is included in your path, and try running the ''​sandbox.m''​ again.
 +  * If you add neuralynx above vandermeerlab in your path Matlab should use the new neuralynx binaries. If not, you may need to delete ''​vandermeerlab/​util/​neuralynx/''​ for this to work.
 +
 +The ''​sandbox.m''​ should run properly now, and you should see the plot you're supposed to see.
 +
 +
 +=== For Linux users ===
 +
 +
 +Follow the instructions above for Mac/OS X users, except you may need to recompile the binaries (note that you will need C and C++ compilers installed. Install the ''​build-essential''​ package on Ubuntu):
 +  * You may want to just delete the existing binaries.
 +  * Edit ''​compile.sh''​ to set ''​PLATFORM=64PC''​ or ''​PLATFORM=32PC''​ depending on your architecture,​ and edit INCLMATLAB and BINMATLAB so that they point to the correct directories for your Matlab installation. If you don't remember, run ''​locate mexsh''​ in the shell and you should see the path.
 +  * You can rename all the files in the binary directory with the shell command: <​code>>​ rename '​s/​_v3//'​ *</​code>​
 +
 +This worked on 64 bit Ubuntu with Matlab R2013b.
 +
  
 === Read up on the data preprocessing pipeline === === Read up on the data preprocessing pipeline ===
  
-[[computing:​datapromotion|[[computing:​computing#​data_management|Data promotion]]]+[[computing:​datapromotion|Data promotion]]
  
 === Be backup-aware === === Be backup-aware ===
  
-If you are using a lab computer, only put data and code on the ''​D:​\''​ drive. This actually has two underlying hard drives (a [[http://​en.wikipedia.org/​wiki/​RAID#​RAID_1|RAID 1 array]] in "​mirroring"​ mode) such that if one fails, your data is safe. However, this does not protect against ​human error (e.g. accidentally deleting data, overwriting a key file, et cetera). Some options to minimize the impact of those:+If you are using a lab computer, only put data and code on the ''​D:​\''​ drive. This actually has two underlying hard drives (a [[http://​en.wikipedia.org/​wiki/​RAID#​RAID_1|RAID 1 array]] in "​mirroring"​ mode) such that if one fails, your data is still available. However, this does not protect against accidentally deleting data, overwriting a key file, any sort of data corruption or damage, //et cetera//. Some options to minimize the impact of those:
  
   * Save your code (and other work that does not take up huge amounts of space) on Dropbox, Google Drive, or similar service that keeps a (limited) revision history   * Save your code (and other work that does not take up huge amounts of space) on Dropbox, Google Drive, or similar service that keeps a (limited) revision history
analysis/course/week1.txt · Last modified: 2018/07/07 10:19 (external edit)