User Tools

Site Tools


analysis:course:week1

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
analysis:course:week1 [2013/09/13 09:44]
mvdm [Grab a data session from the lab database]
analysis:course:week1 [2018/07/07 10:19] (current)
Line 13: Line 13:
 Resources: Resources:
   * Must-read: [[http://​www.ploscompbiol.org/​article/​info%3Adoi%2F10.1371%2Fjournal.pcbi.1000424|Noble,​ A Quick Guide to Organizing Computational Biology Projects]] ([[http://​www.ploscompbiol.org/​article/​fetchObject.action?​uri=info%3Adoi%2F10.1371%2Fjournal.pcbi.1000424&​representation=PDF|direct link to pdf]]) (yes, yours is one)   * Must-read: [[http://​www.ploscompbiol.org/​article/​info%3Adoi%2F10.1371%2Fjournal.pcbi.1000424|Noble,​ A Quick Guide to Organizing Computational Biology Projects]] ([[http://​www.ploscompbiol.org/​article/​fetchObject.action?​uri=info%3Adoi%2F10.1371%2Fjournal.pcbi.1000424&​representation=PDF|direct link to pdf]]) (yes, yours is one)
-  * Optional: [[http://​www.youtube.com/​GitHubGuides|Introduction to version control with GitHub]] ([[http://​git-scm.com/​book/​en/​Getting-Started-Git-Basics|more detailed doc pages]], [[http://​stackoverflow.com/​questions/​tagged/​git|git tagged questions on StackOverflow]])+  * Optional: [[http://​www.youtube.com/​GitHubGuides|Introduction to version control with GitHub]] ([[http://​git-scm.com/​book/​en/​Getting-Started-Git-Basics|more detailed doc pages]], [[http://​stackoverflow.com/​questions/​tagged/​git|git tagged questions on StackOverflow]], [[https://​github.s3.amazonaws.com/​media/​progit.en.pdf|Pro Git manual]], surprisingly readable)
   * Optional: MATLAB documentation on [[http://​www.mathworks.com/​help/​matlab/​matlab_env/​understanding-file-locations-in-matlab.html|File Locations]] and [[http://​www.mathworks.com/​help/​matlab/​matlab_env/​what-is-the-matlab-search-path.html#​br8ch8o|Search Paths]]   * Optional: MATLAB documentation on [[http://​www.mathworks.com/​help/​matlab/​matlab_env/​understanding-file-locations-in-matlab.html|File Locations]] and [[http://​www.mathworks.com/​help/​matlab/​matlab_env/​what-is-the-matlab-search-path.html#​br8ch8o|Search Paths]]
  
Line 24: Line 24:
 If you are using your own computer, you can download MATLAB from the mathworks.com website. For it to work, you need the username to be '​mvdmlab'​ and the network.lic license file available from the lab Dropbox. If you want to use MATLAB off-campus, you also may need to connect to the campus network using a VPN (See the uWaterloo VPN page on how to do this). It may also be possible to acquire your own working copy of MATLAB that does not have these restrictions. If you are using your own computer, you can download MATLAB from the mathworks.com website. For it to work, you need the username to be '​mvdmlab'​ and the network.lic license file available from the lab Dropbox. If you want to use MATLAB off-campus, you also may need to connect to the campus network using a VPN (See the uWaterloo VPN page on how to do this). It may also be possible to acquire your own working copy of MATLAB that does not have these restrictions.
  
 +This course assumes a basic working knowledge of MATLAB, corresponding roughly to the material in the [[http://​www.mathworks.com/​academia/​student_center/​tutorials/​launchpad.html|"​Getting Started with MATLAB"​ Primer]]. If you are unsure, take a look at the table of contents. If there are things you don't recognize, use the Primer itself, or Chapter 2 of the MATLAB for Neuroscientists book to get up to speed.
 === Setting up GitHub === === Setting up GitHub ===
  
Line 30: Line 31:
 If you don't already have a %%GitHub%% account, go to [[http://​www.github.com|GitHub]] and sign up.  If you don't already have a %%GitHub%% account, go to [[http://​www.github.com|GitHub]] and sign up. 
  
-Download and install the GUI client of your choice; for Windows, I recommend [[http://​windows.github.com/​|GitHub Windows]] to get started.+Download and install the GUI client of your choice; for Windows, I recommend [[http://​windows.github.com/​|GitHub Windows]] to get started. ​For installing Git and setting up GitHub on various operating systems, see [[https://​help.github.com/​articles/​set-up-git|GitHub:​ Set Up Git]]
  
 Configure your client. For %%GitHub%% Windows, you'll first need to sign in with your account, then click Tools > Options. Set the "​Default Storage Directory"​ to something reasonable: on lab computers, this should be something on the D: drive (for example, %%D:​\My_Documents\GitHub\%%). Also check that your username and e-mail address look ok (I am ''​mvdm''​). Configure your client. For %%GitHub%% Windows, you'll first need to sign in with your account, then click Tools > Options. Set the "​Default Storage Directory"​ to something reasonable: on lab computers, this should be something on the D: drive (for example, %%D:​\My_Documents\GitHub\%%). Also check that your username and e-mail address look ok (I am ''​mvdm''​).
Line 67: Line 68:
 Use a FTP client such as [[https://​filezilla-project.org/​|Filezilla]] or ''​WinSCP''​ to connect to the lab FTP server, ''​mvdmlab-nas1''​ (129.97.62.84). Configure your FTP client to require "​explicit FTP over TLS" and use ''​BIOL680''​ as username and password. In the ''​BIOL680''​ folder, download the folder ''​R016-2012-10-08''​. A good place to put this folder is in ''​D:​\data\promoted\R016\''​. (In general you want to keep your data separate from your code; for instance, multiple analysis projects may use the same data, so you don't want to duplicate it.) Use a FTP client such as [[https://​filezilla-project.org/​|Filezilla]] or ''​WinSCP''​ to connect to the lab FTP server, ''​mvdmlab-nas1''​ (129.97.62.84). Configure your FTP client to require "​explicit FTP over TLS" and use ''​BIOL680''​ as username and password. In the ''​BIOL680''​ folder, download the folder ''​R016-2012-10-08''​. A good place to put this folder is in ''​D:​\data\promoted\R016\''​. (In general you want to keep your data separate from your code; for instance, multiple analysis projects may use the same data, so you don't want to duplicate it.)
  
-You will have to be on campus to connect. If you still cannot log in to the server, send me your IP address and I will temporarily enable access for you.+Correct FileZilla configuration is the following:​ 
 + 
 +{{ :​analysis:​course:​ftp_config.png?​600 |}} 
 + 
 +You will have to be on campus to connect. If you still cannot log in to the server, send me your IP address and I will temporarily enable access for you. IF it still does not work, get the .zip {{:​analysis:​course:​r016-2012-10-08.zip|here}}.
  
 === Verify things are working === === Verify things are working ===
  
-As explained in the {{:​noble09.pdf|Noble (2009)}} ​paper, create a folder with today'​s date; I do this within a folder called ''​daily''​ so to keep things manageable. Create a ''​sandbox.m''​ file in it and use [[http://​blogs.mathworks.com/​videos/​2011/​07/​26/​starting-in-matlab-cell-mode-scripts/​|Cell Mode]] to check that you can load a data file from the data folder you grabbed (because the loader function is in your path):+As explained in the Noble paper, create a folder with today'​s date; I do this within a folder called ''​daily''​ so to keep things manageable. Create a ''​sandbox.m''​ file in it and use [[http://​blogs.mathworks.com/​videos/​2011/​07/​26/​starting-in-matlab-cell-mode-scripts/​|Cell Mode]] to check that you can load a data file from the data folder you grabbed (because the loader function is in your path):
  
 <code matlab> <code matlab>
Line 84: Line 89:
 plot(tvec(1:​nSamples),​raw_LFP(1:​nSamples));​ plot(tvec(1:​nSamples),​raw_LFP(1:​nSamples));​
 </​code>​ </​code>​
 +
 +You should replace the comment above with a ''​cd''​ command to change directory to where your data is located. Do not place the data in your code folder!
  
 If you get no errors and see a nice neural signal, save your ''​sandbox.m''​ script. Commit and sync to your %%GitHub%% fork. If you do get errors, verify that your path is set up correctly (you can type ''​path''​ to get a listing; it should have the various vandermeerlab folders in it. If not, go back to the Setting up MATLAB steps.) If you get no errors and see a nice neural signal, save your ''​sandbox.m''​ script. Commit and sync to your %%GitHub%% fork. If you do get errors, verify that your path is set up correctly (you can type ''​path''​ to get a listing; it should have the various vandermeerlab folders in it. If not, go back to the Setting up MATLAB steps.)
 +
 +=== For Mac/OS X users ===
 +
 +If you are running Matlab on OS X (and possibly Linux), the above ''​sandbox.m''​ code will probably fail. This is most likely because the ''​vandermeerlab''​ codebase downloaded from GitHub calls on low-level Windows functions. The following steps have worked for someone using OS X 10.8, with Matlab R2013a:
 +  * Head over to the [[http://​neuralynx.com/​research_software/​file_converters_and_utilities]|Neuralynx website]].
 +  * Download the **//​Neuralynx to Matlab Import for Linux and Mac OS X//** package ([[http://​neuralynx.com/​software/​Nlx2Mat_relDec11.tar|direct link]]).
 +  * Extract the archive you have downloaded into a folder, and add that folder to your path shortcut (See [[computing:​matlabsetup|Setting up MATLAB]])
 +  * Navigate to the ''​extracted folder/​binaries/'',​ find the file ''​Nlx2MatCSC_v3.mexmaci'',​ and rename it to ''​Nlx2MatCSC.mexmaci''​ (removing ''​_v3''​)
 +  * Again, make sure this folder is included in your path, and try running the ''​sandbox.m''​ again.
 +  * If you add neuralynx above vandermeerlab in your path Matlab should use the new neuralynx binaries. If not, you may need to delete ''​vandermeerlab/​util/​neuralynx/''​ for this to work.
 +
 +The ''​sandbox.m''​ should run properly now, and you should see the plot you're supposed to see.
 +
 +
 +=== For Linux users ===
 +
 +
 +Follow the instructions above for Mac/OS X users, except you may need to recompile the binaries (note that you will need C and C++ compilers installed. Install the ''​build-essential''​ package on Ubuntu):
 +  * You may want to just delete the existing binaries.
 +  * Edit ''​compile.sh''​ to set ''​PLATFORM=64PC''​ or ''​PLATFORM=32PC''​ depending on your architecture,​ and edit INCLMATLAB and BINMATLAB so that they point to the correct directories for your Matlab installation. If you don't remember, run ''​locate mexsh''​ in the shell and you should see the path.
 +  * You can rename all the files in the binary directory with the shell command: <​code>>​ rename '​s/​_v3//'​ *</​code>​
 +
 +This worked on 64 bit Ubuntu with Matlab R2013b.
 +
  
 === Read up on the data preprocessing pipeline === === Read up on the data preprocessing pipeline ===
  
-[[computing:​computing#​data_management|Data management]]+[[computing:​datapromotion|Data promotion]]
  
 === Be backup-aware === === Be backup-aware ===
  
-If you are using a lab computer, only put data and code on the ''​D:​\''​ drive. This actually has two underlying hard drives (a [[http://​en.wikipedia.org/​wiki/​RAID#​RAID_1|RAID 1 array]] in "​mirroring"​ mode) such that if one fails, your data is safe. However, this does not protect against ​human error (e.g. accidentally deleting data, overwriting a key file, et cetera). Some options to minimize the impact of those:+If you are using a lab computer, only put data and code on the ''​D:​\''​ drive. This actually has two underlying hard drives (a [[http://​en.wikipedia.org/​wiki/​RAID#​RAID_1|RAID 1 array]] in "​mirroring"​ mode) such that if one fails, your data is still available. However, this does not protect against accidentally deleting data, overwriting a key file, any sort of data corruption or damage, //et cetera//. Some options to minimize the impact of those:
  
   * Save your code (and other work that does not take up huge amounts of space) on Dropbox, Google Drive, or similar service that keeps a (limited) revision history   * Save your code (and other work that does not take up huge amounts of space) on Dropbox, Google Drive, or similar service that keeps a (limited) revision history
analysis/course/week1.1379079866.txt.gz · Last modified: 2018/07/07 10:19 (external edit)