User Tools

Site Tools


analysis:nsb2015:week1

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision
Previous revision
analysis:nsb2015:week1 [2015/07/16 16:41]
mvdm created
analysis:nsb2015:week1 [2018/07/07 10:19] (current)
Line 7: Line 7:
   * Use %%GitHub%% to acquire the analysis code we will use   * Use %%GitHub%% to acquire the analysis code we will use
   * Perform some elementary %%GitHub%% operations (pull, add, commit, push) and create a branch for your project   * Perform some elementary %%GitHub%% operations (pull, add, commit, push) and create a branch for your project
-  * Create a well-designed folder structure for your project +  * Create a well-designed folder structure for your projectand be aware of naming conventions ​(in process, ​promoted data)
-  * Choose ​and implement a backup strategy for your project files +
-  * Understand the (pre)processing pipeline from raw data to "promoted" ​data set+
   * Connect to the lab database, download a data set, and test your path setup   * Connect to the lab database, download a data set, and test your path setup
  
Line 16: Line 14:
   * Optional: [[http://​www.youtube.com/​GitHubGuides|Introduction to version control with GitHub]] ([[http://​git-scm.com/​book/​en/​Getting-Started-Git-Basics|more detailed doc pages]], [[http://​stackoverflow.com/​questions/​tagged/​git|git tagged questions on StackOverflow]],​ [[https://​github.s3.amazonaws.com/​media/​progit.en.pdf|Pro Git manual]], surprisingly readable)   * Optional: [[http://​www.youtube.com/​GitHubGuides|Introduction to version control with GitHub]] ([[http://​git-scm.com/​book/​en/​Getting-Started-Git-Basics|more detailed doc pages]], [[http://​stackoverflow.com/​questions/​tagged/​git|git tagged questions on StackOverflow]],​ [[https://​github.s3.amazonaws.com/​media/​progit.en.pdf|Pro Git manual]], surprisingly readable)
   * Optional: MATLAB documentation on [[http://​www.mathworks.com/​help/​matlab/​matlab_env/​understanding-file-locations-in-matlab.html|File Locations]] and [[http://​www.mathworks.com/​help/​matlab/​matlab_env/​what-is-the-matlab-search-path.html#​br8ch8o|Search Paths]]   * Optional: MATLAB documentation on [[http://​www.mathworks.com/​help/​matlab/​matlab_env/​understanding-file-locations-in-matlab.html|File Locations]] and [[http://​www.mathworks.com/​help/​matlab/​matlab_env/​what-is-the-matlab-search-path.html#​br8ch8o|Search Paths]]
 +
  
 Step-by-step:​ Step-by-step:​
Line 38: Line 37:
 If you don't already have a %%GitHub%% account, go to [[http://​www.github.com|GitHub]] and sign up. E-mail me (mvdm at dartmouth dot edu) your account name, so I can give you access to the code repository. If you don't already have a %%GitHub%% account, go to [[http://​www.github.com|GitHub]] and sign up. E-mail me (mvdm at dartmouth dot edu) your account name, so I can give you access to the code repository.
  
-Meanwhile, download and install the Git client of your choice if you don't already have one installed. For Windows, I recommend [[http://​windows.github.com/​|GitHub Windows]] as a user-friendly way to get started. For installing Git and setting up GitHub on various operating systems, see [[https://​help.github.com/​articles/​set-up-git|GitHub:​ Set Up Git]]; note that on the NS&B computers, you need to select "Run as administrator"​ when running the installer.+Meanwhile, download and install the Git client of your choice if you don't already have one installed. ​(On NS&B lab computers, it is probably already installed; in that case you can open it and sign in with your account.) ​For Windows, I recommend [[http://​windows.github.com/​|GitHub Windows]] as a user-friendly way to get started. For installing Git and setting up GitHub on various operating systems, see [[https://​help.github.com/​articles/​set-up-git|GitHub:​ Set Up Git]]; note that on the NS&B computers, you need to select "Run as administrator"​ when running the installer.
  
 Next, configure your client. For %%GitHub%% Windows, after starting up the %%GitHub%% %%GUI%% (the default window that opens when you run %%GitHub%%) you'll first need to sign in with your account, then click Tools > Options. Set the "​Default Storage Directory"​ to something reasonable such as %%D:​\My_Documents\GitHub\%%). Also check that your username and e-mail address look ok (I am ''​mvdm''​). Next, configure your client. For %%GitHub%% Windows, after starting up the %%GitHub%% %%GUI%% (the default window that opens when you run %%GitHub%%) you'll first need to sign in with your account, then click Tools > Options. Set the "​Default Storage Directory"​ to something reasonable such as %%D:​\My_Documents\GitHub\%%). Also check that your username and e-mail address look ok (I am ''​mvdm''​).
Line 44: Line 43:
 === Cloning the NS&B codebase === === Cloning the NS&B codebase ===
  
-Next, we will use Git to create a local copy ("​clone"​) of the NS&B codebase. If you are already familiar with git and have its binaries on your shell path, you can simply do ''​git clone https://​github.com/​mvdm/​nsb2015.git'',​ which will create a new folder ''​nsb2015''​ in your working directory. If this is your first time using it, I recommend opening a new browser (this is important) and navigating to the course repository [[https://​github.com/​mvdm/​nsb2015|website]]. There, click the button "Clone in desktop",​ and your %%GitHub%% client should prompt you to accept the clone. If this fails, you can try to [[http://​stackoverflow.com/​questions/​11000869/​command-line-git-on-windows|add the Git binaries to your path]], opening a shell (command line, ''​cmd.exe''​),​ and typing the ''​git clone''​ command above. If that doesn'​t make sense, ask!+Next, we will use Git to create a local copy ("​clone"​) of the NS&B codebase. If you are already familiar with git and have its binaries on your shell path, you can simply do ''​git clone https://​github.com/​mvdm/​nsb2015.git'',​ which will create a new folder ''​nsb2015''​ in your working directory. If this is your first time using it, try navigating to the course repository [[https://​github.com/​mvdm/​nsb2015|website]]. There, click the button "Clone in desktop",​ and your %%GitHub%% client should prompt you to accept the clone (tested successfully with Chrome, YMMV). If this fails, you can [[http://​joe.blog.freemansoft.com/​2014/​04/​github-clone-to-desktop-with-windows.html|drag]] the URL from your browser into your %%GitHub%% client. As a last resort, you can try to [[http://​stackoverflow.com/​questions/​11000869/​command-line-git-on-windows|add the Git binaries to your path]], opening a shell (command line, ''​cmd.exe''​),​ and typing the ''​git clone''​ command above. If that doesn'​t make sense, ask!
  
 Now, verify that you have created a ''​nsb2015''​ folder with various subfolders and files in it, indicating that you have a local copy of the codebase. Because Git is tracking the contents of this folder, it is now easy to "​pull"​ the latest version from %%GitHub%%, either from the command line: Now, verify that you have created a ''​nsb2015''​ folder with various subfolders and files in it, indicating that you have a local copy of the codebase. Because Git is tracking the contents of this folder, it is now easy to "​pull"​ the latest version from %%GitHub%%, either from the command line:
Line 60: Line 59:
 Open the ''​README.md''​ file in the ''​nsb2015''​ folder. The ''​.md''​ extension is for %%Markdown%%,​ a lightweight set of commands to format text (syntax reference is [[https://​help.github.com/​articles/​markdown-basics | here]]). Open the ''​README.md''​ file in the ''​nsb2015''​ folder. The ''​.md''​ extension is for %%Markdown%%,​ a lightweight set of commands to format text (syntax reference is [[https://​help.github.com/​articles/​markdown-basics | here]]).
  
-Add your name to the list and save the file. Then go to your git shell and type ''​git status''​. Git has noticed the change, but it says that this change is not yet "​staged for commit"​. In other words, git is not tracking this file. Let's fix this:+Add your name to the list and save the file. Then go to your git shell (open one with the gear icon in the top right of your client'​s %%GUI%%) ​and type ''​git status''​. Git has noticed the change, but it says that this change is not yet "​staged for commit"​. In other words, git is not tracking this file. Let's fix this:
  
 <​code>​ <​code>​
Line 69: Line 68:
 If you now do a ''​git status''​ you will see that you are ahead of the origin (the online repository) by 1 commit. This makes sense because you just made a change. Let's push this by doing ''​git push''​. If you get an "​access denied"​ type error, email me (mvdm at dartmouth dot edu) your %%GitHub%% username and I will give you permission. If everything goes to plan you should now be able to see the updated README file [[https://​github.com/​mvdm/​nsb2015 | on GitHub]]. As above, you can also use the %%GUI%% ''​Sync''​ button to accomplish the same steps (albeit in a less transparent manner). If you now do a ''​git status''​ you will see that you are ahead of the origin (the online repository) by 1 commit. This makes sense because you just made a change. Let's push this by doing ''​git push''​. If you get an "​access denied"​ type error, email me (mvdm at dartmouth dot edu) your %%GitHub%% username and I will give you permission. If everything goes to plan you should now be able to see the updated README file [[https://​github.com/​mvdm/​nsb2015 | on GitHub]]. As above, you can also use the %%GUI%% ''​Sync''​ button to accomplish the same steps (albeit in a less transparent manner).
  
-A schematic of these basic operations (pull, commit, push) is shown below.+A schematic of these basic operations (pull, commit, push) is shown below, using the amazing [[https://​www.dokuwiki.org/​plugin:​graphviz|DokuWiki plugin]] for [[http://​www.graphviz.org/​|GraphViz]]:​
  
 <​graphviz dot center> <​graphviz dot center>
Line 83: Line 82:
 === Using GitHub to acquire the FieldTrip toolbox === === Using GitHub to acquire the FieldTrip toolbox ===
  
-Using your experience from the previous section, create a local clone of the [[https://​github.com/​fieldtrip/​fieldtrip|%%FieldTrip%% toolbox]]. If you are using the command line, make sure that you ''​cd''​ to your ''​GitHub''​ folder, i.e. that you are not within some other project such as ''​nsb2015'',​ before cloning. If things worked correctly you should have ''​fieldtrip''​ and ''​nsb2015''​ folders within your %%GitHub%% folder; **not** a ''​fieldtrip''​ folder within your ''​nsb2015''​ folder!+Using your experience from the previous section, create a local clone of the [[https://​github.com/​fieldtrip/​fieldtrip|FieldTrip toolbox]]. If you are using the command line, make sure that you ''​cd''​ to your ''​GitHub''​ folder, i.e. that you are not within some other project such as ''​nsb2015'',​ before cloning. If things worked correctly you should have ''​fieldtrip''​ and ''​nsb2015''​ folders within your %%GitHub%% folder; **not** a ''​fieldtrip''​ folder within your ''​nsb2015''​ folder!
  
 We will use this toolbox extensively for the analysis of local field potentials. Be aware that it is about 1.2GB! We will use this toolbox extensively for the analysis of local field potentials. Be aware that it is about 1.2GB!
Line 104: Line 103:
 </​code>​ </​code>​
  
-This ensures that whenever you click this button, you have a "clean" ​path of only the MATLAB default plus your local versions of the two %%GitHub%% repositories.+This ensures that whenever you click this button, you have a clean **path** (the set of folders, other than the current working directory, whose contents MATLAB can access) ​of only the MATLAB default plus your local versions of the two %%GitHub%% repositories.
  
 Optional: if you don't like the ''​.git''​ folders in your path, you can get clever with [[http://​www.mathworks.com/​help/​matlab/​matlab_prog/​regular-expressions.html|regular expressions]] to remove these: Optional: if you don't like the ''​.git''​ folders in your path, you can get clever with [[http://​www.mathworks.com/​help/​matlab/​matlab_prog/​regular-expressions.html|regular expressions]] to remove these:
Line 128: Line 127:
   * you reach a milestone, such as an analysis that tests a certain hypothesis   * you reach a milestone, such as an analysis that tests a certain hypothesis
  
-Make sure you push these changes ​and additions into your own branchIf you think your additions would be useful or interesting ​to others, you can issue a [[https://​help.github.com/​articles/​using-pull-requests | pull request]] ​to have your contribution incorporated into the ''​master''​ branch.+If you are an owner or collaborator of a %%GitHub%% repository, you will be able to push changes ​you makeI can enable this for you on the NS%B repository, but to be accepted as a collaborator on a large project such as %%FieldTrip%%, you will need to show your work to the owners first (as can be done by creating a Fork or branch and and issuing ​a [[https://​help.github.com/​articles/​using-pull-requests | pull request]]).
  
 === Grab a data session from the lab database === === Grab a data session from the lab database ===
  
-If you can access the NS&B share (usually ''​Z:​\''​), ​the data can be found in the ''​NSB_2014\4_MouseHippocampus\DataAnalysisTutorial\data''​ folder. For this module you will only need the ''​R016-2012-10-08''​ folder, but feel free to just copy all the folders. A good place to put these is in ''​D:​\data\promoted\''​ (Rxxx indicate different rats, followed by the date of each session). In general you want to keep your data separate from your code; for instance, multiple analysis projects may use the same data, so you don't want to duplicate it.+If you can access the NS&B share (usually ''​Z:​\''​), ​an example ​data folder ​can be found in the ''​NSB_2015\4_MouseHippocampus\DataAnalysisTutorial\data''​ folder. For this module you will only need the ''​R016-2012-10-08''​ folder. A good place to put it is in something like ''​D:​\data\promoted\''​ (Rxxx indicate different rats, followed by the date of each session). In general you want to keep your data separate from your code; for instance, multiple analysis projects may use the same data, so you don't want to duplicate it.
  
-If you cannot access ​the NS&B share, use a %%FTP%% client such as [[https://​filezilla-project.org/​|Filezilla]] to connect to the lab %%FTP%% server, ​''​mvdmlab-nas1'' ​(129.97.62.84). Configure your %%FTP%% client ​to require "​explicit %%FTP%% over %%TLS%%"​ and use ''​BIOL680''​ as username and password+The choice of the folder name ''​promoted'' ​indicates that these are data folders for which preprocessing is completedAs explained further in Module 2, preprocessing typically includes the renaming of raw data files, annotation, spike sorting, and a few other stepsIn general, it is useful ​to keep promoted data separate from data still in process.
  
-Correct FileZilla configuration is the following:+If you cannot access the NS&B share, use a %%FTP%% client such as [[https://​filezilla-project.org/​|Filezilla]] to connect to the lab %%FTP%% server at **129.170.31.15**,​ use the following ​login settings (FileZilla):
  
-{{ :analysis:course:ftp_config.png?600 |}}+{{ :analysis:nsb2015:d-datavault-info.png?600 |}}
  
 If you cannot log in to the server, send me your IP address and I will enable access for you. If you cannot log in to the server, send me your IP address and I will enable access for you.
Line 144: Line 143:
 === Verify things are working === === Verify things are working ===
  
-As explained in the Noble paper, create a folder with today'​s date in your project folder. Create a ''​sandbox.m''​ file in it, and use [[http://​blogs.mathworks.com/​videos/​2011/​07/​26/​starting-in-matlab-cell-mode-scripts/​|Cell Mode]] to check that you can load a data file:+As explained in the Noble paper, create a folder with today'​s date in your project folder. Create a ''​sandbox.m''​ file in it, click your previously made shortcut to set up the paths, and use [[http://​blogs.mathworks.com/​videos/​2011/​07/​26/​starting-in-matlab-cell-mode-scripts/​|Cell Mode]] to check that you can load a data file:
  
 <code matlab> <code matlab>
Line 186: Line 185:
 </​code>​ </​code>​
  
-You should see some interesting oscillations -- we will explore these in detail in upcoming modules. If you see this, you have successfully completed this module!+You should see some interesting oscillations -- we will explore these in detail in upcoming modules. If you see this, you have successfully completed this introductory ​module!
  
 {{ :​analysis:​nsb2014:​verify.png?​600 |}} {{ :​analysis:​nsb2014:​verify.png?​600 |}}
  
-=== Folder and file naming === 
- 
-FIXME 
  
 === For Mac/OS X users === === For Mac/OS X users ===
analysis/nsb2015/week1.1437079296.txt.gz · Last modified: 2018/07/07 10:19 (external edit)