User Tools

Site Tools


analysis:course-w16:week4

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
analysis:course-w16:week4 [2016/01/21 13:32]
mvdm [Reconstructing a signal from sampled data (optional)]
analysis:course-w16:week4 [2018/07/07 10:19] (current)
Line 1: Line 1:
 ~~DISCUSSION~~ ~~DISCUSSION~~
- 
-:!: **Under construction,​ do not use!** :!: 
  
 ===== Anatomy of time series data, sampling theory ===== ===== Anatomy of time series data, sampling theory =====
Line 93: Line 91:
 ==== Subsampling (decimating) time series data ==== ==== Subsampling (decimating) time series data ====
  
-In the real world, the frequency at which we can acquire data will be limited the properties of your experimental equipment. For instance, the maximum sampling rate on a typical Neuralynx system is 32 kHz. Thus, the highest-frequency signal we can detect is 16 kHz (the Nyquist frequency). Crucially, however, we cannot rule out the possibility that frequencies above 16 kHz are present in the signal we are sampling from! Thus, we risk **aliasing**:​ generating "​phantom"​ frequencies in our sampled data that don't exist in the true signal. What to do?+In the real world, the frequency at which we can acquire data will be limited ​by the properties of your experimental equipment. For instance, the maximum sampling rate on a typical Neuralynx system is 32 kHz. Thus, the highest-frequency signal we can detect is 16 kHz (the Nyquist frequency). Crucially, however, we cannot rule out the possibility that frequencies above 16 kHz are present in the signal we are sampling from! Thus, we risk **aliasing**:​ generating "​phantom"​ frequencies in our sampled data that don't exist in the true signal. What to do?
  
 The general solution is to apply an //​anti-aliasing filter// to the data before sampling. To illustrate this, let's generate a signal consisting of two frequencies:​ The general solution is to apply an //​anti-aliasing filter// to the data before sampling. To illustrate this, let's generate a signal consisting of two frequencies:​
Line 125: Line 123:
 <code matlab> <code matlab>
 % sample at 12 Hz with different method % sample at 12 Hz with different method
 +tvec1d = decimate(tvec1,​ dt);
 signal2d = decimate(signal1,​dt);​ signal2d = decimate(signal1,​dt);​
  
Line 184: Line 183:
 <code matlab> <code matlab>
 xl = [1 1.04]; xl = [1 1.04];
-linkaxes('​x'​,'​ax1',​ax2);+linkaxes([ax1, ax2], '​x'​);​
 set(ax1,'​XLim',​xl);​ % see what I did there?) set(ax1,'​XLim',​xl);​ % see what I did there?)
 </​code>​ </​code>​
Line 204: Line 203:
 You should obtain something like: You should obtain something like:
  
-{{ :​analysis:​course:​week3_fig1.png?​nolink&​600 |}}+{{ :​analysis:​course-w16:spline_recover.png?​nolink&​600 |}}
  
 Notice how the spline-interpolated sampled signal is a pretty good approximation to the original. In cases where you care about detecting the values and/or locations of signal peaks, such as during spike sorting, performing spline interpolation can often improve accuracy substantially! Notice how the spline-interpolated sampled signal is a pretty good approximation to the original. In cases where you care about detecting the values and/or locations of signal peaks, such as during spike sorting, performing spline interpolation can often improve accuracy substantially!
 +
 ==== Detailed examination of Neuralynx time series data ==== ==== Detailed examination of Neuralynx time series data ====
 +
 +This section will look in some detail at how raw time series data is stored by the Neuralynx system. Even if you do not use this system in your own work, the lessons that can be learned from looking at what can go wrong at the raw data level already are universal!
 +
 +To get into the guts of actual Neuralynx data, we will not use the sanitized wrapper provided by ''​LoadCSC()''​ but instead use the loading function provided by Neuralynx. Using cell mode in a sandbox file as usual, ''​cd''​ into the ''​R016-2012-10-08''​ data folder you downloaded previously in Week 1. Then deploy the Neuralynx loader:
 +
 +<code matlab>
 +%%
 +% cd to your location here
 +fname = '​R016-2012-10-08-CSC03b.ncs';​
 +[Timestamps,​ ~, SampleFrequencies,​ NumberOfValidSamples,​ Samples, Header] = Nlx2MatCSC(fname,​ [1 1 1 1 1], 1, 1, []);
 +</​code>​
 +
 +Note the use of the ''​~''​ output argument, which indicates that we do not wish to assign the second output of the ''​Nlx2MatCSC''​ function to a variable.
 +
 +Most data files come with a "​header"​ that describes some properties of the data. Let's look at the header of our file, which is a LFP recorded from the ventral striatum (nucleus accumbens core) of a rat running back and forth along a linear track:
 +
 +<code matlab>
 +>> Header
 +
 +Header = 
 +
 +    '########​ Neuralynx Data File Header'​
 +    '## File Name C:​\CheetahData\2012-10-08_11-24-30\CSC10.ncs'​
 +    '## Time Opened (m/d/y): 10/​8/​2012 ​ (h:m:s.ms) 11:​25:​2.79'​
 +    '## Time Closed (m/d/y): 10/​8/​2012 ​ (h:m:s.ms) 12:​32:​9.61'​
 +    '​-CheetahRev 5.5.1 '
 +    ''​
 +    '​-AcqEntName CSC10'
 +    '​-FileType CSC'
 +    '​-RecordSize 1044'
 +    ''​
 +    '​-HardwareSubSystemName AcqSystem1'​
 +    '​-HardwareSubSystemType DigitalLynxSX'​
 +    '​-SamplingFrequency 2000'
 +    '​-ADMaxValue 32767'
 +    '​-ADBitVolts 4.57778e-008 '
 +    ''​
 +    '​-NumADChannels 1'
 +    '​-ADChannel 4 '
 +    '​-InputRange 1500 '
 +    '​-InputInverted True'
 +    '​-DSPLowCutFilterEnabled True'
 +    '​-DspLowCutFrequency 1'
 +    '​-DspLowCutNumTaps 0'
 +    '​-DspLowCutFilterType DCO'
 +    '​-DSPHighCutFilterEnabled True'
 +    '​-DspHighCutFrequency 425'
 +    '​-DspHighCutNumTaps 128'
 +    '​-DspHighCutFilterType FIR'
 +    '​-DspDelayCompensation Disabled'​
 +    '​-DspFilterDelay_µs 1984'
 +
 +>> ​
 +</​code>​
 +
 +You should recognize at least a few of the phrases used: for instance, we see that the sampling frequency (often abbreviated as Fs) is given as 2000 (samples per second, or Hz). Important also is the ''​InputRange'',​ given as 1500 (microvolts,​ uV), to indicate that the maximum value (''​ADMaxValue'',​ or 32767) in the raw data corresponds to an actual signal magnitude of 1500uV. Ignore the other fields for now.
 +
 +☛ The waveforms of extracellularly recorded action potentials ("​spikes"​) typically contain frequency components in the 1000-5000Hz range. Do you expect to find spikes in this data file? Why not?
 +
 +The ''​ADMaxValue''​ of 32767 reflects the fact that Neuralynx stores data as 16-bit integers (see the discussion in the Leis chapter if this doesn'​t mean anything to you). Thus, the smallest value it can store is -32768, for a total range of 65536 (16 bits). This means that what is actually a continuous variable -- voltage -- is stored with finite precision.
 +
 +☛ Given Neuralynx'​s 16-bit data format and the specified range of +/- 1500uV for this data session, compute the smallest voltage change (in uV) that can be resolved.
 +
 +☛ Using this knowledge, convert the ''​Samples''​ variable, which is specified in AD values, to millivolts. (Note that the MATLAB data type is given as ''​double''​ but the values are actually ''​int16''​.) Notice that the ''​ADBitVolts''​ field in the header can be used for this unit conversion.
 +
 +Inspect the ''​Timestamps''​ variable, which is given in microseconds (us). 
 +
 +☛ Convert ''​Timestamps''​ to seconds (s).
 +
 +You may have noticed that the ''​Samples''​ variable is size [512 x 10761] and ''​Timestamps''​ is [1 x 10761]. As it turns out, Neuralynx data (of the "​continuously sampled"​ type) is stored in blocks of 512 samples. Only the first sample of each block is timestamped.
 +
 +☛ Compute the total number of samples, and from this number and the sampling frequency, the total time that would be sampled continuously if all samples were acquired without any gaps. Compare this number with the actual time elapsed between the first and last ''​Timestamps''​. What do you conclude?
 +
 +In fact, there are several gaps in the data. Our standard recording protocol requires a "​pre-recording"​ session, followed by a pause, then the actual recording session, another pause, and a "​post-recording"​ session. This can be seen easily by plotting the difference between each sample and its predecessor (''​plot(diff(Timestamps))''​).
 +
 +We wish to restrict the data to only those samples taken when the rat was running on the track. Promoted (i.e. preprocessed and annotated) data folders always have an "​ExpKeys"​ file with some useful metadata, including ''​TimeOnTrack''​ and ''​TimeOffTrack''​ values:
 +
 +<code matlab>
 +>> LoadExpKeys
 +>> ExpKeys
 +
 +ExpKeys = 
 +
 +     ​BehaviorOrder:​ {'​Value' ​ '​Risk'​}
 +          Protocol: '​Hyperdrive'​
 +            Target: {'​Striatum' ​ '​Hippocampus'​}
 +           ​Target2:​ {'​Ventral' ​ '​CA1'​}
 +    TetrodeTargets:​ [2 2 1 1 1 1]
 +     ​TetrodeDepths:​ [2000 1960 6300 6380 6460 1920]
 +       ​TimeOnTrack:​ [1030 2285]
 +      TimeOffTrack:​ [2268 3194]
 +             ​Delay:​ [0.5000 0.5000]
 +         ​goodGamma:​ {'​R016-2012-10-03-CSC04d.ncs' ​ [1x26 char]}
 +           ​goodSWR:​ {'​R016-2012-10-03-CSC02b.ncs'​}
 +         ​goodTheta:​ {'​R016-2012-10-03-CSC02b.ncs'​}
 +        CueToneMap: {'​S3' ​ '​S2' ​ '​S4' ​ '​S1' ​ '​S2' ​ '​S5'​}
 +</​code>​
 +
 +In fact this data contains two recording sessions, called '​Value'​ and '​Risk'​ respectively (this refers to the distributions of food outcomes predicted by audio cues presented as the rat crossed the center of the track; we will not use this information for now, but the full task is described in the [[http://​onlinelibrary.wiley.com/​doi/​10.1111/​ejn.13069/​fullpaper | paper]]). These sessions map onto the first and second elements of ''​TimeOnTrack''​ and ''​TimeOffTrack'',​ which give the times (in seconds) of when the Value and Risk sessions started and ended, respectively.
 +
 +☛ Use the first element of ''​ExpKeys.TimeOnTrack''​ and ''​ExpKeys.TimeOffTrack''​ to find the indices of ''​Timestamps''​ corresponding to the Value session. Then, use these to create a new set of variables ''​TimestampsValue'',​ ''​SamplesValue''​ et cetera. (Note that this is essentially what ''​restrict()''​ does; If you are confused by this, review the documentation on [[http://​www.mathworks.com/​help/​matlab/​math/​matrix-indexing.html|Matrix Indexing]].)
 +
 +☛ Plot the differences between the resulting timestamps (Hint: MATLAB'​s ''​diff()''​ function is useful here!). You should see:
 +
 +{{ :​analysis:​course:​week3_fig4.png?​600 |}}
 +
 +The most common value between timestamps seems to be about 0.26 s. Recall that these timestamps are for the start of a block of 512 values. ​
 +
 +☛ What is the expected difference between 512-sample timestamps if ''​Fs''​ is 2kHz? 
 +
 +Let's test if this indeed the most common value in these data:
 +
 +<code matlab>
 +>> 512.*(1/​2000) == mode(diff(TimestampsValue))
 +
 +ans =
 +
 +     0
 +</​code>​
 +
 +Hmm. (You can ask MATLAB ''​why'',​ but don't expect an informative answer!)
 +
 +☛ Use ''​format long''​ to change MATLAB'​s default display, and inspect the above values to determine the source of the difference.
 +
 +Apparently the typical elapsed time between two 512-sample blocks does not correspond //exactly// to what would be expected if Fs equals 2kHz.
 +
 +☛ Compute Neuralynx'​s true sampling rate from the observed mode of the timestamp diffs.
 +
 +Close enough for practical purposes, but the differences could become significant for very long recording sessions!
 +
 +Next: what is up with these clearly smaller values in the diff plot? Let's investigate:​
 +
 +<code matlab>
 +plot(diff(TimestampsValue))
 +hold on;
 +plot(NumberOfValidSamplesValue == 512,'​r'​)
 +</​code>​
 +
 +If you zoom in, you should find that the odd timestamp diffs occur for those sample blocks that have a number of valid samples that is not 512.
 +
 +☛ Find the indices of these crippled blocks. Look at the ''​Samples''​ inside a few of these to see what Neuralynx does with invalid samples.
 +
 +Tricky! How would we know to exclude these invalid samples? Fortunately,​ we can do so using the ''​NumberOfValidSamples''​ variable, which tells us how many are good and can be included.
 +
 +☛ How does the ''​LoadCSC()''​ function handle these cases?
 +
 +:!: NOTE: the above missing sample weirdness was a rare occurrence for our lab's Neuralynx system; one that was traced to a faulty framegrabber board driver which caused the computer to lock up periodically. Thanks to Neuralynx'​s warning and error reporting system in the acquisition software, we were immediately alerted that something unexpected was happening. In addition, the ''​*events.Nev''​ file contains event strings indicating suspect data blocks.
 +
 +==== Challenges ====
 +
 +★ If you have your own time series data, find out how it is stored: with what precision? In blocks? Does the reported sampling rate match up with what is in the data? How can you convert from the raw data format to voltage (or whatever the quantity you are measuring is)?
 +
 +★ If you implemented your own file loader(s) back in Module 2, implement checks for missing samples and possible sampling frequency misalignments.
 +
 +★ Important! If you have your own idea of something you'd like to accomplish in this course, even if is isn't listed as an official challenge, ask me and we can make it count as one. What you do in this course should be as relevant as possible to your work!
analysis/course-w16/week4.1453401127.txt.gz · Last modified: 2018/07/07 10:19 (external edit)