Number of time length for calibration of training and testing data

[CaelynCheung1996]

In the code, it seems that the number of epochs/time length used to calibrate the training and the testing data are different, is that correct? For training data, all the labeled epochs in a recording are used while only limited epochs in testing recording is used for calibration.

[zekebarger]

Correct! "Calibration" here means estimating the feature means and variances for each brain state, and using those values to standardize the feature scores. Training sets are fully labeled, so we can use all epochs for this calculation. New data sets are typically not fully labeled (otherwise there would be no need for automated scoring), so we just use a small number of labeled epochs provided by the user.

[CaelynCheung1996]

Correct! "Calibration" here means estimating the feature means and variances for each brain state, and using those values to standardize the feature scores. Training sets are fully labeled, so we can use all epochs for this calculation. New data sets are typically not fully labeled (otherwise there would be no need for automated scoring), so we just use a small number of labeled epochs provided by the user.

Thanks for clarifying. I have an additional question about the fixed weight used in the code. Does the one with weights = [.1 .35 .55] close to the class prevalance of your training dataset, where you use the term "reference dataset " in the paper? I am curious how important you find this parameter affect the generalizability of the test performance? I was trying to apply this to video data. I would like to know more about this. Thanks.

[zekebarger]

Good question, those are roughly the class prevalences in the training set. We don't believe that the parameter setting matters very much. And in our very limited testing, changing these values only produced tiny changes in the model's behavior. But we didn't examine this systematically.

[CaelynCheung1996]

Good question, those are roughly the class prevalences in the training set. We don't believe that the parameter setting matters very much. And in our very limited testing, changing these values only produced tiny changes in the model's behavior. But we didn't examine this systematically.

Thanks for the prompt response.