This code implements an iterative screening simulation for LC1234 dataset. The param_configs/ folder contains strategy hyperparameter files. These parameter files also specify the data source, type of model, type of next batch selector, etc. The simulation runs for a maximum number of iterations, where in each iteration it generates the next selected batch and evaluates it. Each iteration can take around 500 seconds. With the 72 hour limit on CHTC, this means we can run around 450 iterations for each parameter set.
Currently there are two main next batch selectors: ClusterBased and InstanceBased. The parameter config files for these specify a hyperparameter set in the billions. We will need to apply random or bayesian grid searching. We might be able to apply a combination of both by running a large number of simulations on chtc concurrently.
This code implements an iterative screening simulation for LC1234 dataset. The
param_configs/folder contains strategy hyperparameter files. These parameter files also specify the data source, type of model, type of next batch selector, etc. The simulation runs for a maximum number of iterations, where in each iteration it generates the next selected batch and evaluates it. Each iteration can take around 500 seconds. With the 72 hour limit on CHTC, this means we can run around 450 iterations for each parameter set.Currently there are two main next batch selectors: ClusterBased and InstanceBased. The parameter config files for these specify a hyperparameter set in the billions. We will need to apply random or bayesian grid searching. We might be able to apply a combination of both by running a large number of simulations on chtc concurrently.