Reconstructing Figure 8

[expectopatronum]

Hi again,
I was able to reconstruct Figure 9 and the corresponding lines in Figure 8 (not exactly but closely)
(MaxMin Hinge 0.1, MaxMin Hinge 0.3, ReLU Hinge 0.1 and ReLU Hinge 0.3) by using fc_classification_l_inf_margin.json and changing margin and activation appropriately.

I am currently trying to figure out how to reproduce the rest. Should 'ReLU Standard' and 'MaxMin Standard' be computed by using fc_classification_l_inf.json and setting the correct activation function?

I assume the two lines 'PGD eps 0.1' and 'PGD eps 0.15' (mentioned in "We also compared to PGD training"?) are computed by "train_pgd.py"? Which config file should be used for this experiment?

Best regards,
Verena

[AtheMathmo]

Hi Verena,

Thanks for your interest in the project!

The standard models are without any Lipschitz constraints in the network. These can be trained using the fc_classification.json configs with the appropriate activations. (For reference, we found an initial learning rate of 0.001 and 0.005 were best for ReLU and MaxMin respectively).

You are correct that the PGD models were trained with the train_pgd.py script. I believe that we used the default fc_classification.json configs corresponding to the networks without Lipschitz constraints. We also tuned learning rates but I'm afraid I can't find the optimal values right now.

Please let me know if you have any other issues reproducing the results. I will also update the README to include this information.

James