Journal Paper

SemLC Journal Paper

Research Questions

1. To which extend does lateral connectivity give rise to a semantic ordering of filters in the first convolutional layer?
   1. How do ranges of short range excitation and long range inhibition influence the overall ordering of filters?
2. How does lateral connectivity affect the orientation tuning of filters?
3. To what extent do representations in the first and later layers change due to lateral connectivity?
4. How do spatial frequency and phase play into the occurrence of lateral connectivity?
5. Can lateral connectivity in deeper layers have a similar effect?

New Experiments

Predefined Filters

• Fixed Filters
  • Gabor?
  • Pretrained on baseline model, then ordered?
• Adaptive Wavelet
  • entirely free
  • DoG
  • wavelet space? Difference-of-Beta-Space most flexible?
• See if correlation plot sinus curves stem from frequency of Gabors

Experimental Conditions

• Baseline
• SemLC-A
• SemLC-P + DoG
• Mix of Sinusoids with Gaussian Envelope

Effect of MH Width on Order

Experimental Conditions

• (Baseline)
• SemLC DoG with width permutations

Orientation Tuning

• present lines of all orientations and plot the activity response as a curve; average and compare with vs without SemLC
• Correlation Plots (as in DL paper)

Experimental Conditions

• (Baseline)
• SemLC (Best Order Condition)

Coverage

• HP Optim with SemLC

Experimental Conditions

• (Baseline)
• (SemLC 1)
• SemLC 2
• SemLC 3
• SemLC 1, 2
• SemLC 1, 2, 3

Intuition: With gaps, gaps become arbitrary since we cannot control/guarantee for the level of abstraction per layer

Autoencoder/Classification

• Compare RDMs
• Alex has Data on this?

Experimental Conditions

• (Baseline)
• SemLC best Order?
• Human/Monkey? <-- ask CCN people

Color, Phase, Frequency

• Build multiple rings, expecting same phases and same frequencies to occur within the same rings, or at least every orientation to only occur once per ring
• Compare to B&W-Models

Other

• Baseline Model
  • https://doi.org/10.1101/2020.05.07.082743
  • https://doi.org/10.1101/2020.07.22.216713
  • https://doi.org/10.1101/407007

Suggestions by the Group

• Contact Kriegeskorte for fMRI Data
• See if Brain-Score data can be used for RDMs
• Search wavelet space by looking at composition of sinusoids
• Apply/Train on lines/gratings to see what SemLC does at deeper layers
• Add to (Alex') saliency model