Reassignment sharpening

[Artoria2e5]

Audacity provides two useful, not-just-FFT algorithms for its spectrogram. One of them is reassignment, which produces fairly sharp spectrograms by moving data according to local estimates of instantaneous frequency and group delay.

With FFT Windows both at 4096, compare:

I'm prrretty sure you can see the difference immediately.

[stuerp]

I'm not a sound engineer and don't have a clue how to implement this unless you can point me to some specs or example code.

[Artoria2e5]

I think it's in https://github.com/audacity/audacity/blob/2e0ad8fd38f18d132ef2da3eb236668821dc103a/au3/src/tracks/playabletrack/wavetrack/ui/SpectrumCache.cpp#L97. Probably start from https://github.com/audacity/audacity/blob/2e0ad8fd38f18d132ef2da3eb236668821dc103a/au3/src/tracks/playabletrack/wavetrack/ui/SpectrumCache.cpp#L202.

[TF3RDL]

I'm not a sound engineer and don't have a clue how to implement this unless you can point me to some specs or example code.

BTW, I have an example of the implementation of reassigned FFT, which is on my own AudioWorklet-based spectrogram visualization example on CodePen, which sharpened the time-frequency resolution especially for spectrogram display (though currently, I had to use a VST plugin like Tritik Visu to display reassigned spectrogram in foobar2000 like this vid on YT)

Though while I'm also not a professional audio engineer, I've implemented some of analysis algorithms myself on several web-based audio-related projects on CodePen despite that

[TF3RDL]

I think a reassigned spectrogram component can represent how auditory system actually works (in terms of frequency separation and tracking) better than a constant-Q transform (as it has poor time resolution on bass frequencies when doing high-resolution CQT analysis, though the same thing also happen on SWIFT and Analog-Style/IIR filter bank modes in your foo_vis_spectrum_analyzer component too as it only models the anatomy of the auditory system), but beware that reassignment method requires a window function (anything other than rectangular), as it won't work without them (on the right part of this image below):

and also have to be careful with window function truncation as reassigned FFT I've implemented on my own aforementioned CodePen project (updated with customizable window functions) uses a derivative of a windowing function for the auxiliary FFT (used for frequency reassignment), as doing so in a wrong way on certain window functions like Hamming can mess up with reassignment method (on the right part below):

Also on unrelated note, the reassigned spectrogram thing (reassigned FFT mode + spectrogram display modes) is on my wishlist for version 2.0.0.0/2.x.x.x of Crossover's foo_enhanced_spectrum_analyzer (Enhanced Spectrum analyzer), an another spectrum analyzer component for foobar2000