Just decoupling the feeder from the chassis makes a huge difference. IME, the silentstep stick did not make a huge difference in the sound of retractions, but decoupling the feeder from the chassis did.
It was expected that this wouldn't fix the E-Motor noise. Because the E-Motor moves very fast during retraction, Marlin sends two step pulses at a time so the interrupt handler doesn't get called more often than the poor little 8bit AVR can handle. So the output is: "step,step,pause ,step,step,pause, ...". Of course it doesn't help to smooth in-between steps when the steps themselves are irregular. Don't take my word for it, look at the code (stepper.cpp).