4.5mm retraction is normal but some need more like 5 or 5.5mm
I have never had it look as bad as what you have though, that's why i think your temp is to high.
the reason why minimum layer time can be bad sometimes is it lowers the speed so much that the head is touching the material for longer, heating all those layers too much, also the slower your print speed the lower you need to set your nozzle temp. but in your last prints it doesn't look like that is the issue.
It also depends on the batch of XT you got as some of mine i print at 225deg as it starts to burn at higher temps.