I've tried a few things and now Cura works again:
Uninstalled Cura 3.5.0 (previous versions being already uninstalled). Also, during uninstall process, I selected to delete my configuration files.
At this point, there's nothing in C:\Program Files\Ultimakerwhatsoever or in C:\ProgramData. The are still files in C:\Users\<user>\AppData\Roaming\cura and C:\Users\<user>\AppData\Local\cura (although only related to previous version - 3.3 and 3.4 in my case - as well as stderr and stdout log files. I moved them somewhere else, just in case.
I made a fresh install (no additionnal component selected: no arduino drivers, no visual studio). No other issue, Cura starts at the end (as before), and after adding custom printer, allowed to send date, then closing Cura, it starts again (and again) (and again). One solution of the issue, but still, we lose all the configuration.
I uninstalled Cura, suppressed the newly created "user" files in both local and roaming, put the ones from previous versions back in place, re-installed Cura, it starts with my old parameters at the end of the install. At this point, Cura runs normally (closes and starts as it should). Moreover, the Cura.exe process in the task manager is bigger than when the bug occured (~200MB instead of ~42MB).
Something apparently failed during the install process, but I don't know what. I'll repost here if the problem occurs again.
Hope this helps!