Currently, the port name(s) are read while KSP is loading, and cannot be changed while KSP is running.
This means relaunching KSP to change the config, especially if the port name was not correctly set or if a new controller is connected.
The idea would be to either add a way to set the com port with the GUI, or add a button to reload the config file and update everything within SimPit