PR: Use spyder-updater to handle updates (Installers)

[mrclary]

Use spyder-updater for minor/micro updates to Spyder's conda-based installer.

All update procedures and user interaction remains unchanged for all scenarios except minor/micro updates to Spyder's conda-based installer. All major updates are still handled by Spyder, as well as all downloads. Minor/micro updates are only allowed for the conda-based installer, as before, but now spyder-updater is launched to provide the graphical user interface for the conda environment updates.

For minor/micro updates to Spyder:

• The user is first notified about its availability, as before.
• However, upon affirmatively proceeding with the download (the first message window), rather than proceeding immediately downloading the Spyder lock files, Spyder will first check for and install updates to spyder-updater. Upon successful completion (or if no spyder-updater updates are available), then Spyder will proceed to download the Spyder lock files.
• Spyder notifies the user upon successful download of the lock files and the user has the option to proceed with the install or install after quitting Spyder; this is as before.
• Upon Spyder quitting, Spyder will launch spyder-updater.

Testing with Spyder Updater

To test integration with Spyder Updater, do the following.

1. Install Spyder 6.1.0a1 using our conda-based installer.
2. Checkout the spyder-updater-test branch from mrclary/spyder fork in your local Spyder repository.
3. Install Spyder in editable mode into Spyder's runtime environment.

   $ conda run --live-stream -p ~/Library/spyder-6/envs/spyder-runtime python path/to/spyder/repo/install_dev_repos.py
   

4. Spoof Spyder version to 5.9.9 to test major update, or 6.0.5 to test minor update. Change the Github url to point to the mrclary fork. The latest commit on the spyder-updater branch contains these changes (to be reverted before merging). You can modify the version number as desired for testing.
5. Start Spyder, ensure that "Check stable releases only" is deselected in preferences, and check for updates. I recommend deselecting "Check for updates on startup".

If checking for major update:

6. Expect a major update to be available (6.1.0a1) and proceed with the download and expect the .pkg/.sh/.exe installer to be downloaded to spyder-<user>/updates/6.1.0a1 in a temporary location.
7. When prompted to install, do not proceed because major installs will execute as previously without engaging the Spyder Updater.

If checking for minor/micro update:

6. Expect a minor/micro update to be available (6.1.0a1) and proceed with the download.
   • Expect the Spyder Updater zip file to be downloaded and extracted into spyder-<user>/updates/spyder-updater/0.1.0 in a temporary location. You should see a conda package and conda lock files.
   • Expect the spyder-updater environment to be installed alongside the spyder-runtime environment.
7. Proceed to install the update when prompted.
   • Expect Spyder to quit and the Spyder Updater to launch and install Spyder 6.1.0a1. Upon completion, Spyder Updater will quit and Spyder should launch.

Return to step 3 to test different scenarios.

[ccordoba12]

Great work here, thanks @mrclary!

[mrclary]

Note: Installing the spyder-updater environment takes approximately 40s. This will only occur for updates from 6.0.x to 6.1, or when Spyder-updater requires an update.

[ccordoba12]

A few more suggestions for you @mrclary.

[ccordoba12]

Note: Installing the spyder-updater environment takes approximately 40s. This will only occur for updates from 6.0.x to 6.1, or when Spyder-updater requires an update.

What if we create the spyder-updater env right after 6.1 is launched for the first time? That way we'll avoid this long delay when updating to 6.1.1

[mrclary]

What if we create the spyder-updater env right after 6.1 is launched for the first time? That way we'll avoid this long delay when updating to 6.1.1

I agree that we should do something here. I'd prefer to leave it for a another PR, and possibly another alpha or beta release, if that's okay.

One issue is that anytime the updater requires updating, it may take this long, even if it is already installed. The updater will need to be checked and updated prior to updating Sypder, so this could happen even after it is first installed on first launch of 6.1+. In addition to your proposal, I'd like to consider adding a message window with progress bar informing the user that Spyder-updater is updating, similar to our download.

[ccordoba12]

I agree that we should do something here. I'd prefer to leave it for a another PR, and possibly another alpha or beta release, if that's okay.

Ok, no problem.

@dalthviz, please follow the procedure described by @mrclary in the OP on Windows to see if it works for you.

[dalthviz]

Gave this a check on Windows using the latest commit where version 6.0.5 is set and an error was raised:

Traceback (most recent call last):
  File "E:\Acer\Documentos\Spyder\Spyder otros\mrclary\spyder\spyder\plugins\updatemanager\workers.py", line 609, in start
    self._install_update()
  File "E:\Acer\Documentos\Spyder\Spyder otros\mrclary\spyder\spyder\plugins\updatemanager\workers.py", line 599, in _install_update
    proc.check_returncode()
  File "C:\Users\dalth\AppData\Local\spyder-6\envs\spyder-runtime\Lib\subprocess.py", line 502, in check_returncode
    raise CalledProcessError(self.returncode, self.args, self.stdout,
subprocess.CalledProcessError: Command 'C:/Users/dalth/AppData/Local/spyder-6\Scripts\conda.exe create --yes --prefix C:\Users\dalth\AppData\Local\spyder-6\envs\spyder-updater --file C:\Users\dalth\AppData\Local\Temp\spyder\updates\spyder-updater\0.1.0\conda-updater-linux-64.lock && C:/Users/dalth/AppData/Local/spyder-6\Scripts\conda.exe install --yes --prefix C:\Users\dalth\AppData\Local\spyder-6\envs\spyder-updater --no-deps --force-reinstall C:\Users\dalth\AppData\Local\Temp\spyder\updates\spyder-updater\0.1.0\spyder-updater*.conda' returned non-zero exit status 1.
Command 'C:/Users/dalth/AppData/Local/spyder-6\Scripts\conda.exe create --yes --prefix C:\Users\dalth\AppData\Local\spyder-6\envs\spyder-updater --file C:\Users\dalth\AppData\Local\Temp\spyder\updates\spyder-updater\0.1.0\conda-updater-linux-64.lock && C:/Users/dalth/AppData/Local/spyder-6\Scripts\conda.exe install --yes --prefix C:\Users\dalth\AppData\Local\spyder-6\envs\spyder-updater --no-deps --force-reinstall C:\Users\dalth\AppData\Local\Temp\spyder\updates\spyder-updater\0.1.0\spyder-updater*.conda' returned non-zero exit status 1.
Traceback (most recent call last):
  File "E:\Acer\Documentos\Spyder\Spyder otros\mrclary\spyder\spyder\plugins\updatemanager\workers.py", line 609, in start
    self._install_update()
  File "E:\Acer\Documentos\Spyder\Spyder otros\mrclary\spyder\spyder\plugins\updatemanager\workers.py", line 599, in _install_update
    proc.check_returncode()
  File "C:\Users\dalth\AppData\Local\spyder-6\envs\spyder-runtime\Lib\subprocess.py", line 502, in check_returncode
    raise CalledProcessError(self.returncode, self.args, self.stdout,
subprocess.CalledProcessError: Command 'C:/Users/dalth/AppData/Local/spyder-6\Scripts\conda.exe create --yes --prefix C:\Users\dalth\AppData\Local\spyder-6\envs\spyder-updater --file C:\Users\dalth\AppData\Local\Temp\spyder\updates\spyder-updater\0.1.0\conda-updater-linux-64.lock && C:/Users/dalth/AppData/Local/spyder-6\Scripts\conda.exe install --yes --prefix C:\Users\dalth\AppData\Local\spyder-6\envs\spyder-updater --no-deps --force-reinstall C:\Users\dalth\AppData\Local\Temp\spyder\updates\spyder-updater\0.1.0\spyder-updater*.conda' returned non-zero exit status 1.

I think the required files were downloaded:

But seems like the defined command tries to use the linux conda-lock file even when I'm on Windows? 🤔

Also, when checking the related traceback, I noticed that there is some code with errors at

spyder/spyder/plugins/updatemanager/workers.py

Lines 601 to 628 in 270860d

	def start(self):
	"""Main method of the worker."""
	try:
	self._check_asset_available()
	if self.asset_info is None and UPDATER_VERSION == parse("0.0.0"):
	proc.check_returncode()
	elif self.asset_info is not None:
	self._download_asset()
	self._install_update()
	except Exception as err:
	# Send untracked errors to our error reporter
	self.error = str(err)
	error_data = dict(
	text=traceback.format_exc(),
	is_traceback=True,
	title=_("Error when updating Spyder-updater"),
	)
	self.sig_exception_occurred.emit(error_data)
	logger.error(err, exc_info=err)
	finally:
	# At this point we **must** emit the signal below so that the
	# "Check for updates" action in the Help menu is enabled again
	# after the check has finished (it's disabled while the check is
	# running).
	try:
	self.sig_ready.emit(self.error is None)
	except RuntimeError:
	pass

Not totally sure but seems to me like proc is not defined for the WorkerUpdateUpdater start method?:

Edit: Left a couple of comments after checking the code in places from where maybe the experienced error is being caused

[ccordoba12]

@dalthviz, please give another try to this one to see if the problems you encountered are fixed now.

[dalthviz]

Checked again and the process still triggers an error:

Traceback (most recent call last):
  File "E:\Acer\Documentos\Spyder\Spyder otros\mrclary\spyder\spyder\plugins\updatemanager\workers.py", line 612, in start
    self._install_update()
  File "E:\Acer\Documentos\Spyder\Spyder otros\mrclary\spyder\spyder\plugins\updatemanager\workers.py", line 599, in _install_update
    proc.check_returncode()
  File "C:\Users\dalth\AppData\Local\spyder-6\envs\spyder-runtime\Lib\subprocess.py", line 502, in check_returncode
    raise CalledProcessError(self.returncode, self.args, self.stdout,
subprocess.CalledProcessError: Command 'C:/Users/dalth/AppData/Local/spyder-6\Scripts\conda.exe create --yes --prefix C:\Users\dalth\AppData\Local\spyder-6\envs\spyder-updater --file C:\Users\dalth\AppData\Local\Temp\spyder\updates\spyder-updater\0.1.0\conda-updater-win-64.lock && C:/Users/dalth/AppData/Local/spyder-6\Scripts\conda.exe install --yes --prefix C:\Users\dalth\AppData\Local\spyder-6\envs\spyder-updater --no-deps --force-reinstall C:\Users\dalth\AppData\Local\Temp\spyder\updates\spyder-updater\0.1.0\spyder-updater*.conda' returned non-zero exit status 1.
Command 'C:/Users/dalth/AppData/Local/spyder-6\Scripts\conda.exe create --yes --prefix C:\Users\dalth\AppData\Local\spyder-6\envs\spyder-updater --file C:\Users\dalth\AppData\Local\Temp\spyder\updates\spyder-updater\0.1.0\conda-updater-win-64.lock && C:/Users/dalth/AppData/Local/spyder-6\Scripts\conda.exe install --yes --prefix C:\Users\dalth\AppData\Local\spyder-6\envs\spyder-updater --no-deps --force-reinstall C:\Users\dalth\AppData\Local\Temp\spyder\updates\spyder-updater\0.1.0\spyder-updater*.conda' returned non-zero exit status 1.
Traceback (most recent call last):
  File "E:\Acer\Documentos\Spyder\Spyder otros\mrclary\spyder\spyder\plugins\updatemanager\workers.py", line 612, in start
    self._install_update()
  File "E:\Acer\Documentos\Spyder\Spyder otros\mrclary\spyder\spyder\plugins\updatemanager\workers.py", line 599, in _install_update
    proc.check_returncode()
  File "C:\Users\dalth\AppData\Local\spyder-6\envs\spyder-runtime\Lib\subprocess.py", line 502, in check_returncode
    raise CalledProcessError(self.returncode, self.args, self.stdout,
subprocess.CalledProcessError: Command 'C:/Users/dalth/AppData/Local/spyder-6\Scripts\conda.exe create --yes --prefix C:\Users\dalth\AppData\Local\spyder-6\envs\spyder-updater --file C:\Users\dalth\AppData\Local\Temp\spyder\updates\spyder-updater\0.1.0\conda-updater-win-64.lock && C:/Users/dalth/AppData/Local/spyder-6\Scripts\conda.exe install --yes --prefix C:\Users\dalth\AppData\Local\spyder-6\envs\spyder-updater --no-deps --force-reinstall C:\Users\dalth\AppData\Local\Temp\spyder\updates\spyder-updater\0.1.0\spyder-updater*.conda' returned non-zero exit status 1.

However, I think the correct lock file is being used. Trying to manually run the command that the traceback shows I see that it is failing due to a PathNotFoundError;

PathNotFoundError: C:\Users\dalth\AppData\Local\Temp\spyder\updates\spyder-updater\0.1.0\spyder-updater*.conda

Maybe on Windows the actual name of the file should be used? So something like C:\Users\dalth\AppData\Local\Temp\spyder\updates\spyder-updater\0.1.0\spyder-updater-0.1.0.dev0-py_0.conda 🤔

[mrclary]

@dalthviz, thanks for testing that. Perhaps you are correct and Windows won't glob the file name. I'll see what I can do about that. But just to confirm, the conda file is there, correct? And the spyder-updater environment was created?

[dalthviz]

But just to confirm, the conda file is there, correct? And the spyder-updater environment was created?

Yep, the conda file is there and indeed the env was created 👍

[dalthviz]

Gave this another check with the latest changes and seems like now after spyder-updater related things get installed another dialog prompting to install a new version (6.1.0a1) appears. However, when I selected the Yes option in the dialog, Spyder closes and seems like an error dialog tries to appear:

After that, I'm unable to open Spyder 🤔

Will try to reinstall and run Spyder on debug mode before doing the update to see if I get any relevant traceback

[mrclary]

Thanks @dalthviz, I'll take a look as well.

[mrclary]

@dalthviz, @ccordoba12. I think I found the issue; the subprocess command to run the update script should be "cmd" instead of "start". I created a pull request for this fix spyder-ide/spyder-updater#3. I also updated the test release on my fork to contain this change.

Delete the spyder-6/envs/spyder-updater environment and try the update through Spyder again. This should create the spyder-updater environment using the new release and work as expected 🤞🏼

[dalthviz]

I tried again removing the spyder-updater env but seems like I still unable to update :/ Will try to reinstall Spyder and do a clean setup (maybe I also need to remove the temp files downloaded for spyder-updater and spyder 6.1.0.a1?) and check again

[dalthviz]

Gave another check with the latest changes and indeed seems like the minor updates are working 🎉

Also, although a major update is not being handled via the updater, I gave it a try. Seems like no new updater related logic is triggered (which I think is expected) but it seems like the process is failing (maybe the logic currently implemented expects to work as if the previous installed version is Spyder 5?):

Is there already an issue for that or could it be worthy to open an issue? Or maybe is failing for me since some other setup is needed to be able to properly trigger that kind of update? 🤔 Kind of out of the scope of this PR but probably something that needs to be addressed in the future so sharing the finding.

[mrclary]

I retested on macOS and everything looks good. When the workflows are done, this should be ready for merge.

[ccordoba12]

Thanks @mrclary for this great improvement and @dalthviz for your extensive testing!

Note: The failures in our tests are unrelated to this and were fixed in PR #24234.