Cannot open notebooks in hidden directory

[Stargrazer82301]

Description of your problem

What steps will reproduce the problem?

1. Installed fresh conda env:

conda create --name env python=3.10    # Also tried with 3.09, 3.11, to no avail
conda install spyder=5.5.1    # Spyder 6 install will not work for some reason, so only trying this with 6
conda install spyder-notebook spyder=5.5.1 -c conda-forge # If I don't put the spyder=5.5.1 in there it tries to install 6, which won't work.

3. Launch spyder, go to Notebook tab, try to open any .ipynb file
4. Get error message: "File Load Error for filename.ipynb
   Invalid response: 404 Not Found"

What is the expected output? What do you see instead?

I expect the notebook to load. Instead, I see the error message.

Please provide any additional information below (stack trace, screenshots, a gif, etc)

The "server info" window within spyder-notebooks has a lot of errors in it:

No QCoreApplication instance found. Application patches not applied. You have to call load_stylesheet function after instantiation of QApplication to take effect. 
[I 2024-09-10 16:52:46.854 ServerApp] jupyter_lsp | extension was successfully linked.
[I 2024-09-10 16:52:46.856 ServerApp] jupyter_server_terminals | extension was successfully linked.
[I 2024-09-10 16:52:46.858 ServerApp] jupyterlab | extension was successfully linked.
[W 2024-09-10 16:52:46.860 JupyterNotebookApp] 'browser' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[I 2024-09-10 16:52:46.861 ServerApp] notebook | extension was successfully linked.
[I 2024-09-10 16:52:46.964 ServerApp] notebook_shim | extension was successfully linked.
[I 2024-09-10 16:52:46.966 ServerApp] spyder_notebook | extension was successfully linked.
[I 2024-09-10 16:52:46.998 ServerApp] notebook_shim | extension was successfully loaded.
[I 2024-09-10 16:52:46.999 ServerApp] jupyter_lsp | extension was successfully loaded.
[I 2024-09-10 16:52:47.000 ServerApp] jupyter_server_terminals | extension was successfully loaded.
[I 2024-09-10 16:52:47.001 LabApp] JupyterLab extension loaded from /Users/cclark/Soft/miniconda3/envs/main/lib/python3.10/site-packages/jupyterlab
[I 2024-09-10 16:52:47.002 LabApp] JupyterLab application directory is /Users/cclark/Soft/miniconda3/envs/main/share/jupyter/lab
[I 2024-09-10 16:52:47.002 LabApp] Extension Manager is 'pypi'.
[I 2024-09-10 16:52:47.009 ServerApp] jupyterlab | extension was successfully loaded.
[I 2024-09-10 16:52:47.011 ServerApp] notebook | extension was successfully loaded.
[I 2024-09-10 16:52:47.013 ServerApp] spyder_notebook | extension was successfully loaded.
[I 2024-09-10 16:52:47.013 ServerApp] Serving notebooks from local directory: /Users/cclark
[I 2024-09-10 16:52:47.013 ServerApp] Jupyter Server 2.14.2 is running at:
[I 2024-09-10 16:52:47.013 ServerApp] http://localhost:8888/tree?token=75eb2a62f6727807acf82cbe568da14f0aca147c5c933e9b
[I 2024-09-10 16:52:47.013 ServerApp]     http://127.0.0.1:8888/tree?token=75eb2a62f6727807acf82cbe568da14f0aca147c5c933e9b
[I 2024-09-10 16:52:47.013 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 2024-09-10 16:52:47.015 ServerApp] 
    
    To access the server, open this file in a browser:
        file:///Users/cclark/Library/Jupyter/runtime/jpserver-30436-open.html
    Or copy and paste one of these URLs:
        http://localhost:8888/tree?token=75eb2a62f6727807acf82cbe568da14f0aca147c5c933e9b
        http://127.0.0.1:8888/tree?token=75eb2a62f6727807acf82cbe568da14f0aca147c5c933e9b
[I 2024-09-10 16:52:47.027 ServerApp] Skipped non-installed server(s): bash-language-server, dockerfile-language-server-nodejs, javascript-typescript-langserver, jedi-language-server, julia-language-server, pyright, python-language-server, r-languageserver, sql-language-server, texlab, typescript-language-server, unified-language-server, vscode-css-languageserver-bin, vscode-html-languageserver-bin, vscode-json-languageserver-bin, yaml-language-server
[W 2024-09-10 16:52:48.856 ServerApp] 404 GET /api/contents/Library/CloudStorage/Dropbox/Work/Scripts/Functional/ACS/ETC/compute_etc_wfc_dark_rate.ipynb?type=notebook&content=1&1726001568794 (e339aeb2d05b4b64a3125a287a89d912@::1) 14.56ms referer=http://localhost:8888/spyder-notebooks/Library/CloudStorage/Dropbox/Work/Scripts/Functional/ACS/ETC/compute_etc_wfc_dark_rate.ipynb
[I 2024-09-10 16:52:48.856 ServerApp] Refusing to serve hidden file or directory '/Users/cclark/Library/CloudStorage/Dropbox/Work/Scripts/Functional/ACS/ETC/compute_etc_wfc_dark_rate.ipynb', via 404 Error
[E 2024-09-10 16:52:48.856 ServerApp] Uncaught exception GET /api/contents/Library/CloudStorage/Dropbox/Work/Scripts/Functional/ACS/ETC/compute_etc_wfc_dark_rate.ipynb?type=notebook&content=1&1726001568794 (::1)
    HTTPServerRequest(protocol='http', host='localhost:8888', method='GET', uri='/api/contents/Library/CloudStorage/Dropbox/Work/Scripts/Functional/ACS/ETC/compute_etc_wfc_dark_rate.ipynb?type=notebook&content=1&1726001568794', version='HTTP/1.1', remote_ip='::1')
    Traceback (most recent call last):
      File "/Users/cclark/Soft/miniconda3/envs/main/lib/python3.10/site-packages/jupyter_server/services/contents/handlers.py", line 154, in get
        model = await ensure_async(
      File "/Users/cclark/Soft/miniconda3/envs/main/lib/python3.10/site-packages/jupyter_core/utils/__init__.py", line 198, in ensure_async
        result = await obj
      File "/Users/cclark/Soft/miniconda3/envs/main/lib/python3.10/site-packages/jupyter_server/services/contents/filemanager.py", line 922, in get
        model = await self._notebook_model(path, content=content, require_hash=require_hash)
      File "/Users/cclark/Soft/miniconda3/envs/main/lib/python3.10/site-packages/jupyter_server/services/contents/filemanager.py", line 861, in _notebook_model
        model = self._base_model(path)
      File "/Users/cclark/Soft/miniconda3/envs/main/lib/python3.10/site-packages/jupyter_server/services/contents/filemanager.py", line 237, in _base_model
        raise web.HTTPError(404, four_o_four)
    tornado.web.HTTPError: HTTP 404: Not Found (file or directory does not exist: 'Library/CloudStorage/Dropbox/Work/Scripts/Functional/ACS/ETC/compute_etc_wfc_dark_rate.ipynb')
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/Users/cclark/Soft/miniconda3/envs/main/lib/python3.10/site-packages/tornado/web.py", line 1790, in _execute
        result = await result
      File "/Users/cclark/Soft/miniconda3/envs/main/lib/python3.10/site-packages/jupyter_server/auth/decorator.py", line 73, in inner
        return await out
      File "/Users/cclark/Soft/miniconda3/envs/main/lib/python3.10/site-packages/jupyter_server/services/contents/handlers.py", line 180, in get
        await self._finish_error(
      File "/Users/cclark/Soft/miniconda3/envs/main/lib/python3.10/site-packages/jupyter_server/services/contents/handlers.py", line 114, in _finish_error
        self.write(message)
      File "/Users/cclark/Soft/miniconda3/envs/main/lib/python3.10/site-packages/tornado/web.py", line 914, in write
        raise RuntimeError("Cannot write() after finish()")
    RuntimeError: Cannot write() after finish()
[W 2024-09-10 16:52:49.511 ServerApp] 404 GET /api/contents/Library/CloudStorage/Dropbox/Work/Scripts/Functional/ACS/ETC/blah.ipynb?type=notebook&content=1&1726001569508 (e339aeb2d05b4b64a3125a287a89d912@::1) 1.90ms referer=http://localhost:8888/spyder-notebooks/Library/CloudStorage/Dropbox/Work/Scripts/Functional/ACS/ETC/blah.ipynb
[I 2024-09-10 16:52:49.512 ServerApp] Refusing to serve hidden file or directory '/Users/cclark/Library/CloudStorage/Dropbox/Work/Scripts/Functional/ACS/ETC/blah.ipynb', via 404 Error
[E 2024-09-10 16:52:49.512 ServerApp] Uncaught exception GET /api/contents/Library/CloudStorage/Dropbox/Work/Scripts/Functional/ACS/ETC/blah.ipynb?type=notebook&content=1&1726001569508 (::1)
    HTTPServerRequest(protocol='http', host='localhost:8888', method='GET', uri='/api/contents/Library/CloudStorage/Dropbox/Work/Scripts/Functional/ACS/ETC/blah.ipynb?type=notebook&content=1&1726001569508', version='HTTP/1.1', remote_ip='::1')
    Traceback (most recent call last):
      File "/Users/cclark/Soft/miniconda3/envs/main/lib/python3.10/site-packages/jupyter_server/services/contents/handlers.py", line 154, in get
        model = await ensure_async(
      File "/Users/cclark/Soft/miniconda3/envs/main/lib/python3.10/site-packages/jupyter_core/utils/__init__.py", line 198, in ensure_async
        result = await obj
      File "/Users/cclark/Soft/miniconda3/envs/main/lib/python3.10/site-packages/jupyter_server/services/contents/filemanager.py", line 922, in get
        model = await self._notebook_model(path, content=content, require_hash=require_hash)
      File "/Users/cclark/Soft/miniconda3/envs/main/lib/python3.10/site-packages/jupyter_server/services/contents/filemanager.py", line 861, in _notebook_model
        model = self._base_model(path)
      File "/Users/cclark/Soft/miniconda3/envs/main/lib/python3.10/site-packages/jupyter_server/services/contents/filemanager.py", line 237, in _base_model
        raise web.HTTPError(404, four_o_four)
    tornado.web.HTTPError: HTTP 404: Not Found (file or directory does not exist: 'Library/CloudStorage/Dropbox/Work/Scripts/Functional/ACS/ETC/blah.ipynb')
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/Users/cclark/Soft/miniconda3/envs/main/lib/python3.10/site-packages/tornado/web.py", line 1790, in _execute
        result = await result
      File "/Users/cclark/Soft/miniconda3/envs/main/lib/python3.10/site-packages/jupyter_server/auth/decorator.py", line 73, in inner
        return await out
      File "/Users/cclark/Soft/miniconda3/envs/main/lib/python3.10/site-packages/jupyter_server/services/contents/handlers.py", line 180, in get
        await self._finish_error(
      File "/Users/cclark/Soft/miniconda3/envs/main/lib/python3.10/site-packages/jupyter_server/services/contents/handlers.py", line 114, in _finish_error
        self.write(message)
      File "/Users/cclark/Soft/miniconda3/envs/main/lib/python3.10/site-packages/tornado/web.py", line 914, in write
        raise RuntimeError("Cannot write() after finish()")
    RuntimeError: Cannot write() after finish()

Versions and main components

• Spyder Version: 5.5.1
• Python Version: 3.10 (also tried 3.09 an 3.11)
• Qt Version Qt: 5.15.8
• PyQt Version: 5.15.9
• Jupyter Notebook Version:
  IPython : 8.27.0
  ipykernel : 6.29.5
  ipywidgets : not installed
  jupyter_client : 8.6.2
  jupyter_core : 5.7.2
  jupyter_server : 2.14.2
  jupyterlab : 4.2.5
  nbclient : 0.10.0
  nbconvert : 7.16.4
  nbformat : 5.10.4
  notebook : 7.2.2
  qtconsole : 5.5.2
  traitlets : 5.14.3
• Operating system: MacOS 13.6.4

Dependencies

Please go to the menu entry Help > Optional Dependencies (or
Help > Dependencies), press the button Copy to clipboard
and paste the contents below:

Mandatory:

applaunchservices >=0.3.0 : 0.3.0 (OK)
atomicwrites >=1.2.0 : 1.4.1 (OK)
chardet >=2.0.0 : 5.2.0 (OK)
cloudpickle >=0.5.0 : 3.0.0 (OK)
cookiecutter >=1.6.0 : 2.6.0 (OK)
diff_match_patch >=20181111 : 20230430 (OK)
intervaltree >=3.0.2 : 3.1.0 (OK)
IPython >=8.13.0,<9.0.0,!=8.17.1 : 8.27.0 (OK)
jedi >=0.17.2,<0.20.0 : 0.19.1 (OK)
jellyfish >=0.7 : 1.1.0 (OK)
jsonschema >=3.2.0 : 4.23.0 (OK)
keyring >=17.0.0 : 25.3.0 (OK)
nbconvert >=4.0 : 7.16.4 (OK)
numpydoc >=0.6.0 : 1.8.0 (OK)
parso >=0.7.0,<0.9.0 : 0.8.4 (OK)
pexpect >=4.4.0 : 4.9.0 (OK)
pickleshare >=0.4 : 0.7.5 (OK)
psutil >=5.3 : 6.0.0 (OK)
pygments >=2.0 : 2.18.0 (OK)
pylint >=2.5.0,<3.1 : 3.0.4 (OK)
pylint_venv >=3.0.2 : 3.0.3 (OK)
pyls_spyder >=0.4.0 : 0.4.0 (OK)
pylsp >=1.10.0,<1.11.0 : 1.10.1 (OK)
pylsp_black >=2.0.0,<3.0.0 : 2.0.0 (OK)
qdarkstyle >=3.2.0,<3.3.0 : 3.2.3 (OK)
qstylizer >=0.2.2 : 0.2.3 (OK)
qtawesome >=1.2.1 : 1.3.1 (OK)
qtconsole >=5.5.1,<5.6.0 : 5.5.2 (OK)
qtpy >=2.1.0 : 2.4.1 (OK)
rtree >=0.9.7 : 1.3.0 (OK)
setuptools >=49.6.0 : 73.0.1 (OK)
sphinx >=0.6.6 : 8.0.2 (OK)
spyder_kernels >=2.5.0,<2.6.0 : 2.5.2 (OK)
textdistance >=4.2.0 : 4.6.2 (OK)
three_merge >=0.1.1 : 0.1.1 (OK)
watchdog >=0.10.3 : 5.0.2 (OK)
zmq >=22.1.0 : 26.2.0 (OK)

Optional:

cython >=0.21 : None (NOK)
matplotlib >=3.0.0 : None (NOK)
numpy >=1.7 : None (NOK)
pandas >=1.1.1 : None (NOK)
scipy >=0.17.0 : None (NOK)
sympy >=0.7.3 : None (NOK)

Spyder plugins:

spyder_notebook.notebookplugin 0.5.1 : 0.5.1 (OK)

[jitseniesen]

Thanks for writing everything down. The first suspicious thing that I see is the message:

Refusing to serve hidden file or directory '/Users/cclark/Library/CloudStorage/Dropbox/Work/Scripts/Functional/ACS/ETC/compute_etc_wfc_dark_rate.ipynb', via 404 Error

Assuming that this is the notebook that you want to open, can you please check whether it is a hidden file or inside a hidden directory?

Does creating a new notebook work?

[Stargrazer82301]

Aha, you were right! The Library directory was hidden (despite none of its subfolders being hidden, and despite not starting with a .). Creating new workbooks in directories not under Library worked fine, although new workbooks under Library threw the same error. Looks like I need sysadmins from my org to fix the hidden-ness, but that's separate problem entirely.

Thanks for figuring it out, despite it not strictly being an issue with spyder-notebooks! I'm not a MacOS user by choice or inclination...

[jitseniesen]

Maybe not strictly our issue but it can probably be handled better on our side. I think Jupyter has an option for allowing hidden files which may sense for us, or perhaps we can give a more informative error message. I'm leaving the issue open for now.

[jitseniesen]

I just realized that somebody else had already opened another issue about this, namely issue #455. I am closing this one so that we don't have two issues about the same thing.