Quarto Preview & IPython cell magic & cell execution

[fenke]

Description

I am using an IPython cell magic (QiTangle) that allows one to define a label and dependencies for python code cells. The extension will add or update a cell (keyed on label) then it checks if the dependencies are defined (they were labels on other cells) and if so, runs the cell's code. If not all dependencies are defined, running the code is blocked. When a cell has no missing dependencies and resolves missing dependencies on other cells all cells who's missing dependencies become resolves are run.

This works as intended on the first run of Quarto Preview, the problem occurs when a document is updated and preview renders it again. Quarto runs the cell's contents regardless of QiTangle's decision to wait because of a cell's dependencies.

Below is a test document, it needs the QiTangle extension, available on PyPi. The extension tries to find a root folder, using a local virtual environment besides the document will stop it from going up to far. The document sets the export to /src/tests/qitangle/qa_example.py with /src in the same folder as the root of your python environment. Similar as in QiTangle's repository.

The fact that the problem occurs in Quarto Preview after a file update suggests that it is a problem with Quarto, but I have no idea what it could be, or why it executes cell code in reruns and not the first run.

---
title: "Cell execution with cell magic"
---


This is the IPython magic extension that Quarto's rendering isn't playing nice with.
```{python}
%load_ext qitangle.entangle

```

It's [available on PyiPi](https://pypi.org/project/qitangle/) and needs _autopep8_ and python >= 3.12

QiTangle exports selected code cells into a python source file, but it does so in an order specified with the cell; each cell gets a label and a list of depencies. A cell's dependencies will be exported before the cell itself. This is a core functionality, being able to write an explanation in an order that is not dictated by the needs of the computer language is an important aspect of literate programming.

Now, move on, specifying the file into which to tangle:

```{python}
%tangle -f --lib tests tests.qitangle.qa_example
```

Our first cell, nothing special:

Some code, nothing exciting
```{python}
%%entangle -f imports
import json, typing
print("Imported some modules")
```

Next, cell 'foo' depends on bar, which isn't defined yet, so nothing happens. Right? Well, in interactive use in VSCode/Positron with (I assume) IPython the code is stored but not executed. As expected. In Quarto preview it works sometimes, like, it works, but then you edit the document and save and it barfs a name-error at me; `NameError: name 'foo' is not defined`. So, only the first time it works as intended, subsequent saves it fails.


```{python}
%%entangle -f bar foo

print("Running in cell bar, calling foo()")

foo()

def bar():     print('function bar() called')

```

Nothing happens - as expected. But when rendering or previewing, quarto render executes the code even though the magic explicitly _does not_. Interestingly, if the call to foo() is admitted nothing bad happens since the interpreter doesn't encounter any undefined functions.

Lets check the report:

```{python}
%tangle --report
```


Lets check the report
```{python}
%tangle --missing
```

Cell 'foo' is missing, lets create it, which will fix dependencies - fixing it for all and all cells that where in limbo are executed

```{python}
%%entangle -f foo imports

print("Running in cell foo")

def foo():     print('function foo() called')
```

And in interactive mode cells 'foo' and 'bar' are now executed. Naturally, Quarto's render doesn't since it has already said goodbye. 

```{python}
%tangle -r
```


```{python}
%tangle -m
```

```{python}
%%entangle -f eof bar
print("Running in eof")
# end-of-file ---------------------------------------------------------
```

```{python}
%tangle -e eof
```

```{python}
%tangle -e

```

[fenke]

Well, sigh-me, it's still true, take a bit of distance from the problem and the solution will hit you. I am answering my own question, it appears.

Quarto probably keeps some kernel process loaded. You see, when you ctrl-c exit the running quarto-preview the next time you invoke it it complains about a broken connection. This update preview seems to clear the user namespace, but extensions remain loaded and preserve their state. In this case, the QiTangle extension. So, in a second run, all dependencies are know and QiTangle has no reason to not run the offending cell, while the refreshed running namespace is without it, and en error is born.

So, using %reload_ext to expunge it's state should resolve the problem.