Skip to content

Commit 674f4c8

Browse files
jph00jph00
committed
docs
1 parent 638bdf5 commit 674f4c8

File tree

1 file changed

+181
-87
lines changed

1 file changed

+181
-87
lines changed

‎nbs/tutorials/extensions.ipynb‎

Lines changed: 181 additions & 87 deletions
Original file line numberDiff line numberDiff line change
@@ -28,11 +28,35 @@
2828
},
2929
{
3030
"cell_type": "markdown",
31-
"id": "521da717-490e-473c-8d4c-c0ae0db3061d",
31+
"id": "73a38ba8",
3232
"metadata": {},
3333
"source": [
34-
"## Built-in export processor: `scrub_magics`\n",
34+
"## Export processors"
35+
]
36+
},
37+
{
38+
"cell_type": "markdown",
39+
"id": "63b0b8a0",
40+
"metadata": {},
41+
"source": [
42+
"The `Processor` class shown later is for **doc processors** — they transform notebooks when building documentation with `nbdev_docs`.\n",
3543
"\n",
44+
"First though we'll discuss the simpler kind of processor: **export processors** — they transform code cells when exporting to `.py` files with `nbdev_export`. These are just plain functions, not classes."
45+
]
46+
},
47+
{
48+
"cell_type": "markdown",
49+
"id": "cccd389c",
50+
"metadata": {},
51+
"source": [
52+
"### Built-in export processor: `scrub_magics`"
53+
]
54+
},
55+
{
56+
"cell_type": "markdown",
57+
"id": "c814315c",
58+
"metadata": {},
59+
"source": [
3660
"nbdev includes one export processor out of the box: `scrub_magics`. It removes Jupyter magic lines (like `%%time`, `%matplotlib inline`, `%%spark`) from your exported `.py` files.\n",
3761
"\n",
3862
"To enable it, add to your `pyproject.toml`:\n",
@@ -50,10 +74,119 @@
5074
"def my_func(): return 42\n",
5175
"```\n",
5276
"\n",
53-
"Will export without the `%%time` line.\n",
77+
"Will export without the `%%time` line."
78+
]
79+
},
80+
{
81+
"cell_type": "markdown",
82+
"id": "497f1841",
83+
"metadata": {},
84+
"source": [
85+
"### Writing an export processor"
86+
]
87+
},
88+
{
89+
"cell_type": "markdown",
90+
"id": "871f112a",
91+
"metadata": {},
92+
"source": [
93+
"An export processor is a function that receives a notebook cell and modifies it in place:\n",
94+
"\n",
95+
"```python\n",
96+
"def my_export_proc(cell):\n",
97+
" \"Transform code cells during export\"\n",
98+
" if cell.cell_type != 'code': return\n",
99+
" # Modify cell.source as needed\n",
100+
" cell.source = transform(cell.source)\n",
101+
"```\n",
102+
"\n",
103+
"For example, here's a formatter using `black`:\n",
104+
"\n",
105+
"```python\n",
106+
"# nbdev_black/__init__.py\n",
107+
"def black_format(cell):\n",
108+
" \"Format code cells with black\"\n",
109+
" if cell.cell_type != 'code': return\n",
110+
" try:\n",
111+
" import black\n",
112+
" cell.source = black.format_str(cell.source, mode=black.Mode())\n",
113+
" except: pass # Leave malformed code unchanged\n",
114+
"```"
115+
]
116+
},
117+
{
118+
"cell_type": "markdown",
119+
"id": "21360d01",
120+
"metadata": {},
121+
"source": [
122+
"### Enabling export processors"
123+
]
124+
},
125+
{
126+
"cell_type": "markdown",
127+
"id": "45062122",
128+
"metadata": {},
129+
"source": [
130+
"Add `export_procs` to your `pyproject.toml`:\n",
131+
"\n",
132+
"```toml\n",
133+
"[tool.nbdev]\n",
134+
"export_procs = [\"nbdev_black:black_format\"]\n",
135+
"```\n",
54136
"\n",
55-
"## What will this cover?\n",
137+
"You can chain multiple processors — they run in order:\n",
56138
"\n",
139+
"```toml\n",
140+
"[tool.nbdev]\n",
141+
"export_procs = [\n",
142+
" \"nbdev_black:black_format\",\n",
143+
" \"my_project.procs:my_custom_proc\",\n",
144+
"]\n",
145+
"```"
146+
]
147+
},
148+
{
149+
"cell_type": "markdown",
150+
"id": "a1f901f7",
151+
"metadata": {},
152+
"source": [
153+
"### Key differences from doc processors"
154+
]
155+
},
156+
{
157+
"cell_type": "markdown",
158+
"id": "2b23f0cd",
159+
"metadata": {},
160+
"source": [
161+
"| | Doc processors | Export processors |\n",
162+
"|---|---|---|\n",
163+
"| **Used by** | `nbdev_docs` | `nbdev_export` |\n",
164+
"| **Interface** | `Processor` class with `cell()` method | Simple function |\n",
165+
"| **Config key** | `doc_procs` | `export_procs` |\n",
166+
"| **Purpose** | Transform notebooks for documentation | Transform code for `.py` files |"
167+
]
168+
},
169+
{
170+
"cell_type": "markdown",
171+
"id": "9fd755ff",
172+
"metadata": {},
173+
"source": [
174+
"## Doc processors"
175+
]
176+
},
177+
{
178+
"cell_type": "markdown",
179+
"id": "0282b545",
180+
"metadata": {},
181+
"source": [
182+
"### What will this cover?"
183+
]
184+
},
185+
{
186+
"cell_type": "markdown",
187+
"id": "40fe7215",
188+
"metadata": {},
189+
"source": [
57190
"With `nbdev`, it's possible to customize and extend it further beyond the standard capabilities through a well thoughtout and scalable framework. Does your particular library or need require you to inject custom quarto additives in certain cells? What about if you want to do something more trivial such as finding shortcuts to replace complicated quarto directives more easily (such as replacing `::: {.column-margin}` with `#| margin`)? \n",
58191
"\n",
59192
"Writing custom plugins with `nbdev` is the easiest method to achieve this, and with this tutorial we will bring you up to speed on how you can use this to create your own plugins to expand and simplify your literate-programming experience with `nbdev` and quarto. \n",
@@ -70,8 +203,14 @@
70203
"id": "ba8290bf-a950-45ae-8eec-b2b0f903b57a",
71204
"metadata": {},
72205
"source": [
73-
"## Getting started, how does `nbdev` make this easy?\n",
74-
"\n",
206+
"### Getting started, how does `nbdev` make this easy?"
207+
]
208+
},
209+
{
210+
"cell_type": "markdown",
211+
"id": "96e6c1ca",
212+
"metadata": {},
213+
"source": [
75214
"First let's visualize just what we're trying to achieve. \n",
76215
"\n",
77216
"Instead of doing the following code which will add `\"Some text\"` to the sidebar (as shown off to the side currently):"
@@ -146,8 +285,14 @@
146285
"id": "f542766c-a777-4d39-b6f8-5b33d393290d",
147286
"metadata": {},
148287
"source": [
149-
"## Bringing in what we need\n",
150-
"\n",
288+
"### Bringing in what we need"
289+
]
290+
},
291+
{
292+
"cell_type": "markdown",
293+
"id": "3afe4878",
294+
"metadata": {},
295+
"source": [
151296
"The actual imports we need to use from `nbdev` is truly not that many! We just need two:\n",
152297
"- `extract_directives`, to read in the list of `#|` written\n",
153298
"- The `Processor` class that will actually perform what we want on notebook cells.\n",
@@ -193,8 +338,14 @@
193338
"id": "709365bd-00bc-4e27-9a95-2a5872181ad6",
194339
"metadata": {},
195340
"source": [
196-
"## Writing a converter\n",
197-
"\n",
341+
"### Writing a converter"
342+
]
343+
},
344+
{
345+
"cell_type": "markdown",
346+
"id": "5118c08f",
347+
"metadata": {},
348+
"source": [
198349
"The first step is creating a quick and easy way to take the `nbdev` directive we want to use (such as `#| div column-margin`) and convert it quickly into something quarto will then read (such as `::: {.column-margin}`). \n",
199350
"\n",
200351
"We can create a string `Template` to perform this for us:"
@@ -424,8 +575,14 @@
424575
"id": "308cd141-5d65-49e9-a81b-c6802418b901",
425576
"metadata": {},
426577
"source": [
427-
"## Writing a `Processor`\n",
428-
"\n",
578+
"### Writing a `Processor`"
579+
]
580+
},
581+
{
582+
"cell_type": "markdown",
583+
"id": "aabc7986",
584+
"metadata": {},
585+
"source": [
429586
"The second-to-last step here is to create the custom `Processor` nbdev utilizes to apply procs (things that modify the contents of cells). The basic understanding of these is simply that you should create a class, have it inherit `Processor`, and any modifications that should be done must be defined in a `cell` function which takes in a `cell` and modifies it in-place."
430587
]
431588
},
@@ -614,8 +771,14 @@
614771
"id": "939edb39-3845-437f-96c7-7ac7465a247d",
615772
"metadata": {},
616773
"source": [
617-
"## How to enable the plugin on your project\n",
618-
"\n",
774+
"### How to enable the plugin on your project"
775+
]
776+
},
777+
{
778+
"cell_type": "markdown",
779+
"id": "9c97a2d3",
780+
"metadata": {},
781+
"source": [
619782
"This requires adding a `doc_procs` key to `[tool.nbdev]` in your `pyproject.toml`.\n",
620783
"\n",
621784
"For example, if this were code that lived in `nbdev`, we can specify where the processor comes from:\n",
@@ -639,90 +802,21 @@
639802
},
640803
{
641804
"cell_type": "markdown",
642-
"id": "477c189a",
805+
"id": "52cad971-b227-4426-880e-c4c1fc09f482",
643806
"metadata": {},
644807
"source": [
645-
"## Export processors\n",
646-
"\n",
647-
"The `Processor` class shown above is for **doc processors** — they transform notebooks when building documentation with `nbdev_docs`.\n",
648-
"\n",
649-
"There's a simpler kind of processor for **export processors** — they transform code cells when exporting to `.py` files with `nbdev_export`. These are just plain functions, not classes.\n",
650-
"\n",
651-
"### Writing an export processor\n",
652-
"\n",
653-
"An export processor is a function that receives a notebook cell and modifies it in place:\n",
654-
"\n",
655-
"```python\n",
656-
"def my_export_proc(cell):\n",
657-
" \"Transform code cells during export\"\n",
658-
" if cell.cell_type != 'code': return\n",
659-
" # Modify cell.source as needed\n",
660-
" cell.source = transform(cell.source)\n",
661-
"```\n",
662-
"\n",
663-
"For example, here's a formatter using `black`:\n",
664-
"\n",
665-
"```python\n",
666-
"# nbdev_black/__init__.py\n",
667-
"def black_format(cell):\n",
668-
" \"Format code cells with black\"\n",
669-
" if cell.cell_type != 'code': return\n",
670-
" try:\n",
671-
" import black\n",
672-
" cell.source = black.format_str(cell.source, mode=black.Mode())\n",
673-
" except: pass # Leave malformed code unchanged\n",
674-
"```\n",
675-
"\n",
676-
"### Enabling export processors\n",
677-
"\n",
678-
"Add `export_procs` to your `pyproject.toml`:\n",
679-
"\n",
680-
"```toml\n",
681-
"[tool.nbdev]\n",
682-
"export_procs = [\"nbdev_black:black_format\"]\n",
683-
"```\n",
684-
"\n",
685-
"You can chain multiple processors — they run in order:\n",
686-
"\n",
687-
"```toml\n",
688-
"[tool.nbdev]\n",
689-
"export_procs = [\n",
690-
" \"nbdev_black:black_format\",\n",
691-
" \"my_project.procs:my_custom_proc\",\n",
692-
"]\n",
693-
"```\n",
694-
"\n",
695-
"### Key differences from doc processors\n",
696-
"\n",
697-
"| | Doc processors | Export processors |\n",
698-
"|---|---|---|\n",
699-
"| **Used by** | `nbdev_docs` | `nbdev_export` |\n",
700-
"| **Interface** | `Processor` class with `cell()` method | Simple function |\n",
701-
"| **Config key** | `doc_procs` | `export_procs` |\n",
702-
"| **Purpose** | Transform notebooks for documentation | Transform code for `.py` files |"
808+
"### Conclusion"
703809
]
704810
},
705811
{
706812
"cell_type": "markdown",
707-
"id": "52cad971-b227-4426-880e-c4c1fc09f482",
813+
"id": "4a2864c2",
708814
"metadata": {},
709815
"source": [
710-
"## Conclusion, nbdev-extensions and a bit about me!\n",
711-
"\n",
712816
"Basically if there's any part of a cell and how it should look either from exporting modules, building documentation, or creating your own special command to perform post-processing it can be done quickly and efficiently with this `Processor` class nbdev provides!\n",
713817
"\n",
714-
"If you're interested in seeing more examples of nbdev-extensions and where you can take it I've (Zachary Mueller) written a library dedicated to it called [nbdev-extensions](https://muellerzr.github.io/nbdev-extensions) where any ideas that may benefit how I approach nbdev I then turn into an extension for the world to use.\n",
715-
"\n",
716-
"Thanks for reading!"
818+
"Zachary Mueller has written a library dedicated to it called [nbdev-extensions](https://muellerzr.github.io/nbdev-extensions) where any ideas that may benefit how I approach nbdev I then turn into an extension for the world to use."
717819
]
718-
},
719-
{
720-
"cell_type": "code",
721-
"execution_count": null,
722-
"id": "8d566a4f-c3ff-4749-97b4-dd1a81c7234a",
723-
"metadata": {},
724-
"outputs": [],
725-
"source": []
726820
}
727821
],
728822
"metadata": {},

0 commit comments

Comments
 (0)