Skip to content

Commit 29322b8

Browse files
deploy: 25e50cf
0 parents  commit 29322b8

File tree

174 files changed

+78922
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

174 files changed

+78922
-0
lines changed

.nojekyll

Whitespace-only changes.

CNAME

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
nbdev.fast.ai

api/clean.html

Lines changed: 1054 additions & 0 deletions
Large diffs are not rendered by default.

api/clean.html.md

Lines changed: 231 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,231 @@
1+
# clean
2+
3+
4+
<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->
5+
6+
To avoid pointless conflicts while working with jupyter notebooks (with
7+
different execution counts or cell metadata), it is recommended to clean
8+
the notebooks before committing anything (done automatically if you
9+
install the git hooks with `nbdev-install-hooks`). The following
10+
functions are used to do that. Cleaning also adds cell `id`s if missing
11+
(required by nbformat 4.5+).
12+
13+
## Trust
14+
15+
------------------------------------------------------------------------
16+
17+
<a
18+
href="https://github.com/AnswerDotAI/nbdev/blob/main/nbdev/clean.py#L25"
19+
target="_blank" style="float:right; font-size:smaller">source</a>
20+
21+
### nbdev_trust
22+
23+
``` python
24+
25+
def nbdev_trust(
26+
fname:str=None, # A notebook name or glob to trust
27+
force_all:bool=False, # Also trust notebooks that haven't changed
28+
):
29+
30+
```
31+
32+
*Trust notebooks matching `fname`.*
33+
34+
## Clean
35+
36+
------------------------------------------------------------------------
37+
38+
<a
39+
href="https://github.com/AnswerDotAI/nbdev/blob/main/nbdev/clean.py#L87"
40+
target="_blank" style="float:right; font-size:smaller">source</a>
41+
42+
### clean_nb
43+
44+
``` python
45+
46+
def clean_nb(
47+
nb, # The notebook to clean
48+
clear_all:bool=False, # Remove all cell metadata and cell outputs?
49+
allowed_metadata_keys:list=None, # Preserve the list of keys in the main notebook metadata
50+
allowed_cell_metadata_keys:list=None, # Preserve the list of keys in cell level metadata
51+
clean_ids:bool=True, # Remove ids from plaintext reprs?
52+
):
53+
54+
```
55+
56+
*Clean `nb` from superfluous metadata*
57+
58+
Jupyter adds a trailing <code></code> to images in cell outputs.
59+
Vscode-jupyter does not.
60+
Notebooks should be brought to a common style to avoid unnecessary
61+
diffs:
62+
63+
``` python
64+
test_nb = read_nb('../../tests/image.ipynb')
65+
assert test_nb.cells[0].outputs[0].data['image/png'][-1] == "\n" # Make sure it was not converted by acccident
66+
clean_nb(test_nb)
67+
assert test_nb.cells[0].outputs[0].data['image/png'][-1] != "\n"
68+
```
69+
70+
The test notebook has metadata in both the main metadata section and
71+
contains cell level metadata in the second cell:
72+
73+
``` python
74+
test_nb = read_nb('../../tests/metadata.ipynb')
75+
76+
assert {'meta', 'jekyll', 'my_extra_key', 'my_removed_key'} <= test_nb.metadata.keys()
77+
assert {'meta', 'hide_input', 'my_extra_cell_key', 'my_removed_cell_key'} == test_nb.cells[1].metadata.keys()
78+
```
79+
80+
After cleaning the notebook, all extra metadata is removed, only some
81+
keys are allowed by default:
82+
83+
``` python
84+
clean_nb(test_nb)
85+
86+
assert {'jekyll', 'kernelspec'} == test_nb.metadata.keys()
87+
assert {'hide_input'} == test_nb.cells[1].metadata.keys()
88+
```
89+
90+
We can preserve some additional keys at the notebook or cell levels:
91+
92+
``` python
93+
test_nb = read_nb('../../tests/metadata.ipynb')
94+
clean_nb(test_nb, allowed_metadata_keys={'my_extra_key'}, allowed_cell_metadata_keys={'my_extra_cell_key'})
95+
96+
assert {'jekyll', 'kernelspec', 'my_extra_key'} == test_nb.metadata.keys()
97+
assert {'hide_input', 'my_extra_cell_key'} == test_nb.cells[1].metadata.keys()
98+
```
99+
100+
Passing `clear_all=True` removes everything from the cell metadata:
101+
102+
``` python
103+
test_nb = read_nb('../../tests/metadata.ipynb')
104+
clean_nb(test_nb, clear_all=True)
105+
106+
assert {'jekyll', 'kernelspec'} == test_nb.metadata.keys()
107+
test_eq(test_nb.cells[1].metadata, {})
108+
```
109+
110+
Passing `clean_ids=True` removes `id`s from plaintext repr outputs, to
111+
avoid notebooks whose contents change on each run since they often lead
112+
to git merge conflicts. For example:
113+
114+
<PIL.PngImagePlugin.PngImageFile image mode=L size=28x28 at 0x7FB4F8979690>
115+
116+
becomes:
117+
118+
<PIL.PngImagePlugin.PngImageFile image mode=L size=28x28>
119+
120+
*Cell* IDs, on the other hand, are always added if missing
121+
122+
``` python
123+
test_cell = {'source': 'x=1', 'cell_type': 'code', 'metadata': {}}
124+
_clean_cell(test_cell, False, set(), True)
125+
test_cell['id']
126+
```
127+
128+
'b3c0571e'
129+
130+
------------------------------------------------------------------------
131+
132+
<a
133+
href="https://github.com/AnswerDotAI/nbdev/blob/main/nbdev/clean.py#L112"
134+
target="_blank" style="float:right; font-size:smaller">source</a>
135+
136+
### process_write
137+
138+
``` python
139+
140+
def process_write(
141+
warn_msg, proc_nb, f_in, f_out:NoneType=None, disp:bool=False
142+
):
143+
144+
```
145+
146+
------------------------------------------------------------------------
147+
148+
<a
149+
href="https://github.com/AnswerDotAI/nbdev/blob/main/nbdev/clean.py#L135"
150+
target="_blank" style="float:right; font-size:smaller">source</a>
151+
152+
### nbdev_clean
153+
154+
``` python
155+
156+
def nbdev_clean(
157+
fname:str=None, # A notebook name or glob to clean
158+
clear_all:bool=False, # Remove all cell metadata and cell outputs?
159+
disp:bool=False, # Print the cleaned outputs
160+
stdin:bool=False, # Read notebook from input stream
161+
):
162+
163+
```
164+
165+
*Clean all notebooks in `fname` to avoid merge conflicts*
166+
167+
By default (`fname` left to `None`), all the notebooks in
168+
`config.nbs_path` are cleaned. You can opt in to fully clean the
169+
notebook by removing every bit of metadata and the cell outputs by
170+
passing `clear_all=True`.
171+
172+
If you want to keep some keys in the main notebook metadata you can set
173+
`allowed_metadata_keys` in `[tool.nbdev]` in `pyproject.toml`. Similarly
174+
for cell level metadata use: `allowed_cell_metadata_keys`. For example,
175+
to preserve both `k1` and `k2` at both the notebook and cell level add
176+
the following to `pyproject.toml`:
177+
178+
``` toml
179+
[tool.nbdev]
180+
allowed_metadata_keys = ["k1", "k2"]
181+
allowed_cell_metadata_keys = ["k1", "k2"]
182+
```
183+
184+
------------------------------------------------------------------------
185+
186+
<a
187+
href="https://github.com/AnswerDotAI/nbdev/blob/main/nbdev/clean.py#L150"
188+
target="_blank" style="float:right; font-size:smaller">source</a>
189+
190+
### clean_jupyter
191+
192+
``` python
193+
194+
def clean_jupyter(
195+
path, model, kwargs:VAR_KEYWORD
196+
):
197+
198+
```
199+
200+
*Clean Jupyter `model` pre save to `path`*
201+
202+
This cleans notebooks on-save to avoid unnecessary merge conflicts. The
203+
easiest way to install it for both Jupyter Notebook and Lab is by
204+
running `nbdev-install-hooks`. It works by implementing a
205+
`pre_save_hook` from Jupyter’s [file save hook
206+
API](https://jupyter-server.readthedocs.io/en/latest/developers/savehooks.html).
207+
208+
## Hooks
209+
210+
------------------------------------------------------------------------
211+
212+
<a
213+
href="https://github.com/AnswerDotAI/nbdev/blob/main/nbdev/clean.py#L191"
214+
target="_blank" style="float:right; font-size:smaller">source</a>
215+
216+
### nbdev_install_hooks
217+
218+
``` python
219+
220+
def nbdev_install_hooks(
221+
222+
):
223+
224+
```
225+
226+
*Install Jupyter and git hooks to automatically clean, trust, and fix
227+
merge conflicts in notebooks*
228+
229+
See
230+
[`clean_jupyter`](https://nbdev.fast.ai/api/clean.html#clean_jupyter)
231+
and `nbdev-merge` for more about how each hook works.

0 commit comments

Comments
 (0)