Skip to content

Commit fcf78ca

Browse files
AA-Turnerchrisjsewell
AA-Turner
and
chrisjsewell
authored
👌 Improve generation of meta nodes (#1080)
This PR updates how HTML meta nodes are generated in MyST-Parser to ensure compatibility with Sphinx v9. ## Changes Made ### `myst_parser/mdit_to_docutils/base.py` 1. **Removed import**: `from docutils.transforms.components import Filter` 2. **Changed return type** of `html_meta_to_nodes()`: - Before: `list[nodes.pending | nodes.system_message]` - After: `list[nodes.meta | nodes.system_message]` 3. **Simplified node generation**: Instead of wrapping meta nodes in `nodes.pending` with a `Filter` transform, meta nodes are now directly returned ### Test Fixtures Updated - `tests/test_renderers/fixtures/docutil_syntax_elements.md` - `tests/test_renderers/fixtures/sphinx_syntax_elements.md` Expected output changed from `<pending>` nodes with internal transform details to direct `<meta>` nodes. ## Assessment: ✅ Correct and Necessary ### Why This Change is Needed 1. **Modern docutils/Sphinx behavior**: In docutils 0.18+ (2021), `nodes.meta` became a standard node type that writers handle directly 2. **Sphinx v9 compatibility**: The old approach using `Filter` transform with pending nodes is deprecated/removed in newer Sphinx versions 3. **Alignment with standards**: This matches how Sphinx's own meta directive works in modern versions ### Why It Works - Writers that support HTML output process `nodes.meta` directly and render them as `<meta>` tags in the HTML `<head>` section - The change removes an unnecessary layer of indirection (pending nodes) - Uses the standard docutils `nodes.meta` approach ## Test Coverage | Test Type | File | Coverage | |-----------|------|----------| | Docutils renderer | `tests/test_renderers/fixtures/docutil_syntax_elements.md` | "Front Matter HTML Meta" case | | Sphinx renderer | `tests/test_renderers/fixtures/sphinx_syntax_elements.md` | Same test case | | CLI/config | `tests/test_renderers/fixtures/myst-config.txt` | `--myst-html-meta` option | | Sphinx HTML build | `tests/test_sphinx/test_sphinx_builds.py::test_extended_syntaxes` | Meta nodes in HTML builds | | Sphinx text build | `tests/test_sphinx/test_sphinx_builds.py::test_extended_syntaxes_text` | Meta nodes with non-HTML builder | ### Non-HTML Builder Coverage The added `test_extended_syntaxes_text` test uses the `text` builder with the `extended_syntaxes` source directory, which has `myst_html_meta` configured. This ensures that meta nodes don't cause issues for non-HTML builders—they simply don't render meta tags (as expected, since `<meta>` is an HTML-specific concept). --------- Co-authored-by: Chris Sewell <chrisj_sewell@hotmail.com>
1 parent e0fc7a3 commit fcf78ca

File tree

5 files changed

+103
-74
lines changed

5 files changed

+103
-74
lines changed

‎myst_parser/mdit_to_docutils/base.py‎

Lines changed: 3 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,6 @@
2828
from docutils.parsers.rst.directives.misc import Include
2929
from docutils.parsers.rst.languages import get_language as get_language_rst
3030
from docutils.statemachine import StringList
31-
from docutils.transforms.components import Filter
3231
from docutils.utils import Reporter, SystemMessage, new_document
3332
from docutils.utils.code_analyzer import Lexer, LexerError, NumberLines
3433
from markdown_it import MarkdownIt
@@ -1883,7 +1882,7 @@ def render_substitution(self, token: SyntaxTreeNode, inline: bool) -> None:
18831882

18841883
def html_meta_to_nodes(
18851884
data: dict[str, Any], document: nodes.document, line: int, reporter: Reporter
1886-
) -> list[nodes.pending | nodes.system_message]:
1885+
) -> list[nodes.meta | nodes.system_message]:
18871886
"""Replicate the `meta` directive,
18881887
by converting a dictionary to a list of pending meta nodes
18891888
@@ -1917,14 +1916,8 @@ def html_meta_to_nodes(
19171916
except ValueError as error:
19181917
msg = reporter.error(f'Error parsing meta tag attribute "{key}": {error}.')
19191918
output.append(msg)
1920-
continue
1921-
1922-
pending = nodes.pending(
1923-
Filter,
1924-
{"component": "writer", "format": "html", "nodes": [meta_node]},
1925-
)
1926-
document.note_pending(pending)
1927-
output.append(pending)
1919+
else:
1920+
output.append(meta_node)
19281921

19291922
return output
19301923

‎tests/test_renderers/fixtures/docutil_syntax_elements.md‎

Lines changed: 4 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -672,38 +672,10 @@ myst:
672672
---
673673
.
674674
<document source="notset">
675-
<pending>
676-
.. internal attributes:
677-
.transform: docutils.transforms.components.Filter
678-
.details:
679-
component: 'writer'
680-
format: 'html'
681-
nodes:
682-
<meta content="Sphinx, documentation, builder" name="keywords">
683-
<pending>
684-
.. internal attributes:
685-
.transform: docutils.transforms.components.Filter
686-
.details:
687-
component: 'writer'
688-
format: 'html'
689-
nodes:
690-
<meta content="An amusing story" lang="en" name="description">
691-
<pending>
692-
.. internal attributes:
693-
.transform: docutils.transforms.components.Filter
694-
.details:
695-
component: 'writer'
696-
format: 'html'
697-
nodes:
698-
<meta content="Un histoire amusant" lang="fr" name="description">
699-
<pending>
700-
.. internal attributes:
701-
.transform: docutils.transforms.components.Filter
702-
.details:
703-
component: 'writer'
704-
format: 'html'
705-
nodes:
706-
<meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type">
675+
<meta content="Sphinx, documentation, builder" name="keywords">
676+
<meta content="An amusing story" lang="en" name="description">
677+
<meta content="Un histoire amusant" lang="fr" name="description">
678+
<meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type">
707679
.
708680

709681
Full Test:

‎tests/test_renderers/fixtures/sphinx_syntax_elements.md‎

Lines changed: 4 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -674,38 +674,10 @@ myst:
674674
---
675675
.
676676
<document source="<src>/index.md">
677-
<pending>
678-
.. internal attributes:
679-
.transform: docutils.transforms.components.Filter
680-
.details:
681-
component: 'writer'
682-
format: 'html'
683-
nodes:
684-
<meta content="Sphinx, documentation, builder" name="keywords">
685-
<pending>
686-
.. internal attributes:
687-
.transform: docutils.transforms.components.Filter
688-
.details:
689-
component: 'writer'
690-
format: 'html'
691-
nodes:
692-
<meta content="An amusing story" lang="en" name="description">
693-
<pending>
694-
.. internal attributes:
695-
.transform: docutils.transforms.components.Filter
696-
.details:
697-
component: 'writer'
698-
format: 'html'
699-
nodes:
700-
<meta content="Un histoire amusant" lang="fr" name="description">
701-
<pending>
702-
.. internal attributes:
703-
.transform: docutils.transforms.components.Filter
704-
.details:
705-
component: 'writer'
706-
format: 'html'
707-
nodes:
708-
<meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type">
677+
<meta content="Sphinx, documentation, builder" name="keywords">
678+
<meta content="An amusing story" lang="en" name="description">
679+
<meta content="Un histoire amusant" lang="fr" name="description">
680+
<meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type">
709681
.
710682

711683
Full Test:

‎tests/test_sphinx/test_sphinx_builds.py‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -242,6 +242,35 @@ def test_extended_syntaxes(
242242
)
243243

244244

245+
@pytest.mark.sphinx(
246+
buildername="text",
247+
srcdir=os.path.join(SOURCE_DIR, "extended_syntaxes"),
248+
freshenv=True,
249+
)
250+
def test_extended_syntaxes_text(
251+
app,
252+
status,
253+
warning,
254+
get_sphinx_app_output,
255+
monkeypatch,
256+
file_regression,
257+
):
258+
"""test setting addition configuration values."""
259+
from myst_parser.mdit_to_docutils.sphinx_ import SphinxRenderer
260+
261+
monkeypatch.setattr(SphinxRenderer, "_random_label", lambda self: "mock-uuid")
262+
app.build()
263+
assert "build succeeded" in status.getvalue() # Build succeeded
264+
warnings = warning.getvalue().strip()
265+
assert warnings == ""
266+
content = get_sphinx_app_output(
267+
app,
268+
buildername="text",
269+
filename="index.txt",
270+
)
271+
file_regression.check(content)
272+
273+
245274
@pytest.mark.sphinx(
246275
buildername="html", srcdir=os.path.join(SOURCE_DIR, "includes"), freshenv=True
247276
)

‎tests/test_sphinx/test_sphinx_builds/test_extended_syntaxes_text.txt‎

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
Test
2+
****
3+
4+
*disabled*
5+
6+
a=1
7+
8+
x=5
9+
10+
x=5
11+
12+
$ a=1 $
13+
14+
a
15+
16+
c=3
17+
18+
b
19+
20+
\begin{equation} b=2 \end{equation}
21+
22+
c=3 d=4
23+
24+
Term **1**
25+
Definition *1*
26+
27+
second paragraph
28+
29+
Term 2
30+
Definition 2a
31+
32+
Definition 2b
33+
34+
Term 3
35+
code block
36+
37+
quote
38+
39+
other
40+
41+
term
42+
definition
43+
44+
other term
45+
other definition
46+
47+
*other term*
48+
49+
[image: fun-fish][image]This is a caption in **Markdown**
50+
51+
[image: fishy][image]This is a caption in **Markdown**
52+
53+
Hallo *there*
54+
55+
linkify URL: www.example.com
56+
57+
* hallo
58+
59+
* there
60+
61+
Numbered code block:
62+
63+
type Result = "pass" | "fail"

0 commit comments

Comments
 (0)