fix docs build and docs around standards, towncrier, etc.

.github/workflows/ci.yml

@@ -70,6 +70,10 @@ jobs:
           poetry-version: "1.8.5"
       - name: "Checking: poetry lock file"
         run: "poetry lock --check"
+    needs:
+      - "ruff-format"
+      - "ruff-lint"
+      - "yamllint"
   yamllint:
     runs-on: "ubuntu-24.04"
     env:
@@ -83,6 +87,9 @@ jobs:
           poetry-version: "1.8.5"
       - name: "Linting: yamllint"
         run: "poetry run invoke yamllint"
+    needs:
+      - "ruff-format"
+      - "ruff-lint"
   pylint:
     runs-on: "ubuntu-24.04"
     strategy:
@@ -118,6 +125,8 @@ jobs:
         run: "docker image ls"
       - name: "Linting: Pylint"
         run: "poetry run invoke pylint"
+    needs:
+      - "poetry"
   pytest:
     strategy:
       fail-fast: true
@@ -153,6 +162,8 @@ jobs:
         run: "docker image ls"
       - name: "Run Tests"
         run: "poetry run invoke pytest"
+    needs:
+      - "poetry"
   publish_gh:
     name: "Publish to GitHub"
     runs-on: "ubuntu-24.04"

docs/admin/install.md

@@ -2,21 +2,21 @@
 
 Option 1: Install from PyPI.
 
-```
+```bash
 pip install pyntc
 ```
 
 Option 2: Manually install via Poetry.
 
-```
+```bash
 git clone https://github.com/networktocode/pyntc.git
 cd pyntc
-pip install poetry
+curl -sSL https://install.python-poetry.org | python3 -
 poetry install
 ```
 
 Option 3: Install from a GitHub branch, such as develop as shown below.
 
 ```bash
-$ pip install git+https://github.com/networktocode/pyntc.git@develop
+pip install git+https://github.com/networktocode/pyntc.git@develop
 ```

docs/admin/uninstall.md

@@ -3,5 +3,5 @@
 Uninstall from environment.
 
 ```bash
-$ pip uninstall pyntc
+pip uninstall pyntc
 ```

docs/admin/upgrade.md

@@ -3,5 +3,5 @@
 Upgrade from PyPI.
 
 ```bash
-$ pip install pyntc --upgrade
+pip install pyntc --upgrade
 ```

docs/dev/contributing.md

@@ -4,15 +4,44 @@ Pull requests are welcomed and automatically built and tested against multiple v
 
 Except for unit tests, testing is only supported on Python 3.9.
 
-The project is packaged with a light development environment based on `Docker` to help with the local development of the project and to run tests within  GitHub Actions.
+The project is packaged with a light development environment based on `Docker` to help with the local development of the project and to run tests within GitHub Actions.
 
-The project is following Network to Code software development guidelines and are leveraging the following:
+The project is following Network to Code software development guidelines and is leveraging the following:
 
-- Black, Pylint, Bandit, flake8, and pydocstyle for Python linting and formatting.
-- pytest, coverage, and unittest for unit tests.
+- Python linting and formatting: `pylint` and `ruff`.
+- YAML linting is done with `yamllint`.
 
 Documentation is built using [mkdocs](https://www.mkdocs.org/). The [Docker based development environment](dev_environment.md#docker-development-environment) can be started by running `invoke docs` [http://localhost:8001](http://localhost:8001) that auto-refreshes when you make any changes to your local files.
 
+## Creating Changelog Fragments
+
+All pull requests to `next` or `develop` must include a changelog fragment file in the `./changes` directory. To create a fragment, use your GitHub issue number and fragment type as the filename. For example, `2362.added`. Valid fragment types are `added`, `changed`, `deprecated`, `fixed`, `removed`, and `security`. The change summary is added to the file in plain text. Change summaries should be complete sentences, starting with a capital letter and ending with a period, and be in past tense. Each line of the change fragment will generate a single change entry in the release notes. Use multiple lines in the same file if your change needs to generate multiple release notes in the same category. If the change needs to create multiple entries in separate categories, create multiple files.
+
+!!! example
+
+    **Wrong**
+    ```plaintext title="changes/1234.fixed"
+    fix critical bug in documentation
+    ```
+
+    **Right**
+    ```plaintext title="changes/1234.fixed"
+    Fixed critical bug in documentation.
+    ```
+
+!!! example "Multiple Entry Example"
+
+    This will generate 2 entries in the `fixed` category and one entry in the `changed` category.
+
+    ```plaintext title="changes/1234.fixed"
+    Fixed critical bug in documentation.
+    Fixed release notes generation.
+    ```
+
+    ```plaintext title="changes/1234.changed"
+    Changed release notes generation.
+    ```
+
 ## Branching Policy
 
 The branching policy includes the following tenets:
@@ -45,4 +74,4 @@ When a new release is created the following should happen.
 - A post release PR is created with.
     - Change the version from `<major>.<minor>.<patch>` to `<major>.<minor>.<patch + 1>-beta` pyproject.toml.
     - Set the PR to the `develop`.
-    - Once tests pass, merge.
+    - Once tests pass, merge.

docs/dev/dev_environment.md

@@ -32,7 +32,6 @@ Once you have Poetry and Docker installed you can run the following commands (in
 ```shell
 poetry shell
 poetry install
-cp development/creds.example.env development/creds.env
 invoke build
 invoke start
 ```
@@ -51,7 +50,7 @@ Poetry is used in lieu of the "virtualenv" commands and is leveraged in both env
 The `pyproject.toml` file outlines all of the relevant dependencies for the project:
 
 - `tool.poetry.dependencies` - the main list of dependencies.
-- `tool.poetry.dev-dependencies` - development dependencies, to facilitate linting, testing, and documentation building.
+- `tool.poetry.group.dev.dependencies` - development dependencies, to facilitate linting, testing, and documentation building.
 
 The `poetry shell` command is used to create and enable a virtual environment managed by Poetry, so all commands ran going forward are executed within the virtual environment. This is similar to running the `source venv/bin/activate` command with virtualenvs. To install project dependencies in the virtual environment, you should run `poetry install` - this will install **both** project and development dependencies.
 
@@ -82,21 +81,18 @@ Each command can be executed with `invoke <command>`. Each command also has its
 ### Utility
 
 ```
-  cli                Enter the image to perform troubleshooting or dev work.
-  clean-container    Remove stopped containers that source for image `pyntc:`
+  cli                       Enter the image to perform troubleshooting or dev work.
+  clean                     Remove stopped containers that source for image `pyntc:`
+  generate-release-notes    Generate Release Notes using Towncrier.
 ```
 
 ### Testing
 
 ```
-  bandit             Run bandit to validate basic static code security analysis.
-  black              Run black to check that Python files adhere to its style standards.
-  coverage           Run the coverage report against pytest.
-  flake8             Run flake8 to check that Python files adhere to its style standards.
-  mypy               Run mypy to validate typing-hints.
-  pylint             Run pylint code analysis.
-  pydocstyle         Run pydocstyle to validate docstring formatting adheres to NTC defined standards.
-  pytest             Run pytest for the specified name and Python version.
-  tests              Run all tests for the specified name and Python version.
-  yamllint           Run yamllint to validate formatting adheres to NTC defined YAML standards.
+  autoformat (a)    Run code autoformatting.
+  pylint            Run pylint for the specified name and Python version.
+  ruff              Run ruff to perform code formatting and/or linting.
+  pytest            Run pytest for the specified name and Python version.
+  tests             Run all tests for the specified name and Python version.
+  yamllint          Run yamllint to validate formatting adheres to NTC defined YAML standards.
 ```

docs/generate_code_reference_pages.py

@@ -0,0 +1,20 @@
+"""Generate code reference pages."""
+
+from pathlib import Path
+
+import mkdocs_gen_files
+
+for file_path in Path("pyntc").rglob("*.py"):
+    module_path = file_path.with_suffix("")
+    doc_path = file_path.with_suffix(".md")
+    full_doc_path = Path("code-reference", doc_path)
+
+    parts = list(module_path.parts)
+    if parts[-1] == "__init__":
+        parts = parts[:-1]
+
+    with mkdocs_gen_files.open(full_doc_path, "w") as fd:
+        IDENTIFIER = ".".join(parts)
+        print(f"::: {IDENTIFIER}", file=fd)
+
+    mkdocs_gen_files.set_edit_path(full_doc_path, file_path)

docs/requirements.txt

@@ -1,9 +1,13 @@
-mkdocs==1.6.0
-# Material for MkDocs theme
-mkdocs-material==9.5.32
-# Render custom markdown for version added/changed/remove notes
-mkdocs-version-annotations==1.0.0
-# Automatic documentation from sources, for MkDocs
-mkdocstrings==0.25.2
-mkdocstrings-python==1.10.8
-griffe==1.1.1
+mkdocs==1.6.1
+markdown-data-tables==1.0.0
+markdown-version-annotations==1.0.1
+mkdocs-gen-files==0.5.0
+mkdocs-glightbox==0.4.0
+mkdocs-macros-plugin==1.3.7
+mkdocs-material==9.6.15
+mkdocs-redirects==1.2.2
+mkdocs-section-index==0.3.10
+mkdocs-redirects==1.2.2
+mkdocs-section-index==0.3.10
+mkdocstrings==0.27.0
+mkdocstrings-python==1.13.0

mkdocs.yml

@@ -14,13 +14,16 @@ theme:
     - "python"
     - "yaml"
   features:
-    - "navigation.tracking"
+    - "content.code.annotate"
+    - "content.code.copy"
+    - "content.tabs.link"
+    - "navigation.footer"
     - "navigation.tabs"
     - "navigation.tabs.sticky"
-    - "search.suggest"
+    - "navigation.tracking"
     - "search.highlight"
     - "search.share"
-    - "navigation.indexes"
+    - "search.suggest"
   favicon: "assets/favicon.ico"
   logo: "assets/networktocode_logo.svg"
   palette:
@@ -39,15 +42,16 @@ theme:
       toggle:
         icon: "material/weather-night"
         name: "Switch to light mode"
+
+validation:
+  absolute_links: "warn"
+  anchors: "warn"
+  omitted_files: "warn"
+  unrecognized_links: "warn"
+
 extra_css:
   - "assets/extra.css"
 
-# needed for RTD version flyout menu
-# jquery is not (yet) injected by RTD automatically and it might be dropped
-# as a dependency in the future
-extra_javascript:
-  - "https://code.jquery.com/jquery-3.6.0.min.js"
-
 extra:
   generator: false
   ntc_sponsor: true
@@ -68,29 +72,51 @@ extra:
       link: "https://twitter.com/networktocode"
       name: "Network to Code Twitter"
 markdown_extensions:
+  - "markdown_version_annotations":
+      admonition_tag: "???"
   - "admonition"
   - "toc":
       permalink: true
   - "attr_list"
+  - "markdown_data_tables":
+      base_path: "docs"
   - "md_in_html"
+  - "pymdownx.details"
+  # Need pymdownx.emoji for Grid icon search
+  - pymdownx.emoji:
+      emoji_index: !!python/name:material.extensions.emoji.twemoji
+      emoji_generator: !!python/name:material.extensions.emoji.to_svg
   - "pymdownx.highlight":
       anchor_linenums: true
   - "pymdownx.inlinehilite"
   - "pymdownx.snippets"
-  - "pymdownx.superfences"
-  - "footnotes"
+  - "pymdownx.superfences":
+      custom_fences:
+        - name: "mermaid"
+          class: "mermaid"
+          format: !!python/name:pymdownx.superfences.fence_code_format
+  - "pymdownx.tabbed":
+      "alternate_style": true
+  - "pymdownx.tilde"
+
 plugins:
   - "search"
-  - "mkdocs-version-annotations"
+  - "gen-files":
+      scripts:
+        - "docs/generate_code_reference_pages.py"
+  - "glightbox":
+      manual: true # See https://blueswen.github.io/mkdocs-glightbox/flexibility/enable-by-image-or-page/
+  - "section-index"
   - "mkdocstrings":
       default_handler: "python"
       handlers:
         python:
           paths: ["."]
           options:
+            heading_level: 1
             show_root_heading: true
-watch:
-  - "README.md"
+            show_root_members_full_path: true
+            show_source: false
 
 nav:
   - Overview: "index.md"
@@ -104,17 +130,43 @@ nav:
       - Uninstall: "admin/uninstall.md"
       - Release Notes:
           - "admin/release_notes/index.md"
-          - v0.0: "admin/release_notes/version_0_0.md"
-          - v0.14: "admin/release_notes/version_0_14.md"
-          - v0.15: "admin/release_notes/version_0_15.md"
-          - v0.16: "admin/release_notes/version_0_16.md"
-          - v0.17: "admin/release_notes/version_0_17.md"
-          - v0.18: "admin/release_notes/version_0_18.md"
-          - v0.19: "admin/release_notes/version_0_19.md"
-          - v0.20: "admin/release_notes/version_0_20.md"
-          - v1.0: "admin/release_notes/version_1_0.md"
-          - v2.0: "admin/release_notes/version_2_0.md"
+          - v0.0: "admin/release_notes/version_0.0.md"
+          - v0.14: "admin/release_notes/version_0.14.md"
+          - v0.15: "admin/release_notes/version_0.15.md"
+          - v0.16: "admin/release_notes/version_0.16.md"
+          - v0.17: "admin/release_notes/version_0.17.md"
+          - v0.18: "admin/release_notes/version_0.18.md"
+          - v0.19: "admin/release_notes/version_0.19.md"
+          - v0.20: "admin/release_notes/version_0.20.md"
+          - v1.0: "admin/release_notes/version_1.0.md"
+          - v2.0: "admin/release_notes/version_2.0.md"
   - Developer Guide:
       - Extending the Library: "dev/extending.md"
       - Contributing to the Library: "dev/contributing.md"
       - Development Environment: "dev/dev_environment.md"
+      - Code Reference:
+          - pyntc: "code-reference/pyntc/__init__.md"
+          - pyntc.errors: "code-reference/pyntc/errors.md"
+          - pyntc.log: "code-reference/pyntc/log.md"
+          - pyntc.devices: "code-reference/pyntc/devices/__init__.md"
+          - pyntc.devices.aireos_device: "code-reference/pyntc/devices/aireos_device.md"
+          - pyntc.devices.asa_device: "code-reference/pyntc/devices/asa_device.md"
+          - pyntc.devices.base_device: "code-reference/pyntc/devices/base_device.md"
+          - pyntc.devices.eos_device: "code-reference/pyntc/devices/eos_device.md"
+          - pyntc.devices.f5_device: "code-reference/pyntc/devices/f5_device.md"
+          - pyntc.devices.ios_device: "code-reference/pyntc/devices/ios_device.md"
+          - pyntc.devices.iosxewlc_device: "code-reference/pyntc/devices/iosxewlc_device.md"
+          - pyntc.devices.jnpr_device: "code-reference/pyntc/devices/jnpr_device.md"
+          - pyntc.devices.nxos_device: "code-reference/pyntc/devices/nxos_device.md"
+          - pyntc.devices.system_features: "code-reference/pyntc/devices/system_features/__init__.md"
+          - pyntc.devices.system_features.base_feature: "code-reference/pyntc/devices/system_features/base_feature.md"
+          - pyntc.devices.system_features.vlans: "code-reference/pyntc/devices/system_features/vlans/__init__.md"
+          - pyntc.devices.system_features.vlans.base_vlans: "code-reference/pyntc/devices/system_features/vlans/base_vlans.md"
+          - pyntc.devices.system_features.vlans.eos_vlans: "code-reference/pyntc/devices/system_features/vlans/eos_vlans.md"
+          - pyntc.devices.tables: "code-reference/pyntc/devices/tables/__init__.md"
+          - pyntc.devices.juniper: "code-reference/pyntc/devices/tables/jnpr/__init__.md"
+          - pyntc.devices.juniper.loopback: "code-reference/pyntc/devices/tables/jnpr/loopback.md"
+          - pyntc.utils: "code-reference/pyntc/utils/__init__.md"
+          - pyntc.utils.converters: "code-reference/pyntc/utils/converters.md"
+          - pyntc.utils.templates: "code-reference/pyntc/utils/templates/__init__.md"
+          - pyntc.arch_decision: "dev/arch_decision.md"