🤖 Set build config via Sphinx ext #2360
Conversation
This is an initial change allowing for restructuring how the config settings are computed and putting that logic into a dedicated Sphinx extension. The idea is that this extension may have multiple callbacks that set configuration for Sphinx based on tags and the environment state. As an example, this in-tree extension implements setting the `is_eol` variable in Jinja2 context very early in Sphinx life cycle. It does this based on inspecting the state of current Git checkout as well as reading a config file listing EOL and supported versions of `ansible-core`. The configuration format is TOML as it's gained a lot of the ecosystem adoption over the past years and its parser made its way into the standard library of Python, while PyYAML remains a third-party dependency. Supersedes ansible#2251.
webknjaz
force-pushed
the
maintenance/extension-driven-build-config
branch
from
8ef3352 to
ab0e151
Compare
oraNod
added
doc builds
| logger.info(str(lookup_err)) | ||
| return | ||
|
|
||
| config.html_context['is_eol'] = _is_eol_build( |
There was a problem hiding this comment.
@oraNod this is what sets that value into the variable defined in conf.py. It basically replaces the old hardcoded value.
| def from_dict(cls, raw_dist: dict[str, list[str]]) -> 'Distribution': | ||
| return cls( | ||
| **{ | ||
| kind.replace('-', '_'): versions |
There was a problem hiding this comment.
The dataclass has to have snake_case attrs so this is doing that conversion. It effectively maps end-of-life in the TOML config to end_of_life.
| 'stable-2.14', | ||
| 'stable-2.13', | ||
| ] | ||
| supported = [ |
There was a problem hiding this comment.
@oraNod did you plan to use this list somehow? I just copied it, but it's not actually used anywhere.
| [distribution.ansible-core] | ||
| end-of-life = [ | ||
| 'stable-2.15', | ||
| 'stable-2.14', | ||
| 'stable-2.13', | ||
| ] | ||
| supported = [ | ||
| 'devel', | ||
| 'stable-2.18', | ||
| 'stable-2.17', | ||
| 'stable-2.16', | ||
| ] |
There was a problem hiding this comment.
@oraNod things here are the same as in the other mapping. Did you expect them to differ at some point? What's the semantic meaning you were going for?
There was a problem hiding this comment.
Hey @webknjaz Sorry for the slow response. I've been focused on prep work for cfgmgmtcamp. My thinking was to use the "supported" list as a way to detect when the core repo creates a new stable branch. We could then get an announcement in matrix that it's time to cut a new stable branch for docs.
Here's a link to my comment where I mentioned this: #1695 (comment)
It was just an idea. And since it's not strictly needed as part of this PR maybe we should do it separately, if at all.
There was a problem hiding this comment.
maybe we should do it separately, if at all
@oraNod yes, it's a good idea to exclude things that are out of the scope right now. However, it's also good to think about the desired data structure overall so it could be designed taking that into account right away.
I'm also confused about this ansible vs. ansible-core thing — both list the same list of EOL entries. What's that about? Did you anticipate these having different values under some circumstances? Any more info?
There was a problem hiding this comment.
@webknjaz Makes sense to narrow the scope to only what is necessary.
For ansible vs. ansible-core there is a period during the release lifecycle where the core version reaches EOL before the Ansible package version. @samccann probably knows those details the best but we do need to mark both of those distributions as EOL at different times.
There was a problem hiding this comment.
@oraNod but ansible has a different versioning scheme. Why does it list the ansible-core branches?
There was a problem hiding this comment.
@webknjaz Because the docs repo uses the same branch names as core.
There was a problem hiding this comment.
I don't fully understand the mapping to ansible then.
There was a problem hiding this comment.
OK, maybe there is also a disconnect in my understanding. I'll try but my explanation could be off so bear with me. It would probably be easier to discuss on a call. That will have to be next week, I'm afraid.
Take the stable-2.18 branch. In the ansible/ansible repo this corresponds to version of 2.18 for core. The ansible/ansible-documentation repo also has a stable-2.18 branch that maps to 2.18 for the core docs build and 11 for the package docs build. It is one branch that maps to two builds that EOL at different times. (I feel like I'm explaining something you already know here. I don't mean to be patronizing.)
Would it be worthwhile to comment the toml so it's clear which branch names map to which ansible versions?
I don't think that is documented anywhere and is kind of tribal knowledge. Part of my goal was indeed to have a central, declarative place where we could clearly see which doc builds are EOL and which are still active.
| DOCSITE_EOL_CONFIG_PATH = DOCSITE_ROOT_DIR / 'end_of_life.toml' | ||
|
|
||
|
|
||
| @dataclass(frozen=True) |
There was a problem hiding this comment.
What's the point of a frozen dataclass if it has mutable attributes?
There was a problem hiding this comment.
It's just a nice default to have.
There was a problem hiding this comment.
Also, the point of immutable data structures is that it's harder to get into a situation where their contents would be changed suddenly by functions where you pass them. Of course, this doesn't shield us from the mutable nested pieces of data, but it's the first step. It might be good to use some marshalling library in the future.
| from dataclasses import dataclass | ||
| from functools import cache | ||
| from pathlib import Path | ||
| from tomllib import loads as parse_toml_string |
There was a problem hiding this comment.
Do we still need to support Python 3.10?
There was a problem hiding this comment.
Could add an extra optional dependency if needed. Though given that 3.11 is hardcoded in the automation, I'd not bother.
This is an initial change allowing for restructuring how the config settings are computed and putting that logic into a dedicated Sphinx extension. The idea is that this extension may have multiple callbacks that set configuration for Sphinx based on tags and the environment state.
As an example, this in-tree extension implements setting the
is_eolvariable in Jinja2 context very early in Sphinx life cycle. It does this based on inspecting the state of current Git checkout as well as reading a config file listing EOL and supported versions ofansible-core.The configuration format is TOML as it's gained a lot of the ecosystem adoption over the past years and its parser made its way into the standard library of Python, while PyYAML remains a third-party dependency.
Supersedes #2251.