Repository navigation
Sub-index pages for specific categories of PEPs (Packaging, etc) #2572
Description
See https://discuss.python.org/t/14111/, where this was discussed initially and where folks have been generally in favour of this.
It would be valuable to be able to list PEPs related to Python Packaging, in a single location in addition to having them in the PEP 0 index. While packaging.python.org could have such a list (and it does, to an extent), that has an additional maintainance burden (it needs to be updated each time a new PEP is proposed). It would be more beneficial to be able to provide such a listing under peps.python.org. This has discoverability advantages, will be easier to maintain (since the PEP sources themselves will note that it's a Packaging-related PEP) and avoids needing to update two spots when a new Packaging-related PEP is proposed.
Activity
pradyunsg commentedon May 3, 2022
A concrete proposal for implementing this, roughly based on the concensus in the aforementioned thread, is:
Trackheader for PEPs, that can have only one value for the initial implementation:Packaging.peps.python.org/track/packagingThe Track header can be expanded to other values in the future (
TypingandReleaseare examples of proposed ideas from the aforementioned thread).Personally, I think it's valuable to implement this for just Packaging as an initial pass since it's a viable "minimum" implementation of this mechanism and... perhaps more importantly... avoids an extended discussion on what these other values should be allowed/updated for. :)
AA-Turner commentedon May 3, 2022
I'm supportive of the idea.
Implementation-wise it wouldn't be challenging & happy to pick it up if we agree.
I'd suggest using the index namespace, as it's recognised that PEP 0 is the index, and this is in effect a 'sub-index' -- so
peps.python.org/pep-0000/packaging.A
brettcannon commentedon May 3, 2022
I support this.
I actually don't know if most people realize there is a PEP 0 and that it's the index.
pradyunsg commentedon May 3, 2022
I don't have strong feelings on the exact URL, so whatever PEP editors prefer works for me!
AA-Turner commentedon May 3, 2022
Fair enough -- I suppose I'm approaching the issue somewhat with an implementation bias, too.
But if that's the worst thing we can argue about... 😉
A
hugovk commentedon May 3, 2022
I also support this!
peps.python.org/pep-0000/packaginglooks a little odd as well, something likepeps.python.org/track/packagingfeels a lot more intuitive.pradyunsg commentedon May 3, 2022
Looking at the implementation generating the index page, currently, PEP 0 is generated in
pep_sphinx_extensions/by adding a hook that generates the entire file in-memory. This file is then populated with the relevant content, as generated inPEPIndexWriter, written and then added to the Sphinx's various internal lists to ensure that it builds it appropriately.It would be feasible to have
pep-0000.rstbe a regular file that's checked into the repository, if it used a custom reST directive that populates the index part of the file. The header and introduction can then be kept in the source, as prose, outside of the Python code (they're in awriter.pyfile currently).This could make having a separate index easier, since the parts that need to differ (header and introduction) can live outside the Python generator, which would also (possibly?) simplify to "only" have a directive for the index generation -- i.e. hook into Sphinx's build system to a lesser extent.
I'm not super familiar with the Sphinx-based PEP generation code though and what constraints it was written with, so if this is not feasible for some reason, I apologise for my ignorance. 😅
PS:
PEPIndexWriterdoesn't actually write the file. It might be a good idea to rename it toPEPIndexGeneratoror something. :)JelleZijlstra commentedon May 3, 2022
Another bikeshed :)
"Track" may not be the best term, because it's really only applicable to packaging, where PEPs go through a very different process from other PEPs. Typing PEPs are on the same "track" as regular PEPs, because they get discussed on python-dev and approved by the SC just like any other PEP. Alternative suggestion: "Area". Then we could have headers like "Area: Packaging", "Area: Typing", "Area: Release", and automatically generate pages for each of these areas.
For now we'd start with just "Area: Packaging", and if the community is interested in a new Area, we could add it later.
hugovk commentedon May 3, 2022
There is a bit of overlap in naming here:
The PEP Type can be Standards Track, Informational, or Process.
https://peps.python.org/pep-0001/#pep-types
Track is only applied to one of the types, but track does also feel the right name for this new field. Maybe it doesn't matter?
AA-Turner commentedon May 3, 2022
Topic?Areaalso seems reasonable.We could also be very boring and have
Index:.A
[-]Providing an additional index page for Python Packaging PEPs[/-][+]Additional index pages[/+]AA-Turner commentedon May 3, 2022
I re-titled the issue as although packaging will be first, the design needs to be sufficiently general for other potential sub-indicies.
A
Rosuav commentedon May 3, 2022
+1 on having subindexes, not overly fussed about keyword. I like "index", honestly, but "track" is fine too.
pradyunsg commentedon May 3, 2022
As long as we don't expand the scope for the first implementation beyond packaging, that works for me.
I do notice that there's two aspects (header name and URL) and we've basically headed straight into bikeshedding them both. I guess that's a good sign (at least, folks aren't opposed to the core idea!). Since we're bikeshedding, listed in order of my personal preference:
Track:has an overlap with Standards Track, first proposed for something like this in 2019 (https://discuss.python.org/t/2645/4 1) and was also used throughout the discussion that led to this issue2. Familiarity bias likely applies here.Area:works for me -- it's fairly similar to Track, except doesn't overlap with any existing vocabulary for PEPs (which probably doesn't matter).Topic:also works... but feels less appropriate than Track or Area.Index:feels odd to me, it's going to be listed in at-least two indexes.I'll personally avoid engaging in futher bikesheding though -- all the options proposed so far are all good-enough as far as I am concerned. :)
Both @AA-Turner (in this issue) and @CAM-Gerlach (in the d.p.o thread) have said that they'd be willing to implement this -- so instead of adding my name to the hat, I'll share this tiny bit of incomplete work I've done already, adding the Track header and exposing it via the JSON "API" and let one of them pick up the rest of the work. :)
Footnotes
This thread was a governance thread and PEP 609 has happened since then. ↩
Which makes me think it was inituitive to the group that was having this discussion. ↩
CAM-Gerlach commentedon May 3, 2022
I originally proposed
Categoryon the Discourse thread, but later went withTrackthat others suggested since it was more specific to the packaging use case and didn't implicitly require categorizing other PEPs (or it would look a little out of place) and implied the differences in process surrounding them, while it could be extended to other specialized subdomains (typing, etc) if desired in the future. But that's somewhat moot if we want to apply it to other topic areas too beyond packaging in the immediate future, and also to some extent if we don't actually display it in the headers on individual rendered PEPs, and "Track" limits extensibility to other domains that don't have a modified process. So I'd preferCategoryorTopic.Comments on naming:
Topic, IMO best captures what it seems like people want this to represent in the general case—the the topic area of the PEPCategoryis similar toTopic, but seems a little more "weighty" which may or may not be what we want to imply, but people might prefer itTrackimplies a difference in process (true of packaging, but not all others) and also conflicts with "Standards Track"AreaI find very vague and a lot less obvious, clear and specific as to what it means, as it has several other meanings and is only unambiguous in the context of phrases like e.g. "topic area"...so why not just use something more specific, likeTopicorCategory?Indexseems very odd in the context of a PEP header, because the fact that it displays in a sub-index is really an implementation detail, not high-level metadata. It would only make sense if we hide it in the rendered PEP, and even so could confuse authors.As for the URL, I think all but one of us here and on the discourse thread more or less agreed on
peps.python.org/<headername>/<topicname>, which is clearest and easiest to type, and doesn't rely on implementation details, so I think the main question to be resolved is the header name.I analyzed and outlined the specifics of what a MVP implementation would require on the Discourse thread, basically:
parser.PEP.__init__pep_index_generator.create_pep_zero, plus modifying the filepath generation logic accordinglywriter.PEPZeroWriter.write_pep0to accept atrackarg that controls using an appropriately modified intro paragraphand it seems like @pradyunsg has already starting on implementing that. The only really non-trivial part is the subindex generation, which requires a few changes to the current logic.
Additionally, the following would also need updates:
.pre-commit-config.yamlthat it has a valid value (currently justPackaging, unless we want to decide on additional standard categories/topics now)I agree with the overall rationale to check PEP 0 into the repo (again) and just inject the specific parts needed, if that makes things simpler and we'd otherwise have to add the new subpages multiple places, though it would make generating sub-indexes perhaps a bit more complicated in other ways since much of the content would vary between indicies, so I'm not sure if it is best to do as part of this change. But I'd defer to @AA-Turner on that.
I can also help implement this since I've reviewed the relevant code and what needs to be done, once we have consensus on the specifics of the implemntation.
[-]Additional index pages[/-][+]Sub-index pages for specific categories of PEPs (Packaging, etc)[/+]pradyunsg commentedon May 4, 2022
I wouldn't block implementing this on waiting on concensus on the naming + URL -- they can both be changed with a find-replace. :)
pradyunsg commentedon May 6, 2022
IIUC, everyone is OK with the URL being
peps.python.org/<header-name>/<header-value-lower-case>(only @AA-Turner wanted something different, and has since implemented this approach in #2579). If that's not the case, please holler!If anyone has strong opinions on what name to pick amongst the ones proposed so far, for the PEP header name, now is a good time to state them. Specifically, if anyone feels strongly against Topic, please say so! :)
ncoghlan commentedon May 15, 2022
I haven't checked the draft implementations, but I'd suggest that
peps.python.org/<headername>/be an auto-generated list of the track pages.I'd still advocate for "Track" as the header name, specifically because people are less likely to assume they know what it means and instead look it up. There will only be a limited number of specifically defined tracks, and it's intended primarily as a subcategorisation of "Standards Track" PEPs (although we'd presumably allow using it for other PEP types).
"Topic" feels weird to me, as "What's the topic of PEP NNN?" already has an answer (the title field), and I don't consider it immediately obvious that "Topic" is short for "Primary Topic Area".
"Category" would probably be OK, but seems a bit easier to confuse with the existing "Type" field than "Track" would be.
CAM-Gerlach commentedon May 15, 2022
I think there's an important conceptual distinction (and decision) to be made here. A few specifically defined, special-purpose tracks (e.g. Packaging, Typing, etc) covering specialized domains and having a defined meaning and some degree of implied procedural weight sounded closer to the initial idea that was discussed on the Discourse thread, which was why "Track" initially made sense to me.
However, it seemed the discussion on this issue evolved more toward wanting something closer to a general-purpose, broader and more purely editorial "Category" label to organize PEPs by topic area; under that model, all PEPs could conceivably have a category, and potentially even belong to more than one.
As a practical matter, both serve the initial motivating purpose (a mechanism to label and display packaging PEPs separately) essentially equally, and realistically it wouldn't make sense to add both, but so I think we should just decide what we want this label to mean—a special-purpose, procedurally-meaningful "track", or a general-purpose, lightweight "category"?
Personally, I don't have a strong opinion on this, and would rather hear and help implement what the community would find more useful.
ncoghlan commentedon May 20, 2022
Note that we know that Track would be useful, as it covers the Packaging use case: a procedurally distinct subset of PEPs covered by a standing delegation from the SC.
We don't have a clear use case for the generalisation to a broader "Category", "Categories", "Topics", or "Tags" notation. However, I also don't think having mutually exclusive procedural tracks defined would preclude adding non-exclusive topic area tagging later if someone came up with a compelling rationale for adding them.
JelleZijlstra commentedon May 20, 2022
For what it's worth, I have two other sub-indices (whatever they end up called) that I'd like to see:
I plan on creating those once the packaging page is done.
pradyunsg commentedon Jun 18, 2022
So... the state of affairs right now seems to be:
Topic: Packaging#2636 was merged, withTopicas the name for the header!Once #2579 is landed, let's close this issue? I think it'll be cleaner to move the discussion for adding additional topics into dedicated issues+PRs for each topic, to avoid cross-talk and confusion.
The only broad thing that I can think of is that we might want to consider whether multi-topic PEPs would be a thing -- which I suggest deferring until it becomes a concern that we actually need to solve (i.e. we add a second topic or, even later, until we identify a PEP that belongs in two topics). :)