Support task groups

[deemp]

Currently, I get this when I run uv run poe

Configured tasks:
  dev                   Run server
  check                 Format, lint, typecheck
  typecheck             Run all typecheckers
  ty-check              Run the 'ty' typechecker
  pyright-check         Run the 'pyright' typechecker
  lint                  Run the 'ruff' linter
  format                Run the 'ruff' formatter
  test                  Run pytest

I'd like to group my commands into sections for better navigation among them. Example:

Configured tasks:
[Server]
  [[Dev]]
    dev                   Run server
[Static analysis]
  [[All]]
    check                 Format, lint, typecheck
  [[Typecheck]]
    typecheck             Run all typecheckers
    ty-check              Run the 'ty' typechecker
    pyright-check         Run the 'pyright' typechecker
  [[Lint]]
    lint                  Run the 'ruff' linter
  [[Format]]
    format                Run the 'ruff' formatter
[Dynamic analysis]
  [[Tests]]
    test                  Run pytest

[nat-n]

Hi @deemp, thanks for the feedback.

Are groups for you just a way to better structure task documentation, or would you want them to have some other significance also?

[deemp]

@nat-n just a way to structure CLI docs.

The idea of such groups comes from the numtide/devshell project.

https://github.com/numtide/devshell/?tab=readme-ov-file#motd

Approach 1

I tried implementing section headings by using special task names. However, this approach failed due to validation. Example:

[tool.poe.tasks."\n[heading]"]

[tool.poe.tasks.normal]
...

I think this could be the most flexible way to insert headings.

Approach 2

However, the Approach 1 might not work well if commands are imported from another file with their own section heading levels. Just like when one wants to insert one Markdown file into another.

https://poethepoet.natn.io/guides/include_guide.html

An alternative way is to let poe handle grouping.

In devshell, I specify the category for each ”task” and then grouping happens automatically.

https://github.com/numtide/devshell/blob/255a2b1725a20d060f566e4755dbf571bbbb5f76/devshell.toml#L41

Approach 3

The Approach 1 and Approach 2 can be combined. Then, it'll be possible to insert headings at arbitrary places. At the same time, during an import, poe will merge task lists that have the same category (belong to the same section).

A problem might be when merging subsections from different sections.

pyproject.toml

[tool.poe.tasks."[Heading 1]"]

[tool.poe.tasks."[[Heading 2]]"]

task 1
task 2

another.toml

[tool.poe.tasks."[Heading 3]"]

[tool.poe.tasks."[[Heading 2]]"]

task 3
task 4

After merging, there should be:

[tool.poe.tasks."[Heading 1]"]

[tool.poe.tasks."[[Heading 2]]"]

task 1
task 2

[tool.poe.tasks."[Heading 3]"]

[tool.poe.tasks."[[Heading 2]]"]

task 3
task 4

rather than

tool.poe.tasks."[Heading 1]"]

[tool.poe.tasks."[[Heading 2]]"]

task 1
task 2
task 3
task 4

[tool.poe.tasks."[Heading 3]"]

[tool.poe.tasks."[[Heading 2]]"]

Approach 4

I prefer this approach because it's both flexible and supports imports.

The Approach 4 is the modified Approach 2. The Approach 4 addresses the problem of the Approach 3.

A category should be a list of headings to represents a path in a tree of sections.

Each heading should be indented according to its depth in the tree. Commands should be indented too.

Then, this config:

[tool.poe.tasks.task1]
category = ["[Heading 1]", "[[Heading 2]]"]

[tool.poe.tasks.task2]
category = ["[Heading 1]", "[[Heading 3]]"]

will produce this CLI:

[Heading 1]
  [[Heading 2]]
    task 1
  [[Heading 3]]
    task 2

[brolewis]

This is a feature I was looking for this week and decided to try and implement a option that was a simple implementation of task groups via a new cateogry option. While it doesn't quite work as described in this issue, I at least wanted to put out what I had to see if it was of interest to the larger community.

[deemp]

@brolewis I like that you indent commands by default. Edited my Approach 4 description similarly.

[nat-n]

Hi @brolewis, @deemp, thanks for your effort to develop this idea.

I'd previously thought about some kind of task grouping feature as a way to share config (such as executor config) across a subset of tasks. So I'd want any new feature like this to be done in a way compatible with this. To me this implies that category is too limited a semantic for grouping tasks. I'd think of task groups as perhaps comparable to dependency-groups in uv or poetry.

As an alternative to tagging tasks with a category, tasks could be defined within a group table. The advantages I see for this being that the group itself is an object that could then have config (like an executor subtable), and it encourages the toml file to look more like the help output.

[tool.poe.groups.server]
heading = "Application Serving"
executor = { type = "uv", group = "server" } # executor group here references a uv dependency group in the project!

[tool.poe.groups.server.tasks.dev]
cmd = "..."

Groups from different config files with the same name would be merged.

it could also enable going in the direction of adding support for namespacing tasks to be executed like poe namespace task-name.

How does this sound for the use cases you have in mind?

The idea of having a deep hierarchy of groups/categories, sounds maybe over complicated, do you have a use case to sell it?

I also would want to think carefully and review options on how to present this nicely in the console (with or without ansi formatting). I'm not sold on devshell's aesthetics. But simply indenting might be too subtle.

[brolewis]

@nat-n Love all these suggestions. This PR was meant to be an opening option to trigger a discussion rather than being the best way so happy to try and implement what seems to be the best option.

[deemp]

@nat-n

As an alternative to tagging tasks with a category, tasks could be defined within a group table.

Good idea. This way of defining groups looks more ergonomic.

---

The idea of having a deep hierarchy of groups/categories, sounds maybe over complicated, do you have a use case to sell it?

My user story: As a project author, I want new contributors to quickly identify all semantically grouped commands (e.g., test, typecheck) so that they can choose a right one for the task at hand.

---

I also would want to think carefully and review options on how to present this nicely in the console (with or without ansi formatting). I'm not sold on devshell's aesthetics. But simply indenting might be too subtle.

I agree that simple indentation might be not enough. Therefore, I suggest to indent by default and allow arbitrary group names.

However, there might be a problem that an arbitrary name can be used in unexpected ways such as changing the color of the group name.

So, maybe we should allow just [], <>, (), #, , and alphanumeric symbols.

---

If there is indentation, it may make sense to enable wrapping of command descriptions.

That is, there can be something like:

Configured tasks:
  [dev]                 Development commands that
                        you can run in the project   
    [serve locally]
      serve-docker      Run service in Docker Compose
                        and expose it on the port 8000
      serve-no-docker   Run service without docker
                        and expose it on the port 8080     

---

it could also enable going in the direction of adding support for namespacing tasks to be executed like poe namespace task-name.

I like the namespacing idea. Namespaces can give explicit control which path to a task to use.

However, I think that in the simplest cases, an explicit path to a task is not necessary as there are no clashes between task names. Therefore, the namespace should probably be an option so that both poe -n namespace.path task-name and poe task-name are accepted.

[brolewis]

I've added a commit to try and use group tables. Let me know if this is closer to your idea, @nat-n.

[nat-n]

Hey, sorry for the slow feedback. Here are my thoughts on this.

I agree that simple indentation might be not enough. Therefore, I suggest to indent by default and allow arbitrary group names.

I would want to use indentation sparingly, horizontal space is limited in many terminals. However we can use styling; poetry for instance groups tasks under a header with a different color.

I'd also prefer to err on the side of being more restrictive with allowed characters, maybe /[\w\d\- _\[\]\(\)]*/

I was thinking more about the namespacing idea, and one use case that would be nice to support (though not essential for the first pass) would be to configure all tasks configured by an include to be added to a specific group. This would be nice for monorepos, collecting tasks from multiple subprojects together at the root.

If the group is configured to namespace its tasks then I would expect this to work again similar to how poetry does namespaced tasks, which allows the same task name to exist in multiple namespaces (which is usually what "namespacing" means), so then either the task name effectively consists of two words: poe [global args...] namespace task-name [task args...], or is composed with a specific join char, which then has to not conflict, e.g. poe [global args...] namespace:task-name [task args...]

@brolewis I'll have a proper look at your PR and make some suggestions.