130 ay 6965 default output paths and support custom staging directories.

[MustafaJafar]

Changelog Description

This PR adds a main setting for configuring output directory for created ROP nodes via AYON creator plugins.

• Add settings toggle to enable/disable path control completely (to allow a studio to completely manage it themselves)
• If enabled,
  • we use the default path as follows {default_path}/`chs('AYON_productName')`/hardcoded_filename
  • intermediate render directories get a hardcoded folder name as follows{default_path}/`chs('AYON_productName')`/hardcoded_foldername/hardcoded_filename
• However, if matching custom staging dir profile is specified in ayon-core settings then we use that custom staging dir directory instead. (This also only applies if "directory management" is enabled" as per the first point).

resolve #130 and resolve #21

Demo

staging templates	custom staging directories

1. follows houdini_cache template
2. follows houdini_Render template
3. follows the default value in Houdini settings as it doesn't match any profile.

Testing notes:

1. Create new instances in Houdini, you should find the new output paths.
2. Publishing new and old instances should work without issues.

[moonyuet]

I tested with publishing the model product. the staging directory set correctly

But later I find out that two of the validators are broken, guess we need to implement the fixes on these two.
They are Validate Primitive to Detail(abc) and Validate Single Frame
The attachments are the json report to show the error of the validator(s).
publish-report-241223-16-42.json
This error of Validate Single Frame showed after the fix in Validate Primitive to Detail(abc)
publish-report-241223-16-44.json

Please fix in the separated PR as they are not related to this PR.

[moonyuet]

See the comment above

[BigRoy]

Nice work!

1. I'm not entirely convinced that ONLY allowing to customize the output directory is sufficient - but maybe it is! ❤️ Would need input from the community to fully grasp whether this is the best approach. Like it may make sense to be more concrete about the filename as well?
2. Whenever we discuss this 'feature' I always keep thinking whether we are doing it the right Houdini-way. I'm wondering if we should have a checkbox in settings that instead disables setting the path completely so that it's possible for a studio to actually define the output path completely by setting their own node attribute defaults directly in Houdini. Again, possibly something we should check with the Houdini-community using AYON. Does it actually make sense that we set it through code instead of relying on the node's defaults (which can be set by a studio, or even an artist specifically) to be adopted instead.
3. Having a single configurable entry does keep it VERY simple to manage - I like that. However, it does disallow completely to go more granular (which wasn't possible anyway, so could be something for later instead).

Also wanted to mention that there is quite some overlap between this feature and the Custom Staging Dir feature which would define where published work renders should go to. As such, even though I like this implementation it may very well happen down the line that this should get superseded by using the custom staging dir setings (which I think are in core as a feature?)

[BigRoy]

My main concerns are mentioned here - other than that I think the implementation is very clean and nice.

However, I do think the:

`chs('AYON_productName')`

part should be part of the setting, not hardcoded in the setting of the path.

As such, the default would be this instead:

staging_dir = "$HIP/ayon/`chs('AYON_productName')`"

I also wonder whether making the new default render/ instead of ayon/ would make way more sense. It'd at least align more with other host integrations like Maya.

@antirotor thoughts?

[MustafaJafar]

However, I do think the:

`chs('AYON_productName')`

part should be part of the setting, not hardcoded in the setting of the path.

This won't work with HDAs, as its file name is not saved in a string parameter.
Also for reference, path in settings is not actually a staging directory it's more of a root output directory and every product can have multiple files like render has render and intermediate render files.
Here's a list of the current paths in this PR (the root is the path in settings: $HIP/ayon)

- Camera, Model, abc pointcache: "{root}/`chs('AYON_productName')`/$OS.abc"
- Arnold 🍑: "{root}/`chs('AYON_productName')`/$OS.$F4{ext}"
- Arnold: 
  - render: "{root}/`chs('AYON_productName')`/$OS.$F4.{ext}"
  - 🍑: "{root}/`chs('AYON_productName')`/ass/$OS.$F4.ass"
- Bgeo pointcache: "{root}/`chs('AYON_productName')`/$OS.$F4.{ext}"
- Cop: "{root}/`chs('AYON_productName')`/$OS.$F4{ext}"
- HDAs: "{root}/HDAs/{node_name}.hda"
- Karma: 
  - render: "{root}/`chs('AYON_productName')`/$OS.$F4.{ext}"
  - checkpoint: "{root}/`chs('AYON_productName')`/checkpoint/$OS.$F4.checkpoint"
  - USD output: "{root}/`chs('AYON_productName')`/usd/$OS_$RENDERID"
- Mantra: 
  - render: "{root}/`chs('AYON_productName')`/$OS.$F4.{ext}"
  - ifd: "{root}/`chs('AYON_productName')`/ifd/$OS.$F4.ifd"
- RS proxy: "{root}/`chs('AYON_productName')`/$OS.$F4.rs"
- Redshift:
  - render: "{root}/`chs('AYON_productName')`/$OS.$AOV.$F4.{ext}"
  - proxy: "{root}/`chs('AYON_productName')`/rs/$OS.$F4.rs
- review: "{root}/`chs('AYON_productName')`/$OS.$F4.{ext}"
- staticmesh: "{root}/`chs('AYON_productName')`/$OS.fbx"
- USD: "{root}/`chs('AYON_productName')`/$OS.usd"
- USD render: "{root}/`chs('AYON_productName')`/usd/$HIPNAME/$OS
- VDB: "{root}/`chs('AYON_productName')`/$OS.$F4.vdb"
- VRay:
  - with AOV: "{root}/`chs('AYON_productName')`/$OS.$AOV.$F4.{ext}"
  - without AOV: "{root}/`chs('AYON_productName')`/$OS.$F4.{ext}"

I also wonder whether making the new default render/ instead of ayon/ would make way more sense. It'd at least align more with other host integrations like Maya.

I wonder if Houdini artists might find it confusing because it doesn't align with Houdini defaults where geo files will be saved in render/ instead of /geo.

[MustafaJafar]

Regarding point 1 and 3 about being more granular about file names and not using hardcoded subdirectories.
This was implemented in the far first few commits in the PR before reverting these changes.
IMO, the settings were too many making them hard to use.
ynput/ayon-core#9

[BigRoy]

This won't work with HDAs, as its file name is not saved in a string parameter. Also for reference, path in settings is not actually a staging directory it's more of a root output directory and every product can have multiple files like render has render and intermediate render files. Here's a list of the current paths in this PR (the root is the path in settings: $HIP/ayon)

I see, thanks for clarifying that. Still - it feels odd to still have that part of the path be hardcoded for all the others. Not sure what the best middle ground is - but in a way, it's still not really customizable.

Also, similar to "custom staging dir" - a studio may want their renders in a different folder than their geo caches. Which also hints at needing settings PER creator instead of a single global one. Yet I do agree - it'd be a lot of settings to maintain. It could be profile based - where the HDA creator just happens to have a different default profile.

Tagging @dee-ynput @antirotor for their thoughts.

---

I also wonder whether making the new default render/ instead of ayon/ would make way more sense. It'd at least align more with other host integrations like Maya.

I wonder if Houdini artists might find it confusing because it doesn't align with Houdini defaults where geo files will be saved in render/ instead of /geo.

Valid point - potentially even more reason to 'remain at the defaults' as per my point 2) above.

---

Side note: I'm personally also NOT a fan of the $HIP getting expanded on create either - but I guess that's also how it was already.

[BigRoy]

After the meeting we had with @antirotor and @MustafaJafar this became the changes:

• Add settings toggle to enable/disable path control completely (to allow a studio to completely manage it themselves)
• If enabled, we use the default path as you have it set up ✅
• However, if matching custom staging dir profile is specified in ayon-core settings then we use that custom staging dir directory instead. (This also only applies if "directory management" is enabled" as per my first point).

[MustafaJafar]

As such, the default would be this instead:

staging_dir = "$HIP/ayon/`chs('AYON_productName')`"

For reference, I gave this a test and it won't work because of we are expanding string hou.text.expandString resulting in the result in my second screenshot.

[MustafaJafar]

Here are some note.

[MustafaJafar]

This should be working now.
Also, I've explained in code why some functions and methods weren't used to obtain the staging dir.
Anyways, I gave it a test run and everything works as before.

[BigRoy]

Overall this works great - however, I still think we should make the expanding of $HIP optional (even if another validator is currently in the way of making that super useful.)

What I'd like to do is:

1. Move the hou.text.expandString(staging_dir) call into HoudiniCreator.get_custom_staging_dir instead of per creator
2. Add an additional setting in settings that enables/disables that if expand_vars: ... if managing the paths is enabled.

I also think we should update ROP Output Directory label to Set ROP Output Directory on Create or something so it's explicit when it triggers from the label, instead of needing to look at the tooltips. What do you think?

Then, after this PR is merged - we should maybe create an issue to look into ways how we should potentially update the render paths after e.g. context switching if needed so that we avoid re-using a scene using the old render paths accidentally. Whether that's a validator, or the creator handling that out of the box is something we can discuss and decide then.

Other than that, great work - very clean!