New dynamic graph implementation

[hansihe]

This is highly WIP.

Not ready for anything other than very early review/input.

A couple of new crates are added:

• rendy-scheduler: Implementation of a graph scheduling algorithm that should be fairly efficient once all TODOs are out of the way. This crate is very loosely coupled to gfx-hal, and is mostly data driven.
• rendy-cache: Resource management. Prototyped in a separate crate, but I expect this to be at least partially integrated into rendy-factory.
• rendy-graph: Completely rewritten, more dynamic, render graph. The interface the end user would interact with. Depends on scheduler and cache.

[ezpuzz]

this is awesome @hansihe ! I will try to build amethyst against your branch and see if anything explodes. thanks for contributing! getting gfx updated will be a huge help for amethyst's mac os compatibility issues.

[hansihe]

@ezpuzz Just to be clear, this will not build as is, there is still a lot of work left.

Mainly just pushed things up because I wanted the WIP on git, and I wanted to share the work I have done so far.

This is more of a long term effort, I can probably give updating the existing render graph to newer gfx-hal in the short term if that's helpful for the project.

[ezpuzz]

@hansihe understood, I will keep an eye on your progress then. yeah essentially we've been blocked on updating the swapchain part of it to work with new gfx and that's blocking mac os compatibility in amethyst. Very interested in your work either way. I have a few other branches in this repo with attempts at getting there but was blocked by a poor grasp of the graph part of things.

[hansihe]

@ezpuzz Is there somewhere outside of github where things are discussed in regards to rendering? I have seen the discord, but the rendering channel seems pretty dead.

[ezpuzz]

@ezpuzz Is there somewhere outside of github where things are discussed in regards to rendering? I have seen the discord, but the rendering channel seems pretty dead.

We should bring back a rendy channel, but we do discuss it sometimes if you search for rendy. I think @AlveLarsson can create a rendy channel and we can chat in discord.

[erlend-sh]

Very exciting work here! 😍

I have restored the #rendering channel on Discord. I expect it could easily become active again if you start asking questions in there. But using this issue extensively is also most welcome.

[hansihe]

I believe the scheduler itself should be in a fairly good state now, it is generating valid scheduling orders of entities, although there are some major improvements that could be made here in the future with regards to interspersing independent dependency chains to a maximal degree. The synchronization strategy generation seems to work decently too now, so I have all the data I need to actually write out the command buffers.

It seems like the gfx folks were open to expanding the gfx-hal API to match the synchronization2 vulkan extension. This would mean we have the ability to pass more of the rich synchronization information we have available down to the actual implementation. As an extra perk, this additional information should make it possible to support DX12 split barriers in gfx-hal.

Right now I am working on the APIs to specify and bind graphics/compute pipelines. Since render passes/subpasses are fully created and managed by the graph, and graphics pipelines need information on the pass/subpass they operate within, I am doing pipeline management and caching within the graph too.

After that is working, I should hopefully be at the point where I can at least render a triangle, although at that point I would also need to think about how we want to do stuff like descriptor management and resource management in general. I have yet to decide on exactly how I want to handle descriptors, but I'm hoping these things will solidify a bit as I can start tinkering around with actual working rendering.

To conclude, I am going slightly insane by the verbosity and flexibility of vulkan, but it is very motivating to feel like I am getting closer to being able to actually draw something.

[hansihe]

This now builds, executes and submits rendering and presenting a single triangle. Still black screen and validation layer warnings though, since I don't yet record barriers.

What is left to do until this can properly render a triangle over multiple frames:

• Record barriers into command buffers. This should be relatively simple, the scheduler already outputs the full synchronization strategy, we just need to apply it.
• Proper resource management. Right now I don't cache resources and keep them alive for the time of flight of the frame. This mainly involves:
  • Resources (images and buffers)
  • Semaphores
  • Events
  • Command buffers

Render passes, graphics pipelines and shader compilation is already cached, though probably not in the best possible way yet.

Once I am at that point, I will probably start experimenting with porting either amethyst-rendy or the PBR renderer I saw floating around.

Don't take this as meaning this is almost complete, there is a lot of kinks that need to be worked out before this will actually be usable.

[hansihe]

The graph-side APIs are probably relatively finished at this point. The two nodes I use for triangle rendering are:

Present

https://github.com/AmethystVR/rendy/blob/new-graph/graph/src/node/present.rs

Example of a presentation node. Although not fully featured with it comes to format and resolution handling, it is fully composable and supports rendering directly to the swapchain image with zero knowledge from the other nodes that interact with the image. This is a big improvement from the previous version of the graph, where we either had to do a copy/blit, or the render node had to have specialized code for performing a present.

There might be some simplifications to how the "time travelling" resource moves work, but this should be relatively minor.

Rendering triangle

https://github.com/AmethystVR/rendy/blob/new-graph/graph/src/node/draw_triangle.rs#L110-L138

Example of a node that performs rendering in a render pass.

Notice that in the construction phase, the node simply declares a render pass, adds a framebuffer attachment, then commits the render pass with an execution closure. This execution closure then performs the recording of the command buffer. Render pass creation and management is automatically and transparently managed by the graph.

A core feature in the design of the whole graph is that I want it to treat tiled renderers on mobile as first class citizens. The scheduler fully supports render passes, and transient attachments, all in a completely composable way. To perform rendering with multiple subpasses, you simply need to call ctx.mark_render_pass(start_entity, end_entity), and the scheduler will take care of everything else. Not doing this automatically is an explicit design decision, as I don't think attempting to merge render passes automatically is a very helpful thing to do. I want it to be really really easy to perform rendering within a render pass, but I want the decision to be completely up to the user. This way there is no weird behavior around things not becoming a render pass when you expect them to because of some small detail.

[hansihe]

Rendering a triangle works!

Triangle example using the new graph interface is here:

https://github.com/amethyst/rendy/blob/affcf482cd71a62910c8f0c88b85dd6ab4d335e0/rendy/examples/triangle_newgraph/main.rs

As I understand it you are involved with amethyst_rendy @ezpuzz? Any comments on this new interface would be appreciated.

Still work to do when it comes to buffer and image resource management, as well as writing out intra-renderpass synchronization. Not to mention many codepaths not yet needed for this simple example, which are still unimplemented.

Once the base graph is somewhat done and working for larger renders, I am looking forward to exploring threading. The current plan on that front is to do graph construction and scheduling on one thread, then the schedule can be used to do the actual execution and command buffer writing across a thread pool. This shouldn't require many changes on the API, except for adding a Send bound on execution closures.