GRU/LSTM stateful = true implementation or workaround guidance

[aawce]

Hi,

We are doing real-time classification and very much like onnxruntime and have been working with it for some time now. However, we need to implement stateful=true in our GRU and LSTMs. We understand that onnx doesn't support stateful, and we have a way of specifying stateful via metadata.

To implement stateful in our ort build, we reviewed other threads suggesting to read the state after a session.run, and pass it back in to the next session.run. This might work for some cases, but it's not clear if it works for some of our use cases where we may want to run batches of sequences/timesteps.

However, option2 is to copy the cpu-provider GRU into our own EP and modify it to keep state. This would be our preferred way to do it BTW for several other reasons, but we'd need help and guidance to do this.

So Q to ort team - what would be recommended approach, and preferably is an expert available to spend a little time guiding us on doing option2? We are willing to roll up our sleeves to do it.

Thks,
Chris

[skottmckay]

Response in #6936