[Question]: Is AMF_NEED_MORE_INPUT working as intended?

[cgutman]

I am attempting to use the AMF_VIDEO_ENCODER_QUERY_TIMEOUT feature to minimize polling on AMFComponent::QueryOutput() while encoding in FFmpeg's AMF encoder. Since FFmpeg calls AMFComponent::SubmitInput() and AMFComponent::QueryOutput() on a single thread, it needs to know prior to calling QueryOutput() whether more input is required to produce a new frame, otherwise QueryOutput() will always wait for the full timeout period and performance of the encoding loop will suffer greatly.
At first glance, checking for AMF_NEED_MORE_INPUT seems perfect for this purpose. The comment for that status value in the Result.h states:

    AMF_NEED_MORE_INPUT                         ,//returned by AMFComponent::SubmitInput did not produce a buffer because more input submissions are required.

However, I'm not seeing that behavior in at least some encoding configurations. One such case is with pre-analysis enabled, which can be demostrated using the EncoderLatency sample using the -ALGORITHM OneInOne -EnablePreAnalysis true parameters. I realize using the EncoderLatency sample's OneInOne mode with PA is not expected to work (since PA requires more than input for an output to be generated and the OneInOne logic doesn't handle that), but I would expect it to fail here with AMF_NEED_MORE_INPUT:

AMF/amf/public/samples/CPPSamples/EncoderLatency/EncoderLatency.cpp

Lines 852 to 855 in 3db6164

	// we're doing frame-in/frame-out so the input
	// should never be full
	res = encoder->SubmitInput(surfaceIn);
	AMF_RETURN_IF_FAILED(res, L"SubmitInput() failed");

Instead, SubmitInput() returns AMF_OK and the code hangs in the output loop with AMF_REPEAT being returned forever:

AMF/amf/public/samples/CPPSamples/EncoderLatency/EncoderLatency.cpp

Lines 860 to 868 in 3db6164

	amf::AMFDataPtr data;
	do
	{
	res = encoder->QueryOutput(&data);
	if (res == AMF_REPEAT)
	{
	amf_sleep(1);
	}
	} while (res == AMF_REPEAT);

Are my expectations of AMF_NEED_MORE_INPUT wrong or is this a bug?

Note: I know I could hardcode logic for known cases where AMF will require additional frames for input (like B-frames or PA), but that seems like an extremely brittle solution for something like FFmpeg which accepts basically arbitrary user encoding parameters. It also means that any change that you make to AMF that would increase the buffering requirement will cause a performance regression with FFmpeg. That's why I would strongly prefer AMF to just tell me when it needs more input rather than making me guess.

[MikhailAMD]

I follow your conversation in FFmpeg mail list and appreciate your efforts.
Currently in AMF encoder there is no clear indication by return code if more input is needed before frame is produced or the frame can be waited. It would be nice to have, but the main AMF model is to use two threads where it doesn't matter. We may consider to introduce proper return codes, but it is not simple - need to take in account: regular B-frames, adaptive-B-frames, look ahead. Another possiblity is to make TIMEOUT parameter dynamic.
And yes, EncoderLatency, OneInOne mode was not designed to be used with PA.
Saying all that, I like your proposal with 1ms - we made some research, it gives way better results. The AMF maintainer will update your TIMEOUT patch and merge.
Thanks