How can I run pytest.main() multiple times from within an embedded Python environment?

[bradleyharden]

As discussed in #3143, running pytest.main() multiple times will give the same result, even if the tests have changed. This is a consequence of Python's module caching. Thus, the recommended approach is to run pytest in a subprocess.

Unfortunately, that's not so simple for me. I'm trying to use pytest within an application that uses Python as an embedded scripting language. Specifically, it looks like the application uses the embeddable package described here, as I see the python3.dll and python3.7.dll files.

I would use the subprocess approach, but I don't know how to invoke the embedded Python interpreter. Moreover, I don't know how to reproduce the rest of the environment created by the application. Is it possible to invoke it directly from the DLL? I don't really know anything about Windows.

I also considered the multiprocessing module. My hope was that I could use it to spawn a fresh interpreter for pytest. However, it looks like I can't import multiprocessing, which suggests that the application explicitly doesn't include it.

Does anyone have ideas on how I could proceed? The list of test modules will be fairly small and known statically. Perhaps, I could manually reload those files before calling pytest.main()? Are there other options?

[bradleyharden]

From some testing this afternoon, I believe I can simply call importlib.reload on each test module before I call pytest.main(). That's easy to do, as all the modules are in one directory. pytest's test collection feature is a bit overboard for my use case.

If that's all I need, then I can close this discussion. But I'm a little worried about what, if any, interaction there is between this issue and #12454. I'll leave this issue open until I resolve the other.

[bradleyharden]

Manually reloading the modules seems to be a solution. It's not ideal, but it works for me.