DD4hepSimulation: pass self to the steeringFile eval

[veprbl]

BEGINRELEASENOTES

• Steering files can now omit instantiating a new SIM = DD4hepSimulation(), and instead use a pre-defined variable with name SIM. This allows for steering files that preserve the existing state of DD4hepSimulation that a custom user code had set prior to the parseOptions() call.

ENDRELEASENOTES

At ePIC we use an entry point like

SIM = DD4hepSimulation()

SIM.Foo = "Xyz"
# ... other common options

SIM.parseOptions()
SIM.run()

https://github.com/eic/npsim/blob/01d54e67c7bbd671d1f87180e55c296286e639c5/src/dd4pod/python/npsim.py

However using this with --steeringFile unconditionally unsets our settings as new DD4hepSimulation object has to be created in the steering. The proposed change allows to write steering files without the

from DDSim.DD4hepSimulation import DD4hepSimulation
SIM = DD4hepSimulation()

This is optional and should not break any existing use case. A next step could be to remove those two lines from output of the --dumpSteeringFile.

[github-actions]

Test Results

   14 files     14 suites   5h 26m 42s ⏱️
  369 tests   369 ✅ 0 💤 0 ❌
2 538 runs  2 538 ✅ 0 💤 0 ❌

Results for commit da9f99b.

[MarkusFrankATcernch]

Happy New Year to all!

@andresailer
Do you agree with these changes ? Then we can merge this MR.

[andresailer]

Hi @veprbl

How does your new steering file look like then?
This is needed because you run npsim instead of ddsim?

[veprbl]

Hi @veprbl

How does your new steering file look like then?

Old file:

from DDSim.DD4hepSimulation import DD4hepSimulation
from g4units import mm, GeV, MeV, degree
SIM = DD4hepSimulation()
SIM.gun.energy = 10*GeV
SIM.gun.particle = "e-"
SIM.gun.position = (0.0, 0.0, 0.0)
SIM.gun.direction = (0.0, 0.0, 1.0)
SIM.gun.distribution = "cos(theta)"
SIM.gun.thetaMin = 45*degree
SIM.gun.thetaMax = 135*degree

New file:

from g4units import mm, GeV, MeV, degree
SIM.gun.energy = 10*GeV
SIM.gun.particle = "e-"
SIM.gun.position = (0.0, 0.0, 0.0)
SIM.gun.direction = (0.0, 0.0, 1.0)
SIM.gun.distribution = "cos(theta)"
SIM.gun.thetaMin = 45*degree
SIM.gun.thetaMax = 135*degree

This is needed because you run npsim instead of ddsim?

npsim implemented so that we can provide reasonable defaults, but the implementation of option parsing in DD4hepSimulation breaks those when --steeringFile is used. I assume, ddsim doesn't have any defaults baked in, so it would not be affected.

I am proposing swapping order of option setting in eic/npsim#29. That would, for now solve the original problem for us, but also make it impossible for users to undo our settings.

[andresailer]

I assume, ddsim doesn't have any defaults baked in, so it would not be affected

ddsim has many defaults baked in, I guess I don't really understand what you mean...

I don't like the SIM appearing out of nowhere in the steering file, that messes up linters, doesn't it?

Also can you answer "yes/no" whether this is because of npsim? (this is just for my understanding)

You want to have a executable with defaults for your detector simulation, instead of setting everything in a steering file and use that, correct?

[veprbl]

I assume, ddsim doesn't have any defaults baked in, so it would not be affected

ddsim has many defaults baked in, I guess I don't really understand what you mean...

I meant ddsim binary itself uses the default-constructed DD4hepSimulation, not the defaults in the grander DDG4.

I don't like the SIM appearing out of nowhere in the steering file, that messes up linters, doesn't it?

Linters are not supposed to care about semantics, but I share your concern. A proper way to do this is to structure config as

from g4units import mm, GeV, MeV, degree

def init_simulation(sim: DD4hepSimulation):
    # do things here
    return sim

That can be made backwards compatible too.

Also can you answer "yes/no" whether this is because of npsim? (this is just for my understanding)

Yes.

You want to have a executable with defaults for your detector simulation, instead of setting everything in a steering file and use that, correct?

Correct. Moreover, the defaults need to apply even if steering files are not used.

[andresailer]

I don't understand the release notes entries.

and instead use a pre-defined SIM.

Where is this pre-defind SIM coming from?

This allows for steering files that don't completely override the simulation settings.

You can already not completely override simulation settings in the steering file. These statements need to be refined and I guess be explicit about not using ddsim.

Regarding linters: pylint cares about "semantics" (certainly pylint would complain if you start using undefined variables), I guess this goes beyond linting, but it is called py"lint", so apologies for the lax use of language.

[veprbl]

I don't understand the release notes entries.

and instead use a pre-defined SIM.

Where is this pre-defind SIM coming from?

That refers to the {"SIM": self} that's passed to eval

This allows for steering files that don't completely override the simulation settings.

You can already not completely override simulation settings in the steering file. These statements need to be refined and I guess be explicit about not using ddsim.

I've updated the wording. Is that better now?

Regarding linters: pylint cares about "semantics" (certainly pylint would complain if you start using undefined variables), I guess this goes beyond linting, but it is called py"lint", so apologies for the lax use of language.

Let me know if you want me to implement my proposal using functions to implement overlays.

[andresailer]

Thanks for all you answers!

Would it be possible to subclass DD4hepSimulation (NPSimSimulation) and use that in the steering file instead?
Probably would need some more modularisation in DD4hep simulation to give more access to subclasses?

[veprbl]

Thanks for all you answers!

Would it be possible to subclass DD4hepSimulation (NPSimSimulation) and use that in the steering file instead? Probably would need some more modularisation in DD4hep simulation to give more access to subclasses?

It would be possible to provide NPSimSimulation to be importable in the user steering files, however that would make those steering files incompatible with ddsim.

Like you point out, that would be more work. I feel pretty confident about fixing external behaviour of DD4hepSimulation.parseOptions() to not overwrite an existing object state. On the other hand, I believe, OOP paradigm is either inherently broken, or is just too hard for a normal programmer to do right, so I'd rather not go that route.