Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add parallel option for sc.asd() #620

Open
cliffckerr opened this issue Oct 14, 2024 · 3 comments
Open

Add parallel option for sc.asd() #620

cliffckerr opened this issue Oct 14, 2024 · 3 comments
Labels
enhancement New feature

Comments

@cliffckerr
Copy link
Member

No description provided.

@cliffckerr cliffckerr added the enhancement New feature label Oct 14, 2024
@RomeshA
Copy link
Contributor

RomeshA commented Oct 14, 2024

Some options might include

  • At each step, draw n proposed changes, try all of them, and pick the best one to use
  • As above, but also update the parameter selection probabilities for the next generation based on the (otherwise unused) steps

@kelvinburke
Copy link
Contributor

  • Can use Parallelize: Send func, args and kwargs once to each process using pool initializer #621 if it gets implemented. We can keep the pool with each process having the function (the model) already passed to it - reducing the time for spin-up - only needing to pass the new parameters.
  • Optionally can also use parallelize with a callback to process and suggest new samples as we get results one-by-one, rather than waiting for all n results.
  • Could also try moving in the sum of improving directions (increase both A and B at the same time if increasing A and B individually was improving). This could be useful if we want to swap budget from one program to another, but reducing or increasing one parameter at a time is worse. Not sure how that would change the probabilities - maybe just it wouldn't change them.

@kelvinburke
Copy link
Contributor

As an alternative, I have tend to found that starting multiple asds in parallel from different starting points tends to give the best results. So, if there was a built in way to do that, and possibly each asd at each step could choose from its own probabilities or perhaps some global combined probabilities and step sizes? Again not sure how that would really work - just putting ideas out there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature
Projects
None yet
Development

No branches or pull requests

3 participants