Importing into a Sagemaker MLFlow instance #200
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I recently had to set up a Sagemaker MLFlow instance and import data from our previous MLFlow instance. The AWS Sagemaker team has provided an MLFlow plugin to make connection easier. https://docs.aws.amazon.com/sagemaker/latest/dg/mlflow-track-experiments.html
With it you set the tracking uri to an ARN of form:
arn:aws:sagemaker:{region}:{account-id}:mlflow-tracking-server/{server-name}
. Because of this I found there were some checks in the code that didn't expect this format that were unnecessary.Another note, is that this plugin makes connecting to the MLFlow server via the python client a breeze, but doing so directly through the REST endpoint is near impossible. I had to change one instance where the http_client was used instead of the mlflow_client to make importing runs with Dataset inputs to work properly.
The last thing I changed was just adding a warning as models with names "my_model" wouldn't import, but "my-model" would work. I have no idea where this limitation comes into play, but I didn't get any errors surface when I ran the export-import tool, so a warning would have helped. I discovered the issue after trying to manually add a registered model in the MLFlow UI.