Updated to allow the token_file_path parameter #1288
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
resolves #1287
Problem
dbt currently does not support the
token_file_path
parameter which is used when connecting from a Snowpark Container.Snowflake Container Services (SPCS) provides an OAuth token file, /snowflake/session/token, to containers to allow them to log into Snowflake without needing a service account. The token in this file expires but is refreshed automatically by the SPCS service. The Snowflake Python Connector added a token_file_path parameter in version 3.11.0 (June 2024) to make it easier for clients to connect without having to read the OAuth token from the file every time a new connection is needed. Instead, by passing the token_file_path parameter set to /snowflake/session/token the driver will automatically handle retrieving new OAuth tokens whenever a new connection is created.
Solution
I have made the necessary changes to the connection.py file and have successfully tested these changes in a Snowpark container. I made my changes branched from the v1.9.0 tag because I was unable to get hatch in the main branch to build an adapter that dbt would pick up. I would deploy it but dbt --version would not show the snowflake adapter and projects would fail saying that it could not find the snowflake adapter. Switching back to the v1.9.0 branch I was able to use
make dev
, build my wheel for SPCS usingbash ./scripts/build-dist.sh
and install locally usingpip install .
. After local regression testing, I deployed the wheel to SPCS and verified it could connect using the token file. I'm guessing that I'm not the only person unable to build from main right now.As you will see in my PR, I also updated the error checking for this and related parameters. When the parameter is added, my code validates that the token file exists and has a useful error message if it does not. I also removed the error for a missing user parameter when token_file_path is used because SPCS OAuth does not need a user specified. Finally, I also added a condition to skip the token parameter logic when token_file_path is being used.
Checklist