Delete a Transform

[BenGalewsky]

Problem

Users can't delete unused transform requests to free up disk space

Approach

This PR goes along with ServiceX PR 915 to provide a command line tool to delete transforms on the server

[codecov]

Codecov Report

Attention: Patch coverage is 69.69697% with 10 lines in your changes missing coverage. Please review.

Project coverage is 86.14%. Comparing base (408fe81) to head (8b3065c).
Report is 16 commits behind head on master.

Files with missing lines	Patch %	Lines
servicex/app/transforms.py	25.00%	6 Missing ⚠️
servicex/app/cache.py	20.00%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #521      +/-   ##
==========================================
+ Coverage   83.05%   86.14%   +3.09%     
==========================================
  Files          27       27              
  Lines        1670     1747      +77     
==========================================
+ Hits         1387     1505     +118     
+ Misses        283      242      -41     

Flag	Coverage Δ
unittests	86.14% <69.69%> (+3.09%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[ponyisi]

So, to bring up the language issue again here ... nothing here removes the local copies of transform results, yes? Is "delete" the correct keyword then? Not maybe "archive" or something similar?

I assume we have an equivalent command to clear a transform from the local cache - do we want to connect the two commands in some way?

[BenGalewsky]

Good call on the deleting the cache... pushed a new commit to do so

[ponyisi]

I'd like @gordonwatts and @kyungeonchoi to comment on the contract here:

• deleting a transform on the client side removes files locally and on the server
• however the server will have its own archiving mechanism in play which will potentially remove the files remotely
  so I think this is perhaps best thought of (from the user side) as a local file cleanup mechanism, and not as a way to do server file management? (which I guess users shouldn't be worried about anyway)

[BenGalewsky]

To be honest, this feature was mostly a stepping stone to the automated lifecycle management, but if you are a polite user and you ran a transform and realized you made a mistake and will never use the results, you can issue this command and it will delete the files locally and off of the server.

If you don't do this, then yes eventually the lifecycle ops will delete this for you

[ponyisi]

This probably is best brainstormed at the retreat, but - the issue as I see it is that a very typical pattern is users just running many different variants of a transform to develop it, so at the end they have a lot of old versions, but they haven't kept track of the obsolete development ones, and they aren't likely to go hunt down the old transform IDs one by one.

If I had to envision a more user-friendly scenario, it would be

• client knows the most recent iteration of transforms with a certain name
• there is a command that allows us to purge the previous versions while keeping the latest ones
• the transform deletion command also takes the sample name as an option instead of the transform ID