Handling of low disk space

[michellutz]

Background and Motivation

To improve handling of low disk space the user could be notified about the status of disk. In addition,
some old test object data should be deleted after a while.

This issue is related to etf-validator/etf-webapp#13.

Proposed change

• Warn user on low disk space in Status View
• Delete old test object data (based on a max lifetime property) e.g. db index files

Alternatives

• Adding a label in home.html that shows the usable disk space.
• Improve the class "TestResultCleaner" to clean also the old test object data.

Funding

TBD

[jenriquesoriano]

It could be useful to show in the status more information regarding the ETF status:
% free memory (free/total)
% free disk space (free/total)
CPU load

[jonherrmann]

I think these are information that are relevant only for the Admin and should not be shown in the standard user interface. Meanwhile, I even think that the Loaded components should not be displayed and the status view should be renamed to Running tests or something like that.

For users, it might be more useful to show a classified status, based on the resource details. For example: "System is under heavy load, tests may take a while", "No space left on disk for new test data. New tests can not be started at this time. Wait for old test data to be cleaned up.". I think these information should be displayed when the user attempts to start a new test or visits the Running tests view.

[jenriquesoriano]

To do:

• Check with the INSPIRE community the actual use of the status tab of the ETF
• Check with the INSPIRE community the kind of messages regarding the resources consumption that might be more useful for the users
• Investigate how to get some additional user navigation info from the ETF/Tomcat logs

[carlospzurita]

As @jonherrmann said, the information on the ETF Status page is not very relevant to a common user of the application. Maybe the messages to display could be:

• Low work load, service ready for use
• Heavy work load, tests run will be queued (In case of low memory or threads availables)
• Performing data clean up, service disabled (In case of low disk space)

Another option could be to check the test objects that the user is providing, to see if the size of the data could be hosted on the system. Also, apply some heuristics to evaluate the space in memory needed for the test run, and how much time it will take; this way some quick test runs could be allowed under heavy work load.

On a related note, it would be interesting to check the usage of the status page. Right now, the page is accessed via Javascript, and no log is generated when displaying the status tab. A tool like Matomo (formerly named Piwik) could be used to capture the client side interactions, and get usage statistics.