| Issue Number | 4820 |
|---|---|
| Summary | Parameter Validation for Scheduled Jobs |
| Created | 2020-04-29 11:48:08 |
| Issue Type | New Feature |
| Submitted By | Englisch, Volker (NIH/NCI) [C] |
| Assigned To | Kline, Bob (NIH/NCI) [C] |
| Status | Closed |
| Resolved | 2020-09-16 08:56:18 |
| Resolution | Fixed |
| Path | /home/bkline/backups/jira/ocecdr/issue.261142 |
It is currently possible to add any parameters to our scheduled jobs regardless if they are used or not. We discussed that it would be nice to have a job fail or notify the users - or even have the scheduler itself perform some kind of parameter validation - when such a bogus parameter has been added.
Implemented on DEV. Please test carefully, ~volker, and check the source code to make sure parameter lists are correct and typo-free. 😉
OK, this is actually not directly part of this ticket but I see that one of the parameters is sometimes listed as "recip" and other times as "recips" (plural 's').
Should we add a ticket for a future release or just ignore it?
That depends. Are the cases where it's singular perhaps intended to have only one recipient? If not, and a good argument could be made that in at least some of the cases more than one recipient should be supported, I'm not averse to changing it to plural.
I also see two jobs using "log-level" while one uses "loglevel"
One job uses camel-case parameters while all other job's parameters are lower case.
One job uses "skip-email", another uses "noemail".
I need to check if "job" and "job-id" are referring to the same thing.
One job uses "host", another "hostname"
I finally went through the entire set of scheduled jobs and ran into a few issues while running the individual jobs. Some of these issues will likely be ignored but I'm going to list those anyway:
"log-level" missing
These jobs specify the "log-level" parameter but an error indicates it
is an invalid parameter:
Glossary Documents Ready For Translation
Glossary Translation Job Notification
Media Documents Ready For Translation
Media Translation Job Notification
Hoover is the only job that still runs with a bogus parameter
Two jobs fail with an error message - likely because these are
temporary or in development:
Load drugs for CDE (temporary)
Error message: Unable to load jobs.drugs.Loader: [Errno 13]
Permission denied: 'd:
cdr
Scheduler
jobs
drugs.py'
Refresh Sitemap
Unable to load jobs.sitemap.Loader: [Errno 13] Permission denied:
'd:
cdr
Scheduler
jobs
sitemap.py'
There were 3 sets of parameters I extracted from the diff report
on github but I didn't see these sets being used anywhere:
'tier'
'ConfigFile', 'Email', 'OutputDir', 'TestMode'
'dump', 'hostname', 'keep', 'tier', 'username'
"log-level" missing
These jobs specify the "log-level" parameter but an error indicates it is an invalid parameter: ...
Isn't that what you created this ticket to do? To report unsupported parameters entered on the Admin interface? I left these in the job definitions so you'd have something to test. I have removed them now.
Hoover ... still runs with a bogus parameter
Typo (SUPPORTER_PARAMETERS instead of
SUPPORTED_PARAMETERS). Fixed
Two jobs fail with an error message...
Our old friend Windows file-systems permissions bugs. I have run
fix-permissions which has addressed that problem (for
now).
There were 3 sets ... I didn't see ... being used anywhere ...
Just because a parameter is supported doesn't mean it's required to be used. For example, you can override the configuration for Hoover (for testing, e.g.), but you typically won't.
Those "not used" parameter sets belonged to the jobs which caused the error messages.
It's all working as expected now.
Tested successfully on QA.
Confirmed on PROD.
Elapsed: 0:00:00.001745