CDR Tickets

A very similar issue was addressed in OCECDR-3819.

I submitted a ticket to remove the bad file: WEBTEAM-6411.

Modify the script so that it notifies us when this happens.

How does this sound:

Add an email address to the FileSweeper config file, for example, something like this, with as many email addresses as desired.

<NotificationEmail>volker@mail.nih.gov</NotificationEmail>
<NotificationEmail>alan@mail.nih.gov</NotificationEmail>
<NotificationEmail>cdr.admin@mail.nih.gov</NotificationEmail>

In the FileSweeper.py script, modify the fatalError(msg) routine to log the error message, as it does now, then send email to all NotificationEmail addresses. I suggest that messages only be sent in the event of a fatal error, not upon successful completion.

Another possible change, tangentially related to this one, could be to store the config file in the database as a miscellaneous document. That would make it easy for us to make changes in emails and other things without needing to get help from CBIIT.

Why not use a CDR group for email notification, as we do elsewhere? As For storing the config file in the repository, wouldn't we need a separate document type in order to accommodate the elements this file uses?

Yes, I think we would want a document type.

This could be a schema controlled doc_type for FileSweeperConfig, or a more general schema for Config files, for example:

<element name = 'Config' type='Config'>
 <complexType name = 'Config'>
  <choice>
   <element name='FileSweeperConfig' type='FileSweeperConfig'>
   ...
  </choice>
 ...

Or alternatively we could make it one of the control types like css that is not schema validated - which has advantages. Then we could add new config files as needed without having to create more document types or editing a schema.

This frees us from requiring CBIIT support. Whether it's worth the trouble depends on how often we need to make changes to this and/or other config files.

I have added the code to send email to one or more people in the event of a fatal error. The error message will be mailed and logged instead of just logged as it was. The list of addresses to which mail is sent is also logged.

To enable emails, create the user group "FileSweeper Error Notification" in the CDR (I've created it on DEV) and check the boxes for desired users. This can also be overridden on the command line - which may or may not be of any value.

I made no changes to the actual FileSweeper operation and so did not test that. I only tested the email capability - by temporarily inserting an instruction in the code to abort with a fatal error.

The new code, including some minor formatting cleanups, is on DEV and in SVN.

I'm marking this as Resolved Fixed.

I've created the new FileSweeper Error Notification group on PROD.
Since we have no way of actively creating the conditions to make the job fail in order to test on PROD I'm going to close the ticket and reopen it later if it turns out the FileSweeper is not working properly.