CDR Tickets

Issue Number 4131
Summary Modify Monthly PDQ Access Report
Created 2016-07-08 13:55:24
Issue Type Improvement
Submitted By Englisch, Volker (NIH/NCI) [C]
Assigned To Englisch, Volker (NIH/NCI) [C]
Status Closed
Resolved 2016-07-08 14:30:07
Resolution Fixed
Path /home/bkline/backups/jira/ocecdr/issue.187777
Description

Our monthly PDQ access report runs on the first of every month using the log-file automatically created by logrotate on the FTP server. Due to a problem on the server the monthly log-file didn't get created until the 5th of July and the scheduled job failed.

Since the log-file was created 5 days late it now contains log data from June and July. In order to run the access report on just the June data the program will have to be modified in order to specify an alternate location for the log-file.

Comment entered 2016-07-08 13:59:45 by Englisch, Volker (NIH/NCI) [C]

Since the location of the log-files is read-only we're not able to create a cleaned-up copy of the log-file in the directory

  /local/pdq_log

I modified the program to allow "redirecting" the location of the log-file to

  /usr/local/cdr/ftplog

by specifying a new command line option

$ PDQAccessReport.py --livemode --localdir
Comment entered 2016-07-08 14:27:30 by Englisch, Volker (NIH/NCI) [C]

The following program has been updated in subversion (trunk):

  • R14091: PDQAccessReport.py

Comment entered 2016-11-09 13:56:23 by Kline, Bob (NIH/NCI) [C]

Story points only reflect the work that remains to be done.

Comment entered 2017-01-11 18:53:10 by Englisch, Volker (NIH/NCI) [C]

I had to make a couple more changes to this report:

  1. The new option was named --uftpdir because I thought originally I would process the files at

    /u/ftp/cdr/

    However, alternate processing is now done from

    /usr/local/cdr/ftplog

    and it seemed silly to keep the option name uftpdir, I've renamed the option to --localdir.

  2. testmode was a valid option to run tests but the option wasn't implemented. Now the output is written to a test file (i.e. PDQPartnerDownloads_test_2016-12.csv) when using this option, it won't overwrite an existing file that's not a test and it's submitted to a developer.

  3. Enhanced log file output

  4. The email with the attached report will now be send to the PDQ Dissemination mailbox instead to myself.

The changes have been tested and are live since the report is run from the DEV server.

  • R14481: PDQAccessReport.py

Elapsed: 0:00:00.001287