CDR Tickets

Issue Number 3699
Summary Create Licensee Access Report
Created 2014-01-15 18:40:46
Issue Type New Feature
Submitted By Englisch, Volker (NIH/NCI) [C]
Assigned To Englisch, Volker (NIH/NCI) [C]
Status Closed
Resolved 2014-07-25 17:14:14
Resolution Fixed
Path /home/bkline/backups/jira/ocecdr/issue.117249
Description

The UNIX team was able to provide me access to the FTP log files which will allow me to create a report identifying the licensees who are accessing the PDQ data from the FTP server.

Comment entered 2014-01-17 17:50:21 by Englisch, Volker (NIH/NCI) [C]

As Bob suggested I created a cronjob that extracts the content of the log file and saves a week worth of data locally. The cronjob runs every Wednesday and copies data for the previous week saving it on DEV under

/usr/local/ftplog/YYYYWW.log

The program is called

archiveFtpLogs.py
Comment entered 2014-07-21 18:48:26 by Englisch, Volker (NIH/NCI) [C]

Since we started this ticket the format for the logfile has changed and I had to rewrite the program I had written. That old program is called

~cdroperator/prod/bin/licenseeReport.py

Also, the program archiveFtpLogs.py may not be needed anymore because CBIIT now creates a monthly log file. It is located at

/local/pdq_log/pdq.log-YYYYMM01.gz

I've created a new program to extract the information for files downloaded by PDQ licensees and creating a report in the directory

/usr/local/cdr/report

The process to create a report follows these steps:

  • On the first of the month CBIIT creates the log file for the previous month.

  • Later that day a cronjob copies that log file to the DEV FTP server at

    /usr/local/cdr/ftplog

  • The program to create the report is started via a cronjob

    ~cdroperator/prod/bin//PDQAccessReport.py

  • The report output is stored under

    /usr/local/cdr/report/LicenseeDownloads_YYYY-MON.csv

    and submitted via email.

The program supports a few options:

PDQAccessReport.py [options]

--testmode  to suppress sending emails to the distribution list
--livemode  for normal processing
--noemail   to suppress sending the email message after processing
--sendagain to suppress processing and only re-send the existing report

This is almost completed except for a few more test runs.

Comment entered 2014-07-25 17:10:11 by Englisch, Volker (NIH/NCI) [C]

I forgot to include the version of the program

  • R12731: PDQAccessReport.py

Comment entered 2014-07-25 17:14:14 by Englisch, Volker (NIH/NCI) [C]

I will close the issue after the next version of the report has been created as part of the regularly scheduled cronjob (August 1st).

Comment entered 2014-08-11 14:48:10 by Englisch, Volker (NIH/NCI) [C]

The report ran but failed because I forgot to include a path to a file name. This has been fixed with

  • R12797: licenseeNames.txt

  • R12798: PDQAccessReport.py

The modified report ran successfully. Closing issue.

Elapsed: 0:00:00.001479