CDR Tickets

BZDATETIME::2012-12-21 16:09:52
BZCOMMENTOR::Volker Englisch
BZCOMMENT::1

The three user accounts have been created on the FTP server
venglisch
cdroperator
testftp

URL = caftps2.nci.nih.gov
Port = 22
Home = /CDR (chrooted environment)

BZDATETIME::2012-12-21 16:11:12
BZCOMMENTOR::Volker Englisch
BZCOMMENT::2

Per discussion with Margaret and Lakshmi we will ask CBIIT to create a CNAME entry of
cancerinfo.nci.nih.gov
in order to connect to the FTP server.

BZDATETIME::2013-01-04 18:46:57
BZCOMMENTOR::Volker Englisch
BZCOMMENT::3

I'm in the process to setup a NFS share created by CBIIT in order to copy our PDQ licensee data to this share. That location will than be accessed by our licensees via SFTP.

BZDATETIME::2013-01-04 18:49:31
BZCOMMENTOR::Volker Englisch
BZCOMMENT::4

Created spreadsheet listing all of our currently active (i.e. active and test) licensees accounts. The document has been forwarded to CBIIT so that they can start creating the accounts on our new CBITT-FTP server
hostname: cancerinfo.nci.nih.gov

Attachment CBIIT_Licensees.xls has been added with description: Licensee FTP Accounts

BZDATETIME::2013-01-07 19:54:44
BZCOMMENTOR::Volker Englisch
BZCOMMENT::5

We've been able to set up the NFS share between CBIIT's SFTP server and our CIPSFTP server. The CBIIT share is currently mounted at
/u/ftp/cdr
There will be sub-directories pub/pdq, qa, and test/pdq to this directory and a licensees home directory will be setup to become
/u/ftp/cdr/pub/pdq
or - visible to the licensee once logged in -
/pdq

I'm in the process of identifying the best way to copy data to this directory because I currently don't have permission to copy an entire directory.

BZDATETIME::2013-01-07 19:56:03
BZCOMMENTOR::Volker Englisch
BZCOMMENT::6

The first notification message to the licensees indicating the move to the CBIIT SFTP server has been submitted.

BZDATETIME::2013-01-09 10:53:22
BZCOMMENTOR::Volker Englisch
BZCOMMENT::7

Attachment Licensee_Notification_2012-01-07.txt has been added with description: First Licensee Notification

BZDATETIME::2013-01-11 17:39:22
BZCOMMENTOR::Volker Englisch
BZCOMMENT::8

I've created two additional scripts on our FTP server.
copyData2Cbiit.py
copyOther2Cbiit.py
The first one copies licensee data from the CIPSFTP production directory
/u/ftp/pub/pdq/full
to the CANCERINFO production directory
/u/ftp/cdr/pub/pdq/full

The second one copies the Terminology data for the NCICB from CIPSFTP to CANCERINGO:
/u/ftp/pub/ncicb
to
/u/ftp/cdr/pub/ncicb

The job control program Jobmaster.py has been modified to run the first of the scripts as part of the weekly publishing and the second one as part of the nightly publishing.

I'm currently in the process of testing the modified programs with a test publishing job on FRANCK.

BZDATETIME::2013-01-17 12:16:34
BZCOMMENTOR::Volker Englisch
BZCOMMENT::9

Everything looks good with the new setup. I will add the new scripts to the Jobmaster steps and start updating the directory on the new SFTP server tonight.

The following scripts have been created/updated:
copyData2Cbiit.py - R11203
copyOther2Cbiit.py - R11144
Jobmaster.py - R11204

The licensees will receive a second email reminder on Monday and Tuesday I will have to contact each of them to provide them with their username and password for the new server.

BZDATETIME::2013-01-21 18:48:08
BZCOMMENTOR::Volker Englisch
BZCOMMENT::10

The second licensee notification has been send out including instructions on the new server name and document location.
I will start submitting user accounts and passwords tomorrow.

Attachment Announcement2.txt has been added with description: Second Licensee Notification

BZDATETIME::2013-01-23 19:11:15
BZCOMMENTOR::Volker Englisch
BZCOMMENT::11

All (minus one) licensees have been contacted to provide them with their username and password on the new SFTP server.

One licensee needs to be added (new test licensee) and for one licensee I'll have to double-check if the user ID is correct.

BZDATETIME::2013-02-27 12:11:38
BZCOMMENTOR::Volker Englisch
BZCOMMENT::12

As we all know Terese is uploading her images to the FTP server from where they are picked up and copied to BACH, our production server.
With our move to CBIIT, the security team will not allow us to copy the image files to the production server, so we'll have to come up with a different approach of getting the files.

When the program was created the idea was to FTP the image files from the FTP server and copy those to a network drive. However, the approach failed because of permission problems that we weren't able to resolve. We implemented the download such that the images were downloaded to BACH and Margaret would manually copy the files to the group drive. In addition, the program moves the files on the FTP server to a different location so that the images download program would only grab new image files.

One solution to the CBIIT security restrictions could be the use the program FileZilla as a tool to download/preview files. The image review team could either use FileZilla to directly copy files from the new SFTP server to a network drive (an intermediary storage on BACH wouldn't be necessary) or view the image files from FileZilla using "Right-click view" without copying.
This approach would not copy the files on the FTP server to a different location, which was done for CIAT and I was wondering how much of an affect this change would have on the work CIAT is performing.

BZDATETIME::2013-05-09 14:49:31
BZCOMMENTOR::Robin Juthe
BZCOMMENT::13

Revised priority based on the discussion in today's CDR meeting.