CDR Tickets

Issue Number 3188
Summary Fix FOIA tool documentation
Created 2010-07-06 08:53:29
Issue Type Bug
Submitted By Kline, Bob (NIH/NCI) [C]
Assigned To Englisch, Volker (NIH/NCI) [C]
Status Closed
Resolved 2014-12-03 18:52:38
Resolution Fixed
Path /home/bkline/backups/jira/ocecdr/issue.107516
Description

BZISSUE::4875
BZDATETIME::2010-07-06 08:53:29
BZCREATOR::Bob Kline
BZASSIGNEE::Volker Englisch
BZQACONTACT::Bob Kline

Please update the online docs for the FOIA tool used to capture weekly snapshots of cancer.gov (some pages are out of date - referring to "public" share, and at least one other - "Creating FOIA Data" - is missing).

Comment entered 2010-07-16 15:44:33 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2010-07-16 15:44:33
BZCOMMENTOR::Volker Englisch
BZCOMMENT::1

Comment entered 2010-09-08 16:19:11 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2010-09-08 16:19:11
BZCOMMENTOR::Volker Englisch
BZCOMMENT::2

The following documents need to get updated:

CDR360452 - Copy FOIA Data from FOIA to UNIXSAG Directory
CDR360458 - Create Cancer.gov FOIA Backup Data
CDR360454 - Create FOIA Test/Production DVD
CDR354513 - Extracting FOIA Data
CDR360450 - Test the FOIA Data on File/DVD

Comment entered 2010-09-08 16:21:10 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2010-09-08 16:21:10
BZCOMMENTOR::Volker Englisch
BZCOMMENT::3

The following documents have been updated:

[ ] CDR360452 - Copy FOIA Data from FOIA to UNIXSAG Directory
[X] CDR360458 - Create Cancer.gov FOIA Backup Data
[ ] CDR360454 - Create FOIA Test/Production DVD
[ ] CDR354513 - Extracting FOIA Data
[ ] CDR360450 - Test the FOIA Data on File/DVD

Comment entered 2010-09-08 16:49:45 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2010-09-08 16:49:45
BZCOMMENTOR::Volker Englisch
BZCOMMENT::4

The following documents have been updated:

[ ] CDR360452 - Copy FOIA Data from FOIA to UNIXSAG Directory
[X] CDR360458 - Create Cancer.gov FOIA Backup Data
[ ] CDR360454 - Create FOIA Test/Production DVD
[X] CDR354513 - Extracting FOIA Data
[ ] CDR360450 - Test the FOIA Data on File/DVD

Comment entered 2010-09-08 17:06:02 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2010-09-08 17:06:02
BZCOMMENTOR::Volker Englisch
BZCOMMENT::5

The following documents have been updated:

[X] CDR360452 - [Outdated] Copy FOIA Data from FOIA to UNIXSAG Directory
[X] CDR360458 - Create Cancer.gov FOIA Backup Data
[X] CDR360454 - [Outdated] Create FOIA Test/Production DVD
[X] CDR354513 - Extracting FOIA Data
[X] CDR360450 - [Outdated] Test the FOIA Data on File/DVD

Comment entered 2010-09-14 17:34:07 by Kline, Bob (NIH/NCI) [C]

BZDATETIME::2010-09-14 17:34:07
BZCOMMENTOR::Bob Kline
BZCOMMENT::6

I reviewed the documents which aren't obsolete and fixed a couple of typos. As soon as a decision has been made (and documented, including replacing "MSD" as the company responsible for storing the off-site fire copy) as to which group is responsible for creating the archive copies, this issue can be closed.

Comment entered 2010-11-29 13:37:21 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2010-11-29 13:37:21
BZCOMMENTOR::Volker Englisch
BZCOMMENT::7

I had send an email to Ravee on Wednesday asking for a meeting to discuss the FOIA process and storage.

Comment entered 2011-05-25 15:33:41 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2011-05-25 15:33:41
BZCOMMENTOR::Volker Englisch
BZCOMMENT::8

I had talked to Carbie last week and it appears that we will need to change the documentation again due to the changes the ops-team is putting in place in order to back up all of the CDR data.
I'll work with Carbie to document the necessary changes.

Comment entered 2011-08-10 18:31:09 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2011-08-10 18:31:09
BZCOMMENTOR::Volker Englisch
BZCOMMENT::9

I installed DVD burning software on my machine and will start copying the weekly FOIA to create an in-house and a fire-copy of the data sets.

Comment entered 2011-08-24 12:31:20 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2011-08-24 12:31:20
BZCOMMENTOR::Volker Englisch
BZCOMMENT::10

I'm still burning the first set of DVDs.

Comment entered 2011-09-30 12:29:33 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2011-09-30 12:29:33
BZCOMMENTOR::Volker Englisch
BZCOMMENT::11

I created the first set of 70 DVDs. I will need to install the system next to label these DVDs.

Comment entered 2011-11-18 14:02:31 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2011-11-18 14:02:31
BZCOMMENTOR::Volker Englisch
BZCOMMENT::12

Carbie is still trying to find the software to setup the DVD label printer. If we can't find it I will probably dig out the Sharpie and label by hand.

Comment entered 2012-02-14 17:39:55 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-02-14 17:39:55
BZCOMMENTOR::Volker Englisch
BZCOMMENT::13

The last comment is long outdated. The label printer has been surplused since nobody had used it in the last two years and without the software to install on a workstation is is worthless.

I need to have a discussion with Lakshmi regarding the future FOIA processes and tasks.

Comment entered 2012-09-27 11:41:59 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-09-27 11:41:59
BZCOMMENTOR::Volker Englisch
BZCOMMENT::14

Giving this task its rightful priority.

Comment entered 2013-08-09 18:16:01 by Englisch, Volker (NIH/NCI) [C]

As part of the move to Shady Grove the FOIA data has been cataloged and it is planned to make all the FOIA data available online and be backed through CBIIT. Mauricio is currently investigating the storage options.

Comment entered 2013-09-23 11:23:33 by Kline, Bob (NIH/NCI) [C]

Do we need to nudge Mauricio?

Comment entered 2013-09-23 17:08:09 by Englisch, Volker (NIH/NCI) [C]

I did that today.

Comment entered 2013-09-27 10:14:23 by Englisch, Volker (NIH/NCI) [C]

Mauricio is still in contact with the CBIIT team to get the appropriate storage assigned in order to make the data available online.

Comment entered 2013-12-17 11:14:08 by Englisch, Volker (NIH/NCI) [C]

I keep pinging Mauricio who is working with the storage team to move this along but there's nothing else I can do until those two entities have hashed out everything.
Until that time I'm setting the status to on-hold.

Comment entered 2014-03-14 13:41:42 by Englisch, Volker (NIH/NCI) [C]

We received information that we will have access to a > 1TB disk drive to store the FOIA data. Currently waiting on CBIIT to provision that drive.

Comment entered 2014-11-05 11:12:52 by Kline, Bob (NIH/NCI) [C]

Where do we stand on this, Volker?

Comment entered 2014-11-05 15:04:47 by Englisch, Volker (NIH/NCI) [C]

Long story. The short version is this:
It was planned to get this disk drive to store the weekly and the past FOIA files and a process would be put in place to have the data automatically backed up. We received the disk drive (but it's too small to hold all of the FOIA data) and nothing has been put in place to automate the process.

Comment entered 2014-11-10 16:40:19 by Englisch, Volker (NIH/NCI) [C]

I will update the documentation to reflect the current situation and then close the issue rather than waiting for the next phase of the process to be completed.

Comment entered 2014-11-20 13:21:28 by Englisch, Volker (NIH/NCI) [C]

While I was working on the FOIA request for Margaret I got pretty annoyed trying to find the correct PDQ-C version on the CDs/DVDs stored in our filing cabinets. Many CDs exist in multiple versions and are hard to find, some of the CDs are incorrectly labeled, some of the CDs were not readable anymore and some of the data on the CDs are incorrectly labeled.
I decided to copy all of the PDQ-C data to the FOIA network drive which has the side-effect that the data gets backed up for off-site storage (hopefully).
We now have the PDQ-C data from Apr. 1994 to Dec. 2002 stored under the directory

\\ncifs-p102-n\CDR_FOIA\FOIA_Data-PDQ-C
Comment entered 2014-12-03 18:52:24 by Englisch, Volker (NIH/NCI) [C]

I have updated the documentation explaining how to create and extract FOIA data. Since Bob and Alan are the only other persons with access to the FOIA network drive they would have to check the documentation for accuracy. The links to the CDR documentation are listed below.

https://cdr.cancer.gov/cgi-bin/cdr/Filter.py?DocId=CDR0000360458&Filter=name:Documentation+Help+Screens+Filter
https://cdr.cancer.gov/cgi-bin/cdr/Filter.py?DocId=CDR0000354513&Filter=name:Documentation+Help+Screens+Filter

Comment entered 2014-12-03 18:53:15 by Englisch, Volker (NIH/NCI) [C]

Adding Alan as watcher.

Elapsed: 0:00:00.001885