CDR Tickets

From the CDR status meeting: We'll assign this to Alan because he's got a better solution. He plans to investigate the use of a more general-purpose report, which lists all of the internal links in the document, ordered by a string representing the linked fragment (the algorithm for determining how those string will be derived is yet to be determined). We'll hold off on doing too much work on this task at least until Margaret is back in town to participate in the discussions.

We discussed this issue at some length in today's CDR status meeting. Here
are some of the issues and ideas that we discussed:

1. Should the report be specific or general?

A specific report would address the specific data elements for which
examples were presented in the issue Description.

A general report would examine all of the link sources and targets in
a document and display information about all of the link targets
along with information about all of the source links pointing to
them.

We quickly realized that a fully general report would be too noisy.
It would produce output, for example, for hundreds or thousands of
citations which would clutter the report with information that no one
is interested in seeing. So if the report is generalized, it would
have to either:

a. Include scoping rules that include specific elements to examine
and then only examine those elements.

or:

b. Include scoping rules that exclude specific elements from the
report.

Either approach would enable us to use the report more broadly than
would be the case if it was written to only examine one hard wired
set of relationships. However a. looks more manageable to me than b.
It would require less experimenting and tuning.

It seems to me that a general report could do what needs to be done
as well as a specific report (though I might be wrong about that),
and that the main determinant of whether or not to write the report
in a general way should be the level of effort. If it's not harder,
or not much harder, to make the report general then that's what we
should do.

2. Should the report be written to support references to external
documents?

I think the ability to report on references to external documents
would not be much harder to write if we don't have to include data
from those external references. If we do, then it adds to the
complexity.

My suspicion is that this inter-document level of generality is less
important than the intra-document level of generality assumed in the
issue Description.

3. What should the output contain?

Bob and Robin discussed output in the form of a spreadsheet with
columns like:

Target text
Target id
Source text
Source id
Section identifier (title?)
XML path of the source
XML path of the target

This would enable the user to sort the output in different ways.

It was also suggested that the output be separated by PDQ Board so
that each board manager (the users of the report) could find her own
data without weeding through data for other boards. This might be
done by making board a column and sorting on it, or by putting each
board's data on a separate spreadsheet tab.

I think that we need to define the output fully and include specific
examples for all of the above spreadsheet columns before we can
decide whether a general or specific report is best and estimate the
level of effort.

4. What is the priority of this task? Should it be in the next release?

Implementing this report looks like it could be time consuming. Our
sense at the meeting was that we should make the priority low so
that, if time permitted, it would be included in the next release.
But if we ran out of time it would be held over for the patch or
release after this one.

Here's a pair of bare-bones examples of what can be done easily to support checking document links within the summary. Let's talk about what else might be needed for the immediate use case. After we get that nailed down we can consider options to make the report more general.

Please capture any changes you decided you want here and I'll implement them.

I have attached a 2-page mockup of the suggested report from CIAT. The board counterparts said they would prefer a general purpose report rather than one that is limited to certain sections and subsections. But eventually if they are able to select specific sections/subsections to run the report for just those sections and subsections, that would be great.

We agreed to make some changes to the table column headings during the status meeting. I'm attaching an updated copy. Bob is going to do a test run pulling in the first 200 characters of the target text.

This is the 7-column version, three columns for the target (linked) information, four columns for the source (linking) information; third column is truncated to the first 200 characters.

Here's a version which makes it more obvious which links point to the same target.

William:

If you have further tweaks you would like to see in the format of the report, it would be good to let me know in time to made the modifications before we sit down to look at it together in this afternoon's status meeting.

The only tweak I have is to drop the text column we limited to 250 characters. The column will not be useful. Also, users prefer the names of the columns they suggested. We may have to consider putting the names of the columns users had suggested in parenthesis.

https://cdr.dev.cancer.gov/cgi-bin/cdr/ocecdr-3650.py?Session=guest&DocId=62864

User-requested column headers restored.

This is awaiting review on DEV.

The column headers do not reflect the mockup I posted. I believe the file you need to use is the one dated : Thursday, 19 Sep 2013 11:53 AM.

OK, I have rewritten the report to use the column headers from that older attachment. I would personally find these column headers more confusing, but I'm not the one who will be using the report. Please review and confirm that I've done it the way you want.

Could you please make minor changes to the following column headings by adding 'Ref' to the titles?

Section/Subsection Containing Fragment Ref | Text in Fragment Ref
+++ +++

Column header changes implemented on DEV; please review.

Verified on Dev. Could you put it on the admin menu under CIAT/OCCM Staff > Reports > Summary and Miscellaneous Documents > Management Reports?

Added to menu. I assumed you mean "Management QC Reports," right?

That is right. Verified on Dev. It looks good. Thank you!

I couldn't run this report from the admin menu on QA. I get the following python script error:

A problem occurred in a Python script.

D:\cdr\Log\tmpurpm14.html contains the description of this error.

Missing a Python package. Should work now. We'll need to include that package in the CBIIT deployment instructions.

Yes. It is working now. Thanks!

Verified on QA.

I am getting a python script error for this one. I tried both IE and FF but got the same error:

A problem occurred in a Python script.

D:\cdr\Log\tmpidp0ww.html contains the description of this error.

You need to invoke it with the same ID format you used successfully on the lower tiers (without the "CDR0000" part).

That worked.Thanks.
Verified on Prod.

I'm only able to get this to run on Firefox. Is that to be expected? I'm getting errors with Excel when I use IE.

We've still got an open ticket for this problem: https://tracker.nci.nih.gov/browse/WEBTEAM-1237 (I just posted a couple of screen shots to that ticket, illustrating the problem as it relates to this report).