CDR Tickets

These CGI scripts will be modified:

☐ AdvancedSearch.py
☐ CdrDocumentation.py
☐ CheckedOutDocs.py
☐ CheckUrls.py
☐ CitationReports.py
☐ CountrySearch.py
☐ DISSearch.py
☐ DocumentsModified.py
☐ DrugReviewReport.py
☐ ExternMapFailures.py
☐ GeneralReports.py
☐ GlossaryProcessingStatusReport.py
☐ GlossaryTermAudioReviewReport.py
☐ GlossaryTermPhrases.py
☐ GlossaryTermSearch.py
☐ Help.py
☐ HelpSearch.py
☐ InvalidDocs.py
☐ LinkedDocs.py
☐ MediaCaptionContent.py
☐ MediaLists.py
☐ MediaTrackingReport.py
☐ MenuHierarchy.py
☐ MiscSearch.py
☐ ModifiedPubMedDocs.py
☐ PoliticalSubUnitSearch.py
☐ PronunciationRecordings.py
☐ PubStatsByDate.py
☐ QcReport.py
☐ RecordingTrackingReport.py
☐ ReplaceCWDReport.py
☐ Request4333.py
☐ Request4486.py
☐ SemanticTypeReport.py
☐ Stub.py
☐ TermHierarchyTree.py
☐ TermUsage.py
☐ UnchangedDocs.py

The following scripts will be removed from the admin menus:

☐ AdHocQuery.py
☐ CdrQueries.py

I have completed the re-write of the 39 scripts which had to be modified for this report. Here are the paths for you to use when testing on DEV, William. Cosmetics may have changed, but functionality should be intact. I believe I have eliminated all of the security holes in these scripts (not just the ones found by the appscan).

I have a question about the script Charlie implemented for the Term Hierarchy Tree report. The logic for that report is fairly complicated, so it's probably better if we discuss the issue in person (face-to-face after next Thursday's meeting, or on the phone).

After you've finished testing I'll have these installed on Stage for a fresh appscan.

Thanks!

CIAT/OCCM Staff > Advanced Search
CIAT/OCCM Staff > Advanced Search > Country
CIAT/OCCM Staff > Advanced Search > Documentation
CIAT/OCCM Staff > Advanced Search > Drug Information Summary
CIAT/OCCM Staff > Advanced Search > Glossary Term
CIAT/OCCM Staff > Advanced Search > Miscellaneous
CIAT/OCCM Staff > Advanced Search > Political SubUnit
CIAT/OCCM Staff > Reports > Citations
CIAT/OCCM Staff > Reports > Citations > Management Reports > Modified PubMed Documents
CIAT/OCCM Staff > Reports > Documentation > CDR Documentation Categories > CDR Help
CIAT/OCCM Staff > Reports > Documentation > CDR Documentation Categories > Operating Instructions 
CIAT/OCCM Staff > Reports > Documentation > CDR Documentation Categories > System Information
CIAT/OCCM Staff > Reports > Drug Information > QC Reports > Advanced Search
CIAT/OCCM Staff > Reports > General Reports > Checked Out Documents
CIAT/OCCM Staff > Reports > General Reports > Documents Modified
CIAT/OCCM Staff > Reports > General Reports > External Map Failures Report
CIAT/OCCM Staff > Reports > General Reports > Invalid Documents
CIAT/OCCM Staff > Reports > General Reports > Linked Documents
CIAT/OCCM Staff > Reports > General Reports > URL Check (Batch job - runs ~15 min)
CIAT/OCCM Staff > Reports > General Reports > Unchanged Documents
CIAT/OCCM Staff > Reports > General Reports > Versions that Replaced CWDs
CIAT/OCCM Staff > Reports > Glossary Terms > Management Reports > Linked or Related Document Reports > Glossary Term Concept by Type Report
CIAT/OCCM Staff > Reports > Glossary Terms > Management Reports > Linked or Related Document Reports > Glossary Term and Variant Search Report
CIAT/OCCM Staff > Reports > Glossary Terms > Management Reports > Processing Reports > Processing Status Report
CIAT/OCCM Staff > Reports > Media > Management Reports > Media Caption and Content Report
CIAT/OCCM Staff > Reports > Media > Management Reports > Media Lists
CIAT/OCCM Staff > Reports > Media > Management Reports > Media Tracking Report
CIAT/OCCM Staff > Reports > Media > Other Reports > Audio Pronunciation Recordings Tracking Report
CIAT/OCCM Staff > Reports > Media > Other Reports > Audio Pronunciation Review Statistics Report
CIAT/OCCM Staff > Reports > Terminology > Other Reports > Drug Review Report
CIAT/OCCM Staff > Reports > Terminology > Other Reports > Menu Hierarchy Report
CIAT/OCCM Staff > Reports > Terminology > Other Reports > Semantic Type Report
CIAT/OCCM Staff > Reports > Terminology > Other Reports > Term By Type
CIAT/OCCM Staff > Reports > Terminology > Other Reports > Term Hierarchy Tree
CIAT/OCCM Staff > Reports > Terminology > Publication Reports > New Published Glossary Terms
CIAT/OCCM Staff > Reports > Terminology > QC Reports > Term Usage

The Documents Modified report (CIAT/OCCM Staff > Reports > General Reports > Documents Modified) comes up with the following python script error:

A problem occurred in a Python script.

D:\cdr\Log\tmpfxev4o.html contains the description of this error.

I had fixed that problem but somehow missed installing the fix. Should be OK now.

The Media Content and Caption report (CIAT/OCCM Staff > Reports > Media > Management Reports > Media Caption and Content Report
) also comes up with a python script error (below) after selecting options and submitting. It didn't matter which selections I made.

A problem occurred in a Python script.

D:\cdr\Log\tmpaf3opd.html contains the description of this error.

Fixed. Please give it another shot.

I have completed testing all the reports. They all appear to be working correctly now. The Term By Type report has not been implemented yet so I couldn't test it. About 3 of the paths are not exactly how we have them under the CIAT/OCCM report menu but I recognized all the reports so I was able to test them.

Thanks. Just bringing up the stub page for the Term By Type report was all the testing you needed to do for that one (the script for the stub page had been changed for this ticket).

There seems to be a discrepancy on the Documentation Page between Stage and DEV.

The path is:
CIAT/OCCM Staff > Reports > Documentation >

On DEV we have:

CDR Documentation Categories
1.CDR Help
2.Operating Instructions
3.System Information

But on Stage we have:

CDR Documentation (PDF) as of 2007-08-22
User Guide
System Documentation
Operation Manual
CDR Documentation (HTML) - Current
User Guide
System Documentation
Operation Manual

Meanwhile, all the reports on Stage do not work. The PDF reports don't work because Adobe Acrobat is currently not installed on Stage but the HTML reports don't work either.

First, you are aware the scan is planned for QA, not Stage, right?

I took out the PDF reports because (a) the version posted was ancient; (b) the PDFs weren't present on the lower tiers; and (c) Verdi, where we had the tools used to generate the PDFs, is gone.

Are the old PDFs currently used for anything?

For the HTML version, we had decided a good while back that it would be best for users to get the current help pages from the production tier, as it appeared that updates to that documentation were being made on PROD, but not propagated down to the lower tiers. That approach doesn't work any more, not that we're hosted in an environment where the servers on the different tiers can't talk to each other. So we've had to revert to an approach to documentation which (a) applies updates from the lower tiers upwards, as we do for all other changes to the software; (b) installs those updates in the version control system; and (c) pulls from the local tier when the user asks for the help pages. That last change will be applied to Stage when Alan's scripts are tested on that tier.

Yes. I am currently testing on QA but QA is very slow so it might take me a while to complete testing. Thanks for the explanation.

I believe that one of the things I wrote in my previous comment was wrong: the Help pages are not under version control (CDR versioning fills that role).

How about the question about whether we still need the PDFs?

I believe we do not need the PDFs. We almost never access them.

App scan has been requested for QA.

Alan: I've assigned this issue to you, to address the additional XXXSearch.py XSS problems Mikol says the scanner found. When I did the first sweep, I considered rewriting the advanced search pages to use the new Page class, which ensures that all the non-static data is scrubbed by the HTML builder module, but decided not to take the time. That may have been the wrong decision. Take a look and decide what the best path to take will be, and let Mikol know when you'll have something for him to re-scan.

I decided to fix the problem revealed by the Appscan by changing a single line in cdrcgi.py. I think it will block XSS attacks (at least of the identified type) for any of the Advanced Search pages. More info is in SECURITYTEAM-368.

Once these issue fixes land safely on PROD, we'll want to remember to create a tag for the revision of trunk which represents the state of PROD at that point.

Please file a ticket with CBIIT today to get this on their schedule (coordinating with the users).