CDR Tickets

Issue Number 4734
Summary [Internal] Rewrite CGI scripts using legacy cdrcgi.header() function
Created 2019-11-23 11:47:34
Issue Type Improvement
Submitted By Kline, Bob (NIH/NCI) [C]
Assigned To Kline, Bob (NIH/NCI) [C]
Status Closed
Resolved 2020-04-27 14:30:04
Resolution Fixed
Path /home/bkline/backups/jira/ocecdr/issue.252780
Description

In the earliest days of the CDR, the available tools for generating and parsing XML and HTML were less capable than they are now, and - partly for this reason - our scripts for building and handling HTML forms and reports used a more error-prone and fragile approach, building up HTML strings by hand, and using regular expressions for parsing them. This also resulted in code whose handling of Unicode strings was not correct. Many of the scripts used the function header() from the cdrcgi module to create the top portion of an HTML form or report page, filling in the rest with markup created by hand. We have since created and refined more robust classes for this purpose, and as part of Kepler, all but five of the 64 CGI scripts which used that function and approach were rewritten. There wasn't sufficient time to rewrite them all, and the purpose of this ticket is to address the rewriting of the remaining five scripts.

  • PubStatus.py

  • ReplaceCWDwithVersion.py

  • SummaryDateLastModified.py

  • SummaryMailerReport.py

  • TermHierarchyTree.py

Comment entered 2020-04-21 14:33:02 by Kline, Bob (NIH/NCI) [C]

is the Summary Mailer report still used, now that we no longer create summary mailers?

Comment entered 2020-04-22 07:25:02 by Kline, Bob (NIH/NCI) [C]

and : for the summary date last modified report, is it necessary to keep the current format, using custom code to pile all the tables into a single worksheet, or can we use the common framework for Excel reports, putting each table on its own tab?

Comment entered 2020-04-22 10:37:04 by Osei-Poku, William (NIH/NCI) [C]

We use this report for our monthly and annual reports. Having all the tables in a single worksheet makes it easier to spot variations and trends. So, that would be our preference.

Comment entered 2020-04-23 12:21:09 by Kline, Bob (NIH/NCI) [C]

Shall I preserve the other anomalies as well, like labeling the summary type column as "Board" (so, for example, Peds summaries have a "board" type of "treatment"), and labeling the audience column as "Type"?

Comment entered 2020-04-23 13:09:58 by Juthe, Robin (NIH/NCI) [E]

Yes, we're still using this report for tracking Advisory Board reviews with mailer IDs.

Comment entered 2020-04-23 13:12:35 by Juthe, Robin (NIH/NCI) [E]

I don't see where these anomalies are - are you still referring to the summaries DLM report?

Comment entered 2020-04-23 13:19:09 by Kline, Bob (NIH/NCI) [C]

Yes, those are in the "system" flavor of the report.

Comment entered 2020-04-23 13:30:31 by Juthe, Robin (NIH/NCI) [E]

Ah, I see it now. I think we should change those column headings to match the content below. "Type" for the summary type information and "Audience" for HP/PAT. Thanks!

Comment entered 2020-04-23 23:34:03 by Kline, Bob (NIH/NCI) [C]

Just out of curiosity, what purpose does the "Audience" column serve, given that that entire block has the same audience, which is displayed right above the block?

Comment entered 2020-04-27 14:30:04 by Kline, Bob (NIH/NCI) [C]

I'm going to mark this ticket as "resolved" so you can look at the reports and make sure I haven't broken anything, but I never did get an answer to my last question.

Comment entered 2020-04-28 16:56:51 by Juthe, Robin (NIH/NCI) [E]

It's redundant, I agree, but it does make it a little easier to quickly identify what you're looking at when scanning the document.

Comment entered 2020-05-13 15:17:58 by Osei-Poku, William (NIH/NCI) [C]

For the Summary Date Last Modified Report, please append the summary title with [Module] when it is included in the results, just like it is on PROD.

Comment entered 2020-05-13 16:22:36 by Kline, Bob (NIH/NCI) [C]

I can do that, but first let me make sure you're aware that "module" in this report means something different (available as a module) than it does for other reports (module only).

Comment entered 2020-05-14 10:17:34 by Osei-Poku, William (NIH/NCI) [C]

The Term Hierarchy Tree report produces the following error message when trying to access it:

 

Traceback (most recent call last): File "D:\inetpub\wwwroot\cgi-bin\cdr\TermHierarchyTree.py", line 321, in <module> Control().run() File "d:\cdr\lib\Python\cdrcgi.py", line 150, in init self.logger.info("started %s", self.subtitle or "controller") File "d:\cdr\lib\Python\cdrcgi.py", line 699, in logger self.logger = cdr.Logging.get_logger(self.LOGNAME, **opts) File "d:\cdr\lib\Python\cdr.py", line 4070, in get_logger handler = logging.FileHandler(path, encoding="utf-8") File "D:\Python\lib\logging_init.py", line 1087, in init StreamHandler.init(self, self.open()) File "D:\Python\lib\logging_init.py", line 1116, in _open return open(self.baseFilename, self.mode, encoding=self.encoding) PermissionError: [Errno 13] Permission denied: 'D:
cdr
Log
TermHierarchyTree.log'

D:\cdr\Log\tmpaw4bq24a.html contains the description of this error.

Comment entered 2020-05-14 10:41:20 by Osei-Poku, William (NIH/NCI) [C]

Yes, it is well understood. Thanks!

Comment entered 2020-05-14 11:03:52 by Kline, Bob (NIH/NCI) [C]

Sorry about that. This is a problem with the lower tiers which was introduced by the Windows server upgrades. When we reported it to CBIIT, the response came down to "too bad, deal with it." At least it's just on the lower tiers, where we can fix it ourselves when it happens (which I have done). Please try again.

Comment entered 2020-05-14 11:43:22 by Kline, Bob (NIH/NCI) [C]

Label restored.

Comment entered 2020-05-14 13:08:30 by Osei-Poku, William (NIH/NCI) [C]

Verified. Thanks!

Comment entered 2020-05-14 18:27:59 by Osei-Poku, William (NIH/NCI) [C]

Verified on DEV. Thanks! We have one more item to test  - Replace CWD with Version report.

Comment entered 2020-05-18 17:37:59 by Osei-Poku, William (NIH/NCI) [C]

Verified on DEV. Thanks!

Comment entered 2020-06-15 10:44:08 by Osei-Poku, William (NIH/NCI) [C]

Verified on QA. Thanks!

Comment entered 2020-08-25 18:53:07 by Osei-Poku, William (NIH/NCI) [C]

Verified on PROD. Thanks!

Elapsed: 0:00:00.000640