CDR Tickets

My proposed approach is to have string values with names JQUERY, JQUERY_UI, and JQUERY_CSS in the cgi control table group for the CDN resources used in the HTMLPage class of cdrcgi.py, and JSON values with names GLOSSARY_SCRIPT and GLOSSARY_CSS in the pp control table group for glossary publish preview.

Direct use of the class-level constants would be replaced with instance properties which use the values from the control table if they are found, falling back on the class-level constants otherwise.

https://ajax.googleapis.com/ajax/libs/jquery/3.5.1/jquery.min.js

https://ajax.googleapis.com/ajax/libs/jqueryui/1.12.1/jquery-ui.min.js

https://ajax.googleapis.com/ajax/libs/jqueryui/1.12.1/themes/smoothness/jquery-ui.css

{code:title=GLOSSARY_SCRIPTS|JavaScript}
[
[
"https://ajax.googleapis.com/ajax/libs/jquery/3.5.1/jquery.min.js",
false
],
[
"https://ajax.googleapis.com/ajax/libs/jqueryui/1.12.1/jquery-ui.min.js",
true
],
[
"https://cdnjs.cloudflare.com/ajax/libs/jplayer/2.9.2/jplayer/jquery.jplayer.min.js",
false
],
[
"https://www.cancer.gov/app-modules/glossary-app/glossary-app.v1.2.2/static/js/main.js",
true
],
[
"https://www.cancer.gov/profiles/custom/cgov_site/themes/custom/cgov/gcov_common/dist/js/Common.js",
true
]
]

{code:title=GLOSSARY_CSS|JavaScript}
[
  "https://www.cancer.gov/profiles/custom/cgov_site/themes/custom/cgov/cgov_common/dist/css/Common.css",
  "https://www.cancer.gov/app-modules/glossary-app/glossary-app.v1.2.2/static/css/main.css"
]

Does this seem reasonable, ~volker?

Using JSON values because we (may) have multiple URLs to include/change?  Why are you using two different formats for the two JSON values?

Yes, this looks like it will do what we were looking for.

Using JSON values because we (may) have multiple URLs to include/change?

Yes, for glossary publish preview. For the HTMLPage class, we have only three values to keep track of. For glossary PP we have lots more libraries.

Why are you using two different formats for the two JSON values?

Because for the glossary PP javascript libraries we need to keep track of the defer flag.

~volker A slightly more sophisticated approach would use JSON for all the values. That would allow us to add other attributes to the elements. So, for example

 [
  {
    "src": "https://code.jquery.com/jquery-3.6.0.min.js",
    "integrity": "sha256-/xUj+3OJU5yExlq6GSYGSHk7tPXikynS7ogEvDej/m4=",
    "crossorigin": "anonymous"
  },
  {
    "src": "https://code.jquery.com/ui/1.13.0/jquery-ui.min.js",
    "integrity": "sha256-hlKLmzaRlE8SCJC1Kw8zoUbU8BxA+8kR3gseuKfMjxA=",
    "crossorigin": "anonymous"
  }
]

... lets us support more secure links to the resources. The downside of JS is that it requires a little more care in editing, but less (for example) than is needed for the much more complex ElasticSearch values. What do you think?

If this modified approach gives us additional functionality that will come in handy at some point I would not worry about "requires more care in editing"!  We should be allowed to always require care in editing and it's not like we're editing these values every other week.

I would, however, need to better understand your latest example as I can't see how it translates to the samples in your first comment.

Does this help?

Not really because I still don't see how your latest example relates to one of the examples given in the first comment.  If you were telling me that in those examples of the first comment the values for the integrity and crossorigin would be empty I think I might have a better understanding.

How about if I word it like this?

The syntax for my earlier example only supported a known set of attributes: "src" and "defer" (using the first (string) value in the list for the "src" attribute and the second (boolean) value in the list to figure out whether to add the "defer" attribute. The new syntax allows for any arbitrary attributes, so if we want to add "integrity" or "crossorigin" we don't have to modify the code to do that. The code would look something like:

for attrs in self.SCRIPT:
    element = self.B.SCRIPT()
    for name, value in attrs.items():
        if value:
            element.set(name, value)
        else:
            element.set(name)

GTN PublishPreview links now are populated on DEV from the values in the ctl table if present. Here's a sample of the values, as viewed in the admin interface:

~volker: Not something this ticket will do anything about, but I did notice while I was scouring through the code working on the ticket, that filter CDR0000000079 has a hard-coded link to a very old version of jQuery (1.4 was released a dozen years ago). The scans haven't caught this so far, but you might want to update the link anyway.

Ditto CDR0000000153 and CDR0000000158. And for a real blast from the past CDR0000797103 has a hard-coded link to http://cdr.dev.cancer.gov/cgi-bin/cdr/STOC_new.js. I'll create a new ticket for fixing these.

I'm guessing the order of these items within one block is not significant, right?

You mean the attributes? That's correct. The order of the blocks (dictionaries) of attributes is significant, though. Don't think you can load jquery-ui before you've loaded jquery.

Order of attributes is never significant semantically. There is such a thing as document normalization, however, which comes into play when you're trying to create diff tools. But it's the responsibility of the creators and users of those tools to prepare (normalize) the document. It's not on us to do so when we're sending HTML to the browser (or XML to a consumer process, for that matter).

On a not completely unrelated topic, Python 3 now preserves the insertion order for dictionaries.

OK, this is done and installed on DEV. The overrides can be viewed using https://cdr-dev.cancer.gov/cgi-bin/cdr/EditControlValues.py (with the right session credentials) under the cdn group. Installing the overrides will be a manual deployment task, because when we get to QA I want to first test that the modified code works without the overrides in place (I've done a little bit of that testing myself on DEV, but we'll do more on QA). The defaults in the code can be reviewed for publish preview and for the admin CGI framework. Here are the links for reviewing the code.

• CGI framework scripts property

• CGI framework stylesheets property

• plugging the properties into the HTML head

• publish preview css_link_attrs property

• publish preview script_link_attrs property

• plugging the publish preview properties into the HTML head

Let me know if you understand how this works sufficiently that you'll at the very least be comfortable updating the override values, ~volker.

Checked into ohm. By the way, on a related note, just a reminder that there have been a lot of pushed commits for the Python code across the repositories over the last couple of weeks, so make sure your clones are up to date. 😉

How would I know what values to set for the non-src attributes?

Depends on the resource. For example, for the jquery links, you look at their CDN page (https://releases.jquery.com/ui/) and use the attributes they tell you to.

That's good to know!  

I was afraid you'd wake up in the middle of the night to write down the integrity value "sha256-hlKLmzaRlE8S..." that just appeared before you. 😃

Ha! The more general answer is that for these resources we're always copying information from markup we're trying to replicate, whether that's markup in the documents we're simulating, or markup we find in instructions for using CDN resources. In no case are we just making up values off the top of our head. 🙂

Are you satisfied with this implementation, ~volker ? Can it go into "DEV Verified"?

Worked for me last time I looked at it.  Good to go!

It looks like we've sufficiently tested your approach:

Installing the overrides will be a manual deployment task, because when we get to QA I want to first test that the modified code works without the overrides in place

I was wondering why we haven't seen errors on PROD since we don't have the "cdn" values setup on any tier except for DEV.  Would you like me to copy those values to QA or STAGE and run a publishing job before adding those to PROD, ~bkline?

Remind me why a publishing job would be the best test of a mechanism which is used for loading JS and CSS for our CGI scripts in the CDR Admin system?

OK, if the publishing job doesn't use or need these values then we skip this as a test.  You as the person who implemented the change is certainly in a better position to identify what needs to happen in order to test this on QA.  I'm just trying to understand if we can close the ticket and it looks to me that we can not at this point. 

How should we proceed with this ticket?

I have installed the overrides on QA and tested both the GTN publish preview as well as the CGI scripts, confirming that

• the overrides are used

• if the values are garbled, then so is the rendering of the pages

• if the values are correct, then so are the pages

You're invited to do any more testing you like, ~volker. You can install them on STAGE and/or PROD if you want (if you install them on PROD they'll be propagated down to the lower tiers when we refresh them from PROD, and it might be easier to tweak existing values when we actually need the overrides than installing them fresh), but it's not absolutely necessary right now, since the "overrides" and what they're "overriding" are the same right now. 🙂

I installed the overrides on STAGE, tested, and installed them on PROD.

Closing ticket.