CDR Tickets

Issue Number 4092
Summary Modify LoadGPEmailers to send to CancerGovTest@mail.nih.gov when not on production
Created 2016-05-12 13:13:06
Issue Type Improvement
Submitted By Learn, Blair (NIH/NCI) [C]
Assigned To Kline, Bob (NIH/NCI) [C]
Status Closed
Resolved 2016-12-16 15:07:18
Resolution Fixed
Path /home/bkline/backups/jira/ocecdr/issue.184121
Description

The LoadGPEmailers script contains logic such that when it's run on the lower tier it sends email to a hard-coded list of test accounts. In SVN, this should be set to CancerGovTest@mail.nih.gov.

Comment entered 2016-12-16 15:07:25 by Kline, Bob (NIH/NCI) [C]

Installed on DEV.

Comment entered 2017-01-05 11:20:44 by Osei-Poku, William (NIH/NCI) [C]

Can you clarify in which mailbox I should expect the test mailer in? CancerGovTest@mail.nih.gov or geneticsdirectory@cancer.gov ? I never got access to CancerGovTest@mail.nih.gov so I am assuming it would be geneticsdirectory@cancer.gov.

Comment entered 2017-01-05 13:27:32 by Kline, Bob (NIH/NCI) [C]

: do you know whom we should contact to get William access to the CancerGovTest mailbox? That's the one Blair told me I should be using.

Comment entered 2017-01-05 14:03:53 by Kline, Bob (NIH/NCI) [C]

: Volker doesn't know who controls access to this mailbox. Can you help us out?

Comment entered 2017-01-05 14:19:19 by Learn, Blair (NIH/NCI) [C]

You should be able to request access for William via a helpdesk ticket (https://service.cancer.gov/)

Comment entered 2017-01-05 15:49:54 by Kline, Bob (NIH/NCI) [C]

Thanks. I have submitted the ticket. Will post progress here.

Comment entered 2017-01-10 17:07:27 by Osei-Poku, William (NIH/NCI) [C]

I do have access to mailbox now. I generated a couple of mailers yesterday but I don't see the mailers in the mailbox so I am not sure if this is working.

Comment entered 2017-01-11 07:56:22 by Kline, Bob (NIH/NCI) [C]

Which tier, please?

Comment entered 2017-01-11 08:31:04 by Osei-Poku, William (NIH/NCI) [C]

On QA.

Comment entered 2017-01-11 10:54:50 by Kline, Bob (NIH/NCI) [C]

I found that the gp_emailer table on QA was not configured properly. I fixed the problem and ran the import job by hand. Please check again.

Comment entered 2017-01-11 12:29:18 by Osei-Poku, William (NIH/NCI) [C]

I was able to see the emails but I noticed 2 issues:

1. The mailer responses do not go back into the test mailbox, they go into the GeneticsDirectory@Cancer.gov mailbox.

2. I am not able to open the diff reports (GP responses). I am getting a 403 error message for two that I submitted.

Comment entered 2017-01-11 14:38:30 by Kline, Bob (NIH/NCI) [C]

The mailer responses do not go back into the test mailbox, they go into the GeneticsDirectory@Cancer.gov mailbox.

That's correct. Blair filed this ticket to avoid having individual email addresses hard-coded in the software for test emailer recipients. The responses have never gone to those hard-coded tester email addresses.

I am not able to open the diff reports (GP responses). I am getting a 403 error message for two that I submitted.

Ah, yes. We're no longer allowed to expose those pages without authentication (see OCECDR-4107), and since the GP mailer CGI code has no way to get a CDR session ID for some arbitrary point in the future, I'm going to need to take that link out of the email notification that CGI script sends. You'll have the review those diff reports from the Admin menu.

Reports > Mailer Reports > GP Emailers List
Comment entered 2017-01-11 14:57:06 by Osei-Poku, William (NIH/NCI) [C]

That's correct. Blair filed this ticket to avoid having individual email addresses hard-coded in the software for test emailer recipients. The responses have never gone to those hard-coded tester email addresses.

To make it easier for us testers, it would be good to have all the test emails go into one mailbox. We also talked about not mixing test emails with production emails.


Ah, yes. We're no longer allowed to expose those pages without authentication (see OCECDR-4107), and since the GP mailer CGI code has no way to get a CDR session ID for some arbitrary point in the future, I'm going to need to take that link out of the email notification that CGI script sends. You'll have the review those diff reports from the Admin menu.

I didn't know this will be the impact of the change in OCECDR-4107. It will certainly make it difficult to review the responses when we receive them. We put the links in the emails so that we can reach the pages quickly when reviewing the emails.

Comment entered 2017-01-11 15:08:24 by Kline, Bob (NIH/NCI) [C]

I sympathize with the problem you're facing, but I don't know that I have a lot of good solutions to offer. Unless has some ideas that haven't occurred to me, best I can think of would be

  1. instead of eliminating the current link in the notification email, replace it with a link to the CDR Admin login page, from which you can go to the page with the links to all the diff reports; or

  2. continue sending you the link, which you paste into your browser, append "&Session=<SESSION-ID>" (from an existing CDR Admin session), and navigate to the edited URL.

Not very satisfactory, I grant you, but we're not free to ignore the security constraints imposed by the fact that this is a public-facing server.

Comment entered 2017-01-11 15:32:35 by Osei-Poku, William (NIH/NCI) [C]

Option 1 is acceptable. Assuming I have to review multiple responses within 30min, for example, would I have to log in each time I click on a new link or logging in once should be enough?

Comment entered 2017-01-11 15:34:32 by Kline, Bob (NIH/NCI) [C]

... it would be good to have all the test emails go into one mailbox ....

I have implemented this new requirement on DEV and QA. Please test.

Comment entered 2017-01-11 15:41:29 by Kline, Bob (NIH/NCI) [C]

Once you have a link to the ListGPEmailers page with a fresh CDR session ID, that page will be available (and refreshable in the event that another genetics professional completes and submits her mailer after you first opened the page) as long as the session is valid (which is indefinitely as long as the server sees a request from that session at least once every 24 hours).

Comment entered 2017-01-11 15:55:11 by Osei-Poku, William (NIH/NCI) [C]

I have generated 5 additional mailers. Could you please run the job manually so that I can review them?

Comment entered 2017-01-11 16:04:54 by Kline, Bob (NIH/NCI) [C]

... link to the CDR Admin login page, from which you can go to the page with the links to all the diff reports ....

I believe I can improve on this option slightly, by adding to the login script the capability of changing where you land after you have successfully logged in. Currently, you always end up on the top page for the complete CDR web admin interface, from which you can navigate down the menus (in this case, to the ListGPEmailers page). With this enhancement, I can have the link in the email specify that you should be dropped directly on the ListGPEmailers page after your CDR (that is, NIH domain) login credentials are accepted. The only conceivable drawback to this approach is that by going directly to this page (which is back on the Linux server), you wouldn't have access to the rest of the CDR admin menus (without pasting the "Session=..." parameter into a hand-made URL). But in this case I can't see that as much of a drawback (you didn't have the full menus reachable from the links you're currently getting in the email alert on production). What do you think about this variation? I can imagine that if we enhance the login script this way, we'll find other uses for the new capability. :-)

Comment entered 2017-01-11 16:09:10 by Osei-Poku, William (NIH/NCI) [C]

Yes, I think this improvement will be very helpful and not having access to the full menu shouldn't be a problem. Thanks!

Comment entered 2017-01-11 16:29:14 by Kline, Bob (NIH/NCI) [C]

DEV or QA?

Comment entered 2017-01-11 16:39:00 by Osei-Poku, William (NIH/NCI) [C]

QA

Comment entered 2017-01-11 17:43:30 by Kline, Bob (NIH/NCI) [C]

Would it also be helpful to have the software move you to the bottom of the page of mailers? Or perhaps have the ListGPEmailers page show the emailers in reverse chronological order (newest at the top)? If so, which option would you prefer?

Comment entered 2017-01-11 17:46:41 by Osei-Poku, William (NIH/NCI) [C]

Reverse chronological order is preferred.

Comment entered 2017-01-11 17:59:12 by Kline, Bob (NIH/NCI) [C]

OK, I think I've got the latest requested changes implemented correctly on DEV. Please test there, and if it's OK I'll deploy the changes to QA.

Comment entered 2017-01-11 18:26:56 by Osei-Poku, William (NIH/NCI) [C]

I generated mailers on DEV. Could you please process them so I can review?

Comment entered 2017-01-11 18:33:24 by Kline, Bob (NIH/NCI) [C]

All set.

Comment entered 2017-01-11 18:44:41 by Osei-Poku, William (NIH/NCI) [C]

All changes appear to be working correctly. However, I noticed that when I submit a No Changes mailer, the email text says that there are changes when in fact I didn't make any changes and the diff report correctly indicates that there are no changes. So, it is only the text of the email that is misleading.

Comment entered 2017-01-11 19:07:57 by Kline, Bob (NIH/NCI) [C]

For another ticket, we changed the comparison logic to ignore differences in attribute order for the script which displays the differences, but we didn't make that change to the actual mailer processing script. I've done that now on DEV. Please test on that tier (I think there are still some mailers which are available for testing).

Comment entered 2017-01-11 19:28:06 by Osei-Poku, William (NIH/NCI) [C]

It is working correctly now. I will review the remaining mailers and update the ticket. Thanks

Comment entered 2017-01-11 19:46:12 by Kline, Bob (NIH/NCI) [C]

Newest changes promoted to QA.

Comment entered 2017-01-11 19:47:46 by Kline, Bob (NIH/NCI) [C]

I have loaded the latest set on QA by hand. I'm calling it a night. :-)

Comment entered 2017-01-12 10:50:31 by Osei-Poku, William (NIH/NCI) [C]

I am getting 403 error message while trying to access the GP mailers on both DEV and QA.

Comment entered 2017-01-12 10:56:03 by Kline, Bob (NIH/NCI) [C]

Are you coming through the CDR Admin menus, or are you using an old bookmark which doesn't have a CDR Session ID?

Comment entered 2017-01-12 10:59:10 by Osei-Poku, William (NIH/NCI) [C]

I am coming through the CDR Admin menus. I tried on 2 machines and still can't get in when I click on the GP Emailers List report on the Mailers menu.

Comment entered 2017-01-12 11:10:12 by Osei-Poku, William (NIH/NCI) [C]

It appears to be a problem with the hosts file on our machines here. Let me look into it and let you know.

Comment entered 2017-01-12 11:21:08 by Kline, Bob (NIH/NCI) [C]

This is odd. I'm assume you are using the oseipokuw account, right? That account is a member of the "Developers" group. The Developers group (and probably some other groups to which your account belongs) has permission for the "GP MAILERS" action. I've added some more logging on DEV. Please try it again there.

Comment entered 2017-01-12 11:45:19 by Osei-Poku, William (NIH/NCI) [C]

I don't think the problem is coming from the CDR servers. It probably has to do with the hosts files that we maintain on our machines to connect to the CDR servers. I am able to get in now through CISCO VPN.

Could you please provide me with the ip address of the gpmailers server?

Comment entered 2017-01-12 11:51:45 by Osei-Poku, William (NIH/NCI) [C]

Verified on DEV and QA.

Comment entered 2017-01-12 12:01:55 by Kline, Bob (NIH/NCI) [C]

DEV: 10.133.198.240
QA: 10.133.200.103
STAGE: 10.133.212.37
PROD: 129.43.254.38 (IP6=2607:f220:41d:21c1::812b:fe26)

Comment entered 2017-01-12 12:12:25 by Osei-Poku, William (NIH/NCI) [C]

Thanks, their full FQDM names would be gpmailers-dev.cancer.gov in that order, right?

Comment entered 2017-01-12 14:12:33 by Kline, Bob (NIH/NCI) [C]

gpmailers-dev.cancer.gov
gpmailers-qa.cancer.gov
gpmailers-stage.cancer.gov
gpmailers.cancer.gov

Comment entered 2017-01-17 12:40:03 by Osei-Poku, William (NIH/NCI) [C]

Thanks!

Elapsed: 0:00:00.000922