Issue Number | 5004 |
---|---|
Summary | Delayed email notifications from running correspondence mailers |
Created | 2021-07-12 15:04:20 |
Issue Type | Bug |
Submitted By | Osei-Poku, William (NIH/NCI) [C] |
Assigned To | Kline, Bob (NIH/NCI) [C] |
Status | Closed |
Resolved | 2021-07-13 17:39:39 |
Resolution | Fixed |
Path | /home/bkline/backups/jira/ocecdr/issue.293872 |
Robin B. reported that there is a long delay (30-60 mins) in receiving email notifications from generating correspondence mailers on PROD. Bonnie also tried it on PROD and it appears to be the same issue as she is yet to receive the email notification. I just tried it on QA and I am yet to receive the email notification as well.
I just tested on QA and I received the email notification instantly. Please attach the email message (I need the original SMTP payload, complete with headers (not just the pasted text from the body). I can see from the logs on QA that eight seconds elapsed between when the script started handling your request and when the email message was submitted to the NIH mail server, so the problem's not in the CDR itself.
It looks like it is an Outlook issue. . I generated a test mailer on QA at about 2:38 PM and I just received the mailer at 5:06 PM but the email was timestamped at 2:40 PM.
Received: from SA9PR09MB4911.namprd09.prod.outlook.com (2603:10b6:806:1f::21)Received: from SA9PR09MB4911.namprd09.prod.outlook.com (2603:10b6:806:1f::21) by MN2PR09MB4988.namprd09.prod.outlook.com with HTTPS; Mon, 12 Jul 2021 21:05:38 +0000Received: from DM6PR09CA0030.namprd09.prod.outlook.com (2603:10b6:5:160::43) by SA9PR09MB4911.namprd09.prod.outlook.com (2603:10b6:806:1f::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4308.20; Mon, 12 Jul 2021 21:05:32 +0000Received: from DM3GCC02FT015.eop-gcc02.prod.protection.outlook.com (2a01:111:f400:7d04::208) by DM6PR09CA0030.outlook.office365.com (2603:10b6:5:160::43) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4308.20 via Frontend Transport; Mon, 12 Jul 2021 21:05:32 +0000Authentication-Results: spf=pass (sender IP is 156.40.79.161) smtp.mailfrom=mail.nih.gov; nih.mail.onmicrosoft.com; dkim=pass (signature was verified) header.d=nih.gov;nih.mail.onmicrosoft.com; dmarc=pass action=none header.from=mail.nih.gov;Received-SPF: Pass (protection.outlook.com: domain of mail.nih.gov designates 156.40.79.161 as permitted sender) receiver=protection.outlook.com; client-ip=156.40.79.161; helo=mail.nih.gov;Received: from mail.nih.gov (156.40.79.161) by DM3GCC02FT015.mail.protection.outlook.com (10.97.8.125) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.4308.20 via Frontend Transport; Mon, 12 Jul 2021 21:05:32 0000Received: from nihsmtpxwayst06.hub.nih.gov (165.112.13.57) by nihexb1.nih.gov (156.40.79.161) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.8; Mon, 12 Jul 2021 14:42:26 -0400DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nih.gov; i=@nih.gov; q=dns/txt; s=NIH; t=1626115347; x=1657651347; h=content-transfer-encoding:mime-version:from:to:subject: message-id:date; bh=PwOt/RKawp1uHARCBXzzTcbrBFLdY1bRmLvGxzIyyEA=; b=KStfh21lBbT+W37/TSbFfn2ema6LAm64PrT2Y9zSUvjTHABT9VWewXZc gxxUmDIy9qVPCM6vfUCtqNzWh1O8nhI410b6pDThejw7ob99XTVo0s2JF 3UqV0R+3a7UIm0a8GSIulds2b8T2MHomexXGgV1goShJu7FV4zy1yye0o bBDdZ4M2ve6Qm8SDBYfsdNpIqjrhia+u4V5FoEGxDrbjxo/QRUuoTpPmu 59lMO2rxCnkgQp40EXR4a4kqi8oaO4dbYMP6D1s5JKZI+VYfdXr6+lvxy /KQoHfJFCTjKeCPUHM0KUNFmSeCLdgBZGsYpGEk16ZdFSBkXax9GRoab Q==;Message-Id: <2da20b$1vadjo@nihsmtpxwayst06.hub.nih.gov>Date: 12 Jul 2021 14:40:12 -0400IronPort-SDR: f40aY6QXNKuq3r+ekEjZrksdQY2V2nuxN1GhUw/i8uSwGxHVWpXjSSI8mVHpBlUH4lIREKjDTN gfpXAy/1RUOstaxwTH4VPMe54GUcff68wSwe+0PlkmCy2FDV4UnTwy8wN2Uq+olKw9yC7L/Pge i1GwG0h3xFU13eiZ9/dFcI4gLziEO9y/qRBPsNDTzoVlXNHgTT7EBklZ9iiYZDMpZaD8/rNL6m /WhpAF+gNZGYSmheTQKlOWO0xmmHrDF1fZUEmz9EddHkv54qDN3/XfTy3li+h1jiONcLIM+pyG k7w=X-SBRS-Extended: LowX-IronPortListener: non-ces-outX-IronPortListener: non-ces-outX-IPAS-Result: =?us-ascii?q?A2GZCQCPjOxg/0la54BaHgEBCxIMQIFOC4F8gREVVlIah?= =?us-ascii?q?EiJBIt+mDSBfAsBAQFGCgQBAQMEhHCCWSY0CQ4CBAEBAQEDAgMBAQcBAQYBA?= =?us-ascii?q?QEBAQEFBAICgQCFLzkNg1aBCAEBAQEBAQEBAQEBAQEBAQEBAQEWAoFeET1OA?= =?us-ascii?q?iYCSUuCEEAqAYJcD59riWeBMoEBihyBFiqHCYZgAokaK04LDAQYghOCShOCU?= =?us-ascii?q?QSEUlqBKA1ZlH2KFZ1jgy6KMZd/ARGLXZcSojCaVoIVKwoYKYM0TxmUHIkjL?= =?us-ascii?q?wI2AgYKAQEDCYwzAQE?=IronPort-HdrOrdr: A9a23:ro+5dqwP4ZuwAETNRCGhKrPwK71zdoMgy1knxilNoH1uA6qlfq WV9pkmPHDP6Qr5NEtBpTn4Ase9qBDnhPlICOsqTNKftWDd0QPCEGgF1/qA/9SJIUHDH4VmpM NdmsZFeb/N5YcTt6nHCG7RKadY/OW6X-IronPort-Anti-Spam-Filtered: trueX-IronPort-AV: E=Sophos;i="5.84,234,1620705600"; d="scan'208";a="66401912"Received: from msg-b12-ltm1_v9.hub.nih.gov (HELO nciws-q2062-v.nci.nih.gov) ([128.231.90.73]) by nihsmtpxwayst06.hub.nih.gov with ESMTP; 12 Jul 2021 14:40:12 -0400Content-Type: text/plain; charset="utf-8"Content-Transfer-Encoding: 7bitMIME-Version: 1.0From: PDQ Operator <NCIPDQoperator@mail.nih.gov>To: oseipokuw@mail.nih.govSubject: [QA] CDR Mailer Job StatusReturn-Path: NCIPDQOperator@mail.nih.govX-InternalExchange: ExInternalX-Exchange-ASBP: 1X-OrganizationHeadersPreserved: nihexb1.nih.govX-MS-Exchange-Organization-ExpirationStartTime: 12 Jul 2021 21:05:32.1703 (UTC)X-MS-Exchange-Organization-ExpirationStartTimeReason: OriginalSubmitX-MS-Exchange-Organization-ExpirationInterval: 1:00:00:00.0000000X-MS-Exchange-Organization-ExpirationIntervalReason: OriginalSubmitX-MS-Exchange-Organization-Network-Message-Id: e0270143-1e95-47fd-363b-08d94578c916X-EOPAttributedMessage: 0X-MS-Exchange-Organization-MessageDirectionality: OriginatingX-MS-Exchange-Organization-AuthSource: nihexb1.nih.govX-MS-Exchange-Organization-AuthAs: InternalX-MS-Exchange-Organization-AuthMechanism: 10X-MS-Exchange-Organization-SCL: -1X-CrossPremisesHeadersPromoted: DM3GCC02FT015.eop-gcc02.prod.protection.outlook.comX-CrossPremisesHeadersFiltered: DM3GCC02FT015.eop-gcc02.prod.protection.outlook.comX-MS-PublicTrafficType: EmailX-OriginatorOrg: mail.nih.govX-MS-Office365-Filtering-Correlation-Id: e0270143-1e95-47fd-363b-08d94578c916X-MS-TrafficTypeDiagnostic: SA9PR09MB4911:X-MS-Exchange-AtpMessageProperties: SAX-SCL-By-Transport-Rule: ironport-filtered in EXOX-MS-Oob-TLC-OOBClassifiers: OLM:5236;X-Microsoft-Antispam: BCL:0;X-Forefront-Antispam-Report: CIP:156.40.79.161;CTRY:US;LANG:en;SCL:-1;SRV:;IPV:CAL;SFV:SKI;H:mail.nih.gov;PTR:InfoDomainNonexistent;CAT:NONE;SFS:;DIR:INB;X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Jul 2021 21:05:32.0904 (UTC)X-MS-Exchange-CrossTenant-Network-Message-Id: e0270143-1e95-47fd-363b-08d94578c916X-MS-Exchange-CrossTenant-Id: 14b77578-9773-42d5-8507-251ca2dc2b06X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=14b77578-9773-42d5-8507-251ca2dc2b06;Ip=[156.40.79.161];Helo=[mail.nih.gov]X-MS-Exchange-CrossTenant-AuthSource: nihexb1.nih.govX-MS-Exchange-CrossTenant-AuthAs: InternalX-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPremX-MS-Exchange-Transport-CrossTenantHeadersStamped: SA9PR09MB4911X-MS-Exchange-Transport-EndToEndLatency: 00:00:06.6285281X-MS-Exchange-Processed-By-BccFoldering: 15.20.4308.026X-Microsoft-Antispam-Mailbox-Delivery: ucf:0;jmr:0;auth:0;dest:I;ENG:(750129)(520011016)(706158)(944506458)(944626604);X-Microsoft-Antispam-Message-Info: v4JuJA4OSszMtD0OstMXwHYqp1F2V0MdWtGWgmZUxthlHRmUXUeul8s9BtPHwea/Q4TzEBu+3vebfx3QvAZ4YKtyf8PqkSiFoZVwyBdJtVxUJYBjC0ChSdyzSWZtGPnatWBrrxWoXGjdY41aSKnqMtQaXcjQVzuQsTVqh6rdGjek0F6YBc6p+mAKFaYgcjiHC+mgws07mbn8yPAMylJ24Xdad4MYHH9dJrkpz9qlhxhCoRCLd2BpYqoF3qDxzWluymJw6cMdvS9AD4RG4QDyo9jlUPh1JcNBfNggviuZE7CUVA73BpVDQp0lPE2hpnm3q0nUmEhjV+n3EOx2SiexXPH+6QnZfCm4cntmuJKFyyC85tcdytKN998KlFzVZu9UjEyWvFZZaS16aUNJdbEAv2p3Ql13i/prye8kbILr861loPK1rXw7NvxIVJcZNoppqz9TLMivn6eDiG1eavWFNLZzm3rz8nZzqZDxnpCZYYoCh6GDYEWLW7Gm902ol6i8zvRpRXpF5saDUl0KJ/N2a9tXjgzaXN2SmakKlZ7UokGanGUPIiogFhPgY9o9rU7mfP+DyDZc5XW26hA9kvXZ2Q==
I can report an issue to CBIIT, but only if I can get a copy of the message without the headers garbled (looks line the breaks between lines have disappeared in what you pasted into JIRA).
If you're using the Web Outlook, you need to choose View > View message details, then copy and pasted into a program which won't garble what comes from the clipboard:
Should look something like this if you're successful. Try attaching it to the issue instead of pasting it into a comment.
I copied and pasted it into notepad which is attached. I am also including a screenshot here.
Thanks, that's what I needed. I have filed ticket NCI-RITM0304304 with CBIIT. Will report back what they say.
From CBIIT:
Robin b. confirmed that the emails notifications were no longer delayed.
File Name | Posted | User |
---|---|---|
image-2021-07-12-20-24-38-264.png | 2021-07-12 20:24:39 | Kline, Bob (NIH/NCI) [C] |
image-2021-07-12-20-26-51-888.png | 2021-07-12 20:26:52 | Kline, Bob (NIH/NCI) [C] |
image-2021-07-13-17-39-14-970.png | 2021-07-13 17:39:15 | Kline, Bob (NIH/NCI) [C] |
MailerOutlookIssueLog.txt | 2021-07-13 09:14:46 | Osei-Poku, William (NIH/NCI) [C] |
OutlookMailerIssue.PNG | 2021-07-12 17:49:30 | Osei-Poku, William (NIH/NCI) [C] |
screenshotOutlookMailerIssueLog.PNG | 2021-07-13 09:17:02 | Osei-Poku, William (NIH/NCI) [C] |
Elapsed: 0:00:00.001488