CDR Tickets

Issue Number 3514
Summary Refresh CDR Database on MAHLER
Created 2012-05-23 15:50:58
Issue Type Improvement
Submitted By Englisch, Volker (NIH/NCI) [C]
Assigned To Englisch, Volker (NIH/NCI) [C]
Status Closed
Resolved 2012-11-08 11:44:22
Resolution Fixed
Path /home/bkline/backups/jira/ocecdr/issue.107842
Description

BZISSUE::5209
BZDATETIME::2012-05-23 15:50:58
BZCREATOR::Volker Englisch
BZASSIGNEE::Volker Englisch
BZQACONTACT::Alan Meyer

The database on MAHLER is more than a year old at this point and I'd like to discuss if it's possible to have CDR refreshed.

Comment entered 2012-05-23 16:22:33 by alan

BZDATETIME::2012-05-23 16:22:33
BZCOMMENTOR::Alan Meyer
BZCOMMENT::1

I have some new filters there, but I presume they will be preserved in the refresh. Otherwise it's fine (and dandy) with me.

Comment entered 2012-06-01 18:09:27 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-06-01 18:09:27
BZCOMMENTOR::Volker Englisch
BZCOMMENT::2

I was hoping to start (and complete) the CDR DB refresh on MAHLER today but it appears that was wishful thinking. This is the first time we're trying to refresh the DB since MAHLER had been rebuild in 2011 and with that rebuilding process the documentation doesn't apply anymore in significant sections. In addition, these changes are adding a lot of time to the process.

a) Mahler is not part of the SAN cluster anymore since the rebuild. This means
that using the backup file from BACH doesn't just require to move the
cluster group from BACH to MAHLER but the backup file now has to be
copied to MAHLER because SQL server does not allow to restore the DB from
a mapped drive. Trying to move the cluster to MAHLER will now result in
an error.
b) Once the DB file has been copied (this takes ~ 1 hour) the restore process
fails with an error message:
The tail of the log for the database "cdr" has not been backed up.
Starting the restore with the 'WITH REPLACE' option gets around this
problem.

I'm not sure how much sense it will make to update the documentation since the next restore will likely be on CBIIT hardware and probably be very different.

Comment entered 2012-06-01 18:14:40 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-06-01 18:14:40
BZCOMMENTOR::Volker Englisch
BZCOMMENT::3

Bob, I've noticed that the PullDevDocs.py process has almost all filter sets listed with incorrect filter set descriptions. I believe it's a left-over from when we had a bug in the code handling unicode characters in filter set names.

I am thinking not to run the process RestoreFilterSets.py at the end and just add the few meaningful changes of the filter sets by hand unless you can think of a reason not to skip this program.

Comment entered 2012-06-01 19:35:29 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-06-01 19:35:29
BZCOMMENTOR::Volker Englisch
BZCOMMENT::4

Alan, the process PushDecDocs.py created two Permatarg errors:

Failure saving CDR0000000178: <Errors>
<Err>Failure executing database query: SQL ERROR CODE: SQL_ERROR;
SQL STATE: 42S02; SQL NATIVE ERROR: 208; SQL ERROR MESSAGE:
[Microsoft][ODBC SQL Server Driver][SQL Server]
Invalid object name 'link_permatarg'.</Err>
</Errors>

Failure saving CDR0000257983: <Errors>
<Err>Failure executing database query: SQL ERROR CODE: SQL_ERROR;
SQL STATE: 42S02; SQL NATIVE ERROR: 208; SQL ERROR MESSAGE:
[Microsoft][ODBC SQL Server Driver][SQL Server]
Invalid object name 'link_permatarg'.</Err>
</Errors>

We can look at these on Tuesday (or I'll have a look on Monday).

Comment entered 2012-06-01 19:54:36 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-06-01 19:54:36
BZCOMMENTOR::Volker Englisch
BZCOMMENT::5

There is an error on MAHLER after the refresh that's preventing us from running a publishing job:
D:\Inetpub\wwwroot\cgi-bin\cdr\Publishing.py in displaySystems
(self=<main.Display instance>)
99 AND num = ?""", (docId, docVersion))
100 docXml = self.__cursor.fetchall()[0][0]
=> 101 tree = etree.XML(docXml.encode('utf-8'))
102 desc = u"*** NO SYSTEM DESCRIPTION FOUND ***"
103 for node in tree.findall('SystemDescription'):
tree undefined, global etree = <module 'lxml.etree' from 'D:\Python\lib\site-packages\lxml\etree.pyd'>, etree.XML = <built-in function XML>, docXml = None, docXml.encode undefined

This may be related to the error reported earlier. I'll have a closer look on Monday.

Comment entered 2012-06-01 19:58:24 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-06-01 19:58:24
BZCOMMENTOR::Volker Englisch
BZCOMMENT::6

Removing Lakshmi from the CC list.

MAHLER has been refreshed with the database from BACH as of 2012-05-31.
Except for accessing the publishing menu everything appears to be working.

Please let me know if you find anything else that's causing problems.

Comment entered 2012-06-01 20:13:49 by alan

BZDATETIME::2012-06-01 20:13:49
BZCOMMENTOR::Alan Meyer
BZCOMMENT::7

(In reply to comment #4)
> Alan, the process PushDecDocs.py created two Permatarg errors:
>
> Failure saving CDR0000000178: <Errors>
> <Err>Failure executing database query: SQL ERROR CODE: SQL_ERROR;
> SQL STATE: 42S02; SQL NATIVE ERROR: 208; SQL ERROR MESSAGE:
> [Microsoft][ODBC SQL Server Driver][SQL Server]
> Invalid object name 'link_permatarg'.</Err>
> </Errors>
>
> Failure saving CDR0000257983: <Errors>
> <Err>Failure executing database query: SQL ERROR CODE: SQL_ERROR;
> SQL STATE: 42S02; SQL NATIVE ERROR: 208; SQL ERROR MESSAGE:
> [Microsoft][ODBC SQL Server Driver][SQL Server]
> Invalid object name 'link_permatarg'.</Err>
> </Errors>
>
> We can look at these on Tuesday (or I'll have a look on Monday).

The CdrServer on Mahler is expecting the link_permatarg table to be
present if any document is saved with a PermaTargId. I'm not sure
why that's happening at the moment, but I'll setup the link_permatarg
table. It was in the Mahler database but not Bach.

I'll probably go ahead and do the same thing on Bach. It won't hurt
anything to have it there.

Comment entered 2012-06-01 20:17:47 by alan

BZDATETIME::2012-06-01 20:17:47
BZCOMMENTOR::Alan Meyer
BZCOMMENT::8

I have created the link_permatarg table on Mahler and Bach.

Comment entered 2012-06-02 09:14:22 by Kline, Bob (NIH/NCI) [C]

BZDATETIME::2012-06-02 09:14:22
BZCOMMENTOR::Bob Kline
BZCOMMENT::9

(In reply to comment #3)
> Bob, I've noticed that the PullDevDocs.py process has almost all filter sets
> listed with incorrect filter set descriptions. I believe it's a left-over from
> when we had a bug in the code handling unicode characters in filter set names.
>
> I am thinking not to run the process RestoreFilterSets.py at the end and just
> add the few meaningful changes of the filter sets by hand unless you can think
> of a reason not to skip this program.

Let's look at it on Tuesday.

Comment entered 2012-06-04 13:39:29 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-06-04 13:39:29
BZCOMMENTOR::Volker Englisch
BZCOMMENT::10

(In reply to comment #7)
> The CdrServer on Mahler is expecting the link_permatarg table to be
> present if any document is saved with a PermaTargId. I'm not sure
> why that's happening at the moment, but I'll setup the link_permatarg
> table. It was in the Mahler database but not Bach.

Perfect!
I've rerun the PushDevDocs.py job to replace the existing documents from MAHLER and this time the process finished without errors.
Also, the error reported in Comment #5 disappeared.

Everything should be back to normal now (except for the filter set descriptions listed in Comment #3).

Comment entered 2012-11-08 11:43:54 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-11-08 11:43:54
BZCOMMENTOR::Volker Englisch
BZCOMMENT::11

I had left this issue open because there was still a minor problem. However, after reading the comments and documentation I don't recall what the problem was.
It may have had to do with restoring menus with unicode characters but I'm not sure.
Since there is no problem using MAHLER after the refresh and since it's unlikely we'll be using any unicode titles for menus anyway Alan suggested not to deal with the issue as an academic exercise but defer this until it becomes an actual problem.

I'll close this issue for now.

Comment entered 2012-11-08 11:44:22 by Englisch, Volker (NIH/NCI) [C]

BZDATETIME::2012-11-08 11:44:22
BZCOMMENTOR::Volker Englisch
BZCOMMENT::12

Issue closed.

Elapsed: 0:00:00.001807