CDR Tickets

BZDATETIME::2005-07-05 14:55:36
BZCOMMENTOR::Bob Kline
BZCOMMENT::1

Made Volker the QA contact.

BZDATETIME::2005-07-07 10:49:16
BZCOMMENTOR::Alan Meyer
BZCOMMENT::2

My initial work on this ran into some problems. I reverted to 2.3 and will try
to figure out it tonight, when no one is using Mahler.

BZDATETIME::2005-07-08 00:17:09
BZCOMMENTOR::Alan Meyer
BZCOMMENT::3

Here's the current (lack of) progress:

SOAPpy:

Some of the test programs worked, some didn't. I'll have to
investigate further and check if the tests that fail also
fail under Python23.

ADO:

I looked at the site-packages/win32com/gen_py directories for
Python23 and Python24. I had imported all the same packages
into Python24 - but the way Python code was constructed under
gen_py was quite different.

I was able to connect to SQL Server using the cdrdb.connect()
function, and get a cursor using cdrdb.conn.cursor(). But
every attempt to execute a query returns "unexpected failure
for query '...')

I tried this from two places - Mahler and from my own
machine, with identical results. Neither one will execute a
query.

It makes me suspect that something has changed in the
win32com module that's causing problems.

Excel:

No further testing at this time. Other things look more
important.

Once again, I've restored Python23.

Bob,

Can you execute SQL Server queries using cdrdb from your
machine with Python24?

BZDATETIME::2005-07-08 11:22:12
BZCOMMENTOR::Bob Kline
BZCOMMENT::4

(In reply to comment #3)
> Here's the current (lack of) progress:
>
> SOAPpy:

Don't worry too much about this package. Nothing that uses it is in
production yet, and in the worst case I can write my own version of
the only thing that does use it at present so that I'm doing my own
SOAP (as I do for cdr2cg).

> ADO: ...
> Can you execute SQL Server queries using cdrdb from your
> machine with Python24?

Can't get to my machine right now (it's refusing Terminal Services
connection, even though I've confirmed the service is running --
probably the NIH firewall(s) at work). I was able to connect to
my own SQL server and execute queries successfully using the cdrdb
module under Python 2.4. We'll tackle this again together on
Tuesday.

BZDATETIME::2005-07-12 14:32:41
BZCOMMENTOR::Alan Meyer
BZCOMMENT::5

Bob and I investigated this further. It appears that Python 2.4.1 has a
known bug that is causing our interface to Microsoft's Active Data Objects
(ADO) interface to SQL Server to fail. 2.3.5 and 2.4.0 both work. The
bug was introduced in 2.4.1 while trying to fix another problem.

I have posted a message on the Python bug tracker asking the fellow who
works on this when he thinks it might be fixed and entered into the
main line distribution. If he replies (or if he doesn't) we can decide
whether to upgrade Python to 2.4.0, or leave it at 2.3.5 and wait for
a later release of 2.4.1+ that fixes this.

For now, I'm suspending the effort to upgrade Python.

BZDATETIME::2005-07-12 20:09:43
BZCOMMENTOR::Alan Meyer
BZCOMMENT::6

This may now be working but I don't know how to test all of the modules.

Things I've tested that appear to work include:

ADO access.
Image library.
The Excel library and PyXLWriter.
Some SOAP tests. Others don't work and I'm suspecting that it's
a problem in the test code, not the library. I also presume
that fpconst worked okay.

I did not install SSL support. I went through the steps to build the
_ssl.pyd DLL, but when I went to install it, I found there was already
an _ssl.pyd in the Python DLL directory. I suspect it was created in
a previous of Python last week. It appears to be a full DLL, and not
just a wrapper as the new versions appear to be. I'd like to discuss
this with Bob before I replace it.

I uninstalled Python 2.3.5 before installing 2.4, and tried to uninstall
all of the modules that were known to the Windows Control Panel Add/Remove
Software program - but I got some permission denied errors (can't get rid
of these can we?) and it's likely that some things were not removed.

We need more testing and a decision about SSL and SOAP before declaring
this as ready to go.

BZDATETIME::2005-07-12 22:06:36
BZCOMMENTOR::Bob Kline
BZCOMMENT::7

(In reply to comment #6)
> I did not install SSL support.

It was already there. I just tested it, and it looks fine.ot removed.

> We need more testing and a decision about SSL and SOAP before declaring
> this as ready to go.

We'll definitely want to let it run for at least a couple of weeks to
give it a chance to fail on Mahler before propogating the upgrade to
Franck and Bach, but I think we can declare SSL support as installed
and tested, and SOAP is as good as it needs to be. As I mentioned earlier,
that module isn't doing anything we can't do ourselves, and isn't used
by anything that's in production.

BZDATETIME::2005-08-17 10:29:48
BZCOMMENTOR::Volker Englisch
BZCOMMENT::8

Since I'm the QA on this bug I thought I'd run a publishing job on MAHLER over
night before setting this to verified. The publishing job - creating 2000
documents per document type finished successfully. However, the push job
finished with a failure status.
http://mahler.nci.nih.gov/cgi-bin/cdr/PubStatus.py?id=2910
and the error message says:
publish: initTermCache: can't get mutex, GetLastError=183

Traceback (most recent call last):
File "d:\cdr\lib\python\cdrpub.py", line 629, in publish
port=self.__pubPort)
File "d:\cdr\lib\python\cdr.py", line 3138, in cacheInit
raise StandardError (err)
StandardError: initTermCache: can't get mutex, GetLastError=183

and there are additional errors listed in the publish.log file before this one:

!3500 Tue Aug 16 23:20:04 2005: Job 2909: checkProblems raises StandardError, ms
g=Aborting on error detected in CDR0000380968.<BR>
!3500 Tue Aug 16 23:20:04 2005: Job 2909: Exception publishing doc 380968 ver 16
in Thread-4
Traceback (most recent call last):
File "d:\cdr\lib\python\cdrpub.py", line 2211, in __publishDocList
self.__publishDoc (doc, filters, destType, destDir, subDir)
File "d:\cdr\lib\python\cdrpub.py", line 2353, in __publishDoc
self.__checkProblems(doc, errors, warnings)
File "d:\cdr\lib\python\cdrpub.py", line 2393, in __checkProblems
raise StandardError(msg)
StandardError: Aborting on error detected in CDR0000380968.<BR>

Of course it could also be the case that this has nothing to do with the Python
upgrade but is rather related to OCECDR-1488.

BZDATETIME::2005-09-01 15:22:02
BZCOMMENTOR::Bob Kline
BZCOMMENT::9

Don't forget to post something on the ActiveState Python mailing list
asking about the next release (and whether they're aware of the bug).

BZDATETIME::2005-09-13 15:46:36
BZCOMMENTOR::Alan Meyer
BZCOMMENT::10

For the record, the bug number for this at Sourceforge.net is #1175396,
"codecs.readline sometimes removes newline chars". Fixes were checked
in to the Python source code in April, but it's not totally clear to
me from the subsequent discussions whether the programmer thinks the
bug is fixed for all possible cases.

Also for the record, there is a simple test case posted that demonstrates
the error with Python 2.4.1, that works fine with 2.4. It's on mahler
in d:\home\alan\test\CodecBug2.4.1.py. When run from the command, it opens
the file "testfile1.txt" and reads it. If Python has the bug (2.4.1 does)
error messages will appear. Otherwise no messages will appear.

BZDATETIME::2005-09-13 16:01:46
BZCOMMENTOR::Alan Meyer
BZCOMMENT::11

(In reply to comment #9)
> Don't forget to post something on the ActiveState Python mailing list
> asking about the next release (and whether they're aware of the bug).

I've posted the question.

BZDATETIME::2005-09-13 20:12:20
BZCOMMENTOR::Alan Meyer
BZCOMMENT::12

(In reply to comment #11)
> (In reply to comment #9)
> > Don't forget to post something on the ActiveState Python mailing list
> > asking about the next release (and whether they're aware of the bug).
>
> I've posted the question.

We've now get an answer from Trent Mick at ActiveState. To wit:

"Yes, I hope to have that fixed in the next release. We hope to have a
new release sometime within the next month, but I don't have a solid
date that I can promise."

I therefore propose to do nothing to Python until the next release
comes out, then restart the Python upgrade process. That is, install
the newest (perhaps 2.4.2?) version on Mahler, test, and if all goes
well for a couple of weeks, install on Franck and then Bach.

BZDATETIME::2005-11-15 19:50:53
BZCOMMENTOR::Alan Meyer
BZCOMMENT::13

Python 2.4.2 is now available from Activestate. I'm
going to try and finish the reports I'm working on before
I get to testing this - unless someone sees a need to do
otherwise.

BZDATETIME::2005-11-17 11:06:45
BZCOMMENTOR::Bob Kline
BZCOMMENT::14

Preliminary testing on my own workstation has not run into any problems. Registering of all of the COM objects we use, as well as exercising of those objects has all been successful.

BZDATETIME::2005-11-29 22:16:43
BZCOMMENTOR::Alan Meyer
BZCOMMENT::15

It might be better to use more generic references in the
documentation for installing the Python packages. A number
of the packages are now available in later versions than
those referenced in the Description.

Here is a more generic listing:

[1] http://prdownloads.sourceforge.net/mysql-python/
Look for files named MySQL-python.exe-......win32-py....zip
[2] http://effbot.org/downloads
Look for files named PIL-...win32-py...exe
[3] http://prdownloads.sourceforge.net/pychecker/
[4] http://prdownloads.sourceforge.net/pyxlwriter/
[5] http://research.warnes.net:9090/~warnes/fpconst/
[6] http://prdownloads.sourceforge.net/pyxml/
[7] http://prdownloads.sourceforge.net/pywebsvcs/
[8] Use COM Makepy tool from PythonWIN IDE.

BZDATETIME::2005-11-29 22:24:20
BZCOMMENTOR::Alan Meyer
BZCOMMENT::16

We are close to having Python fully installed on v2mahler.

The last step I am aware of is to install MS Office so that
we can register the Microsoft Excel 10.0 Object Library.

Everything else has installed uneventfully. Python works
fine. I have not tested everything else. The easiest way
to test will, I think, be to actually run the CDR programs
that use the extra libraries. We can do that as soon as
the CDR is installed.

BZDATETIME::2005-11-29 22:34:44
BZCOMMENTOR::Bob Kline
BZCOMMENT::17

(In reply to comment #16)
> The last step I am aware of is to install MS Office so that
> we can register the Microsoft Excel 10.0 Object Library.

Maybe we can ask the users if the reports that use this library are used infrequently enough that I can port them to use the new ExcelWriter module before they're needed again. If so, we can avoid installing Office on the new machine altogether.

BZDATETIME::2006-02-10 17:38:46
BZCOMMENTOR::Volker Englisch
BZCOMMENT::18

I talked to Carbie yesterday. He expects to have the new FRANCK ready for us around the end of next week.

BZDATETIME::2006-03-10 17:05:35
BZCOMMENTOR::Volker Englisch
BZCOMMENT::19

The new motherboard has been installed and V2FRANCK is up and running.
Carbie will hand over the machine on Monday if it runs without problems over the weekend.

BZDATETIME::2006-03-23 18:11:55
BZCOMMENTOR::Volker Englisch
BZCOMMENT::20

Almost everything that's on the migration list has been completed. The remaining things that need to be added are the installation of the CDRPublishing service.
Once the Windows Resource Kit has been installed I will complete that.

It is time now to test, test, and test some more.

BZDATETIME::2006-03-23 18:45:33
BZCOMMENTOR::Alan Meyer
BZCOMMENT::21

I have installed all of the packages needed on v2Franck
except ssl. ssl is not currently used anywhere, but we
are building it in case COG or another data exchange
partner someday asks us for a secure, encrypted connection
for data transfer.

I built the ssl dlls on my workstation, downloaded the
Python source code, and built the python library. Everything
seemed to work but I was not able to pass the test that
Bob had produced for issue #1504.

I used the new version 0.9.8 ssl. In the past, that wouldn't
work with Python 2.4. That might have been the problem,
though it didn't complain during the compile and link phase
as I recall it doing before. I could try backing up to 0.9.7.

I thought I would try from Mahler, but couldn't get in
via remote desktop, so I decided to suspend this until
Tuesday. We don't need this capability for anything we're
doing now, so it's not high priority.

BZDATETIME::2006-03-30 12:58:11
BZCOMMENTOR::Volker Englisch
BZCOMMENT::22

The last thing besides testing is to get the publishing process to work. Once this problem is resolved we will be able to run a few publishing jobs to test the system.

BZDATETIME::2006-04-18 17:15:41
BZCOMMENTOR::Volker Englisch
BZCOMMENT::23

The new FRANCK is now live and test publishing runs finished successfully.

I believe this task can now be closed.

BZDATETIME::2006-04-20 18:36:30
BZCOMMENTOR::Alan Meyer
BZCOMMENT::24

Installing the new Python on Franck turned out to be more work
than I expected until Volker saved the day by suggesting the
magic word "permissions?"

I had tried several complete re-installs. I traced through the
admin.py program by hand in the debugger. It worked fine. I ran
Python interactively, imported cdrdb, and accessed the database.
That worked fine too. But none of my tests could find the
problem because, of course, I myself was the problem. I had
apparently set the "it works for me but not for the webserver"
switch.

The win32com folder existed from before the update. But
permissions must have somehow been changed on it during the
update. It didn't match the permissions on Mahler.

I gave full control to everyone on that folder on Franck (which
was how it was setup on Mahler) and everything now seems to work
okay.

If publishing goes okay, I'll setup Bach on Tuesday.

BZDATETIME::2006-05-04 22:09:33
BZCOMMENTOR::Alan Meyer
BZCOMMENT::25

I'm posting these notes from installing Python on Bach so that
for our next install we'll be able to recall some of the problems
we overcame on the last one.

1. Attempting to install Pychecker, I got access denied when I
typed:

setup.py install.

Thinking it was permissions, I checked the permissions on
Mahler and saw that on Bach, there was two separate
categories, one for Users, and one for Everyone. Everyone did
not appear on Mahler, and Users had the same permissions on
d:\Python. I then edited the permissions on Bach giving
"Everyone" read, execute and list folder contents. That
caused all the other user categories to suddenly disappear
except Administrators and
S1-5-21-16020293-606-668331-2048475684-513 - which had no
permissions. Whoever that user is, his parents did him a
serious disservice when they named him.

At any rate, that didn't solve anything.

Thinking that it had to do with file associations I went to
the \cdr\bin directory and tried "cdrgetdoc", "showcdrdoc" and
"valdocversion", all without typing the .py extension. All
worked flawlessly.

I notice that in the \Python\Doc directory, "Everyone" and
"Users" co-exist. "Everyone" has full control, but "Users"
can only read, execute and list folder contents.

On the lib directory, "Administrators" have no permissions,
but Users can read, execute and list folder contents.

Later I executed "setup install" for pyxlwriter and it worked.
Maybe my setting permissions for everyone fixed something?

2. Executing PyXML setup.

Got a permission denied error, followed by a crash. Briefly
saw a message referencing a \Python\xml... directory, but when
I went back to look at it, everything was gone.

My next effort produced no error message at all, just exit of
the install.

I then gave adminstrators full control over
Python\Lib\site-packages. That didn't help.

Then tried the same on Python\Lib\site-packages_xmlproc. No
joy.

Then went to add/remove programs and removed PyXML (I had
removed the earlier one before starting). It said it removed
one file and no directories, but left it in the list. When I
tried again it said an error had occurred, or it had
previously been uninstalled, and offered to remove it from the
list.

The next try showed the errors, there were creating,
accessing, and writing in directories on and under
d:\Python\xmldoc.

Went back to d:\Python and gave full control to
administrators. XMLProc then installed successfully. I don't
know whether the old d:\Python lacked that and didn't need it,
or whether installing Python again changed the permissions on
the d:\Python directory. I assume that's what happened.

3. Setting win32com permissions.

I went to the win32 directories which had given us problems on
Franck and tried to set permissions to match Mahler, but I
couldn't do it. When I'd set one set of permissions to match
Mahler, whole groups would disappear from the list of groups
or users for whom permissions could be set.

On win32com, Mahler had:
Adminstrators with full control
"Users" with read, execute, list folder contents
"Everyone" with nothing.

Bach now had:
Administrators with nothing.

I gave Administrators full control to match Mahler and
suddently the Users and System groups disappeared from the
list of group objects. I tried adding Users back in, but it
wouldn't let me. So I gave read, execute and list folder
contents to "Everyone". Our friend S-1-5... was still listed,
he hadn't disappeared, though he had no permission to do
anything and both Allow and Deny were grayed out for all
options.

4. Testing.

Command line utilities worked. Access through the browser
failed.

After trying various things, I gave Administrators full
control over Python and checked the boxes to inherit and
overwrite everything beneath. It warned me that only
heritable object would inherit, which raised my blood pressure
a few (more) points, then processed all the directories.

After that, the Admin subsystem worked.

BZDATETIME::2006-05-04 22:12:24
BZCOMMENTOR::Alan Meyer
BZCOMMENT::26

One more note.

I re-downloaded all of the packages we install with Python.
All turned out to be identical versions with what we have on
Mahler. Since Franck was done more recently than Mahler, we
seem to be in sync on all add-on packages.

BZDATETIME::2006-05-18 14:30:27
BZCOMMENTOR::Bob Kline
BZCOMMENT::27

Closed at status meeting per LG and AHM.