CDR Tickets

BZDATETIME::2010-06-10 13:48:10
BZCOMMENTOR::Bob Kline
BZCOMMENT::1

Bumped priority so this doesn't get permanently submerged.

BZDATETIME::2010-06-29 22:09:27
BZCOMMENTOR::Alan Meyer
BZCOMMENT::2

I had two problems making ParseSchema.exe.

The first problem was that I got a not found error for each of
the libraries named with a "LIBPATH" linker option.

Checking the documentation for cl.exe, it says that one must use
"/link" to signal the presence of flags to be passed to the
linker. Adding "/link" caused the errors to go away. I changed:

From:
LFLAGS =/LIBPATH:$(XML4C)/lib /LIBPATH:$(SABLOT)/lib \
To:
LFLAGS =/link /LIBPATH:$(XML4C)/lib /LIBPATH:$(SABLOT)/lib \

This fixed the linker problem, though I don't understand why our
makefile ever worked for ParseSchema. Have compiler flags
changed since we last built ParseSchema? The compiler is dated
2003 and our ParseSchema was dated 2005 though it's possible we
installed the 2003 compiler after the 2005 build. Or maybe this
never worked and one of us built the old copy of ParseSchema by
hand.

Moving right along, I reran make and came up with problems
finding cdr::Blob::Blob and cdr::CdrDoc::CdrDoc. This is
presumably the problem Bob had when trying to build ParseSchema.
However, checking cdr.lib with:

lib /list cdr.lib

I found those modules were in the library. So I didn't know what
went wrong and first checked paths, but didn't find any problems.

I then tried the following:

make clean
make
make ParseSchema.exe

The errors went away and a working ParseSchema.exe was created.
I don't know why. I renamed the old d:\cdr\bin\ParseSchema.exe
to ParseSchema.exe.old and copied in the new one to the \cdr\bin
directory.

I don't know what was wrong that a clean make fixed. But I can
now say that it all works for me. I recommend the following
test:

Try making ParseSchema.exe

If it fails, save the messages (I didn't do that) and try a clean
make as I did. I'll hold off marking this resolved fixed until
it's demonstrated to work for someone else too.

BZDATETIME::2010-06-30 08:53:08
BZCOMMENTOR::Bob Kline
BZCOMMENT::3

(In reply to comment #2)
> I had two problems making ParseSchema.exe.
>
> The first problem was that I got a not found error for each of
> the libraries named with a "LIBPATH" linker option.
>
> Checking the documentation for cl.exe, it says that one must use
> "/link" to signal the presence of flags to be passed to the
> linker. Adding "/link" caused the errors to go away. I changed:
>
> From:
> LFLAGS =/LIBPATH:$(XML4C)/lib /LIBPATH:$(SABLOT)/lib \
> To:
> LFLAGS =/link /LIBPATH:$(XML4C)/lib /LIBPATH:$(SABLOT)/lib \
>
> This fixed the linker problem, though I don't understand why our
> makefile ever worked for ParseSchema. Have compiler flags
> changed since we last built ParseSchema? The compiler is dated
> 2003 and our ParseSchema was dated 2005 though it's possible we
> installed the 2003 compiler after the 2005 build. Or maybe this
> never worked and one of us built the old copy of ParseSchema by
> hand.

The original ParseSchema sources and makefile are in the Validation directory in the version control archives. I believe you moved them into the Server directory some time last year and started the process of folding the ParseSchema build instructions into the master makefile for the CDR server. In that makefile, the LFLAGS macro was being fed to the linker, not the compiler, so the /link switch was not needed or appropriate. You might need to check to confirm that adding it to the invocation of the linker when the server is being built doesn't break anything. Might be better to put the "/link" in front of $(LFLAGS) when invoking the compiler for building ParseSchema. The original ParseSchema makefile has "/link" as part of the LFLAGS macro, but that's because the macro was only used for invocations of the compiler.

> I recommend the following test:
>
> Try making ParseSchema.exe

I think you're working with a version of the Server directory that I don't have. Go ahead and check in your changes to svn and I'll try them out. You'll need to use svn to move the ParseSchema sources from the Validation directory to the Server directory. Let me know if you need a refresher on how to move things in svn.

BZDATETIME::2010-07-01 23:50:39
BZCOMMENTOR::Alan Meyer
BZCOMMENT::4

Based on feedback from Bob I have done the following:

Changed the positioning of the /link option passed to cl.

Moved the Subversion home of ParseSchema.cpp from it's
location in the Validation directory to the Server directory.

Deleted two unused main programs from the Subversion
repository:

FindCdrIds.cpp
This finds all elements in a schema with cdr:id
attributes. We aren't using the program and can
reproduce it very quickly if we wish in Python.

ValidateDocAgainstSchema.cpp
We have valCdrDoc.py in the Utilities directory which
appears to be more capable.

The new makefile is in Subversion.

There is still one anomaly in the build that I'm not 100% happy
about. Linking ParseSchema produces the following warning:

LINK : warning LNK4089: all references to 'ADVAPI32.dll'
discarded by /OPT:REF

Although we don't specify /OPT:REF anywhere, It appears that it
is on by default. The /OPT:REF option is described here:

http://msdn.microsoft.com/en-us/library/bxwfs976%28VS.80%29.aspx

ADVAPI32.DLL is included in the list of libraries in the $(LIBS)
macro. If I take it out of the list, I get hard errors because
a couple of its functions are referenced in CdrLog.cpp. If I
leave it in, I get the warning.

I presume that what's happening is that the linker first tries to
find everything referenced in the link directive and declares an
error if it can't find something. Then at a later stage, which
it never gets to if the earlier stage emits any errors, it does
the /OPT:REF processing to remove things that were specified to
be linked but were not actually called.

We could remove CdrLog from the objects that we link in
ParseSchema, but that might lead to a (still more) convoluted
makefile. Furthermore, it's not a terrible idea to leave the
make rule alone so that if it ever becomes desirable to use the
CdrLog module in ParseSchema, which I admit is unlikely, we
won't have to modify the Makefile to do it.

We could:

1. Leave well enough alone.

2. Try to turn off warnings for LNK4089.

3. Add a comment to the make file to ignore this warning.

I haven't done any of those yet, but am leaning towards option 3.

BZDATETIME::2010-07-01 23:58:29
BZCOMMENTOR::Alan Meyer
BZCOMMENT::5

(In reply to comment #4)

> ...
> We could:
> 1 ... 2 ... 3 ...

A fourth option is to specify /OPT:NOREF in the linker options.
It will bloat the executable somewhat but otherwise do no harm.
However, it too would require a comment. So maybe 3 is still
best.

BZDATETIME::2010-07-02 11:33:09
BZCOMMENTOR::Bob Kline
BZCOMMENT::6

I vote for the options in this order:

4.
3.
2.
1.

BZDATETIME::2010-07-06 10:32:38
BZCOMMENTOR::Alan Meyer
BZCOMMENT::7

Implemented, tested, and documented option 4 (see comment #5 & #6.)

Marking issue resolved-fixed.

BZDATETIME::2010-07-06 10:39:08
BZCOMMENTOR::Bob Kline
BZCOMMENT::8

I tried building ParseSchema.exe using the modified Makefile, and got the following error:

LINK : fatal error LNK1181: cannot open input file 'CdrCommand.obj'

I had run 'svn up' followed by 'nmake clean' and 'nmake ParseSchema.exe'

BZDATETIME::2010-07-07 00:15:08
BZCOMMENTOR::Alan Meyer
BZCOMMENT::9

After a discussion between myself and Bob I've made numerous
changes to the makefile, including:

1. Eliminated CDRSVR macro.

We now no longer automatically install executables in the
\cdr\bin directory, a potentially dangerous practice.

2. Eliminated SERVICE macro.

It was only used in one spot. It seemed more reasonable to
put the defined object (CdrServer.exe) in that spot than to
define a macro for this one use.

This also eliminates possible confusing in which SERVICE did
not equal service.c but CdrServer.exe.

Also eliminated an unneeded -Fe compiler option.

3. Eliminated OBJS macro.

Everything now goes into the library. Nothing is linked from
the list of object modules. This is potentially less
efficient but also less confusing.

All executables now depend on the library. This should fix
the problem Bob encountered and prevent recurrences of
missing objects in the future.

4. Eliminated use of \cdr\lib.

The library "cdr.lib" is now built and used in only one
place, the server sandbox where all software is now compiled.
This simplifies locating things and eliminates possibly
confusing multiple copies of the same data.

I removed the existing \cdr\lib\cdr.lib so that if one of us
uses an obsolete makefile somewhere we won't get an old
library.

5. Fixed a typo in the _intall rule.

Changed "-net stop..." to "net stop...".

I have not tested this because I don't want to replace the
server on Mahler without consulting everyone.

6. Eliminated all /OPT:REF warnings.

I found a better way to do this than /OPT:NOREF. I now use
the linker flag /ignore:4089, which I've also documented in
the makefile.

7. Eliminated the old log messages.

The only one I saved was the "Initial version" message in
order to retain knowledge of the age of the file. However I
removed the version numbers to keep this as historical
information but avoid confusion about whether this is the
last log message and no other changes and log messages
followed it.

There were no references to Bugzilla in the log messages.

Currently, the only thing that I know is broken is the ccdoc
invocation. ccdoc crashes when we run it. This has been broken
for many years.

Our version of ccdoc dates from 2002-11-14. The latest on
sourceforge is dated 2004-09-30, which may or may not fix the
problem. I haven't tried to install it.

The new makefile version is in Subversion however I must have hit
a wrong key somewhere in the commit and my comments didn't make
it into the log. The comments I would have entered were:

All objects are now in cdr.lib. \cdr\lib is no longer used.
Macros supporting that directory and separate objs are now
gone. Old CVS comments have been removed. See Bugzilla
BZIssue::4663 for details.

BZDATETIME::2010-07-07 09:05:57
BZCOMMENTOR::Bob Kline
BZCOMMENT::10

(In reply to comment #9)

> 5. Fixed a typo in the _intall rule.
>
> Changed "-net stop..." to "net stop...".

That wasn't a typo. The hyphen tells make not to stop processing the rules if invocation of the command results in a non-zero status code.

> Currently, the only thing that I know is broken is the ccdoc
> invocation. ccdoc crashes when we run it. This has been broken
> for many years.

I'll look into that problem.

BZDATETIME::2010-07-29 14:57:21
BZCOMMENTOR::Bob Kline
BZCOMMENT::11

(In reply to comment #10)
> (In reply to comment #9)
>
> > 5. Fixed a typo in the _intall rule.
> >
> > Changed "-net stop..." to "net stop...".
>
> That wasn't a typo. The hyphen tells make not to stop processing the rules if
> invocation of the command results in a non-zero status code.

I put the hyphen back.

>
> > Currently, the only thing that I know is broken is the ccdoc
> > invocation. ccdoc crashes when we run it. This has been broken
> > for many years.
>
> I'll look into that problem.

Fixed. I modified the Makefile instructions slightly, so they work now (basically, I wipe out the existing files and database and start fresh, which presents no problems since the machine is so fast now). See:

http://mahler.nci.nih.gov/cdr/ccdoc/

BZDATETIME::2010-08-03 13:42:51
BZCOMMENTOR::Alan Meyer
BZCOMMENT::12

I propose that we close this one.

BZDATETIME::2010-08-03 15:27:50
BZCOMMENTOR::Bob Kline
BZCOMMENT::13

Done.