CDR Tickets

BZDATETIME::2009-03-05 15:58:00
BZCOMMENTOR::Bob Kline
BZCOMMENT::1

What does the government have for the current system (source code, requirements documentation, design documentation, user documentation)?

BZDATETIME::2009-03-10 13:36:15
BZCOMMENTOR::Margaret Beckwith
BZCOMMENT::2

I think all we have right now is a little bit of user documentation found in the general CIAT documentation guide.

BZDATETIME::2009-03-11 12:46:28
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::3

Margaret has requested the source code, system documentation, design documentation and user documentation from CIAT. CIAT will provide these documents as soon as possible.

BZDATETIME::2009-03-11 17:57:53
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::4

I have attached the CMS training documentation which gives a general overview of the CMS.

It starts with an explanation of the purpose, literature surveillance process including a diagram, overview of system features and interfaces, architecture and security then the rest is on how to use the system.

I will be attaching more documents in the next couple of days.

Attachment TrainDocFinal5-14.doc has been added with description: CMS Training Documentation-Overview of CMS

BZDATETIME::2009-04-20 13:20:33
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::5

(In reply to comment #4)
> Created an attachment (id=1637) [details]
> I will be attaching more documents in the next couple of days.

The remaining document(s) and import utility are included in the CD I gave to Bob in the CDR meeting on 03/12/2009.

BZDATETIME::2009-05-12 11:03:15
BZCOMMENTOR::Margaret Beckwith
BZCOMMENT::6

Upped priority to P4.

BZDATETIME::2009-05-14 23:21:40
BZCOMMENTOR::Alan Meyer
BZCOMMENT::7

I've read the following:

Citation Management System Training Documentation.

This appears to consist of an overview, followed by the
contents some of the help screens. There are some
redundancies in the document, but it appears to me to be
the the most useful document of those we have for anyone
to start with in learning the system - for programmers as
well as users.

NCI CMS Database Diagram.

This is high level only. It names the tables and shows
some relationships between them, but does not document
the contents or usage of any individual tables.

Some of the ASP code.

There's a lot of code. The CMS is only a fraction of the
size of the CDR, but it's a significant fraction.

Documentation in the ASP code is lacking, but the
presence of the help files and training document aids
greatly in understanding the code.

Some of the help files.

There are many of these. They look very explicit and
helpful. The help files appear to include the materials
used for training new users, but also some additional
materials useful in understanding how to administer the
system.

Next Steps
----------

The next steps that would be most useful to me in understanding
the system would be:

1. See demonstrations of the system - as discussed at today's
status meeting.

2. Get a better understanding of the database.

For the second task, it would be ideal if there were a test or
development version of the system with a copy of the database,
like our Mahler or Franck systems, that we could look at and play
with. We'd need direct, programmer level access to the database
in order to learn the most.

If this is not very practical, another approach might be to get a
dump of the database that we could load into SQL Server on
Mahler.

BZDATETIME::2009-05-19 15:05:56
BZCOMMENTOR::Alan Meyer
BZCOMMENT::8

William,

Could we get a copy of the CMS database? How big is it?

Thanks.

BZDATETIME::2009-05-19 15:35:03
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::9

(In reply to comment #8)
> William,
> Could we get a copy of the CMS database? How big is it?
> Thanks.

Its about 1.2GB (without the logs). I can get a copy on a DVD with a .bak file. It looks like you will need MS SQL Server 2K?

BZDATETIME::2009-05-19 15:47:21
BZCOMMENTOR::Alan Meyer
BZCOMMENT::10

(In reply to comment #9)

> Its about 1.2GB (without the logs). I can get a copy on a DVD with a .bak file.
> It looks like you will need MS SQL Server 2K?

That should fit easily on Mahler, and we have SQL Server 2000
there.

We might be able to get the whole thing working on Mahler, but
even if not, having the database will fill in the biggest piece
that's missing from just reading the ASP code.

Thanks.

BZDATETIME::2009-05-20 10:47:18
BZCOMMENTOR::Volker Englisch
BZCOMMENT::11

Bob: Didn't you want to reassign this task to Alan?

BZDATETIME::2009-05-20 11:42:19
BZCOMMENTOR::Bob Kline
BZCOMMENT::12

(In reply to comment #11)
> Bob: Didn't you want to reassign this task to Alan?

Right.

BZDATETIME::2009-05-28 09:17:45
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::13

It seems this is the diagram Lakshmi talked about during the CMS demo on Tuesday.

Attachment CMS_DIAGRAM.pdf has been added with description: CMS Processing Diagram

BZDATETIME::2009-07-09 22:16:11
BZCOMMENTOR::Alan Meyer
BZCOMMENT::14

I've mounted the May 19, 2009 dump of the Citation Maintenance
System database provided by William on Mahler.

The database is named "nci_cms". It is accessible by user "cdr",
but I've made it read-only. For now I think we don't want to
break anything in the database, just inspect the data.

(For future reference, I granted access to CDR by using
Enterprise Manager, clicking down the database tree to nci_cms,
right clicking "Users", and selecting "New Database User". I
also granted SELECT on all tables and views.)

I've prepared a number of SQL scripts to work with the database,
including a complete SQL Server auto-generated database creation
script.

Everything, including the backup file from which the database was
loaded, is in:

mahler\d$\cdr\CitationMaintenanceSystem\Database...

See the Readme.txt file in that directory.

I put the data files in the usual place:

mahler\d$\mssqldata\MSSQL\Data

BZDATETIME::2009-07-09 23:54:45
BZCOMMENTOR::Alan Meyer
BZCOMMENT::15

I am thinking, very provisionally, that a reasonable plan for
dealing with Citation Maintenance after the Lockheed Martin
support ends is to:

1. Develop a new system, integrated with the CDR, to replace the
Citation Maintenance System.

2. While that is being developed, run the existing system on a
server at CIPS or Z-Tech with zero or very minimal
programming changes.

In regards to that possible plan, I have some questions for
William about the existing Citation Maintenance System. These
are questions that will probably require some research to answer:

1. How much technical (i.e., programmer type) support is
currently required, if any, to keep the existing system
running? This might be estimated by finding out how many
hours the technical staff devoted to the system in the last
year.

2. What kind of technical support is required? How much is
programmer support to fix bugs or diagnose errors in the
database? How much is system administration to do backups,
apply system software patches, etc.?

3. If there is a technical problem with the system, who does
CIAT turn to for help? Will any of those people be going to,
or be accessible by, Z-Tech?

4. Who are the experts in the system from a user (i.e.,
non-programmer) point of view? Will any of those people be
going to, or be accessible by, Z-Tech?

5. If the existing system needed to run for an extended period
with no changes whatsoever, up to say one year, would that be
practical? Or are there changes that must be made in order
to keep it running for that period?

6. If the system were re-implemented, starting soon, would there
be someone who could put together a list of problems that
should be fixed and changes that should be made? This should
include bug fixes, requirements that aren't currently met but
need to be, and desirable features that would significantly
improve usability or productivity if they were added.

7. If there is a person who can do that, will the person be
available to Z-Tech?

If you have any ideas about related questions that I didn't think
of but should have, please let me know.

Thanks.

BZDATETIME::2009-07-16 11:37:10
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::16

(In reply to comment #15)

I am including the first set of answers, mostly from Cynthia Bogess. Pete (the primary programmer) would provide additional information sometime next week and I will post that information too.

> In regards to that possible plan, I have some questions for
> William about the existing Citation Maintenance System. These
> are questions that will probably require some research to answer:
> 1. How much technical (i.e., programmer type) support is
> currently required, if any, to keep the existing system
> running? This might be estimated by finding out how many
> hours the technical staff devoted to the system in the last
> year.

From the users' perspective, there is minimal maintenance either on a monthly or yearly basis. They have gone to the programmers to fix problems when they occurred or when they needed ad hoc reports but the problems have not been many. Pete can provide some figures later.

> 3. If there is a technical problem with the system, who does
> CIAT turn to for help? Will any of those people be going to,
> or be accessible by, Z-Tech?

CIAT contacts Pete and Lijuan. Pete would be the best to answer this question. I will post his response next week.

> 4. Who are the experts in the system from a user (i.e.,
> non-programmer) point of view? Will any of those people be
> going to, or be accessible by, Z-Tech?

Cynthia is the primary expert and Minaxi is the backup and they have said they will be going Z-Tech.

> 5. If the existing system needed to run for an extended period
> with no changes whatsoever, up to say one year, would that be
> practical? Or are there changes that must be made in order
> to keep it running for that period?

Yes. The system can run for that period as long as occasional technical problems are fixed as well as the maintenance that Pete and Lijuan do are also taken care of.

> 6. If the system were re-implemented, starting soon, would there
> be someone who could put together a list of problems that
> should be fixed and changes that should be made? This should
> include bug fixes, requirements that aren't currently met but
> need to be, and desirable features that would significantly
> improve usability or productivity if they were added.

Yes. Cynthia can provide this information. Cynthia worked with Margaret and Lakshmi as well as Lockheed (Aspen) Programmers to design the system so she knows all the ins and outs of the system and can provide this information when needed.

BZDATETIME::2009-07-16 14:31:43
BZCOMMENTOR::Alan Meyer
BZCOMMENT::17

(In reply to comment #16)
> (In reply to comment #15)
>
> I am including the first set of answers, mostly from Cynthia
> Bogess. Pete (the primary programmer) would provide additional
> information sometime next week and I will post that information
> too.
> ...

The answers you have provided appear to me to support the
provisional plan in comment #15 of running the existing system
unchanged while developing a new one in parallel.

I'll continue working on the assumption that that is how we will
proceed.

At our next CDR status meeting I would like us all to discuss
that and, if it is agreed, to discuss how to proceed with regard
to both parts of the plan.

One question we'll need to answer to develop a new system is, how
much time would we want from Cynthia for the following tasks:

Document new functionality that should be included in a
re-write of the system.

Answer questions about existing functionality.

Provide review, criticism, test, and feedback on designs and
software that we develop.

Since Cynthia has done this before, she'll probably have some
good ideas about how much time would be needed, and how much she
can make available.

On our side we'll also need to develop an estimated level of
effort. Of course we don't have enough information to do that
yet, but we'll get there.

Judging from what I've seen of the database and the software, it
looks to me like we could host the system on a small Windows
server, or an existing server here at OCE that has some excess
capacity.

I have talked to Reza today to keep him in the loop on all of
this.

BZDATETIME::2009-07-16 14:52:02
BZCOMMENTOR::Margaret Beckwith
BZCOMMENT::18

We are going to have a meeting to discuss this and the genetics directory next Thursday morning at 9:30. I sent out an email earlier today. Lakshmi is leaving in the afternoon to go to India but would like to discuss both of these issues before leaving. William--I don't think I included you on the email, but it would probably be good if you could join as well. We could give you a call.

BZDATETIME::2009-07-16 16:52:49
BZCOMMENTOR::Alan Meyer
BZCOMMENT::19

In looking at the existing Citation Management database, I can
figure out the purpose most of the tables and columns by reading
the data in them and the code that processes them. However,
before I do that, I would like to know if it has already been
done.

Does CIAT or anyone already have database documentation? We've
got the SQLServer generated diagram and schema declarations, but
what I'm looking for is something that explains the meaning of
each table and each column. I think the most likely person to
have such documentation would be one of the programmers.

The way we have done this in the CDR is with our "tables.sql"
file, which has data definition language for each table, index
and view, together with comments giving a several sentence
description of the purpose of each table or view, and a single
line or so describing the meaning of each column.

If nothing like that exists, it might be worthwhile for us to
create it. If a programmer who already knows the tables were to
do it that would be the most efficient way. However, if that is
not practical, I think I could do it.

I estimate the level of effort involved for this task to be
something like the following:

Assumptions:

~35 tables + 1 view
~115 columns

(There are 41 tables and 127 columns in the sample database,
but some of that looks like temporary stuff created for
debugging.)

Estimates:

For someone who already knows the data:

10 minutes per table
5 minutes per column

Total = 35*10 + 115*5 = 695 minutes = ~2 days

For someone who has to figure it all out:

Maybe 2-3 times as long, say ~4-6 days.

I think this would be worthwhile because understanding the
database structure is fundamental to understanding the complete
application. If there data is stored in the existing system that
doesn't show up in our design for a new system, then there's a
good chance we've missed a requirement.

It may also be the case that we'll want to load the existing data
into the new system for continuity. Mapping the fields will be
easier if we have documentation for them.

I'll hold off documenting the data myself unless and until we
find out that:

No existing documentation is available.

No Citation Maintenance System programmer time is available
to produce it.

Can we get the answers before next Thursday's meeting (July 23,
2009)?

BZDATETIME::2009-07-16 16:54:05
BZCOMMENTOR::Alan Meyer
BZCOMMENT::20

I notice that Bob is not on the CC list for this issue.
I'm adding him.

BZDATETIME::2009-07-16 20:10:29
BZCOMMENTOR::Alan Meyer
BZCOMMENT::21

(In reply to comment #19)
> ...
> Estimates:
>
> For someone who already knows the data:
>
> 10 minutes per table
> 5 minutes per column
>
> Total = 35*10 + 115*5 = 695 minutes = ~2 days
>
> For someone who has to figure it all out:
>
> Maybe 2-3 times as long, say ~4-6 days.

Upon reflection, I think we could actually do all of this twice
as fast as the above estimates. Doing this kind of documentation
is a matter of degree. I could probably do something pretty
useful in just one day if that's all we want to spend early in
the project.

BZDATETIME::2009-07-16 20:49:19
BZCOMMENTOR::Alan Meyer
BZCOMMENT::22

I think the next step in this project is to create a requirements
document. I would think it should contain answers to the
following questions:

What data must be maintained?

Who are the users?

What functions must be performed?

What user interfaces are required?

How should this system integrate with the CDR?

What existing data needs to be converted and loaded into the
new system?

To answer these questions I plan to:

Mine Cynthia's "Citation Management System Training
Documentation".

Get a demonstration of the system in action, making notes.

Analyze and document the existing data and, to some extent,
the existing code.

Get advice and documentation from Cynthia and any other users
who can help describe any changes or new features that are
needed or desired in the way the system works now.

Get advice from CIPS (Lakshmi, Margaret, board managers) on
any changes or new features they would like to see, including
the integration of the new system into the CDR.

Categorize requirements:

Categorize priority levels: Necessary vs. Desirable.

Identify high risk items:

Hard to implement functions.

Dependencies on factors outside our control (like NLM
data formats)

Critical performance issues.

By risk:

We need to highlight any high risk requirements, like
dependencies on other systems

Write it all up in a first draft for review.

That's the plan I'm following at the moment.

If anyone has suggestions or existing documentation I can use,
please send them to me.

BZDATETIME::2009-07-21 12:41:55
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::23

(In reply to comment #15)

I have additional additional answers to your questions from the programmers.

> In regards to that possible plan, I have some questions for
> William about the existing Citation Maintenance System. These
> are questions that will probably require some research to answer:
> 1. How much technical (i.e., programmer type) support is
> currently required, if any, to keep the existing system
> running? This might be estimated by finding out how many
> hours the technical staff devoted to the system in the last
> year.

During the period 12/28/08 thru 6/07/09 sixty-nine hours were billed for system maintenance by the programming staff.

> 2. What kind of technical support is required? How much is
> programmer support to fix bugs or diagnose errors in the
> database?
The programmer helps to resolve occasional problems with data loads, ad-hoc reports, minor enhancements and bug fixes. We don’t have an estimate for the sys and database administration charges as these charges were shared with several systems. The system is currently running IIS, MS-SQL Server 2000.

>How much is system administration to do backups,
>apply system software patches, etc.?
We don’t have an estimate for the sys and database administration charges as these charges were shared with several systems. The system is currently running IIS, MS-SQL Server 2000.

> 3. If there is a technical problem with the system, who does
> CIAT turn to for help?

The CIAT client would contact the project staff associated with this effort (Cynthia Boggess, Minaxi Trivedi – currently LM employees).
If the issue could not be resolved, Cynthia Boggess or Minaxi Trivedi would contact development staff within the Lockheed Health Sys group that have been responsible for systems support.

>Will any of those people be going to,
> or be accessible by, Z-Tech?

The development staff with knowledge specific to the Cite Management System are not going to Z-Tech

> 5. If the existing system needed to run for an extended period
> with no changes whatsoever, up to say one year, would that be
> practical?

The system could run for a year with no major enhancements assuming the environment (IIS and SQL Server) were adequate and maintained.

>Or are there changes that must be made in order
>to keep it running for that period?

There is a specific enhancement that has been identified (not critical, but important), and there would be small tasks associated with data loads, etc. that would need attention from development staff.

BZDATETIME::2009-07-23 12:33:12
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::24

(In reply to comment #19)
> In looking at the existing Citation Management database, I can
> figure out the purpose most of the tables and columns by reading
> the data in them and the code that processes them. However,
> before I do that, I would like to know if it has already been
> done.
> Does CIAT or anyone already have database documentation? We've
> got the SQLServer generated diagram and schema declarations, but
> what I'm looking for is something that explains the meaning of
> each table and each column. I think the most likely person to
> have such documentation would be one of the programmers.

The programmers do not have a database documentation but they said that they can put one together for you if you want them to. That is, a documentation explaining the tables and columns in the database.

BZDATETIME::2009-07-23 15:40:07
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::25

With regards to the question about how long it would take Pete to create a basic spreadsheet with all the tables and columns as well as basic descriptions of the columns, he said it would take him between 8 - 10 hours to complete it and he intended to charge CIAT for the time.

BZDATETIME::2009-07-23 16:24:32
BZCOMMENTOR::Margaret Beckwith
BZCOMMENT::26

William, I think it is fine for Pete to go ahead with this. If he gets to 10 hours we would like to see what he has done and decide if it is adequate. I will also let Debbie know that this is being done.

BZDATETIME::2009-07-23 16:26:35
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::27

(In reply to comment #26)
> William, I think it is fine for Pete to go ahead with this. If he gets to 10
> hours we would like to see what he has done and decide if it is adequate. I
> will also let Debbie know that this is being done.

Sure - I will let Pete know to proceed with it. Thanks!

BZDATETIME::2009-07-23 23:31:20
BZCOMMENTOR::Alan Meyer
BZCOMMENT::28

The attached was sent to Reza as documentation of our discussion
this afternoon of issues in planning for bringing the CiteMS
in house before September 30.

Attachment CiteMSPlanning01.rtf has been added with description: Planning document for CTB management

BZDATETIME::2009-07-29 00:10:06
BZCOMMENTOR::Alan Meyer
BZCOMMENT::29

I went over some of the issues regarding hosting the CiteMS
system here at CTB with John Rehmert. He told me about the
security issues I need to consider.

If some of the users of the import utility work at Z-Tech, I
think we can handle the security issues via a VPN connection
between the user's workstation and SQL Server port on our server
here.

If the import utility is run here at NCI, this issue goes away.

The other, obvious approach, to make security issues go away is
to have Z-Tech host the system and have us access it there, as
was done with Aspen / Lockheed Martin.

There is strong interest by both Reza and John in having the
system hosted at Z-Tech. The reasons are:

1. To avoid the security hassles attending every NCI system used
outside the LAN.

2. To avoid having another system to support here, especially
since there are other contracts expiring on September 30 and
other requests for CTB to host systems by that date.

We certainly need to consider this and should probably explore
the issues with Z-Tech.

I'm not the right person to establish contact with them. The
contractual and cost issues are outside my competence. However I
would like to participate in any discussion we have in order to
explain how the system works and ask questions.

Can someone establish contact?

Who handles that?

BZDATETIME::2009-07-29 00:12:02
BZCOMMENTOR::Alan Meyer
BZCOMMENT::30

John Rehmert wanted access to the CiteMS system so he could
look at hosting issues here at CTB.

By his request, I've copied all of the CiteMS files to our
network drive at:

nci6116g\group\oce\ctb\PROJECTS\Citation_Management_System

I notified John and Reza that they're there.

BZDATETIME::2009-07-29 00:31:46
BZCOMMENTOR::Alan Meyer
BZCOMMENT::31

I've read some of the source code for the import utility, looking
for how the program connects to SQL Server.

I found three places where connections can be established, but I
think only one is actually used. It looks like the others were
older techniques for connecting to an Access database, or for
connecting a SQL Server database on a server that was replaced by
a newer one. It appears that the older connection functions are
just there for reference to how it was done in the past.

See:

"Public Sub DataConnect()" in "basMain.bas" for details.

Changing this to work with a server elsewhere looks clean and
easy.

BZDATETIME::2009-07-30 13:04:11
BZCOMMENTOR::Bob Kline
BZCOMMENT::32

We need to make sure this gets wrapped up before September 30.

BZDATETIME::2009-07-30 17:42:31
BZCOMMENTOR::Alan Meyer
BZCOMMENT::33

I checked with John Rehmert regarding the security review that may have already been done for the existing Citation Management System at CIAT.

He said that if the security review was done in the last year or two, it's probably all that needs to be done. He said he could look at whatever documents were generated for that review, if we obtained them for him, and give us his thoughts about whether they're likely to be acceptable to the security department. The security people themselves will have the last word.

If the security review is more than a couple of years old, John thought it more likely that the department here would ask for a new one.

BZDATETIME::2009-07-31 11:29:58
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::34

I have attached the excel file with definitions of Tables and columns (from Pete). He said you should let him know if you have any questions.

Attachment CiteMgtSys_DataDictionary.xlsx has been added with description: Citation Management System Data Dictionary

BZDATETIME::2009-07-31 11:35:23
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::35

Here is another file he included.

Attachment CiteMgtSys_CreateScript.doc has been added with description: Citation Management System Script

BZDATETIME::2009-08-04 17:08:52
BZCOMMENTOR::Alan Meyer
BZCOMMENT::36

Adding Brent Carter to the CC list for this project.

Brent has experience with VB and ASP.

BZDATETIME::2009-08-04 20:58:30
BZCOMMENTOR::Alan Meyer
BZCOMMENT::37

William,

I have some questions for the programmers relating to the
environment needed to compile and run the import utility program.

What compiler should be used to compile the code? VB5? VB6?

How do you currently install the application on a user's machine?

Copy the application and all DLL and OCX files into a single
directory?

Is there any other software we need to operate the import
utility?

Can you package up the DLL and OCX files we need in a zip file,
to be sure we have the versions that you know work?

Here are the library references included in the Cips_CMS.vbp
file:

Reference=*\G{00020430-0000-0000-C000-000000000046}#2.0#0#..\WINNT\System32\stdole2.tlb#OLE Automation
Reference=*\G{00000200-0000-0010-8000-00AA006D2EA4}#2.0#0#..\Program Files\Common Files\System\ADO\msado20.tlb#Microsoft ActiveX Data Objects 2.0 Library
Reference=*\G{56BF9020-7A2F-11D0-9482-00A0C91110ED}#1.0#0#..\WINNT\System32\msbind.dll#Microsoft Data Binding Collection
Reference=*\G{642AC760-AAB4-11D0-8494-00A0C90DC8A9}#1.0#0#..\WINNT\System32\MSDBRPTR.DLL#Microsoft Data Report Designer v6.0
Reference=*\G{6B263850-900B-11D0-9484-00A0C91110ED}#1.0#0#..\WINNT\system32\msstdfmt.dll#Microsoft Data Formatting Object Library
Reference=*\G{3D5C6BF0-69A3-11D0-B393-00A0C9055D8E}#1.0#0#..\Program Files\Common Files\designer\MSDERUN.DLL#Microsoft Data Environment Instance 1.0
Reference=*\G{7C0FFAB0-CD84-11D0-949A-00A0C91110ED}#1.0#0#..\WINNT\System32\msdatsrc.tlb#Microsoft Data Source Interfaces
Object={67397AA1-7FB1-11D0-B148-00A0C922E820}#6.0#0; MSADODC.OCX
Object={831FDD16-0C5C-11D2-A9FC-0000F8754DA1}#2.0#0; MSCOMCTL.OCX
Object={5E9E78A0-531B-11CF-91F6-C2863C385E30}#1.0#0; msflxgrd.ocx
Object={3B7C8863-D78F-101B-B9B5-04021C009402}#1.2#0; richtx32.ocx
Object={F9043C88-F6F2-101A-A3C9-08002B2F49FB}#1.2#0; comdlg32.ocx

Thanks.

BZDATETIME::2009-08-04 21:48:33
BZCOMMENTOR::Bob Kline
BZCOMMENT::38

Alan:

I looked pretty thoroughly, and I don't believe I have a copy of VB5 (or VS98) any more. On the other hand, from what I've read VB6 should be able to compile VB 5 code. We'll see, I guess.

BZDATETIME::2009-08-06 16:49:58
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::39

(In reply to comment #37)
> William,
> I have some questions for the programmers relating to the
> environment needed to compile and run the import utility program.

Here the responses from Pete.

> What compiler should be used to compile the code? VB5? VB6?

VB6

> How do you currently install the application on a user's machine?
> Copy the application and all DLL and OCX files into a single
> directory?

We have the .exe and config file installed on a server (and a shortcut from user's machine).

> Is there any other software we need to operate the import
> utility?

No

> Can you package up the DLL and OCX files we need in a zip file,
> to be sure we have the versions that you know work?

We will try to package these files based on the list below.

BZDATETIME::2009-08-06 18:37:28
BZCOMMENTOR::Brent Carter
BZCOMMENT::40

I discussed deployment with John Rehmert. Assuming we're going to run this from our network and let users VPN in, the ideal case from his perspective is that we run it on Windows Server 2008 with IIS 7 and a SQL Server 2005 database.

I set up a Virtual Machine running Windows Server 2008 with IIS 7 and configured it to run "classic" ASP applications. I also imported the database backup into an instance of SQL Server 2005 Express running on my local machine. I copied the existing ASP code and changed the connection string to point to my SQL Server instance. The code was using a file DSN ("FILEDSN=nci_cms.dsn"). Since we don't seem to have this DSN file, and for consistency, I changed this to use the same DSN-less connection string being used by the VB 6 client application. The only change I made to that connection string was the host name of the server running the database. Before doing significant testing we should probably get the database up and running on one of our database servers.

Alan tested the ASP application from his PC from inside the office and plans to do a test over VPN as well. The site seems to run fine and there are no obvious problems thus far. We were able to log in, view a few different pages, and do searches against the database.

So, assuming Alan has no problems over VPN, this approach seems to be a viable option that shouldn't require a lot of changes (at least for the ASP site - we still need to get the VB 6 client application working).

BZDATETIME::2009-08-06 20:30:04
BZCOMMENTOR::Alan Meyer
BZCOMMENT::41

I will test the system tomorrow from home using the VPN. I
expect it to work fine since the VPN should produce the same
conditions under which I connected successfully from my office
workstation.

William,

Can you arrange for someone at CIAT who knows the system to login
and test the system? Brent and I were able to log in
successfully but it would be good for a user who really knows the
system to test it and be sure that it really is doing the right
things.

If you can do that, Brent or I can send instructions for how to
get into it. It's running right now on Brent's workstation so
he'll want to know when someone will be able to test so he can be
sure that the system is up at that time.

Thanks.

BZDATETIME::2009-08-06 20:37:03
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::42

I am assuming the person at CIAT will have to login via VPN? If you send me the login information, I can get Cynthia to login from home tomorrow.

BZDATETIME::2009-08-06 20:58:24
BZCOMMENTOR::Alan Meyer
BZCOMMENT::43

(In reply to comment #42)
> I am assuming the person at CIAT will have to login via VPN? If
> you send me the login information, I can get Cynthia to login
> from home tomorrow.

Brent has gone home for the evening and may have amendments when
he reads this, but here is the procedure I used today.

1. Cynthia should add a line to the hosts file on her computer.

The file is: "c:\Windows\system32\drivers\etc\hosts"

At the bottom of the file she needs to add this line:

156.40.132.21 CitationManagementSystem

On my workstation this resulted in the following as the last
two lines in the file:

127.0.0.1 localhost
156.40.132.21 CitationManagementSystem

There may be other lines in the file. That's fine.

This change is needed because we have no public name mapping
from our network to Brent's machine. When we put the system
up for real on a production server we probably won't need that
any more.

2. Use the following URL to get in to the system:

http://citationmanagementsystem

The database that is loaded is the backup version that you gave
me dated May 19, 2009. It's not the up to date version. I think
any userid and password that was active at that time would work.

Since the Bugzilla database is visible to lots of people, and
since the production system may have the same userids and
passwords, I won't put those in this posting. If Cynthia has a
problem getting in she can contact us and we'll figure it out.

If Cynthia wants to wait, I'll confirm late tonight or tomorrow
that the VPN connection is working.

BZDATETIME::2009-08-06 21:50:40
BZCOMMENTOR::Alan Meyer
BZCOMMENT::44

In comment #15 I suggested a two part plan:

> 1. Develop a new system, integrated with the CDR, to replace the
> Citation Maintenance System.
>
> 2. While that is being developed, run the existing system on a
> server at CIPS or Z-Tech with zero or very minimal
> programming changes.

With Brent's excellent help, we are well on the road to getting
number 2 working.

Number 1 is a more daunting problem.

As I see it, the reasons for rewriting the software are:

a. To integrate the system with the CDR.

The goal would be to utilize CDR facilities and database
tables where they apply to the CiteMS, rather than having
two systems that do some of the same things.

b. To modernize the functionality of the system.

It is our understanding that some lessons have been learned
since the original implementation that might enable us to do
some things better.

There may also be some new opportunities created for us by
the increasing use of electronic mail and electronic
publishing (e.g., journal articles in PDF format) that might
be worth incorporating in a new system.

c. To modernize the computer technology.

The "Classic" ASP and Visual Basic 6 in which the systems
are written are now years out of date. The Visual Basic can
only be compiled using an old version of Microsoft's
compiler tools.

Newer tools are available with higher capability,
productivity and maintainability than the older ones.

It might also be desirable to re-implement the client import
utility as a web application so the entire program will be
available from anywhere.

It might also be possible to re-implement the import utility
using newer tools from NLM that no longer require an
intermediate file in old Medline format, though that may or
may not be worth the trouble.

However, building a new system is not a trivial undertaking. I
have looked through the system and now estimate it to contain
the following:

Web application:

~125 ASP files.
~25,000 lines of HTML, ASP, and BASIC code.

Import utility:

21 Visual Basic files of various types.
~12,500 lines of form definition and code.

For comparison, the entire CdrServer contains only 10,298 lines
of C++ code (i.e., lines with semicolons), though the CDR Python
programs appear to contain 10-15 times that number.

Obviously, the staff worked very hard to produce this
application. It would be easier to reproduce today because we
have better technology to work with and, especially, because we
have the existing system to show us what works and experienced
users who know what worked well and what didn't. But it would
still be a good sized job.

So, much as I like the idea of re-implementing the system, I
should note that there is an alternative way to achieve "CDR
integration" at lower cost.

We could modify the existing system to directly exchange data
with the CDR. This could be done in different ways, trading off
the degree of integration and modernization against the level of
effort.

If we want a lot of integration and modernization, this will be a
poor choice. We're better off starting fresh with new
technology. If we can accept a small degree of integration and
modernization, this could be, at least in the short term,
cheaper.

BZDATETIME::2009-08-07 08:49:39
BZCOMMENTOR::Brent Carter
BZCOMMENT::45

(In reply to comment #43)
> > I am assuming the person at CIAT will have to login via VPN? If
> > you send me the login information, I can get Cynthia to login
> > from home tomorrow.
> Brent has gone home for the evening and may have amendments when
> he reads this, but here is the procedure I used today.
> 1. Cynthia should add a line to the hosts file on her computer.
> The file is: "c:\Windows\system32\drivers\etc\hosts"
> At the bottom of the file she needs to add this line:
> 156.40.132.21 CitationManagementSystem
> On my workstation this resulted in the following as the last
> two lines in the file:
> 127.0.0.1 localhost
> 156.40.132.21 CitationManagementSystem
> There may be other lines in the file. That's fine.
> This change is needed because we have no public name mapping
> from our network to Brent's machine. When we put the system
> up for real on a production server we probably won't need that
> any more.
> 2. Use the following URL to get in to the system:
> http://citationmanagementsystem

I've updated the website configuration to make it a little easier to connect and verified it is accessible from another computer on the network. So, if connected by VPN, Cynthia should be able to skip step #1 above and just go to this URL: http://156.40.132.21:8001

BZDATETIME::2009-08-07 09:37:51
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::46

(In reply to comment #45)
> (In reply to comment #43)
> I've updated the website configuration to make it a little easier to connect
> and verified it is accessible from another computer on the network. So, if
> connected by VPN, Cynthia should be able to skip step #1 above and just go to
> this URL: http://156.40.132.21:8001

I just found out that Cynthia has taken the day off. She will be working on Monday so it looks like Monday will be the appropriate time to test the connection. I hope that is Okay ?

BZDATETIME::2009-08-07 11:32:31
BZCOMMENTOR::Alan Meyer
BZCOMMENT::47

(In reply to comment #46)
> ...
> I just found out that Cynthia has taken the day off. She will be working on
> Monday so it looks like Monday will be the appropriate time to test the
> connection. I hope that is Okay ?

Monday is fine.

I tested from home today and was able to get in and do
things. I'm not expecting any problems.

BZDATETIME::2009-08-07 15:38:38
BZCOMMENTOR::Brent Carter
BZCOMMENT::48

I set up the VB Import application on a share on the same Windows Server 2008 virtual machine the ASP website is running on. In addition to testing the ASP website, if Cynthia can test this as well that would be great. She should be able to run the application directly from the share.

Once connected over VPN, she can just select "Start > Run" and then type
156.40.132.21\CiteMS in the text box and click the "Ok" button. She should see a Windows Explorer window open. Double clicking "Cips_CMS.exe" should open the import application.

Issues found so far:

1. The report feature doesn't work (it breaks due to a printer issue). This needs more investigation.

2. Even though the application runs from a share, certain files like VB runtime and referenced components have to be registered on the user's local machine. This issue manifests itself with an error such as "Run-time error '339': Component 'MSFLXGRD.OCX' or one of its dependencies not correctly registered: a file is missing or invalid." I tested from two machines - one worked fine and the other has this issue. If Cynthia tests from a machine that has run this application before it will be more likely to work.

To resolve this issue we may need an installer to ensure these files are present although they probably already exist for current users of the system. Another option would be to set up a VM people could log into over VPN and just let them run the application from there. That would certainly be easier from a maintenance standpoint - one box to get configured correctly and then maintain rather than potential configuration and maintenance issues on multiple user machines. There may be licensing considerations with that approach, however.

William: can you please ask the programmers if there is/was any kind of installer program and if so is the source available? If not how did they get dependencies registered on user machines?

There are the issues above and some other small issues to figure out but I think we've got a pretty good proof of concept at this point and no obvious roadblocks.

BZDATETIME::2009-08-10 15:23:49
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::49

Cynthia was able to connect to the Database but not the import utility. The screen shots are attached.

Here is the email from her:

"I was able to connect to the CMS web interface with no problems. I checked both staff and client logins. However, I was unable to connect to the import utility. I am attaching screen shots of the error messages that came up when I tried to login."

Attachment NCI_CMS_Host_test.docx has been added with description: import utility error messages

BZDATETIME::2009-08-11 11:32:28
BZCOMMENTOR::Brent Carter
BZCOMMENT::50

(In reply to comment #49)
> Cynthia was able to connect to the Database but not the import utility.

We were able to reproduce the error Cynthia saw and I've made some updates that fixed the error for us. Could you ask Cynthia to give the import utility another try?

BZDATETIME::2009-08-11 11:33:08
BZCOMMENTOR::Brent Carter
BZCOMMENT::51

I was wondering if it would be possible to get a phone number in case we wanted to speak to the programmers directly - some issues require back and forth and it might make things a little more efficient.

BZDATETIME::2009-08-11 11:38:38
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::52

(In reply to comment #50)
> (In reply to comment #49)
> > Cynthia was able to connect to the Database but not the import utility.
> We were able to reproduce the error Cynthia saw and I've made some updates that
> fixed the error for us. Could you ask Cynthia to give the import utility
> another try?

Sure. I have asked Cynthia to try again.

The lead programmer is Pete Chauvette - 301-519-5564. Email: peter.m.chauvette@lmco.com

BZDATETIME::2009-08-11 14:01:47
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::53

(In reply to comment #52)
> (In reply to comment #50)
> > (In reply to comment #49)
> > > Cynthia was able to connect to the Database but not the import utility.
> > We were able to reproduce the error Cynthia saw and I've made some updates that
> > fixed the error for us. Could you ask Cynthia to give the import utility
> > another try?
> Sure. I have asked Cynthia to try again.

Cynthia was able to login and use the import utility.

BZDATETIME::2009-08-18 13:25:29
BZCOMMENTOR::Alan Meyer
BZCOMMENT::54

In experimenting with the existing web application, Brent was
able to inject SQL into an input screen, something that is a
potential security hole. He is investigating further to see how
many such vulnerabilities he can find and how much work is needed
to secure them.

In order for anyone to exploit this vulnerability they would
still need to be logged in to the system, which means they would
need a valid userid and password. We can strengthen that
security by only allowing users on the NIH network - either
inside NIH or on the VPN - to access the system. Then an
attacker would have to have both an NIH network userid/password
and a CiteMS userid/password.

This will work fine for all users at OCE. Will it be okay for
everyone using the system at CIAT? Is there a need for anyone
outside the NIH network or the VPN to use the web application?

BZDATETIME::2009-08-18 14:54:35
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::55

(In reply to comment #54)
> In experimenting with the existing web application, Brent was
> able to inject SQL into an input screen, something that is a
> potential security hole. He is investigating further to see how
> many such vulnerabilities he can find and how much work is needed
> to secure them.
> In order for anyone to exploit this vulnerability they would
> still need to be logged in to the system, which means they would
> need a valid userid and password. We can strengthen that
> security by only allowing users on the NIH network - either
> inside NIH or on the VPN - to access the system. Then an
> attacker would have to have both an NIH network userid/password
> and a CiteMS userid/password.
> This will work fine for all users at OCE. Will it be okay for
> everyone using the system at CIAT? Is there a need for anyone
> outside the NIH network or the VPN to use the web application?

I think this should be find with CIAT since Cynthia will be logging in via VPN and Minaxi will be on a site-to-site VPN at Z-Tech and if she wants to work from home she can log in using VPN. There are no other users at CIAT apart from Cynthia and Minaxi.

BZDATETIME::2009-08-20 14:56:14
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::56

Alan:
Do you have a specific date or period which this will be completed? The transition team wants to know so that they can plan appropriately since we are only about a month away from transiting to Z-Tech.

BZDATETIME::2009-08-21 00:22:31
BZCOMMENTOR::Alan Meyer
BZCOMMENT::57

(In reply to comment #56)
> Alan:
> Do you have a specific date or period which this will be
> completed? The transition team wants to know so that they can
> plan appropriately since we are only about a month away from
> transiting to Z-Tech.

We don't yet have a specific date. Brent and I have discussed
this at length and Brent has done a lot of analysis of what needs
to be done. We think that the technical challenges are not too
great IF the NCI management and security people are willing to
accept the system without a lot of change.

One of the key people who needs to participate in this process is
out this week and we won't be able to make progress on that issue
until he comes back next week.

Brent and I are going to push as best we can to get things ready
for CIAT testing no later than mid-September, and earlier if
possible. We'll keep everyone informed about progress.

BZDATETIME::2009-08-21 00:26:47
BZCOMMENTOR::Alan Meyer
BZCOMMENT::58

William,

One thing we'll need to do is to ensure that we have a final copy
of the Citation Management System code.

Brent suggested and I agree that we should do the following:

Ask the programmers at CIAT to totally freeze the code, not
change anything unless it is essential to keep the system
running.

Send us the latest copy of the code, as of the freeze.

Send us the version control database for the code.

Can that be arranged for the near future?

BZDATETIME::2009-08-21 09:11:13
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::59

(In reply to comment #58)
> William,
> One thing we'll need to do is to ensure that we have a final copy
> of the Citation Management System code.
> Brent suggested and I agree that we should do the following:
> Ask the programmers at CIAT to totally freeze the code, not
> change anything unless it is essential to keep the system
> running.
> Send us the latest copy of the code, as of the freeze.
> Send us the version control database for the code.
> Can that be arranged for the near future?

Actually, there are plans in place already to deliver the code by 8/26/2009. They plan to freeze any changes to the application by 8/24/2009 and deliver it to you directly either by DVD or FTP. The preference is FTP because they want some form of acknowledgment from OCCM. I could deliver another copy by DVD but they will prefer to hand over to a non-Lockheed employee at this point.

BZDATETIME::2009-08-25 16:59:41
BZCOMMENTOR::Brent Carter
BZCOMMENT::60

I'm attaching a document that summarizes some security issues we found and the security measures we plan to take before hosting the applications.

Attachment Security Analysis and Risk Mitigation Plan.docx has been added with description: Security Analysis and Risk Mitigation Plan

BZDATETIME::2009-08-25 17:55:47
BZCOMMENTOR::Alan Meyer
BZCOMMENT::61

We had a meeting today to discuss next steps for the Citation
Management System. The conclusions we reached were as follows:

o CIAT will provide a frozen copy of all existing Citation
Manager System code on August 26, as planned.

If any problems at CIAT arise that require emergency changes
to the code, CIAT will need to send the patched code to us
ASAP. No non-emergency changes should be made.

o John Rehmert will configure a server for the system.

The server will be inside the NIH network. All users will
need to login to the NIH network, either directly or via the
VPN, in order to access the system.

Although the server will be inside the NIH network, it will
be outside the firewall that protects cancer.gov. If the
server is compromised by a hacker, the hacker will have no
access to cancer.gov or other highly protected systems
through the server.

John will try to get the server ready some time next week,
August 31 - September 4.

o Brent Carter will make the small number of modifications he
recommended in his analysis of the system (see the Bugzilla
comment #60 attachment).

The plan is to complete them by the time the server is ready
and to install them on the new server as soon as it is
available.

o Testing will begin by September 8, or sooner if the system is
ready.

Testing will be done using the May 19, 2009 database backup
that was provided to us, using the new server that will be
used for production.

We will need CIAT to assign testing tasks to whoever is the
best person or persons at CIAT. OCE will assign one or more
PDQ board managers to test here at OCE.

If testers see something questionable, they should make notes
of the problem, check to see if the problem also exists in
the current production system or only in the ported system,
and report as soon as possible to William, Brent or me.

Our goal is to complete all testing in one work week or less,
by September 12.

o A final list of CIAT users must be created.

Presumably, this is the same list of users who are now
authorized to use the system. However, we will need to know:

• Are there any currently authorized users at CIAT who will
  not be using the system in the future?

• Are there any CIAT users who do not have login
  credentials for the NIH network?

o When testing is complete, we plan to take the following steps
to begin live operations:

1. Stop all user work on the system.

2. Backup the database at Lockheed Martin.

3. Copy the database to OCE.

That might be done by ftp or by carrying a DVD with the
data.

We have an ftp site that William should be authorized to
use to upload data. If he does not have the
credentials, he should contact Volker now to figure it
out.

4. Restore the database on our server.

5. Bring the system "up" with the new database.

6. Immediately assign new passwords to all users.

The existing passwords are not unique or secure. NIH
computer applications are expected to have more secure
passwords.

We have been lucky with the system in the past, but here
at NIH we have some systems for which we have seen
15,000 hacking attempts per day and some systems that
were successfully hacked, at a real cost in lost time
and work. Using unique, difficult to guess, passwords
should not be an onerous burden on users

We can discuss the best way to assign passwords at our
next CDR status meeting.

7. Mark any users as "deleted" who will no longer use the
system.

o In addition to the production database, we plan to install a
development copy of the software and data.

The development copy can be used for testing and training
without damaging live data.

The CIAT changeover from Lockheed Martin to Z-Tech is scheduled
for September 29, however we can perform the changeover from the
Lockheed Martin hosted system to the NIH hosted system any time
after testing is complete and before the LM hosting must end.

I propose that we aim to make the changeover some time during the
week of September 14. If worst comes to worst and some terrible
problem occurs, we can continue working on the LM hosted system
until it can be resolved, then re-import the modified data by
repeating steps 1-7 above.

BZDATETIME::2009-08-26 09:28:34
BZCOMMENTOR::Brent Carter
BZCOMMENT::62

(In reply to comment #61)
> Testing will be done using the May 19, 2009 database backup
> that was provided to us, using the new server that will be
> used for production.

Can someone confirm that there have been no changes to the database schema or code since the May 19th backup was provided? If there have been changes or if we can't be 100% sure no changes have been made we should probably get another copy of the database made after the code freeze went into effect.

BZDATETIME::2009-08-26 09:44:13
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::63

(In reply to comment #62)
> (In reply to comment #61)
> > Testing will be done using the May 19, 2009 database backup
> > that was provided to us, using the new server that will be
> > used for production.
> Can someone confirm that there have been no changes to the database schema or
> code since the May 19th backup was provided? If there have been changes or if
> we can't be 100% sure no changes have been made we should probably get another
> copy of the database made after the code freeze went into effect.

There has been at least one change to the code since the last backup was provided.

How do you want the code delivered? Would you want to talk to Pete directly about this?

BZDATETIME::2009-08-26 11:20:19
BZCOMMENTOR::Brent Carter
BZCOMMENT::64

(In reply to comment #63)
> There has been at least one change to the code since the last backup was
> provided.
> How do you want the code delivered? Would you want to talk to Pete directly
> about this?

I spoke to Pete by phone. He's already got a copy of the web and windows application code since the code freeze went into effect. He thinks there haven't been any database changes but just to be 100% sure we have the latest we agreed taking a new backup would be a good idea. The database is pretty large so we think putting it all on DVD makes the most sense. Pete is going to try to get it all on DVD today. William - would you be able to bring it to the weekly meeting tomorrow?

BZDATETIME::2009-08-28 12:33:05
BZCOMMENTOR::Brent Carter
BZCOMMENT::65

The final code was received yesterday on DVD and was copied to the CiteMS project folder on our shared drive.

I finished the code changes outlined in the security analysis document and merged those and all other code changes that have been made into the latest code.

We decided on one additional change: to update the "Change Password/Disable an Existing User" screen so that it requires a certain level of password strength. The new rule is that passwords must contain:

1. at least 8 characters
2. a lower case letter
3. an upper case letter
4. a number from 0 to 9
5. a special character from this list: `~!@#$%^*()-_=+[]{}\|;:?

BZDATETIME::2009-08-28 14:23:05
BZCOMMENTOR::Brent Carter
BZCOMMENT::66

I mentioned in our status meeting yesterday that there were a couple outstanding issues we needed to resolve. One was database related and Min was able to help us get that resolved yesterday. The other was related to some reports that were not returning results properly. I resolved this issue today.

At this point I don't know of any real issues that would prevent a successful deployment once John has a server ready for us. I'll continue to do some more testing to try and ensure we don't have any undiscovered issues.

BZDATETIME::2009-08-28 14:29:05
BZCOMMENTOR::Brent Carter
BZCOMMENT::67

I'm attaching a document that I've been maintaining with the steps taken to get the system up and running, lots of questions & answers, issues & resolutions, changes made, changes to consider in the future, etc.

It is primarily technical in nature and is meant to serve as a reference on what has been done but may be helpful when we consider an upgrade/rewrite of these systems in the future.

Attachment SettingUpAtCTB.docx has been added with description: Setting up CiteMS at CTB

BZDATETIME::2009-08-28 14:37:07
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::68

(In reply to comment #65)
> The final code was received yesterday on DVD and was copied to the CiteMS
> project folder on our shared drive.
> I finished the code changes outlined in the security analysis document and
> merged those and all other code changes that have been made into the latest
> code.
> We decided on one additional change: to update the "Change Password/Disable an
> Existing User" screen so that it requires a certain level of password strength.
> The new rule is that passwords must contain:
> 1. at least 8 characters
> 2. a lower case letter
> 3. an upper case letter
> 4. a number from 0 to 9
> 5. a special character from this list: `~!@#$%^*()-_=+[]{}\|;:?

Cynthia said she will be creating the passwords and will call users on the phone to give them their password. When do you want the passwords created? Is it before the receipt of the final/fresh copy of the database or just after hosting at NCI has started?

BZDATETIME::2009-08-28 14:54:25
BZCOMMENTOR::Brent Carter
BZCOMMENT::69

(In reply to comment #68)

> Cynthia said she will be creating the passwords and will call users on the
> phone to give them their password. When do you want the passwords created? Is
> it before the receipt of the final/fresh copy of the database or just after
> hosting at NCI has started?

There are different ways we could handle this. One is to give us the list of passwords before the final deployment of the database to the production environment. We could then change the passwords as part of or immediately after that process. This has the downside that the username/password list will have to be passed around a bit and will be known to more people than is required.

A second option (and the one I prefer) is for Cynthia to use the admin page in the web application to manually change the passwords just after we've brought the system up. This has the downside that the old "weak" passwords will be in the new live system for a short time but has the advantage that it is simpler and keeps the password between Cynthia and each end user.

Unless anyone has a strong feeling one way or the other (or has another idea altogether) can I suggest we go with option 2?

BZDATETIME::2009-08-28 21:29:53
BZCOMMENTOR::Alan Meyer
BZCOMMENT::70

(In reply to comment #69)

> ...
> Unless anyone has a strong feeling one way or the other (or has
> another idea altogether) can I suggest we go with option 2?

That sounds reasonable to me.

The security issues do not seem too problematic to me because:

1. The system will still be significantly better protected than
it has been for the seven or so years it has run.

2. If anyone hacks into the system and damages the database or
steals userids and passwords, they will only be compromising
a throwaway database. As soon as we switch over to the real
one, any damage will be lost, and any hacked or inserted
userids or passwords will become non-functional.

There are other possible risks, but I think they are exceedingly
small and acceptable.

Unless someone can think of some reason not to, I think we should
proceed with option 2.

BZDATETIME::2009-08-31 11:12:12
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::71

(In reply to comment #70)
> (In reply to comment #69)
> > ...
> Unless someone can think of some reason not to, I think we should
> proceed with option 2.

I have informed Cynthia about using Option 2. That is, using the admin. page to manually change the passwords just after hosting the system at NCI begins.

BZDATETIME::2009-08-31 12:43:02
BZCOMMENTOR::Brent Carter
BZCOMMENT::72

(In reply to comment #71)
> (In reply to comment #70)
> > (In reply to comment #69)
> > > ...
> > Unless someone can think of some reason not to, I think we should
> > proceed with option 2.
> I have informed Cynthia about using Option 2. That is, using the admin. page to
> manually change the passwords just after hosting the system at NCI begins.

Thanks William. Based on the response to the Excel sheet of users I sent out it looks like there are three users to delete. This can be done from the same admin page where passwords are changed by selecting the user and clicking the "Disable" button. We could do this manually in the back end but following the same logic above, I think it makes sense for Cynthia to disable these users when she does the password changes. What do you think?

BZDATETIME::2009-08-31 15:51:41
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::73

(In reply to comment #72)
> (In reply to comment #71)
> > (In reply to comment #70)
> > > (In reply to comment #69)
> > > > ...
> > > Unless someone can think of some reason not to, I think we should
> > > proceed with option 2.
> > I have informed Cynthia about using Option 2. That is, using the admin. page to
> > manually change the passwords just after hosting the system at NCI begins.
> Thanks William. Based on the response to the Excel sheet of users I sent out
> it looks like there are three users to delete. This can be done from the same
> admin page where passwords are changed by selecting the user and clicking the
> "Disable" button. We could do this manually in the back end but following the
> same logic above, I think it makes sense for Cynthia to disable these users
> when she does the password changes. What do you think?

I agree. Cynthia should disable the three accounts at the time of the password changes. I will let her know.

BZDATETIME::2009-09-04 15:46:26
BZCOMMENTOR::Brent Carter
BZCOMMENT::74

I got access to directly administer the CiteMS server today (John is going to let us maintain the applications on this server ourselves). I moved the latest code to production. I did another round of basic testing and for the most part all seems well. Alan and Bob helped me test over VPN and the following three minor issues were found, however, I tried doing the same tests on the current live system and numbers 1 and 2 are existing issues, not something introduced by the move to our network.

1. Search timeout. To reproduce: click the "Main Menu" link and go to the "Record NCI Literature Surveillance Committee Decision" section. Select an item such as "714-x" in the "Summary Topic" dropdown and click submit without specifying any other criteria. The request times out and an error page is displayed. This is typical of systems that allow the end user to potentially specify very broad search criteria. The same search works fine if more criteria are specified.

2. Strange formatting. If you do another search that doesn't timeout but that returns no results, the formatting at the top of the page is a little strange, for example, it displays the search criteria like this: "Search parameters: , , , " This just seems a little awkward but doesn't hurt anything functionally.

3. Bob and I each got a random error when hitting the "Admin Tool" page. Clearing the browser's cache fixed it for both of us and I've tested quite a bit since then and the problem has not happened again. I can't explain the cause but it seems like the type of "weird" error that sometimes happens when developing/testing. Once the cache was cleared it didn't come back so I'm not worried about this one (and can't reproduce to try and fix anyway). If users see this issue we can deal with it then but the cache clearing is an easy workaround.

So, I believe we're ready for user testing next week. The web application is available at http://citems.nci.nih.gov. The import application will need to be installed and configured for each end user machine. Should I just contact them via email and work with them to get set up? I have Cynthia's email but would need the other import application user's contact info. Please let me know how you'd like me to proceed.

BZDATETIME::2009-09-04 17:55:06
BZCOMMENTOR::Bob Kline
BZCOMMENT::75

(In reply to comment #74)

> 2. Strange formatting. If you do another search that doesn't timeout but that
> returns no results, the formatting at the top of the page is a little strange,
> for example, it displays the search criteria like this: "Search parameters:
> , , , " This just seems a little awkward but doesn't hurt anything
> functionally.

I get this display of empty parameters even when the search (with parameters) does return results. I agree that since this isn't a new bug (and doesn't prevent the users from getting their work done) we shouldn't worry about it right now.

BZDATETIME::2009-09-08 10:58:00
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::76

(In reply to comment #74)
> So, I believe we're ready for user testing next week. The web application is
> available at http://citems.nci.nih.gov. The import application will need to be
> installed and configured for each end user machine. Should I just contact them
> via email and work with them to get set up? I have Cynthia's email but would
> need the other import application user's contact info. Please let me know how
> you'd like me to proceed.

Yes. Please email Cynthia directly. Cynthia can also be reached on the phone at 678-906-5609 [I am still waiting for confirmation from Cynthia that she could be reached by phone TODAY]

You can also email Minaxi directly at minaxi.k.trivedi@lmco.com. She can also be reached on the phone at 301-947-9402 [Minaxi can be reached on the phone after NOON TODAY]

Could you please copy me on the emails?

BZDATETIME::2009-09-10 10:38:06
BZCOMMENTOR::Brent Carter
BZCOMMENT::77

(In reply to comment #76)

I spoke to Cynthia and Minaxi on Tuesday but both were unable to access our VPN so we had to wait until NIH got them up and running again. Cynthia called yesterday morning and Minaxi this morning with their VPN working and we got them both set up with the import application. Each performed some basic operations successfully while we were on the phone and will continue to test the rest of the functionality on their own.

William: have you received any feedback yet on any issues they've found with the web or import application? How are they feeling about doing the switchover next week?

BZDATETIME::2009-09-10 10:58:11
BZCOMMENTOR::Margaret Beckwith
BZCOMMENT::78

I have asked the Board Managers and CIAT onsite staff to test the Web site system by next Weds. So far I have heard back from 2 Board Managers that they haven't encountered any problems.

BZDATETIME::2009-09-10 11:38:17
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::79

(In reply to comment #77)
> (In reply to comment #76)

> William: have you received any feedback yet on any issues they've found with
> the web or import application? How are they feeling about doing the switchover
> next week?

So far they have not reported any problems but just as you said, they just started testing so I will give them a day or two to find out how they feel about switching over next week. On the other hand they are aware we have set a deadline of September 14 so I am assuming they are getting ready for the switch.

Meanwhile, here is an email I got from Cynthia this morning:
...................................................

I have actually gotten through quite a bit of testing. I have finished with the admin tool section, systems help section and client reports. And a bit of search testing while verifying reports. So far everything works! I am making a list of things that were not working prior to the testing as I go so it will be ready for them to fix after the testing is complete. I spoke to Minaxi this morning and she was able to testing importing in the import utility and she too has not encountered any problems.

As for speed, I am not really noticing a difference…yet. The kind of things I am doing so far are not big server requests so I will know more later. While doing side by side report comparisons the two were generating data at about the same speed. But at a different time of day this could vary.

BZDATETIME::2009-09-11 10:43:25
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::80

I spoke with Pete yesterday and he said the disk will not be ready on Tuesday morning so he proposed the following schedule:

• Monday COB (9/14) users will stop all data entry into the Cite MS

• Tuesday (9/15) he will make a backup of the database and burn to a disk and deliver it to me by COB.

I also spoke with Robin Harrison and she said she may be able to bring the DVD to OCCM on Wednesday morning.

The only difference is that instead of delivering the Disk on Tuesday, it is going to be delivered on Wednesday. Please let me know if this is Okay.

BZDATETIME::2009-09-11 11:35:12
BZCOMMENTOR::Margaret Beckwith
BZCOMMENT::81

I guess this means that users will not have access to the system for all of Tuesday and most of Wednesday then? Will it take him all day on Tuesday to burn the CD? Thanks.

BZDATETIME::2009-09-11 12:33:44
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::82

1. Here is an email I received from Cynthia this morning:
..............................................................................
During my testing of the CMS website from 3-9pm I noticed that the NCI hosted site was significantly faster then the LM hosted site. Especially when I was testing searches with 2+ search variables and retrieving larger set of citations. However, the import utility hosted at NCI, regardless of the time of day, was slower. Minaxi mentioned this as well. Not horribly slow but noticeably slower with search retrieval and importing.
I testing publishing this morning and everything worked, numbers matches, and citations were divided and assigned to reviewers correctly. I still have to test the client review and decision recording which I hope to finish today. So Monday the majority of the testing will be complete. Minaxi and I will discuss everything Monday, do additional testing and make sure we have not missed anything.
--------------------------------------------------------------------------------

2. Here is an email I received from Minaxi this morning:
...............................................................................
I have imported more than 2000 citations and tested eveything in import utility. It seems to be working fine except it is little slower.
I have also tested running premedline report in CMS and updated citations. That works as well. Also the NOT selected journal report works fine.

BZDATETIME::2009-09-11 15:03:03
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::83

(In reply to comment #81)
> I guess this means that users will not have access to the system for all of
> Tuesday and most of Wednesday then? Will it take him all day on Tuesday to
> burn the CD? Thanks.

Pete said he will talk to DBA to see when he can finish copying the files and will let me know. Hopefully, we can deliver it on Tuesday afternoon instead of Wednesday morning.

BZDATETIME::2009-09-14 19:55:16
BZCOMMENTOR::Brent Carter
BZCOMMENT::84

I got the final database backup from Pete tonight. I did a test restore to my local database so I know the backup is good. I copied it to a network location here and let John's team know where it is. I spoke to them earlier and someone from their team should be available to help some time in the morning. As soon as someone on John's team can restore the database I'll do a quick smoke test as a sanity check and then let you all know it is ready to go. At that point, Cynthia can update the user passwords and delete the inactive users. Then we should be ready to open it up to users for live operation.

BZDATETIME::2009-09-15 09:05:17
BZCOMMENTOR::Brent Carter
BZCOMMENT::85

John's team got the database imported this morning. I did a very basic run through of some features and all seems to be ok. The site is ready for Cynthia to work on the users. I notified her already and asked her to let us know when it is done. Once that is done it might be worth doing some non-destructive testing before opening it up to all the users (i.e. not adding/deleting stuff but just verifying some reports and things of that nature).

BZDATETIME::2009-09-15 09:26:07
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::86

(In reply to comment #85)
> Once that is done it might be worth doing some non-destructive
> testing before opening it up to all the users (i.e. not adding/deleting stuff
> but just verifying some reports and things of that nature).

I have asked Cynthia to also test a few reports before giving out the passwords to users.

BZDATETIME::2009-09-16 11:14:10
BZCOMMENTOR::Brent Carter
BZCOMMENT::87

(In reply to comment #74)
> 3. Bob and I each got a random error when hitting the "Admin Tool" page.
> Clearing the browser's cache fixed it for both of us and I've tested quite a
> bit since then and the problem has not happened again. I can't explain the
> cause but it seems like the type of "weird" error that sometimes happens when
> developing/testing. Once the cache was cleared it didn't come back so I'm not
> worried about this one (and can't reproduce to try and fix anyway). If users
> see this issue we can deal with it then but the cache clearing is an easy
> workaround.

We've had one occurence of the above issue that was identified during testing. Below are bits of relevant text from an email Robin Harrison sent to Cynthia yesterday describing the issue:

• • • • Begin Email ----

I have tested the two accounts and passwords, and I am running into some problems.

I encountered an error message with several functions. For example, if I select Admin Tool, I receive the message: “Error occurred! Please contact technical support.” I see the same message when I follow the path of a citation that I’ve looked up using the Search Database screen.

Then, I tried logging in as the Genetics Reviewer. I kept receiving the same error message I quoted above.

• • • • End Email ----

This is the exact same issue Bob and I saw. It happened to me when testing using multiple users and that is what Robin was doing as well so it sounds like that may be a contributing factor. At this point I'd still take a "wait and see" approach as this has only happened once and it has a very easy workaround. If it continues to happen or if anyone feels this needs investigation / resolution now please let me know and we can try to find a way to reproduce the issue in a test environment.

BZDATETIME::2009-09-16 11:36:42
BZCOMMENTOR::Bob Kline
BZCOMMENT::88

> I receive the message: “Error occurred! Please contact technical support.”

I poked around in the code a little bit, and it looks as if this error message may be coming from the SQL injection detection code. Is that all our own addition, or did they already have something along those lines, which you augmented? In either case, it might be useful to enhance that code so that it logs the string in which it claims it found something dangerous.

BZDATETIME::2009-09-16 12:24:02
BZCOMMENTOR::Brent Carter
BZCOMMENT::89

(In reply to comment #88)
> > I receive the message: “Error occurred! Please contact technical support.”
> I poked around in the code a little bit, and it looks as if this error message
> may be coming from the SQL injection detection code. Is that all our own
> addition, or did they already have something along those lines, which you
> augmented? In either case, it might be useful to enhance that code so that it
> logs the string in which it claims it found something dangerous.

If I remember correctly they had some detection code in an include file but it didn't seem to actually get used/included in any active files. I didn't really add any "detection" code but rather "prevention" code, and only did that by escaping single quotes on the import app login form. So most of what you see was already there.

BZDATETIME::2009-09-16 13:11:50
BZCOMMENTOR::Bob Kline
BZCOMMENT::90

(In reply to comment #89)

> If I remember correctly they had some detection code in an include file but it
> didn't seem to actually get used/included in any active files.

If I understand what I'm looking at, I think it must get included somewhere, because it looks like the only path through which the user would ever see the error message we're getting.

> I didn't really add any "detection" code but rather "prevention" code, and
> only did that by escaping single quotes on the import app login form. So
> most of what you see was already there.

The code checks the form data, the query string, and cookies to see if any of them contains any "black-listed" substrings, including some which would be pretty common (such as '--' or '#'). If any of the offending substrings are found, the user is redirected to the page Error.asp, which is the only place I was able to find the error message we're getting.

Again, I think it might be useful to at least modify that code so that it logs the data which triggered the failure, so we can get closer to figuring out how we can avoid the problem.

BZDATETIME::2009-09-16 16:36:08
BZCOMMENTOR::Brent Carter
BZCOMMENT::91

(In reply to comment #90)
> (In reply to comment #89)
> > If I remember correctly they had some detection code in an include file but it
> > didn't seem to actually get used/included in any active files.
> If I understand what I'm looking at, I think it must get included somewhere,
> because it looks like the only path through which the user would ever see the
> error message we're getting.

I took a quick look and what I was remembering is true for the code we got in May - the only place that "inputValidation.asp" logic is included is in files that have "bk" or "[some date]" indicating they are backup files not currently being used. In the August and production code that file is in fact included in several active pages.

> Again, I think it might be useful to at least modify that code so that it logs
> the data which triggered the failure, so we can get closer to figuring out how
> we can avoid the problem.

I agree - it wouldn't be too much effort and should be a relatively small change. However, since we're live now and given that the file is included in several places we'd want to be careful and of course test we don't break anything... making it a non-trivial effort. I've got some other priorities that are keeping me pretty booked at the moment so (unless someone else wants to take this one) we'd have to decide how important this is and prioritize.

BZDATETIME::2009-09-17 15:01:33
BZCOMMENTOR::Margaret Beckwith
BZCOMMENT::92

Comment from Victoria when working in the system from home:
I’ve been reviewing citations in the CiteMS today and all is going well except it keeps timing out on me. I tend to alternate between reviewing a set of citations and doing other work, so I am letting it sit idle for 20 or 30 minutes at a time. It must be timing out after about 30 minutes because I’ve signed on about 5 times today (at least I’ve memorized my password!). I can’t say for sure that this is different than the “old” system (either way, the old system timed out too quickly, too), but I do think it is a little quicker. Is this something the programmers might be able to adjust?

BZDATETIME::2009-09-17 15:49:18
BZCOMMENTOR::Brent Carter
BZCOMMENT::93

(In reply to comment #92)
> Comment from Victoria when working in the system from home:
> I’ve been reviewing citations in the CiteMS today and all is going well except
> it keeps timing out on me. I tend to alternate between reviewing a set of
> citations and doing other work, so I am letting it sit idle for 20 or 30
> minutes at a time. It must be timing out after about 30 minutes because I’ve
> signed on about 5 times today (at least I’ve memorized my password!). I can’t
> say for sure that this is different than the “old” system (either way, the old
> system timed out too quickly, too), but I do think it is a little quicker. Is
> this something the programmers might be able to adjust?

The standard "session length" is 20 minutes and that is how the system is currently configured. It can be increased but that comes at the expense of additional server resources (holding multiple sessions in memory for longer). The general guidance is to keep the session length as low as possible but high enough that people don't get frustrated. I checked in with Pete and they weren't sure if it had been increased in the old LM hosted application but it sounds like it probably was.

In this case, since we have a very limited number of users and I don't think session is being used to hold lots of data, it should be ok to increase it. This is just a server configuration change so no code changes or significant testing is really required. We'd just make the change and see how things go tomorrow.

Does anyone have any concerns/objections or shall I go ahead and make this change? Shall we try 40 minutes?

BZDATETIME::2009-09-17 15:52:24
BZCOMMENTOR::Margaret Beckwith
BZCOMMENT::94

Unless anyone has any objections, I would be in favor of extending the time.

BZDATETIME::2009-09-18 13:20:00
BZCOMMENTOR::Brent Carter
BZCOMMENT::95

(In reply to comment #94)
> Unless anyone has any objections, I would be in favor of extending the time.

There haven't been any objections so I spoke to Margaret and we decided to go ahead and make the change. If someone has a current session it might still have the old timeout value until they close and re-open their browser.

I set the session timeout to 40 minutes for now so we're not making a drastic change. If users still feel it times out too frequently and there are no adverse effects we can increase it further as needed.

I won't be in Monday morning so in the very unlikely event that there is an issue and it needs to get set back, here are the steps John's team can take to go back to the original setting: RDP into the CiteMS server, open IIS Manager, select the "CtMS" website, double click the "ASP" icon, expand "Services > Session Properties > Time-out," change the value to 00:20:00 and click the "Apply" link in the "Actions" panel on the right.

BZDATETIME::2009-09-22 15:23:54
BZCOMMENTOR::Margaret Beckwith
BZCOMMENT::96

Should I close this issue? I think we decided that we weren't going to work on the one "bug" that had come up sporadically, and we aren't going to make enhancements, so it doesn't seem like there is any reason to keep it open.

BZDATETIME::2009-09-22 15:36:50
BZCOMMENTOR::Alan Meyer
BZCOMMENT::97

I suggest we keep it open until after the CIAT staff has moved
to Z-Tech and has used the system for a week or so. If anything
comes up during that time we'll have this Bugzilla issue for
recording what happened.

Assuming everything is okay, I think we can close it and open
a new issue if something new needs to be done.

BZDATETIME::2009-09-29 21:13:54
BZCOMMENTOR::Alan Meyer
BZCOMMENT::98

William brought in three laptops to CTB today to install the
Import Utility, but we were unable to complete the installation
because it turned out that we needed administrative privileges on
the machines - which neither of us had.

So we went to Z-Tech and were able to install the software there
with the help of Jeff, a Z-Tech sys admin at Z-Tech who was
knowledgeable and helpful.

Menaxi helped with testing, checking to see if the application
appeared to be working.

William now has three laptops configured with the Import Utility.
One will go to Menaxi, one to Cynthia, and one is on his own
machine as a backup if that is ever required.

BZDATETIME::2009-09-29 21:34:17
BZCOMMENTOR::Alan Meyer
BZCOMMENT::99

I updated Brent's file named "CiteMSSetupProcedure.docx", in the
directory:

nci6116g\group\OCE\CTB\PROJECTS\Citation_Management_System

I added a summary of the procedure that William and I used to
install the Visual Basic Import Utility program on a user's
machine. If we have to do it again, hopefully we will find these
instructions.

I also updated "Readme.txt" in the same directory to include
descriptions of additional files and directories added since the
first draft.

BZDATETIME::2009-09-30 09:52:39
BZCOMMENTOR::Volker Englisch
BZCOMMENT::100

Just wondering: Shouldn't this document be included in our online documentation?

BZDATETIME::2009-12-02 11:16:30
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::101

(In reply to comment #100)
> Just wondering: Shouldn't this document be included in our online
> documentation?

Alan will send me the information to be included in the online documentation.

BZDATETIME::2009-12-11 11:08:07
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::102

(In reply to comment #101)
> (In reply to comment #100)
> > Just wondering: Shouldn't this document be included in our online
> > documentation?
>
> Alan will send me the information to be included in the online documentation.

We decided at the CDR status meeting yesterday to lower the priority of this task since the only thing left to do is the documentation.

Lowered priority to P7

BZDATETIME::2010-01-21 18:44:10
BZCOMMENTOR::Alan Meyer
BZCOMMENT::103

Here is a slightly modified version of the document
provided in the previous attachment.

The document could be improved in light of later
experience, but I'm not sure it's worth the effort
unless and until we commit more deeply to retaining
and developing the software.

Attachment CiteMSSetupProcedure.docx has been added with description: Setting up CiteMS at CTB and on user workstations.

BZDATETIME::2010-01-22 15:44:41
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::104

(In reply to comment #103)
> Created an attachment (id=1848) [details]
> Setting up CiteMS at CTB and on user workstations.
>
> Here is a slightly modified version of the document
> provided in the previous attachment.
>
> The document could be improved in light of later
> experience, but I'm not sure it's worth the effort
> unless and until we commit more deeply to retaining
> and developing the software.

I have added this to the online documentation with title CiteMs Setup Procedure and as CDR0000664283. The document only needs to be QAed.

I have marked this issue as resolved.

BZDATETIME::2010-01-22 15:45:25
BZCOMMENTOR::William Osei-Poku
BZCOMMENT::105

Issue Closed. Thanks!