CDR Tickets

BZDATETIME::2010-09-23 13:00:17
BZCOMMENTOR::Bob Kline
BZCOMMENT::1

We have two instances on Verdi:

http://verdi.nci.nih.gov/PDQBoards/ [bare-bones Drupal, customized]
http://verdi.nci.nih.gov/boards/ [with additional packages]

BZDATETIME::2010-10-28 15:15:00
BZCOMMENTOR::Robin Juthe
BZCOMMENT::2

A comment about automatic notifications:

We decided we would like to have notifications triggered by a period of time rather than an event (with the exception of messages). The user will be able to specify how frequently he/she receives a notification (e.g., daily, weekly, monthly). The notification would simply be a notice that the user had messages/documents/etc that had not been viewed on the site.

BZDATETIME::2010-11-02 10:49:23
BZCOMMENTOR::Bob Kline
BZCOMMENT::3

Attachment EBMS-Scope.xls has been added with description: Scope document for EBMS project

BZDATETIME::2010-11-05 12:18:04
BZCOMMENTOR::Bob Kline
BZCOMMENT::4

Modified to reflect latest discussions on notifications and on investigation of possible integration with federated login.

Attachment EBMS-Scope.xls has been added with description: Updated scope document

BZDATETIME::2011-01-06 09:33:51
BZCOMMENTOR::Bob Kline
BZCOMMENT::5

Fixed component assignment.

BZDATETIME::2011-01-06 10:23:32
BZCOMMENTOR::Bob Kline
BZCOMMENT::6

Added Michelle to CC list for task.

BZDATETIME::2011-01-06 15:56:22
BZCOMMENTOR::Michelle Worrest
BZCOMMENT::7

Attachment index_v.1_b&w.psd has been added with description: PDQ Wireframes

BZDATETIME::2011-01-10 11:42:47
BZCOMMENTOR::Michelle Worrest
BZCOMMENT::8

Attachment EBMS_Minutes_1.3.11.doc has been added with description: EBMS UI 1/9/11 Minutes

BZDATETIME::2011-01-12 15:24:31
BZCOMMENTOR::Robin Juthe
BZCOMMENT::9

Michelle, could you please post the latest version of the site map document? We'd like to share it with the rest of the Board managers. Thanks.

BZDATETIME::2011-01-12 16:36:52
BZCOMMENTOR::Michelle Worrest
BZCOMMENT::10

Attachment site_map_1.10.11.pdf has been added with description: PDQ Site Map

BZDATETIME::2011-04-01 17:08:47
BZCOMMENTOR::Bob Kline
BZCOMMENT::11

Here's a link to the draft EBMS data document:

http://bach.nci.nih.gov/ebms-data.html

Let's sit down and go over it when you get a chance.

BZDATETIME::2011-05-06 16:13:39
BZCOMMENTOR::Robin Juthe
BZCOMMENT::12

As mentioned yesterday, Margaret and I met with the rest of the Bd Managers on Wednesday and have some feedback on a couple follow-up questions from Bob’s data document. I will post this in a separate document.

BZDATETIME::2011-05-06 16:15:03
BZCOMMENTOR::Robin Juthe
BZCOMMENT::13

Attachment EBMS - CiteMS coversheets and Communication.doc has been added with description: Feedback from Board Managers

BZDATETIME::2011-05-10 16:41:55
BZCOMMENTOR::Bob Kline
BZCOMMENT::14

Met with Christine to work on timeline information.

BZDATETIME::2011-07-18 16:17:01
BZCOMMENTOR::Bob Kline
BZCOMMENT::15

Robin, Alan, and I met last week, and we came up with some clarifications for and additions to the requirements:

• users will not be able to set their own time zone

• the system will need to support multiple forums

• forum visibility must be controllable by group membership

• managers must be able to create forums on the fly

• count of new summaries will not be included in the home page summary

• grouping options needed for viewing packet feedback (details to follow)

• need way to highlight unseen comments for board manager [1]

• need to be able to tag uploaded documents

• tags will be from a controlled list

• Robin will come up with a preliminary list

• Robin will come up with instructions for what to do with some of the tags

• need to support a Meeting Materials document [2]

[1] This could be just flagging the comments which haven't been seen
since the last time the packet was reviewed by the Board Manager,
or perhaps an optional field to specify suppression of the feedback
posted before a specified date.

[2] For example:

MEETING NAME
ADMINISTRATIVE DOCUMENTS

• document link 1

• document link 2
  SUMMARIES

• summary 1

• article 1

• article 2

• article 3

• summary 2

• article 4

• article 5

• article 6
  .....

Robin/Alan: Does this accord with your notes?

BZDATETIME::2011-07-19 14:56:01
BZCOMMENTOR::Robin Juthe
BZCOMMENT::16

(In reply to comment #15)
> Robin/Alan: Does this accord with your notes?

It does. Margaret and I plan to discuss the following items with the other Board managers in our next meeting (currently slated for Aug 5), after which I will provide more concrete ideas about how we want these to work:

• grouping options needed for viewing packet feedback (details to follow)

• need way to highlight unseen comments for board manager [1]

• need to be able to tag uploaded documents

• tags will be from a controlled list

• Robin will come up with a preliminary list

• Robin will come up with instructions for what to do with some of the tags

• need to support a Meeting Materials document [2]

BZDATETIME::2011-08-05 15:50:46
BZCOMMENTOR::Bob Kline
BZCOMMENT::17

Robin:

Here's a summary of our discussion of packet creation earlier this afternoon.

The sequence of creating a packet will involve the following steps:

1. You pick a board (radio button or pick list). This step has to come before any of the others, and you must pick exactly one board.

2. The interface presents all of the topics associated with that board and you pick exactly one topic. This step has to come before everything else except selecting the board.

3. The interface shows you the CiteMS cycles for which at least one article is associated with the topic you have selected, and has a status indicating the article can be included in review packets, and the article has not been already included in a packet for this topic. You pick one or more of these cycles. This step is required before you can move on to the subsequent steps. All of the remaining steps can be performed in any order.

4. The interface presents all of the articles which are associated with the selected topic and cycles, and have a status indicating the article can be included in review packets, and which have not already been included in a review packet for the selected topic. You decide which articles are to be included in the packet; at least one article must be selected. It makes most sense to me to have all the displayed articles selected by default, but you and your fellow board managers can decide how that should work.

5. The interface shows all of the documents which have been posted to the system with the tag 'summary' and which have been associated with the packet's selected topic (the document posting interface will present allow the poster to pick a topic if the 'summary' tag has been selected), sorted with the most recently posted summary documents before those posted less recently, with summary documents posted longer ago than the last 60 days (or whatever length of time you think appropriate) not shown. You select the summary documents you want associated with the packet (you can pick all of the documents, none of the documents, or any subset of the displayed documents). You can let us know whether the summaries displayed should be all selected by default. Now that more than one summary can be connected with a packet (or none at all), we decided the language in the review interface would be more sufficiently vague to cover all the possible cases (for example, instead of the current label "Warrants no changes to the summary" we'll say instead "Warrants no changes to any summary."

6. The interface displays check boxes for all of the board members associated in the CiteMS system with the selected topic, all checked by default. An additional check box (unchecked by default) will be present to enable the board manager creating the packet to have the interface show check boxes for all members of the selected board, not just those associated with the selected topic (for the occasional situation in which you want to include a reviewer who isn't in the CiteMS defaults for this topic). You decide which reviewers are assigned to the packet. At least one reviewer must be selected.

7. You specify a name for the packet. We didn't talk about this step, and you may decide it might not be necessary. The original prototype identified the packets in the interface by concatenating the name of the topic, the cycle, and "[Name of first reviewer] et al." We might still do this, but because you can now pick more than one cycle, and because you explained that the CiteMS concept of "cycle" and the "cycle" the board managers use to send out the packets aren't as tightly coupled as I originally thought they were, it's probably preferable to give you the flexibility of specifying how you want the packets to be identified in the user interface yourself. Perhaps the software could seed this field with the name of the topic to get you started and you can edit it/add to it as you think appropriate.

We also didn't talk about what can be edited once a packet has been created. The original prototype allowed you to edit just about anything in the packet (except that if you dropped an article from the packet I didn't delete the row from the table associating articles with packets, I just flagged it so it wouldn't show up for the reviewer as something that needs feedback for future review actions; that way I avoided database integrity problems for articles originally assigned to the packet for which board member had already provided feedback, which I didn't want to just throw away). Now that we've made the user interface simpler from the user's point of view, but more complicated from the software's perspective, it's not as reasonable to allow as much freedom for changing your mind about what goes into the packet. I think you could drop or add reviewers (again, without losing existing feedback a review has already provided for articles in the packet), you can change the name of the packet, and you might perhaps want to add or drop articles (but have it work the same as the original prototype, where dropping an article only sets a flag to suppress the article when subsequent reviewers tackle the packet, so you can preserve existing feedback on the article). You might even be able to change the summaries associated with the packet, though that is likely to muddy the waters too much: think of a reviewer who said "let's cite this article in the summary" and then you replace the summary with a different one, so I'd strongly recommend against allowing this type of change. Once a packet has been created, you can't change the board or the topic. You might want to make it possible to suppress a packet so it wouldn't show up in the reviewers' queues any longer, but (as with suppressing articles from a packet), we'd want to implement that with a "suppress" flag for the packet, or perhaps by setting the "suppress" flag for all the articles in the packet, rather than by discarding the information, including any work already done by the reviewers for the packet up to that point.

There's one more variation on all this you might want to consider. Since we're now going to be filtering the articles available for assignment to a packet by only showing the ones which (a) are connected with the selected topic, (b) haven't already been assigned to any packet for this topic, and (c) have a CiteMS status indicating that they are eligible for assignment to packets, it might not add a lot of value to have the step of selecting CiteMS cycles (step 3 above). I imagine that in most (or perhaps all) cases, you're more interested in whether you've assigned all the articles for the packet's topic, than in knowing which cycle the article is associated with in the CiteMS tables. We can still display the CiteMS cycle next to each candidate article for the packet, but it would simplify the interface (both for the user and the software) to be able to leave out a step here, and it would greatly simply things if you decide you want to be able to edit the makeup of the packet after it has been created, and would provide you with more flexibility if you aren't constrained here by which cycle an unassigned article falls in for the CiteMS system when you're doing such editing. Something to consider.

Have I captured everything accurately?

BZDATETIME::2011-08-10 13:41:36
BZCOMMENTOR::Robin Juthe
BZCOMMENT::18

(In reply to comment #17)
> Have I captured everything accurately?

You have! Thanks so much for documenting these. I am uploading an annotated version of the steps you've described with a few comments and answers to your questions about setting defaults, eliminating a step, etc. This has the blessing of all of the Board managers.

BZDATETIME::2011-08-10 13:43:13
BZCOMMENTOR::Robin Juthe
BZCOMMENT::19

Attachment PACKET GENERATION STEPS_annoted after meeting with Bd Mgrs.doc has been added with description: Steps for Generating a Lit Surveillance Packet

BZDATETIME::2011-08-10 14:33:00
BZCOMMENTOR::Bob Kline
BZCOMMENT::20

(In reply to comment #18)

> I am uploading an annotated version of the steps you've described ....

Excellent, thanks!

BZDATETIME::2011-08-30 07:38:10
BZCOMMENTOR::Bob Kline
BZCOMMENT::21

I think http://verdi.nci.nih.gov/ebms/packet is ready for you to play with. Let me know if you run into anything that doesn't look like what we discussed.

Moving on to the interface for posting documents:

Can you give me a preliminary list of some of the tags that might be used?

As tags take over some of the function of determining the answer to the question "what will this document be used for?" or "where will it appear?" can you give some thought to how much you think the "visibility" panel implemented in the original prototype for posting documents has retained its usefulness? If you still think it's needed, which portions would we want to keep, and can you give use cases for those portions?

Thanks!

BZDATETIME::2011-09-01 11:15:47
BZCOMMENTOR::Robin Juthe
BZCOMMENT::22

(In reply to comment #21)

We are reviewing the literature surveillance packets. Looks good so far!

A preliminary list of tags was provided offline. According to my notes, these are:

• meeting

• minutes

• agenda

• support

• summary

• travel

• local

• long-distance

• orientation

• home

As you suggested on Tuesday, it may make more sense for the visibility of documents to be handled further downstream, i.e. in the creation of an agenda packet or literature surveillance packet, or be handled using the tags. I am working on use cases for all document types.

Margaret and I also met yesterday to discuss next steps and we agree that the next major focus should be on the agenda packet creation (possibly revisiting meeting events in the process). This will require significant input from the other Board managers, so we have a meeting scheduled for Sept 14th.

BZDATETIME::2011-09-02 12:34:40
BZCOMMENTOR::Bob Kline
BZCOMMENT::23

The new interface for posting documents has been plugged into http://verdi.nci.nih.gov/ebms/docs; you can now also edit the properties of a previously posted document.

BZDATETIME::2011-12-08 19:35:42
BZCOMMENTOR::Bob Kline
BZCOMMENT::24

The board manager portion of the prototype's Summaries pages is done, with the added header for "Link(s) to Cancer.gov" and the two bugs we saw this morning fixed.

http://verdi.nci.nih.gov/ebms/summaries

BZDATETIME::2011-12-09 10:00:24
BZCOMMENTOR::Robin Juthe
BZCOMMENT::25

Here's a proposed mock-up of the About PDQ page. We plan to have some text directly on the HTML page and the rest be contained in documents uploaded to this page. We would like to add a value to the list of tags – "about" – which wil allow these documents to be directed to this page.

Bob, please give me a call when you have a few minutes to discuss. (301-496-4189)

Attachment ABOUT PDQ PAGE - mockup final.doc has been added with description: About PDQ page - mockup

BZDATETIME::2011-12-09 12:27:27
BZCOMMENTOR::Bob Kline
BZCOMMENT::26

I noticed that for the Adult Treatment and Pediatric boards the board's name is the header for its block, but for the other boards the header is marked up as the same link as is used for the "Board Roster" item. Why don't we make them consistent by using the headers as headers and the link items for the links?

BZDATETIME::2011-12-09 12:30:18
BZCOMMENTOR::Robin Juthe
BZCOMMENT::27

(In reply to comment #26)
> I noticed that for the Adult Treatment and Pediatric boards the board's name is
> the header for its block, but for the other boards the header is marked up as
> the same link as is used for the "Board Roster" item. Why don't we make them
> consistent by using the headers as headers and the link items for the links?

None of the headers should be links. (I copied and pasted the data from elsewhere, and I thought I removed all of those links, but I guess not.) Sorry about that.

BZDATETIME::2011-12-09 15:50:30
BZCOMMENTOR::Bob Kline
BZCOMMENT::28

I have added the About page to the prototype:

http://verdi.nci.nih.gov/ebms/about

Here it is in columns:

http://verdi.nci.nih.gov/ebms/about/2

BZDATETIME::2011-12-09 15:58:00
BZCOMMENTOR::Robin Juthe
BZCOMMENT::29

(In reply to comment #28)
> I have added the About page to the prototype:
> http://verdi.nci.nih.gov/ebms/about
> Here it is in columns:
> http://verdi.nci.nih.gov/ebms/about/2

Looks good! I prefer the first layout, but thanks for giving both a shot.

BZDATETIME::2011-12-15 15:58:30
BZCOMMENTOR::Robin Juthe
BZCOMMENT::30

Here are our thoughts for the home page and some edits to the existing pages based on our meeting yesterday. Let me know if you'd like to discuss any of these comments. Thanks.

Attachment Notes on the Home Page and other pages_12_15_11.doc has been added with description: Home Page Links and other minor changes

BZDATETIME::2011-12-15 16:08:55
BZCOMMENTOR::Bob Kline
BZCOMMENT::31

(In reply to comment #30)

> Here are our thoughts for the home page and some edits to the existing pages
> based on our meeting yesterday. Let me know if you'd like to discuss any of
> these comments.

Thanks, this looks good. Don't see any puzzles or difficult modifications. I'll get these folded in over the next couple of days.

BZDATETIME::2011-12-19 11:30:05
BZCOMMENTOR::Bob Kline
BZCOMMENT::32

[X] Change order of bulleted items on About PDQ Page
[X] Add "Levels of evidence" link to About PDQ Page
[X] Rename "Overview of PDQ" replacing "PDQ on NCI's Web site" (About page)
[X] Add Teleconference checkbox for events
[ ] Create report for upcoming meeting dates [to be spec'd]
[X] Add link to user's next meeting on the Home page
[X] Replace boxed bullets on Home page with links to key tasks

To test these changes you can log on as Victoria to see the key tasks for internal users, and as David Ryan to see the key tasks for external users.

BZDATETIME::2011-12-20 10:11:02
BZCOMMENTOR::Robin Juthe
BZCOMMENT::33

(In reply to comment #32)

Thanks, Bob. Here are some comments:

[X] Change order of bulleted items on About PDQ Page – Looks good!
[X] Add "Levels of evidence" link to About PDQ Page – Looks good, but I didn’t see LOE docs for CAM, Supp Care, or Screening & Prevention. Do you need those documents?
[X] Rename "Overview of PDQ" replacing "PDQ on NCI's Web site" (About page) – Looks good!
[X] Add Teleconference checkbox for events – I expected to see this on the Create Event and Edit Event pages but didn’t see it on either page. Am I missing it?
[X] Add link to user's next meeting on the Home page – Looks good!
[X] Replace boxed bullets on Home page with links to key tasks – We actually didn’t mean for this to be a replacement of the box of notifications that was already there but rather an additional box containing these links. Could you please restore the box with notifications? Those are helpful. Thanks.

BZDATETIME::2011-12-21 16:49:06
BZCOMMENTOR::Bob Kline
BZCOMMENT::34

(In reply to comment #33)

> [X] Add "Levels of evidence" link to About PDQ Page – Looks good, but I didn’t
> see LOE docs for CAM, Supp Care, or Screening & Prevention. Do you need those
> documents?

Yes. The ones you gave me are the ones you see. If you give me the rest I'll plug them in.

> [X] Add Teleconference checkbox for events – I expected to see this on the
> Create Event and Edit Event pages but didn’t see it on either page. Am I
> missing it?

No, I forgot to set the permissions for the new field. Should work now.

> [X] Replace boxed bullets on Home page with links to key tasks – We actually
> didn’t mean for this to be a replacement of the box of notifications that was
> already there but rather an additional box containing these links. Could you
> please restore the box with notifications? Those are helpful. Thanks.

Hmmm. I guess I misunderstood what you meant when you wrote "The rest of the page will be a compilation of shortcut links to key tasks." I have restored code to provide the notifications.

BZDATETIME::2011-12-22 09:27:30
BZCOMMENTOR::Bob Kline
BZCOMMENT::35

Should I tell Chuck that the prototype is ready to be turned over to the person who will be doing wireframes and menu organization?

BZDATETIME::2011-12-23 11:22:16
BZCOMMENTOR::Robin Juthe
BZCOMMENT::36

(In reply to comment #35)
> Should I tell Chuck that the prototype is ready to be turned over to the person
> who will be doing wireframes and menu organization?

As we discussed, there are several small changes to be made to the prototype, but these minor changes should not have a major impact on the creation of wireframes or menu structures.

I will attach a document with several changes I've identified. Some are pretty straightforward and require no further explanation; for those that require further explanation/text, I will post more information separately.

BZDATETIME::2011-12-23 11:27:29
BZCOMMENTOR::Robin Juthe
BZCOMMENT::37

Here is a list of changes I've identified. I would like the rest of the Board Managers and Margaret to weigh in and add anything I've missed, but that probably won't be feasible until after the holidays.

Attachment EBMS - whats left for Bob.doc has been added with description: Minor Changes to Prototype

BZDATETIME::2011-12-23 11:40:53
BZCOMMENTOR::Robin Juthe
BZCOMMENT::38

Hotel Request Form:

Please add the following text to the top of the form:

If you need to stay overnight to attend a PDQ Board meeting, please complete this hotel request form. Robyn Bason at ASI will make your hotel reservation and send the confirmation information to you. Please contact Robyn at basonr@mail.nih.gov or 301-594-9051 if you have any questions or if you wish to provide hotel rewards information.

Please add the following list of preferred hotels (with checkboxes beside each item) with the heading "Preferred hotel:" above the list:

__ Bethesda North Marriott Hotel & Conference Center
__ Hilton Washington DC/Rockville Hotel & Executive Meeting Center
__ The Legacy Hotel & Meeting Centre
__ Other (Please use the notes section to provide the name of your preferred hotel.)

This list of preferred hotels should go below the Nights section and above the Notes section.

Thanks!

BZDATETIME::2011-12-23 12:04:56
BZCOMMENTOR::Robin Juthe
BZCOMMENT::39

Exclusion Criteria for Article Review Sheet:

Please add the following list of possible reasons for excluding an article from PDQ to the article review sheet below the list of dispositions, with a checkbox beside each item. The same list of reasons will be used for each Board. If possible, we would like the selection of at least one reason to be required ONLY if a user indicates that the article warrants no changes to the summary, but that might have to be a later enhancement. A user can select more than one reason. The "Other" option should have a free-text field beside it. Please place the heading "Reason(s) for exclusion from PDQ summary:" above this list with the following statement just beneath the heading: "If you selected, "Warrants no changes to the summary", please indicate which of the following reasons led to your decision to exclude the article."

• Not relevant to PDQ summary topic

• Already cited in PDQ summary

• Review/expert opinion/commentary (no new primary data)

• Provides no new information/novel findings

• Inappropriate study design

• Inadequate study population (small number of patients; underpowered study; accrual target not met )

• Randomized trial with flawed or insufficiently described randomization process

• Unvalidated outcome measure(s) used (e.g., unvalidated surrogate endpoint[s])

• Missing/incomplete outcome data; major protocol deviations

• Inadequate follow-up

• Inappropriate statistical analysis (incorrect tests; lack of intent to treat analysis)

• Inappropriate interpretation of subgroup analyses

• Preliminary findings; need confirmation

• Findings not clinically important

• Other: _______________________________________

I also just noticed that the phrase "Indicate how this article might affect the summary" is out of place - it's not clear if it relates to the disposition or comment field. Please place it just below the "Disposition:" heading.

Thank you!

BZDATETIME::2011-12-23 14:58:44
BZCOMMENTOR::Bob Kline
BZCOMMENT::40

(In reply to comment #39)

> Please add the following list of possible reasons for excluding an article
> from PDQ to the article review sheet below the list of dispositions, with
> a checkbox beside each item.

We'll need to decide how we'll store this information. Since we have the free-form "Other" option for which the reviewer can fill in any text (s)he wants, it wouldn't be appropriate to have a control table for valid values to which we join the response table row. So the two choices would be:

1. Format the choices (including the specified "Other" custom reason) as a
bulleted list at the end of the text in the comments column.

2. Create a new table, but just store the strings entered and/or selected.

What do you think, Alan?

> If possible, we would like the selection of at least one reason to be
> required ONLY if a user indicates that the article warrants no changes
> to the summary, but that might have to be a later enhancement.

I'll do that in the non-prototype version of the system, and in fact I can make the checkboxes only appear if the "Warrants no changes ..." option has been selected above). Is it necessary to implement this in the prototype as well?

> Please place the heading "Reason(s) for exclusion from PDQ summary:"
> above this list with the following statement just beneath the heading:
> "If you selected, "Warrants no changes to the summary", please indicate
> which of the following reasons led to your decision to exclude the
> article."

I'm using the standard Drupal form API here, which placed the title of a field above the field, and the instructions below it. I assume the decisions about the placement of such things falls clearly in the designer's province, so it would be premature for me to do a custom implementation of the form to override Drupal's defaults in the prototype version of the page. Do you agree?

> I also just noticed that the phrase "Indicate how this article might
> affect the summary" is out of place - it's not clear if it relates to
> the disposition or comment field. Please place it just below the
> "Disposition:" heading.

Actually, that phrase was put there as instructions for the checkboxes in the Disposition field (again, following the Drupal standard form API), but since you felt it was more appropriately associated with the comments, I have replaced the string that was formerly given as the instructions for the Comments field with this phrase.

BZDATETIME::2011-12-26 23:44:29
BZCOMMENTOR::Alan Meyer
BZCOMMENT::41

(In reply to comment #40)
> (In reply to comment #39)
>
> > Please add the following list of possible reasons for excluding an article
> > from PDQ to the article review sheet below the list of dispositions, with
> > a checkbox beside each item.
>
> We'll need to decide how we'll store this information. Since we have the
> free-form "Other" option for which the reviewer can fill in any text (s)he
> wants, it wouldn't be appropriate to have a control table for valid values to
> which we join the response table row. So the two choices would be:
>
> 1. Format the choices (including the specified "Other" custom reason) as a
> bulleted list at the end of the text in the comments column.
>
> 2. Create a new table, but just store the strings entered and/or selected.
>
> What do you think, Alan?

I'm coming late to this issue and have probably missed important discussions, but here are my initial thoughts.

My inclination would be to have the control table and have "Other" be one of the controlled values - unless we think that the list of choices will be volatile.

I'd also be inclined to save the information in two parts: a controlled value (one of the items in Robin's list) and a free text comment. The free text comment is particularly important if the board member chooses "Other", but it can still be quite useful in the other cases. If an article is not relevant, the board member might want to say why. If it's already cited, he might want to say where. If the study design is inappropriate, he might want to say how.

Comments like that can be valuable in justifying the board member's choice.

BZDATETIME::2011-12-26 23:55:32
BZCOMMENTOR::Alan Meyer
BZCOMMENT::42

(In reply to comment #39)
> Exclusion Criteria for Article Review Sheet:
>
> Please add the following list of possible reasons for excluding an article from
> PDQ to the article review sheet below the list of dispositions ...

There is another issue that Bob and discussed Thursday that I was going to write up in the new CiteMS Bugzilla issue but it's relevant here so I'll say something about it.

It is my understanding that we are trying to build concrete and defensible documentation of the decisions made by board members. From comments that Robin and Margaret have made at various times, I take it that doing this is part of what we hope to contribute to ASCO.

Using the EBMS for this seems to me an excellent idea, but if we use it, what will do about the board members who don't want to participate electronically? Is their feedback not going to be part of our system? Are we going to have to have two sets of documentation - one electronic in the EBMS and one on paper in files? If so, how would we produce reports that integrate the two?

I don't know all of the considerations here, but my inclination would be to have someone here at OCE enter all of the data into the EBMS that comes back to us on paper. There are some issues in how to do that, some of which involve Drupal userid/password management, but there are probably reasonable ways to overcome them.

I suggest that we discuss this and have a plan, one way or another, to deal with board member responses on paper or by email.

BZDATETIME::2011-12-27 00:16:23
BZCOMMENTOR::Alan Meyer
BZCOMMENT::43

(In reply to comment #42)

> There is another issue that Bob and discussed

Should say "Bob and I".

BZDATETIME::2011-12-27 08:50:39
BZCOMMENTOR::Robin Juthe
BZCOMMENT::44

(In reply to comment #41)
> > If possible, we would like the selection of at least one reason to be
> > required ONLY if a user indicates that the article warrants no changes
> > to the summary, but that might have to be a later enhancement.
> I'll do that in the non-prototype version of the system, and in fact I can make
> the checkboxes only appear if the "Warrants no changes ..." option has been
> selected above). Is it necessary to implement this in the prototype as well?

Building this into the non-prototype version will be fine. Thanks.

> > Please place the heading "Reason(s) for exclusion from PDQ summary:"
> > above this list with the following statement just beneath the heading:
> > "If you selected, "Warrants no changes to the summary", please indicate
> > which of the following reasons led to your decision to exclude the
> > article."
> I'm using the standard Drupal form API here, which placed the title of a field
> above the field, and the instructions below it. I assume the decisions about
> the placement of such things falls clearly in the designer's province, so it
> would be premature for me to do a custom implementation of the form to override
> Drupal's defaults in the prototype version of the page. Do you agree?

I didn't realize this was standard Drupal formatting. It sounds like this is something we can discuss with the designer, but I do think it's odd to have the list of options followed by the instructions pertaining to the selection of one or more of those options.

> > I also just noticed that the phrase "Indicate how this article might
> > affect the summary" is out of place - it's not clear if it relates to
> > the disposition or comment field. Please place it just below the
> > "Disposition:" heading.
> Actually, that phrase was put there as instructions for the checkboxes in the
> Disposition field (again, following the Drupal standard form API), but since
> you felt it was more appropriately associated with the comments, I have
> replaced the string that was formerly given as the instructions for the
> Comments field with this phrase.

It sounds like this text was in the right place to begin with since it was below the list of options, even though that wasn't intuitive to me. It doesn't belong with the comment box. Please reverse this change and we will discuss the placement of these instructions with the designer when we get to that point. Sorry!

BZDATETIME::2011-12-27 09:10:52
BZCOMMENTOR::Robin Juthe
BZCOMMENT::45

(In reply to comment #42)
> Using the EBMS for this seems to me an excellent idea, but if we use it, what
> will do about the board members who don't want to participate electronically?
> Is their feedback not going to be part of our system? Are we going to have to
> have two sets of documentation - one electronic in the EBMS and one on paper in
> files? If so, how would we produce reports that integrate the two?
> I don't know all of the considerations here, but my inclination would be to
> have someone here at OCE enter all of the data into the EBMS that comes back to
> us on paper. There are some issues in how to do that, some of which involve
> Drupal userid/password management, but there are probably reasonable ways to
> overcome them.
> I suggest that we discuss this and have a plan, one way or another, to deal
> with board member responses on paper or by email.

I agree. Paper returns are currently entered into the CiteMS by Bonnie, so I presume the same process will continue with Bonnie entering hard-copy changes/comments into the EBMS so that everything is stored in one place and reports produce meaningful results. We do not want to have two sets of files. We should discuss the issues related to userid/password/permission management though.

BZDATETIME::2011-12-27 16:41:31
BZCOMMENTOR::Bob Kline
BZCOMMENT::46

(In reply to comment #37)

> Minor Changes to Prototype

[x] Add LOE documents for CAM, SPC, S&P

[ ] Create option to save agenda without publishing it to event

[ ] Create forums for different groups

[x] Create interface for board managers to edit travel pages

[x] Add explanatory text for hotel request form

[ ] Drop-down menu for meetings

[ ] Can we use a calendar to enter hotel request dates?

We could, but as some of the data used in the demos illustrated,
it's possible that the correct response might need to be free-form
(for example, Check-in date: "Saturday 18 Feb or Sunday 19 Feb"
with elaboration in the comments field that they would like to
arrive a day early if we're able to book the rooms in one hotel,
but a day later if that hotel isn't available). If we keep the
paper requests we might want to review the values given on the
current forms to see how realistic this example (or other better
examples) might be of the need to allow strings other than
valid dates. Besides, it's hard to imagine how we would further
automate the process for booking the hotels if we enforced
machine-readable dates in this field. Is this discussion
something which would most appropriately be addressed with the
IA and/or web designer?

[ ] Can we add drop-down menu for number of nights?

See comment on previous request.

[x] Add preferred hotels with check boxes.

[ ] Add explanatory text for reimbursement request

I got the language for hotel request form, but not for the
reimbursement request. I assume you'll supply that later, right?

[ ] Drop-down menu for reimbursement request meeting.

[ ] Calendar for reimbursement request dates?

See comment on hotel arrival dates above.

[ ] Add guidelines for monetary amount format.

Do you have specific language in mind for the guidelines?

[ ] Explanatory text for review.

You'll supply the language for this?

[x] Make review comments optional.

Thought we had said earlier this would be required, but I could
easily be mistaken. It's optional now.

[ ] Add list of exclusion criteria to review.

See questions/discussion on this request in comments 39-40.

[x] Remove "Assigned to Ilana Cass et al." from review responses page.

[ ] Modify review responses to show "no changes" responses with reasons
for exclusion?

The question mark means "let's discuss," right?

[x] Display appropriately tagged documents on Roster page.

[ ] Explanatory text for Summaries pages.

You'll supply the language you want here, right?

BZDATETIME::2011-12-27 17:52:38
BZCOMMENTOR::Alan Meyer
BZCOMMENT::47

(In reply to comment #41)
...
> I'm coming late to this issue and have probably missed important discussions,
> but here are my initial thoughts.

Bob has filled me in on some of the past discussion of board member responses.

...
> I'd also be inclined to save the information in two parts: a controlled value
> (one of the items in Robin's list) and a free text comment.
...

I am no longer inclined that way.

I can see that giving board members a form with too many places for comments and encouraging them to justify every check mark could be imposing a burden that would put them off - something we certainly don't want to do.

BZDATETIME::2011-12-28 09:44:47
BZCOMMENTOR::Bob Kline
BZCOMMENT::48

[x] Create option to save agenda without publishing it to event

BZDATETIME::2011-12-28 09:53:44
BZCOMMENTOR::Robin Juthe
BZCOMMENT::49

(In reply to comment #46)
> (In reply to comment #37)
> > Minor Changes to Prototype
> [x] Add LOE documents for CAM, SPC, S&P

Looks good.

> [ ] Create option to save agenda without publishing it to event
> [ ] Create forums for different groups
> [x] Create interface for board managers to edit travel pages

Looks good.

> [x] Add explanatory text for hotel request form

Looks good.

> [ ] Drop-down menu for meetings
> [ ] Can we use a calendar to enter hotel request dates?
> We could, but as some of the data used in the demos illustrated,
> it's possible that the correct response might need to be free-form
> (for example, Check-in date: "Saturday 18 Feb or Sunday 19 Feb"
> with elaboration in the comments field that they would like to
> arrive a day early if we're able to book the rooms in one hotel,
> but a day later if that hotel isn't available). If we keep the
> paper requests we might want to review the values given on the
> current forms to see how realistic this example (or other better
> examples) might be of the need to allow strings other than
> valid dates. Besides, it's hard to imagine how we would further
> automate the process for booking the hotels if we enforced
> machine-readable dates in this field. Is this discussion
> something which would most appropriately be addressed with the
> IA and/or web designer?

I was actually thinking we could have a mini calendar pop up from which the Board member could select a specific date. (Similar to the "add event" page or to many of our CDR reports). I wasn't thinking that the EBMS would guess the date for the person.

> [ ] Can we add drop-down menu for number of nights?
> See comment on previous request.

It's rare for our Board members to stay more than 1 night, but perhaps revising the proposed list of "0", "1", "2", and "3" to "0", "1", "2", and "more than 2" would be a good solution. They can always specify how many nights in the free-text "notes" field. I'm just thinking of the reports we will generate from this information – it would be nice to have both the dates and number of nights in a consistent format.

> [x] Add preferred hotels with check boxes.

Looks good.

> [ ] Add explanatory text for reimbursement request
> I got the language for hotel request form, but not for the
> reimbursement request. I assume you'll supply that later, right?

Correct. I'll supply it in a comment to follow.

> [ ] Drop-down menu for reimbursement request meeting.
> [ ] Calendar for reimbursement request dates?
> See comment on hotel arrival dates above.
> [ ] Add guidelines for monetary amount format.
> Do you have specific language in mind for the guidelines?

Again, I'm thinking of the reports and how it would be best if this data were entered in a consistent way. Would it make sense to pre-populate this column with $00.00?

> [ ] Explanatory text for review.
> You'll supply the language for this?

Yes.

> [x] Make review comments optional.
> Thought we had said earlier this would be required, but I could
> easily be mistaken. It's optional now.

Thanks.

> [ ] Add list of exclusion criteria to review.
> See questions/discussion on this request in comments 39-40.
> [x] Remove "Assigned to Ilana Cass et al." from review responses page.

Looks good.

> [ ] Modify review responses to show "no changes" responses with reasons
> for exclusion?
> The question mark means "let's discuss," right?

It needs discussion along with the issues in comments 39-40, but I actually wasn't too sure how the additional options would affect this report. I didn't know if the report was programmed to display everything supplied on the article review sheet or if you had configured it to display certain contents, in which case it would need to be modified to add these new options. It looks like the latter is true, since I just entered a "no change" comment with an exclusion reason from David Ryan and only the "no change" decision displayed.

> [x] Display appropriately tagged documents on Roster page.

Looks good.

> [ ] Explanatory text for Summaries pages.
> You'll supply the language you want here, right?

Will do!

BZDATETIME::2011-12-28 09:58:14
BZCOMMENTOR::Robin Juthe
BZCOMMENT::50

> [ ] Add explanatory text for reimbursement request

Please add the following text (minus with quotation marks) to the top of the reimbursement request form:

"After you attend a meeting, please complete this Reimbursement Request form for your travel-related expenses. Please select which meeting you attended and enter each expense on a separate line. Receipts are needed for any individual expense over $75. Please upload scanned copies of your receipts or fax them to Robyn Bason at ASI at 301-402-0555. Please contact Robyn at basonr@mail.nih.gov or 301-594-9051 if you have any questions. ASI will reimburse you for your travel expenses and pay your $200.00 honorarium by check 30 days after they receive your properly completed expense statement and all pertinent receipts."

Also:

Please add “($0.55.5/mile)” after "Calculated Cost" in the Travel by Privately-Owned Vehicle section.

BZDATETIME::2011-12-28 10:23:56
BZCOMMENTOR::Bob Kline
BZCOMMENT::51

(In reply to comment #49)

> I wasn't thinking that the EBMS would guess the date for the person.

This is a different problem. It's not that I don't know the date she'll check in. She might not know (or it might be contingent on some other factor, such as which hotel you're able to book her into), and may need to enter in something other than a single date. If we force the value in the field to a valid date format she won't be able to put "Friday or Saturday (see comments)." This is a standard trade-off between our fondness for consistent data entry formats and the users' need for flexibility. As I said in the previous comment, I recommend that we look at the forms the board members have submitted in the past to see how reasonable it is to anticipate the need for this flexibility, and we should ask ourselves how much of an advantage we will gain if we force all the responses into a single format, and whether that advantage is sufficient to offset the flexibility the board members would lose.

> I just entered a "no change" comment with an exclusion reason from
> David Ryan and only the "no change" decision displayed.

Since we haven't had the discussion about how we will store the information I felt it was premature to implement storage of the new information. Since I'm not storing it yet I can't display it.

BZDATETIME::2011-12-28 11:27:22
BZCOMMENTOR::Bob Kline
BZCOMMENT::52

(In reply to comment #50)

> Please add the following text (minus with quotation marks) to the top of the
> reimbursement request form: ....

Done.

> Please add “($0.55.5/mile)” after "Calculated Cost" in the Travel by
> Privately-Owned Vehicle section.

Done.

BZDATETIME::2011-12-28 13:49:37
BZCOMMENTOR::Bob Kline
BZCOMMENT::53

[x] Create forums for different groups

BZDATETIME::2011-12-28 18:10:53
BZCOMMENTOR::Bob Kline
BZCOMMENT::54

[x] Drop-down menu for hotel request meetings
[x] Drop-down menu for reimbursement request meetings

Might not behave completely as expected with some of the meetings entered in the prototype as repeating events.

BZDATETIME::2011-12-28 18:16:42
BZCOMMENTOR::Bob Kline
BZCOMMENT::55

[x] Pre-populate monetary amount for reimbursement request with '$00.00'

BZDATETIME::2011-12-28 18:31:31
BZCOMMENTOR::Bob Kline
BZCOMMENT::56

Status summary for current change requests:

[x] Add LOE documents for CAM, SPC, S&P
[x] Create option to save agenda without publishing it to event
[x] Create forums for different groups
[x] Create interface for board managers to edit travel pages
[x] Add explanatory text for hotel request form
[x] Drop-down menu for hotel request meetings
[ ] Can we use a calendar to enter hotel request dates?
[ ] Can we add drop-down menu for number of nights?
[x] Add preferred hotels with check boxes.
[x] Add explanatory text for reimbursement request
[x] Drop-down menu for reimbursement request meeting.
[ ] Calendar for reimbursement request dates?
[x] Pre-populate reimbursement amount fields.
[ ] Explanatory text for review.
[x] Make review comments optional.
[ ] Add list of exclusion criteria to review [on form but not saved yet]
[x] Remove "Assigned to Ilana Cass et al." from review responses page.
[ ] Modify review responses to show "no changes" responses with reasons
for exclusion?
[x] Display appropriately tagged documents on Roster page.
[ ] Explanatory text for Summaries pages.

BZDATETIME::2011-12-29 16:20:42
BZCOMMENTOR::Alan Meyer
BZCOMMENT::57

(In reply to comment #45)
...
> I agree. Paper returns are currently entered into the CiteMS by Bonnie, so I
> presume the same process will continue with Bonnie entering hard-copy
> changes/comments into the EBMS so that everything is stored in one place and
> reports produce meaningful results. We do not want to have two sets of files.
> We should discuss the issues related to userid/password/permission management
> though.

We discussed the problem briefly in our EBMS meeting today. Our
discussion concluded at least the following:

1. Board member responses to articles sent via paper, fax,
email, or other non-EBMS technique must be entered into the
EBMS.

2. Other board member inputs are less important. We probably do
not, for example, need to enter hotel requests or travel
expenses into the EBMS if they are not already entered by a
board member. However some thought needs to be given to the
issue to be sure that there are no other board member inputs
that must be captured besides responses to articles. Robyn
Bason and others need to be consulted.

3. We must support a mix of EBMS and non-EBMS board member
usage.

Some board members will use the EBMS for everything, some for
nothing, and some in between - using the EBMS sometimes and
using paper, fax or email at other times, even for the same
operations.

We can't assume that all data entry for a board member will
be by that member, or all will be by someone at OCE acting
for that member.

4. Bonnie must be consulted as we try to determine the best way
for her or another OCE person to enter data on behalf of a
board member. She knows the most about the problem.

5. Insufficient thought has gone into the problem of changing
over from the old Citation Management System to the new EBMS.
It may well be impractical to run both systems at the same.

At some point well before implementation we will need a
separate meeting, and probably a series of meetings, to
discuss the issue of the cutover.

BZDATETIME::2011-12-29 16:38:05
BZCOMMENTOR::Robin Juthe
BZCOMMENT::58

Here are a few additional conclusions from today's discussion (pertaining to other EBMS issues mentioned above):

1. The drop-down menu of meetings on the hotel request form will be limited to future meetings, with the soonest meeting appearing at the top of the list.

2. The drop-down menu of meetings on the reimbursement request form will be limited to past meetings, with a cut-off of meetings held in the past year. The most recent meeting will appear at the top of the list.

3. These drop-down lists (items 1 & 2) should possibly also be limited to EXCLUDE meetings that are marked with a "WebEx" flag. This needs further discussion with Board managers before implementation.

4. A calendar pop-up widget will be added to the reimbursement request page. Robyn Bason has been consulted for more information about how Board members request dates for their meetings and what level of flexibility is needed.

5. The issue of a consistent monetary format will be discussed with the IA.

6. On the article review page, the free-text field for "Other" reasons for exclusion will be removed. Instead, a comment next to the "other" option in the exclusion list will be added directing the Board member to indicate his/her other reason in comments section. (Comments will still not be required.)

7. The review responses report will be modified to include the exclusion reasons that were selected by Board members (if he/she indicated that the article warranted no changes to the summary).

BZDATETIME::2012-01-04 14:50:58
BZCOMMENTOR::Robin Juthe
BZCOMMENT::59

(In reply to comment #58)

>1. The drop-down menu of meetings on the hotel request form will be limited to
>future meetings, with the soonest meeting appearing at the top of the list.

>2. The drop-down menu of meetings on the reimbursement request form will be
>limited to past meetings, with a cut-off of meetings held in the past year. The
>most recent meeting will appear at the top of the list.

>3. These drop-down lists (items 1 & 2) should possibly also be limited to
>EXCLUDE meetings that are marked with a "WebEx" flag. This needs further
>discussion with Board managers before implementation.

We discussed the meeting drop-down lists today with all of the Board managers and came to a couple decisions regarding exclusions:

Hotel Request:
Please exclude meetings that have been flagged as "WebEx" and/or "Teleconference".

Reimbursement Request:
Please exclude meetings that have been flagged as "subgroup".
(We agreed to handle exceptions to this - such as the Editor-In-Chief subgroup meeting - outside the system.)

Another question was raised regarding possible exclusions. Would it be possible to exclude meetings from the list if the logged-in user has already submitted a hotel request form or a reimbursement request form for that meeting in the system?

BZDATETIME::2012-01-04 16:26:19
BZCOMMENTOR::Bob Kline
BZCOMMENT::60

(In reply to comment #58)

> 2. The drop-down menu of meetings on the reimbursement request form will be
> limited to past meetings, with a cut-off of meetings held in the past year.

Calendar year? Or past twelve months?

BZDATETIME::2012-01-04 16:26:59
BZCOMMENTOR::Robin Juthe
BZCOMMENT::61

(In reply to comment #60)
> (In reply to comment #58)
> > 2. The drop-down menu of meetings on the reimbursement request form will be
> > limited to past meetings, with a cut-off of meetings held in the past year.
> Calendar year? Or past twelve months?

Past 12 months, please.

BZDATETIME::2012-01-04 16:39:21
BZCOMMENTOR::Bob Kline
BZCOMMENT::62

(In reply to comment #59)

> Another question was raised regarding possible exclusions. Would it be possible
> to exclude meetings from the list if the logged-in user has already submitted a
> hotel request form or a reimbursement request form for that meeting in the
> system?

I'm fairly confident that it can be done. It's conceivable that someone will have forgotten an expense and want to throw in a supplemental reimbursement request. Or someone tells her about finding roaches in the room at the hotel she requested, and she want to submit a revised request. So, can you confirm if anything like these scenarios occur you'll make the user handle it outside the system?

BZDATETIME::2012-01-04 16:42:52
BZCOMMENTOR::Robin Juthe
BZCOMMENT::63

(In reply to comment #62)
> (In reply to comment #59)
> > Another question was raised regarding possible exclusions. Would it be possible
> > to exclude meetings from the list if the logged-in user has already submitted a
> > hotel request form or a reimbursement request form for that meeting in the
> > system?
> I'm fairly confident that it can be done. It's conceivable that someone will
> have forgotten an expense and want to throw in a supplemental reimbursement
> request. Or someone tells her about finding roaches in the room at the hotel
> she requested, and she want to submit a revised request. So, can you confirm
> if anything like these scenarios occur you'll make the user handle it outside
> the system?

This is a possibility, but we will handle the roaches and other edge cases outside the system. :-)

BZDATETIME::2012-01-04 16:55:36
BZCOMMENTOR::Margaret Beckwith
BZCOMMENT::64

You forgot to mention mice. We did have a board member (Mary Daly) who saw a mouse in her room and refused to stay at that hotel (not surprising). But I agree that we can handle these rare cases outside the system!

BZDATETIME::2012-01-10 09:20:29
BZCOMMENTOR::Robin Juthe
BZCOMMENT::65

As I mentioned in last week's CDR meeting, we'd like to change the "Number of Nights" menu on the Hotel Request form to "Check Out" with a pop-up calendar tool. We received input from Robyn Bason that this would be preferable and would be adequate (along with the free-text comment box) for the types of requests they typically receive.

If possible, we would also like to add links to the hotels from this page (linking the name of the hotel to the hotel web site). The links can be found on the Long Distance travel page for each hotel. (BTW, the editing tool for the long-distance page, where I added these links, worked great!)

BZDATETIME::2012-01-17 09:07:57
BZCOMMENTOR::Bob Kline
BZCOMMENT::66

(In reply to comment #65)
> ... change the "Number of Nights" ...
> ... add links to the hotels ....

Both done. Latest checklist:

[x] Add LOE documents for CAM, SPC, S&P
[x] Create option to save agenda without publishing it to event
[x] Create forums for different groups
[x] Create interface for board managers to edit travel pages
[x] Add explanatory text for hotel request form
[x] Drop-down menu for hotel request meetings
[x] Can we use a calendar to enter hotel request dates?
[x] Can we add drop-down menu for number of nights?
[x] Replace number of nights dropdown with Check Out calendar.
[x] Add preferred hotels with check boxes.
[x] Add URL links to preferred hotels.
[x] Add explanatory text for reimbursement request
[x] Drop-down menu for reimbursement request meeting.
[x] Calendar for reimbursement request dates?
[x] Pre-populate reimbursement amount fields.
[ ] Explanatory text for review.
[x] Make review comments optional.
[ ] Add list of exclusion criteria to review [on form but not saved yet]
[x] Remove "Assigned to Ilana Cass et al." from review responses page.
[ ] Modify review responses to show "no changes" responses with reasons
for exclusion?
[x] Display appropriately tagged documents on Roster page.
[ ] Explanatory text for Summaries pages.

BZDATETIME::2012-01-20 17:58:48
BZCOMMENTOR::Bob Kline
BZCOMMENT::67

[x] Add LOE documents for CAM, SPC, S&P
[x] Create option to save agenda without publishing it to event
[x] Create forums for different groups
[x] Create interface for board managers to edit travel pages
[x] Add explanatory text for hotel request form
[x] Drop-down menu for hotel request meetings
[x] Can we use a calendar to enter hotel request dates?
[x] Can we add drop-down menu for number of nights?
[x] Replace number of nights dropdown with Check Out calendar.
[x] Add preferred hotels with check boxes.
[x] Add URL links to preferred hotels.
[x] Add explanatory text for reimbursement request
[x] Drop-down menu for reimbursement request meeting.
[x] Calendar for reimbursement request dates?
[x] Pre-populate reimbursement amount fields.
[o] Explanatory text for review.
[x] Make review comments optional.
[x] Add list of exclusion criteria to review
[x] Remove "Assigned to Ilana Cass et al." from review responses page.
[x] Modify review responses to show "no changes" responses with reasons
for exclusion
[x] Display appropriately tagged documents on Roster page.
[o] Explanatory text for Summaries pages.

I think we're all caught up. If I understood correctly what Robin was telling me yesterday, we're going to hold off on the two items calling for explantory text which are marked [o] for right now (right, Robin?). I have modified the tables to capture the reasons for rejection, and the page showing the board manager the feedback from her board members' review now shows those reasons (if present). It was necessary, because of the table modifications, to clear out the existing reviews, so the only data there now is the result of my own testing today. Please add some review data so the IA and visual designers will have something to look at. I also added some validation code and some dynamic control over the visibility of the rejection reason checkboxes.

Please check over the most recent change requests and confirm that I've implemented them the way you want them (as far as functionality is concerned, ignoring cosmetic issues), and that I haven't left any of them out.

BZDATETIME::2012-01-25 13:00:00
BZCOMMENTOR::Bob Kline
BZCOMMENT::68

We decided recently that we will use plain ASCII strings for searching. We have a choice for how we map the non-ASCII Unicode characters to ASCII equivalents. One method will use a German environment setting, so Göthe will be searchable as Goethe (and other letters with umlauts will be similarly expanded with the e). The other method would map Göthe to Gothe. Which would you prefer?

BZDATETIME::2012-01-25 13:05:46
BZCOMMENTOR::Volker Englisch
BZCOMMENT::69

> One method will use a German environment setting, so Göthe will be searchable
> as Goethe

Do I have a vote? You can probably guess my preference.
How would, for instance, a name like Großmann converted with the non-German environment setting? Is it Grosmann or Grozman or ...?

BZDATETIME::2012-01-25 13:10:35
BZCOMMENTOR::Alan Meyer
BZCOMMENT::70

(In reply to comment #68)

Searching Google for "Goethe" I see 54.5 million hits. Searching for "Gothe" I see "Showing results for goethe". Clicking "Search instead for gothe" I see 3.7 million hits. None of the ones on the first page are about Johann Wolfgang.

I vote for "Goethe".

BZDATETIME::2012-01-25 13:14:34
BZCOMMENTOR::Bob Kline
BZCOMMENT::71

(In reply to comment #69)

> How would, for instance, a name like Großmann converted with the non-German
> environment setting? Is it Grosmann or Grozman or ...?

Grossmann for both methods.

BZDATETIME::2012-01-25 15:02:55
BZCOMMENTOR::Robin Juthe
BZCOMMENT::72

(In reply to comment #67)

I've checked over the following enhancements. You are correct that the two
issues marked [o] are not to be addressed at this time. We're going to ask our
pilot users if they feel an explanation is needed on those pages or if some
training would be adequate. Please see my other comments below.

> [x] Add LOE documents for CAM, SPC, S&P
OK.
> [x] Create option to save agenda without publishing it to event
OK.
> [x] Create forums for different groups
OK. Just want to be sure these groups can be Boards, subgroups, or ad-hoc
groups, right? The examples I saw were for each Board.
> [x] Create interface for board managers to edit travel pages
OK.
> [x] Add explanatory text for hotel request form
OK.
> [x] Drop-down menu for hotel request meetings
This has been added but it is not working properly. (I think this is due to
recurring events, but the events I expect to see in the list are not set up as
recurring events.)
> [x] Can we use a calendar to enter hotel request dates?
OK.
> [x] Can we add drop-down menu for number of nights?
N/A - replaced by next request.
> [x] Replace number of nights dropdown with Check Out calendar.
OK.
> [x] Add preferred hotels with check boxes.
OK.
> [x] Add URL links to preferred hotels.
OK.
> [x] Add explanatory text for reimbursement request
OK.
> [x] Drop-down menu for reimbursement request meeting.
This has been added but it is not working properly. (I think this is due to
recurring events, but the events I expect to see in the list are not set up as
recurring events.)
> [x] Calendar for reimbursement request dates?
OK.
> [x] Pre-populate reimbursement amount fields.
OK.
> [o] Explanatory text for review.
See note above.
> [x] Make review comments optional.
OK.
> [x] Add list of exclusion criteria to review.
OK.
> [x] Remove "Assigned to Ilana Cass et al." from review responses page.
OK.
> [x] Modify review responses to show "no changes" responses with reasons
> for exclusion.
OK.
> [x] Display appropriately tagged documents on Roster page.
OK.
> [o] Explanatory text for Summaries pages.
See note above.

> Please add some review data
I have added a number of reviews and asked other board managers to do the same
so that there will be a variety of data across all of the Boards.

We have noticed a few other things:

• We are getting an error message when creating an ad-hoc group.

• Calendar appointments for ad-hoc groups or individuals are not showing up on
  the appropriate invitees' calendars.

• Is there a way to delete a calendar appointment?

• Please remove the number of upcoming events from the home page for all users.
  This is not useful since the parameters of what is meant by "upcoming" are not
  known. The "Next meeting" line is sufficient.

Thanks!

BZDATETIME::2012-01-25 15:20:54
BZCOMMENTOR::Bob Kline
BZCOMMENT::73

(In reply to comment #72)

> > [x] Create forums for different groups
> OK. Just want to be sure these groups can be Boards, subgroups, or ad-hoc
> groups, right? The examples I saw were for each Board.

Subgroups won't be a problem, because there's no user interface for creating new subgroups, but having forums magically come into existence when new ad-hoc groups are created (and be wiped out when the ad-hoc group is deleted) would be tricky, to put it mildly. Can we make this something we consider as a deferred enhancement?

> > [x] Drop-down menu for hotel request meetings
> This has been added but it is not working properly. (I think this is due to
> recurring events, but the events I expect to see in the list are not set up as
> recurring events.)
> > [x] Drop-down menu for reimbursement request meeting.
> This has been added but it is not working properly. (I think this is due to
> recurring events, but the events I expect to see in the list are not set up as
> recurring events.)

I'll investigate these.

> - We are getting an error message when creating an ad-hoc group.
> - Calendar appointments for ad-hoc groups or individuals are not showing up on
> the appropriate invitees' calendars.

I'll check this out, too.

> - Is there a way to delete a calendar appointment?

If you go into the Edit tab, there's a Delete button at the bottom. I believe it might be possible to set things up so you could "unpublish" the event without actually deleting it, which might be safer.

> - Please remove the number of upcoming events from the home page for all users.
> This is not useful since the parameters of what is meant by "upcoming" are not
> known. The "Next meeting" line is sufficient.

Done.

BZDATETIME::2012-01-25 15:31:32
BZCOMMENTOR::Robin Juthe
BZCOMMENT::74

(In reply to comment #73)
Can we make this something we consider as a deferred
> enhancement?
That's fine. It might be the case that an ad-hoc group would become elevated to an official "subgroup" if it needed its own forum. That could at least be a work-around.

> > - Is there a way to delete a calendar appointment?
> If you go into the Edit tab, there's a Delete button at the bottom. I believe
> it might be possible to set things up so you could "unpublish" the event
> without actually deleting it, which might be safer.

Thanks. Would "unpublishing" the event still remove it from all calendars but keep it in the archives of the system where you would have access to it if we needed to bring it back? Or is this wishful thinking on my part? 🙂

> > - Please remove the number of upcoming events from the home page for all users.
> > This is not useful since the parameters of what is meant by "upcoming" are not
> > known. The "Next meeting" line is sufficient.
> Done.
Looks good - thanks.

BZDATETIME::2012-01-25 15:38:48
BZCOMMENTOR::Bob Kline
BZCOMMENT::75

(In reply to comment #74)

> Thanks. Would "unpublishing" the event still remove it from all calendars but
> keep it in the archives of the system where you would have access to it if we
> needed to bring it back? Or is this wishful thinking on my part? 🙂

I believe so.

BZDATETIME::2012-01-25 15:43:54
BZCOMMENTOR::Bob Kline
BZCOMMENT::76

(In reply to comment #75)
> (In reply to comment #74)
>
> > Thanks. Would "unpublishing" the event still remove it from all calendars but
> > keep it in the archives of the system where you would have access to it if we
> > needed to bring it back? Or is this wishful thinking on my part? 🙂
>
> I believe so.

Hmm, ambiguous. What I meant to say was that I believe it will work as you describe (not that you were dreaming). :-)

BZDATETIME::2012-01-25 17:59:47
BZCOMMENTOR::Bob Kline
BZCOMMENT::77

I am running a script which identifies all of the titles and authors in pubmed articles with non-ascii characters. You can see how the two approaches to folding come out here:

http://verdi.nci.nih.gov/ebms/non-ascii.html

The job wasn't finished when I copied the output here, but the file's pretty big already, and if I wait until the job is done, you probably won't be able to load it into your browser. I added some custom code to handle "Ø" and "ø" as these were coming out as question marks for both methods, and there were a lot of them (and that seemed like a pair of letters where the ASCII equivalent that folks would search for was obvious). You'll see that there are other question marks (other than instances where the original actually had a real question mark). We may want to write custom code for some of them, but we'll never be able to anticipate every possible case, and it probably wouldn't be worth it for many of the characters, either because we wouldn't be able to come up with a useful guess as to what the searcher would key in, or because a character just doesn't show up very often, and there's plenty of other parts of the title to search for (these outliers are mostly, if not entirely, in the titles rather than the author names).

BZDATETIME::2012-01-25 19:24:14
BZCOMMENTOR::Alan Meyer
BZCOMMENT::78

(In reply to comment #77)
> I am running a script which identifies all of the titles and authors in pubmed
> articles with non-ASCII characters. You can see how the two approaches to
> folding come out here:

I looked up a few of these on Pubmed and then viewed the Medline
print format for the records.

Pubmed is using a plain, unaccented character characters with
umlauts. That's the "English locale" option. Although I
previously voted for "oe" rather than "o" in "Goethe", I'm going
to change my vote. Goethe is a famous writer, the spelling of
whose name in English has become highly conventionalized like,
for example, the spelling of "Tchaikovsky", "Moscow" or
"Belgrade". His is not the best name to look at to determine
what most English speakers will type when searching for a name
with an umlaut.

Pubmed does more substitution than either of the locale specific
transcriptions that you checked. For example, Greek letters are
spelled out "alpha", "beta", "gamma".

As I see it, we have a number of options:

1. Use the English locale. Don't worry about failures. Users
will find ways to locate what they want.

2. Use the Medline print format.

2.a. Download both Medline and XML formats, save the XML and
index the Medline.

2.b. Download only the Medline, index and save it.

3. Do our own translations for Greek and some other characters,
as Pubmed does.

My instinct is to go with 1, and shift to 2 or 3 if and when it seems
desirable to do so (Bugzilla priority 6?)

BZDATETIME::2012-01-26 10:04:04
BZCOMMENTOR::Robin Juthe
BZCOMMENT::79

(In reply to comment #78)
> My instinct is to go with 1, and shift to 2 or 3 if and when it seems
> desirable to do so (Bugzilla priority 6?)

I vote for using the English locale too. I, for one, wouldn't know which extra letters to add when searching for the name using the German locale. I agree that we will find other ways (such as the trusty PMID) to locate the citation we need. We do that now.

BZDATETIME::2012-01-26 11:15:14
BZCOMMENTOR::Bob Kline
BZCOMMENT::80

I analyzed all 248,588 articles in the current system and identified all of the non-ASCII characters found in the titles, authors, and abstracts. There are only 239 of them, so I went ahead and created a mapping for them, the results of which you can see at:

http://verdi.nci.nih.gov/ebms/non-ascii-characters.html

Except for <space> (which means the character is replaced by ASCII 32, the space character) and <dropped> (which means the character will be skipped when creating the searchable ASCII string), all of the values in the last column are exactly as they would be represented in the searchable ASCII string. Take a look and let me know if there are any modifications you would like me to make. In particular, you may or may not agree with my decision to represent Greek characters with the Roman equivalent single character for those which have the same (or nearly identical) glyph, instead of consistently mapping all Greek characters using the same approach (so, for example, "μ" becomes "[mu]" but the uppercase is mapped to "M" not "[MU]").

I'm pretty confident that mapping any character which we haven't seen in more than ten years' worth of data to "?" would be a reasonable way to fly. If we run into a Unicode character which had been too obscure to show up, but all of a sudden becomes really popular, we can easily add it to the mapping. I don't see any point in mapping the handful of random CJK characters which have shown up.

BZDATETIME::2012-02-02 09:11:34
BZCOMMENTOR::Robin Juthe
BZCOMMENT::81

I'm attaching some of our initial report ideas, including some changes to the hotel and reimbursement request reports that are in the prototype. These will be later enhancements (not built into the prototype). More to come.

Attachment Proposed EBMS Reports - for Bob.doc has been added with description: EBMS Report Specs

BZDATETIME::2012-02-02 22:02:20
BZCOMMENTOR::Alan Meyer
BZCOMMENT::82

Here are my notes from today's meeting to discuss some transition
implementation issues for the EBMS. We discussed the following
ideas:

1. Test in parallel with just a few Summary topics in the new
system.

The original idea of running both systems fully in parallel in
order to support both online and paper users looks to be too
expensive, because it requires double data entry.

An alternative is to:

Run the old system on ALL articles.

Convert the data one time, loading it into the new system.

Pick a few Summary topics that don't have too many articles
per month, and also run those in the new system. There
will be double data entry, but just for the articles
pertaining to those few topics.

Test for as long as we need to build confidence that the
new system is working and can be put into production.

Discard all data in the new system and re-convert / re-load
the data from the old system to the new one. All data
discarded in the new system will be picked up from the old
system where it was also entered.

This can enable us to extend testing and gain experience at
relatively low cost.

If we want to do this, we need to discuss it with Chuck.

2. Create some sort of printed user manual or online help screens
for training.

This might be done by different people who have different
expertise in the system. It is possible we would benefit from
support from Chuck's team for this.

3. Reproduce sufficient packet print functionality in the new
system to enable Bonnie or other OCE staff to support those
board members who prefer printed paper to online access.

We need to do some analysis to see what is required. It might
be enough to be able to make it possible for Bonnie to
retrieve "cover sheets" and "cover letters" via her web
browser, print them, and use them plus a canned report or two
to figure out what PDFs to print.

4. Provide a way for Bonnie or another OCE staffer to act on
behalf of a board member who has chosen not to work online to
record responses to articles.

Required capabilities probably include:

a. Select a user to act for. Enable the OCE staffer to act
for that user without knowing the user's password.

b. See the user's name on each key screen for recording
responses. This should help to avoid possible mistakes
where the OCE staffer thought she picked board member X to
work for but mistakenly picked board member Y.

c. Log information that tells us who was acting for whom in
any transaction.

d. Prevent a staff member from working on behalf of two
different board members concurrently, for example by using
two different browsers.

(One possibility is to require a pseudo-logoff for one
board member before allowing pseudo-logon for another.)

5. Provide a way for a user to request paper instead of online
access for articles and responses. This might be done via a
user settable profile option, but there are also other ways.

BZDATETIME::2012-02-06 12:07:01
BZCOMMENTOR::Bob Kline
BZCOMMENT::83

We need to decide what to do in the conversion for the articles which NLM doesn't have any more. Of the 249,217 rows in the legacy bib table, NLM says it has nothing matching the PMID in the table for 13,098 rows. Of those 13,098 missing articles, only 1,692 have any record of review feedback. The approach the conversion is taking to pulling the old information into the EBMS database is to get everything about the article from NLM. If we decide we need to bring over the articles which have disappeared from NLM's database, we obviously can't use that approach for those articles. Should we ignore the articles which no longer exist at NLM? If not, do we need all of the missing articles, or just those which got as far as being reviewed? If we include any of the missing articles, should we populate the required fields in the article table with dummy data (e.g., "missing from NLM")? Or do we need to come up with values extracted from whatever was stored in the CiteMS database (which would be significant additional work, possibly impacting the schedule, and complicating the software, which expects the XML document from NLM, which the legacy system didn't store)?

BZDATETIME::2012-02-07 12:21:07
BZCOMMENTOR::Bob Kline
BZCOMMENT::84

(In reply to comment #83)
> We need to decide what to do in the conversion for the articles which NLM
> doesn't have any more. Of the 249,217 rows in the legacy bib table, NLM says
> it has nothing matching the PMID in the table for 13,098 rows. Of those 13,098
> missing articles, only 1,692 have any record of review feedback.

The problem is less severe than I thought, now that I realize that the old system was very careless about leading and trailing spaces, even around the Pubmed IDs themselves. There are actually only 630 articles for which NLM will no longer give us a record, and only 56 of these show any evidence of having made it to the review stage. The most recent article in that group was reviewed back in 2008. So it would seem that if you decide to have the conversion ignore these missing articles the impact on the ability to report on historical review or import activity would be negligible. See attached report.

Attachment missing-articles-report.xls has been added with description: Missing articles which received review in legacy CiteMS system

BZDATETIME::2012-02-09 09:51:12
BZCOMMENTOR::Bob Kline
BZCOMMENT::85

The legacy CiteMS system stored multiple rows for the same reviewer, without any clues about whether those rows represented the same person. I have to merge the information about the same person into a single user account (the name in the Drupal users table must be unique, and we'd want to know who the reviewer really is, anyway), so I need someone to go through the attached list and:

1. identify any sets of multiple rows where the name is the same, but
the names do not represent the same person; and

2. identify any sets of multiple rows where the names are not the same,
but the rows do represent the same person (for example, I bet "Susan
Domcheck" and "Susan Domchek" are really the same person; how about
"Barry Kramer" and "Barnett Kramer"?).

Thanks!

Attachment legacy-citems-reviewers.xls has been added with description: Legacy reviewer list

BZDATETIME::2012-02-09 09:54:21
BZCOMMENTOR::Robin Juthe
BZCOMMENT::86

(In reply to comment #85)
> I need someone to go through the attached list

Sure, we can do that. What do the "null" and "1" flags mean in the second column?

BZDATETIME::2012-02-09 10:36:15
BZCOMMENTOR::Bob Kline
BZCOMMENT::87

(In reply to comment #86)

> What do the "null" and "1" flags mean in the second column?

When a reviewer is no longer an active member of a board, that flag is set to "1"; we still need to create rows for the retired board members, in order to record the article review feedback they provided. There's a date field in the legacy data (not included on the spreadsheet) I use to figure out which value for the flag is the most recent for the same person.

BZDATETIME::2012-02-15 12:51:03
BZCOMMENTOR::Bob Kline
BZCOMMENT::88

I have attached an Excel workbook showing the unique combinations of what the legacy system called "decisions" and "responses" in the database tables. I discovered yesterday that the combinations produced by using the asc_RESPONSE_TYPES table, which Alan and I had conjectured (the lack of documentation for the old system frequently leaves us with nothing but conjecture) was used by the software to control which combinations were allowed when users were recording history for articles making their way through the system, could not possibly be the set of combinations actually used by the system. For example, that table had all of the response values which a reviewer would use ("Warrants no change to the summary," "Merits minor revision to the text," etc.) coupled with "Committee Decision" instead of "Reviewer Decision." What I have done for the attached workbook is instead to pull out all of the unique combinations which actually ended up appearing in the database in connection with specific articles. There are two sheets in the workbook, one showing "decisions" regardless of association with any "responses," and a second sheet showing the counts of "responses" recorded for each "decision" (I'm using quote marks because "Decision" is being used in an odd way, with values like "Previous Full Text Request" and "Reviewer Decision"). I'd like to end up knowing what all these combinations mean, and how I should be mapping/storing them in the new system. Perhaps after Robin and Cynthia have had a chance to glance over the attachment, we can have a conference call to fill in some of the gaps.

Alan's code for populating the ebms_article_state_type table has the following states, sequenced in processing order:

'Imported into the database'
'Rejected by NOT list' (end of the line)
'Rejected in initial review' (end of the line)
'Passed initial review' (ready for the board manager to look at it)
'Rejected by Board Manager' (end of the line; decision from abstract)
'Passed Board Manager'
'Requested full text'
'Full text retrieved'
'Rejected after full text review' (end of the line)
'Passed full text review' (OK for board members to review)

Part of my task will be to decide, for the earlier (before board member review) stages of processing, which "responses" in the legacy data map to which of these states. For example, if I see a "Decision" of "Full Text Request" with a "Response" of "Yes" would that mean I should put two rows in the article state table, one for "Passed Board Manager" and a second for "Requested full text"? Or just a single row for one of those values, and if so, which value? I assume I won't be putting any rows in for the first few values above, right?

Thanks for looking this over. Getting your help in understanding the values in the legacy database will be very helpful.

Attachment citems-responses.xls has been added with description: Combinations of "decisions" and "responses"

BZDATETIME::2012-02-15 16:51:02
BZCOMMENTOR::Bob Kline
BZCOMMENT::89

After some aggressive normalizing of the title strings in the lu_NOT_LISTS table, I was able to reduce the number of titles which I could not match with NLM's journal list from more than half of the thousands of titles to somewhere between one to two hundred titles. I then manually searched NLM's journal list to try and match up the ones I could. The results are attached. Please check my assumptions to confirm that you agree that the journals identified by the NLM ID for the journal matches the journal you want on the NOT list.

Here are the ones that had matches with more than one title in NLM's list:

Antimicrobial agents and chemotherapy
Chinese medical journal
Clinical evidence
Folia biologica
Folia morphologica
Health bulletin
IARC scientific publications
IRB; a review of human subjects research
Irish journal of medical science
Journal of epidemiology and community health
Journal of periodontology
Nursing
Reproduction, nutrition, development
The American journal of Chinese medicine
The Journal of periodontology
Zhonghua yi xue za zhi

Feel free to give me NLM Journal IDs for any of these you want to keep on the list. Otherwise you can always deal with them when new articles roll in from them.

Here are the titles which didn't have multiple exact matches in NLM's list, and I couldn't confidently tell what journal was meant:

ANNALES UNIVERSITATIS MARIAE CURIE-SKLODOWSKA
AUSTRALIAN NURSING JOURNAL (JUNE 1993)
CYTOMETRY : THE JOURNAL OF THE SOCIETY FOR ANALYTICAL CYTOLOGY
HEALTH PSYCHOLOGY'
JOURNAL OF NIPPON MEDICAL SCHOOL = NIHON IKA DAIGAKU ZASSHI
NIPPON YAKURIGAKU ZASSHI. JAPANESE JOURNAL OF PHARMACOLOGY
NURSING STANDARD (ROYAL COLLEGE OF NURSING (GREAT BRITAIN)
PROCEEDINGS OF THE ROYAL SOCIETY OF LONDON. SERIES B. BIOLOGICAL SCIENCES
ROFO. FORTSCHRITTE AUF DEM GEBIETE DER RONTGENSTRAHLEN UND DER NEUEN BILDGEBENDEN VERFAHREN'
ROFO. FORTSCHRITTE AUF DEM GEBIETE DER RONTGENSTRAHLEN UND DER NUKLEARMEDIZIN

Again, you can either give me NLM Journal IDs for these, or deal with them later.

Thanks!

Attachment my-guesses has been added with description: Assumptions about NLM journal IDs

BZDATETIME::2012-02-16 07:57:38
BZCOMMENTOR::Bob Kline
BZCOMMENT::90

I should have added Minaxi and Cynthia to this issue before posting the previous two comments, because I'm going to need input from them on the questions raised there.

BZDATETIME::2012-02-16 10:33:32
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::91

I have reviewed the file and made comments in red. Let me know if you have questions.

Attachment citems-responses_cboggess_comments.xlsx has been added with description: comments on citems-responses spreadsheet

BZDATETIME::2012-02-16 10:48:53
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::92

As for the Not Journals that you are having problems matching with NLM IDs, Minaxi will go through these and get back to you.

BZDATETIME::2012-02-16 11:06:26
BZCOMMENTOR::Bob Kline
BZCOMMENT::93

(In reply to comment #91)

> I have reviewed the file and made comments in red. Let me know if you have
> questions.

Thank you! This is very helpful – exactly the kind of information I was looking for.

BZDATETIME::2012-02-17 14:56:02
BZCOMMENTOR::Minaxi Trivedi
BZCOMMENT::94

(In reply to comment #89)
> Created attachment 2227 [details]
> Assumptions about NLM journal IDs
>
> The results are attached. Please check
> my assumptions to confirm that you agree that the journals identified by the
> NLM ID for the journal matches the journal you want on the NOT list.
>
Yes, I have checked all and agree.

> Here are the ones that had matches with more than one title in NLM's list:
>
I am replacing the list with NLM Id and my notes

> 0315061: Antimicrobial agents and chemotherapy
7513795: Chinese medical journal
100883600: Clinical evidence Not currently indexed for MEDLINE
101294314: Clinical evidence (Online) since 1996
0234640: Folia biologica
0374620: Folia morphologica
0012330: Health bulletin
8009542: IARC scientific publications—currently indexed
0364347: IARC scientific publications— currently indexed for MEDLINE
7906878 : IRB; a review of human subjects research
7806864: Irish journal of medical science Indexed
7806865: Irish journal of medical science not Indexed
7909766: Journal of epidemiology and community health indexed
7806055: Journal of epidemiology and community health not indexed
8000345: Journal of periodontology indexed
7600137: Nursing
8913069 & 8005903: Reproduction, nutrition, development not indexed-- No citations in CiteMS
0354717: The American journal of Chinese medicine not indexed-- No citations in CiteMS
7901431: The American journal of Chinese medicine indexed-- No citations in CiteMS

0173665: The Journal of periodontology not indexed
7511141: Zhonghua yi xue za zhi indexed
0031651: Zhonghua yi xue za zhi not indexed
>
> Here are the titles which didn't have multiple exact matches in NLM's list, and
> I couldn't confidently tell what journal was meant:

I have identified the correct journal and replacing the list with NLM Id and my notes

101214930 : ANNALES UNIVERSITATIS MARIAE CURIE-SKLODOWSKA Not currently indexed for MEDLINE
9317903 : AUSTRALIAN NURSING JOURNAL (JUNE 1993)
9317904: AUSTRALIAN NURSING JOURNAL (JULY 1993)
101235694 & 101235690 : CYTOMETRY : THE JOURNAL OF THE SOCIETY FOR ANALYTICAL CYTOLOGY
8211523: HEALTH PSYCHOLOGY

100935589: JOURNAL OF NIPPON MEDICAL SCHOOL = NIHON IKA DAIGAKU ZASSHI
0420550 NIPPON YAKURIGAKU ZASSHI. JAPANESE JOURNAL OF PHARMACOLOGY
9012906: NURSING STANDARD (ROYAL COLLEGE OF NURSING (GREAT BRITAIN)
101245157 : PROCEEDINGS OF THE ROYAL SOCIETY OF LONDON. SERIES B. BIOLOGICAL SCIENCES Not currently indexed for MEDLINE
7507497: ROFO. FORTSCHRITTE AUF DEM GEBIETE DER RONTGENSTRAHLEN UND DER NEUEN
BILDGEBENDEN VERFAHREN'
7507497: ROFO. FORTSCHRITTE AUF DEM GEBIETE DER RONTGENSTRAHLEN UND DER NUKLEARMEDIZIN
>
>
Please let me know if you have any questions.

Thanks!

BZDATETIME::2012-02-17 15:03:45
BZCOMMENTOR::Bob Kline
BZCOMMENT::95

Thanks, Minaxi. This is very helpful.

BZDATETIME::2012-02-21 14:31:03
BZCOMMENTOR::Bob Kline
BZCOMMENT::96

Here are my questions de jour for the response conversion mapping:

1. Alan doesn't have a state in his list corresponding to "Full Text Obtained - No"; do we need one?

2. I thought what Robin was explaining to me was that the "Committee Decision" response type was used in the CiteMS tables to record the decision that the board manager made after looking at the full text for the article as to whether it was worth looking at. But even though the link from the history table for the responses to the reviewer table is configured to allow NULL values, all of the rows marked "Committee Decision" have a non-NULL link to a board member reviewer. Did I not understand what Robin was trying to tell me? Do the board members participate in this "Committee Decision" after all? Or is this a way to record not just "I want this reviewed by the board members" but "I want this reviewed by Rebecca Nagy"? In other words, the equivalent of the EBMS review packet creation, with specific reviewer assignments?

3. We don't have a state in Alan's tables for the "FYI Distribution" response type. Do we need one?

4. Robin: did we decide that all we need to do to reflect the "On Agenda, but no Decision" response type was just to add the "On agenda" state (which we decided we'll add when we met last Thursday), but avoid putting in the "Final board decision" state (and the row in the new ebms_article_board_decision table)?

5. I'm still puzzled about how we should convert the "Previous Full Text Request" rows in the decision history table. For one thing, in many (most?) cases where there is a pair of rows for the same article, the "Response" value for both rows is the same. That makes sense, of course, when the values are both "Yes"; what's going on when they're both "No"? Are we saying "We did not ask for the full text at some time in the past, and we did not do the same thing at some later time"? Furthermore, most (all?) of these pairs have values in the decision_date column which are very close to each other (often only milliseconds apart). That sort of gives "some time in the past" a pretty surreal meaning, doesn't it?

6. What state should we map "Summary Assigned at Board Meeting" to?

BZDATETIME::2012-02-22 09:13:20
BZCOMMENTOR::Bob Kline
BZCOMMENT::97

(In reply to comment #96)

> 2. I thought what Robin was explaining to me was that the "Committee
> Decision" response type was used in the CiteMS tables to record the decision
> that the board manager made after looking at the full text for the article as
> to whether it was worth looking at. But even though the link from the history
> table for the responses to the reviewer table is configured to allow NULL
> values, all of the rows marked "Committee Decision" have a non-NULL link to a
> board member reviewer. Did I not understand what Robin was trying to tell me?
> Do the board members participate in this "Committee Decision" after all? Or is
> this a way to record not just "I want this reviewed by the board members" but
> "I want this reviewed by Rebecca Nagy"? In other words, the equivalent of the
> EBMS review packet creation, with specific reviewer assignments?

That interpretation can't be right, because the links to reviewers show up for all of the "Committee Decision" rows, not just the ones which have "Yes" for the "Response." For example, there are 26 rows in the history table for PMID 19214961 ("Distress among women receiving uninformative BRCA1/2 results: 12-month outcomes") with the response "No" and each one has a board member linked to the row. They even have a value of "March 2010" for the mailing date.

So not only do I not understand what "Committee Decision" means, I don't even know what "No" means. :-)

To further complicate my confusion, when I looked at this article in the web interface for the legacy system, I see three rows for "Committee Decision" all showing "Yes"!

BZDATETIME::2012-02-22 09:18:35
BZCOMMENTOR::Bob Kline
BZCOMMENT::98

(In reply to comment #97)

> To further complicate my confusion, when I looked at this article in the web
> interface for the legacy system, I see three rows for "Committee Decision" all
> showing "Yes"!

Looking further down the page for that same article, I see a block labeled "On Agenda, But No Decision," and the comment entered in that block says "Discussed at June Psychosocial WG meeting. Added text." Isn't the decision to add text (presumably to the summary) a decision?

BZDATETIME::2012-02-22 10:54:29
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::99

Let me start with full text retrieved…
The purpose of this step in the process is to generate a list of citations that the brd managers said “yes” to in their initial review. When a “yes” full text retrieved is recorded for a citation that citation no longer appears on the list. If this value remains null then the citation will continue to remain on the list. If a “no” is recorded the citation is also removed from the list. A “no” should be accompanied with a note explaining what the “no” in that particular case means. The “no” decision should not be used very often as it is the goal to do everything we can to get a copy of the full text. A “no” decision could mean:
1. Unable to retrieve in full text…maybe it is not available online or in the library or via ILL
2. Full text previously retrieved 2nd copy not needed…but there should be a “yes” full text retrieved previously recorded.
The “no” decision is basically a way to get problematic citations off the ready for full text retrieval list. If there is no note associated with the “no” decision then I have no idea how it has actually been used over the past 10 years by the 6-7 different employees who have recorded it.
As for previous full text request…
The CiteMS was designed around a series of cascading processes/decisions to move a citation through a multi-step review, tracking each decision along the way. Unfortunately not all citations come into review at step 1. The system did not easily accommodate such citations so we worked with what we had. For example when a board member handed a board manager a citation and said that it needed to be discussed in the next review meeting, the citation was added to the system and all necessary decisions to jump that citation to the appropriate step were recorded. And because citations could be reviewed by more than one board or by the same board at different times, multiple decisions could be recorded for the same citations. Each citation is a different set of circumstances. If you want me to interpret specific details of decisions regarding these types of citations just sent me the cmsids.
As for FYI…
This was designed to be used as a check tag or flag; way to send out a copy of a specific citation to all board members or selected board members outside of the review process regardless of whether it was a citation selected for further review or not. So all we need to record here is that the citation was flagged for FYI distribution, when it was flagged and who it was flagged for (which board or which board members). If the citation is worthy of more than just FYI then it will be reviewed as usual.
As for committee decision…
When the CiteMS was designed in 2002, the literature surveillance and review process included a literature committee for each board. These committees would meet to review all the full text copies of all the selected citations. The committee would make a decision to send the citation to the board for further review and either confirm all of the assigned summary topics or select only the ones they wanted to keep.
Literature committees have not been used in many years. So this decision has become a place where the board manager can confirm all of the assigned summary topics or select only the ones they wanted to keep. Or after looking at the full text they may decided to kick the citation out of the review process by recording “no”.
By confirming or selecting summary topics at this point the board manager is essentially finalizing which editorial board members will review this citation as each is assigned specific summary topics. Those topics that get selected, get a “yes” those that are not selected get a “no”. Only the “yes” topic will be displayed on the coversheets. A committee decision is summary topic specific. A citation can have 6 assigned topics upon import and then maybe only 3 will be selected at this decision point.
I hope this helps answer your questions. If not let me know and I’ll try again.

BZDATETIME::2012-02-22 11:45:10
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::100

(In reply to comment #98)
> (In reply to comment #97)
> > To further complicate my confusion, when I looked at this article in the web
> > interface for the legacy system, I see three rows for "Committee Decision" all
> > showing "Yes"!
> Looking further down the page for that same article, I see a block labeled "On
> Agenda, But No Decision," and the comment entered in that block says "Discussed
> at June Psychosocial WG meeting. Added text." Isn't the decision to add text
> (presumably to the summary) a decision?

The "On Agenda, But No Decision" is really just a place to put a citation on hold for the next meeting. So it is for citations that were planned to be discussed and for whatever reason the discussion was not finished or postponed for the next meeting. A note made here should indicate why the citation is on hold, if any discussion happened, or if there is some decision they are waiting on. And the citation will remain on hold until a decision to cite the citation or revise text based on this citation is made. If such a decision is made then the next decision should be recorded...ed brd decision. The note you reference above implies a decision was made and if so should not be recorded with the "On Agenda, No Decision". But maybe there is more going on that the note is not stating.

BZDATETIME::2012-02-23 08:28:08
BZCOMMENTOR::Bob Kline
BZCOMMENT::101

I thought I had posted the comment below a couple of days ago, but I just noticed that I still hadn't clicked "Save Changes." It refers to the article I gave as an example in comments #97 and #98. My plan at this point is to try and line up Cynthia's explanations in subsequent comments with what I see in the history table for this article. The ultimate goal is to come up with conversion logic which preserves the information about what happened with each article, without necessarily using the same convoluted approach used in the original database tables for representing that information. If I get stuck, I'll come back and test your patience further, Cynthia, but let's see how far I get on my own. :-)

I'm going to drill down with this one example to see if I can unravel the significance of what's in the legacy tables. There are 63 rows in the decision history table for this article. For all rows the PMID is 19214961, the CiteMS ID is 200168, the review cycle is February 2010, the SOE is "not selected," and the meeting date is NULL. For all but one of the rows the LOE value is "not selected" (the exception is noted below). I'll walk through the rows in the order of the decision_date column. Each row has a distinct value for that column, though in many cases the values in a clump vary by milliseconds. I'll not show granularity of less than a minute. All the dates are from last year.

============================================================================
9:58 a.m., Monday, March 8 (rows 1-13)
Category: "Committee Decision"
Response: "Yes"
Board: Supportive Care
CIPS Reviewer: Robin Baldwin
Entered By: Nicholas McGraw
Mailed: March 2010
3 rows for topic Anxiety Disorder
(reviewers Nicholas, Kristeller, Kamath)
5 for Depression
(Nicholas, Kristeller, Passik, Glajchen, Kamath)
5 for Adjustment to Cancer: Anxiety and Distress
(Nicholas, Kristeller, Glajchen, Nail, Cripe)
============================================================================
9:17 a.m., Friday, March 19 (rows 14-30)
Category: "Committee Decision"
Response: "Yes"
Board: Cancer Genetics
CIPS Reviewer: "Genetics Reviewer"
Entered By: Bonnie Ferguson
Mailed: March 2010
5 rows Cancer Genetics Risk Assessment and Counseling
(Calzone, Peterson, Hay, Vadaparampil, Weinar)
7 rows Genetics of Breast and Ovarian Cancer
(Daly, Struewing, Robson, Lu, Domchek, Pal, Cass)
5 rows Psychosocial Aspects of Cancer Genetics*
(Calzone, Peterson, Hay, Vadaparampil, Weinar)
============================================================================
2:45 p.m. that same day (rows 31-37)
Category: "Committee Decision"
Response: "No"
Board: Cancer Genetics
CIPS Reviewer: "Genetics Reviewer"
Entered By: Cynthia Boggess
Mailed: March 2010
7 rows Genetics of Breast and Ovarian Cancer
(Daly, Struewing, Robson, Lu, Domchek, Pal, Cass)
============================================================================
1 minute later that same day (rows 38-44)
Category: "Committee Decision"
Response: "No"
Board: Cancer Genetics
CIPS Reviewer: "Genetics Reviewer"
Entered By: Cynthia Boggess
Mailed: March 2010
7 rows Genetics of Breast and Ovarian Cancer
(Daly, Struewing, Robson, Lu, Domchek, Pal, Cass)
============================================================================
8:14 a.m the following Monday, March 22 (rows 45-56)
Category: "Committee Decision"
Response: "No"
Board: Cancer Genetics
CIPS Reviewer: "Genetics Reviewer"
Entered By: Nicholas McGraw
Mailed: March 2010
5 rows: Cancer Genetics Risk Assessment and Counseling
(Calzone, Peterson, Hay, Vadaparampil, Weinar)
7 rows: Genetics of Breast and Ovarian Cancer
(Daly, Struewing, Robson, Lu, Domchek, Pal, Cass)
============================================================================
3:14 p.m. that Friday, March 26 (rows 57-59)
Category: "Reviewer Decision"
Response: "Should be discussed at board meeting"
Board: Cancer Genetics
CIPS Reviewer: "Genetics Reviewer"
Entered By: Bonnie Ferguson
Reviewer: Kathleen Calzone (all three rows)
Decision Note: "merits discussion by Working Group"
2 rows: Cancer Genetics Risk Assessment and Counseling
1 row: Psychosocial Aspects of Cancer Genetics*
[note that two of these three rows are identical, ignoring the 14 millisecond difference in the "decision date" and the key for the row]
============================================================================
3:23 p.m. the following Tuesday, March 30 (row 60)
Category: "Reviewer Decision"
Response: "Warrants no change to the summary"
Board: Supportive Care
CIPS Reviewer: Robin Baldwin
Entered By: Nicholas McGraw
Reviewer: Larry Cripe
1 row: Adjustment to Cancer: Anxiety and Distress
LOE: Nonrandomized controlled trial, cohort study (prospective),
or case-control study (retrospective)
============================================================================
3:31 p.m. Wednesday October 13 (rows 61-63)
Category: "Reviewer Decision"
Response: "Warrants no change to the summary"
Board: Supportive Care
CIPS Reviewer: Robin Baldwin
Entered By: Bonnie Ferguson
Reviewer: Don Nicholas (all three rows)
1 row: Adjustment to Cancer: Anxiety and Distress
1 row: Anxiety Disorder
1 row: Depression
============================================================================

Again, thanks for your patient explanations, Cynthia. I'll get it worked out eventually.

BZDATETIME::2012-02-23 10:35:16
BZCOMMENTOR::Bob Kline
BZCOMMENT::102

There are five more rows in the history table for PMID 19214961 that I didn't include in the previous comment (I wasn't aware that there would be rows which weren't associated with a topic). Here they are:

============================================================================
2:39 p.m. Wednesday March 3
Category: "Full Text Request"
Response: "Yes"
Board: Cancer Genetics
CIPS Reviewer: Genetics Reviewer
Entered By: Genetics Reviewer
============================================================================
10:08 a.m. the next day (Thursday March 4)
Category: "Full Text Obtained"
Response: "Yes"
Board: Cancer Genetics
CIPS Reviewer: Genetics Reviewer
Entered By: Bonnie Ferguson
============================================================================
8:38 a.m. the next day (Friday March 5)
Category: "Full Text Request"
Response: "Yes"
Board: Supportive Care
CIPS Reviewer: Robin Baldwin
Entered By: Robin Baldwin
============================================================================
1:17 p.m. the same day
Category: "Full Text Obtained"
Response: "Yes"
Board: Supportive Care
CIPS Reviewer: Robin Baldwin
Entered By: Nicholas McGraw
============================================================================
4:03 p.m. Wednesday, July 28
Category: "On Agenda, but no Decision"
Response: None
Board: Cancer Genetics
CIPS Reviewer: Genetics Reviewer
Entered By: Genetics Reviewer
Decision Note: Discussed at June Psychosocial WG meeting. Added text.
Meeting Date: June 2010
============================================================================

BZDATETIME::2012-02-23 12:15:29
BZCOMMENTOR::Bob Kline
BZCOMMENT::103

Well, the mists are starting to clear and things are beginning to fall into place. Still seeing some things I can't make sense of, though. One of those things has to do with the decision="Committee Decision" response="No" rows. Sometimes I see what I would expect for such an outcome for an article: a single row in the history table, no topics or reviewers specified. For example, article PMID 11884044, entered by Alicair for Valerie. But then I see all those rows for decision="Committee Decision" response="No" that I listed in comment 101 below. If the "Full Text Request: Yes" event is board-specific, but not topic-specific, what would be the purpose of listing specific topics for which you have decided you don't want board members to look at the article? Clearly Alicair and Valerie didn't think it was useful when they entered the decision for 11884044. But for PMID 19214961 not only are the topics specified for the "Committee Decision: No" rows, but each reviewer who is not going to be asked to look at the article gets his or her own row in the history table, with both topic and reviewer specified. I don't get it.

BZDATETIME::2012-02-23 14:18:05
BZCOMMENTOR::Bob Kline
BZCOMMENT::104

(In reply to comment #103)
> But for PMID 19214961 not only are the topics specified for the "Committee
> Decision: No" rows, but each reviewer who is not going to be asked to
> look at the article gets his or her own row in the history table, with
> both topic and reviewer specified. I don't get it.

It's even stranger than that. There are three rows saying "we're not going to have Mary Daly review this article for the topic "Genetics of Breast and Ovarian Cancer," three more rows saying "we're not going to ask Jeffrey Struewing to review this article for the topic "Genetics of Breast and Ovarian Cancer," three more rows ... [for seven different reviewers, or a total of 21 rows, to say essentially the same thing].

I see that there was an earlier "Reviewer Decision - Yes" set of rows for these these same reviewers (for the same topic). Perhaps the subsequent "Committee Decision: No" rows were an attempt to cancel out the earlier Yes (though three identical sets of "No" rows seems a little bit of overkill)? If that's how the system works, why is the CiteMS web interface reporting a bunch of "Responses Not Returned" for the reviewer/topic combinations which weren't sent out? And why is Kathleen Calzone reported as having provided feedback for the article not just for the "Psychosocial Aspects of Cancer Genetics" topic (which wasn't associated with any of the "Reviewer Decision - No" rows), but also for the "Cancer Genetics Risk Assessment and Counseling" topic, the most recent previous row for which was a "No" committee decision (in fact, she is listed as providing the same feedback for this topic twice)?

Bonus question: when we're recording a decision not to assign an article to a reviewer to review for a topic, what is the significance of the "date mailed" value? Why would we mail them something we don't want them to review?

BZDATETIME::2012-02-23 14:22:20
BZCOMMENTOR::Bob Kline
BZCOMMENT::105

(In reply to comment #104)

> It's even stranger than that. ....

Oops! I see that I wrote "Reviewer Decision - ..." in a couple of places in the previous comment, where I meant to write "Committee Decision - ...." Sorry for adding to the already high level of confusion. :-)

BZDATETIME::2012-02-23 15:17:35
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::106

Let's stop worrying about why and focus on what we need to change or we are both going to go crazy. The current CMS is an effective system but not necessarily the most efficient given it's dated technology and "creative" development.
I agree that the history table could use some streamlining. If it seems like overkill then let's eliminate those rows. Once a citation gets a "no" at any point in the review process, that's the end of the line...there is no need to record all the Ed Brd members that will not get to review that citation. Responses not yet returned we need to keep. Brd Managers need to know who is slacking and not returning their responses. The date mailed is obviously only relevant to what actually gets mailed.
The full text retrieved is brd specific. The comm decision is summary topic specific...as the summary topics are linked to sets of ed brd reviewers. Reviewer responses are also summary topic specific. For each summary topic that received a "yes" committee decision there will be a row for each assigned reviewer to record their response or that they have not yet returned their response. Ed Brd Reviewers review multiple summary topics and will have multiple rows in the history table.

BZDATETIME::2012-02-23 15:56:33
BZCOMMENTOR::Bob Kline
BZCOMMENT::107

(In reply to comment #106)

> Let's stop worrying about why and focus on what we need to change or we are
> both going to go crazy.

I agree, and I'm concentrating at this point on writing up the conversion logic. I gave up a while ago wondering why they implemented the old system the way they did, other than to understand what I need to do with the data.

The biggest puzzle I'm facing right now is how to handle cases in which (as in the example I've given in the previous comments) I find both a "Yes" and a "No" answer to the same question for the same article/topic combination. If I follow the "once an article gets a 'No' it's dead" line of reasoning I would have to throw away the feedback from the reviewers who actually responded in such cases. The contradictory answers also complicate the question of whether to report on reviewers who haven't responded.

BZDATETIME::2012-02-23 17:17:08
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::108

Think of it in terms of citation review paths. A citation can have many review paths per board per topic and at various times. They end with a "no". If a "no" is followed by a "yes" even for the same brd and topic, but some time has passed...we now have a new path under a new set of circumstances. The citation is being re-review or re-considered for some reason.
If the "no" is followed by a "yes" for the same topic and only seconds apart it is probably a mistake. The CiteMS does not allow users to deleted decisions or change them. The only option is to record a second correct decision and make a note of the mistake.
If the "no" is followed by a "yes" for the same topic after an hour or a day or so then maybe somebody changed their minds. If they didn't leave a note I can't explain it.
The data is only as good as the person entering it. If too much is left up to the user then the data will inevitably be compromised. With this in mind what sort of logic can we bring to the new system to eliminate this type of confusion in the future? The system needs to remain flexible to accommodate all sorts of reviewing scenarios, but perhaps we need to require more information be recorded to account for changes in decisions.

BZDATETIME::2012-02-24 13:51:56
BZCOMMENTOR::Bob Kline
BZCOMMENT::109

(In reply to comment #108)

> ... what sort of logic can we bring to the new system to eliminate
> this type of confusion in the future?

I'll have to give that question some further thought (and perhaps discuss it with Alan when he returns). I'm still unclear about what to do with a "Yes" which is followed by a "No" (you've given clear explanations for the cases going in the other direction), especially when I've got evidence which appears to contradict the "No" (such as board member reviews coming in, or reports of board members who failed to return feedback for the articles they were sent).

BZDATETIME::2012-02-24 13:52:43
BZCOMMENTOR::Bob Kline
BZCOMMENT::110

Here is my propsed approach for populating the rows in the ebms_article_state
table, organized by each of the values in the ebms_article_state_type table.

=========================================================================
Imported
=========================================================================

We will put one row in the state table for each row in the ebms_article
table, using values derived from the legacy bib table. Rows in the bib
table which don't have a corresponding row in the ebms_article table
(because NLM no longer considers the article to exist, at least not with
the Pubmed ID that we have). This is true for all states.

The status_dt column will be copied from the bib.Date_input column,
which is defined as NULLable, but in fact no rows in the bib table
contain a NULL for the column.

The user_id column's value will be determined based on the mapping from
the bib.user_id column. The bib.user_id column is also defined as
allowing NULL values, but no rows in the bib table have a NULL for that
column. However, there is one row (Ref_ID 7584, PMID 12020673) with
a value for the user_id column (31) for which no corresponding row
exists in the mt_USERS table. Three possible solutions:

1. Link to user 0 (the row for no user)
2. Make up a user named "Unknown Legacy User"
3. Create a user for the conversion

The comments column will be populated from the bib.notes legacy column.

I recommend that we alter the topic_id column to allow for NULLs, and
set the column to NULL for "Imported" state rows. This will avoid the
problem that for some rows in the bib table there are no corresponding
rows in the asc_ORIGINAL_SUMMARIES table, and other rows in the bib
table have more than one row in the asc_ORIGINAL_SUMMARIES table. We
have come to the conclusion that it is not possible to determine the
meaning intended when rows in the asc_ORIGINAL_SUMMARIES table duplicate
information in the asc_CITE_SUMMARIES table, so we are going to merge
the topic-article associations from both of those tables for the
ebms_article_topic table. Since that approach will capture all of
the topic-article associations, it does not seem necessary to come up
with a solution to the problems introduced by prohibiting NULLs in
the ebms_article_state.topic_id column.

=========================================================================
Rejected by NOT list
=========================================================================

I don't think we can populate these rows for the conversion, as the
only record I can find in the legacy data of articles rejected because
they were published in journals on the "NOT" list is in free-text
comments in the bib.notes column.

=========================================================================
Rejected in initial review
=========================================================================

I don't see this recorded in the legacy system, so presumably we'll only
be using this state for future actions.

=========================================================================
Passed initial review
=========================================================================

I don't see this recorded in the legacy system, so presumably we'll only
be using this state for future actions.

=========================================================================
Rejected by Board Manager
=========================================================================

One row for each article where the "decision" is "Full Text Request"
and the "response" is "No." Again, this is another example of a
state which needs to store NULL for the state_id column, as none
of the instances of this state are topic specific.

The status_dt column will be populated from the mt_DECISION_HISTORY's
decision_date column.

The comments column will be populated from the mt_DECISION_HISTORY's
decision_note column.

The user_id can, I think, be populated by following the mt_REVIEW's
cips_reviewer_id's mapping. In most (all?) cases for this condition,
it's the same as the mt_DECISION_HISTORY value. In other words,
the CIPS reviewer is entering her own decision into the system.

=========================================================================
Passed Board Manager
=========================================================================

One row for each article where the "decision" is "Full Text Request"
and the "response" is "Yes." Column mapping logic same as for "Rejected
by Board Manager" state (including the need to accept NULLs in the
topic_id column - none of these are topic specific).

=========================================================================
Requested full text
=========================================================================

I'm not sure what the distinction is (if any) for future actions, but
as far as I can tell the only way the legacy database has recorded
an action that maps to the "Passed Board Manager" state above is the
request for retireval of the full text for the article, so that condition
will also be handled by insertion of a second row. Column mapping
(except for the state, of course), same as for "Passed Board Manager."

=========================================================================
Full text retrieved
=========================================================================

One row for each article with a history row where the "decision" is
"Full Text Obtained" and the response is "Yes." Column mapping logic
same as for the previous mappings from the history table, except that
for the user_id I'll use the user_name column from the history table,
rather than the CIPS reviewer.

I suspect we might need a state meaning "Full text retrieval failed"
to capture the cases where they've given up trying to get the full
text, and don't want the article to keep showing up on the lists of
articles for which they're still supposed to get the full text.
Alan didn't include such a state, but I'll talk to him about adding
it when he's back in town.

=========================================================================
Rejected after full text review
=========================================================================

One row for each unique article-topic combination (as well as any articles
for which no topic is specified) where the "decision" value is "Committee
Decision" and the "response" is "No." Same column mapping logic as for
the "Requested full text" state. Again, we'll need nulls in the topic
column for most of these, though there are a few thousand rows where a
topic is specified. Unless I get a better explanation for why reviewers
are identified for a decision which essentially means "I don't want any
of the board members to review this article," I'm going to discard the
information about the reviewers who aren't being asked to review the
article, and collapse multiple history rows for each unique article-topic
combination into a single row in the EBMS article state table. Also,
if I see any articles which have some "Committee Decision - No" rows
identifying specific topics, and other "Committee Decision - No" rows
for that same article with no topic, and no "Committee Decision - Yes"
rows for that article, I'm inclined to ignore the topic-specific rows
for that article and just store a NULL for the topic ID, which would
mean "I don't want any board members to look at this article for any
topic."

=========================================================================
Passed full text review
=========================================================================

One row for each unique article-topic combination where the "decision"
is "Committee Decision" and the "response" is "Yes." Two of the rows
have no topic or reviewer specified. Both rows are very old, and I
plan to ignore both of them. Column mapping logic same as for the
"Rejected after full text review" state. In addition to creating the
row in the state table, I will create rows in the ebms_packet,
ebms_packet_article, and ebms_packet_reviewer tables. The created_by
column in the ebms_packet table will point to the CIPS reviewer from
the history table. The packet_title column will be set to the string
"Packet converted from legacy CiteMS database history." I haven't yet
decided what to do with the optional last_seen column, but I may leave
them all as NULLs, or set them all to the date of conversion. I will
create packets by collecting all of the rows with the same topic and
review cycle into a single packet.

=========================================================================
Rejected for agenda
=========================================================================

New state requested by Robin H.; used for actions taken in the new system.

=========================================================================
Queued for later assignment to agenda
=========================================================================

New state requested by Robin H.; used for actions taken in the new system.

=========================================================================
Approved for agenda
=========================================================================

New state requested by Robin H.; used for actions taken in the new system.

=========================================================================
On agenda
=========================================================================

New state requested by Robin H.; used for actions taken in the new system.

=========================================================================
Final board decision taken
=========================================================================

New state requested by Robin H.; used for actions taken in the new system.

BZDATETIME::2012-02-24 15:34:07
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::111

I'm still unclear about what to do with a "Yes"
> which is followed by a "No" (you've given clear explanations for the cases
> going in the other direction), especially when I've got evidence which appears
> to contradict the "No" (such as board member reviews coming in, or reports of
> board members who failed to return feedback for the articles they were sent).

Sorry I thought I had addressed this, but it looks like I did not.
A no after a yes (for the same topic) is most likely a mistake especially if it is followed by additional decisions. If there isn't a note indicating it is a mistake, then we are going to have to assume so unless Bonnie can explain otherwise. Note: in the current system a no at the same decision level does not override the first yes. A yes however can kick a citation back into review at any point.

BZDATETIME::2012-02-24 16:42:25
BZCOMMENTOR::Bob Kline
BZCOMMENT::112

(In reply to comment #111)

> A no after a yes (for the same topic) is most likely a mistake

Thanks, I'll treat it that way. Have a good weekend, and thanks again for all your patient explanations! :-)

BZDATETIME::2012-02-27 12:21:26
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::113

(In reply to comment #110)
> =========================================================================
> Rejected in initial review
> =========================================================================
> I don't see this recorded in the legacy system, so presumably we'll only
> be using this state for future actions.

> =========================================================================
> Passed initial review
> =========================================================================
> I don't see this recorded in the legacy system, so presumably we'll only
> be using this state for future actions.

During the initial review in the CiteMS Import Utility, summary topics for each citation are marked as rejected or left unmarked if they pass the initial review. Citations with unmarked topics will e published for the board manager's review. I am not sure how summary topics are programmatically flagged as rejected but this data has to be stored somewhere in the system and needs to be populated in the conversion if possible.

BZDATETIME::2012-02-27 13:11:30
BZCOMMENTOR::Bob Kline
BZCOMMENT::114

(In reply to comment #113)

> During the initial review in the CiteMS Import Utility, summary topics for each
> citation are marked as rejected or left unmarked if they pass the initial
> review. Citations with unmarked topics will e published for the board manager's
> review. I am not sure how summary topics are programmatically flagged as
> rejected but this data has to be stored somewhere in the system and needs to be
> populated in the conversion if possible.

This is helpful, thanks. I see that both the asc_ORIGINAL_SUMMARIES and the asc_CITE_SUMMARIES tables have a summary_reject flag. I'll see if I can figure out how to interpret these as I work on the merging of the information from the two tables we discussed earlier.

BZDATETIME::2012-02-27 13:17:03
BZCOMMENTOR::Bob Kline
BZCOMMENT::115

(In reply to comment #113)

> ... summary topics for each citation are ... left unmarked if they pass
> the initial review.

It seems that we might either have to make sure the conversion is done when there are no imported article records which are pending the initial decision, or we'll have to do some manual cleanup to mark articles as "rejected" which we converted as "accepted" because the rejection flag wasn't turned on.

BZDATETIME::2012-02-27 13:27:01
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::116

(In reply to comment #115)

> It seems that we might either have to make sure the conversion is done when
> there are no imported article records which are pending the initial decision,

Yes, I agree. Keep in mind that once we import citations it takes about 3 weeks to finish the initial review. So 3 weeks you would have to wait for us to finish so that no citations are pending the initial decision. If you can give us a heads up as to when you are planning the conversion we can wait to import the next batch of citations.

BZDATETIME::2012-02-27 14:43:37
BZCOMMENTOR::Bob Kline
BZCOMMENT::117

(In reply to comment #114)

> I see that both the asc_ORIGINAL_SUMMARIES and the asc_CITE_SUMMARIES
> tables have a summary_reject flag. I'll see if I can figure out how
> to interpret these as I work on the merging of the information from the
> two tables we discussed earlier.

It may be that I can only use the information from the asc_ORIGINAL_SUMMARIES table. I can pick up the user_id from the bib table, on the assumption that the same person doing the import is responsible for setting the summary_reject flag in the asc_ORIGINAL_SUMMARIES table. The tables don't have any record of who set the summary_reject flag in the asc_CITE_SUMMARIES table, so unless you can tell me it's the same person who did the original import who makes the decision about rejection, I'll stick with the first table. Seems appropriate, anyway, given that the names of the two states I'm populating are talking about the "initial" review, right?

BZDATETIME::2012-02-27 14:58:01
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::118

(In reply to comment #117)
> It may be that I can only use the information from the asc_ORIGINAL_SUMMARIES
> table. I can pick up the user_id from the bib table, on the assumption that
> the same person doing the import is responsible for setting the summary_reject
> flag in the asc_ORIGINAL_SUMMARIES table. The tables don't have any record of
> who set the summary_reject flag in the asc_CITE_SUMMARIES table, so unless you
> can tell me it's the same person who did the original import who makes the
> decision about rejection, I'll stick with the first table.

Minaxi usually does all the importing and I usually do all the initial reviewing and setting of the summary_reject flags.
But sometimes I import a few files and she rejects a handful of citations that slipped through the not filter in her double check.
So you cannot make this assumption...sorry...this probably complicates things.

BZDATETIME::2012-02-27 15:36:15
BZCOMMENTOR::Bob Kline
BZCOMMENT::119

(In reply to comment #118)

> So you cannot make this assumption...sorry...this probably complicates things.

Yes. This means if we're going to record the "Rejected in initial review" and "Passed initial review" we're going to have to do it anonymously.

BZDATETIME::2012-02-27 15:54:04
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::120

(In reply to comment #119)
>we're going to have to do it anonymously.

Minaxi and I are the only 2 people who import and initially review so its not all that anonymous. And you could set all the reject and passed initial review to my user name due to the fact that I do the vast majority (98% or so I would guess) of initial reviewing.

BZDATETIME::2012-02-27 16:00:02
BZCOMMENTOR::Bob Kline
BZCOMMENT::121

(In reply to comment #120)

> Minaxi and I are the only 2 people who import and initially review so its not
> all that anonymous. And you could set all the reject and passed initial review
> to my user name due to the fact that I do the vast majority (98% or so I would
> guess) of initial reviewing.

For future imports/initial reviews, of course, the system will record who the actual user was making the decision. I'm just talking about the conversion of the last decade's worth of decisions. If what you say is true all the way back to the beginning of the old system, I have no problem with saying you made all of the initial decisions, as long as Alan doesn't object.

BZDATETIME::2012-02-28 07:53:12
BZCOMMENTOR::Bob Kline
BZCOMMENT::122

(In reply to comment #110)

> Again, this ["Rejected by Board Manager"] is another example of a
> state which needs to store NULL for the state_id column, as none
> of the instances of this state are topic specific.

Sorry, that's a typo for "... needs to store NULL for the topic_id column ...."

BZDATETIME::2012-03-07 14:08:38
BZCOMMENTOR::Alan Meyer
BZCOMMENT::123

(In reply to comment #84)
> Created attachment 2221 [details]
> Missing articles which received review in legacy CiteMS system
...

I'm jumping in very late on this but found some information that
satisfied some of my curiosity about why articles are removed
from Pubmed.

I looked at a couple of these that had responses = "Yes" in Bob's
listing to get an idea about why these might have been removed
from Pubmed. There may be a way to get Pubmed to tell us, but I
don't know what it is so I just tried to find the articles in
various sources. Here's what I found:

PMID: 18774487

This article appeared in the journal "Nature Genetics", was
indexed in Pubmed with PMID=18264098, and was picked up by
CiteMS.

Later, the abstract of the article was republished in the
journal "Urologic Oncology". See:

http://www.sciencedirect.com/science/article/pii/S1078143908001695

The Pubmed indexers appear to have mistakenly treated this as
an original publication even though Urologic Oncology cited
the source and only reprinted the title, authors, and
abstract. I presume that they republished that data because
they were publishing a comment on the article and wanted
their readers to be able to see the abstract without having
to go to Pubmed to do it.

We picked up the second Pubmed citation before they corrected
their error.

PMID: 12681274

This one turns out not to be a scientific article. It's a
list of websites that publish information on quack cancer
cures, with a brief annotation about each website. See:

http://www.thelancet.com/journals/lanonc/article/PIIS1470-2045%2803%2901045-3/fulltext

The subject matter of the article is really about web pages,
not cancer. I presume that Pubmed decided that the article
was really not in scope as a scientific publication and that
their original indexing of it was a mistake.

In both cases someone added the note "No longer available in
Pubmed" to the CiteMS records for the articles. I see there are
607 articles in CiteMS that have this as part of their notes,
which is not far off from the 630 that Bob found by looking in
Pubmed. That might (or might not) be of value in our conversion
since it tells us that the CiteMS maintainers at least knew at
some point that the articles had been withdrawn from Pubmed.

I don't know why either of these got to the stage of soliciting
board member responses in the CiteMS. Perhaps we thought the
comment was interesting in the first case, and the article was of
interest in the second case even though it wasn't scientific
research.

BZDATETIME::2012-03-07 14:34:53
BZCOMMENTOR::Minaxi Trivedi
BZCOMMENT::124

(In reply to comment #123)
> (In reply to comment #84)
> > Created attachment 2221 [details]
> > Missing articles which received review in legacy CiteMS system
> ...
>
> I'm jumping in very late on this but found some information that
> satisfied some of my curiosity about why articles are removed
> from Pubmed.
>
>
> In both cases someone added the note "No longer available in
> Pubmed" to the CiteMS records for the articles. I see there are
> 607 articles in CiteMS that have this as part of their notes,
> which is not far off from the 630 that Bob found by looking in
> Pubmed. That might (or might not) be of value in our conversion
> since it tells us that the CiteMS maintainers at least knew at
> some point that the articles had been withdrawn from Pubmed.
>
>I run the report to update Pre-Medline citations periodically and at that time come across "Wrong UID" which means the citation is no longer available in PubMed. I used to report the list of such wring UID's to the programmer and he used to add the not "No longer available in PubMed" I still run the report, but there are not as many "wrong UIDs" as there used to be in the past and I have not reported them to you.

PubMed's policy to remove "out of scope" citations policy is beyond my understanding as sometimes I have seen citations removed just because it is from a supplement of the journal.

BZDATETIME::2012-03-09 14:50:34
BZCOMMENTOR::Alan Meyer
BZCOMMENT::125

(In reply to comment #109)
> (In reply to comment #108)
>
> > ... what sort of logic can we bring to the new system to eliminate
> > this type of confusion in the future?
>
> I'll have to give that question some further thought (and perhaps discuss it
> with Alan when he returns).

I now think that the sequence numbering plan for the article state table
handles this. See OCECDR-3247 comment #96, point 7.

The idea is that Yes and No responses would have equal sequence priority.
If a Yes comes in after a No, we mark the No as inactive, and vice versa.
At any time, the record will contain all previous decisions, but only one
of them will be currently active. That one will be easily discernible
and, in fact, we should probably hide the inactive states by default and
only show them when requested, or show them in a different color, or
whatever makes the situation perfectly clear.

BZDATETIME::2012-03-09 15:11:39
BZCOMMENTOR::Bob Kline
BZCOMMENT::126

(In reply to comment #125)

> The idea is that Yes and No responses would have equal sequence priority.
> If a Yes comes in after a No, we mark the No as inactive, and vice versa.
> At any time, the record will contain all previous decisions, but only one
> of them will be currently active.

Can I assume you're talking about future states, and not the converted legacy data? If I've understood Cynthia's responses the basic thrust of her advice for the "committee decision" history (that's the context here; we've got another solution for the conflicting entries of the same reviews), is that a "Yes" trumps a "No" regardless of the order of the rows in the history table. That's the approach the conversion is taking to the "committee decision" rows.

BZDATETIME::2012-03-11 20:15:08
BZCOMMENTOR::Alan Meyer
BZCOMMENT::127

(In reply to comment #126)
...
> Can I assume you're talking about future states, and not the converted legacy
> data? If I've understood Cynthia's responses the basic thrust of her advice
> for the "committee decision" history (that's the context here; we've got
> another solution for the conflicting entries of the same reviews), is that a
> "Yes" trumps a "No" regardless of the order of the rows in the history table.
> That's the approach the conversion is taking to the "committee decision" rows.

For the legacy data, I think we should adopt whatever solution most accurately represents the actual history. If "Yes trumps No" is more accurate, then I agree that we should adopt it. We can either add in the No and mark it inactive, or not put in the No at all if we think those are errors - again following whatever principle seems most accurate.

My sense of these decisions is that they are only really critical for articles that are under active consideration. For things that happened a long time ago on articles that are not under active consideration, I wouldn't think that the mistakes that we make will do any harm in ongoing work - though they could result in somewhat inaccurate historical reports.

BZDATETIME::2012-03-12 17:28:48
BZCOMMENTOR::Alan Meyer
BZCOMMENT::128

(In reply to comment #121)
> (In reply to comment #120)
>
> > Minaxi and I are the only 2 people who import and initially
> > review so its not all that anonymous. And you could set all
> > the reject and passed initial review to my user name due to
> > the fact that I do the vast majority (98% or so I would
> > guess) of initial reviewing.
>
> For future imports/initial reviews, of course, the system will
> record who the actual user was making the decision. I'm just
> talking about the conversion of the last decade's worth of
> decisions. If what you say is true all the way back to the
> beginning of the old system, I have no problem with saying you
> made all of the initial decisions, as long as Alan doesn't
> object.

It seems to me reasonable to handle it either way. We could
assign Minaxi as the user for all imports and Cynthia for all
initial reviews, or we could create one or more anonymous user
IDs.

The former method will be right more often than the latter, but
the latter will be wrong less often 🙂

In any case, I can't think of anything that hinges on which
solution we adopt. I think we will look through old data for
some purposes, for example to figure out which journals are
generating many passes and which are generating many rejects, or
how many articles were processed over time. But the user ID is
not the important information in those queries.

BZDATETIME::2012-03-14 10:26:47
BZCOMMENTOR::Bob Kline
BZCOMMENT::129

Alan:

You've got two required columns in the ebms_not_list table that I Don't have data for in the legacy tables: start_date and user_id. Should we make those columns NULLable? If not, what should I put there for the conversion?

BZDATETIME::2012-03-14 14:20:12
BZCOMMENTOR::Bob Kline
BZCOMMENT::130

Minaxi:

Two more journals showed up in the NOT list (presumably added since I last ran the conversion for this list) which the conversion program was unable to map unambiguously. The first is "The American journal of Chinese medicine" which shows up twice in NLM's list of journals:

--------------------------------------------------------
JrId: 409
JournalTitle: The American journal of Chinese medicine
MedAbbr: Am J Chin Med (Gard City N Y)
ISSN: 0090-2942
ESSN:
IsoAbbr: Am J Chin Med (Gard City N Y)
NlmId: 0354717
--------------------------------------------------------
JrId: 410
JournalTitle: The American journal of Chinese medicine
MedAbbr: Am J Chin Med
ISSN: 0192-415X
ESSN:
IsoAbbr: Am. J. Chin. Med.
NlmId: 7901431
--------------------------------------------------------

Can you tell me which of these should be picked up for the NOT list?

The other was "Contemporary nurse : a journal for the Australian nursing profession"; I believe I have successfully tracked this down as

--------------------------------------------------------
JrId: 2147
JournalTitle: Contemporary nurse
MedAbbr: Contemp Nurse
ISSN: 1037-6178
ESSN:
IsoAbbr: Contemp Nurse
NlmId: 9211867
--------------------------------------------------------

... which is what I'll use if you can confirm that this is the right journal.

BZDATETIME::2012-03-14 14:30:43
BZCOMMENTOR::Bob Kline
BZCOMMENT::131

(In reply to comment #130)

> Two more journals showed up in the NOT list ....

Sorry, that's wrong. You had already provided instructions for The American journal of Chinese medicine. So all I need is confirmation that I've identified the correct journal for "Contemporary nurse : a journal for the Australian nursing profession.

BZDATETIME::2012-03-15 11:24:27
BZCOMMENTOR::Bob Kline
BZCOMMENT::132

I have finished the programming for converting the NOT list data, and the QA interface for the conversion has been modified to show the NOT lists for each board.

http://verdi.nci.nih.gov/u/ebms-conversion-qa.py

The EBMS tables just store NLM's ID for the journal, so for this report I'm fetching the titles from NLM on the fly so you'll be able to recognize which journals are represented.

BZDATETIME::2012-03-15 14:22:43
BZCOMMENTOR::Minaxi Trivedi
BZCOMMENT::133

(In reply to comment #132)
> I have finished the programming for converting the NOT list data, and the QA
> interface for the conversion has been modified to show the NOT lists for each
> board.
>

Though we have CAM NOT journal filter in the system, at present it is not being used. When I run the searches, I use NOT filters in PubMed for all other boards except CAM. After importing, I run the query to doublecheck that all NOT journals are eliminated during PubMed search. If they are not, then I reject them.
In the new system, I believe that I will not use the NOT filter while running the PubMed searches, but the citations will be programatically moved to citations from NOT journals. This is to make sure that this does not happen to CAM board citations.

BZDATETIME::2012-03-15 19:05:24
BZCOMMENTOR::Minaxi Trivedi
BZCOMMENT::134

(In reply to comment #131)
> (In reply to comment #130)
>
> > Two more journals showed up in the NOT list ....
>
> Sorry, that's wrong. You had already provided instructions for The American
> journal of Chinese medicine. So all I need is confirmation that I've
> identified the correct journal for "Contemporary nurse : a journal for the
> Australian nursing profession.

Bob,

I can find the journal title "Contemporary nurse" in the journals database which you have tracked down and probably the correct one. I checked CiteMS and we have records from 2001 and all are pointing to "Contemporary nurse".
Just wondering from where did we get the journal title with the subtitle "Contemporary nurse : a journal for the Australian nursing profession"?

BZDATETIME::2012-03-16 10:07:20
BZCOMMENTOR::Bob Kline
BZCOMMENT::135

(In reply to comment #134)
> (In reply to comment #131)
> > (In reply to comment #130)
> >
> > > Two more journals showed up in the NOT list ....
> >
> > Sorry, that's wrong. You had already provided instructions for The American
> > journal of Chinese medicine. So all I need is confirmation that I've
> > identified the correct journal for "Contemporary nurse : a journal for the
> > Australian nursing profession.
>
> Bob,
>
> I can find the journal title "Contemporary nurse" in the journals database
> which you have tracked down and probably the correct one. I checked CiteMS and
> we have records from 2001 and all are pointing to "Contemporary nurse".
> Just wondering from where did we get the journal title with the subtitle
> "Contemporary nurse : a journal for the Australian nursing profession"?

I found quite a few citations for the journal which include that subtitle:

http://trove.nla.gov.au/work/16855206
http://search.informit.com.au/browseJournalTitle;res=IELHEA;issn=1037-6178

The second of these confirms that we're talking about the same journal identified by Medline journal ID 9211867, since NLM lists the ISSN as 1037-6178 also.

BZDATETIME::2012-03-16 10:28:20
BZCOMMENTOR::Bob Kline
BZCOMMENT::136

(In reply to comment #133)

> When I run the searches, I use NOT filters in PubMed for all other boards
> except CAM.

Does this mean that the CAM list has not been maintained?

> In the new system, I believe that I will not use the NOT filter while running
> the PubMed searches, but the citations will be programatically moved to
> citations from NOT journals. This is to make sure that this does not happen to
> CAM board citations.

I'm not clear on what the two appearances of "this" refer to in that last sentence. :-)

If you're saying you don't want any journal articles for CAM searches marked as in journals on the blacklist, then the solution would be to clear out the list of blacklisted journals for the CAM board.

If you decide to keep the blacklist for the CAM board, it would be used to mark journal articles as "Rejected by NOT list," but you would always be able to review those marked articles and override the marking for specific ones.

Can you clarify what you meant?

BZDATETIME::2012-03-16 11:57:15
BZCOMMENTOR::Minaxi Trivedi
BZCOMMENT::137

(In reply to comment #136)
> (In reply to comment #133)
>
> > When I run the searches, I use NOT filters in PubMed for all other boards
> > except CAM.
>
> Does this mean that the CAM list has not been maintained?
YES
>
>
> I'm not clear on what the two appearances of "this" refer to in that last
> sentence. :-)
>
> If you're saying you don't want any journal articles for CAM searches marked as
> in journals on the blacklist, then the solution would be to clear out the list
> of blacklisted journals for the CAM board.
>
> If you decide to keep the blacklist for the CAM board, it would be used to mark
> journal articles as "Rejected by NOT list," but you would always be able to
> review those marked articles and override the marking for specific ones.
>
> Can you clarify what you meant?

We don't want any journal articles for CAM searches marked as in journals on the blacklist. In future, if CAM board decides to use it, I am sure we will be able to add it to the NOT list.

BZDATETIME::2012-03-16 12:10:18
BZCOMMENTOR::Bob Kline
BZCOMMENT::138

(In reply to comment #137)

> We don't want any journal articles for CAM searches marked as in journals on
> the blacklist.

OK, then I will wipe out all of the rows for the CAM board. I'll do this in the conversion, not in the legacy database. That way, we'll be able to go back to the original list if they decide they want you to revive it, bring it up to date, and start using it.

BZDATETIME::2012-03-22 09:23:03
BZCOMMENTOR::Bob Kline
BZCOMMENT::139

I have implemented an HTML/CSS version of Ashleigh's layout, from which I will build the page body template in Drupal for the common design elements as soon as I get sign-off from the users on the layout. The sample page is here:

http://verdi.nci.nih.gov/ebms-template/

BZDATETIME::2012-03-22 15:18:40
BZCOMMENTOR::Bob Kline
BZCOMMENT::140

(In reply to comment #139)
> I have implemented an HTML/CSS version of Ashleigh's layout, from which I will
> build the page body template in Drupal for the common design elements as soon
> as I get sign-off from the users on the layout. The sample page is here:
>
> http://verdi.nci.nih.gov/ebms-template/

The mockup has been modified to replace the banner with an approved image.

BZDATETIME::2012-03-27 18:52:16
BZCOMMENTOR::Alan Meyer
BZCOMMENT::141

I've put the modified ebms.sql with Robin's new state changes in version control. I tested the script on a virtual machine and it appears to work correctly.

Here's the latest version:

--------------------------------------------------------

--------------------------------------------------------

--------------------------------------------------------
19 rows in set (0.00 sec)

BZDATETIME::2012-03-28 09:32:32
BZCOMMENTOR::Bob Kline
BZCOMMENT::142

Here are the counts for the conditions I logged during the extraction of the data from the legacy tables.

54162 No history for article
12974 Articles missing from table for original topic assignment
1736 Article assigned to same topic in different cycles
391 'Yes' and 'No' full text request for same board/article combo
40 NULL response for full text request; no attempt to retrieve
864 'Yes' and 'No' for full text retrieved
77 'No' vs. 'Yes' committee decision on same topic
117 Blanket 'No' vs. topic-specific 'Yes' committee decision
8 Multiple committee decisions with different notes
8 Review received for topic which hadn't been assigned
2 Review received for article which hadn't been assigned for any topic
2 Reviewer set disposition to 'No'
160 'Final' board decision entered more than once

Just to re-cap: the instructions I've gleaned from the email and Bugzilla exchanges resulted in decisions to take the last entry when there were conflicts between "Yes" and "No" for "Full Text Requested" and "Full Text Retrieved" but a "Yes" trumps a "No" for "Committee Decision" regardless of the order of entry.

BZDATETIME::2012-03-28 14:33:28
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::143

(In reply to comment #142)

> 54162 No history for article
Are these citations that were not published or rather they were rejected in my initial review?

> 12974 Articles missing from table for original topic assignment
I am not sure what this means??

> 1736 Article assigned to same topic in different cycles
Any chance you could provide Minaxi and I a few examples of these?

> 391 'Yes' and 'No' full text request for same board/article combo
Just go with the "yes"

> 40 NULL response for full text request; no attempt to retrieve
Set to "No" if null is going to be an issue

> 864 'Yes' and 'No' for full text retrieved
Just go with the "yes"

> 77 'No' vs. 'Yes' committee decision on same topic
Keep both if recorded at different times (more than a few minutes), if recorded with in a few minutes then we can assume someone made a mistake and lets go with the most recent.

> 117 Blanket 'No' vs. topic-specific 'Yes' committee decision
Do you mean a no for all assigned topics?

> 8 Multiple committee decisions with different notes
> 8 Review received for topic which hadn't been assigned
> 2 Review received for article which hadn't been assigned for any topic
> 2 Reviewer set disposition to 'No'
These four items seem like records with mistakes. Can you send us the CMSIDs?

> 160 'Final' board decision entered more than once
Same decision or different decisions? If the same ...my guess is this is just an error. The CMS has a history of timing out so decisions may have been recorded again in fear they were not recorded due to the time out and then resulted in more than on entry.

> Just to re-cap: the instructions I've gleaned from the email and Bugzilla
> exchanges resulted in decisions to take the last entry when there were
> conflicts between "Yes" and "No" for "Full Text Requested" and "Full Text
> Retrieved" but a "Yes" trumps a "No" for "Committee Decision" regardless of the
> order of entry.

To clarify...a "Yes" trumps a "No" programatically in the current CMS dictating what gets kicked on or off lists. If users recorded yes and no multiple times this doesn't indicate a conflict as such but rather changes in decisions for whatever reason...all of which should be recorded in the citation's history.

BZDATETIME::2012-03-29 18:59:04
BZCOMMENTOR::Alan Meyer
BZCOMMENT::144

Here's my proposal for the comment table we discussed this afternoon. I've added it to ebms.sql and will put it in version control unless someone sees a problem with it.

/*

• One or more optional comments can be associated with an

• ebms_article_state row.
  *

• Comments are immutable. To change one's mind about the contents of

• a comment, a user can post another comment on the same state row.
  *

• comment_id Unique auto-generated row ID of this comment.

• article_state_id Of the article state row this comments on.

• user_id Of user posting the comment.

• comment_dt Datetime comment was recorded.

• comment Free text.
  */
  CREATE TABLE embs_article_state_comment (
  comment_id INTEGER NOT NULL AUTO_INCREMENT PRIMARY KEY,
  article_state_id INTEGER NOT NULL,
  user_id INTEGER UNSIGNED NOT NULL,
  comment_dt DATETIME NOT NULL,
  comment TEXT NOT NULL,

FOREIGN KEY article_state_id REFERENCES
ebms_article_state(article_state_id),
FOREIGN KEY user_id REFERENCES users(uid)
)
ENGINE InnoDB;

• • Usually retrieved in association with state row, in date order
    CREATE INDEX ebms_article_state_comment_state_index ON
    ebms_article_state_comment(article_state_id, comment_dt);

BZDATETIME::2012-03-30 10:23:30
BZCOMMENTOR::Bob Kline
BZCOMMENT::145

(In reply to comment #143)

> > 54162 No history for article
> Are these citations that were not published or rather they were rejected in my
> initial review?

Some of each. By "no history for article" I meant there were no rows in the decision history table in the legacy system for the article.

> > 12974 Articles missing from table for original topic assignment
> I am not sure what this means??

We aren't either. We suspect a bug in the system which was detected and fixed a while ago, since all but one instance are from 2003 and earlier, and that one case was back in 2006.

> > 1736 Article assigned to same topic in different cycles
> Any chance you could provide Minaxi and I a few examples of these?

Sure.

CMS ID 23556
Late Effects of Treatment for Childhood Cancer
April 2003 and February 2011

CMS ID 259873
Screening for Cervical Cancer
January 2012 and February 2012

CMS ID 250108
Chronic Lymphocytic Leukemia
October 2011 and November 2011

> > 391 'Yes' and 'No' full text request for same board/article combo
> Just go with the "yes"

Can you confirm that this is what we should do? Earlier feedback for both full text request state as well as for full text retrieval seemed to indicate that in those cases the last entries were most likely to be corrections for earlier mistakes.

> > 40 NULL response for full text request; no attempt to retrieve
> Set to "No" if null is going to be an issue

That's what we'll do.

> > 864 'Yes' and 'No' for full text retrieved
> Just go with the "yes"

See response above for the contradictory entries for "full text request."

> > 77 'No' vs. 'Yes' committee decision on same topic
> Keep both if recorded at different times (more than a few minutes), if recorded
> with in a few minutes then we can assume someone made a mistake and lets go
> with the most recent.

I may have misunderstood, but I thought that earlier feedback for the "committee decision" entries was that, in contrast with my understanding for the full text request/retrieval entries, a "Yes" trumps a "No" regardless of the sequence of the entries. At the bottom of your responses in this thread you repeated your request to have all of the rows in the history table preserved in conversion, so we're considering a rewrite of the conversion logic to do that. In that case, when multiple entries were carried over for the same article/topic combination, only one of them would be marked as "active" so the software would have a way to answer the question "what is the status of this article in the context of this topic?" (or board for states which are board-specific rather than topic-specific). The simplest approach would be to do as you've indicated here: mark the most recent active. A more nuanced approach would be to have different rules for the different types of states about when an earlier entry's state would override that of a later entry. In that case we plan to do something like the following:

1. add the earlier state, marked as active

2. add the later state, marked as inactive

3. add a third row in the state table, repeating the state recorded in the first row, and the date/time stamp from the entry in the second row, with a system-generated comment indicating that the conversion software has created this artificial extra row as a result of a decision to regard the earlier entry as more likely to be correct than the later one

4. mark the first row inactive

5. mark the third row active

This approach would be used to avoid confusing any software which would get tripped up by having entries marked as active followed by later entries for the same article/topic combination marked as invalid. Again, if we go with the simpler decision of marking the latest entry as active, we won't need this wrinkle.

Whichever approach we take, it will be impossible to know for certain that our assumption about which of contradictory entries is correct and which is wrong for everything which gets converted. You can look at the comments for individual cases and come to reasonable conclusions about which entry was a mistake, but for some cases the comments don't resolve the question.

> > 117 Blanket 'No' vs. topic-specific 'Yes' committee decision
> Do you mean a no for all assigned topics?

This refers to "committee decision" rows for which the "response" value is "No" and no topic is identified.

> > 8 Multiple committee decisions with different notes
> > 8 Review received for topic which hadn't been assigned
> > 2 Review received for article which hadn't been assigned for any topic
> > 2 Reviewer set disposition to 'No'
> These four items seem like records with mistakes. Can you send us the CMSIDs?

Some of these you've seen before (on the spreadsheet analyzing the combinations of "decision" and "response" I found in the tables).

The CMS IDs for the articles which had different notes for the same article/topic "committee decision" were 7725 (with two different topics), 10938, 41308, 47002, 62983, 70221, and 208063.

The articles which got reviews for a topic which hadn't been assigned for review are CMS ID 115863 (3 reviews) and CMS ID 116944 (5 reviews).

The articles which got reviews for articles which hadn't been assigned for any topic are CMS ID 114515 and CMS ID 121528.

The two articles for which a reviewer set the disposition to "No" are CMS ID 34733 and CMS ID 180237.

> To clarify...a "Yes" trumps a "No" programatically in the current CMS dictating
> what gets kicked on or off lists. If users recorded yes and no multiple times
> this doesn't indicate a conflict as such but rather changes in decisions for
> whatever reason...all of which should be recorded in the citation's history.

See the discussion above about rewriting the conversion software to preserve all of the rows in the legacy system's history table.

BZDATETIME::2012-03-30 13:50:24
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::146

(In reply to comment #145)
Thanks for your response. This clears up a few things, but brings up a few more...sorry in advance :)There is alot to go through so I am going to break this up in to several replies.

> > > 54162 No history for article
> > Are these citations that were not published or rather they were rejected in my
> > initial review?
> Some of each. By "no history for article" I meant there were no rows in the
> decision history table in the legacy system for the article.

Correct...Citations that were rejected by my initial review and, therefore, never published would not have a anything in the decision history table. But, Citations that were published should have at least one decision in their history table..NCI Reviewer Decision yes or no(the first decision made by brd managers after publication). If these citations do not have an NCI Reviewer Decision of yes or no, then they were NOT reviewed. Citations NOT reviewed should remain in the board managers que until they are reviewed. So if these citations are not still sitting in someone's que, then what happened to them? Did they ever appear in the que? Did they not get published correctly? this may be another bug.

> > > 12974 Articles missing from table for original topic assignment
> > I am not sure what this means??
> We aren't either. We suspect a bug in the system which was detected and fixed
> a while ago, since all but one instance are from 2003 and earlier, and that one
> case was back in 2006.

Ok so these citations are not really a problem.

> > > 1736 Article assigned to same topic in different cycles
> > Any chance you could provide Minaxi and I a few examples of these?
> Sure.

Thanks, I'll go through these and get back to you.

> > > 391 'Yes' and 'No' full text request for same board/article combo
> > Just go with the "yes"
> Can you confirm that this is what we should do? Earlier feedback for both full
> text request state as well as for full text retrieval seemed to indicate that
> in those cases the last entries were most likely to be corrections for earlier
> mistakes.

Sorry I think I am getting my full text requests and retrievals mixed up. See my comment below for full text retrieval.
As for requests, the way the current CMS is designed, board managers should not have to review the same citation for the same topic twice. Occasionally this is not the case. For example, during complete summary revisions or complete re-writes, previously reviewed citations will be re-reviewed. So brd managers could have initially said yes and then months or years later said no or yes again. When a citation is re-reviewed, these new decisions need to be treated separately from the first review. The current system does not do this and this is why we are having all this confusion now. So a depending on the time lap between the decisions, it could be a case of a decision change for the same review or it could be the first decision of a new review.
I am more concerned about the latest decision being a yes than a no. If the citation has a yes followed by a no, I am tempted to ignore the no and focus on the initial review that started with the yes as there will be more decisions to follow in the review.
If the citations started with a no and then got a yes, I am tempted to ignore the first no and focus on the yes that followed.
I am not sure if I am helping or just adding to the confusion.

> > > 40 NULL response for full text request; no attempt to retrieve
> > Set to "No" if null is going to be an issue
> That's what we'll do.

OK

> > > 864 'Yes' and 'No' for full text retrieved
> > Just go with the "yes"
> See response above for the contradictory entries for "full text request."

Sorry for the confusion on this. But the reason I am saying yes to these is because a yes followed by a no would change the status from "we already got it" to "we need to go get it". If there are citations that really need the full text retrieved then they should have a null value. The no doesn't really have much value here (and maybe in the new system we should go with a check tag instead). As I stated eariler it was used just to kick out tricky citations that were hard to get a copy of.
So if there is at least on yes, then I think we should just go with yes.

BZDATETIME::2012-03-30 14:22:21
BZCOMMENTOR::Bob Kline
BZCOMMENT::147

(In reply to comment #146)

> There is alot to go through so I am going to break this up in to several
> replies.

I'm going to do the same thing. That will mean more replies, but they won't be as difficult to wade through as mega-comments would be.

> > > > 54162 No history for article
> > > Are these citations that were not published or rather they were
> > > rejected in my initial review?
> > Some of each. By "no history for article" I meant there were no rows
> > in the decision history table in the legacy system for the article.
>
> Correct...Citations that were rejected by my initial review and, therefore,
> never published would not have a anything in the decision history table.
> But, Citations that were published should have at least one decision in
> theirhistory table..NCI Reviewer Decision yes or no(the first decision
> made by brd managers after publication). If these citations do not have
> an NCI Reviewer Decision of yes or no, then they were NOT reviewed.
> Citations NOT reviewed should remain in the board managers que until
> they are reviewed. So if these citations are not still sitting in
> someone's que, then what happened to them? Did they ever appear in
> the que? Did they not get published correctly? this may be another bug.

I'm confused. In my understanding of the way the legacy system works, there are a number of conditions an article could be in that would result in its showing up in the bib table, but not yet have any rows in the history table:

• you haven't done your initial review of the article yet

• you did the initial review of the article and rejected it

• you passed it in your initial review but haven't published it yet

• you published it but it's still in the board manager's queue for review

I assume there are articles in each of those four piles (and probably lots in that second pile) which contribute to that total of 54,162 articles. What would the bug be that you're worried about?

BZDATETIME::2012-03-30 14:32:43
BZCOMMENTOR::Bob Kline
BZCOMMENT::148

(In reply to comment #146)

> > > > 864 'Yes' and 'No' for full text retrieved
> > > Just go with the "yes"
> > See response above for the contradictory entries for "full text request."
>
> Sorry for the confusion on this. But the reason I am saying yes to these is
> because a yes followed by a no would change the status from "we already got it"
> to "we need to go get it". If there are citations that really need the full
> text retrieved then they should have a null value. The no doesn't really have
> much value here (and maybe in the new system we should go with a check tag
> instead). As I stated eariler it was used just to kick out tricky citations
> that were hard to get a copy of.
> So if there is at least on yes, then I think we should just go with yes.

I am told that the current system makes it difficult or impossible to correct a mistake, so users have resorted to just making a second entry to correct an error. What would a user do in the system if she had entered "yes I was able to get the full text for article X" and then realized that this wasn't right (different article, most pages missing, whatever)? Wouldn't that have resulted in a (mistaken) "Yes" followed by a (correct) "No"? Which would in turn imply (correctly) "we [still] need to go get it," right?

BZDATETIME::2012-03-30 14:41:26
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::149

(In reply to comment #147)
> > > > > 54162 No history for article
> > > > Are these citations that were not published or rather they were
> > > > rejected in my initial review?
> > > Some of each. By "no history for article" I meant there were no rows
> > > in the decision history table in the legacy system for the article.

Above I asked if these citations were not published and you said "some of each". So you implied that there were published and unpublished citations in this 54162.

> > Correct...Citations that were rejected by my initial review and, therefore,
> > never published would not have a anything in the decision history table.
> > But, Citations that were published should have at least one decision in
> > theirhistory table..NCI Reviewer Decision yes or no(the first decision
> > made by brd managers after publication). If these citations do not have
> > an NCI Reviewer Decision of yes or no, then they were NOT reviewed.
> > Citations NOT reviewed should remain in the board managers que until
> > they are reviewed. So if these citations are not still sitting in
> > someone's que, then what happened to them? Did they ever appear in
> > the que? Did they not get published correctly? this may be another bug.
> I'm confused. In my understanding of the way the legacy system works, there
> are a number of conditions an article could be in that would result in its
> showing up in the bib table, but not yet have any rows in the history table:
> * you haven't done your initial review of the article yet
> * you did the initial review of the article and rejected it
> * you passed it in your initial review but haven't published it yet
> * you published it but it's still in the board manager's queue for review
> I assume there are articles in each of those four piles (and probably lots in
> that second pile) which contribute to that total of 54,162 articles. What
> would the bug be that you're worried about?

This is correct. There are no other conditions.
Are there citations that are in the bib table as published that are not in someone's que? Right now the only citations in the que should be from mostly March 2012 and maybe a few for previous months. If there are published citations with no history that are not in someone's que than we have a bug. If all off the pulished citations are those in the current que then everything is fine.

BZDATETIME::2012-03-30 14:44:14
BZCOMMENTOR::Minaxi Trivedi
BZCOMMENT::150

(In reply to comment #148)
> (In reply to comment #146)
>
> > > > > 864 'Yes' and 'No' for full text retrieved
> > > > Just go with the "yes"
> > > See response above for the contradictory entries for "full text request."

>
> I am told that the current system makes it difficult or impossible to correct a
> mistake, so users have resorted to just making a second entry to correct an
> error. What would a user do in the system if she had entered "yes I was able
> to get the full text for article X" and then realized that this wasn't right
> (different article, most pages missing, whatever)? Wouldn't that have resulted
> in a (mistaken) "Yes" followed by a (correct) "No"? Which would in turn imply
> (correctly) "we [still] need to go get it," right?

Bob, I was about to convey the exact same thought.

BZDATETIME::2012-03-30 14:48:17
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::151

(In reply to comment #148)

> I am told that the current system makes it difficult or impossible to correct a
> mistake, so users have resorted to just making a second entry to correct an
> error. What would a user do in the system if she had entered "yes I was able
> to get the full text for article X" and then realized that this wasn't right
> (different article, most pages missing, whatever)? Wouldn't that have resulted
> in a (mistaken) "Yes" followed by a (correct) "No"? Which would in turn imply
> (correctly) "we [still] need to go get it," right?

Maybe the best way to handle this is to give Bonnie a list of all the citations with "no" and have her figure out which really still need the full text. Then set everything else to yes.

BZDATETIME::2012-03-30 16:11:04
BZCOMMENTOR::Bob Kline
BZCOMMENT::152

(In reply to comment #149)
> (In reply to comment #147)
> > > > > > 54162 No history for article
> > > > > Are these citations that were not published or rather they were
> > > > > rejected in my initial review?
> > > > Some of each. By "no history for article" I meant there were no rows
> > > > in the decision history table in the legacy system for the article.
>
> Above I asked if these citations were not published and you said "some of
> each". So you implied that there were published and unpublished citations in
> this 54162.

Well, what I meant to be conveying was there were some of each of

1. articles that were not published
2. articles that were rejected in your initial review

since those were the two conditions you asked about. But as we've since noted, there are two other smaller piles that make up that 54,162, one of which consists of the articles you haven't reviewed yet, and the other of which is made up of the articles you've published but which are still sitting the the queues of the board managers. So even though I didn't think I was saying anything about published citations being in the 54,162, it is in fact true that there are both published and unpublished articles represented in that count.

> Are there citations that are in the bib table as published that are not in
> someone's que? Right now the only citations in the que should be from mostly
> March 2012 and maybe a few for previous months. If there are published
> citations with no history that are not in someone's que than we have a bug. If
> all off the pulished citations are those in the current que then everything is
> fine.

Hard to say without logging onto the system as each of the CIPS reviewers and examining his or here queue, and manually totaling them to confirm that what the user interface presents as the queues matches what I'm seeing in the database. Here are the counts of articles which are in the mt_REVIEW table (that is, they're "published") but have no rows in the history table, grouped by review cycle:

1 July 2003
1 August 2003
1 March 2004
223 January 2007
1 May 2007
1 August 2007
2 March 2009
432 January 2012
409 February 2012
1298 March 2012

The only eyebrow-raiser is the 223 from early 2007. I was sort of hoping they'd all be for the same reviewer, and that reviewer no longer active, but that's not the case:

42 CAM Reviewer
23 Marianne Noone
158 Robin Baldwin

Not sure what's up with those articles. The account for Marianne isn't active any longer, but that's not true for the other two. I've attached the file with the CMS IDs for these.

Attachment 2007-queues.txt has been added with description: Articles still in the board managers' queues from January 2007

BZDATETIME::2012-03-30 16:29:57
BZCOMMENTOR::Bob Kline
BZCOMMENT::153

(In reply to comment #152)

> Not sure what's up with those articles.

A theory occurred to me: perhaps none of the topics assigned to these articles are still active (that is all of the rows in the asc_CITE_SUMMARIES table for these articles link to topics which have their summary_reject flag set). But I checked, and every one of the articles is linked to at least one non-rejected topic.

So perhaps another theory would be that the software doesn't bother to show the reviewer anything that's older than a certain cutoff age. That wouldn't explain what happened if these articles never showed up in anyone's queue, but I don't have any way of knowing if that's true. At any rate, perhaps Alan can dig into the code and find out if this theory has any merit. (I know, that's one of your favorite activities, right, Alan? :-) )

BZDATETIME::2012-03-30 17:07:45
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::154

(In reply to comment #153)
> (In reply to comment #152)
> > Not sure what's up with those articles.
> A theory occurred to me: perhaps none of the topics assigned to these articles
> are still active (that is all of the rows in the asc_CITE_SUMMARIES table for
> these articles link to topics which have their summary_reject flag set). But I
> checked, and every one of the articles is linked to at least one non-rejected
> topic.
> So perhaps another theory would be that the software doesn't bother to show the
> reviewer anything that's older than a certain cutoff age. That wouldn't
> explain what happened if these articles never showed up in anyone's queue,
> but I don't have any way of knowing if that's true. At any rate, perhaps Alan
> can dig into the code and find out if this theory has any merit. (I know,
> that's one of your favorite activities, right, Alan? :-) )

Good ideas...but it looks like something happened in Jan 2007. We will dig through our notes and see if there was a bug, or publishing error or something at that time. I know that there has been at least one time where we had to unpublish and re-publish a batch of citations...could this be the explanation?

BZDATETIME::2012-03-30 17:10:18
BZCOMMENTOR::Minaxi Trivedi
BZCOMMENT::155

(In reply to comment #145)
> (In reply to comment #143)
>

>
> > > 1736 Article assigned to same topic in different cycles
> > Any chance you could provide Minaxi and I a few examples of these?
>
> Sure.
>
> CMS ID 23556
> Late Effects of Treatment for Childhood Cancer
> April 2003 and February 2011
>
> CMS ID 259873
> Screening for Cervical Cancer
> January 2012 and February 2012
>
> CMS ID 250108
> Chronic Lymphocytic Leukemia
> October 2011 and November 2011
>
> All three examples are the fast track requests and the review cycle was changed before publishing. Changing from current to older review cycle(e.g. November 2011 to October 2011) is followed all the time because the citations are still under review.
I am not sure if all 1736 citations would fall into this category. I remember one instance when batch of citations were imported to a wrong review cycle and then it was corrected.
I wonder if looking at all 1736 would help to eliminate those wrongly imported.

Can we use the current date stamp to decide the correct review cycle?

BZDATETIME::2012-03-30 17:39:36
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::156

(In reply to comment #154)

> Good ideas...but it looks like something happened in Jan 2007. We will dig
> through our notes and see if there was a bug, or publishing error or something
> at that time. I know that there has been at least one time where we had to
> unpublish and re-publish a batch of citations...could this be the explanation?

Minaxi dug up an email where we are discussing issues we were having with publishing Jan 2007. The email doesn't cover all the details nor can we remember but it looks like there were citations from selected peds, supp care and cam summary topics that were not publishing correctly.

Here is that email:

Hi Pete,

Here is the list of summaries affected for Jan 2007. These could be the topics which were sitting at the end of the list before you alphabetized them.

Pediatric Board

General Pediatric Treatment
Histiocytosis

Supportive Care Board

Cardiopulmonary Syndromes
Communication*
End of Life*
General Supportive Care*
Normal Adjustment and the Adjustment Disorders

CAM BOARD
Acupuncture*
Aromatherapy*
Cartilage (Bovine and Shark)
Coenzyme Q10
Laetrile / Amygdalin
Milk Thistle
Mistletoe Extracts
Newcastle Disease Virus
PC-SPES
Phytoestrogens*

So the Jan 2007 citations were published, there was an error preventing possibly 223 citations from selected topics listed above from getting published. Pete made some change and then these citations were published. If they were in fact published (which I am sure at the time Minaxi and I tested to make sure they appeared in the right board managers que) then why do they not have any data in the history table? Were they ever reviewed?
We might need to see these 223 as well.

BZDATETIME::2012-03-31 09:05:47
BZCOMMENTOR::Bob Kline
BZCOMMENT::157

(In reply to comment #154)

> I know that there has been at least one time where we had to unpublish
> and re-publish a batch of citations...could this be the explanation?

I don't think that would explain why these aren't showing up in the queues.

BZDATETIME::2012-04-02 11:41:12
BZCOMMENTOR::Bob Kline
BZCOMMENT::158

(In reply to comment #155)

> All three examples are the fast track requests and the review cycle
> was changed before publishing. Changing from current to older review
> cycle(e.g. November 2011 to October 2011) is followed all the time
> because the citations are still under review.
>
> I am not sure if all 1736 citations would fall into this category.
> I remember one instance when batch of citations were imported to a
> wrong review cycle and then it was corrected.
>
> I wonder if looking at all 1736 would help to eliminate those wrongly
> imported.
>
> Can we use the current date stamp to decide the correct review cycle?

I can provide you with the legacy article IDs for all of these if you think that would be helpful, but it may not be needed. The current version of the tables for the new system only records a cycle ID for the import history of an article, and the import history table isn't being populated by the conversion.

Alan:

I noticed that the documentation for the new cycle table is out of date.

/*

• Periods of time used to batch articles associated with a given topic.
  *

• The concept of batching articles associated with a topic in clumps

• related to periods of time (typically a month) is no longer as useful

• as it was in the original Citation Management System. In the new

• system board managers will be free to assemble article review packets

• containing articles from any cycle in the same packet (in fact, this

• practice has been common for a while with some of the boards).

• Nevertheless, the assignment of a topic to an article will still

• carry an association with a cycle, and that cycle will be displayed

• when the review packet is being created.
  *

• cycle_id Automatically generated primary key

• cycle_name Unique name for the cycle (e.g., 'November 2011')

• start_date Datetime of start. Always order by start_date to guarantee

• retrieval in date order since cycle_ids could be created

• out of order due to conversion from old CMS or

• for other reasons.
  */

Since we're not relying on the state table for representing the assignment of topics to articles, we no longer have the association with a cycle described in the table comment quoted above. Should we amend this documentation, to say that the display of the cycle will be taken from the import history tables? This will mean that converted articles won't have a cycle to display, when review packets are being created for them (though a cycle can be determined by looking at the import date for the article, at least approximately).

BZDATETIME::2012-04-03 09:23:14
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::159

(In reply to comment #145)
> > > 77 'No' vs. 'Yes' committee decision on same topic
> > Keep both if recorded at different times (more than a few minutes), if recorded
> > with in a few minutes then we can assume someone made a mistake and lets go
> > with the most recent.
> I may have misunderstood, but I thought that earlier feedback for the
> "committee decision" entries was that, in contrast with my understanding for
> the full text request/retrieval entries, a "Yes" trumps a "No" regardless of
> the sequence of the entries. At the bottom of your responses in this thread
> you repeated your request to have all of the rows in the history table
> preserved in conversion, so we're considering a rewrite of the conversion logic
> to do that. In that case, when multiple entries were carried over for the same
> article/topic combination, only one of them would be marked as "active" so the
> software would have a way to answer the question "what is the status of this
> article in the context of this topic?" (or board for states which are
> board-specific rather than topic-specific). The simplest approach would be to
> do as you've indicated here: mark the most recent active. A more nuanced
> approach would be to have different rules for the different types of states
> about when an earlier entry's state would override that of a later entry. In
> that case we plan to do something like the following:
> 1. add the earlier state, marked as active
> 2. add the later state, marked as inactive
> 3. add a third row in the state table, repeating the state recorded in the
> first row, and the date/time stamp from the entry in the second row, with a
> system-generated comment indicating that the conversion software has created
> this artificial extra row as a result of a decision to regard the earlier entry
> as more likely to be correct than the later one
> 4. mark the first row inactive
> 5. mark the third row active
> This approach would be used to avoid confusing any software which would get
> tripped up by having entries marked as active followed by later entries for the
> same article/topic combination marked as invalid. Again, if we go with the
> simpler decision of marking the latest entry as active, we won't need this
> wrinkle.
> Whichever approach we take, it will be impossible to know for certain that our
> assumption about which of contradictory entries is correct and which is wrong
> for everything which gets converted. You can look at the comments for
> individual cases and come to reasonable conclusions about which entry was a
> mistake, but for some cases the comments don't resolve the question.
> See the discussion above about rewriting the conversion software to preserve
> all of the rows in the legacy system's history table.

You are right there isn't a perfect solution for this. Luckily there are only 77. I think either approach would work.

BZDATETIME::2012-04-03 09:55:31
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::160

(In reply to comment #145)
> > > 117 Blanket 'No' vs. topic-specific 'Yes' committee decision
> > Do you mean a no for all assigned topics?
> This refers to "committee decision" rows for which the "response" value is "No"
> and no topic is identified.

So these will just be inactive. I am not sure why there isn't a summary topic assigned to the no...but a no none the less with no other decision would inactive the citation.

BZDATETIME::2012-04-03 14:08:16
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::161

(In reply to comment #145)
> > > 8 Multiple committee decisions with different notes
> > > 8 Review received for topic which hadn't been assigned
> > > 2 Review received for article which hadn't been assigned for any topic
> > > 2 Reviewer set disposition to 'No'
> > These four items seem like records with mistakes. Can you send us the CMSIDs?
> Some of these you've seen before (on the spreadsheet analyzing the combinations
> of "decision" and "response" I found in the tables).
> The CMS IDs for the articles which had different notes for the same
> article/topic "committee decision" were 7725 (with two different topics),
> 10938, 41308, 47002, 62983, 70221, and 208063.

I am not seeing different notes in these citations. Minaxi is having a look at them ... maybe I am missing it. So we will come back to these 8.
Below are my comments for the others.

The articles which got reviews for a topic which hadn't been assigned for review are
CMS ID 115863 (3 reviews) …late effects was added at the lit surv comm decision which is allowed so there isn’t anything wrong here.
CMS ID 116944 (5 reviews) … test citation can be deleted

The articles which got reviews for articles which hadn't been assigned for any topic are
CMS ID 114515 … this looks like a mistake the lit surv comm decision should be yes
CMS ID 121528 … this looks like a mistake the lit surv comm decision should be yes

The two articles for which a reviewer set the disposition to "No" are
CMS ID 34733
CMS ID 180237
This is odd..let’s delete the “NO” since each of these citations have a “Warrants No Change” for the same reviewer at the same time.

BZDATETIME::2012-04-03 14:26:39
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::162

(In reply to comment #157)
> (In reply to comment #154)
> > I know that there has been at least one time where we had to unpublish
> > and re-publish a batch of citations...could this be the explanation?
> I don't think that would explain why these aren't showing up in the queues.

Ok so these citations were published but never made it into the que and were therefore never reviewed. They are now 5 years old. If Robin wants to review them could we get them in her que?

BZDATETIME::2012-04-03 15:03:39
BZCOMMENTOR::Bob Kline
BZCOMMENT::163

(In reply to comment #162)
> (In reply to comment #157)
> > (In reply to comment #154)
> > > I know that there has been at least one time where we had to unpublish
> > > and re-publish a batch of citations...could this be the explanation?
> > I don't think that would explain why these aren't showing up in the queues.
>
> Ok so these citations were published but never made it into the que and were
> therefore never reviewed. They are now 5 years old. If Robin wants to review
> them could we get them in her que?

I guess the first step is to check with Alan to see if he's had any luck in determining from the code why the old system is suppressing them.

BZDATETIME::2012-04-03 21:06:02
BZCOMMENTOR::Alan Meyer
BZCOMMENT::164

(In reply to comment #163)
> ...
> I guess the first step is to check with Alan to see if he's had any luck in
> determining from the code why the old system is suppressing them.

I'm not sure that I've 't figured it all out but I do have some
ideas that can be further tested.

There are 249 rows in the mt_review table that have no
corresponding rows in asc_search_topics. Most of these are in
review cycle 76 - the one with most of the problems.

Here's what I think may be happening in the system:

All of the code I found that creates a row in the mt_review
table goes on to create one in the asc_search_topics table.
I haven't found any legitimate path for creating a row in the
former without creating one in the latter. But there must
have been such a path at one time, or maybe the data was
there at one time but got deleted somehow.

It looks to me like the asc_search_topics table exists in
order to record board decisions. If summary_accepted = 1,
then a user looking at the screen entitled "Record Editorial
Board Meeting Decision" (produced by "DecisionPicked.asp")
selected "Yes" for "Board Decision". The asc_search_topics
row with the summary_accepted column is assumed to be there
and it is updated to change the summary_accepted value from 0
to 1, and then a row is inserted in the mt_decision_history
table.

It looks to me that when the program searches for citations
for a user that have not been reviewed, it checks for
asc_search_topics.summary_accepted = 0. In other words, if
the summary has already been accepted by the board, the user
(board manager or reviewer) doesn't need to review this
article. It's a done deal.

[If I'm right about this, it may be the case that there is no
information in asc_search_topics that can't be derived from
mt_decision_history - which was where Bob was looking for it,
but which does not seem to be where all of the old CiteMS
code looked for it.]

The problem is, what if there is no row in the asc_search_topics
at all? It looks like the query to locate queued citations won't
find the citation that is awaiting review.

We can test for this in a number of ways:

1. Create a new query to replace the current one that finds
citations in a queue. It should look for mt_review rows that
do NOT have a corresponding asc_search_topics row with
summary_accepted = 1 instead of looking for rows that do have
summary_accepted = 0.

That will take some thought since the existing query depends
on asc_search_topics for other linking information too.

Or alternatively:

2. Insert rows into asc_search_topics with summary_accepted = 0
for every mt_review that doesn't have a corresponding
asc_search_topics row.

I like that solution because it's easy to do, easy to test
(we can test it in the dev/test database), and might fix
other problems that we don't know about. It looks like there
is a lot of code that looks in asc_search_topics and other
functionality might be missing these citations, not just the
functions that produce work queues.

If it really is easy as it seems to look, I would have
thought that the programmer who fixed the program would have
also fixed the data. Maybe it's not as easy as it looks.
Maybe I've got it wrong. Or maybe he found some other
symptom of the problem and didn't realize that this was also
affected. Or maybe he just didn't give this problem his full
attention.

I won't do any more with this right now. I've already spent much
more time on it than I planned and have delayed some other work
in EBMS. I'll wait to discuss it with Bob and see what he thinks
we should do.

BZDATETIME::2012-04-10 17:33:27
BZCOMMENTOR::Robin Juthe
BZCOMMENT::165

Since we are planning to add a "recent activity" block to the home page for both Board members and Board managers, I wanted to provide our thoughts on what sorts of activities should display in this block.

As Board managers, we would be most interested in seeing the following items in our "recent activity" feed:

• a new summary document was posted (for her Board).

• a new discussion was started (in one of her forums).

• a new Board member (on her Board) was added to the EBMS/CiteMS.

It would also be helpful to know who posted the document and who started the discussion. For example:

• Kevin Zbuk posted a summary document. 4/5/2012

• Mark Greene started a new discussion. 4/3/2012

We think Board members would be interested in seeing the folloiwng items in their "recent activity" feed:

• a new packet is ready for their review. (it would be nice if the summary topic could be mentioned.)

• the agenda for a meeting was published to an event he/she is invited to attend.

• a meeting date, time, or location (e.g., WebEx, teleconference) for a meeting he/she is invited to changed.

• a new Board member (on his/her Board) was added to the EBMS/CiteMS.

Here are some examples:

• Genetics of Skin Cancer literature surveillance posted. 4/6/2012

• Agenda for 4/17/2012 Skin WG meeting posted. 4/5/2012

• 4/17/2012 Skin WG meeting time changed. 4/5/2012

• Mark Pomerantz joined the Cancer Genetics Board. 4/3/2012

Another option that we discussed is having a checkbox to indicate whether an update we are making should wind up in the recent activity block for our Board members. Ideally, this will be as automated as possible, but it's another option.

We would initially like to define "recent" as the past 1 month, so all applicable activities that happen in the past month would show in this box. We might want to establish a max number of items in the box though as well.
Bob, maybe we can talk this over later in the week.

BZDATETIME::2012-04-11 09:35:48
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::166

(In reply to comment #165)
This would also be a great way for Minaxi and I to let board managers know not only when their usual monthly citations are ready for review but also if we are publishing results from a special search request.

BZDATETIME::2012-04-11 10:09:48
BZCOMMENTOR::Robin Juthe
BZCOMMENT::167

(In reply to comment #166)
> (In reply to comment #165)
> This would also be a great way for Minaxi and I to let board managers know not
> only when their usual monthly citations are ready for review but also if we are
> publishing results from a special search request.

Good idea - let's add that to the Board manager list. However, I think we may still want to be notified via a system message when the monthly literature or results of a special search are posted since that would trigger a notification in our email inbox. (Recent activity will only be seen when we log into the system.) Granted, we will be in the system a lot so this may not be an issue.

BZDATETIME::2012-04-17 18:20:54
BZCOMMENTOR::Alan Meyer
BZCOMMENT::168

(In reply to comment #164)
> (In reply to comment #163)
> ...
> The problem is, what if there is no row in the asc_search_topics
> at all? It looks like the query to locate queued citations won't
> find the citation that is awaiting review.
>
> We can test for this in a number of ways:
> ...
> 2. Insert rows into asc_search_topics with summary_accepted = 0
> for every mt_review that doesn't have a corresponding
> asc_search_topics row.

I've done a little more work on this. Unfortunately, it's not as
simple as I thought it would be. In order to get one of the
citations from the limbo where it is now into a review queue, I
have to figure out what summary topic to review it under. I can
pick one arbitrarily from the list of summary topics assigned to
the ref_id, but if I do that it won't necessarily be assigned to
the same reviewer to whom it was initially assigned. I can try
to pick a topic reviewed by one of the original reviewers but for
about 20-25% that won't work because the original reviewer is no
longer here, or because topics have been re-assigned since the
reviews were queued. I can try to get all of that as right as
possible, and it is feasible to get 75% or more right and choose
the rest arbitrarily, but it may be more work than it's really
worth.

If we do nothing at all, then when we convert the data the 249
missing review assignments will magically appear in queues
because the articles will have been converted with a last status
of "ready for review".

There are other options too.

So I'm just going to keep some notes and SQL queries and do
nothing more right now. There's other stuff that seems to me to
be higher priority (for example, PermaTargIds, not to mention
other EBMS work.)

BZDATETIME::2012-04-24 11:31:22
BZCOMMENTOR::Robin Juthe
BZCOMMENT::169

I'm attaching an updated list of report ideas and specifications for the new system. In addition to the reports listed in the previous attachment (EBMS Report Specs, which I am marking obsolete), this attachment includes the following new reports:

-Literature Surveillance Review
-Articles by Status
-No Responses

Attachment EBMS Report Specifications - 4_24_12.doc has been added with description: EBMS Report Specs version 2

BZDATETIME::2012-04-27 13:56:48
BZCOMMENTOR::Bob Kline
BZCOMMENT::170

Alan:

I have extracted several cases from the report you and I were reviewing yesterday, with notes and questions. Please take a look and let me know if you think I'm heading in the right general direction.

To bring everyone else up to date:

Alan and I have decided that I should preserve as much of the history as possible, and that I should avoid marking any of the rows as inactive (he will take care of that himself after the fact if at some time in the future he determines that's needed). For "committee decision" rows I do have to do some collapsing since the legacy database tables denormalized the information for these by putting in multiple rows for the same decision, one for each reviewer. The basic approach I'm taking is that if the board, the response, the cycle, the date, the user, the CIPS reviewer, the topic, and the note are the same for multiple adjacent (by time) rows, then for those rows I'll only put one row in the EBMS state table. A further refinement under consideration would also collapse for cases in which multiple reviewers were assigned for multiple topics on an article. In such cases the rows for the same topic aren't adjacent (the order is generally all the rows for reviewer A, topics X, Y, Z, ..., followed by all the rows for reviewer B, topics X, Y, Z, ..., and so on). With a little tricky logic I think I could have only one row for each topic in such cases.

Attachment questions.txt has been added with description: Notes from review of committee decision cases

BZDATETIME::2012-04-27 20:42:30
BZCOMMENTOR::Bob Kline
BZCOMMENT::171

Alan:

Here are the fuller logs of "committee decision" cases where I won't be collapsing adjacent multiple rows. I went ahead and implemented the more complicated logic to collapse multiple rows for the same topic when they weren't adjacent because multiple reviewers were being assigned to multiple topics at the same time. For this log, as well as the previous attachment, you'll want to view the files in a tool which can be expanded to a pretty wide size.

Next I'm going to move on to collecting and analyzing what we'll get when I apply the principle of preserving all of the distinct decision rows across all decision types. Will keep you posted.

Attachment committee-decisions6.txt has been added with description: Cases where I won't be collapsing

BZDATETIME::2012-05-08 13:35:38
BZCOMMENTOR::Bob Kline
BZCOMMENT::172

Cynthia:

It's not likely you'll know the answer to this question, given the age of the data in question, but it can't hurt to ask: during the testing of the conversion, the QA team noticed some puzzling dates for the initial decision rows in the state table, which we pull from the asc_CITE_SUMMARIES table in the legacy system. The example they showed me is for the article with PMID 12020673 (CMS ID 7584). There are four rows in the asc_CITE_SUMMARIES table for this article, one for each of four topics, two of which have the summary_reject value set to 1, and other two have this flag set to 0 (zero). The puzzle is that the date_added column for all four rows has December 16, 2004, when all of the activity displayed in the web interface for this article is recorded as having taken place in 2002 (the legacy system's interface doesn't show you the dates for the initial review decisions). Is it possible that somehow the initial review decisions didn't get recorded for this article back at the outset, with the result that the article kept showing up in your queue, and you had to record those initial review decisions retroactively to get them out of the queue?

BZDATETIME::2012-05-08 14:34:11
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::173

(In reply to comment #172)

This citation has the note "anacostia_test_pc". I searched the database for other citations with this same note and only this one citations is retrieved.
Anacostia is the name of the first CMS server. It held more than just the CMS and had problems from time to time. There is not a name listed with the date last modified of 12/16/2004 which means one of the programers at the time must have used this citation to test something that was going on with the server at the time and added the note. Maybe whatever they were fixing/testing changed the date?

> Cynthia:
> It's not likely you'll know the answer to this question, given the age of the
> data in question, but it can't hurt to ask: during the testing of the
> conversion, the QA team noticed some puzzling dates for the initial decision
> rows in the state table, which we pull from the asc_CITE_SUMMARIES table in the
> legacy system. The example they showed me is for the article with PMID
> 12020673 (CMS ID 7584). There are four rows in the asc_CITE_SUMMARIES table
> for this article, one for each of four topics, two of which have the
> summary_reject value set to 1, and other two have this flag set to 0 (zero).
> The puzzle is that the date_added column for all four rows has December 16,
> 2004, when all of the activity displayed in the web interface for this article
> is recorded as having taken place in 2002 (the legacy system's interface
> doesn't show you the dates for the initial review decisions). Is it possible
> that somehow the initial review decisions didn't get recorded for this article
> back at the outset, with the result that the article kept showing up in your
> queue, and you had to record those initial review decisions retroactively to
> get them out of the queue?

BZDATETIME::2012-05-08 14:40:52
BZCOMMENTOR::Bob Kline
BZCOMMENT::174

(In reply to comment #173)

> Maybe whatever they were fixing/testing changed the date?

Sounds plausible. Thanks, Cynthia.

BZDATETIME::2012-05-18 13:36:09
BZCOMMENTOR::Robin Juthe
BZCOMMENT::175

I'm attaching an updated list of report ideas and specifications for the new
system. In addition to the reports listed in the previous attachment (EBMS
Report Specs version 2, which I am marking obsolete), this attachment includes the following new reports:

-Article Follow-Up
-Literature Surveillance History

Perhaps more importantly, this version also highlights which reports we feel are critical to have in place when we launch the system. These reports are as follows:

-Documents
-Literature Surveillance Reviews
-Articles by Status

We should meet to discuss these in more detail once development for iteration 1 slows down.

Thanks.

Attachment EBMS Report Specifications - 5_1_12.doc has been added with description: EBMS Report specs version 3

BZDATETIME::2012-06-28 20:58:51
BZCOMMENTOR::Alan Meyer
BZCOMMENT::176

This is an output of the Bold/Underline QC report for the HP Bladder Cancer Treatment Summary.

See the next attachment for a PDF version.

Attachment BladderCancerQC.html has been added with description: Bladder Cancer QC report HTML

BZDATETIME::2012-06-28 21:04:50
BZCOMMENTOR::Alan Meyer
BZCOMMENT::177

I have been working on the problem of printing packets for those EBMS users who prefer to receive paper. I haven't yet got an approach to the total problem, but I thought that one of the key problems that needs to be solved is the printing of CDR/PDQ Summaries. I was hoping for a way to script that so that a user (e.g., Bonnie) wouldn't have to call up the CDR Admin system, run a QC report, get the output in her web browser, and go through the steps to print it.

We convert to MS Word to produce editable versions, but Word is not a good intermediary format for printing purposes. It too requires that a user call up an application (Word in this case), and use the mouse to print the document. With Word we also have the extra step of converting from HTML to Word, and then printing.

So I looked for a package that will produce PDF from HTML. I think we can script the output of PDF formats using CUPS (the Common Unix Printing System) on Linux.

I downloaded and tried four different packages. One was troublesome to install and I didn't get it fully working. One crashed on the QC report. One produced a reasonable output for everything but the tables. But the last one appeared to work pretty well. It was "wkhtmltopdf", an open source project that uses the "Webkit" rendering engine built into the Apple Safari browser. It is itself a fork of the KDE Khtml package.

I installed it easily and got it running on an Ubuntu 12.04 system I'm running in a VirtualBox on my desktop. I have attached the PDF output of the HP Bladder Cancer Summary created in HTML in the previous attachment.

Attachment BC.pdf has been added with description: Bladder Cancer QC report PDF

BZDATETIME::2012-06-28 23:44:04
BZCOMMENTOR::Bob Kline
BZCOMMENT::178

(In reply to comment #177)

> I installed it easily and got it running on an Ubuntu 12.04 system I'm running
> in a VirtualBox on my desktop. I have attached the PDF output of the HP
> Bladder Cancer Summary created in HTML in the previous attachment.

I wouldn't go too far down a path which assumes we know what platform we'll be running on. Ideally, we'd be able to come up with a solution which didn't have such a dependency. If we're not able to do that, then I think we need to talk with the folks who are in a position to dictate (or heavily influence) the choice of platform and get some commitments on that decision before we go too much further. As for this particular solution, note that it has the problem common with many of the browsers' printing engines that many of the lines on page boundaries are sliced vertically, with the top halves of the letters on one page, and the bottom halves on the subsequent page, which doesn't make for such easy reading. :-)

BZDATETIME::2012-06-29 00:01:35
BZCOMMENTOR::Alan Meyer
BZCOMMENT::179

(In reply to comment #178)

> I wouldn't go too far down a path which assumes we know what platform we'll be
> running on. Ideally, we'd be able to come up with a solution which didn't have
> such a dependency.

I haven't tested it, but it supposedly works on Windows too. So we're probably safe on that scroe.

> ... As for this particular solution, note that it has the problem
> common with many of the browsers' printing engines that many of the lines on
> page boundaries are sliced vertically, with the top halves of the letters on
> one page, and the bottom halves on the subsequent page, which doesn't make for
> such easy reading. :-)

Carried away by youthful enthusiasm and by the fact that it handled the tables that the other package couldn't, I didn't even notice the page break problem. That's too bad.

I'll keep this one in my back pocket and look for more.

One of the issues we'll have to discuss is the tradeoff between high automation and high quality output. It's not a clear cut case. It may be that spending an extra $5-10,000 per year on labor is justified, especially if we think the amount will go down over time. Of course if we make the paper output ugly enough, we might move the board members to electronic delivery more quickly :^)

BZDATETIME::2012-07-03 12:06:30
BZCOMMENTOR::Alan Meyer
BZCOMMENT::180

I spoke to John Doyle today. He told me a number of things about the environment of our new Drupal servers at CBIIT. If I understand him correctly, he said the following:

1. CBIIT will not support Drupal on Windows, only on Linux.

All of the Drupal applications, including those that are currently running on Windows, will run on Linux at CBIIT.

2. CBIIT had put a development server online.

It's a small server, 1 GB RAM and 60 GB disk, but they told him they can expand it quickly if and when required. John has the root password for the system.

John's current plan is to continue his own development on Windows and use the CBIIT development server as a staging server rather than for true development.

3. There will be a mix of Drupal cores on the system.

I didn't get the details but it seems like they will sometimes use one core for several systems, but use different cores when there might be conflicts between modules selected for different projects.

BZDATETIME::2012-07-03 12:27:00
BZCOMMENTOR::Alan Meyer
BZCOMMENT::181

(In reply to comment #178)

> I wouldn't go too far down a path which assumes we know what platform we'll be
> running on. Ideally, we'd be able to come up with a solution which didn't have
> such a dependency. If we're not able to do that, then I think we need to talk
> with the folks who are in a position to dictate (or heavily influence) the
> choice of platform and get some commitments on that decision before we go too
> much further.

It now looks to me like we can count on having both Windows and Linux available to us. The CDR will still run on Windows and it appears that the EBMS will run on Linux. We need data from both systems and can print on either one or both.

I agree that it makes sense to have as much of the printing solution as practicable be platform independent. However if it turns out that it is significantly easier to print on one of the platforms, or if the tools for good quality output are better on one, then I'm not sure that we should put too much effort into portability. In other words, make all of the parts portable that are easy to make portable, but evaluate hard parts based on the amount of effort required and the impact on print quality.

BZDATETIME::2012-07-27 00:51:45
BZCOMMENTOR::Alan Meyer
BZCOMMENT::182

I have implemented fairly complete citation management import
functionality. There are no real user interfaces yet.
Everything I've done has been tested only with a very bare user
interface, created just to test that the functionality works.

A user (usually Minaxi) can store a search results file from NLM
in either of two formats, the MEDLINE print Format, or the XML
format. The software will autodetect which format is being used.
There seems to be a limit of 200 hits in the MEDLINE format,
whereas the Pubmed XML is unlimited (using the Pubmed "Send To /
file" function.)

She can then upload that file in either of two modes:

"test" mode:

Uploads the file and does everything except store the
records. Specifically, it checks the records for
duplicates, checks to see if new topics need to be
assigned, checks the not list, and checks to see if any
of the new records are the same record but with modified
contents from existing ones, which will then replace the
existing ones.

It produces a report with all of the information from the
test, with Pubmed IDs, EBMS IDs if any, and counts for
each category.

NOT list processing is on by default but can be turned
off in order to import records from NOT listed journals.

I would expect test mode uploading to be heavily used
since it allows a user to see what will happen and spot
surprises before actually updating the database.

"live" mode:

Does all the same work plus:

Updates the article table with new records.

Replaces any records that are newer than what we
have.

Updates the author indexes.

Sets the status of each record, including topic
associations. NOT listed records are imported (if
they are not duplicates) but their status is
automatically set to "Rejected by NOT list".

Creates a permanent record in the database of all of
the information concerning this import action -
allowing the report to be recreated any time it is
desired in the future.

I tested on some small and large samples to check the
functionality and performance. I had gotten some very good times
on smaller sets but even the biggest set I tested seemed likely
to be acceptable. It was a test of 402 records on Prostate
Cancer representing about three weeks of recent Pubmed updates.

Statistics were as follows:

285 articles imported
111 NOT listed (imported but set to Rejected)
7 duplicates
3 topic added
2 replaced

The numbers add up to 408 instead of 402 because some articles
were in more than one category.

The file that I uploaded was 2.74 MB. I only used that file to
find the Pubmed IDs. The actual imports were done by fetching
the articles directly from Pubmed, so if the file is saved at one
time but imported later, it's the latest versions that are
imported.

Timings, not including the import of the search file (several
seconds), were:

test mode: 0:09 minutes:seconds
live mode: 1:24 minutes:seconds

The live mode time was longer than I'd like, but probably still
acceptable. I'm hoping it will be faster on the CBIIT machines
and, even if it's not, 402 records would be an unusually large
search for us.

BZDATETIME::2012-08-22 00:29:46
BZCOMMENTOR::Alan Meyer
BZCOMMENT::183

After our meeting today to discuss display screens for the import
process I reviewed my software that imports citation articles
from Pubmed. I decided that I was handling not lists incorrectly
and revised the code.

The revised code now works as follows:

1. If an article is not in the system at all, add it to the
database whether it's from a not listed journal or not.

It gets an import date and appears on the import report in
the "Imported" category.

2. If the article is:

New for this import topic, and

The import is using the not list, and

The journal is on the not list

Then:

Mark the article as rejected by not list.

Otherwise:

Mark it ready for review.

The logic behind the first condition under number 2 is that
someone might have imported an article already for the current
import topic, for example as part of a fast track. If so, we
don't want to reset it's status to "Rejected by NOT list",
thereby wiping out whatever work has already been done on the
article.

However if the article was imported for a different board or
topic, then for the current board and topic it is indeed rejected
by the NOT list. In that case the article will appear on the
Ready for review list, and on the Topic added list.

In light of all this I'm reporting import results with an extra
category as follows:

If an article is imported, whether not listed or not, it
appears in the list of imported articles.

If it was marked "Ready for review", it appears in a list of
ready for review articles (this is new).

If it was marked "Rejected by NOT list", it appears in a list
of rejected articles.

So the "imported" list will contain some articles that are also
on the ready for review list and some that are also on the
rejected list. The ready for review list will be the one that is
most like the imported list in the old system.

It's complicated, but it reflects the actual processing at import
time and yields a lot of information.

BZDATETIME::2012-08-22 10:16:08
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::184

(In reply to comment #183)
> However if the article was imported for a different board or
> topic, then for the current board and topic it is indeed rejected
> by the NOT list. In that case the article will appear on the
> Ready for review list, and on the Topic added list.

This is confusing. Here is my understanding and let see if we are on the same page.
A citation is imported with a batch of citations for Adult Treatment board AML summary topic, the journal for this particular citation is on the Adult Treatment NOT list so the citation is imported but "Rejected by NOT list". This citation will be in the system but will not appear in any list for reviewing at least not for the Adult treatment board.(Should it not still have the summary topic added so that we know that this citation did come up in our Adult treatment AML search but was just rejected by not list? especially if we are giving users the ability to view and override a not listed citation. This would save them the time of adding the AML topic if they decide to use the citation later. Maybe we could use the little circle with the line through it icon next to the sumtop for an indication of Not listed.)
The same citation is later imported with a batch of citations for Peds AML, but this time the journal is not rejected because it is not on the Peds NOT journal list. So the peds AML sumtop is added and it appears on the Ready for Review List. And will appear on Sharon's list if I do not reject it for publication. But it is still considered rejected by the adult treatment not list and will continue to not appear in any adult treatment review lists.
The same citation is imported a third time for Supp Care Anxiety. But this time it gets rejected by the Supp Care NOT journal list and appears again as rejected by the not list. However this does not change the status of the citation with regards to the Peds Board.

BZDATETIME::2012-08-23 20:47:36
BZCOMMENTOR::Alan Meyer
BZCOMMENT::185

(In reply to comment #184)
> (In reply to comment #183)
> > However if the article was imported for a different board or
> > topic, then for the current board and topic it is indeed rejected
> > by the NOT list. In that case the article will appear on the
> > Ready for review list, and on the Topic added list.
>
> This is confusing.
...

Confusing indeed. It's totally screwed up.

I must have been working too late when I wrote that.

Your description is right and is, in fact, the way the software has been implemented.

Specifically:

... if the article was imported for a different board or
topic, then for the current board and topic it is indeed rejected
by the NOT list. In that case the article will NOT appear on
the Ready for review list, and WILL be on the Topic added list.

I wrote it up exactly backwards.

Sorry for the confusion.

BZDATETIME::2012-08-28 19:06:51
BZCOMMENTOR::Alan Meyer
BZCOMMENT::186

For EBMS printing, we need a mockup of what the review response
sheet for each article to be reviewed should look like.

I'm thinking that it should have a variable part:

Name and user ID of the reviewer.

Name of the Editorial Board.

Name of the Summary topic.

Identification of the article for which this is a response,
e.g.:

Up to two or three authors.
Article title.
Journal citation information.
Pubmed ID
EBMS article ID

and a fixed part that is the same for each response sheet:

4 Disposition checkboxes.
15 Sub-checkboxes for "Warrants no change"
Place to enter comments
Place to enter levels of evidence for the other three
checkboxes.

Instructions for how to fill in and fax, mail, or email a
scan of the response.

I can mock something up, but it might look better if a user does
it.

I'm planning to produce a PDF so I should be able to accommodate
some reasonable font and graphic content if needed.

BZDATETIME::2012-08-29 09:07:11
BZCOMMENTOR::Margaret Beckwith
BZCOMMENT::187

I will work with Victoria to come up with something Alan.

BZDATETIME::2012-08-31 01:06:27
BZCOMMENTOR::Alan Meyer
BZCOMMENT::188

Here are some notes I made regarding our meeting today to discuss
citation management user interface issues:

1. Multiple topics reviewed for one board.

If an article has been assigned two summary topics in the
same editorial board, and if two different reviewers review
those topics, the old system system would (rather
mysteriously) pick one reviewer to review the article. The
decision made by that reviewer would apply to both topics.

In the new system, both topics should be shown on each
reviewer's screen, but each reviewer should have Accept /
Reject buttons only for his topic. When he accepts or
rejects an article, he does so only for that topic.

If one reviewer reviews both topics, that reviewer should see
two pairs of Accept / Reject buttons.

I think this requires no change to Ashleigh's user interface
screen. It is how the screen is generated that is affected
(we won't put Accept/Reject buttons on some lines of the
screen.)

2. Abstract displays.

Victoria said that she always reads the abstract for an
article during the review process. It is cumbersome for her
to have to click a button to bring up the abstract, read it,
and then dismiss the pop-up.

It would be desirable if there were a setting for a user that
specified that abstracts will appear on the review screen.
This would have to appear in a format that spans all of the
columns since abstracts are large blocks of text.

3. Fixing import errors.

Cynthia said that sometimes a complete fiasco can occur in an
import, for example importing articles from a search into the
wrong board and/or topic. In the old CMS there was no good
way to address this. Sometimes the programmer would
manipulate the tables by hand to fix the problem. She
requested a way to fix such errors in the new system.

My own sense of this is that we need to figure out all of the
reasonable options before we make any decision about this.
There are many ways to handle errors - some of which require
little new software and little risk, others of which require
more software and more risk (e.g., of doing other damage to
the database while trying to correct something.) The old
system had a lot of corrupt data.

BZDATETIME::2012-08-31 10:33:22
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::189

(In reply to comment #188)
> 3. Fixing import errors.
> Cynthia said that sometimes a complete fiasco can occur in an
> import, for example importing articles from a search into the
> wrong board and/or topic. In the old CMS there was no good
> way to address this. Sometimes the programmer would
> manipulate the tables by hand to fix the problem. She
> requested a way to fix such errors in the new system.
> My own sense of this is that we need to figure out all of the
> reasonable options before we make any decision about this.
> There are many ways to handle errors - some of which require
> little new software and little risk, others of which require
> more software and more risk (e.g., of doing other damage to
> the database while trying to correct something.) The old
> system had a lot of corrupt data.

What if we just "hide" to errors. Cover them up with a blanket and pretend they never happened 🙂 Could you add in a hide value with the reject or pass at my initial review? Published citations would then be NOT rejected AND NOT hidden.
Not sure how much work this would be or if it would have any risks I am not aware of...just an idea I thought I would toss out there.

BZDATETIME::2012-09-03 15:45:49
BZCOMMENTOR::Alan Meyer
BZCOMMENT::190

(In reply to comment #189)

> What if we just "hide" to errors. Cover them up with a blanket and pretend they
> never happened 🙂 Could you add in a hide value with the reject or pass at my
> initial review? Published citations would then be NOT rejected AND NOT hidden.
> Not sure how much work this would be or if it would have any risks I am not
> aware of...just an idea I thought I would toss out there.

I think that's not a bad idea and doesn't necessarily involve big changes to the system. We already have an ability to put status records in two different states, "active" and "inactive". One possibility is to make everything we had regarding an import "inactive". It's also possible to associate a comment with any article state and with any batch import, which makes it easy to indicate what went wrong with both the articles and the import.

My intuition on this is that we should put the issue on the back burner for now since nothing terrible will happen if mistaken imports occur. I'm also hoping that the number of mistakes will be lower since the user can "test" an import before actually doing it.

BZDATETIME::2012-09-04 21:24:33
BZCOMMENTOR::Alan Meyer
BZCOMMENT::191

In all of the work that I've done on printing for board members I
have only thought about the kind of printing we do in order to
mail packets out to them.

However, Bob pointed out that we also print mounds of paper for
use in board meetings. Do we want to take advantage of the
printing facilities I'm working on to do that?

We have no requirements documentation for it.

Can someone work up the requirements for that? We need answers
to questions like:

• What gets printed?

Do the articles have to be in a particular state, e.g.,

"Deserves citation in the summary",
"Merits revision of the text", or
"Merits discussion".

• Which articles or packets are filtered to look for the state
  (if it's state based?)

• How many copies should be printed?

• Are there other requirements?

I'd like to be in any meeting where we discuss this since I have
a good idea about what kinds of printing facilities we've already
designed.

Thanks.

BZDATETIME::2012-09-04 21:40:30
BZCOMMENTOR::Alan Meyer
BZCOMMENT::192

When printing a packet I'm including the following documents:

Summary version stored with the packet.

All article PDFs in the packet.

For each article, a response sheet for recording reviewer
responses.

Do we also need to print reviewer documents? My understanding
from Bob is that these are copies of the summary that have been
edited by a board member reviewer and posted back into the EBMS.

Should they be included in the printed packets?

My guess is that it could be a pain in the neck for someone to
try to read the original summary and also one, two, or more
copies of the summary that are separately revised by other board
members. In addition, many, perhaps most, of the revisions
posted by board members will come in after the packets are mailed

• so if we print the ones we've got it's likely we won't be
  printing a complete set.

Finally, we have no control over the document formats that people
submit. They might or might not be .doc or .docx. They might or
might not use revision markup - which might or might conform to
any conventions we're using.

So I'm wondering if it's a good idea to print these. But if we
should print them, I'll do it.

What should I do?

BZDATETIME::2012-09-06 10:08:08
BZCOMMENTOR::Robin Juthe
BZCOMMENT::193

(In reply to comment #192)
> When printing a packet I'm including the following documents:
> Summary version stored with the packet.
> All article PDFs in the packet.
> For each article, a response sheet for recording reviewer
> responses.
> Do we also need to print reviewer documents? My understanding
> from Bob is that these are copies of the summary that have been
> edited by a board member reviewer and posted back into the EBMS.
> Should they be included in the printed packets?
> My guess is that it could be a pain in the neck for someone to
> try to read the original summary and also one, two, or more
> copies of the summary that are separately revised by other board
> members. In addition, many, perhaps most, of the revisions
> posted by board members will come in after the packets are mailed
> - so if we print the ones we've got it's likely we won't be
> printing a complete set.
> Finally, we have no control over the document formats that people
> submit. They might or might not be .doc or .docx. They might or
> might not use revision markup - which might or might conform to
> any conventions we're using.
> So I'm wondering if it's a good idea to print these. But if we
> should print them, I'll do it.
> What should I do?

No, we do not need to print returned summary documents from other reviewers in the print packets. Just the original summary (or summaries) associated with the packet need to be printed.

I will work on the requirements for agenda packet printing with Victoria.

BZDATETIME::2012-09-27 23:32:04
BZCOMMENTOR::Alan Meyer
BZCOMMENT::194

I propose to add a new state to our state table for marking article citations that have been "published" by Cynthia.

We have a state now called "Passed initial review". It has the misleading description (written by me) of 'Article "published" for board manager review'.

However, passing initial review and being "published" are two separate actions and two separate states since Cynthia passes articles one by one but publishes all at once in a review cycle.

So, unless I hear objections from someone, I'm going to insert a new state in the sequence between "Passed initial review" and "Rejected by Board Manager".

I don't recall whether we have defined a user interface for publishing. I think a very simple interface is all that is required. It's basically just pushing a button, or maybe selecting a board and pushing a button if we need to publish by board. But I'll not worry about that right now.

BZDATETIME::2012-09-28 09:29:12
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::195

(In reply to comment #194)
> I propose to add a new state to our state table for marking article citations
> that have been "published" by Cynthia.
> We have a state now called "Passed initial review". It has the misleading
> description (written by me) of 'Article "published" for board manager review'.
> However, passing initial review and being "published" are two separate actions
> and two separate states since Cynthia passes articles one by one but publishes
> all at once in a review cycle.
> So, unless I hear objections from someone, I'm going to insert a new state in
> the sequence between "Passed initial review" and "Rejected by Board Manager".
> I don't recall whether we have defined a user interface for publishing. I
> think a very simple interface is all that is required. It's basically just
> pushing a button, or maybe selecting a board and pushing a button if we need to
> publish by board. But I'll not worry about that right now.

No objections...this is exactly what we need. And as for the publishing interface it will be pretty simple but we will want to be able to publish by board or by summary topic as well as by review cycle. This would allow us to publish the entire batch at once or break it down by board or topic.

BZDATETIME::2012-09-28 15:40:20
BZCOMMENTOR::Alan Meyer
BZCOMMENT::196

(In reply to comment #195)
...
> No objections...this is exactly what we need.
...

Done. I bumped all of the higher sequenced states up one sequence number and
inserted the new 'Published' state.

The change is in the ebmsdev database, in the ebms.sql database definition
script, and in version control.

BZDATETIME::2012-09-28 15:47:05
BZCOMMENTOR::Bob Kline
BZCOMMENT::197

(In reply to comment #196)

> The change is in the ebmsdev database, ....

Just to make sure, you updated all the rows in the state table, too (not just the control table), right?

BZDATETIME::2012-09-28 15:59:38
BZCOMMENTOR::Alan Meyer
BZCOMMENT::198

(In reply to comment #197)

> Just to make sure, you updated all the rows in the state table, too (not just
> the control table), right?

No. No updates in the state table are required. I didn't change the state IDs, only the sequence numbers. All the old state_ids still refer to the same states they always referred to. The new row has a higher state ID than all of the existing states, but the sequence number is in the right spot and that should be all that counts.

The next time we run the configuration script, the state IDs will be in order too, but nothing should ever depend on that.

BZDATETIME::2012-09-28 16:57:54
BZCOMMENTOR::Bob Kline
BZCOMMENT::199

(In reply to comment #198)
> (In reply to comment #197)
>
> > Just to make sure, you updated all the rows in the state table, too (not just
> > the control table), right?
>
> No. No updates in the state table are required. ....

Got it, thanks.

BZDATETIME::2012-09-30 01:26:30
BZCOMMENTOR::Alan Meyer
BZCOMMENT::200

We have a single line labeled "AUTHOR" in the EBMS search form.
I have implemented author searching using that line as follows:

1. Enter last name + optional space + initials.

For example:

"Meyer AH"
or
"Meyer"

I'm not supporting searching on first names for now.

2. For multiple authors, separate them with semicolons.

For example:

"Kline RM; Meyer AH"

Spacing won't matter. I'll normalize the spacing so that all
of the following are equivalent:

"Kline RM; Meyer AH"
"Kline RM ; Meyer AH"
"Kline RM; Meyer AH"
" Kline RM ;Meyer AH"
etc.

The key things are that there be at least one space between
the last name and initials, and exactly one semicolon between
each pair of separate authors.

Family names with spaces embedded in them won't work. For
example the following Spanish name won't work:

"Allende Gossens S"

To search that it will be necessary to use a wildcard, e.g.,

"Allende%Gossens S"

See point 4 below.

3. Names will be AND'ed together.

In the above example, an article will only be found if both
authors are named for the article.

4. Use percent signs for wildcards.

For example:

"Johns%n AB"

will retrieve:

"Johnson AB"
"Johnston AB"
"Johnsmorgan AB"
etc.

Wild cards can appear anywhere, any number of times, in
either the last name or initials, but be prepared for long
search times if the wild card is at the front of the string.

If no wild card characters are present, an exact match is
required. For example:

"Johnson A" will not match "Johnson AB"

To get a match in that case enter:

"Johnson A%"

However, that will also match "Johnson AC", "Johnson AD",
etc.

5. I'm not currently supporting searching on collective names,
i.e., names like:

"Ministerial Meeting on Population of the Non-Aligned Movement"

If we need that, someone will have to propose a way to tell
when to search for a collective name and when to search for a
last name + initials. Even if there is a good way, I suggest
we not worry about it for now. Let's get it all working
first.

BZDATETIME::2012-10-02 17:04:26
BZCOMMENTOR::Alan Meyer
BZCOMMENT::201

Here are some more decisions I've made about searching:

1. Published date.

Most of the published dates will work as we have designed the
search input screen. Some, like "2009, Spring", won't.
The best we can do with those is for the user to just enter
"2009" and leave the month blank.

Doing more than that requires a change to our search screen
and our internal indexing - which I think we should defer to
Phase N if we think it's a good idea.

2. Review cycle.

I am implementing this as the review cycle in which an
article was imported.

Unless I am mistaken, we have not brought this information in
from the old system in the conversion - where imports are
done in a radically different way. We will only have it for
articles newly imported in the new system.

We could do some work to infer the cycles from the import
dates but I assume that searching for old cycles isn't very
important and the work is not as high priority as other
things we're doing. So again, phase N.

3. Comments.

We have comments stored in two places: comments associated
with an article state (e.g., a comment entered when an
article is initially reviewed, when a board manager reviews
it, etc.) and comments associated with a tag. However we
only have one line for entering a string to search for in
"comments".

I propose to use it as follows:

a. If the user has entered a tag in the search form, I will
search for any comment text in the tag comments for
articles that have that tag. I'll only look in the
comments associated with that specific tag.

This may be a bit tricky and in general I don't like
tricky interfaces (see below on wildcards, but it makes
sense to me that if a user is asking for articles with
the tag 'X' and puts in a comment string, she's looking
for comments pertaining to that tag and not somewhere
else.

b. If no tag is selected, I'll search for comments anywhere,
in any state comment or any tag comment, attached to the
articles that qualify according to any of the other search
criteria.

I'll require the use of explicit wildcards if wildcards are
desired. For example:

"dogs and cats"
Only finds comments for which "dogs and cats" is the
complete comment.

"dogs and cats%"
Finds comments beginning with the string.

"%dogs and cats%"
Finds comments with the phrase anywhere.

"%dogs%cats"
Finds comments with both words anywhere BUT! "dogs"
must precede "cats" in the comment string.

Personally, I prefer interfaces that don't make tricky and
hidden decisions for the user, in this case - use implicit
wild cards in some fields but not others, or in some
positions but not others. I'm implementing wildcards in all
of the free text fields (i.e., not picked from a checklist or
a drop down), all with the same interface - put in the '%'
where you want it.

As with all searches in our database, text is case
insensitive. Searching for "DOGS", "dogs", or "Dogs" all get
the same results.

No one need comment on any of this unless you think I'm doing
something wrong. If you think it's wrong, please speak now or
hold your peace until phase N.

Thanks.

BZDATETIME::2012-10-02 23:39:22
BZCOMMENTOR::Alan Meyer
BZCOMMENT::202

When we search for articles on editorial board reviewer, what
does that mean?

1. The articles for which a particular board member actually
submitted a review?

2. The articles belonging to packets that have been assigned to
a particular board member, regardless of whether he has done
anything with them or not, or even seen them yet?

3. An article with a topic assigned for which this particular
reviewer is registered - whether or not there have ever been
any responses or packets?

This approach, which makes the most sense in some contexts,
may also find articles that were imported, processed and
completed before this reviewer even joined the editorial
board.

4. Something else?

Thanks.

BZDATETIME::2012-10-03 14:21:12
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::203

(In reply to comment #201)
1. Publication Date…this is fine. Most of the time users will be searching just the year anyway. Actually I don’t think I have ever searched a publication date with a year and a month because some journals use spring and some use April. Unless you know for sure the journal uses spring then searching with it would cause you to miss the citation.
2. Review Cycle… Minaxi and I were under the assumption that the review cycle would be brought over to the new system. In the old system when a new citation is imported a review cycle and summary topic are assigned and the date the citation was imported is recorded. If the same citation comes up again the next month, it is imported again but only the new review cycle and summary topic are assigned. The import date does not change and to my knowledge the date the new summary topic and review cycle were added is not record. So without the review cycle we have no way of differentiating the topic added this month from the one added last month. We have about 50 of these citations every month that “overlap” with the previous month. If these did not have the new review cycle then I would not see them again in my que. So the Review Cycle is not just another date stamp, it represents all new data to be reviewed in a given month…new citations and new summary topics added to citations already in the database. If we lose the review cycle for all past data, although this may not affect the new data, our citation statistics for old data will be different as we will be losing data and the means to identify citations using review cycle. If we had a date stamp for new summary topics added in the old system then this would not be an issue.
3. Comments…this seems fine but it would be nice to not have to worry about which word has to come first in the search string. Any chance we could search the entire string for both words?

BZDATETIME::2012-10-03 14:24:33
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::204

(In reply to comment #202)
> When we search for articles on editorial board reviewer, what
> does that mean?
> 1. The articles for which a particular board member actually
> submitted a review?
> 2. The articles belonging to packets that have been assigned to
> a particular board member, regardless of whether he has done
> anything with them or not, or even seen them yet?
> 3. An article with a topic assigned for which this particular
> reviewer is registered - whether or not there have ever been
> any responses or packets?
> This approach, which makes the most sense in some contexts,
> may also find articles that were imported, processed and
> completed before this reviewer even joined the editorial
> board.
> 4. Something else?
Your understanding is correct.

BZDATETIME::2012-10-03 15:14:43
BZCOMMENTOR::Bob Kline
BZCOMMENT::205

(In reply to comment #203)

> 3. Comments…this seems fine but it would be nice to not have to worry about
> which word has to come first in the search string. Any chance we could search
> the entire string for both words?

That would require the selection and deployment of a full-text indexing package, so that would go on the pile of candidate enhancements for a future release.

BZDATETIME::2012-10-04 10:12:33
BZCOMMENTOR::Robin Juthe
BZCOMMENT::206

(In reply to comment #202)
> When we search for articles on editorial board reviewer, what
> does that mean?
> 1. The articles for which a particular board member actually
> submitted a review?
> 2. The articles belonging to packets that have been assigned to
> a particular board member, regardless of whether he has done
> anything with them or not, or even seen them yet?
> 3. An article with a topic assigned for which this particular
> reviewer is registered - whether or not there have ever been
> any responses or packets?
> This approach, which makes the most sense in some contexts,
> may also find articles that were imported, processed and
> completed before this reviewer even joined the editorial
> board.
> 4. Something else?
> Thanks.

I think that searching on a particular Board member should retrieve all articles that the Board member was assigned to review, regardless of whether he/she sent back any responses. So this would be #2, above. Retrieving articles that were assigned to a given topic before that Board member was even on the Board doesn't make sense to me. I think we would just search by topic if that is what we were interested in seeing.

BZDATETIME::2012-10-04 10:17:27
BZCOMMENTOR::Robin Juthe
BZCOMMENT::207

(In reply to comment #201)
I was also thinking that we would carry over review cycles from the old system to the new one. I use the review cycle parameter fairly often when searching (especially for recent articles). Everything else you have about searching in comments 200 & 201 looks good to me.

BZDATETIME::2012-10-04 14:52:26
BZCOMMENTOR::Alan Meyer
BZCOMMENT::208

I've modified the search back end to look for review cycles for both an import event and a topic added during an import event in the new system.

We haven't done anything about converting the old review cycles. We'll have to think about that.

BZDATETIME::2012-10-05 10:14:02
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::209

Question regarding tagging upon import:
When we import citations from a special search request and apply the citation tag "Special Search Request" and a comment, will this tag/comment be applied to all citations imported regardless of whether the new summary topic is added? Specifically we are wondering about the citations that Minaxi and I refer to as "True Duplicates", These are citations already in the database for the exact same summary topics. So there is no new data to add to these citations except the citation tag and comment.

BZDATETIME::2012-10-05 10:32:57
BZCOMMENTOR::Bob Kline
BZCOMMENT::210

(In reply to comment #209)
> Question regarding tagging upon import:
> When we import citations from a special search request and apply the citation
> tag "Special Search Request" and a comment, will this tag/comment be applied to
> all citations imported regardless of whether the new summary topic is added?
> Specifically we are wondering about the citations that Minaxi and I refer to as
> "True Duplicates", These are citations already in the database for the exact
> same summary topics. So there is no new data to add to these citations except
> the citation tag and comment.

If I understand your question, Alan suggested to me earlier that we decouple tag assignment from import, so you should be able to control directly which articles get which tag/comment assignments.

BZDATETIME::2012-10-05 10:49:57
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::211

(In reply to comment #210)
> If I understand your question, Alan suggested to me earlier that we decouple
> tag assignment from import, so you should be able to control directly which
> articles get which tag/comment assignments.

In the new system we will have the ability to assign specific tags to batches of citations upon import. So if I understand your comment above correctly, two separate actions will be happening at that time: (1)importing all citations in the batch and (2)assigning tags to all citations in the batch. Is this correct?

BZDATETIME::2012-10-05 10:54:07
BZCOMMENTOR::Bob Kline
BZCOMMENT::212

Alan had me take the tag picklist off the import screen, because of the problem you raised (that it's frequently inappropriate to assign the same tag to all of the articles imported).

BZDATETIME::2012-10-05 15:24:38
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::213

(In reply to comment #212)
> Alan had me take the tag picklist off the import screen, because of the problem
> you raised (that it's frequently inappropriate to assign the same tag to all of
> the articles imported).

Frequently inappropriate? it is actually exactly what we want to do frequently. We want to be able to assign one of three tags (batch, special search or fastrack) to ALL citations imported at one time....even the "true duplicates" that will not have any other changes made upon the import except the addition of the tag. We were concerned that "true duplicates" might not get the tag added.

BZDATETIME::2012-10-05 15:27:48
BZCOMMENTOR::Bob Kline
BZCOMMENT::214

(In reply to comment #213)
> (In reply to comment #212)
> > Alan had me take the tag picklist off the import screen, because of the problem
> > you raised (that it's frequently inappropriate to assign the same tag to all of
> > the articles imported).
>
> Frequently inappropriate? it is actually exactly what we want to do frequently.
> We want to be able to assign one of three tags (batch, special search or
> fastrack) to ALL citations imported at one time....even the "true duplicates"
> that will not have any other changes made upon the import except the addition
> of the tag. We were concerned that "true duplicates" might not get the tag
> added.

OK, did I misunderstand what you were telling me to do, Alan?

BZDATETIME::2012-10-05 16:35:12
BZCOMMENTOR::Alan Meyer
BZCOMMENT::215

(In reply to comment #214)
> (In reply to comment #213)
> > (In reply to comment #212)
> > > Alan had me take the tag picklist off the import screen,
> > > because of the problem you raised (that it's frequently
> > > inappropriate to assign the same tag to all of the articles
> > > imported).
> >
> > Frequently inappropriate? it is actually exactly what we want
> > to do frequently. We want to be able to assign one of three
> > tags (batch, special search or fastrack) to ALL citations
> > imported at one time....even the "true duplicates" that will
> > not have any other changes made upon the import except the
> > addition of the tag. We were concerned that "true duplicates"
> > might not get the tag added.
>
> OK, did I misunderstand what you were telling me to do, Alan?

No, you didn't misunderstand me, but it's looking likely that I
didn't fully understand what really should be done.

We have provisions in the database for storing comments in three
different ways.

1. The user interface can call the backend import function and
pass a comment to it along with other parameters. This
comment gets stored in the ebms_import_batch table along with
other parameters concerning the batch. The comment is not
stored separately for each article.

2. We have an ability to attach a comment to a state record.
For example, we can attach a comment to the Import state for
an article. These would be article specific, i.e., we could
do it for some articles and not others, or for all articles
in a batch. We could do it with a user interface that told
the import system to do it to all articles at once, or we
could have users enter comments one at a time.

3. We have an ability to attach a tag with a comment to an
article. Tags can be assigned individually to articles or,
again, the import program could do them in a batch for the
whole import - which was what I originally proposed and
thought we would do until recently when I asked you to take
the tag off the import screen because I wasn't sure how it
would be used.

Cynthia's comment #213 gave me a better understanding of how tags
could be used at import time. We didn't have any list of tags
before except the ones I made up for test purposes. Now I see
what the real use cases are for them.

There are a lot of ways to skin this cat and I'm not sure what is
ideal, but here's what I suggest:

1. Let's put tag selection back on the import screen (sorry Bob
for my jerking you around on this. Let's have a place to
pick one tag and add one comment.

The tag and comment would be optional and, for normal imports
that occur as a result of the monthly Pubmed searches, I
think it would be best not to use tags or comments. The lack
of any special tag would mean the articles came in the normal
way.

2. Cynthia should tell us what should be done with the tag and
comment. Should they be added just to newly imported
citations, or to newly imported citations plus those with a
new topic added, or to all of them?

It would seem most logical to add them to newly imported and
to topic added citations, and not to anything else, but
Cynthia knows more than I do about what's the right thing to
do.

We could have something more complicated than that where a
user can enter multiple tags and comments and we apportion
them separately based on import status, but I think that's
adding significant complexity to the system that I'd rather
avoid - and also makes the user's task more complicated too.

We'll need to create a user interface to enter the data -
basically just a tag selection and a box to enter comments,
and a backend API call.

For the back end, rather than add yet another parameter to
the importArticlesFromNLM() function, I'm tempted to add a
new public function to the ImportBatch class. When the UI
gets back an ImportBatch object from an import it can call it
to add the tags and comments to whole categories of records
in the batch, for example, maybe something like:

obj->addTag(array('imported','topicAdded'), tag, comment);

That would give us the flexibility to do whatever we want in
the user interface and would also support, if we ever want
it, the ability to tag batches of records from a single
import after the fact.

3. Let's not use the existing ebms_batch_import.comment column
for user entered comments at all. I suggest we put the name
of the import file in there since that's something that
Cynthia once asked for but I have no other provision for at
the moment.

What do you think?

BZDATETIME::2012-10-05 18:38:57
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::216

(In reply to comment #215)
> (In reply to comment #214)
> > (In reply to comment #213)
> > > (In reply to comment #212)
This is getting a bit more complicated than I think it needs to be. And I think the flexibility of tag use is to blame. And maybe tagging is not the best option for what Minaxi and I want here.
We import sets of citations (set = 1-200 citations at a time). Every set is for a specific summary topic which we assign upon import. Most sets are the results of our monthly searches. Currently all other sets are either:
(1) results from a special search request for a specific topic that we assign up on import or
(2) citations to be fast tracked for a specific topic that we assign upon import.
(Note: maybe in the future there will be other types of sets to be imported...i.e. if we pull citations from other databases or out of thin air)

If we import a set of citations that is not the result of one of our monthly searches we need to label the ENTIRE set to distinguish it as either :
(1) results from a special search request or
(2) citations to be fast tracked for a specific topic.
In these cases it does not matter whether the citation is new, if only the summary topic is added, or if it is a true duplicate, EVERY citation in that set ALWAYS needs to be labeled in some way.

Why do we need this? If the ENTIRE set is not labeled we will not be able to retrieve all of the citations in that set.
Special search requests are often retrospective covering 2-5 years. Many of these citations will already be in the database and many will have the same summary topic. If this is the case, no new data is added to the records for those that are true duplicates and their status does not change.
So how do we retrieve them?
How will board mangers see all the results from the special search request?
We have no way of identifying that they were part of the special search request citation set after we import them.
Currently we do not import the results of special searches because once imported only the new citations would get reviewed. We do not want these citations to necessarily appear in board manager's que, but we would like board managers to be able to retrieve and review these citation as a complete set; reviewing new ones for the first time and then re-reviewing others for the second time.

BZDATETIME::2012-10-05 22:49:45
BZCOMMENTOR::Alan Meyer
BZCOMMENT::217

(In reply to comment #216)

> This is getting a bit more complicated than I think it needs to
> be. And I think the flexibility of tag use is to blame. And
> maybe tagging is not the best option for what Minaxi and I want
> here.
...

Alright, I think I get it.

I think we can support it with the following changes to our
existing design and implementation:

1. Add a column for import_type to the ebms_import_batch table.

Values would come from a table of import batch types.

2. Add a table of import batch types.

Here's where we store the values and the strings that
represent them in the import user interface.

I think the table changes are a couple of hours work
together.

3. Add an import type field to the import user interface form.

This would be a drop down list of the three currently
identified import types with the default being the regular
monthly topic search type. There might be more than three
types in the future. This field could be in place of the
tag.

Instead of having a tag entry field, we'd have this
import type field. We would retain the comment field on the
import user interface form.

When importing data in a "live" import, the value in the
import type field and any optional comment would both be
stored in the import_batch table.

4. Modify the back end API software for imports to accept the
changes.

The changes would be to:

Add the new import_batch_type as a parameter in the call
to import data into the database.

Add the batch type to the EbmsBatch class as another
field.

Add the batch type values to the class objects when
called, and again when called to instantiate an object
from data retrieved from the database.

I'm estimating that this is another couple of hours work.

5. Import reports would include these values.

I don't think these have been written yet so there is no
revision of existing software to do them.

6. We might add them to the search form?

I'm not sure how a user would identify values to search for
here. We'd have to think about whether the goal is, for
example, to only retrieve articles that were fast tracked, or
to only retrieve articles that came in in a specific batch.

My vote would be to kick this can down the road and see if we
can get by with just a report approach.

Our management is putting pretty strong pressure on us to get
this project out the door as fast as possible, so we'll have to
wait and see what they say about adding things at this time vs.
deferring them to the future.

BZDATETIME::2012-10-07 21:06:56
BZCOMMENTOR::Alan Meyer
BZCOMMENT::218

I'm implementing more of the search functionality and have
questions about the meaning of some of the user inputs.

Advanced Options:

I understand some of the checkboxes but not all of them.

FYI Citation:
Implemented to look for a state for any qualified article
of "Flagged as FYI". If an article has that state, and
it's active, it qualifies. Otherwise it doesn't.

As with most of the search criteria, I assume that this
is board and topic specific if any topic(s) have been
specified in the search.

NCI Reviewer Decision:
Not implemented yet.

I'm thinking this means that there is an entry in the
ebms_article_board_decision table for this article. ANY
of the decision values are okay (Cited, Not cited, Text
approved, Text needs to be written or revised, Hold.) In
other words, it's the same as if we picked all of the
Editorial Board Decisions from that drop down box - which
we can't do.

Full Text Retrieved:
Implemented as a check to see if we have the full text.
Doesn't matter who requested it.

Committee Decision:
Not implemented yet.

I don't know what this means. It could mean that either
one of the two states could pertain to an article:

Passed full text review.
Rejected after full text review.

Or maybe it only means "Passed".

I assume that, if an topic(s) has (have) been specified,
then we're only looking at the above decisions with
respect to those topic(s). Otherwise we look at all
topics.

I need some guidance.

Published to CiteMS:
Leaving aside the anachronistic system name, I assume
this means that a Published" state is attached to this
article. Again, topic specific.

Core Journals:
I know what this means, but we can't do it yet. We'll
have to create the software to load and periodically
refresh the core journals list.

There are different ways to do that. One way is to just
do it ourselves, i.e., look at what journals WE have
actually approved articles from more than any others.

Include Not List Journals:
Not implemented yet.

I assume that this means to include journals that have
been rejected because of being on a NOT list, not to ONLY
include such journals. I also assume it's topic
specific. If one or more topics are specified, only
journals imported for those topics are in the not list.

State/topic relationships:

Many of the input boxes will change their scope depending
upon whether a board or topic has been selected. A few, like
full text retrieved, will not.

So much for the advanced option checkboxes. Here are some more
questions.

Comments:
Not implemented yet.

In a previous Bugzilla posting (comment #201) I proposed
an approach to comments that sometimes looked at comments
in either tags or both states and tags, depending on
whether a tag has been selected.

I'm not sure it's best. I now think it's too tricky and
can produce surprising results. I propose instead to not
search tag comments at all unless we add a new line to
the form for tag comments.

I may well have more questions later, but I'll start implementing
the above decisions, working first on the ones that seem clearest
to give people a chance to respond to the others.

With a form this complex and with this many options I think it
will be easy to put in search criteria that get no hits.
Sometimes it won't be obvious what went wrong when you go back to
the search form because the advanced options, topics, and other
sections will be collapsed and you might not realize that there
are checked items inside.

I don't plan to make any attempt to second guess the search
input. If a user puts in an impossible combination of search
criteria the results will just be zero hits. I won't be picking
one criterion and suppressing another to make it get more even if
a good guess is easy to make.

I guess we'll get used to it and make careful searches 🙂

BZDATETIME::2012-10-08 11:03:04
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::219

(In reply to comment #217)
> (In reply to comment #216)
This looks like a good solution.

> 5. Import reports would include these values.
> I don't think these have been written yet so there is no
> revision of existing software to do them.

Yes the import report would need to display the import type, maybe not for the default (monthly search) but it should for special search request or fast track, etc.

> 6. We might add them to the search form?
> I'm not sure how a user would identify values to search for
> here. We'd have to think about whether the goal is, for
> example, to only retrieve articles that were fast tracked, or
> to only retrieve articles that came in in a specific batch.
> My vote would be to kick this can down the road and see if we
> can get by with just a report approach.

For special search requests we could get by with reports as we do only run 20-50 special search requests each year. As long as the report was formatted in a way that allowed users to review or make changes to the citations.
As for fast tracks, usually it is just one or two citations at a time and are easy enough to retrieve by pmid. But if someone has a big batch of fast tracks then again they would need a report formatted in a way that allowed them to review and make changes to the citations.
Although would creating such a report be less time consuming than adding an element to the search the database search to limit the search by import type and then we could use the existing displays to review these citations.

BZDATETIME::2012-10-08 11:07:57
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::220

(In reply to comment #218)
Advanced Options:

I understand some of the checkboxes but not all of them.

FYI Citation:
Implemented to look for a state for any qualified article
of "Flagged as FYI". If an article has that state, and
it's active, it qualifies. Otherwise it doesn't.

As with most of the search criteria, I assume that this
is board and topic specific if any topic(s) have been
specified in the search.

***Look for articles with this state active or not. (I am not exactly sure how an fyi citation would become inactive.) If you limit this search to active then you will also have to limit it to inactive to retrieve all fyi citations sent in the past. There are not that many and these can be limited by mailer date.

NCI Reviewer Decision:
Not implemented yet.

I'm thinking this means that there is an entry in the
ebms_article_board_decision table for this article. ANY
of the decision values are okay (Cited, Not cited, Text
approved, Text needs to be written or revised, Hold.) In
other words, it's the same as if we picked all of the
Editorial Board Decisions from that drop down box - which
we can't do.

***I think you are confusing this with Editorial Board Reviewer Decision. NCI Reviewer Decision is just the initial review of yes or no to get the full text.

Committee Decision:
Not implemented yet.

I don't know what this means. It could mean that either
one of the two states could pertain to an article:

Passed full text review.
Rejected after full text review.

Or maybe it only means "Passed".

I assume that, if an topic(s) has (have) been specified,
then we're only looking at the above decisions with
respect to those topic(s). Otherwise we look at all
topics.

I need some guidance.

***Yes this is a search for Passed any topic, Rejected any topic, Passed for a specific topic, and rejected for a specific topic. Search is usually limited by review cycle, board and/or topic.

Core Journals:
I know what this means, but we can't do it yet. We'll
have to create the software to load and periodically
refresh the core journals list.

There are different ways to do that. One way is to just
do it ourselves, i.e., look at what journals WE have
actually approved articles from more than any others.

***Yes we will have to come up with the lists ourselves and then maintain them. Each board may want a different list. When we are ready to work on this, Minaxi and I will supply lists of relevant journals for the board managers to select from. We will generate these lists from CMS high frequency journals and from NLM’s core biomedical journals list.

Include Not List Journals:
Not implemented yet.

I assume that this means to include journals that have
been rejected because of being on a NOT list, not to ONLY
include such journals. I also assume it's topic
specific. If one or more topics are specified, only
journals imported for those topics are in the not list.

***This search must be limited by board as each not journal list is board specific. The search could then be limited by a summary topic from the selected board. This would include all journals rejected or not. Would we ever want to see articles rejected only?

State/topic relationships:

Many of the input boxes will change their scope depending
upon whether a board or topic has been selected. A few, like
full text retrieved, will not.

So much for the advanced option checkboxes. Here are some more questions.

Comments:
Not implemented yet.

In a previous Bugzilla posting (comment #201) I proposed
an approach to comments that sometimes looked at comments
in either tags or both states and tags, depending on
whether a tag has been selected.

I'm not sure it's best. I now think it's too tricky and
can produce surprising results. I propose instead to not
search tag comments at all unless we add a new line to
the form for tag comments.
***For now searching for the tag alone is more important. Once we start using tags and gain a better understanding of how we will use and search them, then maybe we can revisit searching comments.

BZDATETIME::2012-10-08 23:43:48
BZCOMMENTOR::Alan Meyer
BZCOMMENT::221

Thanks Cynthia. I have followed your advice in all of the search queries.

I implemented the 'Published' checkbox by looking for the 'Published' state in the state table. Of course that doesn't work right now (it always gets 0 hits) because the Published state was just added (see comments #194-196) post conversion, so no articles are showing as Published.

We'll have to change the conversion process to add that state. I presume we'll want to mark everything that has a "Passed board manager review" or "Rejected at board manager review" as "Published".

BZDATETIME::2012-10-09 09:34:56
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::222

(In reply to comment #221)
> We'll have to change the conversion process to add that state. I presume we'll
> want to mark everything that has a "Passed board manager review" or "Rejected
> at board manager review" as "Published".

The published state should be added to any citation that has "passed medical librarian initial review". If you mark everything that has a "Passed board manager review" or "Rejected at board manager review" as "Published", any citation that is sitting in the board managers que that has not yet been reviewed will not get the published state.
If at the time of conversion, all board managers have empty ques then this will not be an issue.

BZDATETIME::2012-10-18 11:47:10
BZCOMMENTOR::Alan Meyer
BZCOMMENT::223

There are three states in the article state table that are not
tied in any way to a topic or even a board. They are the ones
regarding full text retrieval, namely:

Requested full text
Full text retrieval failed
Full text retrieved

Bob, Dan and I have discussed this issue in the light of the fact
that a number of difficult operations are simplified by requiring
a topic ID for all states.

We are now thinking that we can dispense with these three states
and treat them differently as follows:

If an article passes board manager review and full text
retrieval has never been attempted, we will automatically
generate a request to Bonnie.

If Bonnie retrieves the full text, she'll enter it into the
system and we'll know that it's there.

If the full text is retrieved, the program that constructs
the queue for CIPS reviewers will see that and put the
article in their queue display. If it has not yet been
retrieved it will not put it in their queue display.

If the full text cannot be retrieved, Bonnie will have to
enter something to indicate that there was a failure. The
article will not appear in a reviewer queue.

The main difference from a user's point of view is that a board
manager will never request full text. The request will be
generated automatically the first time an article passes board
manager review. I think all of the other differences are
internal.

There are some internal issues to resolve, but I don't think they
affect users.

Does anyone see a problem with that approach?

BZDATETIME::2012-10-18 12:30:35
BZCOMMENTOR::Robin Juthe
BZCOMMENT::224

(In reply to comment #223)
> Does anyone see a problem with that approach?

This seems fine to me.

BZDATETIME::2012-10-18 14:43:16
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::225

(In reply to comment #224)
> (In reply to comment #223)
> > Does anyone see a problem with that approach?
> This seems fine to me.

Me too!

BZDATETIME::2012-10-18 19:14:45
BZCOMMENTOR::Alan Meyer
BZCOMMENT::226

One of the decisions we made at the CDR status meeting today was that a descriptive tag assigned to an article could be associated with an optional topic_id. We need the ability to enter a tag in which no board or topic is selected, or one in which a topic is selected (thereby implying a board), but we do not need to have tags that have board ids with no topic id.

I had originally implemented the internals of the system the other way around, to support a board_id with no topic_id. I don't remember why I did that and haven't found any notes about it.

In light of the discussion today I am going to change that. I will allow the system to store a tag with no board or topic, or a tag with a topic (and an implied board), but not with a board but no topic.

If we ever need to change that, let's do so in Phase 2.

BZDATETIME::2012-10-18 23:19:06
BZCOMMENTOR::Alan Meyer
BZCOMMENT::227

(In reply to comment #226)

In order to redo the tag table to store topics instead of boards, I plan to delete all of the existing tag data (tag definitions, tag rows and tag comment rows) from the database.

I think that every one of the existing entries was created by me or Veena purely for test purposes. None of them are real.

I would normally check to be sure this is okay with everyone, but I would like to get it done tonight in order to clear the way for a task that Dan is working on that involves tags and I can't think of any reason not to proceed.

When I recreate the data I'll add two new tag types:

Import special search
and
Import fast track

We can change the names later if desired.

BZDATETIME::2012-10-18 23:48:45
BZCOMMENTOR::Alan Meyer
BZCOMMENT::228

(In reply to comment #227)

Done.

I've dropped the three tables and recreated them with the new versions substituting topic for board, and with the two new tag types.

Implementation notes for Bob and Dan:

I made the changes to ebms.sql and tested them by using the modified SQL from that script to recreate the tables after I dropped them from the database.

I decided not to retain a board_id in the ebms_article_tag table because: 1) It would be a denormalization. 2) It's not hard to find the articles by board, or to create a view that does it for us. 3) I don't know yet how often we'll want to search by board or, if we do, whether there will be a performance problem, so we can add the column later if and only if experience shows a performance problem.

I have modified the addArticleTag() function to use topics instead of boards. I've also modified the getTagHistory() function with a straightforward substitution of topic for board. I'll also modify the API functions to use the new versions of everything so I can test.

BZDATETIME::2012-10-19 09:46:53
BZCOMMENTOR::Bob Kline
BZCOMMENT::229

(In reply to comment #228)

> I decided not to retain a board_id in the ebms_article_tag table because: 1) It
> would be a denormalization.

The only case in which that would not be true would be one in which a decision is made at some point to transfer responsibility for a topic (or group of topics) from one board to another. In such a case our statistical reporting would inappropriately credit one board with work actually performed by another. Not a very likely situation, and there's less real impact along these lines in the area of tagging, which a less obviously connected with "work performed" than (say) reviews, where we avoided such denormalization from the start.

BZDATETIME::2012-10-23 13:25:57
BZCOMMENTOR::Alan Meyer
BZCOMMENT::230

In the previous comment, Bob pointed out that a summary topic may have been moved from one board to another. This has presumably happened in the past and may happen again in the future.

This has implications for searching. The way we pick a topic for searching is to first pick a board, then pick a topic from the list of topics for that board. If I search for matches on both topic and board that will automatically exclude all articles for which the topic was once associated with a different board.

I propose to deal with this problem as follows:

If a topic is specified in a search, then I will search for that topic and ignore the board. Even if the topic was once assigned to a different board, I'll still find it.

If a board is specified but no topic is specified, then I will search for the board. In that case I may find old articles that once belonged to the board but their topics have since been moved.

There are old cases where all of this might lead to puzzling results, but if a user tries to figure out what happened, I think he or she will see why it came out the way it did and, importantly, there won't be any articles that cannot be found by topic.

BZDATETIME::2012-10-23 17:09:25
BZCOMMENTOR::Robin Juthe
BZCOMMENT::231

(In reply to comment #230)

This approach for searching sounds good to me.

BZDATETIME::2012-10-23 23:05:19
BZCOMMENTOR::Alan Meyer
BZCOMMENT::232

Robin and I met briefly to discuss a few more search topics. This Bugzilla comment documents what we said.

In order to make rapid progress and meet the schedule, Bob has implemented some simplifications of the search user interface screen. The principal difference that I have noticed is that the Yes / No radio buttons in the ADVANCED SEARCH section are now single buttons.

The difference between having two buttons and having one is that, with two buttons, a user can select articles having the characteristic, articles having the opposite characteristic, or articles for which we don't care (neither Yes nor No was checked.) In the form as implemented I will treat the check box similar to the "Yes" value of a two button form and treat the absence of a check as meaning "don't care". If we need to change that in the future we can, but I'll implement what we can now and we can go back later.

Another point we discussed was the meaning of a check in terms of current states. If something is checked, e.g. NCI REVIEWER DECISION, I will interpret that to mean that the article has an active state of Passed board manager review. It need not be stopped there waiting to go on. It may have gone a lot further in the process, but it at least has once been in that state. So, in most cases, selecting NCI REVIEWER DECISION will also find everything selected by COMMITTEE DECISION, plus many others that passed board manager review but either failed at a later step or haven't yet been evaluated for later steps.

There is one exception to what I have just said. If an article is fast tracked it might have passed COMMITTEE DECISION without ever having the passed NCI REVIEWER DECISION state (Passed board manager review).

We can change that too if desired so that any article that has Passed board manager review or any higher sequenced state will be retrieved for NCI REVIEWER DECISION. But I don't know that many articles would be affected and I propose to defer implementing that unless and until we decide we need it.

Finally, Robin made clear that the INCLUDE REJECTED ARTICLES button only refers to articles that were rejected before publishing. There are some issues with that because, in the newly imported data we can tell the difference between a rejected article for which is still currently rejected and one for which someone changed their mind. But it's not so easy to tell that in the legacy data. I'll bring up the issues for that in another posting later.

BZDATETIME::2012-10-25 16:15:07
BZCOMMENTOR::Alan Meyer
BZCOMMENT::233

There are two input selection items on the search form for:

EDITORIAL BOARD REVIEWER
and
EDITORIAL BOARD RESPONSE

I'm planning to treat these independently. If a user selects:

User = Joe Blow
Response = Merits revision of the text

I will require that all matching articles have Joe Blow listed as
a reviewer and that someone said the article merits revision of
the text. It doesn't have to be Joe.

My reasons for this approach are:

1. It's easier to do (never a bad thing.)

2. It's more flexible (Hmmm, I seem to remember Joe reviewed
it and some reviewer said it merits revision of the text.)

3. It errs on the side of retrieving too much rather than too
little. Better to see an extra article or two you didn't
want than to miss the one you did.

BZDATETIME::2012-10-25 19:29:05
BZCOMMENTOR::Alan Meyer
BZCOMMENT::234

In my ongoing monolog on more than you really want to know about
searching, here's how I am implementing the comment, comment
date, tag, and tag date searching:

1. When to search for comments and dates.

If a comment is entered, I will examine the comment date
fields. Otherwise I'll ignore the comment dates.

This means that a user can search to see if a comment with
some particular content was entered, and can restrict that
search to comments entered within a particular date range.

However a user will NOT be able to just enter a date or date
range and find all records for which some comment had been
entered within the date scope, regardless of what the comment
was.

It might be possible for a user to force the system to do
that by simply entering '%' as the comment search, but I'm
not sure that the results would justify the effort made by
all of those whizzing electrons.

2. Date values.

A user must enter dates as year + optional month + optional
day.

If no value for year is entered, month and day will be
ignored. If no value for month is entered, day will be
ignored.

If there is a year but no month or day, or a year + month but
no day, an implicit range search will be performed. For
example:

Look for comments entered any time in 2012:
[2012] [ ] [ ]

Look for comments entered during November of 2012:
[2012] [Nov] [ ]

Look for comments entered November 8, 2012:
[2012] [Nov] [ 8]

3. Explicit date ranges.

If the RANGE checkbox has been checked, the program will look
at values in the three date fields for the end of the range.

If there are no values in the fields, no range search will be
performed.

If there are values, the range search will look at values up
to, but not including the specified dates. For example:

Find comments entered any time from Jan 1, 2012 through
Dec 31, 2013.
[2012] [ ] [ ]
[2014] [ ] [ ]

Find comments entered any time from Feb 1, 2012 through
Jun 30, 2012
[2012] [Feb] [ ]
[2014] [Jul] [ ]

Find comments entered any time from Feb 10, 2012 through
Feb 11, 2012
[2012] [Feb] [10]
[2014] [Feb] [12]

Again, a user must enter dates in order. A date like the
following will be ignored:

[ ] [Feb] [12]

Starting date MUST be entered. For example:

Find comments entered before Feb 12, 2014.
The "1900" start year in the example is the earliest one
in the drop down list but any date before the first
comment was created will do.
[1900] [ ] [ ]
[2014] [Feb] [12]

But this will be ignored:
[ ] [ ] [ ]
[2014] [Feb] [12]

These conventions are aribtrary. I don't claim they're better
than other conventions. But they look as reasonable and easy to
understand as any others.

4. Tags and tag dates.

The rules here are analogous to those for comments and
comment dates, i.e.,

If no tag is selected, dates are ignored.

Dates and date ranges are interpreted exactly the same as
for comment dates.

BZDATETIME::2012-10-25 20:26:00
BZCOMMENTOR::Alan Meyer
BZCOMMENT::235

What is the "Modified date" in the administrator search?

Is it:

1. The date of the last state change recorded for the article?

2. The date of any state change recorded for the article?

3. Something else?

I presume the date is limited by board and/or topic if those were
selected.

Unless I hear otherwise (fairly soon), I plan to do the
following:

1. Search for articles that have ANY state change (not just the
last one) on the requested date or within the specified date
range.

2. Only look at the state table. I won't look at anything
having to do with tags, packet inclusion or anything else for
which we don't record a state change.

3. Limit by board or topic, as for other state searches. This
means that if a topic is specified I'll ignore the board.
See comment #230.

4. Handle dates and date ranges exactly as for state comments
and tags.
See comment #234.

BZDATETIME::2012-10-26 09:40:15
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::236

(In reply to comment #235)
> What is the "Modified date" in the administrator search?
> Is it:
> 1. The date of the last state change recorded for the article?
> 2. The date of any state change recorded for the article?
> 3. Something else?

Modified date search is a search for any citation that has been modified in any way on a specified date or date range. Modified could be a new sumtop, new decision, change of state, new tag, new comment, edit to a comment, etc...pretty much anything you can do to a citation record would count.

> I presume the date is limited by board and/or topic if those were
> selected.

No, it is usually limited to a specific user or just the date.

> Unless I hear otherwise (fairly soon), I plan to do the
> following:
> 1. Search for articles that have ANY state change (not just the
> last one) on the requested date or within the specified date
> range.
> 2. Only look at the state table. I won't look at anything
> having to do with tags, packet inclusion or anything else for
> which we don't record a state change.

No, this should not just be limited to state changes.

> 3. Limit by board or topic, as for other state searches. This
> means that if a topic is specified I'll ignore the board.
> See comment #230.
> 4. Handle dates and date ranges exactly as for state comments
> and tags.
> See comment #234.

BZDATETIME::2012-10-26 09:53:21
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::237

(In reply to comment #234)
> In my ongoing monolog on more than you really want to know about
> searching, here's how I am implementing the comment, comment
> date, tag, and tag date searching:
> 1. When to search for comments and dates.
> If a comment is entered, I will examine the comment date
> fields. Otherwise I'll ignore the comment dates.
> This means that a user can search to see if a comment with
> some particular content was entered, and can restrict that
> search to comments entered within a particular date range.
> However a user will NOT be able to just enter a date or date
> range and find all records for which some comment had been
> entered within the date scope, regardless of what the comment
> was.

We would actually like to do this. Minaxi currently selects a date range and reviews all comments made during that range as a way to collect feedback...if board managers make comments during their review and in particular if they make a note to request a journal be added to the not list or that specific types of journal articles be excluded.

> It might be possible for a user to force the system to do
> that by simply entering '%' as the comment search, but I'm
> not sure that the results would justify the effort made by
> all of those whizzing electrons.

What if we limited the whizzing electrons to comments made at specific steps in the review process...Comments linked to specific decisions?

> 2. Date values.
> A user must enter dates as year + optional month + optional
> day.
> If no value for year is entered, month and day will be
> ignored. If no value for month is entered, day will be
> ignored.
> If there is a year but no month or day, or a year + month but
> no day, an implicit range search will be performed. For
> example:
> Look for comments entered any time in 2012:
> [2012] [ ] [ ]
> Look for comments entered during November of 2012:
> [2012] [Nov] [ ]
> Look for comments entered November 8, 2012:
> [2012] [Nov] [ 8]
> 3. Explicit date ranges.
> If the RANGE checkbox has been checked, the program will look
> at values in the three date fields for the end of the range.
> If there are no values in the fields, no range search will be
> performed.
> If there are values, the range search will look at values up
> to, but not including the specified dates. For example:
> Find comments entered any time from Jan 1, 2012 through
> Dec 31, 2013.
> [2012] [ ] [ ]
> [2014] [ ] [ ]

But if this search is run on Feb 2014 then you would retrieve more than the specific range.

> Find comments entered any time from Feb 1, 2012 through
> Jun 30, 2012
> [2012] [Feb] [ ]
> [2014] [Jul] [ ]

Why are you not using [2012] [jul] [ ]?

> Find comments entered any time from Feb 10, 2012 through
> Feb 11, 2012
> [2012] [Feb] [10]
> [2014] [Feb] [12]

And again why not [2012] [feb] [11]?

> Again, a user must enter dates in order. A date like the
> following will be ignored:
> [ ] [Feb] [12]
> Starting date MUST be entered. For example:
> Find comments entered before Feb 12, 2014.
> The "1900" start year in the example is the earliest one
> in the drop down list but any date before the first
> comment was created will do.
> [1900] [ ] [ ]
> [2014] [Feb] [12]
> But this will be ignored:
> [ ] [ ] [ ]
> [2014] [Feb] [12]
> These conventions are aribtrary. I don't claim they're better
> than other conventions. But they look as reasonable and easy to
> understand as any others.
> 4. Tags and tag dates.
> The rules here are analogous to those for comments and
> comment dates, i.e.,
> If no tag is selected, dates are ignored.
> Dates and date ranges are interpreted exactly the same as
> for comment dates.

BZDATETIME::2012-10-31 13:31:45
BZCOMMENTOR::Dan Young
BZCOMMENT::238

As per my recollection and notes regarding a discussion of the Literature Surveillance Committee Decision, I'll be moving the FYI state into sequence number 9, alongside the existing 'Passed full text review' and 'Rejected after full text review' states. This also implies that only one of the states can be active for a given article and topic, which the design for the Full Citation page seems to support.

Please let me know if this raises any issues.

BZDATETIME::2012-10-31 19:47:44
BZCOMMENTOR::Alan Meyer
BZCOMMENT::239

(In reply to comment #236 and 237)

I'm working on these and will try to implement everything in accordance with Cynthia's replies to my posting.

If I run into problems I'll report them.

BZDATETIME::2012-11-01 10:37:14
BZCOMMENTOR::Bob Kline
BZCOMMENT::240

I'm working on the EBMS librarians' "Citation Management" reports. Two of the reports in the legacy system are "Citations Imported from Text Files" and "Citations Originally Imported from Text Files." Can someone explain the difference between these two?

Also, there is a report called "Citations Not Selected for Full Text Retrieval." Would this be the same as articles rejected in initial NCI review?

BZDATETIME::2012-11-01 11:20:28
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::241

(In reply to comment #240)
> I'm working on the EBMS librarians' "Citation Management" reports. Two of the
> reports in the legacy system are "Citations Imported from Text Files" and
> "Citations Originally Imported from Text Files." Can someone explain the
> difference between these two?
"Citations Originally Imported from Text Files" is the number of citations imported from the monthly searches only.
"Citations Imported from Text Files" is the number of citations imported from the monthly searches plus any for special search requests or fastracks.
Because the new system will allow us to better identify citations that are monthly, special or fastracked, we will not need the "Citations Originally Imported from Text Files" report. And we can rename the "Citations Imported from Text Files" report as just "Citations Imported".

> Also, there is a report called "Citations Not Selected for Full Text
> Retrieval." Would this be the same as articles rejected in initial NCI review?
Yes, we like to review what has been published but not selected for further review.

BZDATETIME::2012-11-01 11:32:32
BZCOMMENTOR::Bob Kline
BZCOMMENT::242

Excellent! Thanks, Cynthia.

BZDATETIME::2012-11-01 17:27:24
BZCOMMENTOR::Alan Meyer
BZCOMMENT::243

Bob, Dan and I had a discussion about the "FYI" and we had
questions about it. Robin was out today but Victoria was here so
we asked some questions and came to some provisional conclusions
about how to handle FYI.

The existing approaches are as follows:

In the old CiteMS:

FYI is a terminal state (i.e., no further processing will be
done on the article) that is board but not topic specific.
It functions the same as a board manager rejection of an
article for all topics.

In the developing EBMS:

FYI is a terminal state but is board AND topic specific. It
functions exactly the same as the Rejected by Board Manager
state. The only difference is that it implies to a human
that a PDF of the article was somehow sent to one or more
board members, but the method of sending, the date and the
recipient IDs are unknown to the EBMS system - except for
what might be recorded in a free text state comment.

In the developing EBMS software, an article marked FYI for a
particular board and topic can continue through the process
for a different topic as well as for a different board. A
board manager can also change her mind and Pass the article,
causing the FYI state to be inactivated. This is a little
misleading since the article may indeed have already been
sent out FYI to a board member but our record of it will be
inactivated (though not lost.)

Our proposal for the EBMS:

We propose to eliminate FYI as a state. Instead, we propose
to make it a tag. As before, the EBMS software won't know
anything about who might have received the article, but that
information can be recorded in tag comments.

The advantages of this change are:

1. It's more flexible.

FYI can be assigned at any stage of the review process.
An article does not have to be rejected by a board
manager before full text review, or even rejected at all,
in order to be tagged as FYI.

A later operation on the article that would have
inactivated the FYI state, for example reconsideration by
a board manager, will not inactivate the tag that marks
the article as having been sent out FYI. It will only
inactivate the Rejected by Board Manager state.

2. It eliminates having two states that mean the exact same
thing.

A report, for example, of what articles have been
rejected during a board manager review does not have to
look at two different states to find out.

3. Anything that was supported by having an FYI state is
still supported by having an FYI tag.

We can search for FYI articles, display a report of them,
and display the tag and comments in a full citation
display.

Is this a good approach?

BZDATETIME::2012-11-01 18:15:08
BZCOMMENTOR::Robin Juthe
BZCOMMENT::244

(In reply to comment #243)
> Is this a good approach?

I think the approach to use a tag for FYIs could work, but I have a question and a (related) potential drawback:

I would like to still have the option to choose yes, no, or FYI at the full-text review step. Is this possible, or would we have to choose "no" to have any articles we want to flag as "FYI" leave our queue, and then add the "FYI" tag?

Depending on the answer to my question, the potential drawback has to do with searching/reports. I'm thinking that a search for all articles that were rejected by the Board manager at the full-text step could be combined with articles that were given a "no" followed by an FYI tag. These should be distinct.

BZDATETIME::2012-11-01 22:22:31
BZCOMMENTOR::Alan Meyer
BZCOMMENT::245

(In reply to comment #244)
...
> I think the approach to use a tag for FYIs could work, but I
> have a question and a (related) potential drawback:
>
> I would like to still have the option to choose yes, no, or FYI
> at the full-text review step. Is this possible, or would we
> have to choose "no" to have any articles we want to flag as
> "FYI" leave our queue, and then add the "FYI" tag?

I presume what you're asking for here is that an FYI button still
be in the user interface where it was designed to be, but it
would:

Set the article state to "Rejected by Board Manager".
Add the FYI tag to the article.

That's certainly possible but there are some complications. One
is that we'd have to decide what to do with comments. I'm having
trouble finding it in the PDFs for the screen design, but I
presume there is a single place for a comment. The new approach
could logically have two comments - one for the state and one for
the tag. We could put the same comment in both places or in one
only and have the user not click that button but instead
separately click the Rejected and Tag buttons to enter two
different comments.

Dan is the right person to say more about what the issues are
regarding the user interface so I'll defer to him about what
level of effort might be involved.

> Depending on the answer to my question, the potential drawback
> has to do with searching/reports. I'm thinking that a search
> for all articles that were rejected by the Board manager at the
> full-text step could be combined with articles that were given
> a "no" followed by an FYI tag. These should be distinct.

Searching, as currently implemented, won't do what you want.

We can search for articles by any tag. That's implemented.

Searching for articles having the FYI state is also
implemented, but that capability would have to go away if we
make the change because the FYI state won't exist any more.

Searching for articles that have been rejected by a board
manager or after a full text review are unimplemented. The
user interface screen that we have, and the software behind
it, treat a check in the box to mean that the article passed
the review. We have no way to input a failed review and no
implementation of software to look for it.

We also have no way to OR any states. All search criteria
entered by a user are AND'ed together except topics. So even
if we could search for articles that were rejected at the
abstract step or at the full text step, we wouldn't have a
way to get both sets in one search.

Reports are a different story. It would be possible to get all
of the information you want without modifying any other software
since each report is effectively free standing and doesn't have
to interact with other reports the way search criteria do.
However, adding another report does add to the schedule, but if
it replaces a report that is currently defined but not yet
implemented (I don't think many reports have been implemented
yet), the effect on level of effort may be small.

Many moons ago I proposed a search box with a drop down list of
all states in the system, like the drop down for all tags in the
system. A user would select any desired state, set the date
range if desired, and the system would find articles that have
that state, or perhaps that have that state as the "current"
state (maybe with a checkbox for that?). In some ways it would
be more powerful than what we have in the existing design, and it
might be simpler to implement - though it might be less powerful
in other ways. I'm not sure.

If we had that it would solve the problem of searching for any
state but would still not solve the problem of OR'ing two states
together.

Overall, I think we have the following choices:

1. Keep FYI as a state. Proceed as we have been.

Impact on the schedule - don't know that there is any.

You would be able to search for the FYI state, but that state
would be in only one place in the workflow sequence - at the
same sequence level as Passed/Rejected by Board Manager.
There would still be no search for articles that are Rejected
by Board Manager OR FYI.

2. Change FYI to a tag but use the existing methods for setting
states and tags. No new software.

Impact on the schedule - shouldn't cost more.

This makes the combination of rejecting an article at the
abstract review stage and creating an FYI two operations
instead of one, but does provide more flexibility in
assigning FYI to articles (doesn't have to be done with a
rejection, or not with a rejection at just that point), and
does, by at least one interpretation, make the rejection
states more consistent.

3. Change FYI to a tag but add more support for processing it as
if it were a state and adding other new support.

Impact on the schedule - some. Don't know how much.

It would add to the schedule just to do the designs that
would enable us to estimate the cost. I don't see how it
could be less than several days of effort. It might be
significantly more if we also implement searching for all of
the negative states, and more still if we OR states together
in searching.

Personally, I like option two, but I'm not a user and I might
feel differently if I were using the system every day.

I've already spent a couple of hours on this issue.
Unfortunately, in almost every system design we learn things
along the way that can be schedule busters if the schedule doesn't
include time for unanticipated problems.

BZDATETIME::2012-11-01 22:38:30
BZCOMMENTOR::Bob Kline
BZCOMMENT::246

(In reply to comment #245)

> Searching for articles that have been rejected by a board
> manager or after a full text review are unimplemented.

We do have a report for articles rejected by the board manager in her initial review, though.

BZDATETIME::2012-11-01 22:56:04
BZCOMMENTOR::Alan Meyer
BZCOMMENT::247

(In reply to comment #236)

> > I presume the date is limited by board and/or topic if those were
> > selected.
>
> No, it is usually limited to a specific user or just the date.

I went ahead and implemented the board and topic limits anyway, but it still
does exactly what you want.

If the user enters a board or topic, they are used to limit the search.

If the user doesn't enter a board or topic, they don't.

So it's easy to search without regard for board and topic. Just don't select
any.

However, we don't currently have user limits, just date limits. We have no
place on the search form to specify a user, not even a checkbox to say "me". I
haven't implemented any software to limit by user.

BZDATETIME::2012-11-01 23:15:18
BZCOMMENTOR::Alan Meyer
BZCOMMENT::248

Some of the searches run from the search page can retrieve huge numbers of hits and take many minutes to complete. I think some can even timeout the webserver and cause the request to be aborted.

As an experiment, I put a retrieval limit on all searches of 200 hits. As a result, one search that did not complete in several minutes completed in 2 seconds, getting the first 200 hits. Setting it at 500 completed in about 3 seconds.

I'm not sure why this is happening. It may be some new indexes in the right places would solve the problem, or it may be that my search strategies are wrong in one or more key places. Or it may be that a limit is really a good idea. It will require analysis to find out.

In the meantime, I've set a limit of 500 hits. It would be rare for anyone to want to see more than 500 hits so the main problem for users will be that a search will report an inaccurate hit count (500) when in fact there were more hits. Getting exactly 500 will be the clue that this is probably what happened.

It will be easy to change the limiting number or remove the limit later if desired.

BZDATETIME::2012-11-01 23:35:50
BZCOMMENTOR::Alan Meyer
BZCOMMENT::249

(In reply to comment #248)

> Setting it at 500 completed in about 3 seconds.

Some searches won't complete even with the limit. For example, if a user searches for all articles ever assigned to the adult treatment board, we can limit it to 500 but it still takes forever. The Sort Order requirement causes it to try to sort everything to get the particular 500 that best meet the sort order - which takes too long.

We can solve that problem by adding an option to the SORT BY display option which is unsorted. It might be just the thing to use on searches that would otherwise take forever. But on the other hand does anyone want to see a random selection of 500 hits?

There will be times that a user should just click the X to cancel a request and do a different, more practical, search.

BZDATETIME::2012-11-02 09:47:13
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::250

(In reply to comment #249)
> (In reply to comment #248)
> > Setting it at 500 completed in about 3 seconds.
> Some searches won't complete even with the limit. For example, if a user
> searches for all articles ever assigned to the adult treatment board, we can
> limit it to 500 but it still takes forever. The Sort Order requirement causes
> it to try to sort everything to get the particular 500 that best meet the sort
> order - which takes too long.
> We can solve that problem by adding an option to the SORT BY display option
> which is unsorted. It might be just the thing to use on searches that would
> otherwise take forever. But on the other hand does anyone want to see a
> random selection of 500 hits?
> There will be times that a user should just click the X to cancel a request and
> do a different, more practical, search.

Minaxi and I run searches in the CMS that generate well over 200 or even 500 hits. We are not interested in actually seeing all of them; we just want to know the number of citations retrieved. I do not have a problem with a cut off to prevent time outs but I want to see the total retrieved at least.
And as for sorting...the default should be by CMSID. CMSID is pretty much the same as sorting by publication date or review cycle...entry date into the CMS.

BZDATETIME::2012-11-02 10:00:06
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::251

(In reply to comment #247)
> (In reply to comment #236)
> > > I presume the date is limited by board and/or topic if those were
> > > selected.
> >
> > No, it is usually limited to a specific user or just the date.
> I went ahead and implemented the board and topic limits anyway, but it still
> does exactly what you want.
> If the user enters a board or topic, they are used to limit the search.
> If the user doesn't enter a board or topic, they don't.
> So it's easy to search without regard for board and topic. Just don't select
> any.
> However, we don't currently have user limits, just date limits. We have no
> place on the search form to specify a user, not even a checkbox to say "me". I
> haven't implemented any software to limit by user.

I think this will be ok for now but eventually it would be nice to limit by user although I may be the only person who might use this. In the current system I use this to check how many citations I have reviewed each day. In the new system I will have a que that will reduce each day as I review but any modifications made to citation not on my que will be harder to count. I'm big on accountability and this feature, if there is a limit for user, is a good way to measure it.

BZDATETIME::2012-11-02 10:23:19
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::252

(In reply to comment #245)
> (In reply to comment #244)
> ...
> > I think the approach to use a tag for FYIs could work, but I
> > have a question and a (related) potential drawback:
> >
> > I would like to still have the option to choose yes, no, or FYI
> > at the full-text review step. Is this possible, or would we
> > have to choose "no" to have any articles we want to flag as
> > "FYI" leave our queue, and then add the "FYI" tag?
> I presume what you're asking for here is that an FYI button still
> be in the user interface where it was designed to be, but it
> would:
> Set the article state to "Rejected by Board Manager".
> Add the FYI tag to the article.
> That's certainly possible but there are some complications. One
> is that we'd have to decide what to do with comments. I'm having
> trouble finding it in the PDFs for the screen design, but I
> presume there is a single place for a comment. The new approach
> could logically have two comments - one for the state and one for
> the tag. We could put the same comment in both places or in one
> only and have the user not click that button but instead
> separately click the Rejected and Tag buttons to enter two
> different comments.
> Dan is the right person to say more about what the issues are
> regarding the user interface so I'll defer to him about what
> level of effort might be involved.
> > Depending on the answer to my question, the potential drawback
> > has to do with searching/reports. I'm thinking that a search
> > for all articles that were rejected by the Board manager at the
> > full-text step could be combined with articles that were given
> > a "no" followed by an FYI tag. These should be distinct.
> Searching, as currently implemented, won't do what you want.
> We can search for articles by any tag. That's implemented.
> Searching for articles having the FYI state is also
> implemented, but that capability would have to go away if we
> make the change because the FYI state won't exist any more.
> Searching for articles that have been rejected by a board
> manager or after a full text review are unimplemented. The
> user interface screen that we have, and the software behind
> it, treat a check in the box to mean that the article passed
> the review. We have no way to input a failed review and no
> implementation of software to look for it.
> We also have no way to OR any states. All search criteria
> entered by a user are AND'ed together except topics. So even
> if we could search for articles that were rejected at the
> abstract step or at the full text step, we wouldn't have a
> way to get both sets in one search.
> Reports are a different story. It would be possible to get all
> of the information you want without modifying any other software
> since each report is effectively free standing and doesn't have
> to interact with other reports the way search criteria do.
> However, adding another report does add to the schedule, but if
> it replaces a report that is currently defined but not yet
> implemented (I don't think many reports have been implemented
> yet), the effect on level of effort may be small.
> Many moons ago I proposed a search box with a drop down list of
> all states in the system, like the drop down for all tags in the
> system. A user would select any desired state, set the date
> range if desired, and the system would find articles that have
> that state, or perhaps that have that state as the "current"
> state (maybe with a checkbox for that?). In some ways it would
> be more powerful than what we have in the existing design, and it
> might be simpler to implement - though it might be less powerful
> in other ways. I'm not sure.
> If we had that it would solve the problem of searching for any
> state but would still not solve the problem of OR'ing two states
> together.
> Overall, I think we have the following choices:
> 1. Keep FYI as a state. Proceed as we have been.
> Impact on the schedule - don't know that there is any.
> You would be able to search for the FYI state, but that state
> would be in only one place in the workflow sequence - at the
> same sequence level as Passed/Rejected by Board Manager.
> There would still be no search for articles that are Rejected
> by Board Manager OR FYI.
> 2. Change FYI to a tag but use the existing methods for setting
> states and tags. No new software.
> Impact on the schedule - shouldn't cost more.
> This makes the combination of rejecting an article at the
> abstract review stage and creating an FYI two operations
> instead of one, but does provide more flexibility in
> assigning FYI to articles (doesn't have to be done with a
> rejection, or not with a rejection at just that point), and
> does, by at least one interpretation, make the rejection
> states more consistent.
> 3. Change FYI to a tag but add more support for processing it as
> if it were a state and adding other new support.
> Impact on the schedule - some. Don't know how much.
> It would add to the schedule just to do the designs that
> would enable us to estimate the cost. I don't see how it
> could be less than several days of effort. It might be
> significantly more if we also implement searching for all of
> the negative states, and more still if we OR states together
> in searching.
> Personally, I like option two, but I'm not a user and I might
> feel differently if I were using the system every day.
> I've already spent a couple of hours on this issue.
> Unfortunately, in almost every system design we learn things
> along the way that can be schedule busters if the schedule doesn't
> include time for unanticipated problems.

I think the abstract review and flagging for FYI should be two entirely separate decisions that are recorded. I think we need a yes or no for abstract review regardless of FYI. The "no's" and the "FYI's" should be separate counts. I like the idea of making FYI a tag because it will be more flexible...a user could flag a citation as FYI at anytime in the review process regardless of what other decisions have already been recorded or not yet recorded...because FYI should not really change the state of a citation. If FYI is a state then what happens if the abstract review is a "yes" or becomes a "yes" later on? I think it would be much simpler to treat FYI as something separate, something outside the usual review process.

BZDATETIME::2012-11-02 10:33:00
BZCOMMENTOR::Bob Kline
BZCOMMENT::253

(In reply to comment #252)

> I like the idea of making FYI a tag ...

If we adopt this approach (which I agree make much more sense logically), I'll modify the conversion software to apply it for the legacy data.

BZDATETIME::2012-11-02 13:17:35
BZCOMMENTOR::Bob Kline
BZCOMMENT::254

For the reports, I was initially planning to open a separate browser tab for displaying the actual report, so that the report would look similar to what you have in the existing CiteMS, without the trappings of headers and footers or the form used for requesting the report (which would be useless clutter taking up space when you print the report), but avoiding the need to navigate back to the report request form using the Back button. However, there is a risk that because using the technique of opening a separate browser tab or browser window may run afoul of 508 compliance requirements we might be forced to abandon that approach, which would involve some rewriting of the reports software. So what I'm now thinking of doing is to implement the reports so the report content appears under the form used to request the report, in the same browser window/tab as the request was made, and to create style rules which suppress printing of the portions of the page which aren't part of the report itself (banner, headers, menus, the report request form, footers, etc.). You can still use screen or window capture techniques to print an image of how the page looks on the screen should that ever be necessary (for training materials, for example).

I also use the technique of opening up a separate browser tab for links which navigate to a non-EBMS page (for example, to PubMed for viewing article abstracts). Abandoning that technique would, I think, be significantly more annoying for EBMS users than what I'm proposing to do for report output, so I'm not going to change those links unless and until a requirement to do so is imposed by outside forces beyond our control.

Another possibility for the reports would be to do exactly what the legacy system does, which is to replace the page with the report request form with the page for the report output. With this approach you would use the browser's Back button to return to the search request form to submit another report request or to navigate elsewhere in the system using the menu links (as you currently do).

Any thoughts?

BZDATETIME::2012-11-02 13:37:37
BZCOMMENTOR::Alan Meyer
BZCOMMENT::255

(In reply to comment #254)
...
> Any thoughts?

I'm not real familiar with 508 requirements. I didn't know that opening a tab was a problem. But here are some poorly informed thoughts about it.

1. Are we sure we need 508 compliance? The pages we're talking about are not public facing. They're for internal use only, mainly by about ten people.

2. Would there be any difference in 508 compliance if we put the data in a new tab and automatically open that tab?

3. Would it help to have a checkbox, perhaps checked by default, to say open in a new tab? If not checked then we overlay the existing window.

BZDATETIME::2012-11-02 13:42:38
BZCOMMENTOR::Bob Kline
BZCOMMENT::256

(In reply to comment #255)

> 3. Would it help to have a checkbox, perhaps checked by default, to say open in
> a new tab? If not checked then we overlay the existing window.

It wouldn't help the schedule. Let's stifle our urges to implement multiple approaches to the same requirement until we get to phase 2. :-)

BZDATETIME::2012-11-02 14:43:14
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::257

(In reply to comment #254)
> For the reports, I was initially planning to open a separate browser tab for
> displaying the actual report, so that the report would look similar to what you
> have in the existing CiteMS, without the trappings of headers and footers or
> the form used for requesting the report (which would be useless clutter taking
> up space when you print the report), but avoiding the need to navigate back to
> the report request form using the Back button. However, there is a risk that
> because using the technique of opening a separate browser tab or browser window
> may run afoul of 508 compliance requirements we might be forced to abandon that
> approach, which would involve some rewriting of the reports software. So what
> I'm now thinking of doing is to implement the reports so the report content
> appears under the form used to request the report, in the same browser
> window/tab as the request was made, and to create style rules which suppress
> printing of the portions of the page which aren't part of the report itself
> (banner, headers, menus, the report request form, footers, etc.). You can
> still use screen or window capture techniques to print an image of how the page
> looks on the screen should that ever be necessary (for training materials, for
> example).
> I also use the technique of opening up a separate browser tab for links which
> navigate to a non-EBMS page (for example, to PubMed for viewing article
> abstracts). Abandoning that technique would, I think, be significantly more
> annoying for EBMS users than what I'm proposing to do for report output, so I'm
> not going to change those links unless and until a requirement to do so is
> imposed by outside forces beyond our control.
> Another possibility for the reports would be to do exactly what the legacy
> system does, which is to replace the page with the report request form with the
> page for the report output. With this approach you would use the browser's
> Back button to return to the search request form to submit another report
> request or to navigate elsewhere in the system using the menu links (as you
> currently do).
> Any thoughts?

Minaxi and I have never printed any of our reports and I don't see us ever needing to in the future. So I wouldn't spend any extra time making at least our reports print friendly...we just want to see the data. And I would prefer having the report populate on the same page that it is requested. This would make it easier to quickly change variable and generate another report.

BZDATETIME::2012-11-03 09:19:39
BZCOMMENTOR::Bob Kline
BZCOMMENT::258

For the "Citations Published" report, do you still want the rows for topics that have a count of zero? If we do it that way, the list of topics for a given board will be determined by which topics are currently associated with that board, since the system doesn't have knowledge of which topics were connected with which board at every moment in history. As a result, topics which were re-assigned since the review cycle for which the report is requested won't show up under the board for which the publications were originally made. If you run the report without restricting it to a single board, the topic will show up under its new board, but if you run it for a single board, you won't see topics which no longer belong to that board.

BZDATETIME::2012-11-05 08:02:45
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::259

(In reply to comment #258)
> For the "Citations Published" report, do you still want the rows for topics
> that have a count of zero? If we do it that way, the list of topics for a
> given board will be determined by which topics are currently associated with
> that board, since the system doesn't have knowledge of which topics were
> connected with which board at every moment in history. As a result, topics
> which were re-assigned since the review cycle for which the report is requested
> won't show up under the board for which the publications were originally made.
> If you run the report without restricting it to a single board, the topic will
> show up under its new board, but if you run it for a single board, you won't
> see topics which no longer belong to that board.

We do not need to see the topics with zero citations. And we have topics that we do not run searches for anymore and would always have zero. We definitely do not want those to show in current reports.

BZDATETIME::2012-11-05 09:03:35
BZCOMMENTOR::Alan Meyer
BZCOMMENT::260

(In reply to comment #250)

...
> Minaxi and I run searches in the CMS that generate well over 200 or even 500
> hits. We are not interested in actually seeing all of them; we just want to
> know the number of citations retrieved.

To get the count, the search has to run to completion.

I've removed the limit. If something is taking too long a user can just kill the connection by clicking the X to cancel the request. If this is happening too often we'll look to see if there's some way to optimize the search.

BZDATETIME::2012-11-05 16:35:34
BZCOMMENTOR::Robin Juthe
BZCOMMENT::261

(In reply to comment #254)
> So what
> I'm now thinking of doing is to implement the reports so the report content
> appears under the form used to request the report, in the same browser
> window/tab as the request was made, and to create style rules which suppress
> printing of the portions of the page which aren't part of the report itself
> (banner, headers, menus, the report request form, footers, etc.). You can
> still use screen or window capture techniques to print an image of how the page
> looks on the screen should that ever be necessary (for training materials, for
> example).

> Any thoughts?

Victoria and I agree with the proposal to display the reports on the same page as the form used to submit the report request. If this information can be suppressed when printing, that would be ideal. Thanks.

BZDATETIME::2012-11-05 17:26:59
BZCOMMENTOR::Robin Juthe
BZCOMMENT::262

(In reply to comment #245)
> Overall, I think we have the following choices:
> 1. Keep FYI as a state. Proceed as we have been.
> Impact on the schedule - don't know that there is any.
> You would be able to search for the FYI state, but that state
> would be in only one place in the workflow sequence - at the
> same sequence level as Passed/Rejected by Board Manager.
> There would still be no search for articles that are Rejected
> by Board Manager OR FYI.
> 2. Change FYI to a tag but use the existing methods for setting
> states and tags. No new software.
> Impact on the schedule - shouldn't cost more.
> This makes the combination of rejecting an article at the
> abstract review stage and creating an FYI two operations
> instead of one, but does provide more flexibility in
> assigning FYI to articles (doesn't have to be done with a
> rejection, or not with a rejection at just that point), and
> does, by at least one interpretation, make the rejection
> states more consistent.
> 3. Change FYI to a tag but add more support for processing it as
> if it were a state and adding other new support.
> Impact on the schedule - some. Don't know how much.
> It would add to the schedule just to do the designs that
> would enable us to estimate the cost. I don't see how it
> could be less than several days of effort. It might be
> significantly more if we also implement searching for all of
> the negative states, and more still if we OR states together
> in searching.
> Personally, I like option two, but I'm not a user and I might
> feel differently if I were using the system every day.
> I've already spent a couple of hours on this issue.
> Unfortunately, in almost every system design we learn things
> along the way that can be schedule busters if the schedule doesn't
> include time for unanticipated problems.

Unless I am missing something, I still think that the advantages of treating FYIs as a state outweigh the advantages of storing FYIs as a tag.

The advantage of increased flexibility (assigning an article an FYI at any point in the process) with a tag approach is not critical to us. In the current system, we assign FYIs at a single point in the process (full-text review). It isn’t likely that we would get beyond this state and want to go back and make it an FYI. (Although we always could do this from the full-text review screen by adding another decision.) I do see us perhaps knowing that we plan to send something out as an FYI during the abstract review step, but even if we assigned the "FYI" tag at that point, it would still appear in our full-text review queue without additional effort (option 3, I think) to allow the FYI tag to alter the values of a particular state. Is that right?

The issue of searching and reporting is more important. Articles should be passed, rejected, OR assigned as an FYI during full-text review. Articles that are rejected during full-text review are not the same as articles given an FYI disposition. Yes, both states have the same effect of being the “end of the line” for that citation, so I can understand why the software involving these two states is identical, but it’s important for us to be able to search and/or develop a report that will treat these discretely. I know that the current search screen accommodates only YES decisions at the full-text review step, but as I understand it, that is only temporary (this phase). Eventually, we would like to have the other values (no & FYI) available to us on the search screen.

Please let me know if I'm missing something. I'd be happy to talk more about this tomorrow. Thanks!

BZDATETIME::2012-11-05 20:01:13
BZCOMMENTOR::Alan Meyer
BZCOMMENT::263

(In reply to comment #262)
> ...
> Please let me know if I'm missing something. I'd be happy to talk more about
> this tomorrow. Thanks!

The ideas are getting complex. Let's discuss it tomorrow.

BZDATETIME::2012-11-06 15:35:08
BZCOMMENTOR::Alan Meyer
BZCOMMENT::264

(In reply to comment #263)
...
> The ideas are getting complex. Let's discuss it tomorrow.

We just discussed it (Robin, Victoria, Bob, Dan, and Alan.) The decision was, leave FYI as a state, as it was in the old system. No FYI tag will be created.

BZDATETIME::2012-11-08 13:58:59
BZCOMMENTOR::Alan Meyer
BZCOMMENT::265

In the course of making some changes to the database control tables I noticed that we have the following sequence values in the state type table.

All at one sequence level:

Changes not for agenda
For future agenda (with changes)
For future agenda (discussion only)

At the next higher sequence level:

On agenda

I'm now wondering if that's right. All four of them are about the same thing, i.e., whether an article is on the agenda. I'm therefore thinking that all four should be at the same sequence level so that any one of them inactivates any of the others. The main difference would be that, if a user marks an article as "On agenda", then if one of the other three had been assigned, that one is now inactivated (as opposed to just being superseded.)

Does that sound right?

Thanks.

BZDATETIME::2012-11-09 17:11:06
BZCOMMENTOR::Robin Juthe
BZCOMMENT::266

(In reply to comment #265)
> In the course of making some changes to the database control tables I noticed
> that we have the following sequence values in the state type table.
> All at one sequence level:
> Changes not for agenda
> For future agenda (with changes)
> For future agenda (discussion only)
> At the next higher sequence level:
> On agenda
> I'm now wondering if that's right. All four of them are about the same thing,
> i.e., whether an article is on the agenda. I'm therefore thinking that all
> four should be at the same sequence level so that any one of them inactivates
> any of the others. The main difference would be that, if a user marks an
> article as "On agenda", then if one of the other three had been assigned, that
> one is now inactivated (as opposed to just being superseded.)
> Does that sound right?
> Thanks.

This is an old set of values for the Board Manager Action step. (We made some changes when we put together the full citation page.)

Here are the new values:

BOARD MANAGER ACTION
-No further action
-Minor changes not for Board review
-Summary changes for Board review (no paper for discussion)
-Paper and summary changes for discussion
-Paper for Board discussion
-Paper for Working Group discussion

I think it makes sense for each of the above values to receive the same sequence number.

I am thinking through your question about the sequence number of "on agenda" with an example of how I plan to use these two steps in the process. Here's my example:

1. I assign Paper A the state "Paper for working group discussion".
2. I later add Paper A to a working group meeting agenda and give it the "on agenda" state with a meeting date & type.
3. Once the changes are discussed by the working group, I go back to Board manager action for Paper A and assign the state "Paper for Board discussion". 4. I add a new "on agenda" decision for Paper A with the meeting date and type for a Board meeting.

To me, inactivating a state means that the information is no longer relevant or useful (and we decided to gray out and "hide" this information in the design). I would find the information about this paper being on a previous working group meeting agenda (with associated comments) very relevant when preparing a Board meeting agenda, so I don't think step 3 should inactivate step 2. However, it makes sense to me for step 3 to inactivate step 1, and step 4 to inactivate step 2.

This example is more circuitous than most uses of these values, but I tried to illustrate a "messy" example in order to see if it would make sense to have each of these decisions inactivate the previous one. In most cases, we will assign a single Board Manager Action and put the paper on a single agenda. I can see us wanting to know both values at the same time (on a report down the road or even visible to us on the full citation page) - e.g., what is on my agenda and what got it there (which BM Action value)?

So, bottom line: I vote for a higher sequence number for "on agenda".

BZDATETIME::2012-11-09 17:21:25
BZCOMMENTOR::Dan Young
BZCOMMENT::267

(In reply to comment #266)
> (In reply to comment #265)
> > In the course of making some changes to the database control tables I noticed
> > that we have the following sequence values in the state type table.
> > All at one sequence level:
> > Changes not for agenda
> > For future agenda (with changes)
> > For future agenda (discussion only)
> > At the next higher sequence level:
> > On agenda
> > I'm now wondering if that's right. All four of them are about the same thing,
> > i.e., whether an article is on the agenda. I'm therefore thinking that all
> > four should be at the same sequence level so that any one of them inactivates
> > any of the others. The main difference would be that, if a user marks an
> > article as "On agenda", then if one of the other three had been assigned, that
> > one is now inactivated (as opposed to just being superseded.)
> > Does that sound right?
> > Thanks.
>
> This is an old set of values for the Board Manager Action step. (We made some
> changes when we put together the full citation page.)
>
> Here are the new values:
>
> BOARD MANAGER ACTION
> -No further action
> -Minor changes not for Board review
> -Summary changes for Board review (no paper for discussion)
> -Paper and summary changes for discussion
> -Paper for Board discussion
> -Paper for Working Group discussion
>
> I think it makes sense for each of the above values to receive the same
> sequence number.
>
> I am thinking through your question about the sequence number of "on agenda"
> with an example of how I plan to use these two steps in the process. Here's my
> example:
>
> 1. I assign Paper A the state "Paper for working group discussion".
> 2. I later add Paper A to a working group meeting agenda and give it the "on
> agenda" state with a meeting date & type.
> 3. Once the changes are discussed by the working group, I go back to Board
> manager action for Paper A and assign the state "Paper for Board discussion".
> 4. I add a new "on agenda" decision for Paper A with the meeting date and type
> for a Board meeting.
>
> To me, inactivating a state means that the information is no longer relevant or
> useful (and we decided to gray out and "hide" this information in the design).
> I would find the information about this paper being on a previous working group
> meeting agenda (with associated comments) very relevant when preparing a Board
> meeting agenda, so I don't think step 3 should inactivate step 2. However, it
> makes sense to me for step 3 to inactivate step 1, and step 4 to inactivate
> step 2.
>
> This example is more circuitous than most uses of these values, but I tried to
> illustrate a "messy" example in order to see if it would make sense to have
> each of these decisions inactivate the previous one. In most cases, we will
> assign a single Board Manager Action and put the paper on a single agenda. I
> can see us wanting to know both values at the same time (on a report down the
> road or even visible to us on the full citation page) - e.g., what is on my
> agenda and what got it there (which BM Action value)?
>
> So, bottom line: I vote for a higher sequence number for "on agenda".

Robin,

We discussed the 'on agenda' state yesterday and arrived at your decision as well. We'll need to do some work on the database to set up the states you mention, but this is good information. Thanks for the response!

Dan

BZDATETIME::2012-11-13 23:59:31
BZCOMMENTOR::Alan Meyer
BZCOMMENT::268

I've written a document that describes how searching works (or at least how I think it works) in the EBMS as of today.

It's complicated and may not always be what users want or expect, but at least everyone can now know what it does. We can work on changing things that aren't right if we have to.

I wrote this so that Sridhar would be able to test the search functionality, but it's also important for users and we should probably incorporate something like this in any user manual that we write.

Attachment SearchFunctionality.txt has been added with description: How searching works in the EBMS

BZDATETIME::2012-11-29 17:14:18
BZCOMMENTOR::Dan Young
BZCOMMENT::269

(In reply to comment #266)
> Here are the new values:
>
> BOARD MANAGER ACTION
> -No further action
> -Minor changes not for Board review
> -Summary changes for Board review (no paper for discussion)
> -Paper and summary changes for discussion
> -Paper for Board discussion
> -Paper for Working Group discussion

Robin, I was wondering which of these states may represent a completion of the review process for the article/topic combination? For the old four states, we had flagged "No further action" as completed, as the topic was finished at that point. For the five new states apart from "No further action", do you see all of these needing to be either added to an agenda or assigned a final board decision?

I believe the last four are clearly aimed at being added to an agenda, so the question mostly pertains to #2, "Minor changes not for Board review". This looks like it shouldn't end up on agenda, but perhaps might be assigned a board decision to represent the needed changes.

Can you shed any light on this? Thanks.

BZDATETIME::2012-12-05 17:33:09
BZCOMMENTOR::Robin Juthe
BZCOMMENT::270

(In reply to comment #269)
> (In reply to comment #266)
> > Here are the new values:
> >
> > BOARD MANAGER ACTION
> > -No further action
> > -Minor changes not for Board review
> > -Summary changes for Board review (no paper for discussion)
> > -Paper and summary changes for discussion
> > -Paper for Board discussion
> > -Paper for Working Group discussion
> Robin, I was wondering which of these states may represent a completion of the
> review process for the article/topic combination? For the old four states, we
> had flagged "No further action" as completed, as the topic was finished at that
> point. For the five new states apart from "No further action", do you see all
> of these needing to be either added to an agenda or assigned a final board
> decision?
> I believe the last four are clearly aimed at being added to an agenda, so the
> question mostly pertains to #2, "Minor changes not for Board review". This
> looks like it shouldn't end up on agenda, but perhaps might be assigned a board
> decision to represent the needed changes.
> Can you shed any light on this? Thanks.

Dan, I'm sorry I missed this! Only the "No further action" state represents a completion of the review process. Each of the other states (including #2) reflects a decision that could land the article on an agenda and/or merit a final/Editorial Board decision. Hope this helps.

BZDATETIME::2012-12-05 17:52:22
BZCOMMENTOR::Dan Young
BZCOMMENT::271

(In reply to comment #270)
> Dan, I'm sorry I missed this! Only the "No further action" state represents a
> completion of the review process. Each of the other states (including #2)
> reflects a decision that could land the article on an agenda and/or merit a
> final/Editorial Board decision. Hope this helps.

Great, thanks for the clarification. I had already guessed that this was the case, but it's good to hear it from someone who knows definitively.

BZDATETIME::2012-12-13 20:46:40
BZCOMMENTOR::Alan Meyer
BZCOMMENT::272

A Technical Incident Report was filed in OCE Works (Sharepoint based) about searching. I wrote a reply but Sharepoint threw away my formatting. It offered "help on HTML tags", but when I clicked on it it took me somewhere else.

So I tried using <pre> and </pre> wrappers, but Sharepoint threw those away too.

So here it is in Bugzilla in a more readable format. Please see the attachment.

Attachment EBMS_SearchBugs.txt has been added with description: Response to Technical Incident Report on search issues.

BZDATETIME::2012-12-14 09:30:21
BZCOMMENTOR::Bob Kline
BZCOMMENT::273

(In reply to comment #272)

> So I tried using <pre> and </pre> wrappers, but Sharepoint threw those away
> too.

You have to use Internet Explorer to get any Sharepoint-based site to come close to behaving correctly. (I'm not making this up!)

BZDATETIME::2012-12-14 14:54:18
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::274

(In reply to comment #272)
Keep in mind that only Published citations should be retrieved or displayed in the CiteMS unless otherwise specified. Currently both published and unpublished are being retrieved.

The Unpublished includes citations excluded by Journal NOT lists, citations rejected by the medical librarin during initial review and citations just imported that have not been reviewed by the medical librarian.

"Include Rejected Articles" check box
To clarify I think we should rename this "Include Articles Rejected by Medical Librarian" Or "Include Unpublished Citations"

What should be excluded when the
box is NOT checked?

b. All articles rejected before publication?...yes

This would include articles rejected in initial review and
articles rejected by NOT listing.

c. All articles that have not (yet) passed initial review...yes
This would include everything in b. plus anything that has
not yet been initially reviewed by the medical librarian.

BZDATETIME::2012-12-14 17:33:55
BZCOMMENTOR::Alan Meyer
BZCOMMENT::275

Here is some more on searching:

Journal titles
--------------

I fixed the bug in the journal title search. It was broken
internally. Even after it's fixed there will be some quirks.
For example I searched for "New England Journal%" and missed what
I wanted because it is actually stored as "The New England
Journal of Medicine". In order to leave off the "The" I would
have had to say: "%New England Journal%".

Inclusion/exclusion of unpublished articles
-------------------------------------------

We have three separate checkboxes that have some overlapping
responsibilities here.

PUBLISHED TO CITE MS
INCLUDE REJECTED ARTICLES
INCLUDE NOT LIST JOURNALS

All fiddle with the same set of articles, potentially in
contradictory ways, and need to be considered together.

Without considering priorities (not all of these are necessarily
high priority and some of them may require some significant
work), I propose changes as follows:

PUBLISHED TO CITE MS

This is the inverse of what Cynthia wants. It works, but
we should probably replace it with its inverse, something
like:

INCLUDE NOT YET PUBLISHED

with the default being to exclude not yet published
material (what checking the box does now). With the
newly named box checked, we would include:

Articles not yet published that are not yet rejected.

INCLUDE REJECTED ARTICLES

Ideally, we'd only check it if INCLUDE NOT YET PUBLISHED
is already checked.

Not list journals would NOT be included.

INCLUDE NOT LIST JOURNALS

I propose that this just includes NOT listed journal
articles.

Perhaps a good way to see these on the search form would be to
put all of them under Advanced search or Adminstrator search in a
box of their own and indicate that they are hierarchically
related, e.g.:

/Unpublished articles-------------------\
| [ ] INCLUDE NOT YET PUBLISHED |
| [ ] INCLUDE REJECTED ARTICLES |
| [ ] INCLUDE NOT LIST JOURNALS |
-----------------------------------------/

or something like that. If we have a lot of time and want to be
fancy we could have the second box only appear if the first is
checked and the third only if the second is checked. It doesn't
seem to make much sense to check the second without the first, or
the third without the first and second.

I won't do anything until people have had a chance to think about
all this and until priorities are figured out.

BZDATETIME::2012-12-14 17:36:25
BZCOMMENTOR::Alan Meyer
BZCOMMENT::276

(In reply to comment #275)

> I fixed the bug in the journal title search.

The fix is only on the development server, but I'll put it under version control and it should be picked up the next time we refresh the software on QA (Quality Assurance) and, later, on CA (Client Acceptance).

BZDATETIME::2012-12-17 10:16:51
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::277

(In reply to comment #275)
> Perhaps a good way to see these on the search form would be to
> put all of them under Advanced search or Adminstrator search in a
> box of their own and indicate that they are hierarchically
> related, e.g.:
> /Unpublished articles-------------------\
> | [ ] INCLUDE NOT YET PUBLISHED |
> | [ ] INCLUDE REJECTED ARTICLES |
> | [ ] INCLUDE NOT LIST JOURNALS |
> -----------------------------------------/
> or something like that. If we have a lot of time and want to be
> fancy we could have the second box only appear if the first is
> checked and the third only if the second is checked. It doesn't
> seem to make much sense to check the second without the first, or
> the third without the first and second.
> I won't do anything until people have had a chance to think about
> all this and until priorities are figured out.

This sounds like a good plan. I just have a few comments to add.
"Include Not Yet Published" implies that these are all citations that are waiting to be published or will be published soon. This does not really apply to any of the citations excluded by the NOT journal list or even citations that I have rejected. These citations will most likely never be published which is why we only want to see published citations by default. I reject close to 40% of all citations retrieved each month and then another several hundred are excluded by the NOT Journal Lists. (This is a huge chunk of citations that have been laying low behind the scenes and are now appearing in search results in the new system. So definitely a high priority fix.)
"Include Rejected Articles" ... I reject summary topics not articles, but I think most users understand what this means. By checking this box they will be including citations that were rejected for the specific summary topic or board they have selected. So a board or summary topic need to be required variables if this box is checked.
"Include NOT List Journals" this will not work unless editorial board is a required variable. Also Citations that have been excluded by a board's NOT journal list will NOT be reviewed by the medical librarian so they are not "Rejected Articles" they are "Excluded Articles". Therefore, the "Include NOT List Journals" is not a subset of "Include Rejected Articles". They are two separate distinctions...two different dip nets reducing the number of fish in the pond.
/Unpublished articles---------------------------------------\

--------------------------------------------------------------/

BZDATETIME::2012-12-18 16:54:21
BZCOMMENTOR::Alan Meyer
BZCOMMENT::278

Here's an update on the issues assigned to me in the OCE Works
Issue Tracker.

Several of these are, or will shortly be, fixed. I'll post the
fixes on the dev and qa servers before I go home tonight.

SEARCHING
---------

1. Include Rejected Articles.

See Bugzilla comment addressing this

http://verdi.nci.nih.gov/tracker/show_bug.cgi?id=4907#c275

and the following response from Cynthia.

To implement this as desired requires:

Some time from Bob to change the search form and the
corresponding JSON search string.

Some time from Alan (shouldn't be more than a day) to
implement the changes in the Search API.

Current workaround is to always check the box at:

ADVANCED SEARCH / ADVANCED OPTIONS / PUBLISHED TO CITE MS

or perhaps have Bob make it checked by default in the search
form.

2. Reset or clear button.

This is a user interface issue I'll refer to Bob.

Workaround is to click "Search Database" in the menus to get a
clean form rather than use the back button.

3. Review cycle.

This currently works, but only for new data, not legacy data.
Bob, Dan, Laura and I have discussed a fix for this that will
include legacy data.

Time estimate:

My part is a few hours.

Bob will have much more. Dan will have some hours.

4. Journal search.

Fixed.

5. Input date and date range.

There appears to have been a change in the way years were
recorded in the search spec. It used to be that year 1900 =
year 1, and I adjusted accordingly.

I've removed the adjustment and this now works.

All of the date and date range search inputs were affected by
this and all should now be fixed.

6. 500 Max.

Fixed.

Note: If the performance is bad, which it might be for
searches with huge result sets (partly related to sorting
them), then we might consider another field on the search form
where a user could specify the max number of hits to look for.
We could set a default of 500, or whatever, and let a user
clear the field if she wants everything, or set some other
max.

7. Publication date pull down should show most recent dates
first.

For Bob.

8. Jump ahead multiple pages.

For Bob, but maybe lower priority now. See 10 below.

9. Display option to view queue.

I think this is handled by a different function that pulls
up the initial reviewer queue, not the search interface. Dan
would probably know.

10. Sort PMID and CMS ID in reverse order.

Fixed. Does this lower the priority of 7?

11. Finding 'No' decisions as well as 'Yes' decisions.

This is something that got lost during a period of scrambling
to get stuff out the door faster (we're still in that period,
but eventually we'll get out.)

Laura's original requirements and Ashleigh's original design
specified separate YES and NO checkboxes or radio buttons for
the advanced search options but, in the scramble, only YES got
implemented.

If and when the user interface is changed back to the YES/NO
pairs instead of just one box each, I can implement it in the
searching back end.

I don't think it will be hard to handle this but there's an
issue of priorities, which someone above me will have to
decide.

IMPORT BATCH FILE
-----------------

The garbage at the front of the imported file is the Byte Order
Mark added by Internet Explorer 9.

It will take about an hour to fix this and I'll do it shortly.

INCLUDE NOT LIST JOURNALS
-------------------------

I believe this works, but it is only useful on new data entered
in the new system. The legacy system did not import not listed
journal articles, so checking the box on any search that only
sees legacy data will show no difference whether the include not
list box is checked or not.

BZDATETIME::2012-12-18 17:56:55
BZCOMMENTOR::Alan Meyer
BZCOMMENT::279

I have installed the fixes mentioned above on the development
servers (verdi and ebms-dev), and on the qa (ebms-qa) server, to
wit:

All of the date range searches are fixed. A change in the
user interface code for interpreting dates had thrown these
out of whack. But they should all work now.

Imports should now work with Internet Explorer 9 or any other
browser that prepends a Unicode Byte Order Mark to the front
of a Pubmed search result.

BZDATETIME::2012-12-19 17:51:17
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::280

Review Cycles - I think we have opened a can of worms.
Minaxi and I discussed this after the meeting and have several other issues to add to the discussion of review cycles in today's meeting.
In the meeting I said that review cycles are added upon import and are always associated with a summary topic...this is true prior to publication in the legacy system (for importing as well as when I add a summary topic during my review) but after publication summary topics can be added and I am not sure if a review cycle is assigned to that topic. There is currently no means in the legacy system for me to see if a review cycle is added or not when a topic is added post publication.
When importing review cycle is required in both systems. In the legacy system review cycle has to be selected when I added topics during my review but no where else after publication.
In the new system review cycle is only assigned upon import. After a citation is imported, pre- or post publication summary topics can be added and review cycle is not something we have an option to add and once the topic is added there is currently no means for me to see if a review cycle is added to that summary topic or not. In the new system the review cycle seems only linked to the citation itself not the topics.
So the legacy system most likely has summary topics with (topics added prepublication or via import) and without (topics added post publication) review cycles. And I am not sure what impact this may have had on the conversion.
The main reason for having review cycles is so that Minaxi and I can easily identify all new citations and all citations with new topics added to the database each month. So how can we accomplish this with out creating any more problems? And don't say use input date because then you are asking us to remember or track all the dates Minaxi imported citations AND will only work if input date is updated when only summary topics are added to citations. Minaxi and I are trying to consider review cycles differently but we keep coming back to the same problems.
I suggest that we have a meeting to discuss this before our worms turn into snakes.

BZDATETIME::2012-12-19 21:24:50
BZCOMMENTOR::Alan Meyer
BZCOMMENT::281

(In reply to comment #280)
> Review Cycles - I think we have opened a can of worms.
...
> I suggest that we have a meeting to discuss this before our worms turn into
> snakes.

I had already written most of a document when I saw this, but I'm no longer sure that what I've written is good. So I agree with your suggestion.

I propose that we hold off on this until next week since all of us developers already have full plates at least until then.

As preparation, would it be possible for you, and anyone else you think should help, to write up a list of things that you want review cycles to do? I have in mind something like:

Identify articles that should have been reviewed but weren't.

Gather statistics on number of articles processed for each topic over
different time periods, and for each board, and for the whole system.

It may be that we already have information that can be used to provide this without having review cycles at all. It may be that all we need is to do something in the search back end or the reports that we would have had to do anyway, but do it with the state tables we have instead of with review cycles.

Or it may be that we need more support for review cycles than we currently have.

BZDATETIME::2012-12-20 02:49:35
BZCOMMENTOR::Bob Kline
BZCOMMENT::282

(In reply to comment #280)

> The main reason for having review cycles is so that Minaxi and I can easily
> identify all new citations and all citations with new topics added to the
> database each month. So how can we accomplish this with out creating any more
> problems?

I think Alan's import tables are already capturing the information needed to accomplish this goal for imports which occur after conversion. Those tables don't capture what happens after you release the articles for review by ICRDB, but (if I understand your previous comment correctly) neither did the legacy system.

So it seems that the two major questions facing us for this issue are:

Is it OK if reports and queues which filter by cycle don't pick up data from the legacy conversion?

Is it sufficient to identify by review cycle "all citations with new topics added to the database each month" using the import tools, or must we also assign a review cycle to each action adding a new topic to an existing article regardless of how far down stream that action occurs?

BZDATETIME::2012-12-20 11:29:53
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::283

(In reply to comment #282)

Is it OK if reports and queues which filter by cycle don't pick up data from the legacy conversion?

No it’s definitely not ok. The legacy system is very review cycle dependant. We need the capability to search and retrieve data using review cycle. There is no other date stamp that will compensate for review cycle in the legacy system. Minaxi and I have stressed the importance of review cycle in the legacy system from the get go. Yes, Review Cycle seems like just a label assigned with no specific date or date range. It is just a label, but an important one that has served many purposes in the legacy system for 12 years. Purposes that are not filing tables of data or driving status changes but important none the less.

Is it sufficient to identify by review cycle "all citations with new topics added to the database each month" using the import tools, or must we also assign a review cycle to each action adding a new topic to an existing article regardless of how far down stream that action occurs?

Yes, I think we need a review cycle linked to a summary topic regardless of when or how the summary topic is added. In most cases the review cycle will be set by the user either during import, medical librarian review or fast tracking. If a summary topic is added from the full citation display by a board manager or onsite staff (so excluding the medical librarian review section) then the review cycle should be set to the current review cycle by default.

Currently the review cycle is available to add upon import and fast tracking but not during medical librarian review (from my que as well as the med lib section in the full citation display). Review cycles for topics added that are in my que could be automatically assigned the current review cycle so that the design of that page does not need to change. But we need to be able to set the review cycle in the medical librarian section of the full citation display.

BZDATETIME::2012-12-21 14:31:50
BZCOMMENTOR::Robin Juthe
BZCOMMENT::284

Margaret and I met to talk about review cycles today. We came up with a list of what we think review cycles have enabled us to do in the past. We still need to be able to do these things. Based on this list, we have some suggestions for how we might accomplish these tasks in the new system. We also have an idea about converting the legacy data. We look forward to your comments.

NEEDS

Medical Librarians:

• Identify all new data (newly imported citations and new topics assigned to a citation) for review each month.

Board Managers:

• Identify when a certain citation was first associated with her Board (could be on import or later, when a new topic is assigned).

All Users:

• Gather statistics on the number of articles reviewed, approved, etc. over various time frames by topic, by Board, and for the whole system.

• Group report data into manageable “chunks” of information for review.

POTENTIAL SOLUTIONS

New System:

We think that the above tasks could be performed in the new system using a system-generated review cycle (month and year combination, as in the old system). We understand the convenience of assigning a review cycle at various times during a month based on where the medical librarians or Board managers are in their process of review, but perhaps we could rely on pre-determined dates that span two calendar months (e..g, “November 15-December 14”, “December 15-January 14”, etc.) rather than have a single calendar month correlate with a single review cycle. These two examples would correspond with the “December 2012” review cycle and the “January 2013” review cycle, respectively. The review cycle should be topic-specific so that it reflects when a citation and topic were first associated with one another. This action might occur on import (in a batch or a fast track), on one of the queue review pages, or on the full citation page. We shouldn’t need a separate drop-down to add a review cycle; it should be able to be added automatically based on the action being performed and when that action occurs.

In the new system, we also think we will have some new data (including time stamps) that are now available to us to help with searching and reports. For example, the Board managers may rely on the “packet creation date” to inform some of their reports.

Legacy Data:

With our (little) understanding of the behind the scenes programming, it seems like it would be more time consuming from the programming perspective and also from a performance perspective if we expect the existing fields in the search form and in various reports to be able to scan for review cycles in both the new and old systems considering the fact that this information may be stored differently. However, with that being said, it is still important for us to be able to generate reports using the review cycle in the legacy system to be able to obtain historical statistics. Therefore, we propose that the review cycle from the legacy system be converted over as a secondary “tag” (separate from the existing citation-topic tags) that could be searched on and used in various reports. Over time, the ability to search on legacy data should become less important, so it seems like putting this in a separate field (and improving search performance) is worth doing.

BZDATETIME::2012-12-21 19:56:04
BZCOMMENTOR::Minaxi Trivedi
BZCOMMENT::285

Attachment CiteMS_citations display in import utilit.doc has been added with description: This examples show the importance of display of review cycle for review

BZDATETIME::2012-12-26 11:01:02
BZCOMMENTOR::Minaxi Trivedi
BZCOMMENT::286

(In reply to comment #284)
> Margaret and I met to talk about review cycles today. We came up with a list of
> what we think review cycles have enabled us to do in the past. We still need to
> be able to do these things. Based on this list, we have some suggestions for
> how we might accomplish these tasks in the new system. We also have an idea
> about converting the legacy data. We look forward to your comments.
>
> NEEDS
>
> Medical Librarians:
> - Identify all new data (newly imported citations and new topics assigned to a
> citation) for review each month.
>
>I had posted the comment prior to sending the attachment, but it never got posted due to problem with verdi.

When new citation is imported to CiteMS, it gets added either as a new record or new data is added to an existing record. In order to take the decision about newly added data to the existing record, it is very important to see all the old data including review cycle to take a decision to accept or reject the summary topic. Please see the attached examples. CMS ID 260491 was already published for PC-SPES for CAM Board in Feb 2012 review cycle. In December review cycle the same citation came up under CAM overview for the same Board. Since it is the same citation for same board, it would be rejected while reviewing. Similar is the case with CMS ID 271174. It was fast tracked in July 2012 for Adult Leukemia and it came up under Adult AML for Dec. 2012 and it was rejected.
Thus it is very important to view all summary topics and associated board and review cycle at a glance. The new data is going to overlap with legacy data. We would have citations in the database with summary topic from legacy review cycles as well as summary topics with review cycles from the new system.

BZDATETIME::2013-01-02 14:59:08
BZCOMMENTOR::Robin Juthe
BZCOMMENT::287

(In reply to comment #286)

Thanks for these examples, Minaxi. It’s always helpful to think things through with actual data. I think this issue could be addressed with the solution I proposed above. The way we envision this working is that the review cycles assigned to legacy data would be preserved (possibly as another type of tag on a summary topic or something else depending on what the developers suggest) and all previous board/topic/cycle associations would be visible on the full citation page. So, you would still be able to see that an article was imported under a previous review cycle for another topic. Does that address your concern? Sorry if I am misunderstanding your comment.

BZDATETIME::2013-01-02 16:35:35
BZCOMMENTOR::Minaxi Trivedi
BZCOMMENT::288

(In reply to comment #287)
> (In reply to comment #286)
>
> > all previous board/topic/cycle associations would be visible on the full
> citation page. So, you would still be able to see that an article was imported
> under a previous review cycle for another topic. Does that address your
> concern? Sorry if I am misunderstanding your comment.

Robin, your understanding is correct.

BZDATETIME::2013-01-03 08:47:26
BZCOMMENTOR::Bob Kline
BZCOMMENT::289

We had said that the solution to Cynthia's report of unexpected items in her queue (she said that some of the article-topic combinations which were appearing in her queue in the new system had been "deleted" by her) was to ignore the rows in the asc_ORIGINAL_SUMMARIES table. On reflection, I believe it would be better to continue to use that table to create the "Ready for initial review" status, and then after all of the rows in the asc_CITE_SUMMARIES table have been used to create the "Rejected in initial review" and "Passed initial review" statuses, add "Rejected in initial review" status rows for the article-topic combinations which came from the asc_ORIGINAL_SUMMARIES table and which were not represented at all in the asc_CITE_SUMMARIES table. This is an idea suggested several months ago by Alan, who proposed that we add this step as a global change when we went into production with the new system. It is supported by the information we have been given that when we switch from the legacy system to the EBMS all of the queues will have been cleared out, so we wouldn't have the risk of inappropriately marking as rejected those topics which just hadn't gotten around to being reviewed yet. I propose that we incorporate this step as part of the conversion which is run each time we do a build of the system. The users will need to recognize when testing the interim conversion builds there will be topics in the "Rejected in initial review" state only because they were recently added at conversion time and had not yet been reviewed by the librarian.

One difficulty is that because the old system does not appear to have recorded this "deletion" of topics, but simply reflected them by failing to add rows to the asc_CITE_SUMMARIES table, we don't have a good way to assign a date to the "Rejected in initial review" status rows created by this new step in the conversion. Unless someone has a better suggestion I will use a date/time one week after the article was originally imported as the date/time when the "rejection" happened. That will avoid the silliness we'd get if I used the import date/time (implying that the librarian simultaneously thought it was an appropriate topic and an inappropriate topic). It will also (possibly) give us some recently imported articles in the librarian's queue for the interim test conversions (since I would avoid adding status rows with future status dates).

Does this approach seem reasonable?

BZDATETIME::2013-01-03 15:55:27
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::290

(In reply to comment #289)
> Does this approach seem reasonable?
Yes.

BZDATETIME::2013-01-05 15:49:46
BZCOMMENTOR::Bob Kline
BZCOMMENT::291

Here is a proposed approach to carrying over the legacy review cycle information from the old system into the EBMS.

We would add a new table to the system, which records the article ID, the topic ID, the cycle ID, and the ID of the row in the article state table which reflects the state the article was in when the topic was first associated with the article. Each article-topic combination which was ever made would have exactly one row in this new table. The conversion software will populate this table using the review cycle information in the legacy asc_ORIGINAL_SUMMARIES and asc_CITE_SUMMARIES tables, and the API calls to create a new row in the status table for activity which takes place after conversion will ensure that if a row does not already exist in the new table for the article-topic combination represented by the new status row, such a row will be created in the new table. For situations where the user interface requires the user to specify a review cycle (as is true for the most common case in which a topic is associated with an article, namely the import page), the review cycle expressly selected by the user will be recorded. Where there is no user interface for specifying a review cycle (for example, on the "full citation" page which allows a new topic to be associated with the article further downstream in the processing of the article) the software will use an algorithm (for example, if the current date is earlier than the 10th day of the current month, use the review cycle corresponding to this month, otherwise use the review cycle for the following month) to decide which review cycle to record. The details for the algorithm (e.g., which day of the month should be used as the cutoff) can be adjusted before we go into production to match what the users think would work best. As has been discussed earlier, the review cycles available for selection will be automatically created by the system, without the need for an administrator to do anything to make it happen.

We will go back through the code in the new system which generates work queues, reports, and search results, and modify that code to use the information in this new table.

We will also modify the conversion software to populate the tables which record information about article import activity (including review cycles) for the legacy data. This will allow the librarians to find by review cycle and report on import jobs which took place in the old system. It will also result in search results which no longer omit legacy data when the search criteria include a review cycle, which is currently only available for newly imported articles. This conversion presents some challenges, as the legacy system recorded less information than the new import tables will capture. Here is how we propose to address those challenges. The old system does not indicate in any way which actions constitute a single job or batch. We will consider as a single import batch everything which has the same "import" date (without regard to difference in time of day), topic ("summary") ID, and review cycle ID. The asc_ORIGINAL_SUMMARIES table, one of the two tables in which article-topic associations are recorded, does not have a date/time field, so the Date_input for the bib table is used for the imported_date column of the ebms_import_batch table. For many of the rows in the other legacy table which records article-topic associations, the asc_CITE_SUMMARIES table, the date_added column is NULL, and for the very few cases in which an article-topic combination is recorded with a non-NULL date, and that topic is not recorded for the article in the asc_ORIGINAL_SUMMARIES table, the date_added column appears to be ignored by the user interface displays in the CiteMS. For example, CMS ID 11329 was recorded as in the August 2002 review cycle by the asc_CITE_SUMMARIES table with a date_added value of 2011-04-18. The Date_input from the bib table is 2002-08-20, and I was unable to find the 2011 date anywhere on the CiteMS pages for this article. So the plan is to ignore the date_added column for the asc_CITE_SUMMARIES table even when the column's value is not NULL, and instead use the Date_input value from the bib table for deciding which "input" batch to use. For the new system, exactly one topic for an article is recorded with the disposition "imported" and all other topics are recorded as "topic added." All topics also get a row in the ebms_import_action with the disposition "ready for review." The conversion software will arbitrarily pick the first topic it sees for an article in the asc_ORIGINAL_SUMMARIES table (or from the asc_CITE_SUMMARIES table if there are no rows at all for the article in the asc_ORIGINAL_SUMMARIES table) to get the "imported" disposition in the ebms_import_action table. If there is a row in both the asc_ORIGINAL_SUMMARIES table and in the asc_CITE_SUMMARIES table for the same article-topic combination, and the review cycle differs between the two rows (which happens in about 1,700 cases), the conversion software will use the review cycle it finds in the asc_CITE_SUMMARIES table.

There are instances in the mt_REVIEW table (which is board specific, not topic specific) where the review cycle does not match the cycle appearing in a corresponding row in the asc_CITE_SUMMARIES table (which identifies topics, not boards). This is true for roughly 1% of the rows in the asc_CITE_SUMMARIES table. The plan is to ignore the (board-specific) review cycles in the mt_REVIEW table, since the tables we'll be populating here are topic-specific.

Does anyone see any problems with the approach described here? Will this meet the needs which have been expressed for preserving the review cycle information from the legacy system (and for recording review cycles for future activity in the EBMS)? I realize I'm asking you to absorb and respond to a long and complicated description of what we're suggesting, and I apologize in advance for that, but we're trying to do our best to meet everyone's needs given some fairly significant constraints, not the least of which is a body of legacy data which is poorly documented from the systems side (when documented at all) and which has many serious internal inconsistencies and anomalies.

BZDATETIME::2013-01-07 12:04:05
BZCOMMENTOR::Minaxi Trivedi
BZCOMMENT::292

(In reply to comment #291)
> Here is a proposed approach to carrying over the legacy review cycle
> information from the old system into the EBMS.
> I have two comments

>1. The old system does not indicate in any way which actions constitute a single job or batch. We will consider as a single import batch everything which has the same "import" date (without regard to difference in time of day), topic ("summary") ID, and review cycle ID.

Comment: I just want you to know that import is not always done just in one day. It has varied between 1-7 days (approx.) depending on when I decide to import. Sometimes I run the search for one board, import them and run the search for other board and import next day. Sometimes I run all the searches and import them in one go. Also in the past we had to deal with import errors. In that case the import would resume only after the problem is fixed.

2. Where there is no user interface for specifying a review cycle (for example, on the "full citation" page which allows a new topic to be associated with the article further downstream in the processing of the article) the software will use an algorithm (for example, if the current date is earlier than the 10th day of the current month, use the review cycle corresponding to this month, otherwise use the review cycle for the following month) to decide which review cycle to record.

Comment: If the review by medical librarian extends the algorithm period then we would end up with two review cycles for the same batch of citations.

BZDATETIME::2013-01-07 12:48:52
BZCOMMENTOR::Bob Kline
BZCOMMENT::293

(In reply to comment #292)

> Comment: I just want you to know that import is not always done just in one
> day. It has varied between 1-7 days (approx.) depending on when I decide to
> import. Sometimes I run the search for one board, import them and run the
> search for other board and import next day. Sometimes I run all the searches
> and import them in one go. Also in the past we had to deal with import errors.
> In that case the import would resume only after the problem is fixed.

Whatever level of granularity we use for grouping into a single "import job" I only have a place to record the job as having taken place at a single point in time. For future import jobs, each time you give the system a file you got from NLM a new import job is recorded. We can lump everything together for the same topic and review cycle, and for which the citation was first entered into the legacy system in the same month (as opposed to the same day) if you think that's more useful, but I'll still only be able to record a single date/time value for the job. This doesn't affect which cycle is associated with the article-topic combination. That's converted from the information in the asc_ORIGINAL_SUMMARIES and asc_CITE_SUMMARIES table.

> Comment: If the review by medical librarian extends the algorithm period then
> we would end up with two review cycles for the same batch of citations.

Do you mean "exceeds" (... the algorithm period)? If so, are you saying that we would have to modify all of the places in the system where a new topic could be associated with the article? How did the legacy system handle this when a topic got added to an article late in the game? It looks like there are multiple review cycles for the same article in quite a few cases.

BZDATETIME::2013-01-07 14:43:28
BZCOMMENTOR::Minaxi Trivedi
BZCOMMENT::294

(In reply to comment #293)
> (In reply to comment #292)
>
>
1.Do you mean "exceeds" (... the algorithm period)? If so, are you saying that we would have to modify all of the places in the system where a new topic could be associated with the article? How did the legacy system handle this when a topic got added to an article late in the game?
Comment:

Yes I mean “exceeds”. In the legacy system “review cycle” is just a label. It has no specific start date or end date so “exceeding” is never a problem. In the legacy system, the medical librarian must select the review cycle when adding a new topic during the initial review. In the new system if the review cycle is going to be added automatically during this review process, it is okay if all the reviewing is completed during the algorithm period. However, there will be certain months where significantly more citations are retrieved and will require more time to complete the initial review. In such cases, we will need some sort of mechanism to use the same review cycle for the citations until they get the “Published” status.

2.It looks like there are multiple review cycles for the same article in quite a few cases.

This is because the topics were either imported during different review cycles or the topics were added during different review cycles. And in very few cases it could be an error due to pulling up wrong review cycle while adding the topic.

BZDATETIME::2013-01-07 15:00:57
BZCOMMENTOR::Bob Kline
BZCOMMENT::295

(In reply to comment #294)

> ... we will
> need some sort of mechanism to use the same review cycle for the citations
> until they get the “Published” status.

Dan:

Please modify the pages where the user can add a new topic for an article to ask the user to select the review cycle for the new topic in connection with the article, and give that cycle to Alan's API call when you create the new status. You can seed the selection with the cycle which would be selected by the algorithm described above.

Minaxi:

Will that meet your needs?

And did you decide whether you want me to switch from grouping for legacy "import jobs" by date to group instead by month (I would use the first day of the month as the date of the job), or are you OK with having me group the jobs by date?

BZDATETIME::2013-01-07 15:31:20
BZCOMMENTOR::Robin Juthe
BZCOMMENT::296

(In reply to comment #295)
> (In reply to comment #294)
> > ... we will
> > need some sort of mechanism to use the same review cycle for the citations
> > until they get the “Published” status.
> Dan:
> Please modify the pages where the user can add a new topic for an article to
> ask the user to select the review cycle for the new topic in connection with
> the article, and give that cycle to Alan's API call when you create the new
> status. You can seed the selection with the cycle which would be selected by
> the algorithm described above.
> Minaxi:
> Will that meet your needs?
> And did you decide whether you want me to switch from grouping for legacy
> "import jobs" by date to group instead by month (I would use the first day of
> the month as the date of the job), or are you OK with having me group the jobs
> by date?

Bob:

Margaret and I have gone over your proposed approach and it seems like a good solution to the review cycle problem (better than the one we had proposed). An obvious advantage we see is the ability to rely on reports and searches that include both legacy and new data since the information will be stored in the same way. We had hoped to get away from the need to manually assign a review cycle when adding a topic on the full citation page, in our queue, and when fast tracking, but seeding the selection with the appropriate cycle such that it would only need to be modified if it's incorrect will be a big improvement. Thank you for giving this so much thought.

BZDATETIME::2013-01-07 15:45:43
BZCOMMENTOR::Minaxi Trivedi
BZCOMMENT::297

>
> Minaxi:
>
> Will that meet your needs?

Yes 🙂
>
> And did you decide whether you want me to switch from grouping for legacy
> "import jobs" by date to group instead by month (I would use the first day of
> the month as the date of the job), or are you OK with having me group the jobs
> by date?

Cynthia and I are still discussing this. Will get back as soon as possible.

BZDATETIME::2013-01-07 15:49:57
BZCOMMENTOR::Margaret Beckwith
BZCOMMENT::298

I also want to add to Robin's note that I appreciate all of the time and effort that has been spent on this issue and that the proposed solution seems very good. And I wanted to mention to Cynthia and Minaxi, that while I understand their concerns about being able to rely on legacy data, and also appreciate the time that they have spent in reading all of the comments and responding to them, we may need to change the way we do some things in the new system and we all need to be looking for ways to be flexible in doing that.

BZDATETIME::2013-01-07 16:24:35
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::299

(In reply to comment #295)
> (In reply to comment #294)
> And did you decide whether you want me to switch from grouping for legacy
> "import jobs" by date to group instead by month (I would use the first day of
> the month as the date of the job), or are you OK with having me group the jobs
> by date?

I think it will be ok to group legacy import jobs by date. But instead of the 10th of the month we should use 25th.
For example anything imported from June 25th thru July 25th will be July Review Cycle.

BZDATETIME::2013-01-07 17:12:20
BZCOMMENTOR::Bob Kline
BZCOMMENT::300

(In reply to comment #299)
> (In reply to comment #295)
> > (In reply to comment #294)
> > And did you decide whether you want me to switch from grouping for legacy
> > "import jobs" by date to group instead by month (I would use the first day of
> > the month as the date of the job), or are you OK with having me group the jobs
> > by date?
>
> I think it will be ok to group legacy import jobs by date. But instead of the
> 10th of the month we should use 25th.
> For example anything imported from June 25th thru July 25th will be July Review
> Cycle.

I suspect you've gotten confused because comment #291 proposed solutions to two separate problems related to review cycles. The first (and primary) problem was that we didn't have a review cycle associated with every article-topic combination in the system. That problem is to be addressed by the creation and population of a new table in the system.

The second problem was that the conversion had not tried to populate the import history table, which meant that you couldn't ask to be shown the import reports for jobs associated with a particular cycle and topic. I have proposed ways to work around the difficulties involved in populating those two tables, and one of the difficulties was coming up with batching to form discrete jobs (the old system didn't identify such batches).

Here's the important thing to realize at this point in the discussion: I'm not saying we'll be using the "import date" to decide which review cycle to record for a specific job. It's, in fact, the other way around. I'm saying that if two or more articles where imported on the same day, and were assigned the same topic and the same review cycle, then I will record all of those articles as part of the same import job. So I'm not changing the review cycles, I'm taking whatever review cycles I find in the asc_ORIGINAL_SUMMARIES and asc_CITE_SUMMARIES tables. The only effect of grouping the jobs by actions occurring on the same day versus occurring in the same month is how many articles you'll see in the report for a specific import job. Doing it by day rather than lumping a whole month's worth of imports together, will make the legacy reports look much more similar to the reports you'll get for the imports you do after we switch to the new system.

The business about using an algorithm to have the system automatically assign a review cycle for a new topic association happening elsewhere than in an import action (based on where we are in the month) was not for the legacy data, but for actions that happen after conversion, and (as noted earlier in this thread) we've decided we're not going to use such an algorithm to make the decision, but instead to suggest a cycle, which the user can always override.

Does this make any more sense?

BZDATETIME::2013-01-07 18:09:31
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::301

(In reply to comment #300)
> I suspect you've gotten confused because comment #291 proposed solutions to two
> separate problems related to review cycles. The first (and primary) problem
> was that we didn't have a review cycle associated with every article-topic
> combination in the system. That problem is to be addressed by the creation and
> population of a new table in the system.
> The second problem was that the conversion had not tried to populate the import
> history table, which meant that you couldn't ask to be shown the import reports
> for jobs associated with a particular cycle and topic. I have proposed ways to
> work around the difficulties involved in populating those two tables, and one
> of the difficulties was coming up with batching to form discrete jobs (the old
> system didn't identify such batches).
> Here's the important thing to realize at this point in the discussion: I'm not
> saying we'll be using the "import date" to decide which review cycle to record
> for a specific job. It's, in fact, the other way around. I'm saying that if
> two or more articles where imported on the same day, and were assigned the same
> topic and the same review cycle, then I will record all of those articles as
> part of the same import job. So I'm not changing the review cycles, I'm taking
> whatever review cycles I find in the asc_ORIGINAL_SUMMARIES and
> asc_CITE_SUMMARIES tables. The only effect of grouping the jobs by actions
> occurring on the same day versus occurring in the same month is how many
> articles you'll see in the report for a specific import job. Doing it by day
> rather than lumping a whole month's worth of imports together, will make the
> legacy reports look much more similar to the reports you'll get for the imports
> you do after we switch to the new system.
> The business about using an algorithm to have the system automatically assign a
> review cycle for a new topic association happening elsewhere than in an import
> action (based on where we are in the month) was not for the legacy data, but
> for actions that happen after conversion, and (as noted earlier in this thread)
> we've decided we're not going to use such an algorithm to make the decision,
> but instead to suggest a cycle, which the user can always override.
> Does this make any more sense?

The import reports for specific imports are a valuable addition to the new system that we are looking forward to using. They are going to be especially valuable during the current review cycle to ensure all citation are imported successfully. And once we get rolling in the new system, having access to a few months of import reports will be useful in the event of any importing issue that may occur. Minaxi and I never expected that you would be spending time trying to create similar import reports for legacy data. This took us by surprise. I'm sorry but we do not have a need for import reports of legacy data.

We requested the ability to search and retrieve legacy citations using review cycle either alone or in combination with other search variables. Import reports reflect one file of citations imported during a specified review cycle not everything from a review cycle.
So when you used "import jobs" we assumed you were referring to all imports during a specific review cycle. Sorry for the confusion and for the confusion this post will probably generate as well.

BZDATETIME::2013-01-07 18:32:13
BZCOMMENTOR::Bob Kline
BZCOMMENT::302

(In reply to comment #301)

> Minaxi and I never expected that you would be spending time
> trying to create similar import reports for legacy data. This took us by
> surprise. I'm sorry but we do not have a need for import reports of legacy
> data.

I see. Well, that will reduce some of the work we need to do (though some of it already got done). I believe we got off track in comment #283, where (in reply to my question):

> > Is it OK if reports and queues which filter by cycle don't pick up
> > data from the legacy conversion?

You replied:

> No it’s definitely not ok.

One of the reports which filter by cycle is the Import Report, for which the only way to avoid the "definitely not ok" situation of "not picking up data from the legacy conversion" was to go back and populate the import history tables for the legacy imports.

Oh, well. No real harm done.

BZDATETIME::2013-01-07 18:57:52
BZCOMMENTOR::Alan Meyer
BZCOMMENT::303

(In reply to comment #302)
...
> Oh, well. No real harm done.

And in fact, probably a lot of good was done.

In general, I think it's a Good Thing if the data in the database is more consistent rather than less consistent. In addition to the report you mentioned, there might be other reports or searches we implement in the future that would benefit from having this data.

Obviously, there is a cost for this and paying that cost on mere speculation that it might be repaid in the future might not be justified. It probably all depends on how much of the cost has already been paid and how much more there is to pay.

What do you think might be the level of effort to finish up the work you started and create surrogate batch_import data?

BZDATETIME::2013-01-07 19:17:58
BZCOMMENTOR::Bob Kline
BZCOMMENT::304

(In reply to comment #303)

> What do you think might be the level of effort to finish up the work you
> started and create surrogate batch_import data?

We can convert the data later just as easily as now. Let's defer that until we have a legitimate use case for it, concentrating instead on the things the users have actually told us they need. In fact, it's quite possible that if and when we do have a use for the legacy import data, the knowledge of what that use is would inform that way we do it in a useful way.

BZDATETIME::2013-01-07 19:55:44
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::305

(In reply to comment #302)
> (In reply to comment #301)
> > Minaxi and I never expected that you would be spending time
> > trying to create similar import reports for legacy data. This took us by
> > surprise. I'm sorry but we do not have a need for import reports of legacy
> > data.
> I see. Well, that will reduce some of the work we need to do (though some of
> it already got done). I believe we got off track in comment #283, where (in
> reply to my question):
> > > Is it OK if reports and queues which filter by cycle don't pick up
> > > data from the legacy conversion?
> You replied:
> > No it’s definitely not ok.
> One of the reports which filter by cycle is the Import Report, for which the
> only way to avoid the "definitely not ok" situation of "not picking up data
> from the legacy conversion" was to go back and populate the import history
> tables for the legacy imports.
> Oh, well. No real harm done.

Import report is a new report for the new system. It never occured to be that we would go retro for that. I thought we were discussing only the existing reports used in the legacy system all of which DO need to be filtered by review cycle.

BZDATETIME::2013-01-07 20:05:15
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::306

(In reply to comment #305)
> Import report is a new report for the new system. It never occured to be that
> we would go retro for that. I thought we were discussing only the existing
> reports used in the legacy system all of which DO need to be filtered by review
> cycle.

Sorry by "retro" I mean "go backward".

BZDATETIME::2013-01-07 20:19:01
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::307

(In reply to comment #305)
> (In reply to comment #302)
> > (In reply to comment #301)
> > > Minaxi and I never expected that you would be spending time
> > > trying to create similar import reports for legacy data. This took us by
> > > surprise. I'm sorry but we do not have a need for import reports of legacy
> > > data.
> > I see. Well, that will reduce some of the work we need to do (though some of
> > it already got done). I believe we got off track in comment #283, where (in
> > reply to my question):
> > > > Is it OK if reports and queues which filter by cycle don't pick up
> > > > data from the legacy conversion?
> > You replied:
> > > No it’s definitely not ok.
> > One of the reports which filter by cycle is the Import Report, for which the
> > only way to avoid the "definitely not ok" situation of "not picking up data
> > from the legacy conversion" was to go back and populate the import history
> > tables for the legacy imports.
> > Oh, well. No real harm done.
> Import report is a new report for the new system. It never occured to be that
> we would go retro for that. I thought we were discussing only the existing
> reports used in the legacy system all of which DO need to be filtered by review
> cycle.

And just to be clear here...future import reports also need to be filtered by review cycle once we start generating/collecting them in the new system. We just don't need them generated for legacy imports. This report is really most valuable while importing. Once the citations have been imported successfully, we are done with the report. If some issue arises during importing then these reports will be handy to have to help troubleshoot the issue.

BZDATETIME::2013-01-07 21:20:07
BZCOMMENTOR::Bob Kline
BZCOMMENT::308

(In reply to comment #307)

> And just to be clear here...future import reports also need to be filtered by
> review cycle once we start generating/collecting them in the new system.

Understood.

BZDATETIME::2013-01-15 20:18:12
BZCOMMENTOR::Bob Kline
BZCOMMENT::309

(In reply to comment #277)

I've been asked to work on Alan's outstanding tasks while he's cruising around the world. :-)

> "Include Not Yet Published" implies that these are all citations that are
> waiting to be published or will be published soon. This does not really apply
> to any of the citations excluded by the NOT journal list or even citations that
> I have rejected.

Yes, and I like your replacement for this option (INCLUDE UNPUBLISHED ARTICLES).

> "Include Rejected Articles" ... a board or summary topic need to be required
> variables if this box is checked.

I think I can make this box do something intelligent both whether a board and/or topic is specified or not. Imagine that you have asked to see the articles published in Journal J. Only four articles have been published in the journal:

• article A1, which you approved and published for topic T1

• article A2, which you rejected for topic T1

• article A3, which you approved and published for topic T2

• article A4, which you rejected for topic T2

Using the approach I'm thinking of for supporting this checkbox, here are the results you'd get if you:

• don't check INCLUDE REJECTED ARTICLES & don't specify a topic (or board):

• results A1 and A3

• don't check INCLUDE REJECTED ARTICLES but do specify topic T1:

• results A1

• check INCLUDE REJECTED ARTICLES & don't specify a topic:

• results A1, A2, A3, A4

• check INCLUDE REJECTED ARTICLES & specify topic T1:

• results A1 and A2

Specifying a board (or avoiding a board selection) would behave much the same way.

> "Include NOT List Journals" this will not work unless editorial board is a
> required variable.

Here, too, I think we can give you the additional flexibility of doing something intelligent with this checkbox even if you don't specify a board. Let's consider a search for articles published December 25, 2012. Only four articles were published that day

• article A1, published in journal J1, imported for board B1

• article A2, published in journal J1, imported for board B2

• article A3, published in journal J2, imported for board B1

• article A4, published in journal J2, imported for board B2

• article A5, published in journal J2, imported for boards B1 and B2

Assuming J1 is on the NOT list for board B1 and J2 is on the NOT list for board B2, and that you have approved and published the ones that weren't caught by the NOT lists; if you:

• don't click the box & don't specify a board:

• you get articles A2, A3, and A5

• click the box & don't specify a board:

• you get all five articles

• click the box & specify board B1:

• you see articles A1, A3, and A5

• don't click the box but do specify board B1:

• only articles A3 & A5

> /Unpublished articles---------------------------------------\
> | [ ] INCLUDE UNPUBLISHED ARTICLES |
> | [ ] INCLUDE REJECTED ARTICLES |
> | [ ] INCLUDE ARTICLES EXCLUDED BY NOT LIST JOURNALS |
> --------------------------------------------------------------/

I think this makes sense. If you don't check any boxes, you only get articles that have the "published" state (for the topic or board specified if one has been selected, and filtered by the other criteria specified on the search request form): let's call this the "base set (B)." If you check INCLUDE REJECTED ARTICLES you get the base set plus articles that have the "Rejected in initial review" state (again, for the topic or board specified if one has been selected); so, base set (B) + rejected (R). If you check INCLUDE ARTICLES EXLUDED BY NOT LIST JOURNALS you get the base set plus articles excluded by the NOT list (again, that's the NOT list for the board you specified if you did select a board): base (B) + excluded (E). If you check both of these two boxes you'll get B + R + E. If you check INCLUDE UNPUBLISED ARTICLES you'll get B + R + E as well as articles which are in the "Ready for initial review" state (for a topic or board if appropriate), whether or not the other two boxes are checked.

Does that sound right?

BZDATETIME::2013-01-17 12:42:13
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::310

(In reply to comment #309)
> Does that sound right?
Yes, this looks correct.

BZDATETIME::2013-01-20 16:05:26
BZCOMMENTOR::Bob Kline
BZCOMMENT::311

In the wake of my work on some of Alan's TIRs (#2298 and #2307) I have revised his document to reflect changes in the search behavior. For TIR #2298 (only include published articles by default) I have basically followed the decisions laid out in the thread represented by the following comments in this issue:

Comment 275 [Alan] (Here is some more on searching ....)
Comment 277 [Cynthia] (.... I just have a few comments to add. ....)
Comment 309 [Bob] (I've been asked to work on Alan's outstanding tasks ...)
Comment 310 [Cynthia] (Yes, this looks correct.)

Alan: When you get back in town, please review this draft and let me know if you see anything that's wrong. We'll also want to do a code walk-through for what I've done.

The results of this work won't be seen by the users until they get a CA build which includes a fresh conversion of the legacy data (so, not the upcoming build).

Attachment SearchFunctionality.txt has been added with description: How searching works in the EBMS, 2nd draft

BZDATETIME::2013-01-29 17:11:28
BZCOMMENTOR::Minaxi Trivedi
BZCOMMENT::312

After today’s meeting, Cynthia and I discussed the conversion schedule. We feel that conversion with an empty queue is definitely a good idea. We have come up with a solution to accomplish this without having to stop the work. We will change the dates of March review cycle and make it to 20 days instead of regular 30 days. The remaining 10 days will be incorporated into April review cycle. We will run the searches and save text files. Cynthia will review the text files offline and delete citations that are false drops or citations that are not relevant to summary topic. This will remove at least 10-15% of citations versus 40% rejected regularly. We will also work around to mark the citations that need to be rejected. Once the new system goes live, we will import the text files and publish them.

BZDATETIME::2013-01-29 17:37:30
BZCOMMENTOR::Alan Meyer
BZCOMMENT::313

(In reply to comment #312)
> After today’s meeting, Cynthia and I discussed the conversion schedule. We feel
> that conversion with an empty queue is definitely a good idea. We have come up
> with a solution to accomplish this without having to stop the work. ...

That sounds workable and should solve that part of the problem that has to do with unpublished citations. It will mean that statistics are skewed for the month (that month will have a higher pass/reject ratio than usual and a lower imported count), but that may not matter.

Handling that of the problem seems particularly useful because there are probably more obsolete citations in the system that were never published than obsolete citations in any other state. So clearing the pre-publishing queue would obviate the need for any heuristics for obsolete citations in the initial review states. We'd just mark them all as rejected in initial review with a conversion note saying that they were rejected programmatically in the conversion process.

We'll still need to do the analysis regarding what the issues are with articles in other, post-publishing states, that have not been resolved before conversion.

BZDATETIME::2013-01-29 17:49:13
BZCOMMENTOR::Alan Meyer
BZCOMMENT::314

(In reply to comment #313)
...
> Handling that of the problem ...

Was meant to be:

> Handling that part of the problem ...

BZDATETIME::2013-01-29 18:59:20
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::315

(In reply to comment #313)
> (In reply to comment #312)
It will mean that statistics are skewed for the
> month (that month will have a higher pass/reject ratio than usual and a lower
> imported count), but that may not matter.
I don't think this should be an issue. We are used to stat skews. Each month the number of citations we retrieve can range from 1800-3500. Pubmed may decided to do a bunch of back indexing or incorporate new journals. Or maybe Sharon wants to fast track 800+ citations. But annually these skewed stats will all average out.
The best part about this solution is that we do not have to stop work, we can avoid a big back log of citations and we can stay pretty much on schedule.

BZDATETIME::2013-01-30 18:08:28
BZCOMMENTOR::Bob Kline
BZCOMMENT::316

Here's an initial stab at a plan for allowing the users to continue working in March. As I understand it, the fundamental problem we're trying to avoid is having users faced with queues filled up with articles from the past that should be forgotten in the context of the question "what articles are still awaiting processing?" but didn't get to a terminal state in the legacy system.

In the database where I did my most recent conversion test, there are only five non-terminal states which have any records that are marked "current":

10944 Passed initial review
1224 Published
9344 Passed Board Manager
32617 Passed full text review
667 On agenda

There are other non-terminal states, but with the latest conversion logic, there aren't any "current" records with those states, in part because most of those states are new downstream states which weren't tracked in the old system (for example, "Paper and summary changes for discussion"), and in part because we decided recently that anything that got left in the "Ready for initial review" state would be marked as rejected if it had been waiting for review more than a week at the time of conversion.

I don't think the "On agenda" state presents a problem, so I'll consider the other four in turn.

For the first state above ("Passed initial review"), we had decided earlier that the system would by default ignore rows in the state table with this as the current state for those rows created before conversion. So the Publish Citations page, when it first comes up, only shows articles that got approval from the librarians after the conversion took place, although if a review cycle is selected in the filtering form for the page, the article for the selected cycle which were passed by the librarian, but didn't get a row in the legacy REVIEW table will show up. Also, the librarians have decided that they're going to compress their schedule in March so that by the end of the month everything that's been approved for the current cycle will have been published. So I think we're covered for the "Passed initial review" state.

The state I'm the least confident of in the list of five above is the "Published" state. I'm going to ask Dan to provide a concise summary of the logic used to populate the board manager's queue for articles which are waiting for review of the abstract.

The next state ("Passed Board Manager") will, by default, not cause any articles to show up in the board managers' queues of articles waiting for full text review, because at the point of conversion there will be no articles for which the new system has the full text PDF. Bonnie's queue of articles awaiting retrieval of the full text will also be empty, because by default the system uses the same approach as described above for showing the librarian's queue of articles waiting to be "published" when Bonnie's queue is constructed: if the board manager passed the article based on its abstract before the conversion, the article doesn't appear on Bonnie's list. However, as with the Publishing queue, she can select a review cycle to force legacy articles to show up in the full text retrieval queue. This will allow her to selectively fetch PDFs for the March cycle, without being pestered into the future by having all of the legacy documents showing up in here queue forever. So here, too, I think we have what we need.

Finally, all of the articles in the "Passed full text review" state will also have been assigned to review packets, so none of those articles will appear in the picklist of articles waiting to be assigned to board members for review. And Robin has asked us to mark all of the legacy packets as Archived, so the old packets themselves won't be cluttering up lists of things waiting to be reviewed.

What else do we need to be considering? What have I gotten wrong? If I understand what we're doing, I think we're going to be in good shape.

BZDATETIME::2013-01-31 10:20:39
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::317

(In reply to comment #316)
> What else do we need to be considering? What have I gotten wrong? If I
> understand what we're doing, I think we're going to be in good shape.

Regarding Published state…
We need to distinguished between legacy citations “published and waiting for Board Manager review” from legacy citations “published already reviewed”. So do you mean that citations marked current for the “published” state are published and waiting for Board Manager review?
In the legacy system every citation published is put in someone’s que. Once the citation is reviewed by the board manager then the citation is “published already reviewed”. Since Minaxi and I will be working offline for the March Review Cycle (no new citations will be added until after the conversion) and the Feb review cycle will be finished completely by the conversion date there should be no “active states” or any citations waiting for some decision to be made. After the conversion there should only be “published and already reviewed” citations. So I don’t think the published state will be an issue either as long as all Feb 2013 citations “published and ready for board manager review” are reviewed by the conversion date.

BZDATETIME::2013-01-31 10:25:37
BZCOMMENTOR::Bob Kline
BZCOMMENT::318

(In reply to comment #317)

> Regarding Published state…
> We need to distinguished between legacy citations “published and waiting for
> Board Manager review” from legacy citations “published already reviewed”. So
> do you mean that citations marked current for the “published” state are
> published and waiting for Board Manager review?

Yes. The "Published" state row for the article-topic combination will still exist after the board manager has done her review, but it will no longer be marked as the current state.

BZDATETIME::2013-02-25 18:00:29
BZCOMMENTOR::Minaxi Trivedi
BZCOMMENT::319

EBMS-Testing

When I tried to import “Adult Prostate Cancer” file with 168 citations in TEST mode, it worked fine. But when I tried to import the file, it stopped after few citations and the attached error message appeared. I searched the database and could see only 36 citations of the total 168. I have been able to import other files without any problem.

BZDATETIME::2013-02-25 18:03:41
BZCOMMENTOR::Minaxi Trivedi
BZCOMMENT::320

Here is the import error attachment.

Attachment CiteMS import_error_Feb 25.doc has been added with description: CiteMS import error

BZDATETIME::2013-02-25 18:18:52
BZCOMMENTOR::Alan Meyer
BZCOMMENT::321

(In reply to comment #320)
> Created attachment 2305 [details]
> CiteMS import error
>
> Here is the import error attachment.

I see what's wrong. I'll fix it tomorrow and report back.

Could you also either make an attachment and include the file that caused the error, or else just give me the PMID of the citation that blew up. I'll use it for testing.

Thanks.

BZDATETIME::2013-02-25 18:31:44
BZCOMMENTOR::Minaxi Trivedi
BZCOMMENT::322

Here is the textfile attached.

Attachment adult_prostate_Feb13.txt has been added with description: Prostate cancer text file

BZDATETIME::2013-02-26 16:40:22
BZCOMMENTOR::Minaxi Trivedi
BZCOMMENT::323

EBMS Testing Import statistics

I tested importing 7 files and checked the statistics of import. For 4 files the numbers matched, but for three files the number for “Articles ready for Review” was off by 2-3 numbers. I decided to go in detail of adult skin cancer file. It has the following statistics:
24 unique IDs in the batch
14 articles imported
7 articles NOT listed
10 duplicate articles
21 articles ready for review
10 articles with topic added
0 articles replaced
0 articles with errors

The statistics shows 21 articles ready for review. When I checked the articles ready for review in the queue, it shows 17 articles, which is the correct no.
It is bit confusing to understand these numbers.
I am attaching the file with all PMIDs with my notes.

This is not a high priority at present because it seems to be a report issue, but eventually it needs to be fixed.

BZDATETIME::2013-02-26 16:43:16
BZCOMMENTOR::Minaxi Trivedi
BZCOMMENT::324

Here is the attachment of file with PMIDs and my analysis.

Attachment adult skin statistics.doc has been added with description: Import statistics of adult skin cancer file

BZDATETIME::2013-02-26 17:04:42
BZCOMMENTOR::Alan Meyer
BZCOMMENT::325

(In reply to comment #322)
> Created attachment 2306 [details]
> Prostate cancer text file
>
> Here is the textfile attached.

The bug was caused by a failure of my program to check the length of a field it was attempting to insert in the database.

I believe this is now fixed on verdi, ebms-dev, ebms-qa, and in version control.

The fix will go to ebms-test the next time we migrate files to that server.

I also found a couple of other places where I assumed that the length would be okay and fixed those too. In all of these cases the fields will be truncated, but the full data remains in the full record and should not cause problems.

BZDATETIME::2013-02-26 17:29:14
BZCOMMENTOR::Alan Meyer
BZCOMMENT::326

(In reply to comment #323)
> EBMS Testing Import statistics
>
> I tested importing 7 files and checked the statistics of import. ...

Hi Minaxi,

Can you please attach the file that you used for the import?

Thanks.

BZDATETIME::2013-02-26 17:31:50
BZCOMMENTOR::Minaxi Trivedi
BZCOMMENT::327

Here is the file

Attachment adult_skin_Feb13.txt has been added with description: adult skin cancer file

BZDATETIME::2013-02-27 14:01:17
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::328

I am having a formatting issue with the CiteMS Search the Database page. This issue is unique to my profile or possibly an issue with my computer even though I have tried adjusting my resolution, font size, etc. Nothing seems to correct the formatting issue. All the other pages in the CiteMS are formatting correctly except the Search the Database screen. I will attach a screen shot of what I am seeing.

BZDATETIME::2013-02-27 14:03:11
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::329

Screen shot of formatting issue for search the database screen.

Attachment SearchDatabaseLayoutIssue.docx has been added with description: Search the Database screen shot

BZDATETIME::2013-02-27 14:37:38
BZCOMMENTOR::Bob Kline
BZCOMMENT::330

(In reply to comment #328)
> I am having a formatting issue with the CiteMS Search the Database page. This
> issue is unique to my profile or possibly an issue with my computer even though
> I have tried adjusting my resolution, font size, etc. Nothing seems to correct
> the formatting issue. All the other pages in the CiteMS are formatting
> correctly except the Search the Database screen. I will attach a screen shot of
> what I am seeing.

Can you tell me what happens if you clear the browser's cache? Do this by navigating to the search page (rather than using an existing one, which may have been used for submitting a search previously), then holding down the shift key while clicking on the refresh icon in the browser's address bar. (An alternated method for the last step would be holding down the shift key while pressing the F5 key).

BZDATETIME::2013-02-27 14:43:02
BZCOMMENTOR::Dan Young
BZCOMMENT::331

(In reply to comment #328)
> I am having a formatting issue with the CiteMS Search the Database page. This
> issue is unique to my profile or possibly an issue with my computer even though
> I have tried adjusting my resolution, font size, etc. Nothing seems to correct
> the formatting issue. All the other pages in the CiteMS are formatting
> correctly except the Search the Database screen. I will attach a screen shot of
> what I am seeing.

Hi Cynthia,

I was able to reproduce this formatting error locally - it looks to be specific to Internet Explorer 8 (and perhaps older versions) and the search page. The page will be styled correctly once the updated file is pushed to CA. Until then, I hope the page is usable, if formatted strangely.

• Dan

BZDATETIME::2013-02-27 15:51:28
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::332

(In reply to comment #330)
> (In reply to comment #328)
> > I am having a formatting issue with the CiteMS Search the Database page. This
> > issue is unique to my profile or possibly an issue with my computer even though
> > I have tried adjusting my resolution, font size, etc. Nothing seems to correct
> > the formatting issue. All the other pages in the CiteMS are formatting
> > correctly except the Search the Database screen. I will attach a screen shot of
> > what I am seeing.
> Can you tell me what happens if you clear the browser's cache? Do this by
> navigating to the search page (rather than using an existing one, which may
> have been used for submitting a search previously), then holding down the shift
> key while clicking on the refresh icon in the browser's address bar. (An
> alternated method for the last step would be holding down the shift key while
> pressing the F5 key).

Nothing changes when I do this.

BZDATETIME::2013-02-27 15:53:09
BZCOMMENTOR::Bob Kline
BZCOMMENT::333

(In reply to comment #332)

> Nothing changes when I do this.

Thanks for checking. This appears to be an issue with older versions of Internet Explorer. I believe Dan has a workaround for the problem.

BZDATETIME::2013-02-27 15:53:52
BZCOMMENTOR::Cynthia Boggess
BZCOMMENT::334

(In reply to comment #331)
> Hi Cynthia,
> I was able to reproduce this formatting error locally - it looks to be specific
> to Internet Explorer 8 (and perhaps older versions) and the search page. The
> page will be styled correctly once the updated file is pushed to CA. Until
> then, I hope the page is usable, if formatted strangely.
> - Dan

The page still works as long as I put the correct data in the right line which can be tricky as nothing lines up correctly.