CDR Tickets

Requires discussion of how this fits into the Schema, configuration, functional requirements

Based on discussions with Margaret, we will need video metadata from the media doc in CDR so that it can be sent to Percussion for reuse across cancer.gov.

This would include the video's youtube ID, and other metadata.

The other components to this story will include:

• Having GK create a glovideo with the video metadata so that it can be reused in other Percussion content (not PDQ summaries).

• A video placed in PDQ Summary would be separate from the glovideo content type, and we would use a filter to put the correct info into the PDQ summary, and we would not use a Percussion inline link to the video.

From a business rule standpoint: when a video needs to be deleted from CDR - the CDR team will need to open a CGOV ticket so that we can do an impact analysis to remove it from the percussion content so that it will not affect other content on Cancer.gov that may be reusing the video. This follows the same process as a PDQ.

... we will need video metadata ....

Here are some possible things we could capture (see documentation):

• width

• height

• autoplay

• start

• end

• loop

• showinfo

• rel

• playsinline

• playlist

• modestbranding

• cc_load_policy

• color

• controls

• disablekb

• enablejsapi

• fs

• iv_load_policy

• frameborder

Some further issues to be thrashed out:

1. Will we want to adapt the Media document type, which was originally intended for use with multimedia which is actually stored in the CDR, or come up with a new document type which does not expect to have a blob associated with it?

2. Which metadata items should be stored with the base document for the video clip? Which of them should be stored in the element in the document in which the video will be embedded (the "using" document). Which should be allowed in both places (with the values in the "using" document overriding those found in the base document for the video clip)?

3. Which items should be required and which have defaults (and what should the defaults be)?

I'm taking this out of the "in progress" column. Perhaps we should move this to the second iteration since we still have a lot of unanswered questions.

~bkline, can you put together a proposed technical approach that can be reviewed with the WCMS team? Since there are dependencies, it would be important to understand the impacts of proposed changes in the CDR, as well as brainstorm alternatives if needed. Let me know approximately when you think you could be ready for a discussion, and I'll setup a meeting. I'll move this into Iteration 2.

I'm pretty sure I can get something ready for this Thursday, and for sure no later than next Tuesday.

I'm starting out with a simple approach, supporting roughly the same level of functionality as we have for embedded video in the EBMS. Once we've settled on what will be supported downstream (i.e., GateKeeper and cancer.gov) I'll come up with a proposed set of schema changes for the CDR.

The initial solution adds a new element to the DTD named EmbeddedVideo, with the following attributes:

• source - initially (forever?) only "YouTube" is supported (required)

• caption - string without markup for display above the video (optional)

• width - integer specifying the width of the iframe containing the video in pixels (required)

• height - integer specifying the height of the iframe containing the video in pixels (required)

• unique_id - string used by the video hosting service to identify the video (required)

Here's an example.

<EmbeddedVideo
    source="YouTube"
    caption="NCI Video Journey Into Nanotechnology"
    width="400"
    height="300"
    unique_id="jC8CUIID2HA"/>

The element would be allowed wherever a table could appear within the document types for which embedded video will be supported (summaries and glossary terms?). This would be translated into HTML which wraps the optional caption and iframe for the video within a div element. For example:

<div class="embedded-video">
  <p>NCI Video Journey Into Nanotechnology</p>
  <iframe 
    width="400"
    height="300"
    frameborder="0"
    src="https://www.youtube.com/embed/jC8CUIID2HA?showinfo=0"
    allowfullscreen="false"></iframe>
</div>

Here's a sample of how it might look on a dictionary page (I would have used a summary but summary publish preview is not working on DEV right now):

http://cdr-dev.cancer.gov/nanotechnology.html

So CDR will need size, source, unique ID, title, caption (not required). We would need to add a layout style to choose whether or not the title is displayed

For captions - whatever is supported for images will be similarly supported for videos.

~padillasr - Can you provide the percussion templates for videos?

Cancer.gov Code:

 
<figure class="video center size100">
   <h4>(TITLE)The Outstanding Investigator Award (R35)</h4>
   <div id="ytplayer-LC8g5egkvz8" class="flex-video widescreen" data-video-id="(UNIQUE_ID)LC8g5egkvz8" data-video-title="(TITLE)The Outstanding Investigator Award (R35)">
   <noscript>       
      <p>
         <a href="https://www.youtube.com/watch?v=(ID)LC8g5egkvz8" target="_blank" title="(TITLE)The Outstanding Investigator Award (R35)">View this video on YouTube.</a>
      </p>
   </noscript>
  </div>
   <figcaption class="caption-container">(CAPTION)
<p>NCI established the Outstanding Investigator Award (R35) in 2015 for principal investigators who have achieved significant research accomplishments. Featured in this video is Holly Prigerson, Ph.D., of Weill Cornell Medicine, who is among the first group of 62 researchers to receive the award.</p>
   </figcaption>
</figure> 

Available video template options in Percussion when content owners currently insert a video are:

• Video100NoTitle

• Video100Title

• Video50NoTitle

• Video50NoTitleRight

• Video50TitleRight

• Video75NoTitle

• Video75Title

Haven't heard back from anyone about which of the %TextElements; (from the DTD) GateKeeper supports for the Caption elements, so I wrote some scripts to find out what's in the exported documents right now for the existing Caption elements. Here's what I found:

Caption
  @language
  GeneName
  Reference
    @refidx
  Emphasis
  ExternalRef
    @xref
  ScientificName
  Superscript

So that's what I propose we support for the Caption children of EmbeddedVideo elements:

<!ELEMENT EmbeddedVideo (VideoTitle?, Caption*)>
<!ATTLIST EmbeddedVideo
       hosting CDATA #FIXED "youtube"
       unique_id CDATA #REQUIRED
       template (Video100NoTitle|Video100Title|Video50NoTitle|Video50NoTitleRight|Video50TitleRight|Video75NoTitle|Video75Title)>
<!ELEMENT VideoTitle (#PCDATA)>
<!ELEMENT Caption (#PCDATA | GeneName | Reference | Emphasis | ExternalRef | ScientificName | Superscript | Strong | Subscript)>

We could redefine Caption as above (more restrictively) or leave it as it is in the DTD, and just continue to support only the subset by agreement as before. I threw in "Strong" and "Subscript" for good measure (and symmetry).

If this works for GateKeeper and further downstream (and for the users), I'll work on the upstream schema and filter changes.

N.b.: I have changed the name of the 'source' attribute to 'hosting' (to avoid conceptual confusion with the internal information on the original source for the video).

~sunvw and ~volker: Will the proposed structure work for GateKeeper and further downstream? Do we have to retain the "image-" portion of the names for the size values?

Thanks,
Bob

~bryanp - Can you take a look at the proposed structure?

can you determine whether we must keep the "image-" portion of the enumerated size value names?

The image-* portion is used to specify the size using CSS. I would assume we'd want to use the same or similar size attributes and replacing image with video or so. This will require for the CSS to be updated and likely - but I'm not certain about this yet - the Gatekeeper XSLT.

We'd like to include an element to record the duration of the video in the CDR record (for our own internal use--not for publishing downstream).

Bob is going to start detailing the elements that will be included in the internal CDR document (looking at the media doc schema as a starting point).

~bryanp - bump.

Figure classes on templates:

• Video100NoTitle: "video center size100"

• Video100Title: "video center size100"

• Video50NoTitle: "video center size50"

• Video50NoTitleRight: "video right size50"

• Video50TitleRight: "video right size50"

• Video75NoTitle: "video center size75"

• Video75Title: "video center size75"

I have done some further investigation, and I believe it will be possible to prevent the publishing software from trying to export video files for the Media documents used for videos which are externally hosted (which would fail, because there won't be a data blob for these documents). That means we can use the existing Media document type for this issue instead of creating a separate document type for media not stored in the repository. As I recall, using the existing document type was the approach preferred by the users. If I've missed something which would make this approach unworkable, I believe we would be able to switch to using a separate, new document type for the embedded video metadata without too much difficulty.

Here is what I've added to the common schema base document:

  <!-- Link to a Media document for a video hosted on YouTube,
       to be embedded in a player UI in the rendered linking
       document. The title, caption, and presentation template
       default values can be overridden here for this specific
       usage. See https://tracker.nci.nih.gov/browse/OCECDR-3558. -->
  <complexType       name            = 'EmbeddedVideo'>
    <sequence>
      <element       name            = 'VideoID'
                     type            = 'MediaID'/>
      <element       name            = 'SpecificMediaTitle'
                     type            = 'string'
                     minOccurs       = '0'/>
      <element       name            = 'SpecificMediaCaption'
                     type            = 'SpecificMediaCaption'
                     minOccurs       = '0'/>
      <element       name            = 'SpecificVideoPresentationTemplate'
                     type            = 'VideoPresentationTemplate'
                     minOccurs       = '0'/>
    </sequence>
    <attribute       name            = 'cdr:id'
                     type            = 'string'
                     minOccurs       = '0'/>
  </complexType>

  <!-- Mirrors the MediaCaption type in Media.xml, but without
       the audience and language attributes. This is used as
       part of an EmbeddedVideo block used in a CDR document
       which links to a Media document for a video. -->
  <complexType       name            = 'SpecificMediaCaption'
                     mixed           = 'true'>
    <choice          minOccurs       = '0'
                     maxOccurs       = 'unbounded'>
      <group         ref             = 'PhraseOrParaElements'/>
      <group         ref             = 'CommonTextElements'/>
    </choice>
  </complexType>

  <!-- Controls three aspects of the embedded video's presentation:
       positioning, size, and whether the title is displayed.
       These are the allowed combinations. See the WCMS documentation
       for further details. -->
  <simpleType        name            = 'VideoPresentationTemplate'>
    <restriction     base            = 'string'>
      <enumeration   value           = 'Video100NoTitle'/>
      <enumeration   value           = 'Video100Title'/>
      <enumeration   value           = 'Video50NoTitle'/>
      <enumeration   value           = 'Video50NoTitleRight'/>
      <enumeration   value           = 'Video50TitleRight'/>
      <enumeration   value           = 'Video75NoTitle'/>
      <enumeration   value           = 'Video75Title'/>
    </restriction>
  </simpleType>

And here are the changed/extended parts of the Video schema:

  <complexType       name            = 'VideoData'>
    <sequence>
      <element       name            = 'VideoType'
                     type            = 'VideoType'/>
      <element       name            = 'VideoEncoding'
                     type            = 'VideoEncoding'/>
      <element       name            = 'Color'
                     type            = 'Color'/>
      <element       name            = 'RunSeconds'
                     type            = 'integer'
                     minOccurs       = '0'/>

      <!-- Next three elements are new for OCECDR-3558 -->
      <element       name            = 'VideoHosting'
                     type            = 'VideoHosting'/>
      <element       name            = 'HostingID'
                     type            = 'string'/>
      <element       name            = 'DefaultVideoPresentationTemplate'
                     type            = 'VideoPresentationTemplate'
                     minOccurs       = '0'/>
    </sequence>
  </complexType>

  <!-- new for OCECDR-3558 -->
  <simpleType        name            = 'VideoHosting'>
    <restriction     base            = 'string'>
      <!-- lowercase, following the lead of the URLs, not mixed case
           like the display on their web pages. -->
      <enumeration   value           = 'youtube'/>
    </restriction>
  </simpleType>

The users will probably want to add some new media processing status values, and possibly some new category values as well. Let's discussion these proposed schema changes (as well as decide on the document types to which we should add the new EmbeddedVideo element) in tomorrow's CDR status meeting.

I realized that I could only go so far in restricting the Caption definition, because the elements allowed for the markup we will support are themselves defined very broadly, and there's no way to block recursive use without breaking documents which use such definitions for markup of blocks other than Captions. We'll have to put off a more nuanced approach to the schema definition for the day when we replace our own implementation for XML schema (based on a pre-release draft of the standard) with a standards-compliant implementation.

~volker or ~bryanp or ~padillasr: does one of you have a specific reference I can plug in above for the semantics of the template names where I have the "see the WCMS documentation ..." comment above? The users will want to know, for example, what the numbers mean. Thanks!

I'm certain the numbers represent percentages. However, I don't know if a video displayed as 100% is the same size as an image displayed at full size. I'll let you know once I find a sample video on Cancer.gov.

Bob will add the EmbeddedVideo element block to summaries, glossary term concept docs, and drug information summaries.

Margaret, William, and Robin will look at the media schema to determine whether additional categories and/or processing statuses are needed.

I prepared a sample summary document with different video sizes for Tuesday's meeting:
file:///L:/OCPL/_Cross/CDR/temp/Carcinoma-summary2.htm
(Please ignore the messed up mega menu on that page.)

Here is a sample dictionary document:
file:///L:/OCPL/_Cross/CDR/temp/metastasis-DictionaryTerms2.htm

~bkline, it appears that the DTD changes you've listed don't allow for a title to be included.

Good catch – fixed.

Do you know if the question about markup in the title is still up in the air? Can you (~volker) confirm that the users are on board with the restriction that it be plain text, and if not, negotiate with the WCMS team to see what they would be willing to support?

Thanks.

I'm no Percussion expert but it appears to me that the title field in Percussion for the video is limited to ASCII. Therefore this limitation would be of technical nature regardless of the user's wishes.

We do have a Percussion expert on the watch list (~learnb) and he will certainly correct me if I'm wrong.
(Go ahead, Blair, I can take it. :-) )

From the requirements perspective - I would just like to add that in order for videos to be consistent across the site (and since we do not allow markup for titles on Cancer.gov), it would be consistent for CDR to also not send over special markup for titles (or if it was sent over, it would be stripped out)

Thanks, Victoria. That's sort of the answer I was expecting.

OK, you're wrong. :-)

Percussion's fields are stored as Unicode, not ASCII. It's a bit of hair-splitting, but potentially relevant depending on what the users want to do with the field.

I'd have to do some digging to tell you how a specific content type uses a given field, but for most, it's "plain text" in the sense that Unicode characters such as ñ, ® and the like are allowed. Markup is generally limited to body fields.

Schema changes are on DEV (css still pending). CDR258011 now links to the new test video Media document CDR779579.

Margaret, William, and Robin will look at the media schema to determine whether additional categories and/or processing statuses are needed

Which means the schema changes I just installed are provisional.

~bkline, when I'm trying to open the test summary I'm getting an error message in XMetaL: "Text not allowed here."
The cursor is positioned within the EmbeddedVideo tags.

Hi ~juther, it looks like we will need a ticket to adjust the CSS in XMetaL for the new Video content.
The elements VideoHosting, HostingID, DefaultVideoPresentationTemplate, etc. need to be setup.

Ah, we can't have the cdr:ref attribute directly attached to the EmbeddedVideo element, because if we do, the CDR server will throw away the child elements are replace them with the string for the constructed title of the linked document. What I will have to do is to add a VideoLink element as a child of the EmbeddedVideo block and put the cdr:ref attribute there. Will let you know when the schema and the document are fixed.

Give it a try now.

Perfect! Now we're cooking with gas!

Shouldn't the EmbeddedVideo element also receive a cdr:id? Please?!
It's being converted into a MediaLink for the vendor output and will be easier to set it in the CDR than through the filters.

Sure, why not? Added.

MediaLink and not a VideoLink? Are they going to expect to find an exported media file somewhere?

Just checking if you're paying attention. :-)
It's going to be a EmbeddedVideo element in the vendor output following your DTD changes above.

Do we know if the VideoTitle and/or Captions will only be present in English or should be expect Spanish text as well?

I think we should expect Spanish as well.  We ended up translating everything for the metastasis animation.

Thanks,
Margaret

From: NCI JIRA Mail (NIH/NCI)
Sent: Monday, February 06, 2017 4:31 PM
To: Beckwith, Margaret (NIH/NCI) [E] <mbeckwit@icic.nci.nih.gov>
Subject: [NCI JIRA Tracker] (OCECDR-3558) Animation in CDR

[1]Englisch, Volker (NIH/NCI) [C] commented on [2]OCECDR-3558

[3]Re: Animation in CDR

Do we know if the VideoTitle and/or Captions will only be present in English or should be expect Spanish text as well?

[4]

[5]Add Comment

This message was sent by Atlassian JIRA (v7.2.0#72002-sha1:36e1562)

 
----------------------------------------------------------------------------------------
[1] https://tracker.nci.nih.gov/secure/ViewProfile.jspa?name=volker
[2] https://tracker.nci.nih.gov/browse/OCECDR-3558
[3] https://tracker.nci.nih.gov/browse/OCECDR-3558
[4] https://tracker.nci.nih.gov/browse/OCECDR-3558#add-comment
[5] https://tracker.nci.nih.gov/browse/OCECDR-3558#add-comment

I'm wondering if this is a complete list:
For the size50 we're using right, right-noTitle, and center-noTitle but there is no template for center with title.
Is this correct?

An EmbeddedVideo element is currently only allowed as a shared media between English and Spanish but not as a media for each individual concept. Is this correct?

I expected the new element to be used like a MediaLink and be allowed to be inserted either as a shared element or split into each concept. The later makes more sense to me because the video will always be targeted to one language unless Spanish subtitles exist.

Starting a checklist of what needs to happen to put this ticket to bed.

Done
✔ Install schema changes
✔ Add entries to link table
✔ Add sample video for review
✔ Modifications to export filter

To be done
❗ Add CSS rules for XMetaL
❗ Testing of published documents with video
❗ Update the DTD files pdqCG.dtd and pdq.dtd (CDR and GateKeeper)
❗ Update GateKeeper filters for Summary, GlossaryTerm and DrugInformationSummary docs
❗ Republish documents (explain to me again why we need this, ~volker)
❗ Modifications to publishing control queries

I have merged in your additions from the following comment, ~volker. Please review the lists again.

Thanks.

We also need to:

• update the DTD files pdqCG.dtd and pdq.dtd

On the Gatekeeper side we need to

• Update filter changes for Summaries, GlossaryTerms, DrugInformationSummary

• Update DTD

• Republish documents

Bob will add the EmbeddedVideo element block to the GTC English and Spanish term definition blocks.

Please add the following category to the media document schema: biological process.

Both requested changes have been made.

~volker: I'm looking at the publishing control queries in order to make sure the publishing system doesn't choke trying to export media documents for which the CDR doesn't really have the media, and I think the best way to accomplish this goal is to add a join on the query_term_pub table for paths of /Media/.../ImageEncoding and /Media/.../SoundEncoding. The ImageEncoding and SoundEncoding elements are required for image and sound media documents respectively, so this should work. It doesn't yet, though, because not all of the Media documents in the pub_proc_cg table match the modified query. I assume that this is because although the query_term_def table has both of these paths, they must have been added after many of the Media documents were last saved. I will verify this theory by running a re-indexing job on DEV for all of the Media documents, and comparing the queries with and without the join I'm proposing.

Please give this proposal some analysis and tell me if you agree that it's the right approach.

Are the ImageEncoding and SoundEncoding elements allowed for video?
I was thinking along these lines of restricting the queries, too. I am a little surprised not all images are indexed for these paths but it's possibly we've never run a full re-index job for Media on DEV since those paths have been added..

Each of those elements are in a separate block for the type of physical media, and a document can have only one of those blocks and still be valid (and publishable).

So, let's assume my explanation of the gaps is correct (I'm still going to confirm it): can you see any holes in the approach I'm proposing?

For my own notes: I have already taken care of the new link and query term indexing rules for this ticket on all four tiers.

I think that's the right approach.

I have installed the modified publishing control document in svn and on DEV. At some point, ~volker, I'll need you to verify that the right media documents still get published (probably as part of the test publishing jobs you'll have to run for this ticket anyway).

Thanks.

I published three pages to BLUE which are including summary videos displayed with different templates:

• https://www-blue-dev.cancer.gov/types/bladder/patient/bladder-treatment-pdq#section/all

• https://www-blue-dev.cancer.gov/types/small-intestine/patient/small-intestine-treatment-pdq#section/all

• https://www-blue-dev.cancer.gov/types/small-intestine/hp/small-intestine-treatment-pdq#section/all

Awesome! Danke sehr.

~buracklb - If you have some time, please take a sneak peek at the summaries on Blue with videos and verify they look as expected from the IA/Design perspective. Thanks!

So I think this would be the current state of the ticket:

✔ Install schema changes
✔ Add entries to link table
✔ Add sample video for review
✔ Modifications to export filter
✔ Modifications to publishing control queries
✔ Add CSS rules for XMetaL
✔ Testing of published documents with video (still ongoing, I see)
✔ Update the DTD files pdqCG.dtd and pdq.dtd on the CDR server (@volker: please check what I did)
✔ Update the DTD files pdqCG.dtd and pdq.dtd on GateKeeper
✔ Update GateKeeper filters for Summary, GlossaryTerm and DrugInformationSummary docs

Turning this over to you, ~volker, as the remaining work is mostly under your control.

Thanks!

The three pages Volker pushed to Blue look good to me. STRONG WORK!

I tested the new EmbeddedVideo block in the CDR and the caption is now displaying as expected in Publish Preview. Thanks, Volker!

The following filters have been updated to process the new EmbeddedVideo elements:

• R14641: CDR000157.xml (Vendor Filter: Summary)

• R14641: CDR271370.xml (Module: Vendor Filter Templates)

• R14641: CDR410709.xml (Module: Multimedia Denormalization)

• R14641: CDR616047.xml (Denormalization Filter: GlossaryTermName)

• R14641: CDR616048.xml (Vendor Filter: GlossaryTermName)

Both DTD files have been updated in subversion:

• R14649: pdq.dtd

• R14649: pdqCG.dtd

Should I expect to see the templates at work in PP? So far, it doesn't seem to be working well as the samples you provided above.

It depends which document type you're trying. It will work for summaries but not for glossaries. As discussed last week, the glossaries require a Gatekeeper release.

Yes, I am aware of that and I am testing only summaries. Just to clarify, the videos are displayed correctly. However, I expect to see the video to go to the right of the page if I select a template with the word "right" in it but it doesn't appear to be the case when testing. Her.e is a sample summary on DEV CDR0000269044

I'm assuming you have your browser window too narrow. Only on the desktop size will the video be displayed small and to the right. On tablet and lower device sizes the display will always be full size.

Your sample looks good on my screen.

~volker: I just looked at your version of the DTDs, and I noticed that it allows EmbeddedVideo as inline markup. Was that intentional?

That is right. I am testing on a 12" laptop :-). Thanks!

That's an oversight. I mimicked the definition of the MediaLink but we don't use the thumbnails anymore.

The PDQ DTD has been fixed. The EmbeddedVideo element has been removed as an inline element.

@volker: as soon as this last ticket is marked as Resolved, I'll do the deployment to QA.

Thanks.

Margaret and I created a test media doc for the metastasis video on QA (CDR786047).

We had a few questions/comments about the schema:

1. File Type - the animations Terese is sending are mp4 files. The available options are AVI, MJPG, and MPEG2. Could we add MP4 to the picklist? (or is that the same thing as one of these formats?)

2. Proposed Use - could we add an optional "ExternalRef" element to this block? Many of the animations will likely be proposed for use on other Cancer.gov pages, so it would be good to be able to keep track of that information within the Media Doc.

3. Video Type - what is the intended purpose of this element? We filled it in with "animation" for test purposes, but we weren't too sure what would go here. We may eventually want to come up with a picklist as opposed to using free-text for this field.

We added the video to a summary (CDR350260) using the EmbeddedVideo block, but it is not displaying in Publish Preview. This may have something to do with #1 above.

Thanks!

We added the video to a summary (CDR350260) using the EmbeddedVideo block, but it is not displaying in Publish Preview.

It appears that the filters did not get updated as part of the build process. Once the filters have been updated you should be able to view the video.

I have added MPEG4 to the list of valid values for file type, and I added ExternalRef to the ProposedUse block (DEV and QA). Here's what Alan wrote about VideoType:

  <simpleType        name            = 'VideoType'>
    <restriction     base            = 'string'/>
    <!-- Enumeration to be determined.
         We many want to start out without the restriction of a
         valid values list, as we have done for audio media. -->
  </simpleType>

Yes, the video is displaying now. Thanks, Volker!

Verified on QA.