Sneak Preview: More Efficient Image Sequence Handling

By Isak - June 17, 2016 (Last updated: April 9, 2017)

Another sneak preview before the release of Vidispine 4.6. It might be the last before release, but keep your eyes open for another one to be sure. Vidispine 4.6 contains an important database schema change when it comes to handling image sequences. The result is much better efficiency and smaller database.

OK, we admit it. The old way of handling image sequences files was really inefficient. For an hour of 25 fps content, that is 90k rows for the files in the database, 90k components, 90k times X number of component metadata. You see where this is heading?

With 4.6, a file sequence is an entity on its own. So, a file sequence is a file, it’s just represented as multiple physical files on the file system. See what we mean? No? An example.

In order for file sequences to be recognized on a Storage, add a <sequence> element to the storage definition. Like this:

<StorageDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  ...
  <sequence>
    <regex>.*-(\d)+.png</regex>
  </sequence>
</StorageDocument>

Now copy three files to it, foo-00001.png, foo-00002.png, foo-00003.png.

<FileDocument xmlns="http://xml.vidispine.com/schema/vidispine">
    <id>VX-1702</id>
    <path>foo-0000/.png</path>
    <uri>file:/srv/media/foo-0000*.png#file=1-3</uri>
    <state>CLOSED</state>
    <size>-1</size>
    <timestamp>2016-06-16T15:15:36.303+02:00</timestamp>
    <refreshFlag>1</refreshFlag>
    <storage>VX-1</storage>
    <metadata/>
    <range count="3" start="1"/>
    <type>FILE_SEQUENCE</type>
</FileDocument>

Whoa! One FileDocument with lots of new stuff to it. As you can imagine, if you want to import this sequence, all you need to do is

POST /API/storage/file/VX-1702/import

Much easier. And of course, when you show the item

<ItemDocument xmlns="http://xml.vidispine.com/schema/vidispine" id="VX-526">
  <shape>
    <id>VX-726</id>
    <essenceVersion>0</essenceVersion>
    <tag>original</tag>
    <mimeType>image/png</mimeType>
    <containerComponent>
      <file>
        <id>VX-1702</id>
        <path>foo-0000/.png</path>
        <uri>file:/srv/media/foo-0000*.png#file=1-3</uri>
        <state>CLOSED</state>
        <size>-1</size>
        <timestamp>2016-06-16T15:15:36.303+02:00</timestamp>
        <refreshFlag>1</refreshFlag>
        <storage>VX-1</storage>
        <metadata/>
        <range start="1" count="3"/>
        <type>FILE_SEQUENCE</type>
      </file>
      ...

File sequences work for API import, auto-import, copy jobs. Not yet for raw import.