Native AWS Elemental MediaConvert in Vidispine API Through Vidinet

By Patrik - October 29, 2018 (Last updated: November 4, 2018)

AWS Elemental Media Services are cloud-based services with pay-as-you-go pricing which provide customers the capability to ingest, process, package and deliver video content at scale. Vidispine has added MediaConvert as a Vidinet service and made it natively integrated with the Vidispine API, with MediaConvert output presets available to use for output transcoding.

Vidinet, Vidispine API and AWS Elemental MediaConvert

Vidinet allows customers to use AWS Elemental MediaConvert directly from the Vidispine API, with all AWS Elemental output system presets accessible through the Vidispine API as native transcode presets. MediaConvert can be used for all output transcoding in the Vidispine API. Running AWS Elemental transcoding jobs through the Vidispine API also simplifies price calculations using the unique cost estimation feature in the Vidispine API.

In this blog post, we’ll walk you through what this means in terms of functionality in the Vidispine API, from setting up AWS Elemental MediaConvert to doing an actual transcode using MediaConvert from the Vidispine API.

MediaConvert transcoding and presets

You will have access to all existing system output presets from AWS Elemental MediaConvert in the Vidispine API. An AWS Elemental MediaConvert transcoder will automatically be chosen if you select to transcode using one of the MediaConvert presets (see note in Developer Preview section).

Cost estimates / pricing

Prices for MediaConvert transcodes are the same as you would get using MediaConvert directly, with the same pay-as-you-go pricing structure with a per minute charge.

As you may know, it is possible to ask the Vidispine API for a cost estimate on any Vidinet media service, such as transcoding and QC. The cost estimate is of course also available for AWS Elemental MediaConvert jobs. Usually cost estimates are calculated on your source material, using bitrate as a base. With MediaConvert we instead calculate the cost based on the target (output) preset, using the same factors as AWS Elemental uses, such as codec, resolution, frame rate and quality of output.

Note: This is not your AWS Elemental MediaConvert account that is connected to the Vidispine API. MediaConvert executes on Vidispine’s AWS account and you will be billed through us.

Developer Preview

What we are releasing now is a Developer Preview that has some limitations in what MediaConvert presets you can use, functionality and regions you can choose (Ireland only).

While you have access to all MediaConvert system output presets from the Vidispine API, we don’t support all of them yet. When trying to do a cost estimate, or transcode using one of the not supported presets, Vidispine will return an error with an explanation.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<CostEstimateDocument xmlns="https://xml.vidispine.com/schema/vidispine">
  <id>VlgtNj1mYmU1OWE2NC00MDA0LTQzNjYtODE4Ny1lN2ZhM2E2ODZkOTE=</id>
  <url>https://my-vidispine-api.myvidispine.com/API/cost/estimate/VlgtNj1mYmU1OWE2NC00MDA0LTQzNjYtODE4Ny1lN2ZhM2E2ODZkOTE=</url>
  <state>FINISHED</state>
  <service>
    <resource>VX-6</resource>
    <name>My Elemental MediaConvert Transcoder</name>
    <type>ELEMENTAL_MEDIACONVERT</type>
    <state>FAILED</state>
    <message>Can't provide cost estimate, specified jobCriterion 'H_265' isn't accepted for job type ELEMENTAL_MEDIACONVERT</message>
  </service>
</CostEstimateDocument>

 

Specifically, we don’t support the following codecs and features in MediaConvert:

  • HEVC (codec)
  • HLS (container)
  • Dash (container)
  • Audio Only, Dolby Audio, Audio Normalization (features)

The Vidispine Knowledge Base contains an updated list of supported MediaConvert presets to be used as a reference.

For now, you also have to specify the Vidinet resource you want to use when transcoding using MediaConvert. See the following knowledge base article to learn how to specify the resource.

Because AWS Elemental MediaConvert only works with S3 buckets in the same region as MediaConvert is running in, you can only use items on a source storage in the same region, and you also have to specify the destination storage using custom job metadata. To specify a transcoding destination storage use the following custom job metadata when executing your transcode job:

&jobmetadata=destinationStorageId%3dVX-8

In this case, I have set the destinationStorageId to VX-8 (you need to encode the = to %3d.

Setting up Elemental

Ok, let’s get started with MediaConvert. To be able to use AWS Elemental MediaConvert in Vidispine API you need to add the MediaConvert service to your Vidinet account.

AWS Elemental MediaConvert in Vidinet

Select the service, give it a name and select your region (AWS eu-west-1 (Ireland)). When you launch the service, you will be presented with instructions on how to set up the service. I’ll go through those instructions here.

First, make sure you have set up the AWS S3 bucket(s) that you need to use as storage in Vidispine API, as MediaConvert only works with storages that are S3 buckets. When you have set up your storage, you need to add a new bucket policy according to the knowledge base article, or as in the instructions.

Next step is to connect your AWS Elemental MediaConvert service to you Vidispine API. Use the provided access keys to create the resource, per the published ResourceDocument.

POST /API/resource/Vidinet 

<ResourceDocument xmlns="https://xml.vidispine.com/schema/vidispine">
  <vidinet>
    <url>vidinet://{accesskey}:{secretkey}@ce4e3be7-7dac-42fa-8151-ac4fb119cb8e</url>
    <endpoint>https://services.vidinet.net</endpoint>
    <type>ELEMENTAL_MEDIACONVERT</type>
  </vidinet>
</ResourceDocument>

Almost there, now it’s time to add all MediaConvert presets to your Vidispine API instance as shape-tags containing the mediaconvert element. To install MediaConvert system presets use:

PUT /APIinit/preset-mediaconvert-templates

With that done you have access to MediaConvert natively from your Vidispine API

Using Elemental in Vidispine API

Now you can use the MediaConvert presets as any other transcode preset, and when using one of them you will trigger a transcode using a Vidinet MediaConvert service. Let’s say that you want to run a transcode using the AWS MediaConvert preset “System-Generic_Hd_Mp4_Avc_Aac_16x9_1280x720p_24Hz_4.5Mbps”. That preset is the same as the shape-tag  “__mediaconvert_Generic_Hd_Mp4_Avc_Aac_16x9_1280x720p_24Hz_4.5Mbps” in Vidispine.

First, let’s get a cost estimate:

POST https://{yourapi}.myvidispine.com/API/cost/item/VX-61/transcode?tag=__mediaconvert_Generic_Hd_Mp4_Avc_Aac_16x9_1280x720p_24Hz_4.5Mbps

This call returns a CostEstimateDocument, pointing out where you can find the cost estimate.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<CostEstimateDocument xmlns="https://xml.vidispine.com/schema/vidispine">
    <id>VlgtMTE9ZjJiMzUwZmEtOTNmNy00YWU5LWFjNDEtYTgxNjEyNTA5MGQ4</id>
    <url>https://{yourapi}.myvidispine.com/API/cost/estimate/VlgtMTE9ZjJiMzUwZmEtOTNmNy00YWU5LWFjNDEtYTgxNjEyNTA5MGQ4</url>
</CostEstimateDocument>

And the cost estimate is as follows

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<CostEstimateDocument xmlns="https://xml.vidispine.com/schema/vidispine">
    <id>VlgtMTE9ZjJiMzUwZmEtOTNmNy00YWU5LWFjNDEtYTgxNjEyNTA5MGQ4</id>
    <url>https://{yourapi}.myvidispine.com/API/cost/estimate/VlgtMTE9ZjJiMzUwZmEtOTNmNy00YWU5LWFjNDEtYTgxNjEyNTA5MGQ4</url>
    <state>FINISHED</state>
    <service>
        <resource>VX-11</resource>
        <name>The Company Ltd MediaConvertname>
        <type>ELEMENTAL_MEDIACONVERT</type>
        <state>FINISHED</state>
        <cost unit="USD">0.0476</cost>
    </service>
</CostEstimateDocument>

Great, now I know what the cost will be (my chosen video was 60 seconds, so I get the actual per minute price for this transcode), so let’s run the transcode.

POST https://{yourapi}.myvidispine.com/API/cost/item/VX-61/transcode?tag=__mediaconvert_Generic_Hd_Mp4_Avc_Aac_16x9_1280x720p_24Hz_4.5Mbps&resourceId=VX-7&jobmetadata=destinationStorageId%3dVX-8

Note: you have to tell the Vidispine API what transcoder to use, i.e., the resourceId, in this case, VX-7 is the MediaConvert transcoder (see this KB article), and you have to specify the destination storage to make sure MediaConvert picks the right storage (same region) to store output.

Using customer MediaConvert presets

You can, of course, use custom presets when using MediaConvert through the Vidispine API, with the same restrictions as you have on the system presets. The simplest solution to create custom presets is to create your preset from the AWS MediaConvert console, and then create a new Vidispine API preset using the resulting JSON.

1. Start by creating your custom preset in the MediaConvert console, either from scratch or by duplicating an existing and make the needed changes.

2. From the newly created preset choose to Export JSON.

Export Custom MediaConvert Preset

3. Take the resulting JSON and copy that into a Vidispine TranscodePresetDocument. The JSON from the MediaConvert preset goes into <outputsettings> in the XML.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<TranscodePresetDocument xmlns="https://xml.vidispine.com/schema/vidispine">
  <description>XDCAM MPEG HD422, 1280x720p, 50fps, 50mbps</description>
  <name>Patrik1-Generic_Hd_Mp4_Avc_Aac_16x9_1280x720p_24Hz_4.5Mbps</name>
  <audio/>
  <video/>
  <mediaconvert>
    <outputSetting>
      {
        "Description": "same. removing all PRO (interlace + single pass)",
        "Category": "GENERIC-HD",
        "Name": "Patrik1-Generic_Hd_Mp4_Avc_Aac_16x9_1280x720p_24Hz_4.5Mbps",
        ...
      }
    </outputSetting>
  </mediaconvert>
</TranscodePresetDocument>

4. Create your new Vidispine API shape tag using:

PUT /API/shape-tag/Patrik1-Generic_Hd_Mp4_Avc_Aac_16x9_1280x720p_24Hz_4.5Mbps

with the above TranscodePresetDocument as payload. This would create your new shape-tag that can be used directly in the Vidispine API.

More information

Read more about AWS Elemental MediaConvert on AWS Media Services. Head for the Vidispine API documentation for details on how to use MediaConvert. If you’re interested in a deep-dive into AWS Elemental MediaConvert as well as how to use it, we have a webinar coming up with engineers from both AWS and Vidispine going through nitty-gritty details. If you have questions or find something odd, please contact us at https://support.vidispine.com.