Overview¶
GPAC gained (starting in version 2.4.0) the ability to deal with dynamic metadata such as SCTE-35. The information in this page also applies to other metadata such as ID3 markers (e.g. Nielsen), timecodes (TEMI, QT), or virtually any type of dynamic metadata.
GPAC is able to:
- inspect,
- edit to some extent via NHML,
- insert dynamically via the C/JS/Python/... API,
- remux or transmux between MPEG-TS and ISOBMFF-based formats (MP4, MPEG-DASH, HLS, CMAF),
- create 23001-18 Event Tracks and DASH content for both inband (CSAI, SGAI) or out-of-band (SSAI),
- align IDRs with events in case of transcoding.
A special dec_scte35 filter was created to segment SCTE-35 content or insert cue start property on splice points to align transcoded GOPs with SCTE-35 events. The dec_scte35 filter needs to be explicitely added to the graph (or requested from the dasher with the evte_agg option).
Inspection¶
The content can be inspected like any other content in GPAC. Packet-based inspection may allow to dump useful information for analysis and reprocessing.
GPAC inspect¶
would output:
[...]
<Packet number="970" PID="1" framing="complete" dts="1636452" cts="1640956" dur="1502" sap="0" ilace="0" corr="0" seek="0" bo="N/A" roll="0" crypt="0" vers="0" size="6799" lp="0" depo="0" depf="0" red="0" CRC32="0x17CDFEAC">
<SCTE35>
<scte35:SpliceInfoSection xmlns:scte35="urn:scte:scte35:2013:bin" sap_type="3" protocol_version="0" encrypted_packet="0" pts_adjustment="183000" tier="4095">
<scte35:SpliceInsert splice_event_id="4026531841" splice_event_cancel_indicator="False" out_of_network_indicator="True" program_splice_flag="True" duration_flag="False" splice_immediate_flag="False">
<scte35:SpliceTime pts_time="1813812"/>
</scte35:SpliceInsert>
</scte35:SpliceInfoSection>
</SCTE35>
</Packet>
[...]
MP4Box -dxml¶
scte35_dump.xml would contain for instance such descriptions for a sample:
[...]
<NHNTSample number="94" DTS="41224943904144" dataLength="98" CTSOffset="0" isRAP="yes" mediaOffset="0" sourceByteOffset="13584" duration="48048" depends="24" >
<EventMessageInstanceBox Size="98" Type="emib" Version="0" Flags="0" Specification="EventMessageTrack" Container="file" presentation_time_delta="122121" event_duration="239239" event_id="1" scheme_id_uri="urn:scte:scte35:2013:bin" value="" message_data="0xFC302500000000697800FFF01405F00000027FEFFE1D0AC9C1FE000DB07B0001010100001231F90A" >
<scte35:SpliceInfoSection xmlns:scte35="urn:scte:scte35:2013:bin" sap_type="3" protocol_version="0" encrypted_packet="0" pts_adjustment="27000" tier="4095">
<scte35:SpliceInsert splice_event_id="4026531842" splice_event_cancel_indicator="False" out_of_network_indicator="True" program_splice_flag="True" duration_flag="True" splice_immediate_flag="False">
<scte35:SpliceTime pts_time="487246273"/>
<scte35:BreakDuration duration="897147"/>
</scte35:SpliceInsert>
</scte35:SpliceInfoSection>
</EventMessageInstanceBox>
</NHNTSample>
[...]
Edit metadata¶
It is possible to edit data using the NHML serialization format.
For example some inspected content can be re-injected, see this example from the testsuite:
<?xml version="1.0" encoding="UTF-8" ?>
<NHNTStream version="1.0" timeScale="90000" mediaType="meta" mediaSubType="sc35" mime_type="text/plain" text_encoding="utf-8">
<NHNTSample DTS="100000" isRAP="yes">
<EventMessageInstanceBox Version="0" Flags="0" presentation_time_delta="165000" event_duration="135000" event_id="0" scheme_id_uri="urn:scte:scte35:2013:bin" value="">
<SpliceInfoSection xmlns="http://www.scte.org/schemas/35" sapType="3" protocolVersion="0" ptsAdjustment="0" tier="4095">
<TimeSignal>
<SpliceTime ptsTime="329696250"/>
</TimeSignal>
<SegmentationDescriptor segmentationEventId="3658" segmentationEventCancelIndicator="0" segmentationEventIdComplianceIndicator="1" programSegmentationFlag="1" segmentationDurationFlag="1" deliveryNotRestrictedFlag="1" segmentationDuration="135000" segmentationTypeId="52" segmentNum="0" segmentsExpected="0" subSegmentNum="0" subSegmentsExpected="0">
</SegmentationDescriptor>
</SpliceInfoSection>
</EventMessageInstanceBox>
</NHNTSample>
</NHNTStream>
This is particularly useful when dynamically editing and injecting metadata.
Remux and transmux¶
GPAC is able to remux any to any, including but not limited to:
- TS to TS
- MP4 (23001-18) to MP4 (23001-18)
- TS to MP4 (23001-18)
- TS to DASH out-of-band EventStream element
- NHML to TS or MP4 or DASH
See specific instructions in the next section when segmenting for OTT packaging (MPEG-DASH, HLS).
The command-line is trivial:
The format is selected automatically depending on the file extension (or can be forced with :ext=).
Examples provided by the community:
- Input is TS (embedding the "scte35 property"):
- TS -> MP4 (emsg) : gpac -i input.ts -o output.mp4:scte35=inband
- TS -> MP4 (evte: call scte35dec explicitely) : gpac -i input.ts @#video scte35dec @ @@ -o output.mp4 (MP4Box -add input.ts -add input.ts#video:@scte35dec -new output.mp4)
- TS -> HLS (outband) : gpac -i input.ts -o output.m3u8:segdur=6:dmode=dynamic:tsb=-1:hlsc
- TS -> DASH (auto mode: outband, <EventStream> 'xml+bin') : gpac -i input.ts -o output.mpd
- TS -> DASH (inband, emsg) : gpac -i input.ts -o output.mpd:scte35=inband
- TS -> DASH (evte: force segmentation) : gpac -i input.ts -o output.mpd:evte_agg
- TS -> TS : gpac -i input.ts -o output.ts
- Input is EVTE/MP4 (Event Stream track with its own GPAC PID):
- MP4 -> DASH (evte, auto mode): gpac -i input.mp4 -o output.mpd
- MP4 -> DASH (outband, <EventStream> 'xml+bin'): gpac -i input.mp4 scte35dec:prop -o output.mpd:scte35=xmlbin
- NB: the input can also be in a third format: raw SCTE35 frames embedded in MPEG-2 TS Sections (see gpac -h scte35dec.mode).
Manifest, Segmentation and 23001-18 ISOBMFF Event Track¶
Introduction¶
The core concept is that metadata can be signalled "in-band" or "out-of-band".
"Out-of-band" (sometimes also referred as "outband") means that the content is in the manifest. In DASH this is done with EventStream in the manifest. In HLS, input can be in the variant playlist.
"Inband" means that the metadata leaves inside the media content. This can be done with extra emsg boxes inside an existing (typically video) track, or as a separate Event Track (evte). In both cases, recommendations require some manifest signalling (e.g. with GPAC's dasher :inband_event option).
The :scte35 option controls the signalling using DATERANGE and CUE-OUT/CUE-IN. If you need other modes, contact us.
How it works¶
The source metadata is usually presented as a MPEG-TS section PID input or emsg boxes in ISOBMFF or evteevent tracks. This may require GPAC to analyze or extract metadata for you. There are many use-cases so this can't be fully automated. Please read this document carefully to set the appropriate filters (dec_scte35) or options (scte35, evte_agg) appropriately.
Adaptive streaming with DASH or HLS requires the content to be segmented. Simple segmentation (where metadata content can be replicated over several segments without any change) works for all kind of streams. More advanced SCTE-35 segmentation is presented below.
Note: HLS was only tested with out-of-band signalling.
SCTE-35 segmentation¶
For SCTE-35 streams the dec_scte35 filter must be either explicitly loaded or the dasher evte_agg option must be set.
Note that it can detect the segment duration from the dasher to create the right segmentation for you (both with in-band and out-of-band):
Whenever possible the content is in the form of 23001-18 ISOBMFF Event Tracks. emib and emeb boxes are inserted automatically.
Same works with HLS (with in-band):
Special case of in-band signalling¶
In the case where GPAC wouldn't automatically detect your in-band content, you can add it manually to the manifest. Here is an example with ID3 tags e.g. for Nielsen:
gpac -i file.mp4 -o file.mpd::inband_event=https://aomedia.org/emsg/ID3@https://aomedia.org/emsg/ID3@audio,https://aomedia.org/emsg/ID3@www.nielsen.com:id3:v1@audio
Align IDRs with events in case of transcoding¶
The following example will transcode a MPEG-TS file while inserting IDRs at SCTE-35 cue points and copying all the other data: