Table of Contents

Class AutoReframeBlock

Namespace
VisioForge.Core.MediaBlocks.AI
Assembly
VisioForge.Core.AI.dll

AI auto-reframe MediaBlock that converts landscape footage into a fixed (typically vertical 9:16) output by dynamically cropping around a detected subject. It taps frames for object detection on a sampled cadence, follows the selected subject, smooths the crop position over time, and scales the crop to the configured output resolution.

public sealed class AutoReframeBlock : MediaBlock, IMediaBlockInternals, IVideoProcessingBlock, IMediaBlock, IDisposable

Inheritance

MediaBlock

Implements

IMediaBlockInternals
IVideoProcessingBlock
IMediaBlock

Inherited Members

MediaBlock.GetPipelineContext()
MediaBlock.SetPipelineContext(BlockPipelineContext)
MediaBlock.SetPipeline(MediaBlocksPipeline)
MediaBlock.Context
MediaBlock.Name
MediaBlock.IsBuilt
MediaBlock.Owner
MediaBlock.Type
MediaBlock.ID
MediaBlock.Input
MediaBlock.Inputs
MediaBlock.Output
MediaBlock.Outputs
MediaBlock.HasInputs
MediaBlock.HasOutputs
MediaBlock.Build()
MediaBlock.CreateElements()
MediaBlock.AddElementsToPipeline()
MediaBlock.RemoveElementsFromPipeline()
MediaBlock.DeepCopy(string)
MediaBlock.Reset()
MediaBlock.ToYAMLBlock()
MediaBlock.ClearPads()
MediaBlock.Dispose()

Remarks

Internally the block is a chain of VideoSampleGrabber (RGBA frame tap) → videocrop → videoscale → capsfilter → videoconvert. Detection runs on the tapped frames; the resulting crop window is applied live by updating the videocrop left/right/ top/bottom properties from the streaming thread (the tap runs upstream of videocrop in the same chain call, so a crop update applies to the very frame that triggered it). The crop window size is fixed for the whole run (it depends only on the source size, the source pixel aspect ratio, and the output aspect ratio), so only the crop position changes and downstream caps do not renegotiate per frame. Anamorphic (non-square-pixel) sources are handled by sizing the crop in pixel space so its display aspect matches the output aspect; the scale to the square-pixel output then restores true display proportions.

Constructors

AutoReframeBlock(AutoReframeSettings)

Initializes a new instance of the VisioForge.Core.MediaBlocks.AI.AutoReframeBlock class.

public AutoReframeBlock(AutoReframeSettings settings)

Parameters

settings AutoReframeSettings

The auto-reframe settings. Must not be null.

Properties

ActiveProvider

Gets the active ONNX execution provider.

public OnnxExecutionProvider ActiveProvider { get; }

Property Value

OnnxExecutionProvider

Input

Gets the primary input pad for this media block.

public override MediaBlockPad Input { get; }

Property Value

MediaBlockPad

Inputs

Gets the collection of input pads for this media block.

public override MediaBlockPad[] Inputs { get; }

Property Value

MediaBlockPad[]

LastInferenceTimeMs

Gets the detector run time of the most recent detection, in milliseconds. Zero before the first detection. Not updated on frames that skip detection and only reuse the smoothed crop.

public float LastInferenceTimeMs { get; }

Property Value

float

Output

Gets the primary output pad for this media block.

public override MediaBlockPad Output { get; }

Property Value

MediaBlockPad

Outputs

Gets the collection of output pads for this media block.

public override MediaBlockPad[] Outputs { get; }

Property Value

MediaBlockPad[]

Type

Gets or sets the functional type of this media block (source, sink, effect, encoder, etc.). This property must be overridden in derived classes to specify the block's role in the pipeline.

public override MediaBlockType Type { get; }

Property Value

MediaBlockType

Exceptions

NotImplementedException

Thrown when the derived class has not implemented this property.

NotImplementedException

Thrown when the derived class has not implemented the setter.

Methods

Build()

Builds and initializes this media block, creating any necessary GStreamer elements and configuring them. This method must be overridden in derived classes to implement block-specific initialization logic.

public override bool Build()

Returns

bool

true if the block was successfully built; otherwise, false.

CleanUp()

Releases all resources and performs cleanup operations for this MediaBlock. This method disposes of GStreamer elements, releases memory, and ensures proper shutdown of the block. Called automatically during pipeline teardown.

public void CleanUp()

Dispose(bool)

Releases unmanaged and - optionally - managed resources. Override this method in derived classes to implement block-specific cleanup logic.

protected override void Dispose(bool disposing)

Parameters

disposing bool

true to release both managed and unmanaged resources; false to release only unmanaged resources.

GetCore()

Gets the core GStreamer element wrapper (the frame-tap element).

public BaseElement GetCore()

Returns

BaseElement

The VisioForge.Core.GStreamer.Base.BaseElement wrapper, or null if not built yet.

GetElement()

Gets the primary GStreamer element (the frame-tap element).

public Element GetElement()

Returns

Element

The Gst.Element, or null if not built yet.

IsAvailable()

Determines whether this media block is available. Correct NuGet SDK redist must be included.

public static bool IsAvailable()

Returns

bool

true if this media block is available; otherwise, false.

IMediaBlockInternals.SetContext(MediaBlocksPipeline)

Associates this MediaBlock with a pipeline and initializes its internal context. This method is called automatically when the block is added to a pipeline, providing access to shared resources and pipeline-wide configuration.

void IMediaBlockInternals.SetContext(MediaBlocksPipeline pipeline)

Parameters

pipeline MediaBlocksPipeline

The MediaBlocksPipeline instance that will manage this block.

OnReframeUpdated

Raised on every processed frame with the crop window applied, the followed subject (if any), and the source frame timestamp.

public event EventHandler<ReframeEventArgs> OnReframeUpdated

Event Type

EventHandler<ReframeEventArgs>