Table of Contents

Class OpenVocabularyDetectorBlock

Namespace
VisioForge.Core.MediaBlocks.AI
Assembly
VisioForge.Core.AI.dll

An open-vocabulary object-detection media block (OWLv2 / Grounding DINO). Detects objects described by free-text VisioForge.Core.Types.X.AI.OpenVocabularyDetectorSettings.Prompts instead of a fixed class list. Implements the VisioForge.Core.MediaBlocks.MediaBlock. Implements the VisioForge.Core.MediaBlocks.IMediaBlockInternals.

public class OpenVocabularyDetectorBlock : MediaBlock, IMediaBlockInternals, IVideoProcessingBlock, IMediaBlock, IDisposable

Inheritance

MediaBlock

Implements

IMediaBlockInternals
IVideoProcessingBlock
IMediaBlock

Inherited Members

MediaBlock._isBuilt
MediaBlock._pipeline
MediaBlock._pipelineCtx
MediaBlock.GetPipelineContext()
MediaBlock.SetPipelineContext(BlockPipelineContext)
MediaBlock.SetPipeline(MediaBlocksPipeline)
MediaBlock.Context
MediaBlock.Name
MediaBlock.IsBuilt
MediaBlock.Owner
MediaBlock.Type
MediaBlock.ID
MediaBlock.Input
MediaBlock.Inputs
MediaBlock.Output
MediaBlock.Outputs
MediaBlock.HasInputs
MediaBlock.HasOutputs
MediaBlock.Build()
MediaBlock.CreateElements()
MediaBlock.AddElementsToPipeline()
MediaBlock.RemoveElementsFromPipeline()
MediaBlock.DeepCopy(string)
MediaBlock.Reset()
MediaBlock.ToYAMLBlock()
MediaBlock.ClearPads()
MediaBlock.Dispose(bool)
MediaBlock.Dispose()

Remarks

Open-vocabulary models are far heavier than YOLO, so inference runs on a background worker (single-slot, latest-wins) exactly like VisioForge.Core.MediaBlocks.AI.OcrBlock: the streaming thread submits a clean frame copy and only redraws the most recent cached detections, so the pipeline never stalls. Each detection's Label is the matched prompt string and its ClassId is that prompt's zero-based index. Prompts can be swapped at runtime via VisioForge.Core.MediaBlocks.AI.OpenVocabularyDetectorBlock.SetPrompts(System.String[]); the change applies on the next processed frame.

Constructors

OpenVocabularyDetectorBlock(OpenVocabularyDetectorSettings)

Initializes a new instance of the VisioForge.Core.MediaBlocks.AI.OpenVocabularyDetectorBlock class.

public OpenVocabularyDetectorBlock(OpenVocabularyDetectorSettings settings)

Parameters

settings OpenVocabularyDetectorSettings

The detector settings. Must specify a valid model path, tokenizer files, and prompts.

Exceptions

ArgumentNullException

Thrown when settings is null.

Properties

ActiveProvider

Gets the execution provider the detector's inference session actually engaged. Valid after the block has been built; reports VisioForge.Core.Types.X.AI.OnnxExecutionProvider.CPU otherwise.

public OnnxExecutionProvider ActiveProvider { get; }

Property Value

OnnxExecutionProvider

DroppedFrameCount

Gets the number of frames that arrived while the background worker was still busy and were therefore skipped for detection (the live video is unaffected).

public long DroppedFrameCount { get; }

Property Value

long

Input

Gets the primary input pad.

public override MediaBlockPad Input { get; }

Property Value

MediaBlockPad

Inputs

Gets the array of all input pads.

public override MediaBlockPad[] Inputs { get; }

Property Value

MediaBlockPad[]

LastInferenceTimeMs

Gets the wall-clock time, in milliseconds, the most recent inference took. Inference runs on a background thread; use this to tune VisioForge.Core.Types.X.AI.OnnxInferenceSettings.FramesToSkip to the hardware.

public float LastInferenceTimeMs { get; }

Property Value

float

Output

Gets the primary output pad.

public override MediaBlockPad Output { get; }

Property Value

MediaBlockPad

Outputs

Gets the array of all output pads.

public override MediaBlockPad[] Outputs { get; }

Property Value

MediaBlockPad[]

Type

Gets the type of the media block.

public override MediaBlockType Type { get; }

Property Value

MediaBlockType

Methods

Build()

Constructs the internal GStreamer elements and pads for this block.

public override bool Build()

Returns

bool

true if successful, false otherwise.

CleanUp()

Cleans up internal resources, specifically the GStreamer element.

public void CleanUp()

Dispose(bool)

Releases unmanaged and - optionally - managed resources.

protected override void Dispose(bool disposing)

Parameters

disposing bool

true to release both managed and unmanaged resources; false to release only unmanaged resources.

GetCore()

Gets the core GStreamer element wrapped by this block.

public BaseElement GetCore()

Returns

BaseElement

The VisioForge.Core.GStreamer.Base.BaseElement wrapper, or null if not built yet.

GetElement()

Gets the GStreamer element instance.

public Element GetElement()

Returns

Element

The Gst.Element, or null if not built yet.

SetConfidenceThreshold(float)

Updates the confidence threshold on the running detector. The adapter reads it per frame, so the change takes effect on the next processed frame without rebuilding the pipeline.

public void SetConfidenceThreshold(float threshold)

Parameters

threshold float

Minimum detection score to keep, in the range [0, 1].

SetIoUThreshold(float)

Updates the IoU (non-maximum-suppression) threshold on the running detector. The adapter reads it per frame, so the change takes effect on the next processed frame without rebuilding the pipeline.

public void SetIoUThreshold(float threshold)

Parameters

threshold float

Overlap above which duplicate boxes are merged, in the range [0, 1].

SetPrompts(string[])

Replaces the detection prompts at runtime. The new prompts are re-tokenized under the block's prompt lock and take effect on the next processed frame.

public void SetPrompts(string[] prompts)

Parameters

prompts string[]

The new detection prompts.

IMediaBlockInternals.SetContext(MediaBlocksPipeline)

Sets the pipeline context for this block.

void IMediaBlockInternals.SetContext(MediaBlocksPipeline pipeline)

Parameters

pipeline MediaBlocksPipeline

The pipeline.

OnObjectsDetected

Event raised when one or more objects are detected in a video frame.

public event EventHandler<ObjectsDetectedEventArgs> OnObjectsDetected

Event Type

EventHandler<ObjectsDetectedEventArgs>

See Also

MediaBlock
IMediaBlockInternals