Table of Contents

Namespace VisioForge.Core.MediaBlocks.AI

Classes

AudioEventDetectorBlock

An audio-event-detection media block: it taps the audio stream, resamples it to 16 kHz mono, and runs a YAMNet (AudioSet, 521-class) ONNX classifier over short windows to recognize real-world sounds — siren, dog bark, glass break, alarm, music, speech and hundreds more — raising VisioForge.Core.MediaBlocks.AI.AudioEventDetectorBlock.OnAudioEvent as sounds are detected. Audio passes through unchanged.

AutoReframeBlock

AI auto-reframe MediaBlock that converts landscape footage into a fixed (typically vertical 9:16) output by dynamically cropping around a detected subject. It taps frames for object detection on a sampled cadence, follows the selected subject, smooths the crop position over time, and scales the crop to the configured output resolution.

BackgroundRemovalBlock

An AI background-removal (matting) media block. Implements the VisioForge.Core.MediaBlocks.MediaBlock. Implements the VisioForge.Core.MediaBlocks.IMediaBlockInternals.

FaceRecognitionBlock

A face recognition (face identity) media block. Runs a two-stage pipeline: a YuNet detector locates faces and their five landmarks, each face is aligned and embedded (SFace / ArcFace), and the embedding is matched 1:N against an enrolled VisioForge.Core.Types.X.AI.FaceGallery. Implements the VisioForge.Core.MediaBlocks.MediaBlock. Implements the VisioForge.Core.MediaBlocks.IMediaBlockInternals.

LicensePlateRecognizerBlock

A license plate recognizer (ANPR/LPR) media block. Runs a specialized two-stage pipeline: a dedicated license-plate detector locates plates in the frame, each plate is cropped, and a plate-specific OCR model reads its characters. Implements the VisioForge.Core.MediaBlocks.MediaBlock. Implements the VisioForge.Core.MediaBlocks.IMediaBlockInternals.

ObjectAnalyticsBlock

Turnkey MediaBlock that performs multi-object tracking, tripwire line crossing, and polygon zone occupancy on top of ONNX object detection, drawing overlays and raising analytics events.

OcrBlock

An OCR (optical character recognition) media block that recognizes text in video frames using a multi-stage PaddleOCR (PP-OCR) ONNX pipeline. Implements the VisioForge.Core.MediaBlocks.MediaBlock. Implements the VisioForge.Core.MediaBlocks.IMediaBlockInternals.

OnnxInferenceBlock

A generic ONNX inference media block. Implements the VisioForge.Core.MediaBlocks.MediaBlock. Implements the VisioForge.Core.MediaBlocks.IMediaBlockInternals.

OpenVocabularyDetectorBlock

An open-vocabulary object-detection media block (OWLv2 / Grounding DINO). Detects objects described by free-text VisioForge.Core.Types.X.AI.OpenVocabularyDetectorSettings.Prompts instead of a fixed class list. Implements the VisioForge.Core.MediaBlocks.MediaBlock. Implements the VisioForge.Core.MediaBlocks.IMediaBlockInternals.

PIIRedactionBlock

A PII redaction media block. Detects faces (YuNet), vehicle license plates (FastALPR detector), and on-screen text (PaddleOCR detector, with an optional regex filter on the recognized text) and obscures them in-place with a Gaussian blur, pixelation, or a solid fill. Implements the VisioForge.Core.MediaBlocks.MediaBlock. Implements the VisioForge.Core.MediaBlocks.IMediaBlockInternals.

SpeakerDiarizationBlock

A speaker-diarization ("who spoke when") media block. It taps the audio stream, resamples it to mono 16 kHz, and on a background worker runs a pyannote segmentation ONNX model over overlapping 10 s windows and turns every speech turn it finds into a voiceprint with a WeSpeaker/3D-Speaker embedding ONNX model. When the stream ends, it clusters the voiceprints into speakers and publishes the timeline. Implements the VisioForge.Core.MediaBlocks.MediaBlock. Implements the VisioForge.Core.MediaBlocks.IMediaBlockInternals.

VLMBlock

A vision-language media block that runs a Florence-2 ONNX pipeline on video frames to caption them, detect objects, run OCR, or ground phrases (selected by VisioForge.Core.Types.X.AI.VLMSettings.Task). Implements the VisioForge.Core.MediaBlocks.MediaBlock. Implements the VisioForge.Core.MediaBlocks.IMediaBlockInternals.

VideoEmbeddingBlock

A media block that encodes sampled video frames into CLIP image embeddings for semantic video search. Implements the VisioForge.Core.MediaBlocks.MediaBlock. Implements the VisioForge.Core.MediaBlocks.IMediaBlockInternals.

YOLOObjectDetectorBlock

A YOLOv8/v11 object-detection media block. Implements the VisioForge.Core.MediaBlocks.MediaBlock. Implements the VisioForge.Core.MediaBlocks.IMediaBlockInternals.