Namespace VisioForge.Core.Types.X.AI
Classes
- AudioEventDetectorSettings
-
Settings for the audio-event-detection block (YAMNet / AudioSet, 521 classes).
- AudioEventScore
-
A single AudioSet class score for one processed window, reported by the audio-event block's scores event.
- AutoReframeSettings
-
Settings for the AI auto-reframe block, which converts landscape footage into a fixed (typically vertical 9:16) output by dynamically cropping around the detected subject.
- BackgroundRemovalSettings
-
Settings for the background-removal (matting) block/engine.
- ByteTrackerSettings
-
Settings for the ByteTrack multi-object tracker.
- DetectionFilterSettings
-
Settings for filtering detections before they are passed to the tracker.
- DiarizedTranscriptBuilder
-
Merges a speech-to-text transcript with a speaker-diarization timeline, labeling each transcript segment with the speaker who talks over most of it.
- DiarizedTranscriptSegment
-
A transcript segment labeled with the speaker who most overlaps it — the output of VisioForge.Core.Types.X.AI.DiarizedTranscriptBuilder merging a speech-to-text transcript with a diarization timeline.
- FaceGallery
-
An in-memory gallery of enrolled face identities used for 1:N face recognition. Each identity holds one or more L2-normalized face embeddings; a query embedding is matched by cosine similarity (the maximum over all stored embeddings of all identities). The gallery is thread-safe and can be persisted to and loaded from disk.
- FaceRecognitionSettings
-
Settings for the face recognition (face identity) block. The block runs a two-stage pipeline: a face detector (YuNet) locates faces and their five landmarks, each face is aligned to a canonical 112x112 crop and turned into an embedding (SFace / ArcFace), and the embedding is matched 1:N against an enrolled VisioForge.Core.Types.X.AI.FaceGallery by cosine similarity.
- FrameEmbeddingIndex
-
An in-memory index of per-frame CLIP embeddings used for semantic video search. Each entry stores a frame's presentation timestamp, the source tag it came from, and the embedding vector produced by a
VideoEmbeddingBlock. A query embedding (for example the CLIP text embedding of a natural-language phrase) is matched by cosine similarity and the closest frames are returned. The index is thread-safe and can be persisted to and loaded from disk. - FrameEmbeddingSearchResult
-
A single hit returned by VisioForge.Core.Types.X.AI.FrameEmbeddingIndex.Search(System.Single[],System.Int32): the video timestamp of an indexed frame, the source tag it was indexed under, and the cosine similarity score against the query embedding.
- LicensePlateRecognizerSettings
-
Settings for the license plate recognizer (ANPR/LPR) block. The block runs a specialized two-stage pipeline: a dedicated license-plate detector (YOLO) locates plates in the frame, then a plate-specific OCR model reads the characters of each cropped plate.
- LineZoneSettings
-
Settings for a directed line (tripwire) zone.
- ObjectAnalyticsOverlaySettings
-
Settings controlling the visual overlay rendered by object analytics.
- ObjectAnalyticsSettings
-
Aggregate settings for the object analytics block, combining detector, tracker, filter, overlay, line-zone, and polygon-zone configuration.
- OcrSettings
-
Settings for the OCR block, which runs a multi-stage PaddleOCR (PP-OCR) pipeline on video frames: text detection (DBNet) → optional angle classification → text-line recognition (CRNN/SVTR + CTC).
- OnnxInferenceSettings
-
Settings for a generic ONNX inference block/engine.
- OpenVocabularyDetectorSettings
-
Settings for the open-vocabulary object detector block/engine (OWLv2 and Grounding DINO).
- PIIFaceRedactionSettings
-
Face-category settings for the PII redaction block. Faces are located with the OpenCV Zoo YuNet detector (MIT); only detection runs — no identity recognition.
- PIIPlateRedactionSettings
-
License-plate-category settings for the PII redaction block. Plates are located with the FastALPR YOLOv9-T end-to-end detector (MIT); only detection runs — the plate characters are never read.
- PIIRedactionSettings
-
Settings for the PII redaction block, which automatically obscures personally identifiable information in video frames: faces (YuNet), vehicle license plates (FastALPR detector), and on-screen text (PaddleOCR detector, with an optional regex filter on the recognized text). Each category can be enabled independently and all use detection only — nothing is identified, recognized, or exported unless the text regex filter explicitly requires recognition.
- PIITextRedactionSettings
-
On-screen-text-category settings for the PII redaction block. Text regions are located with the PaddleOCR PP-OCR detection model and confirmed with the recognition model (both Apache-2.0): recognition is what filters the detector's false positives (on a real scene the detector alone fires on textures and edges, so detection without recognition would redact most of the frame). By default every recognized text region is redacted; set VisioForge.Core.Types.X.AI.PIITextRedactionSettings.TextFilterRegex to redact only the regions whose recognized text matches (for example e-mail addresses or phone numbers). The detection, recognition, and dictionary models are all required whenever VisioForge.Core.Types.X.AI.PIITextRedactionSettings.Enabled is
true. - PTZAutoTrackingSettings
-
Settings for the PTZ auto-tracking controller, which converts object-detection positions into continuous PTZ velocity commands so an ONVIF PTZ camera follows a tracked object.
- PolygonZoneSettings
-
Settings for a polygon zone used for occupancy counting.
- SileroVadSettings
-
Settings for the Silero neural voice-activity detector that segments speech inside the speech-to-text block.
- SpeakerDiarizationSettings
-
Settings for the speaker-diarization block ("who spoke when"): a pyannote segmentation model finds who is talking inside each 10 s window of audio, a WeSpeaker/3D-Speaker embedding model turns every such turn into a voiceprint, and the voiceprints are clustered into speakers once the recording has ended.
- SpeakerSegment
-
One diarized speech turn: a time span attributed to a single speaker.
- SpeechSegment
-
A single recognized speech segment: a span of transcribed text with its start/end time on the media timeline.
- SpeechToTextSettings
-
Settings for the speech-to-text block (Whisper ASR + optional Silero VAD).
- VLMRegion
-
A single grounded region produced by a Florence-2 task: a text label (an object category, a region description, or a recognized text block) together with its bounding box in source-frame pixel coordinates.
- VLMSettings
-
Settings for the Florence-2 vision-language model (VLM) block. The block runs a four-session ONNX pipeline (vision encoder, token embedder, text encoder, and a merged decoder) to caption frames, detect objects, run OCR, or ground phrases, selected by VisioForge.Core.Types.X.AI.VLMSettings.Task.
- VideoEmbeddingSettings
-
Settings for the video embedding block, which encodes sampled video frames into CLIP image embeddings for semantic search. The block runs a CLIP dual-tower model: a vision tower turns each sampled frame into an L2-normalized embedding, and a text tower (used via
EncodeText) turns a natural-language query into an embedding in the same space so the two can be compared by cosine similarity. - YoloDetectorSettings
-
Settings for the YOLO object detector block/engine.
Enums
- AutoReframeTargetSelection
-
Selects which detected object the auto-reframe block follows when several are present.
- BackgroundRemovalModel
-
The segmentation/matting model family the background-removal block runs. Each family expects a distinct input size and pixel normalization, so the block must be told which one the supplied
.onnxfile belongs to. The model produces a single-channel foreground alpha mask that the block uses to composite the replacement background. - BackgroundReplacementMode
-
How the background-removal block replaces the pixels the matting model marks as background (low foreground alpha).
- DetectionAnchor
-
Defines which point of a detection bounding box is used as its reference anchor for line-crossing and polygon-zone evaluation.
- FaceEmbeddingModel
-
The face-embedding model family used by the face recognizer. The family only selects the input preprocessing of the aligned 112x112 crop (channel order and normalization); the embedding length is always read from the model itself, so any vector size works.
- LineCrossingDirection
-
The direction of a directed line crossing.
- ObjectDetectorModel
-
The object-detection model family the detector decodes. Each family has a distinct ONNX output layout and a distinct frame-preprocessing convention (resize mode, pixel normalization, channel order), so the detector must be told which one the supplied
.onnxfile belongs to. - OnnxExecutionProvider
-
Execution provider used by the ONNX Runtime inference session.
- OpenVocabularyModel
-
The open-vocabulary detection model family a supplied ONNX model belongs to. This selects the text tokenizer, the frame-preprocessing convention, and the output decoder used by the open-vocabulary detector engine.
- PIICategory
-
The category of personally identifiable information a redacted region belongs to.
- PIIRedactionStyle
-
The visual style used to redact a detected PII region in the video frame.
- PTZTargetSelection
-
Strategy used by the PTZ auto-tracking controller to pick which detected object to follow when more than one object is present in a frame.
- SpeakerEmbeddingModel
-
Selects which speaker-embedding ("voiceprint") model the diarization block is configured for. The value is a declarative hint: the ONNX file actually loaded is VisioForge.Core.Types.X.AI.SpeakerDiarizationSettings.EmbeddingModelPath, and the feature front-end auto-configures itself from that file's ONNX metadata. The enum picks the default download URL and documents the recommended clustering threshold for each model; both models are Apache-2.0.
- SpeechToTextTask
-
The Whisper task: plain transcription or speech translation to English.
- VLMTask
-
The task a Florence-2 vision-language model performs on each processed frame. The task selects the natural-language prompt fed to the model and how its output is interpreted (free text vs. grounded regions).
- WhisperModelSize
-
The Whisper (GGML) model variant. Larger models are more accurate but slower and use more memory; quantized variants (Q5/Q8) trade a little accuracy for a smaller footprint and faster CPU inference.
- ZoneExitReason
-
The reason a tracked object exited a polygon zone.