Table of Contents

Namespace VisioForge.Core.MediaBlocks.Special

Classes

BarcodeDetectorBlock

A specialized media block that detects and decodes barcodes from video frames in real-time. This block analyzes incoming video frames for various barcode formats including QR codes, DataMatrix, Code128, Code39, EAN-13, and other standard 1D/2D barcodes.

Key features:

  • Real-time barcode detection and decoding from video streams
  • Support for multiple simultaneous barcode detection
  • Configurable operating modes (passthrough or sink)
  • Event-based notification when barcodes are detected
  • Minimal performance impact on video pipeline

Common use cases:

  • QR code scanning in mobile applications
  • Inventory management systems with barcode scanning
  • Document processing with embedded barcodes
  • Access control systems using QR codes
  • Product tracking in retail/warehouse environments

Technical details:

  • Processes video frames using computer vision algorithms
  • Can operate in InputOutput mode (passthrough) or Input mode (sink)
  • Raises events with decoded barcode data and metadata
  • Requires appropriate barcode detection libraries in the GStreamer plugin set

Implements the VisioForge.Core.MediaBlocks.MediaBlock. Implements the VisioForge.Core.MediaBlocks.IMediaBlockInternals. Implements the IDisposable.

BufferEventArgs

Event arguments for buffer processing events.

CustomColorspaceXBlock

A media block that converts RGB video frames to YV12 format. This block uses a custom GStreamer plugin that implements RGB to Y'CbCr conversion with 4:2:0 chroma subsampling in C#.

CustomMediaBlock

A flexible media block that allows integration of any GStreamer element or bin into the MediaBlocks pipeline. This block serves as a bridge between the MediaBlocks abstraction and raw GStreamer elements, enabling use of specialized or custom GStreamer components not wrapped by standard blocks.

Key features:

  • Support for any GStreamer element by name or bin description
  • Dynamic pad configuration with automatic or manual pad handling
  • Property setting through key-value parameters
  • Support for elements with dynamic pads (pad-added event)
  • Optional caps filtering on output pads
  • Property introspection for debugging

Common use cases:

  • Integrating third-party GStreamer plugins
  • Using experimental or platform-specific elements
  • Creating complex processing chains with bin descriptions
  • Prototyping new functionality before creating dedicated blocks
  • Accessing GStreamer elements not yet wrapped in MediaBlocks

Technical details:

  • Can create elements from factory names or parse bin descriptions
  • Supports multiple input and output pads with configurable media types
  • Handles dynamic pad creation for elements like demuxers
  • Provides property mapping for various data types (int, float, string, enum)
  • Uses identity elements for dynamic pad connection management

Implements the VisioForge.Core.MediaBlocks.MediaBlock. Implements the VisioForge.Core.MediaBlocks.IMediaBlockInternals. Implements the IDisposable.

CustomTransformBlock

A media block that allows custom transformation with independent input/output caps. Users receive input samples via event and can push output samples when ready.

CustomVideoEncoderBlock

Custom video encoder block for integrating third-party or specialized video encoders into the media pipeline. This block provides a flexible framework for adding GStreamer video encoder elements that are not directly supported by the SDK, enabling integration of proprietary codecs, experimental encoders, or platform-specific encoding solutions. Features automatic color space conversion to I420 format and dynamic property configuration through a key-value interface.

Key features:

  • Integration of any GStreamer-compatible video encoder
  • Automatic video format conversion to I420
  • Dynamic property configuration via dictionary
  • Support for hardware and software encoders
  • Custom codec parameter tuning
  • Plugin-based encoder extensions

Common use cases:

  • Proprietary codec integration (custom H.264 variants)
  • Hardware-specific encoders (vendor SDKs)
  • Experimental codec testing (AV2, VVC)
  • Legacy format support (RealVideo, Indeo)
  • Specialized compression (lossless, HDR)
  • Custom streaming encoders

The block handles the complexity of GStreamer element creation and pipeline integration while providing a simple configuration interface for encoder-specific parameters. Implements the VisioForge.Core.MediaBlocks.MediaBlock. Implements the VisioForge.Core.MediaBlocks.IMediaBlockInternals. Implements the IDisposable.

DataMatrixDecoder

Provides asynchronous DataMatrix decoding capabilities for video frames using the LibDMTX library. This class processes video frames to detect and decode DataMatrix 2D barcodes with configurable frame skipping and processing synchronization to optimize CPU usage.

DataMatrixDecoderBlock

A specialized media block that detects and decodes DataMatrix 2D barcodes from video frames in real-time. This block analyzes incoming video frames for DataMatrix codes using the LibDMTX library with optimized frame processing, configurable frame skipping, and asynchronous decoding to minimize performance impact on the video pipeline.

Key features:

  • Real-time DataMatrix detection and decoding from video streams
  • Asynchronous processing with automatic frame dropping if busy
  • Configurable frame skip to reduce CPU usage (analyze every Nth frame)
  • Support for multiple DataMatrix codes in a single frame
  • Event-based notification when DataMatrix codes are detected
  • Support for various video formats (RGB, BGR, RGBA, BGRA)
  • Configurable decode timeout and symbol size hints

Common use cases:

  • Industrial tracking with DataMatrix codes on products
  • Document processing with embedded DataMatrix codes
  • Inventory management systems
  • Quality control and verification systems
  • Medical device and pharmaceutical tracking

Technical details:

  • Uses LibDMTX for DataMatrix detection and decoding
  • Processes frames asynchronously to avoid blocking the pipeline
  • Automatically converts video formats to RGB for LibDMTX
  • Thread-safe with proper synchronization
  • Supports both passthrough and sink modes

Implements the VisioForge.Core.MediaBlocks.MediaBlock. Implements the VisioForge.Core.MediaBlocks.IMediaBlockInternals. Implements the IDisposable.

DataProcessorBlock

A media block that allows custom in-place processing of audio/video buffers. Inherit from this class and override the ProcessBuffer method to implement custom processing logic.

DataSampleGrabberBlock

A specialized media block that intercepts and provides access to raw media data frames passing through the pipeline. This block acts as a transparent passthrough while capturing frame data for analysis, processing, or monitoring. Unlike video/audio-specific grabbers, this block can handle any media type including metadata streams.

Key features:

  • Zero-copy frame access with minimal performance impact
  • Support for any media type (video, audio, metadata, subtitles)
  • Passthrough design maintains pipeline flow
  • Event-based notification for each frame
  • Access to raw buffer data and timing information
  • Configurable media type filtering

Common use cases:

  • Capturing metadata streams from specialized sources
  • Monitoring data flow for debugging purposes
  • Extracting timing information from media streams
  • Analyzing proprietary data formats
  • Building custom analytics on media streams
  • Implementing data stream validators

Technical details:

  • Uses GStreamer pad probes for non-invasive data access
  • Provides frame data through event callbacks
  • Maintains original buffer timestamps and metadata
  • Can be configured for specific media types or auto-detect
  • Thread-safe frame delivery via events

Implements the VisioForge.Core.MediaBlocks.MediaBlock. Implements the VisioForge.Core.MediaBlocks.IMediaBlockInternals. Implements the IDisposable.

DebugTimestampBlock

A debugging utility block that logs and optionally overlays timestamp information on media streams. This block intercepts media frames to display Decode Time Stamp (DTS) and Presentation Time Stamp (PTS) values, which are crucial for diagnosing synchronization and timing issues in media pipelines.

Key features:

  • Logs DTS and PTS values to the debug console with custom labels
  • Optional visual overlay of timestamp information on video frames
  • Passthrough design with minimal performance impact
  • Configurable logger name for identifying different streams
  • Support for any media type (video, audio, etc.)
  • Platform-specific availability (not available on iOS)

Common use cases:

  • Debugging A/V synchronization issues
  • Analyzing timestamp discontinuities in streams
  • Verifying proper timestamp generation in encoders
  • Troubleshooting playback timing problems
  • Educational visualization of media timing concepts
  • Quality assurance for streaming applications

Technical details:

  • DTS represents when a frame should be decoded
  • PTS represents when a frame should be presented
  • Can combine with TextOverlay for visual timestamp display
  • Timestamps are logged in nanoseconds (GStreamer format)
  • Useful for identifying timestamp gaps or jumps

Implements the VisioForge.Core.MediaBlocks.MediaBlock. Implements the VisioForge.Core.MediaBlocks.IMediaBlockInternals. Implements the IDisposable.

DecodeBinBlock

An intelligent decoding block that automatically detects and decodes various media formats into raw streams. DecodeBin analyzes incoming media data, identifies the format and codecs, and constructs an appropriate decoding pipeline to produce raw video, audio, and subtitle outputs.

Key features:

  • Automatic format detection and codec selection
  • Support for virtually any media format GStreamer can decode
  • Dynamic pad creation for discovered streams
  • Separate outputs for video, audio, and subtitle streams
  • Built-in format conversion for consistent output
  • Configurable stream selection (video/audio/subtitle)

Common use cases:

  • Decoding media files with unknown or variable formats
  • Processing network streams with dynamic content
  • Building flexible media players that handle any format
  • Extracting raw streams from container formats
  • Media analysis and transcoding pipelines
  • Multi-stream processing applications

Technical details:

  • Uses GStreamer's decodebin element for intelligent decoding
  • Automatically constructs decoder chains based on stream caps
  • Includes queue elements for each stream to handle threading
  • Video output includes format conversion to standard formats
  • Audio output can optionally include format conversion
  • Handles dynamic stream addition during playback

Implements the VisioForge.Core.MediaBlocks.MediaBlock. Implements the VisioForge.Core.MediaBlocks.IMediaBlockInternals. Implements the IDisposable.

DecryptorBlock

A specialized media block that decrypts AES-encrypted media streams in real-time. This block processes encrypted media data and outputs the decrypted content, enabling secure media distribution and playback of protected content.

Key features:

  • AES decryption with configurable key sizes (128, 192, 256 bits)
  • Support for various AES modes (ECB, CBC, CFB, OFB, CTR, GCM)
  • Real-time stream decryption with minimal latency
  • Transparent passthrough design
  • Compatible with any media type (video, audio, data)
  • Configurable initialization vectors and padding

Common use cases:

  • Playback of DRM-protected content
  • Secure media streaming applications
  • Decrypting downloaded encrypted media files
  • Building secure media distribution systems
  • Implementing custom content protection schemes
  • Compliance with content security requirements

Technical details:

  • Uses hardware-accelerated AES when available
  • Maintains stream timing and metadata
  • Zero-copy operation where possible
  • Thread-safe decryption processing
  • Compatible with standard AES implementations
  • Supports streaming and file-based media

Implements the VisioForge.Core.MediaBlocks.MediaBlock. Implements the VisioForge.Core.MediaBlocks.IMediaBlockInternals. Implements the IDisposable.

DecryptorPlayerBlock

Provides a complete media playback solution for encrypted files by combining decryption, demuxing, and decoding.

EncryptorBlock

A specialized media block that encrypts media streams using AES encryption in real-time. This block processes unencrypted media data and outputs encrypted content, enabling secure media storage and transmission for content protection.

Key features:

  • AES encryption with configurable key sizes (128, 192, 256 bits)
  • Support for various AES modes (ECB, CBC, CFB, OFB, CTR, GCM)
  • Real-time stream encryption with minimal latency
  • Transparent passthrough design
  • Compatible with any media type (video, audio, data)
  • Configurable initialization vectors and padding

Common use cases:

  • Creating DRM-protected content for secure distribution
  • Encrypting media files before storage or transmission
  • Building secure streaming applications
  • Implementing custom content protection systems
  • Protecting sensitive video/audio recordings
  • Compliance with data protection regulations

Technical details:

  • Uses hardware-accelerated AES when available
  • Preserves stream timing and metadata
  • Zero-copy operation where possible
  • Thread-safe encryption processing
  • Compatible with standard AES implementations
  • Pairs with DecryptorBlock for round-trip encryption

Implements the VisioForge.Core.MediaBlocks.MediaBlock. Implements the VisioForge.Core.MediaBlocks.IMediaBlockInternals. Implements the IDisposable.

FrameDoublerBlock

A media block that doubles the frame rate of video streams by duplicating frames. For example, converts 30fps video to 60fps by duplicating each frame with adjusted timestamps.

LiveSyncBlock

Live stream synchronization block that ensures frames are delivered on time for real-time streaming. The LiveSync block synchronizes live streams by managing frame timing, dropping late frames, and duplicating frames when needed to maintain consistent playback. This is essential for live streaming scenarios where maintaining real-time delivery is critical.

Key features:

  • Real-time frame synchronization for live streams
  • Automatic dropping of late frames to prevent backlog
  • Frame duplication when frames arrive too late
  • Maintains consistent playback timing
  • Low-latency operation for live streaming
  • Suitable for live camera feeds, RTSP streams, and other real-time sources

Common use cases:

  • Live camera streaming to ensure real-time delivery
  • RTSP stream synchronization for IP cameras
  • Live broadcasting to maintain consistent frame timing
  • Real-time video conferencing applications
  • Live sports or event streaming
  • Any scenario requiring strict real-time frame delivery

Technical details:

  • Monitors frame timestamps and delivery timing
  • Drops frames that arrive too late to maintain real-time playback
  • Duplicates previous frame if current frame hasn't arrived yet
  • Operates with minimal latency for live scenarios
  • Handles format changes and events properly
  • Configurable latency and timeout parameters

Implements the VisioForge.Core.MediaBlocks.MediaBlock. Implements the VisioForge.Core.MediaBlocks.IMediaBlockInternals. Implements the IDisposable.

MultiQueueBlock

A specialized buffering block that provides multiple synchronized queues for handling multiple media streams. MultiQueue is designed to manage multiple data streams with intelligent buffering and synchronization, ensuring smooth playback and preventing starvation in multi-stream scenarios.

Key features:

  • Multiple independent queues with shared buffering logic
  • Automatic synchronization between streams
  • Dynamic buffer size management
  • Prevention of pipeline stalls in multi-stream scenarios
  • Thread decoupling for each stream
  • Configurable number of input/output pairs

Common use cases:

  • Handling multiple streams from demuxers (video + audio + subtitle)
  • Synchronizing streams with different data rates
  • Buffering before muxing multiple streams
  • Load balancing in parallel processing pipelines
  • Preventing thread blocking in complex pipelines
  • Managing streams with varying latencies

Technical details:

  • Each queue runs in its own thread for isolation
  • Implements intelligent buffering strategies
  • Shares memory limits across all queues
  • Handles format changes dynamically
  • Provides backpressure when buffers are full
  • More efficient than multiple separate Queue blocks

Implements the VisioForge.Core.MediaBlocks.MediaBlock. Implements the VisioForge.Core.MediaBlocks.IMediaBlockInternals. Implements the IDisposable.

NullRendererBlock

Provides a sink element that consumes media data without producing any output, useful for testing and analysis.

ParseBinBlock

Provides automatic parsing and demuxing of media streams without decoding, exposing elementary streams.

QueueBlock

A fundamental buffering block that provides thread decoupling and data buffering in media pipelines. The Queue block acts as a buffer between pipeline elements, allowing upstream and downstream components to operate at different rates and in different threads.

Key features:

  • Thread boundary creation for pipeline parallelization
  • Configurable buffer sizes (bytes, time, buffers)
  • Prevents pipeline stalls and deadlocks
  • Smooth data flow between elements with different processing speeds
  • Automatic format negotiation passthrough
  • Low latency buffering with minimal overhead

Common use cases:

  • Decoupling slow sources from fast sinks
  • Creating thread boundaries in pipelines
  • Buffering before computationally intensive operations
  • Smoothing bursty data flows
  • Preventing blocking in real-time pipelines
  • Isolating pipeline segments for stability

Technical details:

  • Runs source and sink pads in separate threads
  • Implements configurable high/low watermarks
  • Provides backpressure when full
  • Handles format changes and events properly
  • Can operate in push or pull mode
  • Memory-efficient ring buffer implementation

Implements the VisioForge.Core.MediaBlocks.MediaBlock. Implements the VisioForge.Core.MediaBlocks.IMediaBlockInternals. Implements the IDisposable.

SRTPDecryptorBlock

A specialized media block that decrypts SRTP (Secure Real-time Transport Protocol) streams in real-time. This block processes encrypted SRTP packets and outputs decrypted RTP packets, enabling secure media reception and playback of protected content.

Key features:

  • SRTP decryption per RFC 3711 with configurable cipher suites
  • Support for AES-128-ICM and AES-256-ICM decryption
  • HMAC-SHA1 authentication verification with 80-bit or 32-bit tags
  • Real-time SRTP stream decryption with minimal latency
  • Transparent passthrough design for RTP/RTCP packets
  • Compatible with standard SRTP implementations
  • Separate decryption contexts for RTP and RTCP
  • Replay attack protection

Common use cases:

  • Receiving secure VoIP and video conferencing streams
  • Decrypting real-time media broadcasts
  • Building secure streaming client applications with WebRTC
  • Implementing SIP/RTP security (SIPS/SRTP) receivers
  • Compliance with security regulations for real-time communications
  • Decrypting encrypted live video feeds and IP camera streams

Technical details:

  • Uses AES decryption in Counter Mode (ICM/CTR) for low latency
  • Verifies message authentication and detects tampering
  • Maintains RTP timing and sequence numbers
  • Zero-copy operation where possible
  • Thread-safe decryption processing
  • Compatible with standard SRTP key exchange mechanisms (SDES, DTLS-SRTP)
  • Supports streaming and file-based media

Implements the VisioForge.Core.MediaBlocks.MediaBlock. Implements the VisioForge.Core.MediaBlocks.IMediaBlockInternals. Implements the IDisposable.

SRTPEncryptorBlock

A specialized media block that encrypts RTP streams using SRTP (Secure Real-time Transport Protocol) in real-time. This block processes unencrypted RTP packets and outputs SRTP encrypted packets, enabling secure media streaming and transmission for content protection.

Key features:

  • SRTP encryption per RFC 3711 with configurable cipher suites
  • Support for AES-128-ICM and AES-256-ICM encryption
  • HMAC-SHA1 authentication with 80-bit or 32-bit tags
  • Real-time RTP stream encryption with minimal latency
  • Transparent passthrough design for RTP/RTCP packets
  • Compatible with standard SRTP implementations
  • Separate encryption contexts for RTP and RTCP

Common use cases:

  • Securing VoIP and video conferencing streams
  • Protecting real-time media broadcasts
  • Building secure streaming applications with WebRTC
  • Implementing SIP/RTP security (SIPS/SRTP)
  • Compliance with security regulations for real-time communications
  • Encrypting live video feeds and IP camera streams

Technical details:

  • Uses AES encryption in Counter Mode (ICM/CTR) for low latency
  • Provides message authentication and replay protection
  • Preserves RTP timing and sequence numbers
  • Zero-copy operation where possible
  • Thread-safe encryption processing
  • Compatible with standard SRTP key exchange mechanisms (SDES, DTLS-SRTP)
  • Pairs with SRTPDecryptorBlock for round-trip encryption

Implements the VisioForge.Core.MediaBlocks.MediaBlock. Implements the VisioForge.Core.MediaBlocks.IMediaBlockInternals. Implements the IDisposable.

SourceSwitchBlock

Provides dynamic switching between multiple input sources, allowing seamless source changes during pipeline operation.

SuperMediaBlock

Provides a container for combining multiple media blocks into a single reusable unit.

TeeBlock

Provides a media stream splitter that duplicates input data to multiple outputs without modification.

UniversalDecoderBlock

A universal decoder block that replaces decodebin with controlled decoder management. This block automatically selects appropriate decoders for input streams and provides explicit control over memory management to avoid the uncontrolled memory leaks of decodebin.

UniversalDemuxDecoderBlock

A unified demuxer and decoder block that handles the complete pipeline from source to decoding. This block uses UniversalSourceSettings and automatically selects appropriate demuxers, parsers, and decoders based on the actual capabilities reported by GStreamer elements.

UniversalTransformBlock

Universal transform block that allows arbitrary video transformations with configurable caps. Supports 1:N frame transformations through a virtual method that must be overridden.

Enums

UniversalDecoderBlock.DecoderStrategy

Decoder selection strategy.