Table of Contents

Class AudioEventDetectorSettings

Namespace
VisioForge.Core.Types.X.AI
Assembly
VisioForge.Core.dll

Settings for the audio-event-detection block (YAMNet / AudioSet, 521 classes).

public class AudioEventDetectorSettings

Inheritance

Inherited Members

Remarks

Unlike the vision AI settings, this type does NOT derive from VisioForge.Core.Types.X.AI.OnnxInferenceSettings: YAMNet consumes a 16 kHz mono audio waveform rather than an image, so the image-size / letterbox / NCHW knobs do not apply. The YAMNet ONNX model is not shipped in the SDK NuGet packages: the application supplies it and points VisioForge.Core.Types.X.AI.AudioEventDetectorSettings.ModelPath at it. The block never downloads anything itself. The block accumulates tapped PCM into an unbounded queue (audio is never discarded — a source that outruns the worker only grows the backlog), runs inference on a background worker over VisioForge.Core.Types.X.AI.AudioEventDetectorSettings.WindowSeconds windows, smooths the per-class scores over VisioForge.Core.Types.X.AI.AudioEventDetectorSettings.SmoothingWindows windows, and raises one event per continuous sound with hysteresis (VisioForge.Core.Types.X.AI.AudioEventDetectorSettings.Threshold to enter, VisioForge.Core.Types.X.AI.AudioEventDetectorSettings.Threshold * VisioForge.Core.Types.X.AI.AudioEventDetectorSettings.ReleaseRatio to leave).

Every property here is read once, when the block is built. Set them before starting the pipeline: changing one on a running pipeline has no effect until the block is rebuilt (the next start).

Constructors

AudioEventDetectorSettings()

Initializes a new instance of the VisioForge.Core.Types.X.AI.AudioEventDetectorSettings class.

public AudioEventDetectorSettings()

AudioEventDetectorSettings(string)

Initializes a new instance of the VisioForge.Core.Types.X.AI.AudioEventDetectorSettings class with a model path.

public AudioEventDetectorSettings(string modelPath)

Parameters

modelPath string

The absolute path to the YAMNet ONNX model file (yamnet.onnx).

Properties

ClassFilter

Gets or sets an optional allowlist of AudioSet class display names (case-insensitive, e.g. "Siren", "Dog", "Glass"). When non-empty, only these classes can raise events; all other classes are ignored. null or empty (the default) detects every class. Names must match an AudioSet class exactly: a name that does not is ignored with a warning, and a non-empty filter in which NO name matches fails the block's build with an error rather than quietly detecting everything.

public IList<string> ClassFilter { get; set; }

Property Value

IList<string>

DeviceId

Gets or sets the hardware device id used when a GPU provider is selected. Defaults to 0.

public int DeviceId { get; set; }

Property Value

int

EnableScoresEvent

Gets or sets a value indicating whether the block raises a lightweight top-K scores event for every processed window (useful for a live meter). Off by default to avoid per-window event traffic.

public bool EnableScoresEvent { get; set; }

Property Value

bool

ModelPath

Gets or sets the absolute path to the YAMNet ONNX model file. Required. The model takes a 1-D 16 kHz mono float waveform and returns a per-frame [frames, 521] AudioSet score vector.

public string ModelPath { get; set; }

Property Value

string

Provider

Gets or sets the execution provider for the ONNX session. Defaults to VisioForge.Core.Types.X.AI.OnnxExecutionProvider.Auto, which uses the fastest hardware provider present in the loaded ONNX Runtime native build and falls back to the CPU. YAMNet is tiny — on the CPU, roughly 0.6 ms for the default 0.975 s window and 1.8 ms for the 5 s maximum, over 1000× real time either way — so the CPU is usually sufficient.

public OnnxExecutionProvider Provider { get; set; }

Property Value

OnnxExecutionProvider

ReleaseRatio

Gets or sets the hysteresis release ratio. An active event ends when its smoothed score falls below VisioForge.Core.Types.X.AI.AudioEventDetectorSettings.Threshold * VisioForge.Core.Types.X.AI.AudioEventDetectorSettings.ReleaseRatio. A value below 1 keeps a single continuous sound from flickering into many events. Defaults to 0.7. A value outside 0.01..1 (or NaN) is clamped into that range with a warning naming the value it replaced: the floor keeps an active event releasable (a ratio of 0 yields a release threshold of 0, which no score could ever fall below, latching every started event until end-of-stream), and the ceiling keeps the release threshold from exceeding the enter threshold.

public float ReleaseRatio { get; set; }

Property Value

float

SmoothingWindows

Gets or sets the number of consecutive windows the per-class scores are averaged over (a moving average) to suppress one-window blips before thresholding. 1 disables smoothing. Defaults to 3. The average covers the windows seen so far rather than the configured count, so it ramps in at the start of a stream: a sound that is present from the very first window is reported immediately, with no start-of-stream latency. The trade-off is that smoothing only takes effect once enough windows have accumulated — a one-window blip in the very first window is thresholded unsmoothed and can raise an event. Raising this trades detection latency (for sounds that start later in the stream) for fewer false positives.

public int SmoothingWindows { get; set; }

Property Value

int

Threshold

Gets or sets the confidence threshold a class must reach for an event to start. Defaults to 0.5. The model's scores live in 0..1, and a value outside 0.01..1 is clamped into it with a warning: the ceiling keeps the threshold reachable, and the floor keeps an active event releasable (it ends below VisioForge.Core.Types.X.AI.AudioEventDetectorSettings.Threshold * VisioForge.Core.Types.X.AI.AudioEventDetectorSettings.ReleaseRatio, which at a threshold of 0 no score could ever fall below, latching every class until end-of-stream). Clamping does not rescue a mis-scaled value — a 0..100 slider bound straight through still detects nothing — so the warning names the value it replaced.

public float Threshold { get; set; }

Property Value

float

TopK

Gets or sets the number of highest-scoring classes reported in the scores event when VisioForge.Core.Types.X.AI.AudioEventDetectorSettings.EnableScoresEvent is true. Defaults to 5 and is clamped to 1..521 (the class count).

public int TopK { get; set; }

Property Value

int

WindowSeconds

Gets or sets the inference window length, in seconds. Each window is classified independently and its per-frame scores are averaged. Defaults to 0.975 s — one 0.96 s YAMNet patch plus a small margin. Clamped to the range 0.96–5 s: YAMNet frames its input into 0.96 s patches and zero-pads anything shorter, so a sub-patch window is still scored — but against synthetic silence more than against the audio, which is why the floor exists.

public double WindowSeconds { get; set; }

Property Value

double