A WPF Spectrum Analyzer for Audio Visualization (C#)

UPDATE:The WPF Spectrum Analyzer is now part of the WPF Sound Visualization Library. That is where you will find source code for the latest and most-stable versions of the Spectrum Analyzer. Also, this post has been updated and superseded by this one. I’m leaving this one here for historical purposes, but check out the other article for a more performant and scalable version the control.

A coworker and I were recently talking about doing audio visualization with Windows Presentation Foundation (WPF). One of the more common visualization techniques for music applications is to slap in a simple banded spectrum analyzer. I thought such spectrum analyzers would be easy to find, as they’re included in many applications written in toward all sorts of platforms. A bit surprisingly, however, was that such visualizations in WPF aren’t as common as I would have expected. I thought I could use the rarity of such controls as an opportunity to play around with some fast rendering techniques in WPF. I was fairly pleased with the outcome, so I thought I would share the results.

The Spectrum Analyzer Control

The Spectrum Analyzer Control

About Playing Audio

If any of you have tried to do anything with audio in .NET, you’ve probably found the out-of-the-box functionality to be pretty limited. WPF has some enhancements in this area, but it is nowhere near some of the alternatives. Going the other way, one may choose to use direct references to DirectSound to get the power they need. However, this leaves it up to you, the coder, to deal with all the scaling back when the user’s machine doesn’t meet the needs. I find it much easier to use some of the third party sound APIs out there who take care of all these gritty details

My “sound suite” of choice is the BASS API. BASS is loaded with features. Along with getting sound set up to play, it also has the ability to read all sorts of file formats, do effects processing, record sound, and more. However, BASS is unmanaged. To interface BASS with .NET, one will need the aptly named BASS.NET. At a passing glance, one may see BASS.NET as a thin .NET wrapper over the BASS API. However, it actually has some very useful additions. Benefits include tag reading support, useful methods for parsing FFT data, and even some visualizers included (but they are targeted to WinForms/GDI+). It was actually the source code for these visualizers that I used as a starting point for this WPF control. BASS.NET’s author, Bernd Niedergesaess (a.k.a radio42), was kind enough to let me share this source code with you today even though he really did all of the heavy-lifting/pioneering for this project during his development of BASS.NET.

The Spectrum Analyzer

Now, I’m not new to writing Spectrum Analyzers. Previous software engineering jobs had me writing controls for Windows-based Spectrum Analyzers in the past. However, these weren’t for audio visualization, but were more for scientific measurements of radio and microwave signals. One of the things I noticed about audio visualizations is that a lot of accuracy is ignored in favor of making the visualizer look and feel in line with what we perceive. I won’t go into great amounts of detail on this, but when you see funky methods being applied to FFT results in the source code, this is probably what is going on. One of the most common examples of this is that FFT data is never displayed on the Y-axis linearly, but rather using a square root function and a scaling factor. Without this, you see most of the visualizer moving very little, except for the occasional large burst of energy. This is NOT how your ears/brain perceive the sound you’re listening to.

So, let’s take a look at what the control looks like in XAML:

1
2
3
4
5
6
7
8
9
10
11
<my:SpectrumAnalyzer x:Name="spectrumAnalyzer"
                     Height="160"
                     HorizontalAlignment="Stretch"
                     VerticalAlignment="Bottom"
                     BarSpacing="5"
                     AveragePeaks="False"
                     MinimumFrequency="20"
                     MaximumFrequency="20000"
                     BarCount="32"
                     PeakFallDelay="10"
                     BarHeightScaling="Decibel" />

It all looks pretty straightforward. Besides the attributes I’ve shown on the control here, I also have included dependency properties to set brushes for the bars and their peaks, as well as a dependency property for the linearity of displayed channel data. Here’s the XAML for the Spectrum Analyzer control itself.

1
2
3
4
5
6
7
8
9
<UserControl x:Class="BandedSpectrumAnalyzer.SpectrumAnalyzer"
             xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
             xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml">
    <Canvas>
        <Image Name="SpectrumImage"
               HorizontalAlignment="Stretch"
               VerticalAlignment="Stretch" />
    </Canvas>
</UserControl>

Shockingly simple, no? That’s because the meat of this control is in the code-behind. I’ve chosen to do a more classic pixel-based rendering technique for this UserControl. This is good and bad, but not the norm for WPF development. It’s good because it gives me a bit more control in terms of performance. It’s bad in that WPF is meant to be scaled freely, and now it is my responsibility to make that work well. Now, here’s where the real work happens.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Windows;
using System.Windows.Controls;
using System.Windows.Media;
using System.Windows.Media.Imaging;
using System.Windows.Threading;
using Un4seen.Bass;

namespace BandedSpectrumAnalyzer
{
    /// <summary>
    /// Interaction logic for SpectrumAnalyzer.xaml
    /// </summary>
    public partial class SpectrumAnalyzer : UserControl
    {
        #region Fields
        private DispatcherTimer animationTimer;
        private RenderTargetBitmap anaylzerBuffer;
        private DrawingVisual drawingVisual = new DrawingVisual();
        private float[] channelData = new float[2048];
        private float[] channelPeakData;
        private int scaleFactorLinear = 9;
        private int scaleFactorSqr = 2;
        private int maxFFTData = 4096;
        private BASSData maxFFT = (BASSData.BASS_DATA_AVAILABLE | BASSData.BASS_DATA_FFT4096);
        private double bandWidth = 1.0;
        private double barWidth = 1;
        private int maximumFrequencyIndex = 2047;
        private int minimumFrequencyIndex = 0;
        private int sampleFrequency = 44100;
        private int[] barIndexMax;
        private int[] barLogScaleIndexMax;
        private BassEngine bassEngine;
        #endregion

        #region Dependency Property Declarations
        public static readonly DependencyProperty MaximumFrequencyProperty;
        public static readonly DependencyProperty MinimumFrequencyProperty;
        public static readonly DependencyProperty BarCountProperty;
        public static readonly DependencyProperty BarSpacingProperty;
        public static readonly DependencyProperty PeakFallDelayProperty;

        protected static readonly DependencyProperty StreamHandleProperty =
            DependencyProperty.Register("StreamHandle",
            typeof(int),
            typeof(SpectrumAnalyzer));

        public static readonly DependencyProperty FrequencyScaleIsLinearProperty =
            DependencyProperty.Register("FrequencyScaleIsLinear",
            typeof(bool),
            typeof(SpectrumAnalyzer));


        public static readonly DependencyProperty BarHeightScalingProperty =
            DependencyProperty.Register("BarHeightScaling",
            typeof(BarHeightScaling),
            typeof(SpectrumAnalyzer));

        public static readonly DependencyProperty AveragePeaksProperty =
            DependencyProperty.Register("AveragePeaks",
            typeof(bool),
            typeof(SpectrumAnalyzer));

        public static readonly DependencyProperty BarBrushProperty =
            DependencyProperty.Register("BarBrush",
            typeof(Brush),
            typeof(SpectrumAnalyzer));

        public static readonly DependencyProperty PeakBrushProperty =
            DependencyProperty.Register("PeakBrush",
            typeof(Brush),
            typeof(SpectrumAnalyzer));
        #endregion

        #region Dependency Properties
        /// <summary>
        /// The maximum display frequency (right side) for the spectrum analyzer.
        /// </summary>
        /// <remarks>This value should be somewhere between 0 and half of the maximum sample rate. If using
        /// the maximum sample rate, this would be roughly 22000.</remarks>
        [Category("Common")]
        public int MaximumFrequency
        {
            get { return (int)GetValue(MaximumFrequencyProperty); }
            set
            {
                SetValue(MaximumFrequencyProperty, value);
            }
        }

        /// <summary>
        /// The minimum display frequency (left side) for the spectrum analyzer.
        /// </summary>
        [Category("Common")]
        public int MinimumFrequency
        {
            get { return (int)GetValue(MinimumFrequencyProperty); }
            set
            {
                SetValue(MinimumFrequencyProperty, value);
            }
        }

        /// <summary>
        /// The number of bars to show on the sprectrum analyzer.
        /// </summary>
        /// <remarks>A bar's width can be a minimum of 1 pixel. If the BarSpacing and BarCount property result
        /// in the bars being wider than the chart itself, the BarCount will automatically scale down.</remarks>
        [Category("Common")]
        public int BarCount
        {
            get { return (int)GetValue(BarCountProperty); }
            set
            {
                SetValue(BarCountProperty, value);
            }
        }

        /// <summary>
        /// The brush used to paint the bars on the spectrum analyzer.
        /// </summary>
        [Category("Common")]
        public Brush BarBrush
        {
            get { return (Brush)GetValue(BarBrushProperty); }
            set
            {
                SetValue(BarBrushProperty, value);
            }
        }

        /// <summary>
        /// The brush used to paint the peaks on the spectrum analyzer.
        /// </summary>
        [Category("Common")]
        public Brush PeakBrush
        {
            get { return (Brush)GetValue(PeakBrushProperty); }
            set
            {
                SetValue(PeakBrushProperty, value);
            }
        }

        /// <summary>
        /// The delay factor for the peaks falling. This is relative to the
        /// refresh rate of the chart.
        /// </summary>
        [Category("Common")]
        public int PeakFallDelay
        {
            get { return (int)GetValue(PeakFallDelayProperty); }
            set
            {
                SetValue(PeakFallDelayProperty, value);
            }
        }

        /// <summary>
        /// The spacing, in pixels, between the bars.
        /// </summary>
        [Category("Common")]
        public double BarSpacing
        {
            get { return (double)GetValue(BarSpacingProperty); }
            set
            {
                SetValue(BarSpacingProperty, value);
            }
        }

        /// <summary>
        /// If true, the bar height will be displayed linearly with the intensity value.
        /// Otherwise, the bars will be scaled with a square root function.
        /// </summary>
        [Category("Common")]
        public BarHeightScaling BarHeightScaling
        {
            get { return (BarHeightScaling)GetValue(BarHeightScalingProperty); }
            set
            {
                SetValue(BarHeightScalingProperty, value);
            }
        }

        /// <summary>
        /// If true, this will display the frequency scale (X-axis of the spectrum analyzer)
        /// in a linear scale. Otherwise, the scale will be logrithmic.
        /// </summary>
        [Category("Common")]
        public bool FrequencyScaleIsLinear
        {
            get { return (bool)GetValue(FrequencyScaleIsLinearProperty); }
            set
            {
                SetValue(FrequencyScaleIsLinearProperty, value);
            }
        }

        /// <summary>
        /// If true, each bar's peak value will be averaged with the previous
        /// bar's peak. This creates a smoothing effect on the bars.
        /// </summary>
        [Category("Common")]
        public bool AveragePeaks
        {
            get { return (bool)GetValue(AveragePeaksProperty); }
            set
            {
                SetValue(AveragePeaksProperty, value);
            }
        }

        protected int StreamHandle
        {
            get { return (int)GetValue(StreamHandleProperty); }
            set
            {
                SetValue(StreamHandleProperty, value);
                if (StreamHandle != 0)
                {
                    BASS_CHANNELINFO info = new BASS_CHANNELINFO();
                    Bass.BASS_ChannelGetInfo(StreamHandle, info);
                    sampleFrequency = info.freq;
                }
                else
                {
                    sampleFrequency = 44100;
                }
                BarMappingChanged(this, EventArgs.Empty);
            }
        }
        #endregion

        #region Dependency Property Validation
        private static object CoerceMaximumFrequency(DependencyObject d, object value)
        {
            SpectrumAnalyzer spectrumAnalyzer = (SpectrumAnalyzer)d;
            if ((int)value < spectrumAnalyzer.MinimumFrequency)
                return spectrumAnalyzer.MinimumFrequency + 1;
            return value;
        }

        private static object CoerceMinimumFrequency(DependencyObject d, object value)
        {
            int returnValue = (int)value;
            SpectrumAnalyzer spectrumAnalyzer = (SpectrumAnalyzer)d;
            if (returnValue < 0)
                return returnValue = 0;
            spectrumAnalyzer.CoerceValue(MaximumFrequencyProperty);
            return returnValue;
        }

        private static object CoerceBarCount(DependencyObject d, object value)
        {
            int returnValue = (int)value;
            returnValue = Math.Max(returnValue, 1);
            return returnValue;
        }

        private static object CoercePeakFallDelay(DependencyObject d, object value)
        {
            int returnValue = (int)value;
            returnValue = Math.Max(returnValue, 0);
            return returnValue;
        }

        private static object CoerceBarSpacing(DependencyObject d, object value)
        {
            double returnValue = (double)value;
            returnValue = Math.Max(returnValue, 0);
            return returnValue;
        }
        #endregion

        #region Constructors
        static SpectrumAnalyzer()
        {
            // MaximumFrequency
            FrameworkPropertyMetadata maximumFrequencyMetadata = new FrameworkPropertyMetadata(20000);
            maximumFrequencyMetadata.CoerceValueCallback = new CoerceValueCallback(SpectrumAnalyzer.CoerceMaximumFrequency);
            MaximumFrequencyProperty = DependencyProperty.Register("MaximumFrequency", typeof(int), typeof(SpectrumAnalyzer), maximumFrequencyMetadata);

            // MinimumFrequency
            FrameworkPropertyMetadata minimumFrequencyMetadata = new FrameworkPropertyMetadata(0);
            minimumFrequencyMetadata.CoerceValueCallback = new CoerceValueCallback(SpectrumAnalyzer.CoerceMinimumFrequency);
            MinimumFrequencyProperty = DependencyProperty.Register("MinimumFrequency", typeof(int), typeof(SpectrumAnalyzer), minimumFrequencyMetadata);

            // BarCount
            FrameworkPropertyMetadata barCountMetadata = new FrameworkPropertyMetadata(24);
            barCountMetadata.CoerceValueCallback = new CoerceValueCallback(SpectrumAnalyzer.CoerceBarCount);
            BarCountProperty = DependencyProperty.Register("BarCount", typeof(int), typeof(SpectrumAnalyzer), barCountMetadata);

            // BarSpacing
            FrameworkPropertyMetadata barSpacingMetadata = new FrameworkPropertyMetadata(5.0);
            barSpacingMetadata.CoerceValueCallback = new CoerceValueCallback(SpectrumAnalyzer.CoerceBarSpacing);
            BarSpacingProperty = DependencyProperty.Register("BarSpacing", typeof(double), typeof(SpectrumAnalyzer), barSpacingMetadata);

            // PeakFallDelay
            FrameworkPropertyMetadata peakFallDelayMetadata = new FrameworkPropertyMetadata(5);
            peakFallDelayMetadata.CoerceValueCallback = new CoerceValueCallback(SpectrumAnalyzer.CoercePeakFallDelay);
            PeakFallDelayProperty = DependencyProperty.Register("PeakFallDelay", typeof(int), typeof(SpectrumAnalyzer), peakFallDelayMetadata);
        }

        public SpectrumAnalyzer()
        {
            PeakBrush = new SolidColorBrush(Colors.GreenYellow);
            BarBrush = new LinearGradientBrush(Colors.ForestGreen, Colors.DarkGreen, new Point(0, 1), new Point(0, 0));

            InitializeComponent();

            animationTimer = new DispatcherTimer(DispatcherPriority.ApplicationIdle);
            animationTimer.Interval = TimeSpan.FromMilliseconds(25);
            animationTimer.Tick += new EventHandler(animationTimer_Tick);

            DependencyPropertyDescriptor backgroundDescriptor = DependencyPropertyDescriptor.FromProperty(BackgroundProperty, typeof(SpectrumAnalyzer));
            backgroundDescriptor.AddValueChanged(this, AppearanceChanged);
            DependencyPropertyDescriptor barBrushDescriptor = DependencyPropertyDescriptor.FromProperty(BarBrushProperty, typeof(SpectrumAnalyzer));
            barBrushDescriptor.AddValueChanged(this, AppearanceChanged);

            DependencyPropertyDescriptor maxFrequencyDescriptor = DependencyPropertyDescriptor.FromProperty(MaximumFrequencyProperty, typeof(SpectrumAnalyzer));
            maxFrequencyDescriptor.AddValueChanged(this, BarMappingChanged);
            DependencyPropertyDescriptor minFrequencyDescriptor = DependencyPropertyDescriptor.FromProperty(MinimumFrequencyProperty, typeof(SpectrumAnalyzer));
            maxFrequencyDescriptor.AddValueChanged(this, BarMappingChanged);
            DependencyPropertyDescriptor barCountDescriptor = DependencyPropertyDescriptor.FromProperty(BarCountProperty, typeof(SpectrumAnalyzer));
            maxFrequencyDescriptor.AddValueChanged(this, BarMappingChanged);

            BarMappingChanged(this, EventArgs.Empty);

            if (!DesignerProperties.GetIsInDesignMode(this))
            {
                bassEngine = BassEngine.Instance;
                UIHelper.Bind(bassEngine, "ActiveStreamHandle", this, StreamHandleProperty);
                animationTimer.Start();
            }
        }

        void animationTimer_Tick(object sender, EventArgs e)
        {
            UpdateSpectrum();
        }
        #endregion

        #region Event Overrides
        protected override void OnRender(DrawingContext dc)
        {
            base.OnRender(dc);
            anaylzerBuffer = new RenderTargetBitmap((int)RenderSize.Width, (int)RenderSize.Height, 96, 96, PixelFormats.Pbgra32);
            if (SpectrumImage != null)
            {
                SpectrumImage.Source = anaylzerBuffer;
            }
            UpdateSpectrum();
        }

        protected override void OnRenderSizeChanged(SizeChangedInfo sizeInfo)
        {
            base.OnRenderSizeChanged(sizeInfo);
            BarMappingChanged(this, EventArgs.Empty);
        }
        #endregion

        #region Private Utility Methods
        private void UpdateSpectrum()
        {
            if (bassEngine == null || drawingVisual == null || anaylzerBuffer == null || RenderSize.Width < 1 || RenderSize.Height < 1)
                return;

            if (!bassEngine.IsPaused && (StreamHandle == 0 || (GetFFTBuffer(StreamHandle, (int)maxFFT) < 1)))
                return;

            // Clear Canvas
            anaylzerBuffer.Clear();

            using (DrawingContext drawingContext = drawingVisual.RenderOpen())
            {
                // Draw background if applicable.    
                if (Background != null)
                    drawingContext.DrawRectangle(Background, null, new Rect(0, 0, RenderSize.Width, RenderSize.Height));

                // Draw Spectrum Lines
                RenderSpectrumLines(drawingContext);
            }

            anaylzerBuffer.Render(drawingVisual);
        }

        private int GetFFTBuffer(int handle, int length)
        {
            return Un4seen.Bass.Bass.BASS_ChannelGetData(handle, this.channelData, length);
        }

        private void RenderSpectrumLines(DrawingContext dc)
        {
            double fftBucketHeight = 0f;
            double barHeight = 0f;
            double lastPeakHeight = 0f;
            double peakYPos = 0f;
            double height = this.RenderSize.Height;
            int barIndex = 0;
            double peakDotHeight = Math.Max(barWidth / 2.0f, 1);
            double barHeightScale = (height - peakDotHeight);
            const double minDBValue = -90;
            const double maxDBValue = 0;
            const double dbScale = (maxDBValue - minDBValue);

            for (int i = minimumFrequencyIndex; i < maximumFrequencyIndex; i++)
            {
                // If we're paused, keep drawing, but set the current height to 0 so the peaks fall.
                if (bassEngine.IsPaused)
                {
                    barHeight = 0f;
                }
                else // Draw the maximum value for the bar's band
                {
                    switch (BarHeightScaling)
                    {
                        case BandedSpectrumAnalyzer.BarHeightScaling.Decibel:
                            double dbValue = 20 * Math.Log10((double)channelData[i]);
                            fftBucketHeight = ((dbValue - minDBValue) / dbScale) * barHeightScale;
                            break;
                        case BandedSpectrumAnalyzer.BarHeightScaling.Linear:
                            fftBucketHeight = (channelData[i] * scaleFactorLinear) * barHeightScale;
                            break;
                        case BandedSpectrumAnalyzer.BarHeightScaling.Sqrt:
                            fftBucketHeight = (((Math.Sqrt((double)this.channelData[i])) * scaleFactorSqr) * barHeightScale);
                            break;
                    }

                    if (barHeight < fftBucketHeight)
                        barHeight = fftBucketHeight;
                    if (barHeight < 0f)
                        barHeight = 0f;
                }

                // If this is the last FFT bucket in the bar's group, draw the bar.
                int currentIndexMax = FrequencyScaleIsLinear ? barIndexMax[barIndex] : barLogScaleIndexMax[barIndex];
                if (i == currentIndexMax)
                {
                    // Peaks can't surpass the height of the control.
                    if (barHeight > height)
                        barHeight = height;

                    if (AveragePeaks && barIndex > 0)
                        barHeight = (lastPeakHeight + barHeight) / 2;

                    peakYPos = barHeight;

                    if (channelPeakData[barIndex] < peakYPos)
                        this.channelPeakData[barIndex] = (float)peakYPos;
                    else
                        this.channelPeakData[barIndex] = (float)(peakYPos + (PeakFallDelay * this.channelPeakData[barIndex])) / ((float)(PeakFallDelay + 1));

                    double xCoord = BarSpacing + (barWidth * barIndex) + (BarSpacing * barIndex) + 1;

                    // Draw the bars
                    if (BarBrush != null)
                        dc.DrawRectangle(BarBrush, null, new Rect(xCoord, (height - 1) - barHeight, barWidth, barHeight));

                    // Draw the peaks
                    if (PeakBrush != null)
                        dc.DrawRectangle(PeakBrush, null, new Rect(xCoord, (height - 1) - this.channelPeakData[barIndex], barWidth, peakDotHeight));

                    lastPeakHeight = barHeight;
                    barHeight = 0f;
                    barIndex++;
                }
            }
        }
        #endregion

        #region Dependency Property Changed Handlers
        private void AppearanceChanged(object sender, EventArgs e)
        {
            UpdateSpectrum();
        }

        private void BarMappingChanged(object sender, EventArgs e)
        {
            barWidth = Math.Max((int)((RenderSize.Width - (BarSpacing * (BarCount + 1))) / (double)BarCount), 1);
            maximumFrequencyIndex = Math.Min(Utils.FFTFrequency2Index(MaximumFrequency, maxFFTData, sampleFrequency) + 1, 2047);
            minimumFrequencyIndex = Math.Min(Utils.FFTFrequency2Index(MinimumFrequency, maxFFTData, sampleFrequency), 2047);
            bandWidth = Math.Max(((double)(maximumFrequencyIndex - minimumFrequencyIndex)) / RenderSize.Width, 1.0);

            int actualBarCount = Math.Max((int)((RenderSize.Width - BarSpacing) / (barWidth + BarSpacing)), 1);
            channelPeakData = new float[actualBarCount];

            int indexCount = maximumFrequencyIndex - minimumFrequencyIndex;
            int linearIndexBucketSize = (int)Math.Round((double)indexCount / (double)actualBarCount, 0);
            List<int> maxIndexList = new List<int>();
            List<int> maxLogScaleIndexList = new List<int>();
            double maxLog = Math.Log(actualBarCount, actualBarCount);
            for (int i = 1; i < actualBarCount - 1; i++)
            {
                maxIndexList.Add(minimumFrequencyIndex + (i * linearIndexBucketSize));
                int logIndex = (int)((maxLog - Math.Log(actualBarCount - i, actualBarCount)) * indexCount) + minimumFrequencyIndex;
                maxLogScaleIndexList.Add(logIndex);
            }
            maxIndexList.Add(maximumFrequencyIndex);
            maxLogScaleIndexList.Add(maximumFrequencyIndex);
            barIndexMax = maxIndexList.ToArray();
            barLogScaleIndexMax = maxLogScaleIndexList.ToArray();
        }
        #endregion
    }

    public enum BarHeightScaling
    {
        Decibel,
        Sqrt,
        Linear
    }
}

The UpdateSpectrum() method is handling the writing of our bar drawing to the buffer. Using a DrawingVisual and DrawingContext directly gives me great performance. I currently have the timer on this Spectrum Analyzer to draw every 25 ms (which translates to about 40 FPS). Even at this speed, I find the application barely registering 1% CPU, and that’s in the confines of a virtual machine. The actual method I’m using to draw FFT data is inside of RenderSpectrumLines(). You’ll note that we take the peak value in the frequency range of a bar and display that. This is all pretty standard fare for this sort of spectrum analyzer. I encourage you to read up on Fast Fourier Transforms and Audio Processing if this interests you.

Spectrum Analyzer Control With A Light Theme

Spectrum Analyzer Control With A Light Theme

That’s pretty much it! As you can see, I dropped the control in a Window, added some basic playback controls, tag reading, and fancy WPF reflection to give it a nice look. I encourage you to download the source code if you’re interested in including something like this in your own WPF application. If you have questions, see room for improvement, or have useful information I left out, please leave me a comment!

Download the Spectrum Analyzer and Source Code As Part of the WPF Sound Visualization Library
Download Source Code For This Post (Deprecated!)

Update 2/6/2011:

  • Cleaned up some unused references. (Had System.Drawing in there, oops!)
  • Added XML comments to dependency properties on SpectrumAnalyzer UserControl
  • Revised theme selection layout
  • Exposed BarSpacing Property (was accidentally made private)
  • Added AveragePeaks Dependency Property
  • Gradiated backgrounds to demonstrate true WPF transparency support
  • Added option to make the X Axis (Frequency) scale logrithmicly
  • Added another height scaling configuration, decibel. “BarHeightIsLinear” property replaced with “BarHeightScaling”
  • Optimized bar bucket check.
  • Moved all of the Bass logic into an INotifyPropertyChanged class so binding is possible

Extending ZenLibrary – Creating New Rules

Hello everyone, this is an article about my ZenLibrary program aimed at those programmers who wish to either extend it or just know more about what’s going on behind the scenes. If you’re just interested in obtaining the program, click here.

I’ve *attempted* to make it easy for any programmer to extend the rules in ZenLibrary. Since I’m releasing this tutorial at roughly the same time as the actual program, I haven’t actually had a chance to get any feedback on how I might improve the extensibility interface. I do, however, want to make it clear I have no interest in making an elaborate end-user based rule scripting engine. Doing so would a.) screw up the simplicity of the program and b.) drastically change the scope of this software. Throwing in some user-defined regular expressions is about as far as I am willing to go.

Part 0 – Getting Set Up and Getting The Source

Creating new rules in ZenLibrary does NOT require getting the source code. One simply needs to reference “ZenLibrary.RuleBase.dll” that comes with the program (version 0.3 and later). However, I think having the source on hand will make debugging things a bit easier on your part. But, it’s all your call.

This project was done in C#, using WPF, and in Visual Studio 2008. Hopefully none of that made you too uncomfortable. You’ll pretty much need Visual Studio 2008 to deal with this solution. I suppose the Express edition of C# could work, but I haven’t tried it.

The source code is controlled in subversion. Everyone has read permissions on the repository. If you’d like to contribute to the project and need write access, drop me a line. This is an open source project, released using the LGPL.

ZenLibrary SVN Host: http://inchoatethoughts.com/zenlibrarysvn

Part 1 – How It Works

Before we get into how to add our own rules, let’s talk a bit about how this program runs. First, the user chooses a directory to scan through and the rules he wants to test against, and then he presses start. The program now launches a new thread (as to not stall the user interface thread) and begins to scan each directory. This is where our first set of tests begins to run. There are two different types of tests, or “rules” (I’ll be using the terms interchangeably). There are “per-directory” rules. These rules run on any directory with music files. These sorts of rules are useful for doing things like checking for the existence of a particular image file (e.g. album artwork) inside the directory. After the “per-directory” tests run, a set of “per-file” tests will be run against each of the MP3 files in the directory. The “per-files” are useful for checking the content of specific tags, checking for proper file name, etc.

Rule Scanning Flow

Rule Scanning Flow

Forgive the terrible flowchart. Continuing on. This is a .NET program, so anyone extending this software will have the full .NET library at their disposal, which is pretty powerful in and of itself. However, since this is an audio library scanner, the program also makes use of TagLib#. TagLib, for those who aren’t already familiar, is a pretty powerful tag reader. It actually works on all sorts of media, not just MP3s with ID3 tags. TagLib# is then a .NET version of TagLib. It’s released under the LGPL, much like ZenLibrary, so packaging it and distributing it with this software is not a problem. ZenLibrary references TagLib, so when designing rules, don’t forget you have a good hunk of tagging technology to make us of. If you’re looking to do things with tags, just add a reference to “taglib-sharp.dll” as well.

Part 2 – Creating a Rule

Our task today is to essentially create one of the rules/tests that make up the meat of this program. I’ve done some work to make this as easy as possible. Here is what you’ll need to do:

  • Inherit from the “Rule” class
  • Give the rule a name (overload the Name property)
  • Define the TestType (“per-file” or “per-directory”)
  • Give it some test logic

That’s it! If you want a more elaborate rule with configurations that are persisted between application sessions, there is a ConfigurableRule class to inherit from and a few more overloads you’ll need to provide. But, let’s ignore that for now. We’re going to create a very simple rule. It’s purpose will be to detect for the presence of the “discnumber” tag in all of our music files.

Diagram for Abstract Rule Classes

Diagram for Abstract Rule Classes

Set Up The Project

First order of business is getting our add-in project created. Open up Visual Studio and create a new “Class Library” project. It doesn’t really matter what you name it, but you may want to indicate your name and that it is a file with ZenLibrary Rules in it. Pretend your name is Billy Bob. You might name the project “ZenLibrary.BillyBobsRules.” I’ve named my project “SampleRuleLibray.” Once you’ve created the Class Library project, go to your references folder, right click and select “Add Reference,” click the browse tab and go find “ZenLibrary.RuleBase.dll” (it comes packaged with ZenLibrary releases). For this particular “Discnumber” tag rule, we’re also going to be using the TagLib# library as well. Once again, add a reference using “Browse” and select “taglib-sharp.dll,” which should also be in the ZenLibrary directory.

References Defined

References Defined

Inherit From The Rule Class

Now that we have our project set up, we can go ahead and create our rule class. Create a new class file (I called mine “SampleRule.cs”). This class should inherit from “ZenLibrary.RuleBase.Rule.”

1
2
3
4
5
6
7
8
9
using ZenLibrary.RuleBase;

namespace SampleRuleLibrary
{
    public class SampleRule : Rule
    {
       
    }
}

Give The Rule A Name

The rule needs a name. The name shows up in ZenLibrary’s rule panel on the left side and is how the user will identify your rule from the other rules. To specify a name, just override the “Name” property’s get. Pretty simple!

1
2
3
4
5
6
7
8
9
10
11
12
using ZenLibrary.RuleBase;

namespace SampleRuleLibrary
{
    public class SampleRule : Rule
    {
        public override string Name
        {
            get { return "Discnumber Tag"; }
        }        
    }
}

Define The Test Type

The test type determines if your test will be run per music directory or per file. It’s just a simple optimization effort, really. The per-directory test exists so that we don’t have to check for the existence of “folder.jpg” in the directory on every file scan. Conversely, the per-file test exists so we don’t have to rewrite logic to scan all of the files every time we run on a music directory. To define our Test Type, we override the “TestType” property’s get and specify “TestType.FileScan” for per-file and “TestType.DirectoryScan” for per-directory. Our test case is something that needs to be done a per-file basis, so we’ll use “TestType.FileScan” in the example.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
using ZenLibrary.RuleBase;

namespace SampleRuleLibrary
{
    public class SampleRule : Rule
    {
        public override string Name
        {
            get { return "Discnumber Tag"; }
        }

        public override TestType TestType
        {
            get { return TestType.FileScan; }
        }        
    }
}

Give The Test Some Logic

Finally, the meat of the program. Our test. This is where the magic happens. The magic that you define, that is. Anything can happen here, really. Override the “RunTest()” method. You’ll be provided with either a “System.IO.FileInfo” for the file you’re scanning (on a per-file scan) or a “System.IO.DirectoryInfo” for the scanning directory (on a per-directory scan). If the scan is per-file, the DirectoryInfo parameter will be the directory where the file is. If the scan is per-directory, the FileInfo parameter will be null. From here on out, it’s up to you what to do with them. All that matters now is that you return a “RuleTestResult” at the end indicating whether the test has passed or failed. If the test has passed, mark “TestPassed” in the “RuleTestResult.IsPassed” property to true and return it. If it is has failed, mark “RuleTestResult.IsPassed” as false. Also, assign the “RuleTestResult.RuleTestFailedString.” This string is the message that will be displayed in the results box on the ZenLibrary UI. Additionally, you’ll want to specify the location where it failed by assigning “RuleTestResult.ResultPath.” This can pretty much always be the “FullName” property of the provided “DirectoryInfo.”

For the sample test case, I’ve added some logic, using TagLib#, to check whether the “Discnumber” tag is “0.” This is an indication that the “Discnumber” tag has not been assigned, and thus, fails our test.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
using ZenLibrary.RuleBase;

namespace SampleRuleLibrary
{
    public class SampleRule : Rule
    {
        public override string Name
        {
            get { return "Discnumber Tag"; }
        }

        public override TestType TestType
        {
            get { return TestType.FileScan; }
        }

        public override RuleTestResult RunTest(System.IO.DirectoryInfo directoryInfo, System.IO.FileInfo fileInfo)
        {            
            TagLib.File file = TagLib.File.Create(fileInfo.FullName);
            RuleTestResult result = new RuleTestResult();
            if (file.Tag.Disc != 0)
            {
                result.TestPassed = true;
                return result;
            }
            result.ResultPath = directoryInfo.FullName;
            result.RuleTestFailedString = string.Format("File "{0}" does not have the discnumber tag defined.", fileInfo.FullName);
            result.TestPassed = false;
            return result;
        }
    }
}

Boom! We’re done! Yep, that’s really all there is to it. Compile your Class Library and drop the output DLL in the same directory as ZenLibrary.exe. You don’t need to register it or anything. ZenLibrary will automatically detect its presence and instantiate any rules within.

The Sample Rule Running!

The Sample Rule Running!

If you want to see more elaborate rules with custom configuration dialogs and the likes, you can grab the source code from the subversion repository and check out the included rules in the ZenLibrary.RuleBase assembly. You may also want to check out how to write a light plug-in infrastructure via reflection by taking a look in “RuleSet.cs” to see how I instantiate all the rules without programmers having to manually register them.

You can also download the sample rule project here.

Thanks for reading!

Custom Drawing Controls in C# – Manual Double Buffering

I feel like this article is about four years too late. All the cool kids have moved on to WPF or something really awesome, like XNA, for their graphics needs. However, there are still a lot of us playing around in the .NET 2.0 mucky muck for various reasons. Of those still doing .NET 2.0 / WinForms / GDI+ programming, I’m surprised to see how many people draw their custom controls the wrong way. Yep, I said “wrong.” I blame the majority of the C# books out there. Most of them, in the chapter on System.Drawing, just tell everyone to draw their stuff in OnPaint() or in response to a paint event. PLEASE, STOP DOING THIS. I’ll explain why.

One should not have a lot going on in one’s OnPaint method. It should be very small, very straight forward. In 95% of my custom controls, my paint event looks something like this:

1
2
3
4
5
protected override void OnPaint(PaintEventArgs e)
{
    if (!isDisposing && backbufferGraphics != null)
        backbufferGraphics.Render(e.Graphics);
}

That’s it. Just two lines. Here’s what’s going on. OnPaint is the .NET equivalent of the Win32 WM_PAINT message being raised. It happens when a region of your control is “invalidated.” Invalidation occurs when Windows thinks your control needs to be redrawn for some reason. Examples include your control being resized, another window moving in front of it, your window losing and regaining focus, etc. Because OnPaint() is called so frequently and unpredictably, it is not wise to have a lot of complex and CPU-intensive operations going on inside of it.

You may be wondering what we do instead. You be smart and efficient about it. Because the drawing logic is often complex and expensive, you’ll want to do it only as often as necessary. This means having an off-screen buffer that you draw to. When it comes time to actually paint your control on to the screen, your control can just go take a look at our off-screen buffer and copy it to the display. Pretty simple. Now, there is added complexity when you consider that you may be drawing to this buffer at the exact same time your control needs to draw it to the screen. In this scenario, your control would have to site around and wait with an invalidated region until you’ve finished drawing to your buffer. What this usually looks like to the end user is an irritating flicker on the control. The flicker they see is a rapid switch between invalidated control regions (which are usually painted pure white, black, or some other random bit of color) and the drawn buffer.

A quick look around the net will reveal that the solution to flicker is “double buffering.” Now, mind you, none of this is a new concept. People have been doing this for about as long as computer graphics have existed. The difference is, it’s a lot easier now. .NET literally makes it as easy as setting a property on your forms or controls. Beyond that, a programmer used to have to actually manage two buffers manually and swap between them. This is more automated with the introduction of the BufferedGraphics and BufferedGraphicsContext classes. Basically, you draw to the BufferedGraphics class, and it takes care of the double buffering for you. Here is an example constructor and buffer creation routine.

Constructor setting up the form’s double buffering properties:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
public DoubleBufferedControl()
{
    InitializeComponent();

    // Set the control style to double buffer.
    this.SetStyle(ControlStyles.AllPaintingInWmPaint | ControlStyles.UserPaint, true);
    this.SetStyle(ControlStyles.SupportsTransparentBackColor, false);
    this.SetStyle(ControlStyles.OptimizedDoubleBuffer, true);

    // Assign our buffer context.
    backbufferContext = BufferedGraphicsManager.Current;
    initializationComplete = true;

    RecreateBuffers();

    Redraw();
}

And here is an example of creating the buffers

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
private void RecreateBuffers()
{
    // Check initialization has completed so we know backbufferContext has been assigned.
    // Check that we aren't disposing or this could be invalid.
    if (!initializationComplete || isDisposing)
    return;

    // We recreate the buffer with a width and height of the control. The "+ 1"
    // guarantees we never have a buffer with a width or height of 0.
    backbufferContext.MaximumBuffer = new Size(this.Width + 1, this.Height + 1);

    // Dispose of old backbufferGraphics (if one has been created already)
    if (backbufferGraphics != null)
        backbufferGraphics.Dispose();

    // Create new backbufferGrpahics that matches the current size of buffer.
    backbufferGraphics = backbufferContext.Allocate(this.CreateGraphics(),
    new Rectangle(0, 0, Math.Max(this.Width, 1), Math.Max(this.Height, 1)));

    // Assign the Graphics object on backbufferGraphics to "drawingGraphics" for easy reference elsewhere.
    drawingGraphics = backbufferGraphics.Graphics;

    // This is a good place to assign drawingGraphics.SmoothingMode if you want a better anti-aliasing technique.

    // Invalidate the control so a repaint gets called somewhere down the line.
    this.Invalidate();
}

Now, there is one critical place where we make sure to call RecreateBuffers(). That is when the control is resized. The buffers we create in RecreateBuffers() are all sized to match the control’s size. If the buffer wasn’t the size of the control, the control would repaint with a large region of invalid graphics. So, just make sure you have a call to RecreateBuffers in OnResize for your control:

1
2
3
4
5
6
protected override void OnResize(EventArgs e)
{
    base.OnResize(e);
    RecreateBuffers();
    Redraw();
}

That’s pretty much all you need to get a custom double buffered control going. You may have noted that I made a few calls to “Redraw().” Redraw() is where I put my actual drawing logic. What goes in Redraw() is the meat of how you render your control. Just remember, call it only when needed. Don’t go and put the call to Redraw() in OnPaint(). That would be against everything this article tried to teach. This method of rendering is very powerful and I’ve created a number of real-time fast-rendering controls with this mechanism. I’ve been using this method for many years now, so it is tried and true. One of the first times I started double buffering on a custom control was for a custom chart and spectrogram I made for some spectrum analyzer software I was working on. Here is a screenshot:

Realtime Graph and Spectrogram

Realtime Graph and Spectrogram

Thanks for reading. If it helps anyone, I’ve attached a very simple sample project below that contains all the code I’ve shown above. If you have any questions or comments, feel free to use the comments below.

Download Sample Project