WPF SVL supports themes/templating!

I’ve just launched my first CodePlex project, The WPF Sound Visualization Library. This thing is actually an extension of the Spectrum Analyzer control I made and part of a much larger secret project. Essentially, I’m taking all the WPF controls I’m creating in my other big project, dumping them out to their own project, slapping an MIT license on it, and releasing it to the public. They all have the common theme of being controls related to sound visualization/playback. Everything was designed so the look could be customized. I’m always looking for ways to improve their flexibility, so if you have ideas, let me know.

Stereo Waveform Timeline

Introducing The Waveform Timeline

First up is a Waveform. I’ve seen a few people struggling to create these on the BASS forums, so I hope this is useful to the community. Right now, it only has stereo support. I hope to support Mono, Quad, and 5.1 channel displays soon. Aside from displaying a Waveform, this thing has the ability to set track position and a repeat section in the audio stream it is rendering.

One of the challenges in displaying a Waveform in WPF is making sure that it doesn’t kill performance while not appearing blurry. I’ve taken advantage of BitmapCache and some special scaling code so that this is handled well. The Waveform Timeline is broken down into four template parts: the waveform canvas, the timeline canvas, the progress indicator canvas, and the repeat region overlay canvas.

Spectrum Analyzer

Our Old Friend - The Spectrum Analyzer

Next is my old friend the Spectrum Analyzer. Nothing really new to report here. I have lots of detail on this in my previous blog posts. I should note, however, there are a few minor fixes and an added property or two since the last time I posted Spectrum Analyzer source code to this blog.

Album Art Display

Album Art Display - For Those Who Bought Music Before iTunes

Finally, we have an album art display. Basically, it’ll take the album art image (presumably from MP3/AAC tags) and display it in a jewel case. I don’t know how much longer the kids will even remember jewel cases, but I still think they look cool in media applications. I’ve had the Album Art Display in a few of my example applications already, but I’ve refactored it a bit so it was a Control rather than a UserControl.

Check Out The CodePlex Project!

—

The Spectrum Analyzer Control

1. It didn’t support scaling.
2. It was a UserControl, which means that didn’t support custom templates/themes.

These two inadequacies severely crippled much of what makes WPF so great.

Supporting Scaling – Go Go Gadget Vectors!

My first pass at the Spectrum Analyzer was based on a very bad assumption. I assumed, foolishly, that a classic pixel-based rendering loop where I manually controlled the buffers would offer me performance benefits and control that I would not be able to achieve if I used vector-based rendering techniques. When I say “vector-based” I mean rendering Shapes on a Canvas. “Pixel-based” refers to me using a DrawingContext to draw shapes in a fashion similar to how things were done in the ol’ GDI+ days. I’ve left my old post in tact if you want to see how the pixel-based technique was done. Assumption aside, using pixel-based rendering has some consequences when a ScaleTransform gets applied to my control. Basically, things get blurry. It’s the same as resizing an image in Photoshop (that is, in the real world, where we don’t have CSI-like “enhance” commands). Since I’m a big proponent of using ScaleTransforms to “zoom” in on a Window’s content as the Window resizes, I felt like this inadequacy in my control was eating away at my soul.

To set about changing this, I had to first mentally break down the control into vector components. Luckily, the Spectrum Analyzer isn’t terribly complex. It’s essentially a variable number of rectangles laid out horizontally. Then, added to that, is another set of rectangles (of the same quantity) that make up the “falling peaks.”

Spectrum Analyzer - Now With Vector Scaling Support!

Template Support

This brings us to crafting the control as an ACTUAL Control rather than a UserControl. Really, the only component I require is a Canvas on which I can draw all these Rectangles. I’ve set up my Control using the conventional “PART_” naming convention (see TemplatePartAttribute). To actually get the bars or falling peaks to draw, an application author will also need to define the BarStyle and/or PeakStyle dependency properties on the control. Without these, they’ll still technically “draw,” but they’ll have no brush and be invisible.

Here’s an example of a style and template for the SpectrumAnalyzer control.

Revised Control Code
I’ve revised the rendering process a bit. When I first wrote the control code, there was a render method that drew a bunch of rectangles onto a DrawingVisual. Instead of drawing directly, I now just change the coordinates/size of a bunch of rectangles that are stored as fields on the Spectrum Analyzer. I let WPF handle their rendering. Thus, my “RenderSpectrumLines” method became a “UpdateSpectrumShapes” method. Also, since this is now a control, there is no XAML to go along with it per se. However, there would be a default template for it if it were placed in a Control Library. Here’s the updated code:

On Performance / Conclusion
So, where does this leave us? A peak at the Task Manager delivers a pretty telling tale. On my system, the control continues to use very little processor power. What’s even better, however, is you’ll see far less churning of the memory usage. It seems that when you let WPF handle the rendering, it gets to have better control of what sort of buffers get created and how quickly they’re cleared from memory. When we did it manually with our RenderTargetBitmap, we were at the mercy of the garbage collector.

I think there are still a number of controls where one will need to fall back on to pixel based rendering (e.g., a spectrogram), but this one lent itself well to being drawn with vectors. That said, I think the current iteration of this Control is far closer to how one should design their controls for other people to use. People incorporating this into their own applications should find it much more flexible in style, as well as much more performant.

Download Source Code

—

A coworker and I were recently talking about doing audio visualization with Windows Presentation Foundation (WPF). One of the more common visualization techniques for music applications is to slap in a simple banded spectrum analyzer. I thought such spectrum analyzers would be easy to find, as they’re included in many applications written in toward all sorts of platforms. A bit surprisingly, however, was that such visualizations in WPF aren’t as common as I would have expected. I thought I could use the rarity of such controls as an opportunity to play around with some fast rendering techniques in WPF. I was fairly pleased with the outcome, so I thought I would share the results.

The Spectrum Analyzer Control

About Playing Audio

If any of you have tried to do anything with audio in .NET, you’ve probably found the out-of-the-box functionality to be pretty limited. WPF has some enhancements in this area, but it is nowhere near some of the alternatives. Going the other way, one may choose to use direct references to DirectSound to get the power they need. However, this leaves it up to you, the coder, to deal with all the scaling back when the user’s machine doesn’t meet the needs. I find it much easier to use some of the third party sound APIs out there who take care of all these gritty details

My “sound suite” of choice is the BASS API. BASS is loaded with features. Along with getting sound set up to play, it also has the ability to read all sorts of file formats, do effects processing, record sound, and more. However, BASS is unmanaged. To interface BASS with .NET, one will need the aptly named BASS.NET. At a passing glance, one may see BASS.NET as a thin .NET wrapper over the BASS API. However, it actually has some very useful additions. Benefits include tag reading support, useful methods for parsing FFT data, and even some visualizers included (but they are targeted to WinForms/GDI+). It was actually the source code for these visualizers that I used as a starting point for this WPF control. BASS.NET’s author, Bernd Niedergesaess (a.k.a radio42), was kind enough to let me share this source code with you today even though he really did all of the heavy-lifting/pioneering for this project during his development of BASS.NET.

The Spectrum Analyzer

Now, I’m not new to writing Spectrum Analyzers. Previous software engineering jobs had me writing controls for Windows-based Spectrum Analyzers in the past. However, these weren’t for audio visualization, but were more for scientific measurements of radio and microwave signals. One of the things I noticed about audio visualizations is that a lot of accuracy is ignored in favor of making the visualizer look and feel in line with what we perceive. I won’t go into great amounts of detail on this, but when you see funky methods being applied to FFT results in the source code, this is probably what is going on. One of the most common examples of this is that FFT data is never displayed on the Y-axis linearly, but rather using a square root function and a scaling factor. Without this, you see most of the visualizer moving very little, except for the occasional large burst of energy. This is NOT how your ears/brain perceive the sound you’re listening to.

So, let’s take a look at what the control looks like in XAML:

It all looks pretty straightforward. Besides the attributes I’ve shown on the control here, I also have included dependency properties to set brushes for the bars and their peaks, as well as a dependency property for the linearity of displayed channel data. Here’s the XAML for the Spectrum Analyzer control itself.

Shockingly simple, no? That’s because the meat of this control is in the code-behind. I’ve chosen to do a more classic pixel-based rendering technique for this UserControl. This is good and bad, but not the norm for WPF development. It’s good because it gives me a bit more control in terms of performance. It’s bad in that WPF is meant to be scaled freely, and now it is my responsibility to make that work well. Now, here’s where the real work happens.

The UpdateSpectrum() method is handling the writing of our bar drawing to the buffer. Using a DrawingVisual and DrawingContext directly gives me great performance. I currently have the timer on this Spectrum Analyzer to draw every 25 ms (which translates to about 40 FPS). Even at this speed, I find the application barely registering 1% CPU, and that’s in the confines of a virtual machine. The actual method I’m using to draw FFT data is inside of RenderSpectrumLines(). You’ll note that we take the peak value in the frequency range of a bar and display that. This is all pretty standard fare for this sort of spectrum analyzer. I encourage you to read up on Fast Fourier Transforms and Audio Processing if this interests you.

Spectrum Analyzer Control With A Light Theme

That’s pretty much it! As you can see, I dropped the control in a Window, added some basic playback controls, tag reading, and fancy WPF reflection to give it a nice look. I encourage you to download the source code if you’re interested in including something like this in your own WPF application. If you have questions, see room for improvement, or have useful information I left out, please leave me a comment!

Download the Spectrum Analyzer and Source Code As Part of the WPF Sound Visualization Library
Download Source Code For This Post (Deprecated!)

Update 2/6/2011:

• Cleaned up some unused references. (Had System.Drawing in there, oops!)
• Added XML comments to dependency properties on SpectrumAnalyzer UserControl
• Revised theme selection layout
• Exposed BarSpacing Property (was accidentally made private)
• Added AveragePeaks Dependency Property
• Gradiated backgrounds to demonstrate true WPF transparency support
• Added option to make the X Axis (Frequency) scale logrithmicly
• Added another height scaling configuration, decibel. “BarHeightIsLinear” property replaced with “BarHeightScaling”
• Optimized bar bucket check.
• Moved all of the Bass logic into an INotifyPropertyChanged class so binding is possible

I’ve *attempted* to make it easy for any programmer to extend the rules in ZenLibrary. Since I’m releasing this tutorial at roughly the same time as the actual program, I haven’t actually had a chance to get any feedback on how I might improve the extensibility interface. I do, however, want to make it clear I have no interest in making an elaborate end-user based rule scripting engine. Doing so would a.) screw up the simplicity of the program and b.) drastically change the scope of this software. Throwing in some user-defined regular expressions is about as far as I am willing to go.

Part 0 – Getting Set Up and Getting The Source

Creating new rules in ZenLibrary does NOT require getting the source code. One simply needs to reference “ZenLibrary.RuleBase.dll” that comes with the program (version 0.3 and later). However, I think having the source on hand will make debugging things a bit easier on your part. But, it’s all your call.

This project was done in C#, using WPF, and in Visual Studio 2008. Hopefully none of that made you too uncomfortable. You’ll pretty much need Visual Studio 2008 to deal with this solution. I suppose the Express edition of C# could work, but I haven’t tried it.

The source code is controlled in subversion. Everyone has read permissions on the repository. If you’d like to contribute to the project and need write access, drop me a line. This is an open source project, released using the LGPL.

Part 1 – How It Works

Before we get into how to add our own rules, let’s talk a bit about how this program runs. First, the user chooses a directory to scan through and the rules he wants to test against, and then he presses start. The program now launches a new thread (as to not stall the user interface thread) and begins to scan each directory. This is where our first set of tests begins to run. There are two different types of tests, or “rules” (I’ll be using the terms interchangeably). There are “per-directory” rules. These rules run on any directory with music files. These sorts of rules are useful for doing things like checking for the existence of a particular image file (e.g. album artwork) inside the directory. After the “per-directory” tests run, a set of “per-file” tests will be run against each of the MP3 files in the directory. The “per-files” are useful for checking the content of specific tags, checking for proper file name, etc.

Rule Scanning Flow

Forgive the terrible flowchart. Continuing on. This is a .NET program, so anyone extending this software will have the full .NET library at their disposal, which is pretty powerful in and of itself. However, since this is an audio library scanner, the program also makes use of TagLib#. TagLib, for those who aren’t already familiar, is a pretty powerful tag reader. It actually works on all sorts of media, not just MP3s with ID3 tags. TagLib# is then a .NET version of TagLib. It’s released under the LGPL, much like ZenLibrary, so packaging it and distributing it with this software is not a problem. ZenLibrary references TagLib, so when designing rules, don’t forget you have a good hunk of tagging technology to make us of. If you’re looking to do things with tags, just add a reference to “taglib-sharp.dll” as well.

Part 2 – Creating a Rule

Our task today is to essentially create one of the rules/tests that make up the meat of this program. I’ve done some work to make this as easy as possible. Here is what you’ll need to do:

• Inherit from the “Rule” class
• Give the rule a name (overload the Name property)
• Define the TestType (“per-file” or “per-directory”)
• Give it some test logic

That’s it! If you want a more elaborate rule with configurations that are persisted between application sessions, there is a ConfigurableRule class to inherit from and a few more overloads you’ll need to provide. But, let’s ignore that for now. We’re going to create a very simple rule. It’s purpose will be to detect for the presence of the “discnumber” tag in all of our music files.

Diagram for Abstract Rule Classes

Set Up The Project

First order of business is getting our add-in project created. Open up Visual Studio and create a new “Class Library” project. It doesn’t really matter what you name it, but you may want to indicate your name and that it is a file with ZenLibrary Rules in it. Pretend your name is Billy Bob. You might name the project “ZenLibrary.BillyBobsRules.” I’ve named my project “SampleRuleLibray.” Once you’ve created the Class Library project, go to your references folder, right click and select “Add Reference,” click the browse tab and go find “ZenLibrary.RuleBase.dll” (it comes packaged with ZenLibrary releases). For this particular “Discnumber” tag rule, we’re also going to be using the TagLib# library as well. Once again, add a reference using “Browse” and select “taglib-sharp.dll,” which should also be in the ZenLibrary directory.

References Defined

Inherit From The Rule Class

Now that we have our project set up, we can go ahead and create our rule class. Create a new class file (I called mine “SampleRule.cs”). This class should inherit from “ZenLibrary.RuleBase.Rule.”

Give The Rule A Name

The rule needs a name. The name shows up in ZenLibrary’s rule panel on the left side and is how the user will identify your rule from the other rules. To specify a name, just override the “Name” property’s get. Pretty simple!

Define The Test Type

The test type determines if your test will be run per music directory or per file. It’s just a simple optimization effort, really. The per-directory test exists so that we don’t have to check for the existence of “folder.jpg” in the directory on every file scan. Conversely, the per-file test exists so we don’t have to rewrite logic to scan all of the files every time we run on a music directory. To define our Test Type, we override the “TestType” property’s get and specify “TestType.FileScan” for per-file and “TestType.DirectoryScan” for per-directory. Our test case is something that needs to be done a per-file basis, so we’ll use “TestType.FileScan” in the example.

Give The Test Some Logic

Finally, the meat of the program. Our test. This is where the magic happens. The magic that you define, that is. Anything can happen here, really. Override the “RunTest()” method. You’ll be provided with either a “System.IO.FileInfo” for the file you’re scanning (on a per-file scan) or a “System.IO.DirectoryInfo” for the scanning directory (on a per-directory scan). If the scan is per-file, the DirectoryInfo parameter will be the directory where the file is. If the scan is per-directory, the FileInfo parameter will be null. From here on out, it’s up to you what to do with them. All that matters now is that you return a “RuleTestResult” at the end indicating whether the test has passed or failed. If the test has passed, mark “TestPassed” in the “RuleTestResult.IsPassed” property to true and return it. If it is has failed, mark “RuleTestResult.IsPassed” as false. Also, assign the “RuleTestResult.RuleTestFailedString.” This string is the message that will be displayed in the results box on the ZenLibrary UI. Additionally, you’ll want to specify the location where it failed by assigning “RuleTestResult.ResultPath.” This can pretty much always be the “FullName” property of the provided “DirectoryInfo.”

For the sample test case, I’ve added some logic, using TagLib#, to check whether the “Discnumber” tag is “0.” This is an indication that the “Discnumber” tag has not been assigned, and thus, fails our test.

Boom! We’re done! Yep, that’s really all there is to it. Compile your Class Library and drop the output DLL in the same directory as ZenLibrary.exe. You don’t need to register it or anything. ZenLibrary will automatically detect its presence and instantiate any rules within.

The Sample Rule Running!

If you want to see more elaborate rules with custom configuration dialogs and the likes, you can grab the source code from the subversion repository and check out the included rules in the ZenLibrary.RuleBase assembly. You may also want to check out how to write a light plug-in infrastructure via reflection by taking a look in “RuleSet.cs” to see how I instantiate all the rules without programmers having to manually register them.

You can also download the sample rule project here.

Thanks for reading!

One should not have a lot going on in one’s OnPaint method. It should be very small, very straight forward. In 95% of my custom controls, my paint event looks something like this: