We’ve now presented several approaches to filtering within an FPGA on this blog. We talked about two of the simplest filters, a generic filter implementation, a cheaper version of the same, and even how to build a moving average filter.

What we haven’t discussed is how to test these filters.

Just to make matters worse, these filters are only the tip of the iceberg of the filters we’d like to test. Not only do we have the current set of filters to test, but I’d also like to present Verilog logic for Symmetric filters, Half-band filters, Hilbert transforms and even some slower versions of those same filters.

Each of these filters is going to need a test bench. At the same time, these test benches are going to have a lot of code in common between each other. Let’s consider, today, what functionality a generic Verilator based filtering test bench requires, so that we can then create a filter test bench engine that can be used, and modified if needed, to test all of these filters.

Abstracting a logic filter

Fig 1: A Generic Filter portlist
A list of the ports of a generic filter: reset, setting taps (i_tap_wr, i_tap), and samples (i_ce, i_sample, and o_sample)

The first step is going to be abstracting what a filter looks like in general so that one basic test bench can support many filters. For this, we can draw upon the design of our most generic filter, and see a set of ports looking roughly like Fig 1. There’s an i_reset strobe to clear the filter’s internal memory, an i_tap_wr signal to indicate that the filter is being loaded with new taps through i_tap, and an i_ce signal to indicate both that a new sample is ready at i_sample and that a sample is ready to be read out at o_sample.

Almost all of our FIR filters, to date, have had a form similar to this one. The biggest exception is the boxcar filter–but with a simple wrapper even this filter can be made to fit this form.

Now, given this form, let’s consider the questions that need to be answered by a test bench:

  1. Does the filter have the impulse response we expect it to have?

  2. Does this filter faithfully apply the impulse response to any signal we give it?

  3. Will a carefully constructed input stream overflow the filter?

  4. What is the frequency response of this filter? If the filter is a lowpass filter, what is it’s passband ripple and stopband depth (as measured)?

IIR filters will have different needs from FIR filters. For now, we’ll just focus on the FIR variety in determining how to abstract this test bench.

Digital FPGA based filters in particular also have some properties that will affect how we interact with them from one test bench to the next. For many of our filters, these are parameters within our design. Those same parameters will affect how we interact with the filter to answer the questions above. The properties that a test harness for all filters will need to know are:

  • IW, the number of bits per input sample

  • TW, the number of bits per tap

  • OW, the number of bits per output sample

  • DELAY, the delay between an impulse input and the first sample that responds to that input.

  • NTAPS, the number of taps in the filter

  • NCLOCKS, the minimum number of clocks between successive input clock enables

Our purpose today will be to build a generic filtering test harness. The goal for this harness will be to handle all of the various filter properties we just discussed above, to interact with a filter of the form shown in Fig 1., and then to answer about the filter’s performance based upon the questions above. You can imagine that this test harness will have the form shown in Fig 2, where the filter itself fits within a virtual socket in this virtual test harness.

Fig 2: C++/Verilator Filtering Test Harness
A picture showing inputs to a filter test harness, as well as conclusions one would like to draw

The harness won’t quite be a complete test bench, but rather a component of a complete test bench. As such it should make it easier to build filter test benches using this harness.

Building a test harness

All of these actions and properties can easily be encapsulated by a C++ test object that will provide us conceptualized access to our filter.

We’ll need to make the C++ class that encapsulates this object easily inheritable, as (for example) the meaning of the taps loaded into one filter (such as a generic one) might be very different from another (such as a symmetric filter). Hence, most of these methods will be virtual.

Let’s outline what one such filter test harness might look like. We’ll let it inherit from a basic TESTB class, such as we’ve discussed before. This TESTB class, if you recall, primarily provided us with the ability to tick() the clock by raising and then lowering the clock input pin, as well as recording our simulation to a VCD file. The final function of this TESTB class is to give us access to a simple reset() function which will set the i_reset pin and toggle the clock.

We’ll define our test harness class with some defaults–things that can be overridden later by the test bench that uses this harness. Further, since we don’t know what the class name of our actual filter is going to be until build time, we’ll make the test harness class, FILTERTB, a template class. That way, the compiler can fill in the template details once they are known.

template <class VFLTR> class FILTERTB : public TESTB<VFLTR> {
	int	*m_hk;
	FILE	*result_fp;

Remember all of those properties we said that a digital filter would have? These will be variables of our object.

	int	m_delay, m_iw, m_ow, m_tw, m_ntaps, m_nclks;
public:
	FILTERTB(void) {
		m_hk = NULL;
		m_delay = 2;
		m_iw    = 16;
		m_ow    = 16;
		m_tw    = 12;
		m_ntaps = 128;
		m_nclks = 1;
		result_fp = NULL;
	}

The test harness will need to know what these properties are. Hence, we’ll start out by handling the setting and referencing of these various properties. This won’t actually change anything about the Verilator generated filter code itself, nor will these values connect to the values often set by parameters within the filter (though it probably should), these routines will just let the actual test bench tell tell the test harness what properties are associated with the filter.

There’s the number of bits allocated to the input sample, IW.

	int	IW(int k)    { m_iw = k;    return m_iw; }
	int	IW(void) const    { return m_iw; }

OW is the number of bits allocated to output samples.

	int	OW(void) const    { return m_ow; }
	int	OW(int k)    { m_ow = k;    return m_ow; }

TW is the number of bits allocated to each of the taps.

	int	TW(void) const    { return m_tw; }
	int	TW(int k)    { m_tw = k;    return m_tw; }

DELAY is the number of clocks between an input and the first filter response that it impacts.

	int	DELAY(void) const { return m_delay; }
	int	DELAY(int k) { m_delay = k; return m_delay; }

Slower filters, may require multiple clocks between each global CE signal, i_ce. We’ll keep track of this feature/capability in the CKPCE, the clocks per CE parameter.

	int	CKPCE(void) const { return m_nclks; }
	int	CKPCE(int k) {
		m_nclks = k;
		if (m_nclks <= 1)
			m_nclks=1;
		return m_nclks;
	}

The minimum number of clocks per CEs is just one, something enforced above.

NTAPS is the number of filter coefficients, or taps in this filter–it is the length of the impulse response. Our test harness will use this to know how many samples following an impulse need to be examined.

	int	NTAPS(void) const { return m_ntaps; }

We’ll cache this impulse response (more later), so any time the number of taps changes (such as with the boxcar filter), the cache will need to be reloaded as well.

	int	NTAPS(int k) {
		m_ntaps = k;
		clear_cache();
		return m_ntaps;
	}

Now with those properties aside, we can start looking at methods (actions) this filtering test harness will need to handle. As you’ll notice from below, we’ll make most of these virtual–so that different types of filters can change them if need be.

We discussed the tick() method as a generic method that will call Verilator and simply step the clock by one tick. It will also record any results into a VCD file (or other file) that may need to be recorded–if desired.

	virtual	void	tick(void);

You may remember some time ago how I recommended debugging DSP algorithms graphically. That recommendation applies to filters as well. For those cases where you want to output the results of your filtering operation to examine, our test harness allows us to open a file wherein we’ll place both input values and output values.

	void	record_results(const char *fname) {
		result_fp = fopen(fname, "w");
	}

In order to apply test vectors and receive independent results, we’ll need to be able to reset the filter to clear any internal state.

	virtual	void	reset(void);

For our test structure, we want to be able to load filter coefficients into our design. This will involve sending a vector of integers, of a given length, to the i_tap port.

	virtual	void	load(int  ntaps,  int *data);

The apply() method will send the given data to the filter as input samples, and return the result from applying the filter.

	virtual	void	apply(int nlen, int *data);

The test() method is nearly identical to the apply() method, with the single exception that the test() method reset()s the filter first. This will be important for sending any particular test vectors to the filter, and for making certain that there aren’t any unintended consequences in the result.

	virtual	void	test(int  nlen, int *data);

As it will turn out, lots of methods are going to depend upon the filter’s impulse response. Let’s overload the [] operator to return coefficients from that response.

	int	operator[](const int tap);

Further, rather than calculating the impulse response. over and over again, we’ll calculate it once and cache the results. That means that any time there is a change in the filter’s coefficients, we’ll need to be able to clear this cache.

	void	clear_cache(void) {
		if (m_hk)
			delete[] m_hk;
		m_hk = NULL;
	}

While we have a method above for loading coefficients into our filter, let’s create another method that not only loads coefficients into our filter but also verifies that the resulting impulse response contains the coefficients that we want.

As with many of these methods, this method is virtual so that filters with less than generic tap structures, such as symmetric filters as one example, can overload this function and provide their own test method.

	virtual	void	testload(int  nlen, int *data);

With all of the above functionality, there’s no reason why we can’t apply particular known inputs to this filter in order to measure its frequency response. Since this is likely to be a common request, we’ll make a method for doing this and place it into our test harness as well. The nfreq describes how many frequencies we are interested examining, and the result will be placed into the response buffer. The algorithm will work by placing complex exponential test vectors into the filter. We can use mag to control the amplitude of these test vectors, with a mag of one being the largest magnitude the filter will accept without overflow.

	virtual	void	response(int nfreq, COMPLEX *response, double mag= 1.0);

Finally, we’ll add two further canned tests. The first, test_overflow(), checks whether a carefully constructed input designed to overflow the filter, does or does not. A true value means that the filter passed this test.

	bool	test_overflow(void);

The second test is appropriate for lowpass filters only. This test will measure the highest passband frequency, the lowest stopband frequency, the depth of the stopband, and the ripple across the passband. It will work for most lowpass frequency response functions, but perhaps not for all lowpass filters.

	void	measure_lowpass(double &fp, double &fs, double &depth, double &ripple);
};

Together, these are the methods (actions) we’ll teach our test harness how to apply to all filters, These methods are declared in this C++ header file. We’ll discuss how to implement them in the next section. For now, let me challenge you to ask the question: have we gotten everything? Are there other particular capabilities that a generic filter test harness should have–capabilities that we’ve overlooked? Feel free to write me and tell me about anything I’ve missed.

Test harness method details

Let’s now turn to the details of how this test harness is implemented. In general, we’ll just work our way through the various methods associated with this test harness C++ class, discussing how to implement each along the way.

Handling different data widths

The first thing we are going to implement is a helper method for adjusting the number of bits in a number. Since Verilator will return numbers with only N bits in them, we’d like to be able to sign extend these values to their full int capability.

static long	sbits(uint64_t val, int b) {
	long	s;

	s = (val << (sizeof(val)*8-b));
	s >>= (sizeof(val)*8-b);
	return	s;
}

Likewise, if the value is unsigned, we’ll also like to extend the value–this time with zero bits.

static unsigned	long ubits(uint64_t val, int b) {
	return	val &= (1<<b)-1;
}

These are just two helper functions that we’ll need when sending bit limited data to Verilator, and when interpreting Verilator’s results.

Advancing the state of the test bench

As you may recall from the discussion of the TESTB class, it has a tick() method within it to handle toggling the clock for us. What it doesn’t have is a means of recording samples for later study with Matlab or Octave. We’ll create that here.

template<class VFLTR> void	FILTERTB<VFLTR>::tick(void) {
	bool	ce;
	long	vec[2];

	ce = (TESTB<VFLTR>::m_core->i_ce);
	vec[0] = sbits(TESTB<VFLTR>::m_core->i_sample, IW());

	TESTB<VFLTR>::tick();

	vec[1] = sbits(TESTB<VFLTR>::m_core->o_result, OW());

	if ((ce)&&(result_fp))
		fwrite(vec, sizeof(long), 2, result_fp);
}

As you can see from above, anytime the i_ce flag is true, the input value and corresponding output value will be written to a file (if open) for later analysis.

Resetting the Filter

The reset() function just sets the i_reset signal for one clock tick. Just to be safe, we’ll make sure the other indication lines are zeros as well.

template<class VFLTR> void	FILTERTB<VFLTR>::reset(void) {
	TESTB<VFLTR>::m_core->i_tap   = 0;
	TESTB<VFLTR>::m_core->i_sample= 0;
	TESTB<VFLTR>::m_core->i_ce    = 0;
	TESTB<VFLTR>::m_core->i_tap_wr= 0;

	TESTB<VFLTR>::reset();

	TESTB<VFLTR>::m_core->i_reset = 0;
}

We’ll use this function to make certain the filter starts from a clean state at the beginning of any tests.

There is a consequence, though, to creating a clean test bench that only provides expected and reasonable inputs, such as setting the reset with all of these other inputs set to zero. In particular, this method of testing logic may not be as complete as a more formal approach that will examine all possible inputs to our filter module–both the reasonable as well as the unreasonable.

In our approach today, we’ll just stick to the reasonable filter inputs.

Applying the filter to a known input

The core function of any linear filter operation is to apply the filter to a given set of data samples. That’s the purpose of the apply() function below. We’ll start by making sure we’re not holding wires high from any prior context that didn’t clean up properly behind itself.

template<class VFLTR> void	FILTERTB<VFLTR>::apply(int nlen, long *data) {
// printf("FILTERTB::apply(%d, ...)\n", nlen);
	TESTB<VFLTR>::m_core->i_reset  = 0;
	TESTB<VFLTR>::m_core->i_tap_wr = 0;
	TESTB<VFLTR>::m_core->i_ce     = 0;
	tick();

Now that we’re set up, let’s walk through each of the input samples. For each input sample, we’ll place it on the i_sample input to the filter, and then read the result from the o_result output of the filter.

	for(int i=0; i<nlen; i++) {
		// Make sure the CE line is high
		TESTB<VFLTR>::m_core->i_ce     = 1;

		// Strip off any excess bits
		TESTB<VFLTR>::m_core->i_sample= ubits(data[i], IW());

		// Apply the filter
		tick();

		// Sign extend the result
		data[i] = sbits(TESTB<VFLTR>::m_core->o_result, OW());

Although all of the filter’s we’ve looked at so far accept one input value and produce one output value on each clock, not every filter is of this type. There’s a lot to be gained by sharing resources across clocks if the sample rate can be guaranteed to be less than half of the system clock rate. To make sure that we can still test filters of this variety, we’ll tick our clock m_nclk-1 additional times here–while holding i_ce low.

		if (m_nclks > 1) {
			TESTB<VFLTR>::m_core->i_ce     = 0;
			for(int k=1; k<m_nclks; k++)
				tick();
		}
	}
	TESTB<VFLTR>::m_core->i_ce     = 0;
}

This is fundamentally how to feed any digital filter, so having this capability within our test harness, only makes.

Loading new filter coefficients

For any filter, that doesn’t have fixed taps, we’ll need to be able to load a new set of coefficients into the filter, As we’ve designed this, doing so requires setting the i_tap_wr input, and then ticking the clock once to load each new coefficient into the filter.

template<class VFLTR> void	FILTERTB<VFLTR>::load(int  ntaps, long *data) {
	TESTB<VFLTR>::m_core->i_reset = 0;
	TESTB<VFLTR>::m_core->i_ce    = 0;
	TESTB<VFLTR>::m_core->i_tap_wr= 1;
	for(int i=0; i<ntaps; i++) {
		// Strip off any excess bits
		TESTB<VFLTR>::m_core->i_tap = ubits(data[i], TW());

		// Apply the filter
		tick();
	}
	TESTB<VFLTR>::m_core->i_tap_wr= 0;

When we are done loading a new set of coefficients into the filter, we’ll clear our cached impulse response values.

	clear_cache();
}

These will need to be rebuilt, but we’ll hold off on that task until they are referenced.

Getting the results from a test vector

A function that is very similar to the apply() method is the test() method below. The idea behind the test() method is that we wish to determine the filter’s response to a given test vector. This method differs from the apply() method in only a few small respects.

First, the test() function begins with a reset() to clear any internal values within the filter. The apply() method didn’t clear any internal filter state.

template<class VFLTR> void	FILTERTB<VFLTR>::test(int  nlen, long *data) {
	const	bool	debug = false;
	assert(nlen > 0);

	reset();

We then call the filter nlen times to feed the test vector into the filter. Well, not quite. We’re going to pass the test vector followed by a series of zeros–one for each delay value. That way, we can quietly remove any filter delay effects from the test vectors response–something not done within the apply() function.

	TESTB<VFLTR>::m_core->i_reset  = 0;
	TESTB<VFLTR>::m_core->i_tap_wr = 0;

	int	tstcounts = nlen+DELAY();
	for(int i=0; i<tstcounts; i++) {
		long	v;

		v = 0;
		TESTB<VFLTR>::m_core->i_ce = 1;

Here’s the next difference: after the first nlen samples, the next several samples are all set to zero.

		// Strip off any excess bits
		if (i >= nlen)
			TESTB<VFLTR>::m_core->i_sample = 0;
		else
			TESTB<VFLTR>::m_core->i_sample = ubits(data[i], IW());

		// Apply the filter
		tick();

		// Sign extend the result
		v = TESTB<VFLTR>::m_core->o_result;

Some filters will require multiple clocks to come to an answer. Let’s make sure we give those filters the clocks they need. We’ll start by making sure that the i_ce line is zero, and then we’ll tick off any extra clocks we are missing.

		TESTB<VFLTR>::m_core->i_ce = 0;
		for(int k=1; k<m_nclks; k++)
			tick();

Further, we’ll only record the results after an initial DELAY() number of samples have been passed into the filter.

		if (i >= DELAY())
			data[i-DELAY()] = sbits(v, OW());
	}
	TESTB<VFLTR>::m_core->i_ce = 0;
}

As you’ll see shortly, this test() method is going to feature prominently in any more sophisticated test—such as measuring the impulse response in the next section.

Measuring the Impulse Response

The impulse response of a filter is defined by its response to a single non-zero value. Many of our test components will depend upon this value, so it’s important that we have a means to both initially measure it, and then to use that measurement later without requiring it to be measured over and again. This is the purpose of the [] operator within the test harness.

Our operator starts out with some very basic parameter checks, just to make certain that we handle responses for out of bounds requests properly.

template<class VFLTR> int	FILTERTB<VFLTR>::operator[](const int tap) {

	if ((tap < 0)||(tap >= 2*NTAPS()))
		return 0;

Measuring a filter’s impulse response is actually just as simple as creating an input to the filter with one non-zero entry (the first one), and then recording how the filter responds to it, hence the name impulse response. Here we do exactly that, save that we we use the maximum negative value for our impulse. We pick this choice for two reasons: first, because the maximum negative value is larger in magnitude than the maximum positive value and so this becomes a test for overflow. Second, by using a really large number, we can also make certain that our result isn’t lost when/if any bits are dropped at the end.

	else if (!m_hk) {
		int	nlen = 2*NTAPS();
		m_hk = new long[nlen];

		// Create an input vector with a single impulse in it
		for(int i=0; i<nlen; i++)
			m_hk[i] = 0;

		// Set our one value to nonzero.  This is the impulse.
		m_hk[0] = -(1<<(IW()-1));

		// Apply the filter to the impulse vector
		test(nlen, m_hk);

Once we get our test vector back, the impulse response coefficient we are looking for will be the response from the filter times the value of our impulse. Hence, we’ll divide the initial impulse value back out and record each sample of our impulse response.

		// Set our m_hk vector based upon the results
		for(int i=0; i<nlen; i++) {
			int	shift;
			shift = IW()-1;
			m_hk[i] >>= shift;
			m_hk[i] = -m_hk[i];
		}
	}

Once we’ve determined the impulse response once, we’ll just look it up for future requests–sparing us the cycles of calculating it over and over again.

	return m_hk[tap];
}

Checking whether the filter load worked

The testload() function is simply a load() filter coefficients function followed by a request for the impulse response, and then a comparison of whether or not the impulse response truly matches the one we wanted.

We’ll use the boolean, mismatch to capture the condition where things don’t match. This will allow us to place the assertion that the impulse response matches the one we loaded at the end of the routine.

template<class VFLTR> void	FILTERTB<VFLTR>::testload(int nlen, long *data) {
	bool	mismatch = false;
	load(nlen, data);
	reset();

	for(int k=0; k<nlen; k++) {

We’ll now compare the measured coefficient, (*this)[k], against the filter coefficient we expect, found in data[k].

		int	m = (*this)[k];
		if (data[k] != m) {
			printf("Err: Data[%d] = %ld != (*this)[%d] = %d\n", k, data[k], k, m);
			mismatch = true;
		}
	}

We’ll make the assertion here that everything matched above.

	if (mismatch) {
		fflush(stdout);
		assert(!mismatch);
	}

We’ll also insist that any coefficients beyond the ones we expect to set should have been read as zero.

	for(int k=nlen; k<2*DELAY(); k++)
		assert(0 == (*this)[k]);
}

This function has been made virtual so that other filter types, such as those whose coefficients are partially fixed (symmetric), half-band, Hilbert, etc), may adjust both how their actual coefficients are loaded, and then also how those loaded coefficients correspond with resulting impulse response.

Checking for overflow

The purpose of the test_overflow() routine is to test whether or not, if the filter is given a carefully constructed input designed to overflow the filter, that the filter will return the correct result. If the filter’s response doesn’t match what we think it should, this will be an indication of a potential overflow.

Fig 3: The Overflow Check Test-Vector
A test vector for use when checking for overflow, having maximum values that match the signs of the filters impulse response

The waveform (test vector) itself is chosen from all maximum values, as shown in Fig 3. In this figure, a desired filter’s impulse response is shown in black. The discretized version of that same response is shown in blue–those are the results that the [] operator will return. The samples in red show the test vector we’ll use.

The magnitude of the coefficients in the test vector is full scale–whether the maximum positive integer that can be represented in IW bits or the maximum negative integer. The signs are adjusted so that when the test sequence lines up with the internal taps of our filter, as shown in Fig 3, the individual coefficients times the test vector’s coefficients will produce the maximum possible positive value. Once all these maximum possible positive values are accumulated, we’ll have the maximum output value that the filter can produce. Any failure to match this maximum output value will be an indication that the filter couldn’t keep up with full-scale values.

That will be our approach below.

template<class VFLTR> bool	FILTERTB<VFLTR>::test_overflow(void) {
	int	nlen = 2*NTAPS();
	long	*input  = new long[nlen],
		*output = new long[nlen];
	long	maxv = (1<<(IW()-1))-1;
	bool	pass = true, tested = false;

	// maxv = 1;

	for(int k=0; k<nlen; k++) {
		// input[v] * (*this)[(NTAPS-1)-v]
		if ((*this)[NTAPS()-1-k] < 0)
			input[k] = -maxv-1;
		else
			input[k] =  maxv;
		output[k]= input[k];
	}

Now that we’ve put this test signal together, let’s see what the filter does with it.

	test(nlen, output);

Now, for every one of those filter outputs, …

	for(int k=0; k<nlen; k++) {
		long	acc = 0;
		bool	all = true;

let’s calculate what the filter output should have been. This involves evaluating the convolution equation (the slow way), and then verifying that we get the output we expect.

		for(int v = 0; v<NTAPS(); v++) {
			if (k-v >= 0) {
				acc += input[k-v] * (*this)[v];
				if (acc < 0)
					all = false;
			} else
				all = false;
		}

It would also be nice to have some confidence that the test vector worked as desired. This is the purpose of the all and tested variables. If the test vector works as desired, then it should hit (at some point) a sweet spot where all of the signs line up. If this happens, then we set all to true.

		if (all)
			tested = true;

Any failure of the output to match our calculated (expected) value indicates that something went wrong–perhaps a filter overflow, perhaps something else in the implementation of the filter. Either way, you’ll want to know. Here are two ways to find out: the assertion below, which will cause the test to end abruptly with an error, and a boolean pass. Hence, if you comment out the assertion, the pass boolean will still tell you the answer.

		pass = (pass)&&(output[k] == acc);
		assert(output[k] == acc);
	}

Finally, now that we are done comparing every output with its expected value, let’s clean up any allocated values and return.

	delete[] input;
	delete[] output;
	return (pass)&&(tested);
}

As you may have noticed, this test not only checks that the output doesn’t overflow, but it also offers a rudimentary check that the filter even works in general–by comparing the filter’s output to a locally calculated convolution. However, as a general convolution checker, this routine is not sufficient. For this reason, we’ll need some other tests to fully convince us that this filter works.

Saved for Later

There are two other required components of this generic filtering test harness: a function to calculate the frequency response of the filter,

template<class VFLTR> void	FILTERTB<VFLTR>::response(int nfreq,
		COMPLEX *rvec, double mag) {
	// ...
}

and a second function to answer the question of how well did the filters’s frequency response actually match a desired response. As we mentioned above, this second function (as built) will only be relevant to lowpass filters.

template<class VFLTR> void FILTERTB<VFLTR>::measure_lowpass(double &fp,
		double &fs, double &depth, double &ripple) {
	// ...
}

Building these two components of our test harness, however, will have to wait for another day and another post.