Locally resetting an AXI component

Earlier this week, I came across an AXI component designed by a user who was struggling to figure out why his component wasn’t working. In the end the first answer to why his core wasn’t working was that he was neither simulating, nor formally verifying his design. The second answer was that he had a clock domain crossing issue that wasn’t part of his post.

Looking through his logic, however, I found a third reason why he might be struggling–his reset logic. This reset code contained the following logic,

input	wire	LOCAL_RESET;

always @(posedge M_AXI_ACLK)
if (M_AXI_ARESETN == 0 || LOCAL_RESET == 1'b1)
	axi_arvalid <= 1'b0;
else if (!axi_arvalid && /* whatever else */)
	// rest of logic follows

always @(posedge M_AXI_ACLK)
if (M_AXI_ARESETN == 0 || LOCAL_RESET == 1'b1)
	state <= IDLE;
else case(state)
begin
	// State machine logic
end

always @(posedge M_AXI_ACLK)
if (M_AXI_ARESETN == 0 || LOCAL_RESET == 1'b1 || state == IDLE)
	read_index <= 0;
else if (M_AXI_RVALID && M_AXI_RREADY && read_index != MAX_INDEX)
	read_index <= read_index + 1;

always @(posedge M_AXI_ACLK)
if (M_AXI_ARESETN == 0 || LOCAL_RESET == 1'b1)
	axi_rready <= 1'b0;
else
	// rest of logic follows

The design also contained an associated slave, also containing a LOCAL_RESET wire, although none of the logic within the slave depended upon the same LOCAL_RESET.

It shouldn’t be too hard to come up with a scenario that will cause such a design to read results into the wrong address, or worse to hang the rest of the design by dropping axi_rready just before M_AXI_RVALID gets set.

Although this didn’t turn out to be what the user was struggling from in the end, it still brings up an interesting question: How shall an AXI component be locally reset? By locally reset, I’m referring to resetting the component without pulling S_AXI_ARESETN low and resetting every component on the bus.

It’s not as simple as it might sound.

Locally resetting an AXI Slave component

Fig 1. Resetting an AXI slave

Resetting an AXI Slave is the easier type of component to reset. The key to resetting a slave with an AXI-lite interface is to reset the registers within the design, rather than the entire design, as shown in Fig. 1 on the right.

As an example, in our example AXI slave design, we included the following logic for updating an register in a design having an AXI-lite interface:

initial	r0 = 0;
always @(posedge S_AXI_ACLK)
if (!S_AXI_ARESETN)
begin
	r0 <= 0;
	// ...
end else if (axil_write_ready)
begin
	case(awskd_addr)
	2'b00: r0 <= wskd_r0;
	// ...
	endcase
end

This register logic is the only logic that needs to adjust in order to implement a local reset for an AXI-Lite slave.

First, we’d need a reset signal from somewhere. Such a signal could come from either an external input port (that was synchronous with the design clock),

input	wire	LOCAL_RESET;

or it could come from one of the bus-accessible registers within the design,

initial	LOCAL_RESET = 1;
always @(posedge S_AXI_ACLK)
if (!S_AXI_ARESETN || LOCAL_RESET)
begin
	LOCAL_RESET <= 0;
	// ...
end else if (axil_write_ready)
begin
	if (awskd_addr == RESET_ADDRESS)
		LOCAL_RESET <= wskd_data[RESET_BIT];
end

However you create your local reset signal, once the LOCAL_RESET signal is available, the registers within the design can be reset using it.

initial	r0 = 0;
always @(posedge S_AXI_ACLK)
if (!S_AXI_ARESETN || LOCAL_RESET)
begin
	r0 <= 0;
	// ...
end else if (axil_write_ready)
begin
	case(awskd_addr)
	2'b00: r0 <= wskd_r0;
	// ...
	endcase
end

Notice that I’m not resetting any of the AXI signals, but just the various registers that would be set by them. This is the key to not hanging the bus.

Returning an error during the local reset

If you want, you could also adjust the responses to various requests as well, so that during a reset the bus would return an error. In this case, you’d change,

always @(*)
	S_AXI_BRESP = 2'b00;

into something where any write during a LOCAL_RESET would produce an error. The key to this transformation, however, is making certain that the S_AXI_BRESP signal still only every changes when the outgoing transaction isn’t stalled. This is then as simple as,

// Define the AXI response possibilities
localparam [1:0]	OKAY = 2'b00,
			SLVERR = 2'b10;

always @(posedge S_AXI_ACLK)
if (!S_AXI_BVALID || S_AXI_BREADY)
	S_AXI_BRESP <= (LOCAL_RESET) ? SLVERR : OKAY;

A similar update to S_AXI_RRESP could accomplish the same effect there as well.

The difficult part of this change, however, is that a bus error will tend to cause any user program to crash. Bus error handling is typically done by the operating system and, well, let me ask, if you aren’t using an operating system then is your embedded software ready to handle that challenge?

Retrofitting an existing component

Fig 2. Using a firewall to locally reset an existing core

But what if your AXI slave component already exists? In this case, you might need to retrofit a slave core with a local reset option. Pulling the S_AXI_ARESETN line low of that component only might look like a tempting option, but what would this do to any ongoing bus transactions? It would hang the bus, bringing down the rest of the design. This is unacceptable.

However, if this is the sort of functionality you are looking for, just without hanging the bus, then there is a way to accomplish it. Simply place an AXI firewall between your component and the rest of the bus, and then adjust the reset line to the core below the firewall as you see fit. Any ongoing slave transaction hiccups will be caught by the AXI firewall, and used to prolong the reset of the component core–returning AXI SLVERRs (a type of AXI bus error) until the core is fully re-integrated back into your system.

An Example

Perhaps an example might clarify this whole idea. I used the register based reset approach when resetting the AXI-Lite version of my Wishbone scope.

If you aren’t familiar with the (now possibly misnamed) Wishbone Scope, it’s a basic bus-based internal logic analyzer. It’s a really easy way to capture, record, and report back to you what’s going on within your design with one caveat: It will only work if the bus never hangs.

This is one of those times and reasons why I like Wishbone: If you just drop the cycle line, then everything resets. I can then go back into a design and (usually) find out what’s going on if I have to.

Not so with AXI: the AXI bus protocol has no bus abort capability. Any misbehaving AXI bus component will hang the bus. This is why it’s so important to formally verify your AXI components–simulation just doesn’t tend to be thorough enough to check all the functionality of an AXI based interaction. Formal methods, on the other hand, will check every possibility.

Fig 3. Crossing clock domains with a reset request

So how does the reset work in my AXI-Lite scope?

Well, first, it’s not quite so simple as we described above. The scope is designed to be able to (optionally) handle inputs coming from a second clock domain–one associated with the data source. That means that the reset won’t be complete until it has crossed from the bus clock domain to the data clock domain and back again.

So let’s follow this through. There are three parts to the internal reset request logic. First, if a reset has been requested, we clear it on any completion–independent of any new request.

always @(posedge bus_clock)
begin
	// Reset logic
	if (bw_reset_complete)
		// Clear the reset on completion
		bw_reset_request <= 1'b0;

This implements the “clear request” logic of Fig. 3, above and to the right.

Next, if a reset has been requested that hasn’t yet completed (logic above), we keep the request going until it does complete. This would be the case in Fig. 3 between the reset request and the reset acknowledgment.

	else if (bw_reset_request)
		// While the reset is ongoing, keep it going until its
		// cleared
		bw_reset_request <= 1'b1;

Finally, the core is responsive to a reset request from the bus–both from the bus reset wire !S_AXI_ARESETN (called i_reset below) as well as from a particular write to the control register. Specifically, if there’s ever a write to the control register of this scope, and the reset bit within that write is low, then we begin a local reset cycle.

	else if (i_reset || (write_to_control && !i_bus_data[RESET_BIT]))
		// Initiate a new reset request
		//    Note that we won't initiate a new reset request
		//    while one is already pending.  Once the pending
		//    one completes, we'll be in the reset state anyway
		bw_reset_request <= 1'b1;
end

Why check for if the reset bit of the control word is low rather than high? It just makes working with the core easier. Normally, writes to the control word contain how long you want to wait from trigger to capture. This requires resetting the core, so the reset request will naturally get set on any such write–even without raising a bit to explicitly request it.

This reset logic then crosses clock domains via a handshaking protocol, shown in Fig. 3 above–one we’ve discussed the basics of before. It’s not fast, but it works. All of the capture logic is then reset from a reset signal in the data clock domain, dw_reset–that’s my equivalent of the LOCAL_RESET signal we’ve been discussing above.

Once complete, the return signal, bw_reset_complete, is set and the logic above clears. Until the reset completes, any read from the control register will return with the reset bit set–indicating that the reset is ongoing. This is useful for those cases where the data clock isn’t running for some reason. In those cases, if you come back and read from the scope, using either the CPU or the debugging bus), you’ll quickly notice that the scope is still in reset–telling you exactly what’s going on.

What about the rest of the bus logic?

It doesn’t change.

The bus still operates like it normally would, unless or until you reset the entire bus, which is just the way we wanted this core to work. The approach is fairly easy to design and easy to implement, and (so far) it’s worked quite well for me.

That handles AXI slave components, but how would you locally reset an AXI master?

Locally resetting an AXI Master component

Fig 4. Steps to resetting an AXI master

Resetting an AXI master is a bit more challenging. Basically, the master has to wait for all outstanding transactions to complete in order to complete a reset. This requires something of a state machine.

Let’s walk through the steps of how this might work.

1. First, a reset is requested, the design then transitions from any WORKING state to an INRESET state in order to wait for things to settle out.

2. While in the INRESET state, no additional AXI transactions are initiated. Existing transactions are allowed to complete. This step itself requires a couple of sub-steps.

   1. BREADY and RREADY need to be set to one if they aren’t already

      I personally like to keep BREADY and RREADY high any time a transaction is outstanding, and even if no transactions are outstanding. I consider this good practice, although I can imagine reasons why a design might not be able to do this.

   2. As soon as !AxVALID || AxREADY is true on any of the address request channels, the AxVALID line should to be dropped. That way, no new transaction requests are issued.

      This assumes that you don’t set WVALID high prior to its associated AWVALID signal. While the specification allows you to do this, it doesn’t typically buy you any better performance since the interconnect won’t know what to do with the data apart from the address anyway. Still, if you have chosen to allow WVALID to be set prior to AWVALID, then you’ll need to make sure to flush the AWVALID lines as well, just like we’re about to flush the WVALID signals below.

   3. As soon as !WVALID || WREADY, you will want to set S_AXI_WSTRB to zero. This will prevent your core from actually writing any more values following the reset request–assuming this was what you wanted.

      You’ll also want to set WVALID to 1 as long as you have write transactions outstanding that need to be completed.

   4. The write channel signaling is a bit trickier. You will need to complete any ongoing write operations. My recommendation would be to set WVALID to 1 as long as you have an outstanding transaction to complete. This works since you’re setting WSTRB to zero at the same time, and so you don’t really care any more about what data you are sending, just that the data transactions clear through the system.

   Together, these steps will keep you from starting any new transactions.

3. While in the INRESET state, any and all existing transactions must be allowed to complete.

   You were keeping track of how many bursts requests you’ve initiated and how many expected responses are still outstanding, right?

   As long as transactions remain outstanding, you’ll need to stay in this INRESET state.

4. Once all transactions have completed, your core may then leave the reset state and return to IDLE (or WORKING–depending upon your core’s design requirements).

Shall we take a look at an example AXI master to see how this might be done?

AXI Master Example

All three of my AXI DMA algorithms have some capability for aborting transactions. This includes my AXI stream to memory core, my AXI memory to stream core, and my AXI DMA core. Unlike the major vendor cores, these three are all open source. Each contains an AXI-Lite control port, and a full memory mapped AXI4 master port.

Fig 5. AXI Stream to Memory core components

Since each core contains the same basic reset logic within them, let’s just look at the S2MM core as an arbitrary example. As illustrated in Fig. 5, this core is responsible for accepting stream data, possibly synchronizing to the nearest packet start, and then writing this stream data to memory at a user defined location.

The S2MM core core has an AXI-lite interface for control. If the core is busy, an abort command can be given to it through this interface which will cause a form of reset. Such an abort command will be issued if ever the ABORT_KEY is written to the upper byte of the control register while the core is in operation.

always @(*)
begin
	w_cmd_abort = 0;
	w_cmd_abort = (axil_write_ready && awskd_addr == CMD_CONTROL)
		&& (wskd_strb[3] && wskd_data[31:24] == ABORT_KEY);
	if (!r_busy)
		w_cmd_abort = 0;
end

This user-abort signal is then registered, and held high until the user-commanded, local “reset” is complete and the design returns to idle (i.e. !r_busy).

initial	cmd_abort = 0;
always @(posedge i_clk)
if (i_reset)
	cmd_abort <= 0;
else if (!r_busy)
	cmd_abort <= 0;
else
	cmd_abort <= cmd_abort || w_cmd_abort;

There’s also a similar signal, axi_abort_pending. This is set not only based upon the user command, but it also gets set if the core ever encounters any bus errors.

initial	axi_abort_pending = 0;
always @(posedge i_clk)
if (i_reset || !r_busy)
	axi_abort_pending <= 0;
else begin
	// Abort on any bus error
	if (M_AXI_BVALID && M_AXI_BREADY && M_AXI_BRESP[1])
		axi_abort_pending <= 1;
	if (cmd_abort)
		axi_abort_pending <= 1;
end

Once either of these signals are set, either cmd_abort or axi_abort_pending, the core begins a local reset sequence that will return it to its idle state. This includes aborting all current and ongoing transactions.

First, the core needs to set BREADY to one. This particular core follows my default philosophy of keeping BREADY high in the first place,

	assign	BREADY = 1;

so no additional action is required there.

Next, AWVALID needs to be dropped. In this particular core, a combinatorial register is used to determine when to set AWVALID in the first place. This register is called w_phantom_start. w_phantom_start is always cleared in the case of any pending abort.

always @(*)
begin
	// We start again if there's more information to transfer
	w_phantom_start = !aw_none_remaining;

	// But not if the amount of information we need isn't (yet)
	// in the FIFO.
	if (next_fill < {{(LGFIFO-LGMAXBURST){1'b0}}, r_max_burst})
		w_phantom_start = 0;

	// Insist on a minimum of one clock between burst starts,
	// since our burst length calculation takes a clock to do
	if (phantom_start)
		w_phantom_start = 0;

	// If we're still writing the last burst, then don't start
	// any new ones
	if (M_AXI_WVALID && (!M_AXI_WLAST || !M_AXI_WREADY))
		w_phantom_start = 0;

	// Finally, don't start any new bursts if we aren't already
	// busy transmitting, or if we are in the process of aborting
	// our transfer
	if (!r_busy || cmd_abort || axi_abort_pending)
		w_phantom_start = 0;
end

This signal guarantees that AWVALID is returned to zero as soon as possible following any abort request. Notice, in the actual logic block that sets AWVALID below, that this signal is only ever changed if !M_AXI_AWVALID || M_AXI_AWREADY. This is key to following the protocol and you’ll see this pattern often.

initial	axi_awvalid = 0;
always @(posedge i_clk)
if (i_reset)
	axi_awvalid <= 0;
else if (!M_AXI_AWVALID || M_AXI_AWREADY)
	axi_awvalid <= w_phantom_start;

Clearing WVALID on a reset takes a bit more work. Or rather, the logic for WVALID doesn’t really change. It still gets set at the beginning of every burst, and remains set until the burst is complete.

initial	axi_wvalid = 0;
always @(posedge i_clk)
if (i_reset)
	axi_wvalid <= 0;
else if (!M_AXI_WVALID || M_AXI_WREADY)
begin
	if (M_AXI_WVALID && !M_AXI_WLAST)
		axi_wvalid <= 1;
	else
		axi_wvalid <= w_phantom_start;
end

Admittedly, this arrangement is somewhat specific to how this stream to memory core works. In particular, this core only ever sets WVALID if it has enough information to hold it high until any burst that it is a part of is complete. This is part of my philosophy of either using the bus or getting off the bus so someone else can use it. While AXI doesn’t require this philosophy, this design approach helps to minimize any required bus resources, such as those used by the interconnect, and so it helps keep everything else working without getting slowed down by this core unless this core actually needs the bus to talk to a peripheral.

The WSTRB signal similarly straightforward: anytime the bus isn’t stalled, WSTRB is adjusted. If there’s no abort pending, WSTRB is set to all ones. (As currently written, the core can only transfer aligned words of full size. This may change in the future as I have opportunity.) Otherwise, we stop writing on either user abort or upon receiving a bus error as soon as possible by setting WSTRB to zero. As before, the !M_AXI_WVALID || M_AXI_WREADY condition is key here to following protocol.

always @(posedge i_clk)
if (!M_AXI_WVALID || M_AXI_WREADY)
	axi_wstrb <= (axi_abort_pending) ? 0:-1;

The last step is to know when we are done so we can know when to leave this reset state. This means that we need to know when there are no more requests remaining to be made, and when the last outstanding request has been returned. As we’ll see below, the return to idle logic has to be a part of the normal return to idle logic–the task just returns to idle a bit earlier.

Fig 6. The S2MM operation is like a double-hourglass

Getting there requires checking two counters. You can think of this sort of like the double hourglass shown in Fig. 6 on the left. Initially, the first counter is loaded by the user with the number of requests that need to be made. This is the aw_requests_remaining counter. Then, as write address burst requests get accepted by the bus, the counter decrements and a second counter, aw_bursts_outstanding, gets incremented. (The units of the two counters aren’t quite the same–the first counter counts beats, the second bursts–but we can ignore that for now.) Finally, when a burst is complete, the aw_bursts_outstanding counter gets decremented. Once both counters reach zero, the operation is complete.

The way the local reset, a.k.a. abort, works is that it empties out the first chamber of requests still to be issued, but then it lets the rest of the operations proceed to completion.

Let’s look at this in more detail.

The first counter keeps track of the number of remaining bus word transactions. When the core, isn’t busy, this number is set from the AXI-Lite interface registers,

initial	aw_none_remaining = 1;
initial	aw_requests_remaining = 0;
always @(posedge i_clk)
if (!r_busy)
begin
	aw_requests_remaining <= cmd_length_w;
	aw_none_remaining     <= zero_length;

It gets cleared on any local reset request, whether on a user abort or upon receiving a returned bus error.

end else if (cmd_abort || axi_abort_pending)
begin
	aw_requests_remaining <= 0;
	aw_none_remaining <= 1;

Finally, on the first beat of any AWVALID, possibly even before AWREADY, we adjust the number of words remaining to be written. That’s the meaning of phantom_start below–it’s true on the first clock cycle of any new burst write request. Unlike AWVALID, phantom_start is only ever high for the first cycle. It’s like what the AWVALID signal would be if AWREADY were always true.

end else if (phantom_start)
begin
	aw_requests_remaining
		<= aw_requests_remaining - (M_AXI_AWLEN + 1);
	aw_none_remaining <= (aw_requests_remaining == (M_AXI_AWLEN+1));
end

You might also wish to notice the other flag associated with our counter: aw_none_remaining. This flag is equivalent to aw_requests_remaining == 0, and it is used to simplify the logic following so I don’t have to check every bit of the counter in order to know if it’s zero.

We still need one other counter before we can know if we’ve completed our bus transactions, and that’s the counter that keeps track of the number of outstanding burst transactions. This particular core maintains both a counter and a flag for this purpose–just like we did above. The counter keeps track of the number of outstanding bursts, whereas the flag keeps track of whether only a single burst is outstanding or not.

initial	aw_last_outstanding   = 0;
initial	aw_bursts_outstanding = 0;
always @(posedge i_clk)
if (i_reset)
begin
	aw_bursts_outstanding <= 0;	// The counter
	aw_last_outstanding <= 0;	// The flag
end else case ({ phantom_start,
			M_AXI_BVALID && M_AXI_BREADY })
2'b01:	begin
	aw_bursts_outstanding <= aw_bursts_outstanding - 1;
	aw_last_outstanding <= (aw_bursts_outstanding == 2);
	end
2'b10:	begin
	aw_bursts_outstanding <= aw_bursts_outstanding + 1;
	aw_last_outstanding <= (aw_bursts_outstanding == 0);
	end
default: begin end
endcase

Now that these two counters and their associated flags have been defined, we finally have enough information to determine if all of our outstanding bus transactions have completed or not.

always @(*)
if (!r_busy)
	w_complete = 0;
else
	// We are complete if nothing more is to be requested,
	w_complete = !M_AXI_AWVALID && (aw_none_remaining)
		// *and* the last burst response has now been returned
		&&(aw_last_outstanding) && (M_AXI_BVALID);

This w_complete flag is then used to send our state machine back to idle, by clearing the r_busy flag. Once cleared, the core has finished any user commanded reset, and can return to its idle state.

initial	r_busy     = 0;
initial	r_complete = 0;
always @(posedge i_clk)
if (i_reset)
begin
	r_busy     <= 0;
	r_complete <= 0;
end else if (!r_busy)
begin
	// Core is idle, waiting for a command to start
	if (w_cmd_start)
		r_busy <= 1'b1;

	// Any write to the control register will clear the
	// completion flag
	if (axil_write_ready && awskd_addr == CMD_CONTROL)
		r_complete <= 1'b0;
end else if (w_complete)
begin
	// Clear busy once the transaction is complete
	//  This includes clearing busy on any error
	r_complete <= 1;
	r_busy <= 1'b0;
end

The key point here is that even though there are several steps required to reset an AXI master, those steps actually fit in nicely with the logic of building the AXI master in the first place.

A second key point is simply this: plan ahead for any local reset functionality you may wish to build into your own design.

Conclusion

Knowing how to reset a single user core in the context of a larger bus is an important part of building any bus component. The task isn’t nearly as simple as it might sound, simply because a local reset shouldn’t require a change in behavior from any of the other cores attached to the same system bus. Further, if you don’t do it right, you risk locking up the entire system–something that is generally considered quite bad.

One specific worry includes usage counters found within any bus interconnects. Such counters are required to keep track of the number of outstanding transactions, so that the interconnect can know when to adjust any channel assignments without risking returns getting sent to the wrong bus masters.

The good news is that by following the steps outlined above, you can rest assured that your core can still be reset separate from the rest of the design–even in the presence of any ongoing bus activity.