The ZipCPU's Interrupt Controller
For those unfamiliar with the ZipCPU, it was designed from the ground up to be a simple, low logic CPU. At every location and opportunity, I have tried to cut logic from the design to make it simpler. One of the places where I tried to keep the design simple was the interrupt controller.
Traditionally, when a CPU receives an interrupt, the hardware CPU logic then looks up an “interrupt vector” from a table located at a known location in memory. The index to the table is given by the interrupt number. This interrupt vector will contain the address of a software routine known as an interrupt handler. Once the CPU reads this value from memory, it will then jump to that address in order to execute what is known as an “Interrupt Service Routine” or ISR.
In a pseudo-assembly, these actions of the CPU might look like:
Of course, these instructions would never appear in any CPU software. Rather, they are implied by and executed within the CPU itself.
Depending upon the CPU, other actions may take place as well. The current interrupt may be disabled, and some architectures will even disable all interrupts. The old program counter is usually stored somewhere (as shown in pseudocode above), and sometimes there’s even a flag register that needs to be stored as well (also shown in pseudocode above). Modern multitasking computers may also store the stack register register itself, and so on. Where these registers get stored is CPU and Operating System (O/S) dependent. Older CPUs used to store these values in memory on the stack. Many newer CPUs just place these values into special purpose registers.
The size of the vector table itself is also CPU dependent. Some architectures support 16 interrupts. Some support more. Each interrupt gets its own interrupt handler address in the table, and as a result the CPU can have many interrupt service routines (ISRs). This allows the CPU to jump to a routine specific to the type of interrupt taking place. Should the keyboard interrupt trigger, for example, the CPU could transition straight to the software necessary for handling keyboard input. Other possible interrupt sources include the display, mouse, DMA, timer, hard drive, SD-card, and many more.
One fascinating fact of the Linux O/S is that it tends to set every interrupt address to the same interrupt service routine–regardless of the peripheral that tripped it.
The ZipCPU uses a different approach, somewhat inspired by the Linux approach of having only a single interrupt service routine.
The ZipCPU doesn’t look up an interrupt address from memory. Instead, it just clears the internal pipeline and then switches from the user to the supervisor register sets. It also has only one interrupt input.
From the software standpoint, the
ZipCPU doesn’t call the
interrupt handler directly.
That’s not the entry point to the interrupt
handler. Instead, it just
looks like the return of a special zip_rtu()
function call made from
supervisor mode.
As a quick background, if you will recall the
ZipCPU starts
in supervisor mode upon
any reset. When it then hits an RTU (return to userspace)
instruction, i.e. when the software calls the zip_rtu()
function, the
ZipCPU
switches from supervisor to user mode. It will then run instructions from the
user register set until some time later following either a user trap
(system call request), an exception (something went wrong, a
bus
error for example), or an
interrupt. At this point in time,
the ZipCPU switches to the supervisor
register set, and continues executing instructions from there.
This often leads to a ZipCPU supervisor program that looks something like:
The problem with this approach is that the CPU needs to know, once it switches to supervisor mode, which device triggered the interrupt. This is the purpose of the ZipCPU’s interrupt controller. The interrupt controller also handles enabling particular interrupts–something we’ll get to in a moment.
The Concept
The basic concept of an interrupt
controller
is that it monitors several interrupt
lines or sources, and triggers an
interrupt when any of the lines is
high. If we consider a generic
interrupt numbered k
, we might
write:
Of course, this isn’t quite sufficient. We also need a way to clear this interrupt state once set. We also need to insure that it starts in an idle state, and only triggers after the interrupt has taken place, as shown in Fig. 4 on the right.
So, let’s try this again. We’ll initialize our interrupt’s state to zero.
Likewise, on an interrupt, we’ll set the interrupt state.
Finally, if the user writes to our interrupt register, we’ll clear the interrupt state. To be specific to one interrupt, we’ll clear the interrupt state any time the user writes a ‘1’ to the interrupt state associated with this interrupt in the interrupt state register, and ignore any write otherwise.
Now, if we aggregate all of these interrupts together, we can create a single interrupt input for the CPU.
Except … what if we don’t want to be interrupted? Or, alternatively, what if we don’t want to be interrupted by a particular interrupt?
Therefore, let’s create what’s known as an interrupt mask, so that only interrupts found identified by the mask will interrupt the CPU.
There are actually two parts to the ZipCPUs interrupt mask logic within its interrupt controller. First, an individual interrupt mask selects particular interrupts to be enabled or disabled. Second, there’s a master interrupt enable wire that needs to be set in order for any interrupt wire to create an interrupt within that controller.
The code for this “interrupt enable” register might now look something like:
We can also apply this same logic to the master interrupt enable or MIE register within the controller. We’ll just use the MSB to control this value.
When put together, the logic for generating an outgoing interrupt will now look something like,
As a result, an outgoing interrupt will be created any time one of several interrupt source wires goes high, provided that both that individual enable and the MIE bit is set.
The controller logic I’m presenting above is actually from a second generation ZipCPU interrupt controller–one that hasn’t (yet) propagated into all of my projects yet. The first generation controller had a big problem with the lack of an ability to atomically set or clear particular interrupt enable lines. In this first controller, writing a word with the high order bit set high, that is with the MIE bit set, would enable all interrupts as well as those whose interrupt enable lines were set. However, if you were to wrote a word to the controller with the high order bit clear, you could then disable any interrupt masks but you would also disable the MIE bit! These two operations, in the original design, could not be decoupled. As a result of that earlier approach, disabling any interrupt source within the controller also disabled the master interrupt enable bit. If you only wanted to disable a single interrupt’s enable bit, you’d need to first disable all interrupts as well as the single interrupt you wanted to disable, and then you’d need to re-enable all interrupts in a second transaction.
The newer, and updated logic above, avoids that problem.
Let’s see how this works, shall we?
The Code
The biggest difference between the outlined logic above, and the actual
controller as implemented,
is that the currently implemented
controller
uses word rather than bit operators. Hence, for IUSED
possible
interrupts
there’s a word of IUSED
bits in length that contains all of the bits in the
current
interrupt
state,
and another one that contains a list of all of the interrupt sources that are currently enabled.
The third state register controls whether this
controller
will generate
interrupts
at all. Setting r_mie
to zero, will disable all
interrupt
generation from this
controller.
Reads to this core will just read our status register, but writes can have a couple of side effects. In particular, we’ll want to support a basic write to the peripheral, as well as writes that either enable or disable interrupts.
Now comes the fun parts!
First, how shall we determine which
interrupts.
are active? Remember, an
interrupt
is active independent of whether or not it is enabled. An
interrupt
becomes “active” when the incoming
interrupt
line is high, and it gets deactivated when a 1
is written to its
interrupt
state line. We’ll keep the
interrupt
state in our bottom 15 bits, giving us something like:
The next important piece of logic controls our interrupt enable lines. These
will be controlled by bits i_wb_data[16 +: IUSED]
. If you’ve never used
this A[X +: W]
notation before, it’s equivalent to A[(X+W)-1:X]
. I
often find it much easier to read–especially since, like an old codger,
I try to make certain all of my code fits within an 80 column screen,
using 8-character tabs.
We’ll start out the interrupt enable logic by disabling all of the enable bits on any synchronous reset.
Otherwise, if the user writes a word to our controller
with bit i_wb_data[15]
held high, then we’ll enable any
interrupts
whose enable bits are also set during the write.
The same is true for disabling
interrupts.
Writing to the
interrupt controller
with bit i_wb_data[15]
clear, will be used to clear any enable lines that
are set during the write.
The logic for the master enable bit for this controller is almost identical. The big difference is that we aren’t using word logic but rather bit logic to set it.
Notice the important part of this control design: the r_mie
bit is only
adjusted when i_wb_data[DW-1]
is also high. This allows other bits to be
adjusted without adjusting this one–fixing the problem from the
last controller.
Ok, that handles most of our interrupt state and enable controls.
How can we now go about using them to generate our outgoing interrupt?
First, we’ll set a helper bit, w_any
, to capture whether any enabled bits
are also active.
Now, if w_any
is set, and if the master
interrupt
enable is also set, then we can send an
interrupt
to the CPU.
Notice how o_interrupt
is registered, or set on a clock edge. That’s
important, since a CPU
may have a lot of logic riding on this incoming signal. By registering it
here and now, we avoid compounding any internal timing problems the
CPU
might already have. Indeed, ZipCPU
used to have a lot more
logic
dependent upon this one signal than it has
now.
All that’s left, then, is to set our bus outputs.
The first one is o_wb_data
. In this case, we want to set bit 31
to
reflect whether or not the master interrupt enable is set,
and bit 15
to reflect whether any
interrupt
would be currently triggering.
After that, the register is split into two halves. The top half indicates which interrupts are enabled,
and the bottom half indicates which interrupts are currently active.
Only there’s a tricky problem here: if we don’t set all of the bits, such
as if IUSED
is something less than the full width, then the synthesis
tools will complain at us about undefined bits–and rightly so.
I used to get around this by setting part of the bits in the basic always
block, and using a generate block to set the rest. Verilog, however, has
this wonderful feature where the “last assignment wins.” Therefore, we can
start by assigning all of the bits within o_wb_data
to zero,
after which we can set the specific bits of interest as before.
As a result, any bit that isn’t set specifically, will be set by default to zero. Even better, the synthesizer can remove the logic surrounding any of these bits which might get stuck at zero.
This particular peripheral is also unique because it has only one address,
and hence there’s no logic that needs to take place on a read other than just
returning o_wb_data
. Hence, we can set o_wb_ack
to be equal to
i_wb_stb
, and set our stall line to zero as well.
Voila! A functioning interrupt controller!
Formally Verifying the Interrupt Controller
What about formally verifying something this simple?
Let’s see if we can’t turn the formal verification section into a set of rules that we will check, and then check at each point that our controller works as intended.
The first step, though, is a bit preliminary. First, we want to create our
f_past_valid
logic, so we can tell when any $past()
values are properly
constrained and usable within any assertions.
Then we want to make certain our design starts with an initial reset. This is really needed more for the Wishbone bus than anything else, but it often makes good practice anyway.
Here are the values we want to make certain are responsive to a reset. Notice that we also force them to be properly responsive to the first or initial clock as well.
Ok, let’s start walking through what this controller needs to do.
- Rule #1: Any incoming
interrupt
should be able to set the
r_int_state
bits
The word level logic here might be a touch confusing. Basically,
we’re asserting that if a bit was high in i_brd_ints
on the last
clock, that r_int_state
in the next clock should reflect that.
- Rule #2: An interrupt should be generated if received and enabled
In this case, r_int_state
captures an
interrupt
being received, and r_int_enable
captures one being enabled. If this is
true one clock ago, and if r_mie
was high, then o_interrupt
should now
be high as well.
This is really the first part of the inverse of Rule #2 above, expressing when the interrupt line should be clear.
Rule #4 is the other half of rule #3.
- Rule #4: If no active interrupts are enabled, then no outgoing interrupt shall be asserted either
Again, we are asserting when the outgoing interrupt line should be clear (zero). In this case, if there are no active and enabled interrupts, then neither should the outgoing interrupt line be high.
From here, let’s switch from how the interrupt controller should deal with interrupts, and look at how it should deal with transactions from the bus.
- Rule #5: It should be possible to disable one (or all) interrupts as a result of writing a word from the bus
To put this another way, if it was set one clock ago within
$past({i_wb_data[31],i_wb_data[16 +: IUSED]})
, then the enables on the
current clock, { r_mie, r_int_enable }
should be zero.
- Rule #6: It should be possible to enable one (or more) interrupts
- Rule #7: It should be possible to acknowledge an interrupt, and so deactivate it
- Rule #8: The interrupt enables should be stable without a write
The next big portion of our proof is to include our bus property file, to verify that, yes, we are properly responding to bus requests.
The tricky part here is that this slave doesn’t have an i_wb_cyc
line coming
into it. According to the Wishbone
spec, it doesn’t need one. It can (and
does) get by with just the i_wb_stb
signal. Our
interface properties,
however, require an i_wb_cyc
line. Therefore, let’s imagine we had an
i_wb_cyc
line, and that our imagined i_wb_cyc
followed the rules of the
bus.
Now we can include our bus properties.
You may note that we set the width of the address bus to .AW(1)
. It’s not.
It’s really zero. I have yet, however, to learn how to remove bits
automatically based upon a parameter, so I instead set this to one and then set
the address bit(s) themselves to 1'b0
.
Since o_wb_ack == i_wb_stb
, we’ll never have any outstanding requests, so
asserting the proper value for the number of outstanding (not responded to)
requests becomes easy.
I then threw some odds and ends into the proof, just for good measure.
For example, if there hasn’t been either a write or reset request, then any previously active interrupt should remain active.
What’s missing? A big and important part of any proof: the proof that, in
spite of all of our constraints, our logic can still work. So, let’s add some
cover()
statements.
The first and most obvious item to cover is, yes, it is possible to produce an outgoing interrupt.
A good second example would be that, given a prior active interrupt, it can be cleared by a write from the bus.
Let’s also make certain our enables work, by covering a low o_interrupt
,
but yet a high w_any
or similarly a low o_interrupt
, an active
interrupt
line, and a high r_mie
.
This should be (roughly) enough to convince ourselves that this interrupt controller is working.
But how to use it? Well, the first step towards using an interrupt controller is to wire it up. To do that, let’s invoke AutoFPGA to make our connections for us.
AutoFPGA
The first step to integrating this controller into a design using AutoFPGA is to create an AutoFPGA configuration file for this controller. The second step will be using AutoFPGA to connect various interrupt sources to this controller. So let’s start out by walking through an AutoFPGA configuration file for this interrupt controller, and then walk through how to connect the interrupt from the ZipTimer peripheral that we discussed previously to our new interrupt controller. If you’d like, you can follow along as I work my way through the buspic interrupt controller configuration file in my ArrowZip design, and again through the bus timer configuration to see how some simple AutoFPGA tags can be used to merge this into a design.
As we get into this, let me remind you of two things about AutoFPGA: First, AutoFPGA is designed to be able to compose a design from multiple sources via a copy-and-paste approach, with some amount of variable substitution. Second, AutoFPGA can also attach bus masters and slaves together, assign addresses, and for our purposes today, it can assign interrupts to a bit-vector that will then be the input to an interrupt controller like the one we’ve discussed above.
We’ll start out by examining how to integrate this interrupt controller into the ArrowZip project. In many of my projects, the interrupt controller is a part of the wrapper of the ZipCPU that I call the ZipSystem. Not so in the ArrowZip design. Sorry. There wasn’t room in the ArrowZip design for the ZipSystem. Instead, in the ArrowZip design the interrupt controller is a proper peripheral on the main wishbone bus along with every other peripheral.
As a result, we’ll be primarily just integrating a simple peripheral into the wishbone bus structure using AutoFPGA.
The first step, as always, is to pick a name for our peripheral. I’m going to
use the name buspic
, since this Programmable Interrupt Controller (PIC) is
attached to the
bus instead of the
ZipSystem.
I’m also going to create the tag @DEVID
for later substitution
and set it to the same name, but this time in all capitals.
In case anything else depends upon the presence of the BUSPIC within the
design, we’ll also create an @ACCESS
tag. This will create a define
macro in the top of the
main.v
design element, that can then later be
commented out (if you’d like). Other Verilog components within
main.v
that might depend upon the presence of this one can then use this macro
to determine whether this peripheral is present in the design or not.
Even better, a quick script within the
Makefile
of our
rtl directory
converts these defines into a design.h
file as well, which
can then be used within software components to determine what components
are and are not built into the design.
Let’s now create a
bus
connection for this component. The first step is to
tell AutoFPGA
how many addresses this slave has. In our case, it’s just the
one, so @NADDR=1
. We then want to connect to the
bus
I’ve named wb
. Further, since our output register is available to be
read at all times, and since this peripheral never stalls, we can tell
AutoFPGA to
connect it up as a peripheral of type SINGLE
, as in, @SLAVE.TYPE=SINGLE
.
That will create return wires for this peripheral to return to the
bus,
using the @PREFIX
tag to determine their names. It will also create another
value to determine when a bus
operation is selecting this device, buspic_sel
. In total, this will looks
something like,
We’ll use these various connections in a moment to actually connect our device to the bus.
Before we do that, though, let’s tell
AutoFPGA that this
particular device is a programmable interrupt controller (PIC). Specifically,
AutoFPGA
accepts a number of tags associated with any PIC. These are first the name of a
multi-element vector, in our case bus_int_vector
, and second the number of
elements in that vector–which for us is 15 elements.
Now, when any peripheral wants to route its interrupt wire to the PIC named
buspic
, it will be assigned a bit within this bus_int_vector
of bits.
We’ll get back to that in a bit.
Our interrupt
controller
needs an output wire, to be fed to the
ZipCPU. Let’s call that
bus_interrupt
, and declare it at the top of our
main.v
file.
@MAIN.DEFNS
describes one of those tags that’s just copied directly
into a given location within an output file–in this case
main.v.
Another straight copy tag is @MAIN.INSERT
. The value for this tag will
be copied directly into
main.v
after variable substitution.
In this case, the primary variable that we wish to substitute in, as we copy
this longer string into our
main.v
file, is the @$(PREFIX)
tag. As you may
recall, AutoFPGA
tags may be referenced by either a @$(*)
or a @$*
. Hence, we could
reference @$(PREFIX)
or @$PREFIX
equally. Both will get substituted with
our prefix, buspic
. I personally like the @$(*)
form better, because in
this case the underscore following the tag will not be confused with the
identifier for the tag itself.
We can also specify some alternate logic, for if and when the define
macro
at the top of
main.v
is commented out.
In that case, the @MAIN.ALT
code will replace the @MAIN.INSERT
code.
Further, in this alternate “the buspic is not included” case, AutoFPGA
will provide default values for the bus returns: buspic_stall = 0
,
buspic_data = 0
, and so on, so you can ignore them here.
If you’ve followed this blog much, you know that I like to have command line access to the peripherals within my designs using a command I call wbregs. This allows me access to memory mapped peripherals in roughly the same was as the PEEK & POKE of yesteryear.
Unlike PEEK & POKE,
though, wbregs
allows named values.
In a similar manner, if you want access to this peripheral from the C++
bus interface, you’ll need to know the address of this peripheral. This
is the purpose of the @REGS.*
tags. The first one, @REGS.N=1
, tells
AutoFPGA that this
peripheral has only one named register. The second line tells
AutoFPGA
that this register has a zero word offset from the base address
of this peripheral, that the C++ name for this register should be R_PIC
,
and that wbregs
should accept a name PIC
in order to access this register.
These two lines affect the regdefs.h
and regdefs.cpp
files generated
by AutoFPGA.
We’d also like to reference this interrupt controller from within a program for the ZipCPU.
For example, given an interrupt number, such as X
, it would be nice to know
how it maps to the various bit masks within our peripheral. We can also
throw in macros to enable and disable interrupts using this
controller’s
if we would like.
These macros will get copied to the top part of the board.h
file, used to
describe peripherals for any CPU within the design. The first one of these,
BUSPIC(X)
, will also get called by the
AutoFPGA
generated code to define values to describe each interrupt assignment.
We’d also like to know, in that same file, the location of this peripheral in address space. We discussed some time ago how to access memory mapped peripherals in general, so here’s how it is done with AutoFPGA.
First, we define the name that we want to reference this peripheral as from within our ZipCPU C code.
the type this value will point to,
and a macro that can be used to determine if the design (originally) had this peripheral within it,
Finally, we can put these all together to define an constant address
pointer referencing this peripheral. This final line, after variable
substitution, will be copied directly into the board.h
file.
AutoFPGA will then place a line into the board.h
file looking something like:
Our interrupt controller’s configuration file contains one more group of tags. These are used to place information into a make include file, which can then be used to determine which files to send to yosys (or any other synthesizer) and how to determine dependencies.
For our controller,
we create a group of files within the
cpu
subdirectory named BUSPIC
. This file group contains the icontrol.v
file
only in our case, since this interrupt
controller
implementation doesn’t have any submodules.
At this point, our interrupt controller has now been connected to the rest of the design. The one remaining detail is that the rest of the design hasn’t been connected to this interrupt controller. So let’s take a look at how to connect the ZipTimer’s interrupt output to this controller.
The ArrowZip design contains a ZipTimer AutoFPGA configuration within the ZipCPU’s configuration file, zipbones.txt. Two particular lines within this configuration need to be discussed.
First, the @INT.BUSTIMER.WIRE
line defines a wire that will contain this
interrupt signal.
This alone will define the wire, bustimer_int
, pasting a line,
into the main.v file. That only defines it.
To connect it up to a particular
interrupt controller,
just name the @PREFIX
of the
controller
you want to connect to. In
our case, that’s
the buspic
we just defined above.
AutoFPGA will now pick a bit within bus_int_vector[14:0]
, and assign
bustimer_int
to that vector, giving it the name BUSTIMER
found within the
key name, @INT.BUSTIMER.WIRE
. For example, if you look through the
main.v
AutoFPGA
generated file within the
ArrowZip repository, you’ll find
the lines,
From here you can see how the seven interrupts used within this design are assigned.
But how would you access these definitions within some C code, to know which
interrupts got assigned where? For that, check out the
AutoFPGA
generated board.h
file. Towards the bottom, you’ll find these
interrupt
assignment statements.
These lines are created by
AutoFPGA
based upon the
interrupts
connected to the peripheral having the @PREFIX
of buspic
, and the
@PIC.WIRE
key.
Looking over this code, you should now understand why we defined the
BUSPIC(X)
macro earlier. We can now reference these interrupt ID’s
by these macros. In the case of our
timer,
we can now reference the BUSPIC_BUSTIMER
interrupt
within our code.
Alternatively,
AutoFPGA
has defined a value for the @INT.BUSTIMER.ID
key that you can
use as well, to place this identifier as necessary wherever you need to within
your design.
That’s all that’s necessary to connect a SLAVE.TYPE=SINGLE
peripheral to the
bus, and then to fill out an
interrupt
(or other) vector register with values
composed from several other peripherals within the design. Yes, this
capability is currently very flexible–I’ve even used it to compose system
reset wires from multiple independent sources. It’s just that useful.
What if you didn’t want to include this buspic
in your design? You could
simply remove it from the
AutoFPGA
command line. Sure, you’d then get some errors about
interrupts
that didn’t have corresponding interrupt
controllers,
but that’s what you’d want, right?
Conclusion
At this point, we’ve now created (or rather updated) an interrupt controller, we’ve then formally verified it and connected it into our design. The peripheral now has an address on our bus, and the bus logic knows to check the results of this peripheral for responses. Similarly, we’ve also seen how to connect various interrupt lines created from among our several peripherals to this controller using AutoFPGA. All that remains is to demonstrate this controller within a design.
At this point, though, I’m going to stop and break this article into pieces. The next step in using interrupts with the ZipCPU isn’t really a Verilog or FPGA design problem. Using interrupts is more of an advanced software lesson, so it will have somewhat of a different flavor to it.
My hope is to use this controller to demonstrate and discuss not only how to control timing delays within an embedded CPU, but also to demonstrate how to run multiple tasks within the ZipCPU as well.
All of these will be if and as the Lord wills.
While he was yet speaking, there came also another, and said, The fire of God is fallen from heaven, and hath burned up the sheep, and the servants, and consumed them; and I only am escaped alone to tell thee. (Job 1:16)