nmutil package

Submodules

nmutil.byterev module

This work is funded through NLnet under Grant 2019-02-012

License: LGPLv3+

nmutil.byterev.byte_reverse(m, name, data, length)

byte_reverse: unlike nmigen word_select this takes a dynamic length

nmigen Signal.word_select may only take a fixed length. we need bigendian byte-reverse, half-word reverse, word and dword reverse.

nmutil.clz module

class nmutil.clz.CLZ(width)

Bases: nmigen.hdl.ir.Elaboratable

combine_pairs(m, iteration, pairs)
elaborate(platform)
generate_pairs(m)

nmutil.concurrentunit module

concurrent unit from mitch alsup augmentations to 6600 scoreboard

This work is funded through NLnet under Grant 2019-02-012

License: LGPLv3+

  • data fans in
  • data goes through a pipeline
  • results fan back out.

the output data format has to have a member “muxid”, which is used as the array index on fan-out

Associated bugreports:

class nmutil.concurrentunit.InMuxPipe(num_rows, iospecfn, maskwid=0)

Bases: nmutil.multipipe.PriorityCombMuxInPipe

class nmutil.concurrentunit.MuxOutPipe(num_rows, iospecfn, maskwid=0)

Bases: nmutil.multipipe.CombMuxOutPipe

class nmutil.concurrentunit.PipeContext(pspec)

Bases: object

eq(i)
matches(another)

Returns a list of Assert()s validating that this context matches the other context.

ports()
class nmutil.concurrentunit.ReservationStations(num_rows, maskwid=0, feedback_width=None)

Bases: nmigen.hdl.ir.Elaboratable

Reservation-Station pipeline

Input: num_rows - number of input and output Reservation Stations

Requires: the addition of an “alu” object, from which ispec and ospec are taken, and inpipe and outpipe are connected to it

  • fan-in on inputs (an array of BaseData: a,b,mid)
  • ALU pipeline
  • fan-out on outputs (an array of FPPackData: z,mid)

Fan-in and Fan-out are combinatorial.

elaborate(platform)
i_specfn()
o_specfn()
ports()
nmutil.concurrentunit.num_bits(n)

nmutil.divmod module

This work is funded through NLnet under Grant 2019-02-012

License: LGPLv3+

nmutil.divmod.trunc_divs(n, d)
nmutil.divmod.trunc_rems(n, d)

nmutil.dynamicpipe module

Meta-class that allows a dynamic runtime parameter-selectable “mixin”

This work is funded through NLnet under Grant 2019-02-012

The reasons why this technique is being deployed is because SimpleHandshake needs to be dynamically replaced at the end-users’ choice, without having to duplicate dozens of classes using multiple-inheritanc “Mix-in” techniques.

It is however extremely unusual, and has been explicitly limited to this one module. DO NOT try to use this technique elsewhere, it is extremely hard to understand (meta-class programming).

class nmutil.dynamicpipe.DynamicPipe(*args)

Bases: object

class nmutil.dynamicpipe.MaskCancellableRedir(mod, *args)

Bases: nmutil.singlepipe.MaskCancellable

class nmutil.dynamicpipe.Meta

Bases: abc.ABCMeta

mlock = <unlocked _thread.lock object>
recursing = <_thread._local object>
registry = {}
class nmutil.dynamicpipe.SimpleHandshakeRedir(mod, *args)

Bases: nmutil.singlepipe.SimpleHandshake

nmutil.extend module

This work is funded through NLnet under Grant 2019-02-012

License: LGPLv3+

nmutil.extend.exts(exts_data, width, fullwidth)
nmutil.extend.extz(exts_data, width, fullwidth)

nmutil.formaltest module

class nmutil.formaltest.FHDLTestCase(methodName='runTest')

Bases: unittest.case.TestCase

assertFormal(spec, mode='bmc', depth=1, solver='')
assertRaises(exception, msg=None)

Fail unless an exception of class expected_exception is raised by the callable when invoked with specified positional and keyword arguments. If a different type of exception is raised, it will not be caught, and the test case will be deemed to have suffered an error, exactly as for an unexpected exception.

If called with the callable and arguments omitted, will return a context object used like this:

with self.assertRaises(SomeException):
    do_something()

An optional keyword argument ‘msg’ can be provided when assertRaises is used as a context object.

The context manager keeps a reference to the exception as the ‘exception’ attribute. This allows you to inspect the exception after the assertion:

with self.assertRaises(SomeException) as cm:
    do_something()
the_exception = cm.exception
self.assertEqual(the_exception.error_code, 3)
assertRaisesRegex(exception, regex=None)

Asserts that the message in a raised exception matches a regex.

Args:

expected_exception: Exception class expected to be raised. expected_regex: Regex (re.Pattern object or string) expected

to be found in error message.

args: Function to be called and extra positional args. kwargs: Extra kwargs. msg: Optional message used in case of failure. Can only be used

when assertRaisesRegex is used as a context manager.
assertRepr(obj, repr_str)
assertWarns(category, msg=None)

Fail unless a warning of class warnClass is triggered by the callable when invoked with specified positional and keyword arguments. If a different type of warning is triggered, it will not be handled: depending on the other warning filtering rules in effect, it might be silenced, printed out, or raised as an exception.

If called with the callable and arguments omitted, will return a context object used like this:

with self.assertWarns(SomeWarning):
    do_something()

An optional keyword argument ‘msg’ can be provided when assertWarns is used as a context object.

The context manager keeps a reference to the first matching warning as the ‘warning’ attribute; similarly, the ‘filename’ and ‘lineno’ attributes give you information about the line of Python code from which the warning was triggered. This allows you to inspect the warning after the assertion:

with self.assertWarns(SomeWarning) as cm:
    do_something()
the_warning = cm.warning
self.assertEqual(the_warning.some_attribute, 147)

nmutil.gtkw module

This work is funded through NLnet under Grant 2019-02-012

License: LGPLv3+

nmutil.gtkw.write_gtkw(gtkw_name, vcd_name, gtkw_dom, gtkw_style=None, module=None, loc=None, color=None, base=None, zoom=None, marker=-1, clk_period=1e-06, time_resolution_unit='ps')

Write a GTKWave document according to the supplied style and DOM.

Parameters:
  • gtkw_name – name of the generated GTKWave document
  • vcd_name – name of the waveform file
  • gtkw_dom – DOM style description for the trace pane
  • gtkw_style – style for signals, classes and groups
  • module – default module
  • color – default trace color
  • base – default numerical base
  • loc – source code location to include as a comment
  • zoom – initial zoom level, in GTKWave format. Can also be “formal” when the file comes from a formal engine
  • marker – initial location of a marker
  • clk_period – clock period in seconds, helping to set a reasonable initial zoom level. Use together with time_resolution_unit.
  • time_resolution_unit – use “ps” or “ns”. Derived from the units of the “timescale” on the VCD file. Used with “clk_period” to set a default zoom level

gtkw_style format

Syntax: {selector: {attribute: value, ...}, ...}

“selector” can be a signal, class or group

Signal groups propagate most attributes to their children

Attribute choices:

  • module: absolute path of the current module
  • submodule: same as above, but relative
  • color: trace color
  • base: numerical base for value display
  • display: alternate text to display in the signal pane
  • comment: comment to display in the signal pane
  • bit: select a bit from a wide signal. MSB is zero, unfortunately
  • closed (for groups): this group starts closed

gtkw_dom format

Syntax: [signal, (signal, class), (group, [children]), comment, ...]

The DOM is a list of nodes.

Nodes are signals, signal groups or comments.

  • signals are strings, or tuples: (signal name, class, class, ...)
  • signal groups are tuples: (group name, class, class, ..., [nodes])
  • comments are: {'comment': 'comment string'}

In place of a class name, an inline class description can be used. (signal, {attribute: value, ...}, ...)

An anonymous group can be used to apply a style to a group of signals. ({attribute: value}, [signal, signal, ...])

nmutil.iocontrol module

IO Control API

This work is funded through NLnet under Grant 2019-02-012

License: LGPLv3+

Associated development bugs: * http://bugs.libre-riscv.org/show_bug.cgi?id=538 * http://bugs.libre-riscv.org/show_bug.cgi?id=148 * http://bugs.libre-riscv.org/show_bug.cgi?id=64 * http://bugs.libre-riscv.org/show_bug.cgi?id=57

Important: see Stage API (stageapi.py) in combination with below

Main classes: PrevControl and NextControl.

These classes manage the data and the synchronisation state to the previous and next stage, respectively. ready/valid signals are used by the Pipeline classes to tell if data may be safely passed from stage to stage.

The connection from one stage to the next is carried out with NextControl.connect_to_next. It is not necessary to have a PrevControl.connect_to_prev because it is functionally directly equivalent to prev->next->connect_to_next.

class nmutil.iocontrol.NextControl(stage_ctl=False, maskwid=0)

Bases: nmigen.hdl.ir.Elaboratable

contains the signals that go to the next stage (both in and out) * valid_o: output indicating to next stage that data is valid * ready_i: input from next stage indicating that it can accept data * data_o : an output - MUST be added by the USER of this class

connect_to_next(nxt, do_data=True, do_stop=True)

helper function to connect to the next stage data/valid/ready. data/valid is passed TO nxt, and ready comes IN from nxt. use this when connecting stage-to-stage

note: a “connect_from_prev” is completely unnecessary: it’s just nxt.connect_to_next(self)

elaborate(platform)
ports()
ready_i_test
class nmutil.iocontrol.Object

Bases: object

eq(inp)
ports()
class nmutil.iocontrol.PrevControl(i_width=1, stage_ctl=False, maskwid=0, offs=0)

Bases: nmigen.hdl.ir.Elaboratable

contains signals that come from the previous stage (both in and out) * valid_i: previous stage indicating all incoming data is valid.

may be a multi-bit signal, where all bits are required to be asserted to indicate “valid”.
  • ready_o: output to next stage indicating readiness to accept data
  • data_i : an input - MUST be added by the USER of this class
elaborate(platform)
eq(i)
ports()
ready_o

public-facing API: indicates (externally) that stage is ready

valid_i_test
class nmutil.iocontrol.RecordObject(layout=None, name=None)

Bases: nmigen.hdl.rec.Record

ports()
nmutil.iocontrol.add_prefix_to_record_signals(prefix, record)

recursively hunt through Records, modifying names to add a prefix

nmutil.latch module

This work is funded through NLnet under Grant 2019-02-012

License: LGPLv3+

class nmutil.latch.SRLatch(sync=True, llen=1, name=None)

Bases: nmigen.hdl.ir.Elaboratable

elaborate(platform)
ports()
nmutil.latch.latchregister(m, incoming, outgoing, settrue, name=None)

based on a conditon, “settrue”, incoming data will be “latched” into a register and passed out on “outgoing”.

  • if “settrue” is ASSERTED, outgoing is COMBINATORIALLY equal to incoming
  • on the same cycle that settrue is DEASSERTED, outgoing REMAINS equal (indefinitely) to the incoming value
nmutil.latch.mkname(prefix, suffix)
nmutil.latch.sr_sim(dut)
nmutil.latch.test_sr()

nmutil.mask module

This work is funded through NLnet under Grant 2019-02-012

License: LGPLv3+

class nmutil.mask.Mask(sz)

Bases: nmigen.hdl.ir.Elaboratable

elaborate(platform)
nmutil.mask.masked(m_out, m_in, mask)

nmutil.multipipe module

Combinatorial Multi-input and Multi-output multiplexer blocks conforming to Pipeline API

This work is funded through NLnet under Grant 2019-02-012

License: LGPLv3+

Multi-input is complex because if any one input is ready, the output can be ready, and the decision comes from a separate module.

Multi-output is simple (pretty much identical to UnbufferedPipeline), and the selection is just a mux. The only proviso (difference) being: the outputs not being selected have to have their ready_o signals DEASSERTED.

https://bugs.libre-soc.org/show_bug.cgi?id=538

class nmutil.multipipe.CombMultiInPipeline(stage, p_len, p_mux, maskwid=0, routemask=False)

Bases: nmutil.multipipe.MultiInControlBase

A multi-input Combinatorial block conforming to the Pipeline API

p.data_i : StageInput, shaped according to ispec
The pipeline input
p.data_o : StageOutput, shaped according to ospec
The pipeline output
r_data : input_shape according to ispec
A temporary (buffered) copy of a prior (valid) input. This is HELD if the output is not ready. It is updated SYNCHRONOUSLY.
elaborate(platform)
process(i)
class nmutil.multipipe.CombMultiOutPipeline(stage, n_len, n_mux, maskwid=0, routemask=False)

Bases: nmutil.multipipe.MultiOutControlBase

A multi-input Combinatorial block conforming to the Pipeline API

p.data_i : stage input data (non-array). shaped according to ispec n.data_o : stage output data array. shaped according to ospec

elaborate(platform)
process(i)
class nmutil.multipipe.CombMuxOutPipe(stage, n_len, maskwid=0, muxidname=None, routemask=False)

Bases: nmutil.multipipe.CombMultiOutPipeline

class nmutil.multipipe.InputPriorityArbiter(pipe, num_rows)

Bases: nmigen.hdl.ir.Elaboratable

arbitration module for Input-Mux pipe, baed on PriorityEncoder

elaborate(platform)
ports()
class nmutil.multipipe.MultiInControlBase(in_multi=None, p_len=1, maskwid=0, routemask=False)

Bases: nmigen.hdl.ir.Elaboratable

Common functions for Pipeline API

connect_to_next(nxt, p_idx=0)

helper function to connect to the next stage data/valid/ready.

elaborate(platform)
ports()
set_input(i, idx=0)

helper function to set the input data

class nmutil.multipipe.MultiOutControlBase(n_len=1, in_multi=None, maskwid=0, routemask=False)

Bases: nmigen.hdl.ir.Elaboratable

Common functions for Pipeline API

connect_to_next(nxt, n_idx=0)

helper function to connect to the next stage data/valid/ready.

elaborate(platform)
ports()
set_input(i)

helper function to set the input data

class nmutil.multipipe.NonCombMultiInPipeline(stage, p_len, p_mux, maskwid=0, routemask=False)

Bases: nmutil.multipipe.MultiInControlBase

A multi-input pipeline block conforming to the Pipeline API

p.data_i : StageInput, shaped according to ispec
The pipeline input
p.data_o : StageOutput, shaped according to ospec
The pipeline output
r_data : input_shape according to ispec
A temporary (buffered) copy of a prior (valid) input. This is HELD if the output is not ready. It is updated SYNCHRONOUSLY.
elaborate(platform)
process(i)
class nmutil.multipipe.PriorityCombMuxInPipe(stage, p_len=2, maskwid=0, routemask=False)

Bases: nmutil.multipipe.CombMultiInPipeline

an example of how to use the combinatorial pipeline.

nmutil.nmoperator module

nmigen operator functions / utils

This work is funded through NLnet under Grant 2019-02-012

License: LGPLv3+

eq: –

a strategically very important function that is identical in function to nmigen’s Signal.eq function, except it may take objects, or a list of objects, or a tuple of objects, and where objects may also be Records.

class nmutil.nmoperator.Visitor

Bases: object

a helper class for iterating single-argument compound data structures. similar to Visitor2.

array_iter(ai)
iterate(i)

iterate a compound structure recursively using yield

record_iter(ai)
class nmutil.nmoperator.Visitor2

Bases: object

a helper class for iterating twin-argument compound data structures.

Record is a special (unusual, recursive) case, where the input may be specified as a dictionary (which may contain further dictionaries, recursively), where the field names of the dictionary must match the Record’s field spec. Alternatively, an object with the same member names as the Record may be assigned: it does not have to be a Record.

ArrayProxy is also special-cased, it’s a bit messy: whilst ArrayProxy has an eq function, the object being assigned to it (e.g. a python object) might not. despite the input having an eq function, that doesn’t help us, because it’s the ArrayProxy that’s being assigned to. so…. we cheat. use the ports() function of the python object, enumerate them, find out the list of Signals that way, and assign them.

arrayproxy_iter2(ao, ai)
dict_iter2(o, i)
iterator2(o, i)
record_iter2(ao, ai)
nmutil.nmoperator.cat(i)

flattens a compound structure recursively using Cat

nmutil.nmoperator.eq(o, i)

makes signals equal: a helper routine which identifies if it is being passed a list (or tuple) of objects, or signals, or Records, and calls the objects’ eq function.

nmutil.nmoperator.shape(i)

nmutil.noconflict module

nmutil.noconflict.classmaker(left_metas=(), right_metas=())
nmutil.noconflict.get_noconflict_metaclass(bases, left_metas, right_metas)

Not intended to be used outside of this module, unless you know what you are doing.

nmutil.noconflict.remove_redundant(metaclasses)
nmutil.noconflict.skip_redundant(iterable, skipset=None)

Redundant items are repeated items or items in the original skipset.

nmutil.picker module

Priority Picker: optimised back-to-back PriorityEncoder and Decoder and MultiPriorityPicker: cascading mutually-exclusive pickers

This work is funded through NLnet under Grant 2019-02-012

License: LGPLv3+

PriorityPicker: the input is N bits, the output is N bits wide and only one is enabled.

MultiPriorityPicker: likewise except that there are M pickers and each output is guaranteed mutually exclusive. Optionally: an “index” (and enable line) is also outputted.

MultiPriorityPicker is designed for port-selection, when there are multiple “things” (of width N) contending for access to M “ports”. When the M=0 “thing” requests a port, it gets allocated port 0 (always). However if the M=0 “thing” does not request a port, this gives the M=1 “thing” the opportunity to gain access to port 0.

Given that N may potentially be much greater than M (16 bits wide where M may be e.g. only 4) we can’t just ok, “ok so M=N therefore M=0 gets access to port 0, M=1 gets access to port 1” etc.

class nmutil.picker.MultiPriorityPicker(wid, levels, indices=False, multiin=False)

Bases: nmigen.hdl.ir.Elaboratable

implements a multi-input priority picker Mx inputs of N bits, Mx outputs of N bits, only one is set

Each picker masks out the one below it, such that the first gets top priority, the second cannot have the same bit that the first has set, and so on. To do this, a “mask” accumulates the output from the chain, masking the input to the next chain.

Also outputted (optional): an index for each picked “thing”.

elaborate(platform)
ports()
class nmutil.picker.PriorityPicker(wid, lsb_mode=False, reverse_i=False, reverse_o=False)

Bases: nmigen.hdl.ir.Elaboratable

implements a priority-picker. input: N bits, output: N bits

  • lsb_mode is for a LSB-priority picker
  • reverse_i=True is for convenient reverseal of the input bits
  • reverse_o=True is for convenient reversal of the output bits
elaborate(platform)
ports()

nmutil.pipeline module

This work is funded through NLnet under Grant 2019-02-012

License: LGPLv3+

class nmutil.pipeline.AutoPipe(stage, assigns)

Bases: nmutil.singlepipe.UnbufferedPipeline

elaborate(platform)

handles case where stage has dynamic ready/valid functions

class nmutil.pipeline.AutoStage(inspecs, outspecs, eqs, assigns)

Bases: nmutil.stageapi.StageCls

ispec()
ospec()
process(i)
setup(m, i)
class nmutil.pipeline.ObjectProxy(m, name=None, pipemode=False, syncmode=True)

Bases: object

eq(i)
get_specs(liked=False)
classmethod like(m, value, pipemode=False, name=None, src_loc_at=0, **kwargs)
ports()
class nmutil.pipeline.PipeManager(m, pipemode=False, pipetype=None)

Bases: object

Stage(name, prev=None, ispec=None)
get_specs(stage, name, liked=False)
class nmutil.pipeline.PipelineStage(name, m, prev=None, pipemode=False, ispec=None)

Bases: object

Pipeline builder stage with auto generation of pipeline registers.

class nmutil.pipeline.SimplePipeline(m)

Bases: object

Pipeline builder with auto generation of pipeline registers.

nmutil.pipeline.get_assigns(_assigns)
nmutil.pipeline.get_eqs(_eqs)
nmutil.pipeline.like(value, rname, pipe, pipemode=False)
nmutil.pipeline.likedict(specs)
nmutil.pipeline.likelist(specs)

nmutil.pipemodbase module

This work is funded through NLnet under Grant 2019-02-012

License: LGPLv3+

Associated bugreports: * https://bugs.libre-soc.org/show_bug.cgi?id=538

class nmutil.pipemodbase.PipeModBase(pspec, modname)

Bases: nmigen.hdl.ir.Elaboratable

PipeModBase: common code between nearly every pipeline module

process(i)
setup(m, i)

links module to inputs and outputs

class nmutil.pipemodbase.PipeModBaseChain(pspec)

Bases: nmutil.dynamicpipe.DynamicPipe

PipeModBaseChain: common code between stage-chained pipes

Links a set of combinatorial modules (get_chain) together and uses pspec.pipekls to dynamically select the pipeline type Also conforms to the Pipeline Stage API

ispec()

returns the input spec of the first module in the chain

ospec()

returns the output spec of the last module in the chain

process(i)
setup(m, i)

links module to inputs and outputs

nmutil.plru module

class nmutil.plru.PLRU(BITS)

Bases: nmigen.hdl.ir.Elaboratable

PLRU - Pseudo Least Recently Used Replacement

PLRU-tree indexing: lvl0 0

/ / lvl1 1 2

/ / lvl2 3 4 5 6

/ // / … … … …

elaborate(platform=None)
ports()

nmutil.queue module

class nmutil.queue.Queue(width, depth, fwft=True, pipe=False)

Bases: nmigen.lib.fifo.FIFOInterface, nmigen.hdl.ir.Elaboratable

elaborate(platform)

nmutil.ripple module

This work is funded through NLnet under Grant 2019-02-012

License: LGPLv3+

class nmutil.ripple.MoveMSBDown(width)

Bases: nmigen.hdl.ir.Elaboratable

based on a partition mask, moves the MSB down to the LSB position. only the MSB is relevant, other bits are ignored. works by first rippling the MSB across the entire partition (TODO: split that out into its own useful module), then ANDs the (new) LSB with the partition mask to isolate it.

elaborate(platform)
class nmutil.ripple.RippleLSB(width)

Bases: nmigen.hdl.ir.Elaboratable

based on a partition mask, the LSB is “rippled” (duplicated) up to the beginning of the next partition.

elaborate(platform)

nmutil.sim_tmp_alternative module

Run-time selection of simulator engine

Usage:

from nmutil.sim_tmp_alternative import Simulator

Then, use Simulator as usual.

This should be backwards compatible to old developer versions of nMigen.

To use cxxsim, export NMIGEN_SIM_MODE=cxxsim from the shell. Be sure to check out the cxxsim branch of nMigen, and update yosys to the latest commit as well.

To use pysim, just keep NMIGEN_SIM_MODE unset. Alternatively, export NMIGEN_SIM_MODE=pysim.

Example:

$ export NMIGEN_SIM_MODE=...  # pysim or cxxsim, default is pysim
$ python ...

or, even:

$ NMIGEN_SIM_MODE=...  python ...
nmutil.sim_tmp_alternative.Simulator(*args, **kwargs)

Wrapper that allows run-time selection of simulator engine

nmutil.sim_tmp_alternative.is_engine_cxxsim()

Returns True if the selected engine is cxxsim

nmutil.sim_tmp_alternative.is_engine_pysim()

Returns True if the selected engine is pysim

nmutil.sim_tmp_alternative.nmigen_sim_environ_variable = 'pysim'

Detected run-time engine from environment

nmutil.sim_tmp_alternative.nmigen_sim_top_module = 'top.'

Work-around for cxxsim not defining the top-level module

nmutil.singlepipe module

Pipeline API. For multi-input and multi-output variants, see multipipe.

This work is funded through NLnet under Grant 2019-02-012

License: LGPLv3+

Associated development bugs: * http://bugs.libre-riscv.org/show_bug.cgi?id=148 * http://bugs.libre-riscv.org/show_bug.cgi?id=64 * http://bugs.libre-riscv.org/show_bug.cgi?id=57

Important: see Stage API (stageapi.py) and IO Control API (iocontrol.py) in combination with below. This module “combines” the Stage API with the IO Control API to create the Pipeline API.

The one critically important key difference between StageAPI and PipelineAPI:

  • StageAPI: combinatorial (NO REGISTERS / LATCHES PERMITTED)
  • PipelineAPI: synchronous registers / latches get added here

RecordBasedStage:

A convenience class that takes an input shape, output shape, a “processing” function and an optional “setup” function. Honestly though, there’s not much more effort to just… create a class that returns a couple of Records (see ExampleAddRecordStage in examples).

PassThroughStage:

A convenience class that takes a single function as a parameter, that is chain-called to create the exact same input and output spec. It has a process() function that simply returns its input.

Instances of this class are completely redundant if handed to StageChain, however when passed to UnbufferedPipeline they can be used to introduce a single clock delay.

ControlBase:

The base class for pipelines. Contains previous and next ready/valid/data. Also has an extremely useful “connect” function that can be used to connect a chain of pipelines and present the exact same prev/next ready/valid/data API.

Note: pipelines basically do not become pipelines as such until handed to a derivative of ControlBase. ControlBase itself is not strictly considered a pipeline class. Wishbone and AXI4 (master or slave) could be derived from ControlBase, for example. UnbufferedPipeline: ——————

A simple stalling clock-synchronised pipeline that has no buffering (unlike BufferedHandshake). Data flows on every clock cycle when the conditions are right (this is nominally when the input is valid and the output is ready).

A stall anywhere along the line will result in a stall back-propagating down the entire chain. The BufferedHandshake by contrast will buffer incoming data, allowing previous stages one clock cycle’s grace before also having to stall.

An advantage of the UnbufferedPipeline over the Buffered one is that the amount of logic needed (number of gates) is greatly reduced (no second set of buffers basically)

The disadvantage of the UnbufferedPipeline is that the valid/ready logic, if chained together, is combinatorial, resulting in progressively larger gate delay.

PassThroughHandshake:

A Control class that introduces a single clock delay, passing its data through unaltered. Unlike RegisterPipeline (which relies on UnbufferedPipeline and PassThroughStage) it handles ready/valid itself.

RegisterPipeline:

A convenience class that, because UnbufferedPipeline introduces a single clock delay, when its stage is a PassThroughStage, it results in a Pipeline stage that, duh, delays its (unmodified) input by one clock cycle.

BufferedHandshake:

nmigen implementation of buffered pipeline stage, based on zipcpu: https://zipcpu.com/blog/2017/08/14/strategies-for-pipelining.html

this module requires quite a bit of thought to understand how it works (and why it is needed in the first place). reading the above is strongly recommended.

unlike john dawson’s IEEE754 FPU STB/ACK signalling, which requires the STB / ACK signals to raise and lower (on separate clocks) before data may proceeed (thus only allowing one piece of data to proceed on ALTERNATE cycles), the signalling here is a true pipeline where data will flow on every clock when the conditions are right.

input acceptance conditions are when:
  • incoming previous-stage strobe (p.valid_i) is HIGH
  • outgoing previous-stage ready (p.ready_o) is LOW
output transmission conditions are when:
  • outgoing next-stage strobe (n.valid_o) is HIGH
  • outgoing next-stage ready (n.ready_i) is LOW

the tricky bit is when the input has valid data and the output is not ready to accept it. if it wasn’t for the clock synchronisation, it would be possible to tell the input “hey don’t send that data, we’re not ready”. unfortunately, it’s not possible to “change the past”: the previous stage has no choice but to pass on its data.

therefore, the incoming data must be accepted - and stored: that is the responsibility / contract that this stage must accept. on the same clock, it’s possible to tell the input that it must not send any more data. this is the “stall” condition.

we now effectively have two possible pieces of data to “choose” from: the buffered data, and the incoming data. the decision as to which to process and output is based on whether we are in “stall” or not. i.e. when the next stage is no longer ready, the output comes from the buffer if a stall had previously occurred, otherwise it comes direct from processing the input.

this allows us to respect a synchronous “travelling STB” with what dan calls a “buffered handshake”.

it’s quite a complex state machine!

SimpleHandshake

Synchronised pipeline, Based on: https://github.com/ZipCPU/dbgbus/blob/master/hexbus/rtl/hbdeword.v

class nmutil.singlepipe.BufferedHandshake(stage, in_multi=None, stage_ctl=False)

Bases: nmutil.singlepipe.FIFOControl

class nmutil.singlepipe.ControlBase(stage=None, in_multi=None, stage_ctl=False, maskwid=0)

Bases: nmutil.stageapi.StageHelper, nmigen.hdl.ir.Elaboratable

Common functions for Pipeline API. Note: a “pipeline stage” only exists (conceptually) when a ControlBase derivative is handed a Stage (combinatorial block)

NOTE: ControlBase derives from StageHelper, making it accidentally compliant with the Stage API. Using those functions directly BYPASSES a ControlBase instance ready/valid signalling, which clearly should not be done without a really, really good reason.

connect(pipechain)

connects a chain (list) of Pipeline instances together and links them to this ControlBase instance:

in <—-> self <—> out
^

v |

[pipe1, pipe2, pipe3, pipe4]
^ | ^ | ^

v | v | v |

out—in out–in out—in

Also takes care of allocating data_i/data_o, by looking up the data spec for each end of the pipechain. i.e It is NOT necessary to allocate self.p.data_i or self.n.data_o manually: this is handled AUTOMATICALLY, here.

Basically this function is the direct equivalent of StageChain, except that unlike StageChain, the Pipeline logic is followed.

Just as StageChain presents an object that conforms to the Stage API from a list of objects that also conform to the Stage API, an object that calls this Pipeline connect function has the exact same pipeline API as the list of pipline objects it is called with.

Thus it becomes possible to build up larger chains recursively. More complex chains (multi-input, multi-output) will have to be done manually.

Argument:

  • pipechain:
    • a sequence of ControlBase-derived classes

    (must be one or more in length)

Returns:

  • a list of eq assignments that will need to be added in an elaborate() to m.d.comb
connect_to_next(nxt)

helper function to connect to the next stage data/valid/ready.

data_r
elaborate(platform)

handles case where stage has dynamic ready/valid functions

ports()
set_input(i)

helper function to set the input data (used in unit tests)

class nmutil.singlepipe.FIFOControl(depth, stage, in_multi=None, stage_ctl=False, fwft=True, pipe=False)

Bases: nmutil.singlepipe.ControlBase

FIFO Control. Uses Queue to store data, coincidentally happens to have same valid/ready signalling as Stage API.

data_i -> fifo.din -> FIFO -> fifo.dout -> data_o

elaborate(platform)

handles case where stage has dynamic ready/valid functions

class nmutil.singlepipe.MaskCancellable(stage, maskwid, in_multi=None, stage_ctl=False, dynamic=False)

Bases: nmutil.singlepipe.ControlBase

Mask-activated Cancellable pipeline

Arguments:

  • stage. see Stage API above
  • maskwid - sets up cancellation capability (mask and stop).
  • in_multi
  • stage_ctl
  • dynamic - allows switching from sync to combinatorial (passthrough)
    USE WITH CARE. will need the entire pipe to be quiescent before switching, otherwise data WILL be destroyed.

stage-1 p.valid_i >>in stage n.valid_o out>> stage+1 stage-1 p.ready_o <<out stage n.ready_i <<in stage+1 stage-1 p.data_i >>in stage n.data_o out>> stage+1

|

+–process->–^

elaborate(platform)

handles case where stage has dynamic ready/valid functions

class nmutil.singlepipe.MaskNoDelayCancellable(stage, maskwid, in_multi=None, stage_ctl=False)

Bases: nmutil.singlepipe.ControlBase

Mask-activated Cancellable pipeline (that does not respect “ready”)

Based on (identical behaviour to) SimpleHandshake. TODO: decide whether to merge into SimpleHandshake.

Argument: stage. see Stage API above

stage-1 p.valid_i >>in stage n.valid_o out>> stage+1 stage-1 p.ready_o <<out stage n.ready_i <<in stage+1 stage-1 p.data_i >>in stage n.data_o out>> stage+1

|

+–process->–^

elaborate(platform)

handles case where stage has dynamic ready/valid functions

class nmutil.singlepipe.PassThroughHandshake(stage, in_multi=None, stage_ctl=False)

Bases: nmutil.singlepipe.FIFOControl

class nmutil.singlepipe.PassThroughStage(iospecfn)

Bases: nmutil.stageapi.StageCls

a pass-through stage with its input data spec identical to its output, and “passes through” its data from input to output (does nothing).

use this basically to explicitly make any data spec Stage-compliant. (many APIs would potentially use a static “wrap” method in e.g.

StageCls to achieve a similar effect)
ispec()
ospec()
class nmutil.singlepipe.RecordBasedStage(in_shape, out_shape, processfn, setupfn=None)

Bases: nmutil.stageapi.Stage

convenience class which provides a Records-based layout. honestly it’s a lot easier just to create a direct Records-based class (see ExampleAddRecordStage)

ispec()
ospec()
process(i)
setup(m, i)
class nmutil.singlepipe.RegisterPipeline(iospecfn)

Bases: nmutil.singlepipe.UnbufferedPipeline

A pipeline stage that delays by one clock cycle, creating a sync’d latch out of data_o and valid_o as an indirect byproduct of using PassThroughStage

class nmutil.singlepipe.SimpleHandshake(stage=None, in_multi=None, stage_ctl=False, maskwid=0)

Bases: nmutil.singlepipe.ControlBase

simple handshake control. data and strobe signals travel in sync. implements the protocol used by Wishbone and AXI4.

Argument: stage. see Stage API above

stage-1 p.valid_i >>in stage n.valid_o out>> stage+1 stage-1 p.ready_o <<out stage n.ready_i <<in stage+1 stage-1 p.data_i >>in stage n.data_o out>> stage+1

|

+–process->–^

Truth Table

Inputs Temporary Output Data ——- ———- —– —- P P N N PiV& ~NiR& N P i o i o PoR NoV o o V R R V V R

——- - - - - 0 0 0 0 0 0 >0 0 reg 0 0 0 1 0 1 >1 0 reg 0 0 1 0 0 0 0 1 process(data_i) 0 0 1 1 0 0 0 1 process(data_i) ——- - - - - 0 1 0 0 0 0 >0 0 reg 0 1 0 1 0 1 >1 0 reg 0 1 1 0 0 0 0 1 process(data_i) 0 1 1 1 0 0 0 1 process(data_i) ——- - - - - 1 0 0 0 0 0 >0 0 reg 1 0 0 1 0 1 >1 0 reg 1 0 1 0 0 0 0 1 process(data_i) 1 0 1 1 0 0 0 1 process(data_i) ——- - - - - 1 1 0 0 1 0 1 0 process(data_i) 1 1 0 1 1 1 1 0 process(data_i) 1 1 1 0 1 0 1 1 process(data_i) 1 1 1 1 1 0 1 1 process(data_i) ——- - - - -

elaborate(platform)

handles case where stage has dynamic ready/valid functions

class nmutil.singlepipe.UnbufferedPipeline(stage, in_multi=None, stage_ctl=False)

Bases: nmutil.singlepipe.FIFOControl

class nmutil.singlepipe.UnbufferedPipeline2(stage=None, in_multi=None, stage_ctl=False, maskwid=0)

Bases: nmutil.singlepipe.ControlBase

A simple pipeline stage with single-clock synchronisation and two-way valid/ready synchronised signalling.

Note that a stall in one stage will result in the entire pipeline chain stalling.

Also that unlike BufferedHandshake, the valid/ready signalling does NOT travel synchronously with the data: the valid/ready signalling combines in a combinatorial fashion. Therefore, a long pipeline chain will lengthen propagation delays.

Argument: stage. see Stage API, above

stage-1 p.valid_i >>in stage n.valid_o out>> stage+1 stage-1 p.ready_o <<out stage n.ready_i <<in stage+1 stage-1 p.data_i >>in stage n.data_o out>> stage+1

| |

+- process-> buf <-+

p.data_i : StageInput, shaped according to ispec
The pipeline input
p.data_o : StageOutput, shaped according to ospec
The pipeline output
buf : output_shape according to ospec
A temporary (buffered) copy of a valid output This is HELD if the output is not ready. It is updated SYNCHRONOUSLY.

Inputs Temp Output Data ——- - —– P P N N ~NiR& N P (buf_full) i o i o NoV o o V R R V V R

——- - - - 0 0 0 0 0 0 1 process(data_i) 0 0 0 1 1 1 0 reg (odata, unchanged) 0 0 1 0 0 0 1 process(data_i) 0 0 1 1 0 0 1 process(data_i) ——- - - - 0 1 0 0 0 0 1 process(data_i) 0 1 0 1 1 1 0 reg (odata, unchanged) 0 1 1 0 0 0 1 process(data_i) 0 1 1 1 0 0 1 process(data_i) ——- - - - 1 0 0 0 0 1 1 process(data_i) 1 0 0 1 1 1 0 reg (odata, unchanged) 1 0 1 0 0 1 1 process(data_i) 1 0 1 1 0 1 1 process(data_i) ——- - - - 1 1 0 0 0 1 1 process(data_i) 1 1 0 1 1 1 0 reg (odata, unchanged) 1 1 1 0 0 1 1 process(data_i) 1 1 1 1 0 1 1 process(data_i) ——- - - -

Note: PoR is NOT involved in the above decision-making.

elaborate(platform)

handles case where stage has dynamic ready/valid functions

nmutil.stageapi module

Stage API

This work is funded through NLnet under Grant 2019-02-012

License: LGPLv3+

Associated development bugs: * http://bugs.libre-riscv.org/show_bug.cgi?id=148 * http://bugs.libre-riscv.org/show_bug.cgi?id=64 * http://bugs.libre-riscv.org/show_bug.cgi?id=57

Stage API:

stage requires compliance with a strict API that may be implemented in several means, including as a static class.

Stages do not HOLD data, and they definitely do not contain signalling (ready/valid). They do however specify the FORMAT of the incoming and outgoing data, and they provide a means to PROCESS that data (from incoming format to outgoing format).

Stage Blocks really should be combinatorial blocks (Moore FSMs). It would be ok to have input come in from sync’d sources (clock-driven, Mealy FSMs) however by doing so they would no longer be deterministic, and chaining such blocks with such side-effects together could result in unexpected, unpredictable, unreproduceable behaviour.

So generally to be avoided, then unless you know what you are doing. https://en.wikipedia.org/wiki/Moore_machine https://en.wikipedia.org/wiki/Mealy_machine

the methods of a stage instance must be as follows:

  • ispec() - Input data format specification. Takes a bit of explaining.

    The requirements are: something that eventually derives from nmigen Value must be returned OR an iterator or iterable or sequence (list, tuple etc.) or generator must yield thing(s) that (eventually) derive from the nmigen Value class.

    Complex to state, very simple in practice: see test_buf_pipe.py for over 25 worked examples.

  • ospec() - Output data format specification.

    format requirements identical to ispec.

  • process(m, i) - Optional function for processing ispec-formatted data.

    returns a combinatorial block of a result that may be assigned to the output, by way of the “nmoperator.eq” function. Note that what is returned here can be extremely flexible. Even a dictionary can be returned as long as it has fields that match precisely with the Record into which its values is intended to be assigned. Again: see example unit tests for details.

  • setup(m, i) - Optional function for setting up submodules.

    may be used for more complex stages, to link the input (i) to submodules. must take responsibility for adding those submodules to the module (m). the submodules must be combinatorial blocks and must have their inputs and output linked combinatorially.

Both StageCls (for use with non-static classes) and Stage (for use by static classes) are abstract classes from which, for convenience and as a courtesy to other developers, anything conforming to the Stage API may choose to derive. See Liskov Substitution Principle: https://en.wikipedia.org/wiki/Liskov_substitution_principle

StageChain:

A useful combinatorial wrapper around stages that chains them together and then presents a Stage-API-conformant interface. By presenting the same API as the stages it wraps, it can clearly be used recursively.

StageHelper:

A convenience wrapper around a Stage-API-compliant “thing” which complies with the Stage API and provides mandatory versions of all the optional bits.

class nmutil.stageapi.Stage

Bases: object

Static “Stage” API. does not require instantiation (after derivation)

see “Stage API” above. Note: python does not require derivation from this class. All that is required is that the pipelines have the functions listed in this class. Derivation from this class is therefore merely a “courtesy” to maintainers.

static ispec()
static ospec()
class nmutil.stageapi.StageChain(chain, specallocate=False)

Bases: nmutil.stageapi.StageHelper

pass in a list of stages (combinatorial blocks), and they will automatically be chained together via their input and output specs into a combinatorial chain, to create one giant combinatorial block.

the end result conforms to the exact same Stage API.

  • input to this class will be the input of the first stage
  • output of first stage goes into input of second
  • output of second goes into input into third
  • … (etc. etc.)
  • the output of this class will be the output of the last stage

NOTE: whilst this is very similar to ControlBase.connect(), it is really important to appreciate that StageChain is pure combinatorial and bypasses (does not involve, at all, ready/valid signalling OF ANY KIND).

ControlBase.connect on the other hand respects, connects, and uses ready/valid signalling.

Arguments:

  • chain:a chain of combinatorial blocks conforming to the Stage API NOTE: StageChain.ispec and ospect have to have something to return (beginning and end specs of the chain), therefore the chain argument must be non-zero length
  • specallocate:if set, new input and output data will be allocated and connected (eq’d) to each chained Stage. in some cases if this is not done, the nmigen warning “driving from two sources, module is being flattened” will be issued.

NOTE: DO NOT use StageChain with combinatorial blocks that have side-effects (state-based / clock-based input) or conditional (inter-chain) dependencies, unless you really know what you are doing.

process(i)
class nmutil.stageapi.StageCls

Bases: object

Class-based “Stage” API. requires instantiation (after derivation)

see “Stage API” above.. Note: python does not require derivation from this class. All that is required is that the pipelines have the functions listed in this class. Derivation from this class is therefore merely a “courtesy” to maintainers.

ispec()
ospec()
class nmutil.stageapi.StageHelper(stage)

Bases: nmutil.stageapi.Stage

a convenience wrapper around something that is Stage-API-compliant. (that “something” may be a static class, for example).

StageHelper happens to also be compliant with the Stage API, it differs from the stage that it wraps in that all the “optional” functions are provided (hence the designation “convenience wrapper”)

ispec(name=None)
new_specs(name)

allocates new ispec and ospec pair

ospec(name=None)
process(i)
set_specs(p, n)

sets up the ispecfn and ospecfn for getting input and output data

setup(m, i)

nmutil.util module

This work is funded through NLnet under Grant 2019-02-012

License: LGPLv3+

nmutil.util.eq32(is_32bit, dest, src)
nmutil.util.flatten(v)
nmutil.util.rising_edge(m, sig)
nmutil.util.sel(m, r, sel_bits, field_width=None, name=None, src_loc_at=0)

Forms a subfield from a selection of bits of the signal r (“register”).

Parameters:
  • m – nMigen Module for adding the wires
  • r – signal containing the field from which to select the subfield
  • sel_bits – bit indices of the subfield, in “MSB 0” convention, from most significant to least significant. Note that the indices are allowed to be non-contiguous and/or out-of-order.
  • field_width – field width. If absent, use the signal r own width.
  • name – name of the generated Signal
  • src_loc_at – in the absence of name, stack level in which to find it
Returns:

a new Signal which gets assigned to the subfield
nmutil.util.treereduce(tree, op, fn)

treereduce: apply a map-reduce to a list. examples: OR-reduction of one member of a list of Records down to a

single data point: treereduce(tree, operator.or_, lambda x: getattr(x, “data_o”))
nmutil.util.wrap(process)

Module contents