RoCEv2Server

For conceptual usage, see:

• RoCEv2 Protocol

• Device

Implementation

This Python API is implemented directly in PyRogue. It is a thin pyrogue.Device wrapper that owns a Boost.Python-exported rogue.protocols.rocev2.Server (see Server). The FPGA-side metadata-bus handshake lives in surf (surf.ethernet.roce.RoCEv2Engine); the application Root drives the host↔FPGA bring-up (getHostParams → engine.setupConnection → completeConnection) — it is no longer a child device of this server.

The 15 LocalVariable status/configuration fields (FpgaIp, FpgaGid, HostQpn, HostGid, HostIp, MrAddr, MrRkey, RxFrameCount, RxByteCount, MaxPayload, RxQueueDepth, HostRqPsn, HostSqPsn, FpgaQpn, FpgaLkey) are auto-rendered by autoclass below from the docstrings in python/pyrogue/protocols/_RoCEv2.py.

class pyrogue.protocols.RoCEv2Server(*, rocev2Cfg, **kwargs)

RoCEv2 RC receive server — pyrogue Device.

Mirrors the interface of UdpRssiPack so the two can be used interchangeably inside a Root.

Parameters:

rocev2Cfg (RoCEv2ServerCfg) – Bring-up / host-NIC configuration (ip, deviceName, ibPort, gidIndex, maxPayload, rxQueueDepth). See RoCEv2ServerCfg.

static detectGidIndex(device, ip, ibPort=1)

Resolve the host’s RoCE v2 IPv4 GID-table index from sysfs.

The mlx5 GID-table index of the RoCE v2 IPv4 GID drifts across NIC / FPGA reloads, so callers that pass gidIndex=-1 resolve it fresh each session. Matches the entry whose type is ‘RoCE v2’ AND whose IPv4-mapped GID (::ffff:a.b.c.d) is on the same /24 as ip — this skips the RoCE v1 slot that shares the same IP.

Returns the integer index, or None if no matching GID is found.

Parameters:

• device (str)

• ip (str)

• ibPort (int)

property stream

Direct access to the C++ stream master — receives every channel the FPGA emits. Use getChannel(n) instead to get a filtered view of a single RDMA-immediate channel id (the per-channel analogue of UdpRssiPack.application(n)).

getChannel(channel)

Return a channel-filtered view (channel id from immediate value bits [7:0]).

The RDMA SEND-with-Immediate carries the channel id in the low 8 bits of the immediate value (Server.cpp runThread decodes ntohl(wc.imm_data) & 0xFF), so valid channel ids are 0..255. Out- of-range values would previously blow up with OverflowError inside the Filter ctor — validate up front and raise a clear rogue.GeneralError instead.

Parameters:

channel (int)

printConnInfo()

Print the RC connection summary (QP/GID/MR + buffer sizing).

Return type:

None

getHostParams()

Read the host RC QP params for the FPGA hand-off.

Returns the five flat values the FPGA engine’s setupConnection() consumes (hostQpn, hostRqPsn, hostSqPsn, mrAddr, mrLen) as a RoCEv2HostParams, so the Root can call engine.setupConnection(**server.getHostParams()._asdict(), pmtu=..., ...).

Return type:

RoCEv2HostParams

completeConnection(fpgaQpn, *, pmtu, minRnrTimer, rnrRetry, retryCount, fpgaLkey=0)

Finish the host RC QP once the FPGA side is up.

The Root calls this with the FPGA result (fpgaQpn/fpgaLkey from the engine’s RoCEv2FpgaParams) plus the transport knobs. Validates the transport knobs at call time (the invariant displaced from __init__), derives fpgaRqPsn internally, drives the C++ server, and sets the FPGA status vars. rnrRetry/retryCount are accepted for symmetry with the engine call site but are not forwarded to the C++ surface.

Return type:

None

add(node)

Add a child node to the device.

Return type:

None

Parameters:

node (Any)

addCustomBlock(block)

Add a pre-defined memory block to the device.

Return type:

None

Parameters:

block (Any)

addInterface(*interfaces)

Add stream or memory interfaces to manage.

Parameters:

*interfaces (Any | Iterable[Any]) – One or more interfaces or iterables of interfaces.

Return type:

None

addNode(nodeClass, **kwargs)

Construct and add a node of nodeClass.

Return type:

None

Parameters:

• nodeClass (type[Node])

• kwargs (Any)

addNodes(nodeClass, number, stride, **kwargs)

Add a series of nodes with indexed names.

Return type:

None

Parameters:

• nodeClass (type[Node])

• number (int)

• stride (int)

• kwargs (Any)

addProtocol(*protocols)

Add protocol entities (alias of addInterface).

Return type:

None

Parameters:

protocols (Any)

addRemoteVariables(number, stride, pack=False, **kwargs)

Add a repeating block of remote variables.

Parameters:

• number (int) – Number of variables to add.

• stride (int) – Byte stride between instances.

• pack (bool, default: False) –

  If True, also add a LinkVariable named <name>_All which combines all entries into one underscore-delimited display string.

  The indexed RemoteVariable container remains available at the original name. The packed alias uses each element variable’s getDisp() and setDisp() behavior, so formatting follows the underlying variable type. Values are joined in reverse index order: for a variable array Field[0], Field[1], Field[2], the packed value Field_All reads like Field[2]_Field[1]_Field[0] and writing "1_2_3" updates Field[2] = 1, Field[1] = 2, and Field[0] = 3.

  This is intended for orthogonal register fields that are convenient to expose both as individual variables and as one combined value.

• **kwargs (Any) – Arguments forwarded to RemoteVariable.

Return type:

None

addToGroup(group)

Add this node to a group and propagate to children.

Parameters:

group (str) – Group name to add.

Return type:

None

property address: Any

Return the device base address.

callRecursive(func, nodeTypes=None, **kwargs)

Call a named method on this node and matching children.

Parameters:

• func (str) – Method name to call.

• nodeTypes (Iterable[type[Node]] | None, default: None) – Node types to include.

• **kwargs (Any) – Arguments forwarded to the method call.

Return type:

None

checkBlocks(*, recurse=True, variable=None, **kwargs)

Deprecated alias for waitBlocks().

Return type:

None

Parameters:

• recurse (bool)

• variable (Any | None)

• kwargs (Any)

classmethod classLogName()

Return the canonical Python logger name for this Node class.

Return type:

str

command(**kwargs)

Decorator to add inline methods as commands.

The decorated function is used as LocalCommand(function=...). The command wrapper supplies keyword arguments root, dev, cmd, and arg; the function may accept any subset.

Return type:

Callable[[Callable[..., Any]], Callable[..., Any]]

Parameters:

kwargs (Any)

property commands: OrderedDict[str, BaseCommand]

Return direct child commands.

commandsByGroup(incGroups=None, excGroups=None)

Return commands filtered by group.

Return type:

OrderedDict[str, BaseCommand]

Parameters:

• incGroups (str | list[str] | None)

• excGroups (str | list[str] | None)

countReset()

Call countReset on all child devices.

Return type:

None

property description: str

Return the node description.

property deviceList: list[Device]

Return a recursive list of devices.

property devices: OrderedDict[str, Device]

Return direct child devices.

devicesByGroup(incGroups=None, excGroups=None)

Return devices filtered by group.

Return type:

OrderedDict[str, Device]

Parameters:

• incGroups (str | list[str] | None)

• excGroups (str | list[str] | None)

enableChanged(value)

Hook for reacting to enable state changes.

Return type:

None

Parameters:

value (Any)

property expand: bool

Return the expand state.

filterByGroup(incGroups, excGroups)

Return True if the node passes include/exclude filters.

Parameters:

• incGroups (str | list[str] | None) – Group name or group names to include.

• excGroups (str | list[str] | None) – Group name or group names to exclude.

Return type:

bool

find(*, recurse=True, typ=None, **kwargs)

Find all child nodes that are a base class of ‘typ’ and whose properties match all of the kwargs. For string properties, accepts regexes.

Parameters:

• recurse (bool, default: True) – If True, recurse into child nodes.

• typ (type[Node] | None, default: None) – Base class type to match.

• **kwargs (Any)

Returns:

List of nodes that match the criteria.

Return type:

list[Node]

property forceCheckEach: bool

Deprecated alias for forceWaitEach.

property forceWaitEach: bool

Force block operations to wait after each transaction.

genDocuments(path, incGroups=None, excGroups=None)

Generate Sphinx documentation pages for this device.

Parameters:

• path (str) – Output directory path.

• incGroups (str | list[str] | None, default: None) – Group name or group names to include.

• excGroups (str | list[str] | None, default: None) – Group name or group names to exclude.

Return type:

None

getNodes(typ, excTyp=None, incGroups=None, excGroups=None)

Get a filtered ordered dictionary of nodes. pass a class to typ to receive a certain type of node exc is a class type to exclude, incGroups is an optional group or list of groups that this node must be part of excGroups is an optional group or list of groups that this node must not be part of

Parameters:

• typ (type[Node])

• excTyp (type[Node] | None, default: None) – Type to exclude.

• incGroups (str | list[str] | None, default: None) – Group name or group names to include.

• excGroups (str | list[str] | None, default: None) – Group name or group names to exclude.

Returns:

Ordered dictionary of nodes.

Return type:

OrderedDict[str, Node]

getYaml(readFirst=False, modes=['RW', 'RO', 'WO'], incGroups=None, excGroups=None, recurse=True)

Return current values as YAML text.

Parameters:

• readFirst (bool, default: False) – If True, perform a full hardware read before exporting.

• modes (list[str], default: ['RW', 'RO', 'WO']) – Variable modes to include. Allowed values are 'RW', 'WO', and 'RO'.

• incGroups (str | list[str] | None, default: None) – Group name or group names to include.

• excGroups (str | list[str] | None, default: None) – Group name or group names to exclude.

• recurse (bool, default: True) – If True, recurse into child devices.

Returns:

YAML-formatted representation of the node subtree.

Return type:

str

property groups: list[str]

Return the node groups.

property guiGroup: str | None

Return the GUI group label.

hardReset()

Call hardReset on all child devices.

Return type:

None

property hidden: bool

Return True if the node is hidden.

hideVariables(hidden, variables=None)

Hide a list of variables or variable names.

Parameters:

• hidden (bool) – True to hide variables, False to show them.

• variables (Iterable[Any] | None, default: None) – Iterable of variable names or variables.

Return type:

None

inGroup(group)

Return True if the node is in the provided group or groups.

Parameters:

group (str) – Group name or list of groups to test.

Return type:

bool

initialize()

Call initialize on all child devices.

Return type:

None

property isCommand: bool

Return True if this node is a command.

property isDevice: bool

Return True if this node is a device.

property isVariable: bool

Return True if this node is a variable (excluding commands).

isinstance(typ)

Return True if this node is an instance of typ.

Return type:

bool

Parameters:

typ (type[Node])

linkVariableGet(**kwargs)

Decorator to add inline linkedGet functions.

The decorated function is used as LinkVariable(linkedGet=...). The linked-get wrapper supplies keyword arguments dev, var, read, index, and check; the function may accept any subset.

Return type:

Callable[[Callable[..., Any]], Callable[..., Any]]

Parameters:

kwargs (Any)

loadYaml(name, writeEach, modes, incGroups=None, excGroups=None)

Load YAML configuration from files or directories.

Parameters:

• name (str | list[str]) – Input file, directory, zip-path, or list of those entries.

• writeEach (bool) – Write each variable as it is applied.

• modes (list[str]) – Variable modes to include. Allowed values are 'RW', 'WO', and 'RO'.

• incGroups (str | list[str] | None, default: None) – Group name or group names to include.

• excGroups (str | list[str] | None, default: None) – Group name or group names to exclude.

Returns:

Returns True when load completes.

Return type:

bool

logName()

Return the canonical logger name for this Node instance.

Return type:

str

makeRecursive(func, nodeTypes=None)

Create a recursive wrapper for a named method.

Parameters:

• func (str) – Method name to call.

• nodeTypes (Iterable[type[Node]] | None, default: None) – Node types to include.

Return type:

Callable[..., None]

manage(*interfaces)

Manage additional interfaces for start/stop.

Return type:

None

Parameters:

interfaces (Any)

property name: str

Return the node name.

node(name)

Return a direct child node by name.

Parameters:

name (str) – Child node name.

Return type:

Node

property nodeCount: int

Return the total node count.

property nodeList: list[Node]

Return a recursive list of nodes.

nodeMatch(name)

Match a node name, including array-style accessors.

Parameters:

name (str) – Node name or array accessor string.

Returns:

Matching nodes.

Return type:

list[Node]

property nodes: OrderedDict[str, Node]

Return an ordered dictionary of direct child nodes.

property offset: Any

Return the device offset.

property parent: Node

Return the parent node.

property path: str

Return the full node path.

printYaml(readFirst=False, modes=['RW', 'RO', 'WO'], incGroups=None, excGroups=None, recurse=False)

Print the YAML representation to stdout.

Parameters:

• readFirst (bool, default: False) – If True, perform a full hardware read before exporting.

• modes (list[str], default: ['RW', 'RO', 'WO']) – Variable modes to include. Allowed values are 'RW', 'WO', and 'RO'.

• incGroups (str | list[str] | None, default: None) – Group name or group names to include.

• excGroups (str | list[str] | None, default: None) – Group name or group names to exclude.

• recurse (bool, default: False) – If True, recurse into child devices.

Return type:

None

readAndCheckBlocks(recurse=True, variable=None, checkEach=None, *, waitEach=None)

Deprecated alias for readAndWaitBlocks().

Return type:

None

Parameters:

• recurse (bool)

• variable (Any | None)

• checkEach (bool | None)

• waitEach (bool | None)

readAndWaitBlocks(recurse=True, variable=None, waitEach=None, *, checkEach=None)

Read blocks and wait for read transactions to complete.

Parameters:

• recurse (bool, default: True) – If True, recurse into child devices.

• variable (Any | None, default: None) – Optional variable to read.

• waitEach (bool | None, default: None) – Wait for each block transaction before starting the next one.

• checkEach (bool | None, default: None) – Deprecated alias for waitEach.

Return type:

None

readBlocks(*, recurse=True, variable=None, waitEach=None, checkEach=None, index=-1, **kwargs)

Read blocks in the background.

Parameters:

• recurse (bool, default: True) – If True, recurse into child devices.

• variable (Any | None, default: None) – Optional variable to read.

• waitEach (bool | None, default: None) – Wait for each block transaction before starting the next one.

• checkEach (bool | None, default: None) – Deprecated alias for waitEach.

• index (int, default: -1) – Optional index for array variables.

• **kwargs (Any) – Additional arguments passed through to the transaction.

Return type:

None

removeFromGroup(group)

Remove this node from a group.

Parameters:

group (str) – Group name to remove.

Return type:

None

property root: Root

Return the root node.

property running: bool

Return True if the device is running.

saveYaml(name, readFirst, modes=['RW', 'RO', 'WO'], incGroups=None, excGroups=None, autoPrefix='', autoCompress=False)

Save YAML configuration or status to a file.

Parameters:

• name (str | None) – Destination file path. If empty, a timestamped name is generated.

• readFirst (bool) – Read values from hardware before exporting.

• modes (list[str], default: ['RW', 'RO', 'WO']) – Variable modes to include. Allowed values are 'RW', 'WO', and 'RO'.

• incGroups (str | list[str] | None, default: None) – Group name or group names to include.

• excGroups (str | list[str] | None, default: None) – Group name or group names to exclude.

• autoPrefix (str, default: '') – Prefix for auto-generated filenames.

• autoCompress (bool, default: False) – Generate a .zip file when auto-generating names. Default False

Returns:

Returns True when export completes.

Return type:

bool

classmethod setClassLogLevel(level, *, includePython=True, includeRogue=True)

Set the logging level for this Node class logger family.

Return type:

str

Parameters:

• level (int | str)

• includePython (bool)

• includeRogue (bool)

setLogLevel(level, *, includePython=True, includeRogue=True)

Set the logging level for this Node instance logger.

Return type:

str

Parameters:

• level (int | str)

• includePython (bool)

• includeRogue (bool)

setName((Slave)arg1, (str)arg2) → None :

C++ signature :

void setName(rogue::interfaces::memory::Slave {lvalue},std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)

setPollInterval(interval, variables=None)

Set the poll interval for a group of variables.

Parameters:

• interval (float) – Polling interval in seconds.

• variables (Iterable[Any] | None, default: None) – Iterable of variable names or variables. If None, all variables with nonzero poll interval are updated.

Return type:

None

setYaml(yml, writeEach, modes, incGroups=None, excGroups=None)

Set variable values from YAML text.

Parameters:

• yml (str) – YAML text containing values to apply.

• writeEach (bool) – Write each variable as it is applied.

• modes (list[str]) – Variable modes to include. Allowed values are 'RW', 'WO', and 'RO'.

• incGroups (str | list[str] | None, default: None) – Group name or group names to include.

• excGroups (str | list[str] | None, default: None) – Group name or group names to exclude.

Return type:

None

property variableList: list[BaseVariable]

Return a recursive list of variables and commands.

property variables: OrderedDict[str, BaseVariable]

Return direct child variables (excluding commands).

variablesByGroup(incGroups=None, excGroups=None)

Return variables filtered by group.

Return type:

OrderedDict[str, BaseVariable]

Parameters:

• incGroups (str | list[str] | None)

• excGroups (str | list[str] | None)

verifyBlocks(*, recurse=True, variable=None, waitEach=None, checkEach=None, **kwargs)

Verify blocks in the background.

Parameters:

• recurse (bool, default: True) – If True, recurse into child devices.

• variable (Any | None, default: None) – Optional variable to verify.

• waitEach (bool | None, default: None) – Wait for each block transaction before starting the next one.

• checkEach (bool | None, default: None) – Deprecated alias for waitEach.

• **kwargs (Any) – Additional arguments passed through to the transaction.

Return type:

None

waitBlocks(*, recurse=True, variable=None, **kwargs)

Wait for block transactions and notify variable listeners.

Parameters:

• recurse (bool, default: True) – If True, recurse into child devices.

• variable (Any | None, default: None) – Optional variable to wait on.

• **kwargs (Any) – Additional arguments passed through to the transaction.

Return type:

None

writeAndVerifyBlocks(force=False, recurse=True, variable=None, checkEach=None, *, waitEach=None)

Write, verify, and wait for block transactions to complete.

Parameters:

• force (bool, default: False) – Force the write even if values are unchanged.

• recurse (bool, default: True) – If True, recurse into child devices.

• variable (Any | None, default: None) – Optional variable to write/verify.

• waitEach (bool | None, default: None) – Wait for each block transaction before starting the next one.

• checkEach (bool | None, default: None) – Deprecated alias for waitEach.

Return type:

None

writeBlocks(*, force=False, recurse=True, variable=None, waitEach=None, checkEach=None, index=-1, **kwargs)

Write all blocks held by this device.

Parameters:

• force (bool, default: False) – Force the write even if values are unchanged.

• recurse (bool, default: True) – If True, recurse into child devices.

• variable (Any | None, default: None) – Optional variable to write.

• waitEach (bool | None, default: None) – Wait for each block transaction before starting the next one.

• checkEach (bool | None, default: None) – Deprecated alias for waitEach.

• index (int, default: -1) – Optional index for array variables.

• **kwargs (Any) – Additional arguments passed through to the transaction.

Return type:

None

Configuration dataclasses

RoCEv2Server takes its host-NIC bring-up config as a RoCEv2ServerCfg. The transport / QP-tuning knobs shared between the FPGA engine.setupConnection() and the host server.completeConnection() calls are bundled in RoCEv2TransportCfg. The pmtu and minRnrTimer fields take values from the RoCEv2Mtu and RoCEv2RnrTimer enums.

class pyrogue.protocols.RoCEv2ServerCfg(ip, deviceName, ibPort=1, gidIndex=-1, maxPayload=4096, rxQueueDepth=None)

Bring-up / transport configuration for RoCEv2Server.

Bundles the ibverbs + QP-tuning knobs so the RoCEv2Server(...) call site stays compact. Most defaults mirror the historical RoCEv2Server constructor; maxPayload now defaults to 4096 (one MTU_4096 PMTU) rather than the C++ DefaultMaxPayload of 9000 — pass None to restore the 9000-byte default.

Parameters:

• ip (str) – FPGA IP address. Used to derive the FPGA GID (IPv4-mapped IPv6).

• deviceName (str) – ibverbs device name (e.g. ‘rxe0’ for softRoCE, ‘mlx5_0’ for HW NIC).

• ibPort (int, default: 1) – ibverbs port number (default: 1).

• gidIndex (int, default: -1) – GID table index for the host NIC’s RoCEv2 IPv4 address. -1 (the default) auto-detects the index matching ip by scanning sysfs (/sys/class/infiniband/<dev>/ports/<port>/gid_attrs/types + gids); see RoCEv2Server.detectGidIndex().

• maxPayload (int | None, default: 4096) – Maximum payload bytes per RDMA SEND (the size of each pre-posted receive slot). Defaults to 4096 (one MTU_4096 PMTU); pass None to use the C++ DefaultMaxPayload (9000).

• rxQueueDepth (int | None, default: None) – Number of pre-posted receive slots. None (the default) uses the C++ DefaultRxQueueDepth (256).

class pyrogue.protocols.RoCEv2TransportCfg(pmtu=RoCEv2Mtu.MTU_4096, minRnrTimer=12, rnrRetry=7, retryCount=3)

Transport / QP-tuning configuration for the RoCEv2 RC connection.

Symmetric companion to RoCEv2ServerCfg — that one carries the host-NIC config, this one carries the four transport/QP-tuning knobs that are NOT held by either the stateless surf engine or the host-side server. A single instance is forwarded by the orchestrating Root into BOTH engine.setupConnection(...) and server.completeConnection(...) so the FPGA and host sides stay in sync on pmtu/minRnrTimer. Note that rnrRetry/retryCount are applied FPGA-side only — the host server accepts them for call-site symmetry but does not program them into the host QP (see RoCEv2Server.completeConnection()), so those two knobs govern only the FPGA’s retry behavior.

Defaults match the proven acceptance-gate config in rocev2PrbsTest.py: the same invariants that RoCEv2Server.completeConnection() enforces are mirrored here as __post_init__ validation so a bad cfg fails loudly at construction.

Parameters:

• pmtu (int, default: <RoCEv2Mtu.MTU_4096: 5>) – Path MTU code (a RoCEv2Mtu value). Defaults to RoCEv2Mtu.MTU_4096 (5).

• minRnrTimer (int, default: 12) – IB Min RNR NAK timer code 0..31 (a RoCEv2RnrTimer value). Defaults to 12 — the documented measured clean-validation value.

• rnrRetry (int, default: 7) – RNR retry count. Defaults to 7.

• retryCount (int, default: 3) – Transport retry count. Defaults to 3.

class pyrogue.protocols.RoCEv2Mtu(*values)

Path MTU codes (libibverbs ibv_mtu enum).

MTU_1024 = 3

MTU_2048 = 4

MTU_256 = 1

MTU_4096 = 5

MTU_512 = 2

class pyrogue.protocols.RoCEv2RnrTimer(*values)

IB-spec Min RNR NAK Timer Field codes (IBA Vol1 Table 45). Member name encodes the RNR wait in milliseconds; code 0 is the special 655.36ms slot.

MS_0_01 = 1

MS_0_02 = 2

MS_0_03 = 3

MS_0_04 = 4

MS_0_06 = 5

MS_0_08 = 6

MS_0_12 = 7

MS_0_16 = 8

MS_0_24 = 9

MS_0_32 = 10

MS_0_48 = 11

MS_0_64 = 12

MS_0_96 = 13

MS_10_24 = 20

MS_122_88 = 27

MS_15_36 = 21

MS_163_84 = 28

MS_1_28 = 14

MS_1_92 = 15

MS_20_48 = 22

MS_245_76 = 29

MS_2_56 = 16

MS_30_72 = 23

MS_327_68 = 30

MS_3_84 = 17

MS_40_96 = 24

MS_491_52 = 31

MS_5_12 = 18

MS_61_44 = 25

MS_655_36 = 0

MS_7_68 = 19

MS_81_92 = 26