Classes
struct	KernelValue
	A value in a block under construction: an access node with the subset and base type of the memlet reading it. More...
class	BlockBuilder
	Builds the dataflow of one block from expressions. More...
class	KernelBuilder
	Builds loop nests over transient buffers into a sequence. More...
struct	RowMajorB
	The copy read_b_row_major() makes of B, and the contraction that reads it. More...
class	OzakiMatMulNode
	Y = A @ B with B given as its Ozaki slices, prepared once for many products, e.g. More...
class	OzakiMatMulNodeSerializer
class	OzakiMatMulExpander
struct	ContractionPrecision
	How contractions (MatMul and GEMM nodes) of Float and Double compute, as users select it. More...
class	ContractionPrecisionPass
	Applies the transformation `precision` selects to every contraction it can; others stay native. More...
class	FPContraction
	Floating-point contraction (-ffp-contract=fast): applies FuseFMA in every block until no multiplication feeding an addition is left to contract. More...
class	FuseFMA
	Contracts a floating-point multiplication feeding an addition into a fused multiply-add. More...
class	OzakiContraction
	Emulates a floating-point matmul (MatMul or GEMM node) with exact Int8 matmuls (Ozaki scheme), at about the input's precision. More...
class	QuantizeContraction
	Computes a floating-point matmul (MatMul or GEMM node) on integers, quantized with a scale per row of A and per column of B. More...
class	UnfuseFMA
	Splits a fused multiply-add into a multiplication and an addition. More...

Typedefs
typedef passes::VisitorPass< FPContraction >	FPContractionPass

Enumerations
enum class	OzakiVariant { Slicing , Modular }
	How the Ozaki scheme represents the operands in Int8. More...
enum class	Rescaling { Exponents , Direct }
	How QuantizeContraction applies the scales of A's rows and B's columns to the Int32 products. More...

Functions
bool	b_stored_transposed (const data_flow::LibraryNode &node)
	Whether the k x n B of the MatMul or GEMM `node` is stored transposed, its rows along its columns' elements.
RowMajorB	read_b_row_major (builder::StructuredSDFGBuilder &builder, data_flow::LibraryNode &node)
	Lets the MatMul or GEMM `node`, whose k x n B is stored transposed, read a row-major copy of B made by a TensorCopy node in a block before its own; each matrix of a batch is transposed on its own.
data_flow::LibraryNodeCode	LibraryNodeType_OzakiMatMul ("fp::OzakiMatMul")
bool	expand_ozaki_matmuls (builder::StructuredSDFGBuilder &builder, analysis::AnalysisManager &analysis_manager)
	Expands every OzakiMatMul node of the SDFG into the slicing of A, the Int8 MatMuls and the rescaling; whether any.
void	register_fp_math_plugin (plugins::Context &context)

Typedef Documentation

◆ FPContractionPass

typedef passes::VisitorPass<FPContraction> sdfg::fp_math::FPContractionPass

Enumeration Type Documentation

◆ OzakiVariant

enum class sdfg::fp_math::OzakiVariant

strong

How the Ozaki scheme represents the operands in Int8.

Enumerator
Slicing	Ozaki-I: split each operand into 7-bit slices and sum the s(s+1)/2 leading products of slices.
Modular	Ozaki-II: scale the operands to integers, multiply their residues modulo coprime moduli and reconstruct the product with the Chinese remainder theorem; one product per modulus.

◆ Rescaling

enum class sdfg::fp_math::Rescaling

strong

How QuantizeContraction applies the scales of A's rows and B's columns to the Int32 products.

Enumerator
Exponents	Through the scales' power-of-two exponents in one step, so that no intermediate overflows where the result does not (needs floor and exact powers of two).
Direct	Multiplied by s_A[i] / q and then s_B[j] / q, which needs nothing beyond multiplication but overflows for a row whose magnitude lies within a factor K q of the type's largest value, about 10^33 for Float and K = 768.

Function Documentation

◆ b_stored_transposed()

bool sdfg::fp_math::b_stored_transposed ( const data_flow::LibraryNode & node )

Whether the k x n B of the MatMul or GEMM node is stored transposed, its rows along its columns' elements.

◆ expand_ozaki_matmuls()

bool sdfg::fp_math::expand_ozaki_matmuls	(	builder::StructuredSDFGBuilder &	builder,
		analysis::AnalysisManager &	analysis_manager )

Expands every OzakiMatMul node of the SDFG into the slicing of A, the Int8 MatMuls and the rescaling; whether any.

◆ LibraryNodeType_OzakiMatMul()

data_flow::LibraryNodeCode sdfg::fp_math::LibraryNodeType_OzakiMatMul ( "fp::OzakiMatMul" )

inline

◆ read_b_row_major()

RowMajorB sdfg::fp_math::read_b_row_major	(	builder::StructuredSDFGBuilder &	builder,
		data_flow::LibraryNode &	node )

Lets the MatMul or GEMM node, whose k x n B is stored transposed, read a row-major copy of B made by a TensorCopy node in a block before its own; each matrix of a batch is transposed on its own.

The contraction is a new node for a GEMM, which replaces node.

Scales of B's columns, as the direct rescaling computes them, then reduce each column along the copy's rows, and the slices or quantized B are row-major as well.

◆ register_fp_math_plugin()

void sdfg::fp_math::register_fp_math_plugin ( plugins::Context & context )