Raptor/validation/operations

Operation Validation Suite

This directory contains the ONNX models used by validation/validate.py to validate individual operations through compilation, PIM simulation, and comparison with the ONNX-MLIR reference runtime.

Naming

Every model uses the same path convention:

<category>/<case>/<category>_<case>.onnx

The <category>/<case> pair is the operation ID printed by the validator. Use lowercase snake_case for both components. Keep the generator function, graph name, directory, and filename based on the same operation ID when adding a test.

Generate and validate

Regenerate all generated models from the repository root:

.venv/bin/python validation/operations/gen_tests.py

Run the complete suite with deadlock detection:

.venv/bin/python validation/validate.py \
  --raptor-path build_release/Release/bin/onnx-mlir \
  --onnx-include-dir onnx-mlir/include \
  --operations-dir validation/operations \
  --raptor-extra-arg=--pim-detect-communication-deadlock \
  --raptor-extra-arg=--pim-export-spatial-dataflow=none

Use --compile-only for compiler and deadlock checks, then --run-only to reuse those artifacts for reference execution, simulation, and comparison. The validator prints separate latency and throughput operation tables before its summary and writes all of their rows to validation_results.csv.

Complete inventory

The suite contains 177 models. Tensor shapes, attributes, and constants are defined in gen_tests.py and in the checked-in ONNX models.

Add (5)

Case	Description
after_gemm	Gemm followed by Add with a broadcast bias vector.
basic	Elementwise Add on two inputs with identical shapes.
broadcast_row	Elementwise Add with row-vector broadcasting.
channel_broadcast_1024	NCHW per-channel broadcasting over 1024 channels.
leading_dimension_broadcast	Trailing-dimension broadcasting across leading dimensions.

Concat (3)

Case	Description
channel_axis	Concatenates two runtime NCHW tensors along the channel axis.
negative_axis	Concatenates tensors using a negative axis.
three_inputs_channel_axis	Concatenates three runtime NCHW tensors along the channel axis.

Conv (42)

Case	Description
batch_2	Batched Conv with SAME_UPPER padding and bias.
batch_4_pointwise	Pointwise Conv with batch size four.
input_224_7x7_stride2	224x224 RGB Conv with 64 output channels, a 7x7 kernel, stride two, and bias.
depthwise_1024_channels	Depthwise pointwise Conv with 1024 groups.
depthwise_grouped	Depthwise-style grouped Conv with one input channel per group.
dilated_3x3	Conv with a dilated 3x3 kernel.
dynamic	Conv with runtime input and weight tensors.
explicit_padding	3x3 Conv with symmetric explicit padding.
grouped_many_groups	Pointwise Conv with many groups and high channel counts.
grouped_two_groups	Two-group pointwise Conv with bias.
huge_pointwise_1024	Pointwise Conv with 1024 input and output channels.
huge_pointwise_1024_dynamic	The 1024-channel pointwise Conv with runtime weights.
kernel_3x3	Basic 3x3 Conv without padding.
kernel_equals_input_spatial	Conv whose kernel covers the full spatial input.
large_input_channels_1x1	Pointwise Conv with 1024 input channels and modest output width.
large_output_channels_1x1	Pointwise Conv with modest input width and 1024 output channels.
large_spatial	3x3 Conv on a larger spatial input.
multi_channel	3x3 Conv with multiple input and output channels.
non_square_kernel_1x3	Conv with a non-square 1x3 kernel.
non_square_kernel_3x1	Conv with a non-square 3x1 kernel.
non_uniform_stride	Conv with different height and width strides.
output_channel_grouping_minimal	Minimal 64-to-256 pointwise Conv for output-channel grouping.
pointwise_1x1	Basic pointwise channel-mixing Conv.
pointwise_tiled_chain	Relu and chained pointwise Convs with a tiled intermediate.
real_asymmetric_padding	Conv with asymmetric explicit padding.
relu_conv_store	Conv, Relu, and a second Conv to validate an intermediate stored result.
same_lower_3x3	3x3 Conv with SAME_LOWER padding.
same_padding_3x3	3x3 Conv with SAME_UPPER padding.
kernel_2x2	Hand-authored Conv with a 2x2 kernel.
stride_2	3x3 Conv with stride two.
strategy_depthwise_16	16-channel depthwise 3x3 Conv for depthwise lowering.
strategy_input_k_tiled	32-channel 3x3 Conv sized to exercise input-K tiling.
strategy_output_channel_tiled	8-to-192 3x3 Conv sized to exercise output-channel tiling.
strategy_streamed_packed	3-to-16 3x3 Conv on 128x128 input for streamed packed lowering.
strategy_streamed_patch	3-to-16 3x3 Conv on 64x64 input for streamed patch lowering.
strategy_tiled_2d	32-to-192 3x3 Conv sized to exercise 2D tiling.
with_bias_3x3	Multi-channel 3x3 Conv with bias.
with_constant	Hand-authored SAME_UPPER Conv with constant weight and bias.
without_kernel_shape_attr	Conv whose kernel shape is inferred from its weight tensor.
yolo11n_depthwise_head	YOLO11n pointwise-to-depthwise head boundary at 80x80, preserving row fragments.
yolo11n_heavy	Two largest standard YOLO11n Conv-SiLU blocks by MAC count at 64x80x80.
yolo11n_stem	First two YOLO11n Conv-SiLU blocks at 640x640, including the distributed activation boundary.

Div (6)

Case	Description
after_gemm	Gemm followed by Div with a broadcast divisor vector.
basic	Elementwise Div by a same-shape constant tensor.
channel_broadcast_1024	Div with NCHW per-channel broadcasting over 1024 channels.
leading_dimension_broadcast	Div with trailing-dimension broadcasting.
runtime_scalar_rhs	Div of a runtime tensor by a scalar initializer.
scalar_constant	Div with scalar broadcasting on a 2D tensor.

Gather (5)

Case	Description
3d_input_axis1	Gathers along axis one of a 3D input.
axis0_matrix_indices	Gathers rows using a 2D indices tensor.
axis1	Gathers selected columns from a 2D tensor.
negative_axis	Gathers using a negative axis.
negative_indices	Gathers with negative indices along axis zero.

Gemm (21)

Case	Description
alpha_beta	Applies non-default alpha and beta scaling with bias.
bias_rank2_broadcast	Broadcasts a rank-2 bias across output rows.
dynamic	Uses both matrix operands at runtime.
dynamic_alpha	Uses runtime operands with non-default alpha scaling.
dynamic_beta	Uses runtime operands and bias with non-default beta scaling.
dynamic_bias	Uses runtime matrix operands and runtime bias.
dynamic_bias_alpha_beta	Combines runtime operands and bias with alpha and beta scaling.
dynamic_transpose_b	Transposes a runtime right-hand matrix.
huge_1024	Uses 1024-wide inner and output dimensions.
large	Exercises larger rectangular matrices.
large_k_small_n	Uses a large reduction dimension and narrow output.
non_square	Uses different reduction and output widths.
scalar_bias	Broadcasts a scalar bias to the full output.
square_weights	Basic Gemm with square weights.
small	Tiny Gemm for fast focused validation.
small_k_large_n	Uses a modest reduction dimension and wide output.
transpose_a	Transposes the left-hand matrix.
transpose_a_and_b	Transposes both matrix operands.
transpose_b	Transposes the right-hand weight matrix.
transpose_b_with_bias	Combines a transposed weight matrix with bias.
with_bias	Basic matrix product with vector bias.

Gemv (5)

Case	Description
all_constant	Vector-matrix product with all inputs constant.
constant_weight	Basic single-row vector-matrix product with constant weights.
non_uniform_bias	Adds a non-uniform constant bias pattern.
uniform_bias	Adds a uniform constant bias pattern.
scalar_bias	Adds a scalar broadcast bias.

MatMul (12)

Case	Description
basic	Direct 2D MatMul with constant right-hand matrix.
batched_3d	Batched 3D MatMul with matching batch dimensions.
batched_3d_dynamic	Batched 3D MatMul with both operands at runtime.
batched_left_constant	Batched 3D MatMul with constant left-hand matrix.
batched_lhs_broadcast	Broadcasts a 2D left-hand matrix across a batched right-hand tensor.
batched_rhs_broadcast	Broadcasts a 2D right-hand matrix across a batched left-hand tensor.
dynamic	Direct 2D MatMul with both operands at runtime.
huge_1024	Uses 1024-wide inner and output dimensions.
left_constant	Direct 2D MatMul with constant left-hand matrix.
matrix_vector	Matrix-vector multiplication producing a 1D output.
vector_matrix	Vector-matrix multiplication producing a 1D output.
yolo_attention	YOLO11n rank-4 dynamic MatMul-scale-transpose-MatMul attention chain.

Mul (6)

Case	Description
after_conv	Conv followed by per-channel scaling.
after_conv_scalar_constant	Conv followed by Mul with a scalar constant.
basic	Elementwise Mul on two inputs with identical shapes.
channel_broadcast_1024	Mul with NCHW per-channel broadcasting over 1024 channels.
leading_dimension_broadcast	Mul with trailing-dimension broadcasting.
scalar_constant	Mul with scalar broadcasting.

Pool (15)

Case	Description
avg_basic	AveragePool with a 2x2 kernel and unit stride.
avg_ceil_mode	AveragePool with ceil mode enabled.
avg_explicit_padding	Explicitly padded AveragePool excluding pad from the divisor.
avg_include_pad	Explicitly padded AveragePool including pad in the divisor.
avg_large_channels	AveragePool with a large channel count and small spatial extent.
avg_non_uniform_stride	AveragePool with different height and width strides.
avg_real_asymmetric_padding	AveragePool with asymmetric explicit padding.
max_after_conv	Conv followed by MaxPool.
max_basic	MaxPool with a 2x2 kernel and unit stride.
max_ceil_mode	MaxPool with ceil mode enabled.
max_global_style_kernel_equals_input	MaxPool whose kernel covers the full spatial input.
max_non_square_kernel	MaxPool with a non-square kernel.
max_real_asymmetric_padding	MaxPool with asymmetric explicit padding.
max_same_upper	MaxPool with SAME_UPPER padding.
max_stride2_multichannel	Multi-channel MaxPool with stride two.

ReduceMean (17)

Case	Description
4d_spatial	Reduces height and width of an NCHW tensor while preserving rank.
4d_spatial_keepdims_0	Reduces NCHW height and width while dropping those axes.
after_conv	Conv followed by a spatial ReduceMean.
all_axes_keepdims_0	Reduces all axes to a scalar.
all_axes_keepdims_1	Reduces all axes while preserving rank.
basic	Reduces a feature dimension while preserving rank.
channel_axis_nchw	Reduces the channel axis of an NCHW tensor.
keepdims_0	Reduces a feature dimension and drops that axis.
large_dimension_1024	Reduces a dimension of length 1024.
legacy_axes_1_2_keepdims_1	Opset-18 reduction over multiple positive axes.
legacy_axis1_keepdims_0	Opset-18 reduction over one axis while dropping it.
legacy_axis1_keepdims_1	Opset-18 reduction over one axis while preserving rank.
legacy_empty_axes_noop	Opset-18 empty-axes no-op followed by Relu.
legacy_nchw_spatial	Opset-18 spatial reduction on NCHW input.
legacy_negative_axis	Opset-18 reduction using a negative axis.
legacy_reduce_all_keepdims_1	Opset-18 all-axis reduction with the axes input omitted.
negative_axis	ReduceMean using a negative axis.

Relu (4)

Case	Description
4d	Standalone Relu on an NCHW tensor.
after_conv	Conv followed by Relu.
after_gemm	Gemm followed by Relu.
basic	Standalone Relu on a 2D tensor.

Reshape (4)

Case	Description
4d_to_2d_flatten	Flattens a 4D tensor into a 2D view.
infer_dim_minus_one	Uses -1 to infer one output dimension.
same_rank	Changes shape without changing rank.
zero_copies_input_dim	Uses 0 to copy an input dimension.

Resize (6)

Case	Description
height_only	Nearest-neighbor resize of only the height dimension.
nearest_2x	Nearest-neighbor upsampling by a factor of two.
nearest_downsample	Nearest-neighbor downsampling.
non_uniform_scales	Nearest-neighbor resize with different spatial scales.
width_only	Nearest-neighbor resize of only the width dimension.
with_sizes	Resize using explicit output sizes instead of scales.

Sigmoid (3)

Case	Description
4d	Standalone Sigmoid on an NCHW tensor.
after_gemm	Gemm followed by Sigmoid.
basic	Standalone Sigmoid on a 2D tensor.

Slice (8)

Case	Description
2d_basic	Slices a 2D tensor with explicit axes and unit steps.
after_conv	Conv followed by a spatial crop.
default_axes	Omits axes and steps to use positional defaults.
large_channel_1024	Slices a channel range from a 1024-channel tensor.
nchw_spatial_crop	Crops the spatial axes of an NCHW tensor.
negative_axis	Slices using a negative axis.
negative_indices	Slices using negative indices.
step2	Slices using a positive step greater than one.

Softmax (5)

Case	Description
3d_last_axis	Softmax over the last axis of a 3D tensor.
basic	Softmax over the last dimension of a 2D tensor.
channel_axis	Softmax over the channel axis of an NCHW tensor.
large_dimension_1024	Softmax over a last dimension of length 1024.
negative_axis	Softmax using a negative axis.

Split (4)

Case	Description
basic	Splits a 2D tensor into two explicit output sizes.
equal_three_way	Splits a 2D tensor evenly into three outputs.
negative_axis	Splits using a negative axis.
uneven_channel_axis_4d	Splits an NCHW channel axis into uneven outputs.

Sub (6)

Case	Description
after_gemm	Gemm followed by Sub with a broadcast constant vector.
basic	Elementwise Sub on two runtime inputs with identical shapes.
broadcast_row	Sub with a broadcast row-vector right-hand constant.
channel_broadcast_1024	Sub with NCHW per-channel broadcasting over 1024 channels.
constant_lhs_broadcast	Sub with a broadcast constant left-hand operand.
leading_dimension_broadcast	Sub with trailing-dimension broadcasting.