MDMM hardware-aware metrics (FPGA + PACA), tests and docs by ArghyaRanjanDas · Pull Request #46 · cern-nextgen/PQuantML

ArghyaRanjanDas · 2026-06-25T00:04:39Z

Summary

Add FPGAAwareSparsity and PACAPattern MDMM metrics to both the Keras and Torch backends, ported onto dev's per-backend layout. Supersedes MDMM tests and hardware-aware metric functions #37, which was built on the old single pruning_methods/ tree and targeted main.
Fix the Pydantic serialization warnings reported on MDMM tests and hardware-aware metric functions #37 (the constraint_type enum and the discriminated pruning_parameters union). Root cause: the MDMM enum fields defaulted to bare strings, and Pydantic does not validate/coerce defaults, so the runtime value stayed a str. They now default to enum members.
Move metric validation into MDMMPruningModel (ge/le + Literal) and drop the per-metric asserts. PACA auto-forces Equality/target 0 via a model_validator, so the MDMM layer stays metric-agnostic.
Run pattern selection on-graph (keras.ops bit-pack + bincount, native torch.unique, no NumPy round-trip of the weight); centralise distance/layout/resource constants in core/constants.py; fix the get_layer_sparsity float/int dtype bug on the TF backend.
Add config_mdmm_fpga.yaml, config_mdmm_paca.yaml, the mdmm_fpga_config() / mdmm_paca_config() helpers, and document the new metric types in reference.md and getting_started.md.

Test plan

cd tests && pytest test_mdmm.py test_mdmm_metrics.py (Keras / TensorFlow)
cd tests && KERAS_BACKEND=torch pytest test_mdmm.py test_mdmm_metrics.py (Keras / Torch)
cd tests && KERAS_BACKEND=torch pytest test_torch_mdmm.py (native Torch)
Load config_mdmm_fpga.yaml and config_mdmm_paca.yaml, call model_dump(), confirm no PydanticSerializationUnexpectedValue warnings.
Existing methods unaffected: pytest test_pdp.py test_ap.py.

Notes for review

Most of the MDMM tests and hardware-aware metric functions #37 inline comments are already satisfied by dev's restructure: metric_type is read off the config and resolved via the metric registry, and the MetricType / ConstraintType enums already exist. This PR keeps that flat-params + registry approach rather than nested metric models.
The patterns.py no-op OIHW->OIHW permute is replaced with real per-backend layout handling (Keras kernels are HWIO, Torch OIHW), and the convert_to_numpy pattern step is gone.
The shipped FPGA config uses Equality/target 0; happy to change the default constraint if you prefer a different target for structured group-sparsity.

Add the two hardware-aware sparsity metrics for MDMM to both the Keras and Torch backends, on dev's per-backend layout: - FPGAAwareSparsityMetric: fraction of zero DSP/BRAM weight groups, modelling FPGA resource packing (rf / precision / bram_width). - PACAPatternMetric: drives conv kernels onto a small set of dominant binary patterns. Pattern selection stays on-graph (keras.ops bit-pack + bincount / native torch.unique, no NumPy round-trip of the weight) and exposes a projection mask applied during fine-tuning through a generic, metric-agnostic hook in MDMM. Data model (MDMMPruningModel): - New metric_type values + flat Optional fields (precision, target_resource, bram_width, num_patterns_to_keep, beta, distance_metric) validated by Pydantic (ge/le + Literal); a model_validator forces PACA -> Equality / target 0. - Fix the reported Pydantic serialization warnings (constraint_type 'Equality' and the discriminated pruning_parameters union): enum fields now default to enum members instead of bare strings, which Pydantic does not coerce on defaults. Also: distance/layout/resource constants centralised in core/constants.py, the get_layer_sparsity float/int dtype fix on the TF backend, and config_mdmm_fpga.yaml + config_mdmm_paca.yaml with mdmm_fpga_config() / mdmm_paca_config() helpers.

- test_mdmm.py / test_mdmm_metrics.py: Keras MDMM, FPGA DSP/BRAM math, PACA pattern selection + projection masking, full phase cycles, Pydantic validation, and a regression test that MDMM configs serialize without Pydantic warnings. - test_torch_mdmm.py: native-torch MDMM + metrics. - run_tests.sh: run the new files under both tensorflow and torch backends.

- reference.md: MDMM metric-type overview + FPGAAwareSparsity and PACAPatternSparsity parameter tables. - getting_started.md: constraint-based MDMM usage with the hardware-aware configs.

nastiapetrovych · 2026-07-09T12:48:07Z

Add these tests to run_tests.sh

pytest test_mdmm.py test_mdmm_metrics.py
KERAS_BACKEND=torch pytest test_mdmm.py test_mdmm_metrics.py
KERAS_BACKEND=torch pytest test_torch_mdmm.py

nastiapetrovych · 2026-07-09T15:53:22Z

Hi @ArghyaRanjanDas, thank you for your updated PR.

I reviewed it successfully and left you couple of comments on what should be changed before we merge it to the dev branch. Let me know if you have any questions regarding it.

nastiapetrovych · 2026-07-01T16:45:43Z

+  damping: 1.0
+  use_grad: false
+  constraint_lr: 1.0e-3
+  # PACAPatternMetric-specific


Clarification about PACAPatternMetric-specific values; are these values result in the best results?

They were the values (hand-tuned) that worked best for the SmartPixels. Do you think it would be better to scan for the best hyperparameters?

If Yes, Kindly let me know if you also have any dataset in mind, that you would like this to be tuned to.

I would say, that in this situation it would be nice just to leave the best values based on your point of view. If they are the same, leave them as it's.

nastiapetrovych · 2026-07-03T14:38:22Z

+
+def convert_conv_layout(w, src, dst=CANONICAL_CONV_LAYOUT):
+    """Transpose a 4D conv weight from `src` to `dst` layout (no-op if already equal)."""
+    if src == dst:


why not to make just one condition:
if src =! dst or perm != (0, 1, 2, 3):
and return w in the end instead?

Yeah sure! So something like this?

def convert_conv_layout(w, src, dst=CANONICAL_CONV_LAYOUT): perm = _perm(src, dst) if src != dst or perm != (0, 1, 2, 3): return ops.transpose(w, perm) return w

yes, perfect:)

nastiapetrovych · 2026-07-03T14:41:36Z

+
+def _kernel_pattern_distances(kernel_patterns, kernels, dominant_patterns, distance_metric):
+    """Distance from every kernel to every dominant pattern. -> (M_kernels, alpha)."""
+    tk = ops.cast(kernel_patterns, kernels.dtype)          # (M, K) binary support of kernels


these namings are slightly confusing

Ohh okay :)
I will name them something better.

thank you a lot!

nastiapetrovych · 2026-07-03T14:43:40Z

+        dot = ops.sum(k_e * projected, axis=-1)
+        denom = ops.norm(k_e, axis=-1) * ops.norm(projected, axis=-1) + keras.backend.epsilon()
+        return 1.0 - dot / denom
+    raise ValueError(f"Unsupported distance metric: {distance_metric!r}")


there is no need in value error, as pydantic will raise it:
distance_metric: Optional[Literal["hamming", "valued_hamming", "cosine"]] = Field(default=None)

nastiapetrovych · 2026-07-03T14:47:30Z

 _METRIC_REGISTRY = {
    "UnstructuredSparsity": UnstructuredSparsityMetric,
    "StructuredSparsity": StructuredSparsityMetric,
+    "FPGAAwareSparsity": FPGAAwareSparsityMetric,


as torch and keras mdmm implementations share the same constraints, metric registry and functions ( _compute_hard_mask), why not to reorganise it in the way that modules reuse that shared code?

I agree with the principle, the backend is split into two. But I think all the other method also does that, and to unify them into one keras backend that would require a considerable amount of repo-wide refactoring?

I think we should really post-pone this large refactoring, unless we decide to start moving everything to keras.

Makes sense for now

nastiapetrovych · 2026-07-09T13:08:02Z

+PACA_DISTANCE_METRICS = (DISTANCE_HAMMING, DISTANCE_VALUED_HAMMING, DISTANCE_COSINE)
+
+# FPGAAwareSparsityMetric target hardware resources
+TARGET_RESOURCE_DSP = "DSP"


same here, why not to make FPGA_TARGET_RESOURCES_REGISTRY, etc?

nastiapetrovych · 2026-07-09T13:11:07Z

@@ -67,6 +67,8 @@ class ActivationPruningModel(BasePruningModel):
 class MetricType(str, Enum):


It can be as a general metric class, and these subclasses inherit from it and reimplement some methods

That also sounds great! Implementing that

nastiapetrovych · 2026-07-09T13:12:17Z

+        return zero_bram.sum() / float(bram_norms.numel())
+
+
+class PACAPatternMetric:


why these metric classes aren't Pydantic?

This are compute objects as the metric holds live tensor and bound method. So I think we would have to allow arbitrary_types_allowed=True if we want to move to Pydantic, which I think would not be good code policy.

Also during config parsing, Pydantic is already validating by the time MDMM.build constructs a metric, every value has passed the Literal/ge/le checks in MDMMPruningModel. So validating again inside the class would be double validation.

Actually, if we do that per-metric config sub-models from the other thread, there will be effectively a Pydantic class per metric where it owns and validates the parameters. And the runtime class will just then consume those validated values.

Kindly let me know how that sounds to you or if I am missing anything

I think it wouldn't be a problem if we make a good module structure planning -> we can discuss it via meet

nastiapetrovych · 2026-07-09T13:21:12Z

@@ -0,0 +1,72 @@
+# file: /src/pquant/configs/config_mdmm_fpga.yaml


too much redundant parameters in config files

Sure Happy to trim them. these two were clones from the config_mdmm.yaml to stay consistent and trimming them might make it a bit confusing for the user (ommited = default or ommited = forgotten). Do you think we should trim just these two or all of them? the minimal versions should be ~15 lines

Maybe we can make separate variation of files, including different mdmm parameters for different settings? All others blocks (training, hpo, quantization) should stay the same within the files.

nastiapetrovych · 2026-07-09T13:21:31Z

+pruning_parameters:
+  pruning_method: mdmm
+  enable_pruning: true
+  disable_pruning_for_layers: [] # Disable pruning for these layers, even if enable_pruning is true


same, too much redundant parameters

Same as above

ArghyaRanjanDas · 2026-07-24T01:38:10Z

Add these tests to run_tests.sh

pytest test_mdmm.py test_mdmm_metrics.py
KERAS_BACKEND=torch pytest test_mdmm.py test_mdmm_metrics.py
KERAS_BACKEND=torch pytest test_torch_mdmm.py

Hi, you might have missed it, they are already added, lines 7–9 of tests/run_tests.sh‎

nastiapetrovych · 2026-07-24T13:09:06Z

Thanks @ArghyaRanjanDas for these modifications.

Yes, I have missed run_test.sh changes, thanks for pointing them out. If you need to discuss any of these comments, feel free to reach out to me:)

ArghyaRanjanDas added 3 commits June 24, 2026 20:52

docs: document MDMM metric types (FPGA / PACA)

020c77f

- reference.md: MDMM metric-type overview + FPGAAwareSparsity and PACAPatternSparsity parameter tables. - getting_started.md: constraint-based MDMM usage with the hardware-aware configs.

nastiapetrovych assigned ArghyaRanjanDas Jun 30, 2026

nastiapetrovych self-requested a review June 30, 2026 15:20

nastiapetrovych added feature new feature added fix labels Jun 30, 2026

nastiapetrovych requested changes Jul 9, 2026

View reviewed changes

nastiapetrovych mentioned this pull request Jul 9, 2026

MDMM tests and hardware-aware metric functions #37

Closed

		@@ -67,6 +67,8 @@ class ActivationPruningModel(BasePruningModel):
		class MetricType(str, Enum):

		return zero_bram.sum() / float(bram_norms.numel())


		class PACAPatternMetric:

		@@ -0,0 +1,72 @@
		# file: /src/pquant/configs/config_mdmm_fpga.yaml

Uh oh!

Conversation

ArghyaRanjanDas commented Jun 25, 2026

Summary

Test plan

Notes for review

Uh oh!

nastiapetrovych commented Jul 9, 2026

Uh oh!

nastiapetrovych commented Jul 9, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ArghyaRanjanDas Jul 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ArghyaRanjanDas Jul 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ArghyaRanjanDas Jul 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ArghyaRanjanDas commented Jul 24, 2026

Uh oh!

nastiapetrovych commented Jul 24, 2026

Uh oh!

Reviewers

ArghyaRanjanDas Jul 24, 2026 •

edited

Loading

ArghyaRanjanDas Jul 24, 2026 •

edited

Loading

ArghyaRanjanDas Jul 24, 2026 •

edited

Loading