GraNatPy - Gradient Naturalization for Image Enhancement and Evaluation

This repository is a collection of metrics to evaluate procedurally generated image datasets.

Gradient Naturalization

Notabily, we include a Python implementation of the paper:

Y. Gong and I. F. Sbalzarini. A natural-scene gradient distribution prior and its application in light-microscopy image processing. IEEE Journal of Selected Topics in Signal Processing, 10(1):99–114, 2016.

The original code can be found here.

In short, Naturalness is a gradient-based metric that can be computed for an arbitrary image; Naturalization is the method that can enhance images based on this metric.

The Naturalness metric reflects the similarity in the gradient distribution of a sample image to a gradient distribution learned from large image datasets. The closer the Naturalness is to 1, the closer the image is to what we perceive as natural. This is useful for evaluating the perceived quality of synthetically created images.

From their abstract:

[...] We argue here that the gradient distribution of natural-scene images may provide a versatile and well-founded prior for light-microscopy images that does not impose assumptions about the geometry of the ground-truth signal, but only about its gradient spectrum. We provide motivation for this choice from different points of view, and we illustrate the resulting regularizer for use on light-microscopy images. We provide a simple parametric model for the resulting prior, leading to efficiently solvable variational problems. We demonstrate the use of these models and solvers in a variety of common image-processing tasks, including contrast enhancement, noise-level estimation, denoising, blind deconvolution, and dehazing. [...]

The GraNatPy package contains the following classes and functions:

• GraNat: convenience class wrapping the full pipeline: loading, naturalization, visualization and saving
• naturalize_rgb_image: core function to naturalize an RGB or grayscale image
• naturalize_single_channel: core function to naturalize a single image channel
• compute_all_metrics: compute and print the dNf (delta Naturalness factor), MSE, PSNR, SSIM and NRMSE between two images
• compare_images: compare a reference image against a set of similar (e.g., synthetic) images and print a summary
• overlay_naturalness_heatmap: visualize per-region naturalness as a heatmap overlay on the original image

Installation

The simplest way is to install the package via pip:

pip install granatpy

Alternatively, clone the repository and install in editable mode:

git clone https://github.com/casus/GraNatPy
cd granatpy
pip install -e .

Quick Start

from granatpy import GraNat

# Load, naturalize and save
GraNat.load_image("photo.tif").naturalize(show=True).save("output.png")

# Visualize naturalness heatmap
from granatpy import overlay_naturalness_heatmap
import imageio.v3 as imageio
import numpy as np

img = imageio.imread("photo.tif")
fig, ax, heatmap = overlay_naturalness_heatmap(img, grid_rows=20, grid_cols=30)

# Compare images
from granatpy import compare_images

images = {
    "photo_low.tif",
    "photo_high.tif",
}
real = "photo_real.tif"
results = compare_images(real, images)

You can also find an example notebook in example/example.ipynb

Metrics

We include several metrics for measuring various dataset properties and to compare real and synthetic image pairs; they are saved in metrics.py.

from granatpy import compare_images to compare a real and synthetic image, passed as Numpy arrays. This returns PSNR, MSE, SSIM, NRMSE, Naturalness factors for both images, and dNf as the absolute difference between the two.

You can calculate the Shannon entropies of images or datasets with image_entropy and dataset_entropy, respectively.

LPIPS -- Comparing image pairs

To compare the perceptual similarity of image pairs, we provide two options:

1. Torchmetrics Implementation (Recommended)

This uses the standard torchmetrics library, which is fully compatible with PyPI package installations and modern PyTorch.

from granatpy.metrics import compute_lpips_torch

compute_lpips_torch("./real.tif", "./synthetic.tif")

2. Original richzhang/PerceptualSimilarity Implementation (Paper Version)

If you require the exact implementation used in our paper (based on richzhang/PerceptualSimilarity), you can use:

from granatpy.metrics import compute_lpips

compute_lpips("./real.tif", "./synthetic.tif")

Note: The first call to compute_lpips will automatically attempt to install the specific version of lpips from the official GitHub repository (richzhang/PerceptualSimilarity@082bb24) using pip.

For more usage information, refer to: github.com/richzhang/PerceptualSimilarity

FID -- Comparing datasets

The Fréchet Increption Distance (FID) is useful to assess the quality of generated datasets by calculating the distance between two latent feature distributions; typically a synthetic and a real dataset. To calculate the FID, use the torch-fidelity package included with our dependencies and call it via the command-line as follows:

fidelity --gpu 0 --fid --input1 path/to/synthetic/data/folder --input2 path/to/real/data/folder --samples-find-deep --samples-find-ext tif --kid-subset-size 100 --rng_seed 420

--samples-find-ext: allows you to query for specific data types, we use TIF files.
--rng_seed: allows you to create reproducible results.

For more usage information, see: github.com/toshas/torch-fidelity.

License

This project is licensed under the GPL-3.0 License. See LICENSE for details.

For attribution of example images, see ATTRIBUTION.md.