Example of a texture artifact

Original

VTM-20.0,
208 times smaller

JPEG AI Reference,
tools on, hop,
204 times smaller

Key features

• 50,370 confirmed visual artifacts
• 3 artifact types: texture/boundary degradation, color distortion, text corruption
• 6 neural codecs (JPEG AI + 5 modern learned codecs)
• 350,000 source images processed
• Localized, axis-aligned bounding-box annotations
• Crowdsourced subjective validation (15 assessors per artifact)
• Includes original, neural-compressed, and traditional-compressed images

Artifact distribution

Note

This page provides access to the public part of the dataset (≈ 2/3 of all samples).

The remaining part is kept hidden and used exclusively for internal benchmarking and future evaluation of artifact-detection methods.
This split prevents overfitting of learning-based approaches and enables fair comparison of future methods on unseen data.

Downloads

Dataset

Methodology

The dataset was created to study visual artifacts introduced by learning-based image compression, with a primary focus on JPEG AI.

We processed approximately 350,000 images from the Open Images [1] dataset using JPEG AI. To verify that the proposed artifact-detection methods generalize beyond JPEG AI, we selected 3,000 source images and recompressed them using five additional modern neural codecs: Cheng 2020 (attention), EVC, ELIC, STF, and QRes-VAE.

For comparison, each image was also compressed using a traditional codec (HM-18.0) at a matched bitrate.

We then applied our own artifact-detection methods, described in detail in the paper, to identify and localize three common artifact types in neural image compression:

• texture / boundary degradation,
• color distortions,
• text corruption.

For each method, the output is a localized artifact instance, including:

• an artifact strength score (method-specific metric value)
• an axis-aligned bounding box of the affected region

We consider candidate regions where the neural-compressed image is more distorted than the traditional-compressed image at the same bitrate.

Subjective Validation

All artifact candidates were validated via crowdsourced subjective assessment using two platforms: Toloka and Yandex Tasks.

Each artifact was evaluated by 15 independent participants, and at least 12 positive votes were required to confirm its presence.
Separate validation protocols were used for each artifact type, with dedicated training and verification stages to ensure annotation quality.

Only artifacts that passed subjective validation are included in the dataset.

Overview of the dataset pipeline. Original images are compressed by neural and traditional codecs; our metrics detect candidate artifacts; candidates then undergo multi-stage crowdsourced validation (training, exam, verification questions), and only confirmed items are included in the final dataset.

Dataset Diversity

We analyze the diversity of the dataset using two aggregate measures: Range and Uniformity. The results demonstrate that the created dataset exhibits a high level of variability across diversity metrics, indicating its ability to cover a wide range of image content and artifact cases. This makes the dataset particularly valuable for thorough evaluation and for the development of artifact-detection methods for neural-based image compression.

Dataset diversity analysis. Distribution of dataset characteristics by Range (horizontal axis) and Uniformity (vertical axis). Each point corresponds to a dataset/subset; each subplot shows one source-image metric (Absolute Luminance, CF, Contrast, SI, SharpnessScore). Kodak, MSU LIC Benchmark, and SCLIC [2] are external reference benchmarks; the remaining points correspond to our subsets.

Annotation Format

Each dataset sample corresponds to a single artifact instance.

For every artifact we provide:

• the original image;
• the image compressed by a neural codec (with codec settings and bitrate);
• the image compressed by a traditional codec at a matched bitrate;
• the artifact type;
• a method-specific metric value;
• an axis-aligned bounding box in original-image coordinates;
• subjective voting results.

Citation

@misc{tsereh2024jpegai,
  author={Tsereh, Daria and Mirgaleev, Mark and Molodetskikh, Ivan and Kazantsev, Roman and Vatolin, Dmitriy},
  title={JPEG AI Image Compression Visual Artifacts: Detection Methods and Dataset},
  howpublished={arXiv:2411.06810 [cs.CV]},
  year={2024},
  url={https://arxiv.org/abs/2411.06810}
}

Reference

1. Kuznetsova, A., Rom, H., Alldrin, N., et al.: The Open Images Dataset V4. https://storage.googleapis.com/openimages/web/index.html

2. Hofer, N., Böhme, R.: Challenging cases of neural image compression: A dataset of visually compelling yet semantically incorrect reconstructions. In: Proceedings of the 33rd ACM International Conference on Multimedia, pp. 13318–13324 (2025)