MSU Video Colorization Benchmark
Explore the best video colorization algorithms!
Our benchmark evaluates the best video colorization methods for color propagation and automatic colorization.
Everyone is welcome to participate! Apply your video colorization method to our dataset and submit the results to see how well it performs. Check the Submitting section to learn the details.
Over 2000 People
have participated in the verified
pairwise subjective comparison
36 High-Resolution Test Clips
with diverse complexity of spatial
and temporal information
11 Methods Tested
including video color propagation
and automatic video colorization
7 Metrics
and their correlations
for detailed comparison
Structural Distortion Maps
for explicit artifacts detection
Speed/Quality Scatter Plots
and tables with metrics
for a comprehensive comparison
What’s New
- July, 2024: Benchmark Release
Introduction
For a comprehensive assessment of the video colorization task, we used various objective metrics and subjective comparison. Over 2000 valid participants have selected the most visually appealing colorization result in a number of pairwise comparisons. In addition, we calculated the FPS (frames per second) to compare the speed of both algorithms and metrics.
Check the Methodology section to learn the details.
Scroll below for comparison charts, tables, and interactive visual comparisons of video colorization models result.
Leaderboards
† denote color propagation with 1 anchor frame
‡ denote color propagation with 2 anchor frames
* denote automatic colorization
Charts
Visualizations
Submitting
To add your video colorization method to the benchmark, follow these steps:
1. Download the dataset |
2. Apply your video colorization method to the dataset |
3. Send to video-colorization-benchmark@videoprocessing.ai
|
If you have any suggestions or questions, please contact us: video-colorization-benchmark@videoprocessing.ai
Get Notifications About the Updates of This Benchmark
Do you want to be the first to discover the best new video colorization algorithm? We can notify you about this benchmark’s updates: simply submit your preferred email address using the form below. We promise not to send you unrelated information.
Further Reading
Check the “Methodology” section to learn how we prepare our dataset.
Check the “Metrics” section to learn more about metrics and their correlations.
Check the “Participants” section to learn which video colorization method implementations we use.
MSU Video Quality Measurement Tool
Widest Range of Metrics & Formats
- Modern & Classical Metrics SSIM, MS-SSIM, PSNR, VMAF and 10+ more
- Non-reference analysis & video characteristics
Blurring, Blocking, Noise, Scene change detection, NIQE and more
Fastest Video Quality Measurement
- GPU support
Up to 11.7x faster calculation of metrics with GPU - Real-time measure
- Unlimited file size
Main MSU VQMT page on compression.ru
![](/assets/img/benchmarks/frm/subjectify.png)
Crowd-sourced subjective
quality evaluation platform
- Conduct comparison of video codecs and/or encoding parameters
What is it?
Subjectify.us is a web platform for conducting fast crowd-sourced subjective comparisons.
The service is designed for the comparison of images, video, and sound processing methods.
Main features
- Pairwise comparison
- Detailed report
- Providing all of the raw data
- Filtering out answers from cheating respondents
Subjectify.us
-
MSU Benchmark Collection
- Super-Resolution for Video Compression Benchmark
- Video Colorization Benchmark
- Defenses for Image Quality Metrics Benchmark
- Learning-Based Image Compression Benchmark
- Super-Resolution Quality Metrics Benchmark
- Video Saliency Prediction Benchmark
- Metrics Robustness Benchmark
- Video Upscalers Benchmark
- Video Deblurring Benchmark
- Video Frame Interpolation Benchmark
- HDR Video Reconstruction Benchmark
- No-Reference Video Quality Metrics Benchmark
- Full-Reference Video Quality Metrics Benchmark
- Video Alignment and Retrieval Benchmark
- Mobile Video Codecs Benchmark
- Video Super-Resolution Benchmark
- Shot Boundary Detection Benchmark
- The VideoMatting Project
- Video Completion
- Codecs Comparisons & Optimization
- VQMT
- MSU Datasets Collection
- Metrics Research
- Video Quality Measurement Tool 3D
- Video Filters
- Other Projects