MSU Video Quality Measurement Tool
- Projects, ideas: Dr. Dmitriy Vatolin
- Implementation: Alexey Moskvin, Oleg Petrov, Sergey Putilin, Sergey Grishin, Arsaev Marat, Georgy Osipov
Basic Information About VQMT
- What’s new in MSU VQMT 10?
- Brief Description
- Main Features
- Plots Examples
- Comparison with Analogs
- Contributors
- Testimonials
- Changelog
What’s new in MSU VQMT 10?
Download FREE non-commercial version with limitations or Buy PRO.
Special offer for recent customers:
50% off! Know
more…
- New look of main window
Brief Description
Demo Video: Basic Use Case Overview
MSU Video Quality Measurement Tool (VQMT) is a program for objective video quality assessment. It provides functionality for both full-reference (two videos are examined) and single-reference (one video is analyzed) comparisons. Basic scheme of the tool is illustrated on below.
VQMT input and output
Video quality is a characteristic of a video passed through a video transmission/processing system, a formal or informal measure of perceived video degradation (typically, compared to the original video). Video processing systems may introduce some amounts of distortion or artifacts in the video signal, so video quality evaluation is an important problem. MSU VQMT is a program for video quality measurements. This application allows you to create objective comparison of video codecs and perform video processing filters analysis.
Visualization examples for several metrics
Program allows to answer the next questions:
- One codec has more blurring than another. On what frames? Average mark? *
- One codec has more blocking than another. On what frames? Average mark? *
- One codec has lower quality than another. On what frames? Average mark in PSNR, VQM, SSIM? *
* Corresponding “bad frames” are saved automatically as BMP files.
Current stable version: 10.1 [Changelog]
Main Features
Feature | FREE version | PRO version | |
---|---|---|---|
Flexible Functionality
The tool provides functionality for calculating:
|
available in FREE version
|
available in PRO version
|
|
GPU and other computing devices
The tool supports performing some metrics on CUDA compatible devices (nVIDIA video cards) and all other OpenCL compatible devices (wide set of video cards, CPUs, multicore systems) |
available in FREE version
|
available in PRO version
|
|
Objective Metrics Support
|
available in FREE version
|
available in PRO version
|
|
Video and Image Formats Support
|
available in FREE version
|
available in PRO version
|
|
ROI Support (via masking)
Allows to calculate metrics in Regions of Interest (ROI). ROI should be specified by additonal provided video with mask for each frame. |
available in FREE version
|
available in PRO version
|
|
Metrics Visualization
Video visualization file is generated for combination [metric, color component, reference]. Color and luminance of each pixel in visuzalization video is defined by metric value in that pixel (very useful for codec and filter debugging). |
available in FREE version
|
available in PRO version
|
|
Results in CSV format
Metrics results and average values are saved in set of *.CSV (comma-separated values) file which can be opened everywhere. |
available in FREE version
|
available in PRO version
|
|
Results in JSON format
All metric results and information about file opening are saved in single JSON file or stdout of command line utility. |
available in FREE version
|
available in PRO version
|
|
Usage in companies
Free version license agreement allows only personal usage. |
not available in FREE version
|
available in PRO version
|
|
Comparative Analysis
Multiple references can be compared against the original simultaneously. |
2 refs
limited in FREE version
|
Unlim*
full in PRO version
|
|
Professional Bit Depth Support
(over 8 bits per color component)
|
not available in FREE version
|
available in PRO version
|
|
Batch Processing
VQMT provides GUI and command-line (CL) interfaces. It is possible using CL-tool to calculate several metrics for a given set of color components simultaneously and for more files (in parallel), thus achieving significant speed advantage comparing to case of using GUI tool. |
not available in FREE version
|
available in PRO version
|
|
Over HD Support
Processing of video with resolution equal or higher than 1280x720: HD, FullHD, 4K, 8K video. |
not available in FREE version
|
available in PRO version
|
Our plans
-
Planned featuresOffer featureStream supportAllows VQMT read and estimate quality of streamed videoFrame matchingDetect shift-type mismatch between frames and automatically resolve itGeometry matchingDetect geometry mismatches and automatically resolve itMultiple files in GUIComparison of an arbitrary number of files in GUI at once. Now only 2 distorted files is possible to compare with original (in GUI)BD-rate calculation, measure scheduleScheduling great amount of measurements and instruments to parse resultsNew metricsMetrics planned: ME TI, Google SI, Google TI, ML Blurring, ML Blocking, ML NoiseHuman visual attention aware metricsWeighted measure with bigger weight for more salient regionsPython wrapSimple Python interface to run VQMT and achieve results indirectly from Python codeSaving result historyDon't delete computed results on VQMT close. Reopen previous results
-
In next versionSpeedupUp to 30% speedup for majority of metricsVMAF accelerationOptimizing VMAF, using multiple cores or GPU capabilitiesParallel workRunning metrics on multiple cores for better productivitySubsamplingSkipping frames to achieve target bitrate or speedup processAuto colorspaceAllows you not to specify colorspace in command linePSNR over RGB and YUVYou can measure PSNR over all planesOnline visualizationSee metric visualization without saving it onlineOnline metricSee metric value for desired frame without global calculation processMore statistical dataMultiple methods of averaging and more statistical data output
VQMT back SDK
Back SDK allows companies to integrate it’s own products with VQMT metrics. Proceed to purchase SDK: Buy SDK.
VQMT SDK is provided as Linux shared library. VQMT back SDK compatible with C/C++, you can download interface here: vqmt_sdk.h
Here you can also find SDK usage sample: sdk_sample_yuv.cpp
Sample output:
$ ./vqmt_sdktest ~/video/cinepack.yuv ~/video/loseless.yuv yuv420p 1280 720 all
psnr: 39.985
psnr PSNR (256): 39.985
psnr APSNR: 39.985
psnr APSNR (256): 39.985
ssim: 0.960733
ssim precise: 0.96347
ssim GPU identical: 0.980512
msssim: 0.989333
msssim precise: 0.993107
3ssim: 0.948871
vqm: 0.503724
blocking: 40.4037
blurring: 0.0115356
blurring delta: 2.29336
delta: 0.000120949
msad: 0.873773
mse: 2.64237
dfm: 0
bi_psnr: 43.8853 2.65798
niqe: 8.24557
scd: 0
vmaf: 73.8748
ne: 0.00107096
bfm: 0
Plots Examples
These plots were built in MATLAB, based on measurement results from MSU VQMT.
Per-frame Y-PSNR for several codecs
Y-PSNR dependency vs bitrate and frame number
Average Y-PSNR for several codecs and bitrates
Colored surface of Y-PSNR vs frame number and bitrate
Example of the error in codec, detected with VQMT.
Another examples of plots built based on VQMT output.
Comparison with Analogs
Contributors
Special thanks to following contributors.
Studios
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
Hardware & Devices
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
Software & Solutions
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
Education
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
Television & Media Industry
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
Others
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
Testimonials
We have testimonials from the following big companies:
- Analog Devices(analog.com)
- Samsung (samsung.com)
- Intel (intel.com)
- Hewlett-Packard (hp.com)
- Thomson (thomson.net)
- Harmonic (harmonicinc.com)
- Matrox (matrox.com)
- Zoran (zoran.com)
- Kodak (kodak.com)
- Honeywell (honeywell.com)
Small companies:
- AudioCodecs (audiocodes.com)
- Envivio (envivio.fr)
- Ambric Inc (ambric.com)
- Smart phones (smartphones.no)
- TestPro (testpro.co.il)
- Bouygues Telecom (bouyguestelecom.fr)
- CDRinfo (cdrinfo.com)
- Infoteck (infoteck.ru)
- Cubiworld (cubiworld.ch)
- INESC Porto (inescporto.pt)
And universities:
- University of Technology Sydney (it.uts.edu.au)
- Fraunhofer (igd.fraunhofer.de)
- Brunel University (brunel.ac.uk)
- Alpen-Adria University (edu.uni-klu.ac.at)
- California State University (csudh.edu)
- Universeit Gent (intec.ugent.be)
Changelog
- 11.1 Geometry correction — May 2019
- Comparison on different resolution (geometry correction)
- New metrics (SI, TI)
- Saving bitrates
- And bugfixes
- 11.0 1400+ raw formats, wider HDR — Feb 2019
- Support for 1400+ new RAW formats
- Support for standard names for RAW formats
- Support new RGB ↔ YUV tables
- Support HDR videos and more HDR image formats: 3 more times video formats now can be lossless
- Result table with metric values inside VQMT
- Automatic command-line generation from GUI
- Plot of bitrate
- Actual VMAF support: modern models, 4k models, confidence intervals
- New blurring metric
- New Noise Estimation metric
- Linux command line considerably improved
- Better performance on some cases
- New quality control standards
- Measurement on subsampled U and V
- Latest CUDA engine
- And bugfixes
- Only x64-bit edition from this version
- 10.2 — Feb 2019
- PREMIUM licensing introduced
- And bugfixes
- 10.1 New metrics: VMAF, NIQE — Apr 2018
- Support of perspective and modern VMAF metric. Many settings guarantee full metric support.
- Now it’s possible to determine quality even without a reference via new NIQE metric.
- Improved usability: Drag & Drop files and changing of file order by single click.
- The -stdin parameter allows you to transmit data directly from the output of another program, such as FFmpeg, without saving multi-gigabyte files. Acceleration and simplification (PRO and DEMO only)
- New formats with support for HDR: extended support for TIFF and PXM family. (HDR only in PRO and DEMO)
- Better usability of command line - a lot of improvements while maintaining full backward compatibility (PRO and DEMO only)
- Built-in profiler. It will show which operations take the most time and allow you to accurately measure the performance of VQMT (PRO and DEMO only)
- Better JSON output
- TIFF as format for bad frames
- More functions in Linux version: more metrics, plugins support, OpenCL support (PRO and DEMO only)
- And bugfixes
- Now we recommend specifying original input using new key ‘-orig’ instead of traditional ‘-in’
- 10.0 BETA New main window, Linux support — Apr 2017
- New look of main window
- Saving and loading VQMT projects
- Status bar with diagnostic in main window
- Report generation
- Saving result plot in arbitrary image formats
- New clear VQMT folder structure
- Improved using vqmt console in PRO version
- Command line Linux utility
- 9.1 — Apr 2017
- Fixed incorrect results while using second processed video
- Added legend to result plot
- Saving log from GUI
- Fixed copying plot to clipboard
- Using *.YUV bug fixed: first frame could be duplicated
- Improved h265 support
- Index file building bug fixed
- Version 9.0 BETA New result window — Dec 2016
- New look of result window
- Using interface while calculation is processing and while viewing results
- Viewing multiple results at the same time
- Switch between calculation log and results plot
- Extended information about progress: count of processed frames, FPS, elapsed time, estimated time
- Pretty and configurable result plot
- Saving CSV after calculation finished
- Saving JSON result file in GUI
- Saving results’ plot as SVG file
- Version 8.1 — Nov 2016
- Added checking for updates.
- Using single image as source do not truncate all other inputs to one frame.
- Fixed crashes when use plugin metric.
- Improved video previewing, fixed opening visualization video inside VQMT.
- Fixed possible incorrect negative PSNR.
- Fixed mask behavior.
- Recognize patterns 720p etc. in the name of RAW file to detect resolution automatically.
- Hotkeys in Preview and Fullscreen window considerably improved.
- Allow to change video source and frame inside Fullscreen window.
- Bugfixes in Preview window performed.
- Fixed green frame in some types of file, incorrect reading of some color spaces.
- Version 8.0 BETA Preview window — Oct 2016
- Added new visualization methods: Lossless Video, Lossy Video and TIFF files.
- Preview window replaced.
- Full-screen preview, display selection.
- Side-by-side preview.
- Inspecting visualization inside VQMT.
- Version 7.1 — Oct 2016
- Crashes fixed.
- Improved command line output in PRO version.
- Preview display fix.
- Seeking and offsetting YUV files fixed.
- Version 7.0 BETA OpenCL support, Metrics reorganized — Jun 2016
- Added support for OpenCL device interface. Efficient calculation on different devices.
- Speedup of PSNR and SSIM metrics.
- Similar metrics joined.
- Version 6.2 JSON support — Jun 2016
- Added support for JSON output in console.
- Incorrect results with RGB metric on RGB video fixed.
- Version 6.1 BETA — Apr 2016
- Fixed bug: hangs in console while using files in network directories.
- Fixed incorrect behavior of .Y4M files in GUI.
- Fixed crashes when the length of all files was not able to be detected.”
- Version 6.0 BETA Acceleration — Mar 2016
- VQMT became about 3 times faster.
- 16 and 32 bpp floating point TIFF files supported (PRO version).
- Added “-threads” command line argument to PRO version, which allows to control CPU usage.
- Advanced mask processing - allowed shifts from exact black color for indication black area.
- Fixed crashes and incorrect results while using mask.
- Version 5.2 — Apr 2016
- Fixed bug: hangs in console while using files in network directories.
- Files incorrect behaviour of .Y4M files in GUI.”
- Version 5.1 Ranges and offsets support — Mar 2016
- Support comparing ranges of input videos.
- Fixed crashed while using YUV files with more than 8bpp.
- Corrected RGB <-> YUV conversion procedures.
- Fixed incorrect transformation for some input formats.
- 10, 14 and 16 bpp RGB format plain order changed to R, G, B.
- Version 5.0 BETA Open wizard, Command line refactor — Sep 2015
- Added Open File Wizard to help with opening and previewing files;
- Use wizard or file picker for selection of input file;
- Wizard supports drag&drop mechanism;
- Now user can select mode for opening file (FFmpeg, AviFile, automatic AviSynth, RAW file, image or image sequence and others);
- Special mode to compare the results of opening file using different modes;
- Automatic selection of the best open mode for specified file;
- More settings for opening process customization;
- Added automatic generation of video index file that helps file to be opened and correctly previewed;
- Number of supported codecs available for opening and previewing increased (supported for new versions of video codecs);
- Added support for images as input files: *.jpg, *.png, *.tif, more formats of *.bmp and many others;
- Added support for using image sequences as input video, automatic detection of sequence;
- Previewing raw files (*.yuv, etc.) “on the fly”;
- Detection of resolution of raw files from file name;
- PRO version console interface is more user friendly with full compatibility to previous version;
- Version 4.4 GPU support improved — Sep 2015
- Number of files available for metric calculation increased;
- The number of supported devices for running CUDA metrics increased;
- Unable to run metric for single file bug fixed;
- Unable to view results if some minor error occurred bug fixed;
- CUDA metrics crash fixed;
- Version 4.3 BETA — May 2015
- Number of files available for metric calculation increased. Now metric can be calculated for the files that not available for preview;
- Speed up of file opening in metric calculation process for some types of files;
- Standard VQMT plug-ins is now supported;
- Memory leak fixed in VQMT.
- Version 4.2 BETA — Apr 2015
- Added native support for *.mkv, *.flv and some other containers and codecs;
- Stability fixes.
- Version 4.1 BETA Native reading video files via FFmpeg — Mar 2015
- Added FFmpeg file reading support, the number of supported formats greatly increased. Using AviSynth is not recommended;
- Fixed x64 version crashed;
- Stability fixes.
- Version 3.2 AviSynth support improved — Feb 2015
- Existence of AviSynth determining fixed;
- AviSynth for VQMT as standalone installer;
- AviSynth plug-in opening fixed;
- Fixed unsuccessful file opening in AviSynth mode;
- AviSynth mode now supported in console;
- All dependencies now are immediately in installer, no more redistributable packages needed;
- Main menu and desktop labels fixed to determine Pro, Free and Pro Demo license;
- Executable file metadata errors fixed;
- Cosmetic fixed in Interface and file naming: revision number added to version naming;
- Fixed crashes in 64-bit version on multiple platforms;
- Fixed crashes and hangs after: the press of Process button, viewing of analyses result, other events.
- Version 3.1 — Nov 2012
- Changed to CUDA 5.0 toolkit, added Kepler support (Compute Capability 3.0)
- Stability fixes
- Fixed major bug with masking
- Version 3.0 GPU support, 64-bit version — Jun 2011
- Added stSSIM metric
- Added “.y4m” raw video internal support
- Added Autoupdate feature for free version (our PRO customers receive updates automatically)
- Added GPU realization for SSIM-based metrics (SSIM, 3-SSIM, MS-SSIM. Requires CUDA-capable device. See metrics info page for additional info)
- Added subjective comparison for the most popular metrics (see metrics info)
- Added 64-bit version of MSU VQMT (up to 10% speedup)
- Program crashes due memory lack when -metr ALL specified with large (i.e. 1280x720) video frames.
- Version 2.7.3 — Oct 2010
- Fixed bug with MSSIM metric causing source frame change
- Fixed some metrics inaccuracy causing different metric values by enabling\disabling visualization
- Program crashes due memory lack when -metr ALL specified with large (i.e. 1280x720) video frames
- Version 2.7.2 — Aug 2010
- Fixed bug causing incorrect metric values, when using 3SSIM and MSSSIM simultaneously
- Fixed bug causing incorrect PSNR metric values in CSV files
- Fixed bug causing no metric calculation for large (>4gb) files
- Not existing directory specified in “-cod” parameter will be created now and processing will not cancel.
- Program crashes due memory lack when -metr ALL specified with large (i.e. 1280x720) video frames
- Version 2.7.1 — Jun 2010
- Fixed bug in CVS file generation. Sometimes first frame metric value was empty
- Fixed bug causing incorrect MSE metric values after calculating SSIM metric
- Version 2.7 MSSSIM, 3SSIM metrics added — May 2010
- MSSSIM (fast and precise) metric implemented
- 3SSIM metric implemented
- Fixed bug in calculation of VQM metric under Windows 7
- Fixed bug during program launch on some machines
- Version 2.6 Windows Vista & Windows 7 support — Jan 2010
- Fixed bug in Scene Change Detection plugin when working under Windows Vista or Windows 7
- Fixed bug in saving visualization video when running on Windows Vista or Windows 7
- Fixed dependency with vcomp.dll
- Version 2.5 — Nov 2009
- Video with any resolution is now supported by all metrics. Video with resolution which is not appropriate for some metric is now expanded (via data duplication, separately for each metric) to make resolution acceptable
- 1.95 times speed up of command line tool multiple metrics calculation on average (PRO version only)
- YUV files with size more than 2Gb are supported now
- Output directory for *.CSV and visualization files is automatically set to folder of last specified reference file
- Fixed bug in processing of *.YUV files with non-standard resolution
- Fixed bug in loading the mask from *.YUV files
- Fixed bug in masking of L (LUV colorspace) component
- Fixed bug in processing of non-standard resolution *.AVS files
- Fixed bug in calculation of SSIM (precise) for second reference file
- Fixed bug in conversion from RGB32 to YUV color spaces for video with non-standard resolutions (affects calculation of metric for *.AVI files)
- Version 2.01 beta Masking added — Apr 2009
- 1.5 times speed up of command line tool multiple metrics calculation on average (PRO version only)
- Masking is added
- Fixed bug in 4:2:2 raw files with more than 8 bits per component support
- Version 2.0 beta HDTV support, Deep RAW files — Mar 2009
- *.MOV, *.VOB, *.WMV, *.MP4, *.MPG, *.MKV, *.FLV formats support simplified
- HDTV support (PRO version only)
- Raw files with more than 8 bits color depth per component are supported (PRO version only)
- Alternative SSIM and PSNR are added for compatibility with other implementations
- New version of *.CSV files with average metric values (PRO version only)
- Minor acceleration
- Preview buttons are added
- Options save is improved
- All MSU plugins are renamed (names are now more correct in GUI and simpler to call from PRO console)
- MSU Noise Estimation plugin bug with incorrect (identical) values for some videos is fixed
- MSU Noise Estimation and MSU BI-PSNR plugins provide correct information about their home pages now
- MSU BI-PSNR plugin crash during visualizing a metric for video sequences with dimensions less than 255 is fixed
- Version 1.52
- Error in saving CSV file for comparative analysis is fixed
- Version 1.51
- Error at the opening YUV-files is fixed
- Version 1.5
- Set of interface fixes
- Version 1.4
- Bugfixing in BMP processing (visualization saving, etc)
- Version 1.3
- Bug with YV12 yuv files fixed
- Version 1.2
- Now it is possible to compress visualization
- Plugin mechanism released
- Problem with some DV codecs
- Version 1.0
- More YUV file types are supported, including YV12, YUY2, YUV
- Supports unicode
- Visualization dialog was extensively reworked
- Interface is more user-friendly
- Version 0.81
- New AVI Reading system (support large AVI Files, VP 70)
- Bugfixing in final dialog
- Version 0.8
- New dialog with visualization of the comparison and for comparison of the selected frames added.
- Version 0.75
- Improved codecs support (x264)
- Now data from YUV-AVI files is extracted without conversion
- Bugfixing: #NAN in VQM calculation, calculation of RGB-metrics for YUV-files
- MSU Blocking Measure changed
- Version 0.74
- MSE, MSAD metrics added
- Saving of mean value of the metric added
- Improved codecs support (DivX3, WMV, mjpeg2000)
- Delta metric was changed
- Bugfixing (AviSynth - wrong result for comparison of three files)
- Version 0.73
- All colorspaces from AviSynth are supported
- I420 (IYUV) support added
- XviD and B-frames support added
- Bugfixing (YUV-files, SSIM)
- Version 0.72
- AVS Support added
- Bugfixing
- Doesn’t work with XVID
- Version 0.71
- First public beta
Contacts
E-mail: video-measure@graphics.cs.msu.ru