Main page — VQMT

MSU Video Quality Measurement Tool

Basic Information About VQMT

What’s new in MSU VQMT 10?

Download FREE non-commercial version with limitations or Buy PRO.

Special offer for recent customers:
50% off! Know more…

Brief Description

Demo Video: Basic Use Case Overview

MSU Video Quality Measurement Tool (VQMT) is a program for objective video quality assessment. It provides functionality for both full-reference (two videos are examined) and single-reference (one video is analyzed) comparisons. Basic scheme of the tool is illustrated on below.


VQMT input and output

Video quality is a characteristic of a video passed through a video transmission/processing system, a formal or informal measure of perceived video degradation (typically, compared to the original video). Video processing systems may introduce some amounts of distortion or artifacts in the video signal, so video quality evaluation is an important problem. MSU VQMT is a program for video quality measurements. This application allows you to create objective comparison of video codecs and perform video processing filters analysis.


Visualization examples for several metrics

Program allows to answer the next questions:

  1. One codec has more blurring than another. On what frames? Average mark? *
  2. One codec has more blocking than another. On what frames? Average mark? *
  3. One codec has lower quality than another. On what frames? Average mark in PSNR, VQM, SSIM? *

* Corresponding “bad frames” are saved automatically as BMP files.

Current stable version: 10.1 [Changelog]

Main Features

Feature FREE version PRO version
Flexible Functionality

The tool provides functionality for calculating:

  • metric values for each frame
  • average value for sequence
  • metric values for specific color component
available in FREE version
available in PRO version
GPU and other computing devices

The tool supports performing some metrics on CUDA compatible devices (nVIDIA video cards) and all other OpenCL compatible devices (wide set of video cards, CPUs, multicore systems)

available in FREE version
available in PRO version
Objective Metrics Support
  • PSNR (4 variations)
  • Delta
  • MSAD
  • MSE
  • SSIM (2 variations, GPU, OpenCL support)
  • MS-SSIM (2 variations, GPU, OpenCL support)
  • 3SSIM (GPU, OpenCL support)
  • stSSIM
  • VMAF
  • VQM
  • MSU Blurring Metric
  • MSU Blocking Metric
  • MSU Brightness Flicking Metric – opensource plugin
  • MSU Brightness Independent PSNR – opensource plugin
  • MSU Drop Frame Metric – opensource plugin
  • MSU Noise Estimation Metric
  • MSU Scene Change Detector
  • NIQE
available in FREE version
available in PRO version
Video and Image Formats Support
  • Video files: *.AVI, *.DAT, *.DIVX, *.F4V, *.FLV, *.H265, *.M2TS, *.M2V, *.M4V, *.MKV, *.MOV, *.MP4, *.MPG, *.MTS, *.MXF, *.OGM, *.OGV, *.QT, *.TS, *.VOB, *.WMV, *.265, *.3G2, *.3GP, *.3GPP and others
  • AviSynth: *.AVS
  • RAW files: *.YUV (scientific), *.Y4M
  • Image files and image sequences: *.BMP, *.CUT, *.DDS, *.EXR, *.G3, *.GIF, *.HDR, *.ICO, *.IFF, *.LBM, *.J2K, *.J2C, *.JNG, *.JP2, *.JPG, *.JIF, *.JPEG, *.JPE, *.JXR, *.WDP, *.HDP, *.MNG, *.PCD, *.PCX, *.PCT, *.PICT, *.PIC, *.PNG, *.PBM, *.PGM, *.PPM, *.PSD, *.RAS, *.SGI, *.RGB, *.RGBA, *.BW, *.TGA, *.TARGA, *.TIF, *.TIFF, *.WAP, *.WBMP, *.WBM, *.WEBP, *.XBM, *.XPM
available in FREE version
available in PRO version
ROI Support (via masking)

Allows to calculate metrics in Regions of Interest (ROI). ROI should be specified by additonal provided video with mask for each frame.
available in FREE version
available in PRO version
Metrics Visualization

Video visualization file is generated for combination [metric, color component, reference]. Color and luminance of each pixel in visuzalization video is defined by metric value in that pixel (very useful for codec and filter debugging).
available in FREE version
available in PRO version
Results in CSV format

Metrics results and average values are saved in set of *.CSV (comma-separated values) file which can be opened everywhere.
available in FREE version
available in PRO version
Results in JSON format

All metric results and information about file opening are saved in single JSON file or stdout of command line utility.
available in FREE version
available in PRO version
Usage in companies

Free version license agreement allows only personal usage.
not available in FREE version
available in PRO version
Comparative Analysis

Multiple references can be compared against the original simultaneously.

 2 refs
limited in FREE version
 Unlim*
full in PRO version
Professional Bit Depth Support

(over 8 bits per color component)

  • RAW files
    • P010 (10 bit 4:2:0 yuv)
    • P014 (14 bit 4:2:0 yuv)
    • P016 (16 bit 4:2:0 yuv)
    • P210 (10 bit 4:2:2 yuv)
    • P214 (14 bit 4:2:2 yuv)
    • P216 (16 bit 4:2:2 yuv)
    • P410 (10 bit 4:4:4 yuv)
    • P414 (14 bit 4:4:4 yuv)
    • P416 (16 bit 4:4:4 yuv)
    • P410_RGB (10 bit 4:4:4 rgb)
    • P414_RGB (14 bit 4:4:4 rgb)
    • P416_RGB (16 bit 4:4:4 rgb)
    • Float (32 bit 4:4:4 yuv)
    • Float_RGB (32 bit 4:4:4 rgb)
  • TIFF files with 1-16-bit integers or 32-bit floats
  • Whole set of PXM (PBM, PGM and PPM)
not available in FREE version
available in PRO version
Batch Processing

VQMT provides GUI and command-line (CL) interfaces. It is possible using CL-tool to calculate several metrics for a given set of color components simultaneously and for more files (in parallel), thus achieving significant speed advantage comparing to case of using GUI tool.
not available in FREE version
available in PRO version
Over HD Support

Processing of video with resolution equal or higher than 1280x720: HD, FullHD, 4K, 8K video.
not available in FREE version
available in PRO version
* multiple references allowed in command line interface

Our plans

  • Planned features
    Offer feature
    Stream support
    Allows VQMT read and estimate quality of streamed video
    Frame matching
    Detect shift-type mismatch between frames and automatically resolve it
    Geometry matching
    Detect geometry mismatches and automatically resolve it
    Multiple files in GUI
    Comparison of an arbitrary number of files in GUI at once. Now only 2 distorted files is possible to compare with original (in GUI)
    BD-rate calculation, measure schedule
    Scheduling great amount of measurements and instruments to parse results
    New metrics
    Metrics planned: ME TI, Google SI, Google TI, ML Blurring, ML Blocking, ML Noise
    Human visual attention aware metrics
    Weighted measure with bigger weight for more salient regions
    Python wrap
    Simple Python interface to run VQMT and achieve results indirectly from Python code
    Saving result history
    Don't delete computed results on VQMT close. Reopen previous results
  • In next version
    Speedup
    Up to 30% speedup for majority of metrics
    VMAF acceleration
    Optimizing VMAF, using multiple cores or GPU capabilities
    Parallel work
    Running metrics on multiple cores for better productivity
    Subsampling
    Skipping frames to achieve target bitrate or speedup process
    Auto colorspace
    Allows you not to specify colorspace in command line
    PSNR over RGB and YUV
    You can measure PSNR over all planes
    Online visualization
    See metric visualization without saving it online
    Online metric
    See metric value for desired frame without global calculation process
    More statistical data
    Multiple methods of averaging and more statistical data output

VQMT back SDK

Back SDK allows companies to integrate it’s own products with VQMT metrics. Proceed to purchase SDK: Buy SDK.

VQMT SDK is provided as Linux shared library. VQMT back SDK compatible with C/C++, you can download interface here: vqmt_sdk.h

Here you can also find SDK usage sample: sdk_sample_yuv.cpp

Sample output:

$ ./vqmt_sdktest ~/video/cinepack.yuv ~/video/loseless.yuv yuv420p 1280 720 all
psnr:                      39.985
psnr PSNR (256):           39.985
psnr APSNR:                39.985
psnr APSNR (256):          39.985
ssim:                      0.960733
ssim precise:              0.96347
ssim GPU identical:        0.980512
msssim:                    0.989333
msssim precise:            0.993107
3ssim:                     0.948871
vqm:                       0.503724
blocking:                  40.4037
blurring:                  0.0115356
blurring delta:            2.29336
delta:                     0.000120949
msad:                      0.873773
mse:                       2.64237
dfm:                       0
bi_psnr:                   43.8853 2.65798
niqe:                      8.24557
scd:                       0
vmaf:                      73.8748
ne:                        0.00107096
bfm:                       0

Plots Examples

These plots were built in MATLAB, based on measurement results from MSU VQMT.


Per-frame Y-PSNR for several codecs

Y-PSNR dependency vs bitrate and frame number

Average Y-PSNR for several codecs and bitrates

Colored surface of Y-PSNR vs frame number and bitrate

Example of the error in codec, detected with VQMT.

Another examples of plots built based on VQMT output.

Comparison with Analogs

Contributors

Special thanks to following contributors.

Studios

Walt Disney Studios Home Box Office, Inc. Olympic Broadcasting Services 20th Century Fox Film Corporation Fox Broadcasting Company
NBCUniversal

Hardware & Devices

Intel Corporation Nvidia Corporation Apple Inc. Cisco Systems, Inc. Sony Computer Entertainment, Inc.
Dolby Laboratories, Inc. Ericsson Motorola Solutions, Inc. AMETEK, Inc. Evertz Microsystems Limited
Telestream, Inc Analog Devices, Inc. Extron Electronics Lockheed Martin Exar
Chips&Media, Inc. HDLogix, Inc. Comtronic Computer, Inc.

Software & Solutions

Skype Microsoft Corporation Oracle Corporation Facebook, Inc. QuickPlay Media, Inc.
MediaTek, Inc. RealNetworks, Inc. AVerMedia Technologies Inc. Ittiam Systems Elemental Technologies, Inc.
Luxoft Smith Micro Software, Inc. BroadSoft

Education

University of Waterloo Simon Fraser University WrocЕ‚aw University of Technology University of West Bohemia University of Vienna
Fraunhofer-Institut fГјr Digitale Medientechnologie IDMT Julius Maximilian University of WГјrzburg Federal University of ParГЎ Hamburg University of Technology

Television & Media Industry

Sky Deutschland AG Bell Media DirecTV Kaltura Netflix Inc.
Vubiquity, Inc Telus Communications Beijing TL-Chinasoft Technology Co., Ltd Veset RGB Networks
Burosch Audio-Video-Technik TelefГіnica, S.A. Allegro DVT Octasic Inc. Vlaamse Radio- en Televisieomroeporganisatie

Others

Sorenson Communications InterDigital Intuitive Surgical, Inc. TASER International, Inc. National Teleconsultants LLC
VISLINK Whitney Medical Solutions Movidius Gaikai iMemories, Inc.
Avaya Inc. Snell Ltd CEVA, Inc. Rockwell Collins, Inc. CDNetworks

Testimonials

We have testimonials from the following big companies:

Small companies:

And universities:

Changelog

Contacts

E-mail: video-measure@graphics.cs.msu.ru

21 Feb 2019
See Also
MSU Video Upscalers Benchmark 2021
The most comprehensive comparison of video super resolution (VSR) algorithms by subjective quality
MSU Video Quality Measurement Tool: Picture types
VQMT 13.1 Online help: List of all picture types available in VQMT and their aliases
MSU Video Quality Measurement Tool: Usage of VQMT metrics in CLI
VQMT 13.1 Online help: Description of VQMT metrics, their parameters and using in CLI
MSU Video Quality Measurement Tool: VQMT various lists and tables
VQMT 13.1 Online help: Information about VQMT visualization formats, read modes, etc...
MSU Video Quality Measurement Tool: Command-line help
VQMT 13.1 Online help: Information about all VQMT CLI options for Pro/Premium versions
MSU Video Quality Measurement Tool: Configuration description
VQMT 13.1 Online help: Description of the format of the VQMT configuration file
Site structure
MSU Graphics & Media Lab Video Group
2019–2020