Comparison of Scaler Kernels in y4mscaler
Frequency Response

This page illustrates the spectral effects of the various kernels implemented in y4mscaler's default scaling engine.

The synthetic source image used here is a discrete "chirp", with a series of horizontal bands of increasing spatial frequency, like this:

Each horizontal band is a grayscale sine wave of different frequency. The frequencies range from zero (constant white) at the top to the Nyquist limit (alternating black and white pixels) at the bottom.

In the table below, each row corresponds to a different "M/N" downscaling factor, and each column corresponds to a different kernel in the scaler engine. The source frame for each row is "N/M" times the final horizontal size, and is then scaled horizontally using y4mscaler:

    y4mscaler -I bg=rgb:128,128,128 -O yscale=1/1 -O xscale=M/N -S option=KERNEL -S mode=mono
The amplitude of each band in the output image is measured (by measuring the max and min values) and those amplitudes are plotted in the little graphs shown below. This yields an empirical measurement of the frequency response of as a function of scale factor and kernel. The ideal frequency response is perfect low-pass filter, which is plotted in green. (Both axes are linear; x-axis is spatial frequency, from DC to Nyquist limit; y-axis is transfer function [0,1].)

Here is the frequency response table:

M/N box linear quadratic cubic cubicB cubicCR cubicK4 sinc:4 sinc:6

The "box" filter performs the worst with respect to aliasing, but it is the fastest. The "sinc:N" filters perform the best, with the sharpest cutoff, but it is the slowest. (Cutoff sharpness and processing time increase with N.) "cubic" is usually the best compromise between visual quality and processing speed in the graphics world. "cubicB" and "cubicCR" are all of the same order and require the same amount of processing, but "cubicB" rolls off earlier and "cubicCR" later. (Earlier roll-off will cause more softening, which may help to squeeze out extra bits when MPEG encoding.)

You may also want to see the resulting scaled chirp frames themselves, or even a brief comparison with yuvscaler.

return to y4mscaler

maximum impact research
Digital Media Group
<dmg at>
Last modified: Sun Dec 11 14:55:29 EST 2005

©2003 Matthew Marjanovic.
This material may not be republished in any form without express written consent of the author.