Staccato Signals

A comparison of QuickTime and CoreGraphics image scaling

I recently used a photo management utility to scale some of my photos and I was less than thrilled with the results. After a brief exchange with the author, I learned that he uses NSGraphicContext image interpolation to scale the image. As a user, I really wanted to get better photos out of this process, and as a developer I started wondering if there's anything he could do to improve his software short of writing a custom scaling algorithm. The answer turns out to be: when image quality is important, do not use CoreGraphics or AppKit to scale your images -- use QuickTime instead.

I whipped up a small application that renders the same image in four different ways:

  1. Using QuickTime graphics importer to load and scale the image at the same time, then render the scaled image without additional scaling to a window.
  2. Using QuickTime graphics importer to load the image at its native size, then scale the image into a window using CoreGraphics high quality image interpolation.
  3. Using QuickTime graphics importer to load the image at its native size, then scale the image into a window using CoreGraphics low quality image interpolation.
  4. Using QuickTime graphics importer to load the image at its native size, then scale the image into a window using CoreGraphics without image interpolation.

At scaling factor of ½, the results were unremarkable. The CG image without interpolation was poor, and the remaining three were indistinguishable.

However, at scaling factors that are not powers of two, the difference between QD and high quality CG scaling was quite noticeable. QuickTime scaling produced the best output, and CG with no interpolation produced the worst. Ironically, CG with high quality scaling made the image look worse than CG with low quality scaling; the so-called high-quality scaling algorithm makes the image much too blurry.

The resulting images are shown in the movie below. Pay particular attention to the regions marked with red, green, and blue rectangles. The red rectangles show regions in which distortion of straight edges is particularly noticeable when scaled with CG and no interpolation. The green and blue rectangles show areas in which loss of detail is particularly noticeable when rendered with CG and high quality interpolation, compared to QuickTime. Finally, the rectangles themselves are a very good demonstration of what the different rendering methods do to sharp straight edges.

If you want to play with this yourself, the code is below. Build this as a Mac OS X Carbon application, and put a sample.jpg in the application's Resources folder:

PLAIN TEXT
C++:
  1. #include <Carbon/Carbon.h>
  2. #include <CoreFoundation/CoreFoundation.h>
  3. #include <QuickTime/QuickTime.h>
  4.  
  5. //
  6. //  WARNING: This code is for demonstration purposes only. Do not use it verbatim, you'll regret it.
  7. //
  8.  
  9. int main()
  10. {
  11.     float   scaling = 8/float(9);
  12.     bool    offset = false;
  13.  
  14.     FSRef         imageRef;
  15.     CFURLGetFSRef(
  16.         CFBundleCopyResourceURL(
  17.             CFBundleGetMainBundle(),
  18.             CFSTR("sample"),
  19.             CFSTR("jpg"),
  20.             0
  21.         ),
  22.         &imageRef
  23.     );
  24.    
  25.     FSSpec        imageSpec;
  26.     FSGetCatalogInfo(&imageRef, kFSCatInfoNone, 0, 0, &imageSpec, 0);
  27.    
  28.     ComponentInstance     importer;
  29.     GetGraphicsImporterForFile(&imageSpec, &importer);
  30.    
  31.     Rect          fullImageBounds;
  32.     GraphicsImportGetNaturalBounds(importer, &fullImageBounds);
  33.    
  34.     ImageDescriptionHandle        imageDesc =
  35.         reinterpret_cast<ImageDescriptionHandle>(NewHandle(sizeof(ImageDescription)));
  36.        
  37.     GraphicsImportGetImageDescription(importer, &imageDesc);
  38.  
  39.     short   depth = ((imageDesc).depth <= 16) ? 16 : 32;
  40.     OSType  pixelFormat = (depth == 32) ? k32ARGBPixelFormat : k16BE555PixelFormat;
  41.  
  42.     GraphicsImportSetQuality(importer, codecLosslessQuality);
  43.  
  44.     // Load full image    
  45.     OffsetRect(&fullImageBounds, -fullImageBounds.left, -fullImageBounds.top);
  46.    
  47.     UInt32  fullRowStride = fullImageBounds.right * depth / 8;
  48.     Ptr     fullImageBuffer = NewPtr(fullRowStride * fullImageBounds.bottom);
  49.    
  50.     GWorldPtr       fullImageWorld;
  51.     QTNewGWorldFromPtr(&fullImageWorld, pixelFormat, &fullImageBounds, NULL, NULL, 0, fullImageBuffer, fullRowStride);
  52.     GraphicsImportSetGWorld (importer, fullImageWorld, NULL);
  53.     PixMapHandle          fullImagePixMap = GetGWorldPixMap(fullImageWorld);
  54.     LockPixels(fullImagePixMap);
  55.     GraphicsImportDraw(importer);
  56.  
  57.     CGDataProviderRef     fullImageDataProviderRef =
  58.         CGDataProviderCreateWithData(
  59.             NULL,
  60.             GetPixBaseAddr(fullImagePixMap),
  61.             fullImageBounds.bottom * GetPixRowBytes(fullImagePixMap),
  62.             NULL
  63.         );
  64.    
  65.     CGImageRef            fullImage =
  66.         CGImageCreate(
  67.             fullImageBounds.right, fullImageBounds.bottom,
  68.             (fullImagePixMap).cmpSize,
  69.             (fullImagePixMap).pixelSize,
  70.             GetPixRowBytes(fullImagePixMap),
  71.             CGColorSpaceCreateDeviceRGB(),
  72.             kCGImageAlphaPremultipliedFirst,
  73.             fullImageDataProviderRef,
  74.             NULL,
  75.             true,
  76.             kCGRenderingIntentDefault
  77.         );
  78.  
  79.  
  80.     // Load scaled image
  81.     Rect    scaledImageBounds = {
  82.         0, 0, fullImageBounds.bottom * scaling, fullImageBounds.right * scaling
  83.     };
  84.  
  85.     MatrixRecord  scaledMatrix;
  86.     SetIdentityMatrix(&scaledMatrix);
  87.     ScaleMatrix(&scaledMatrix, X2Fix(scaling), X2Fix(scaling), X2Fix(0), X2Fix(0));
  88.     GraphicsImportSetMatrix(importer, &scaledMatrix);
  89.  
  90.     UInt32      scaledRowStride = scaledImageBounds.right * depth / 8;
  91.     Ptr         scaledImageBuffer = NewPtr(scaledRowStride * scaledImageBounds.bottom);
  92.     GWorldPtr   scaledImageWorld;
  93.     QTNewGWorldFromPtr(&scaledImageWorld, pixelFormat, &scaledImageBounds, NULL, NULL, 0, scaledImageBuffer, scaledRowStride);
  94.     GraphicsImportSetGWorld (importer, scaledImageWorld, NULL);
  95.     PixMapHandle          scaledImagePixMap = GetGWorldPixMap(scaledImageWorld);
  96.     LockPixels(scaledImagePixMap);
  97.     GraphicsImportDraw(importer);
  98.  
  99.     CGDataProviderRef     scaledImageDataProviderRef =
  100.         CGDataProviderCreateWithData(
  101.             NULL,
  102.             GetPixBaseAddr(scaledImagePixMap),
  103.             scaledImageBounds.bottom * GetPixRowBytes(scaledImagePixMap),
  104.             NULL
  105.         );
  106.    
  107.     CGImageRef            scaledImage =
  108.         CGImageCreate(
  109.             scaledImageBounds.right, scaledImageBounds.bottom,
  110.             (scaledImagePixMap).cmpSize,
  111.             (**scaledImagePixMap).pixelSize,
  112.             GetPixRowBytes(scaledImagePixMap),
  113.             CGColorSpaceCreateDeviceRGB(),
  114.             kCGImageAlphaPremultipliedFirst,
  115.             scaledImageDataProviderRef,
  116.             NULL,
  117.             true,
  118.             kCGRenderingIntentDefault
  119.         );
  120.  
  121.     CGContextRef context;
  122.  
  123.     // Create window with QT-scaled image
  124.     Rect      qtBounds = {
  125.         50,
  126.         50,
  127.         50 + scaledImageBounds.bottom,
  128.         50 + scaledImageBounds.right
  129.     };
  130.    
  131.     WindowRef     qtWindow;
  132.     CreateNewWindow(
  133.         kDocumentWindowClass,
  134.         kWindowCollapseBoxAttribute | kWindowStandardHandlerAttribute | kWindowCompositingAttribute,
  135.         &qtBounds,
  136.         &qtWindow
  137.     );
  138.     SetWindowTitleWithCFString(qtWindow, CFSTR("QuickTime scaling"));
  139.     ShowWindow(qtWindow);
  140.    
  141.     QDBeginCGContext(GetWindowPort(qtWindow), &context);
  142.     CGContextSetInterpolationQuality(context, kCGInterpolationNone);
  143.     CGContextDrawImage(context, CGRectMake(0, 0, scaledImageBounds.right, scaledImageBounds.bottom), scaledImage);
  144.     CGContextFlush(context);
  145.     QDEndCGContext(GetWindowPort(qtWindow), &context);
  146.    
  147.  
  148.     // Create window with CG-scaled image (high quality interpolation)
  149.     Rect      cgHighBounds = qtBounds;
  150.     if (offset) {
  151.         OffsetRect(&cgHighBounds, 50 + scaledImageBounds.right, 0);
  152.     }
  153.    
  154.     WindowRef     cgHighWindow;
  155.     CreateNewWindow(
  156.         kDocumentWindowClass,
  157.         kWindowCollapseBoxAttribute | kWindowStandardHandlerAttribute | kWindowCompositingAttribute,
  158.         &cgHighBounds,
  159.         &cgHighWindow
  160.     );
  161.     SetWindowTitleWithCFString(cgHighWindow, CFSTR("CoreGraphics high quality"));
  162.     ShowWindow(cgHighWindow);
  163.  
  164.     QDBeginCGContext(GetWindowPort(cgHighWindow), &context);
  165.     CGContextSetInterpolationQuality(context, kCGInterpolationHigh);
  166.     CGContextDrawImage(context, CGRectMake(0, 0, scaledImageBounds.right, scaledImageBounds.bottom), fullImage);
  167.     CGContextFlush(context);
  168.     QDEndCGContext(GetWindowPort(cgHighWindow), &context);
  169.    
  170.     // Create window with CG-scaled image (low quality interpolation)
  171.     Rect      cgLowBounds = qtBounds;
  172.     if (offset) {
  173.         OffsetRect(&cgLowBounds, 0, 50 + scaledImageBounds.bottom);
  174.     }
  175.    
  176.     WindowRef     cgLowWindow;
  177.     CreateNewWindow(
  178.         kDocumentWindowClass,
  179.         kWindowCollapseBoxAttribute | kWindowStandardHandlerAttribute | kWindowCompositingAttribute,
  180.         &cgLowBounds,
  181.         &cgLowWindow
  182.     );
  183.     SetWindowTitleWithCFString(cgLowWindow, CFSTR("CoreGraphics low quality"));
  184.     ShowWindow(cgLowWindow);
  185.  
  186.     QDBeginCGContext(GetWindowPort(cgLowWindow), &context);
  187.     CGContextSetInterpolationQuality(context, kCGInterpolationLow);
  188.     CGContextDrawImage(context, CGRectMake(0, 0, scaledImageBounds.right, scaledImageBounds.bottom), fullImage);
  189.     CGContextFlush(context);
  190.     QDEndCGContext(GetWindowPort(cgLowWindow), &context);
  191.  
  192.     // Create window with CG-scaled image (no interpolation)
  193.     Rect      cgNoneBounds = qtBounds;
  194.     if (offset) {
  195.         OffsetRect(&cgNoneBounds, 50 + scaledImageBounds.right, 50 + scaledImageBounds.bottom);
  196.     }
  197.    
  198.     WindowRef     cgNoneWindow;
  199.     CreateNewWindow(
  200.         kDocumentWindowClass,
  201.         kWindowCollapseBoxAttribute | kWindowStandardHandlerAttribute | kWindowCompositingAttribute,
  202.         &cgNoneBounds,
  203.         &cgNoneWindow
  204.     );
  205.     SetWindowTitleWithCFString(cgNoneWindow, CFSTR("CoreGraphics without interpolation"));
  206.     ShowWindow(cgNoneWindow);
  207.  
  208.     QDBeginCGContext(GetWindowPort(cgNoneWindow), &context);
  209.     CGContextSetInterpolationQuality(context, kCGInterpolationNone);
  210.     CGContextDrawImage(context, CGRectMake(0, 0, scaledImageBounds.right, scaledImageBounds.bottom), fullImage);
  211.     CGContextFlush(context);
  212.     QDEndCGContext(GetWindowPort(cgNoneWindow), &context);
  213.  
  214.     RunApplicationEventLoop();
  215. }

Nov 1, 2004 in

Leave a Comment

Please note: Comment moderation is enabled and may delay your comment. There is no need to resubmit your comment.

© 1998-2010 Ben Artin.
License