Multisampling Anti-Aliasing: A Closeup View

May 22, 2003 / by aths / page 6 of 8 / translated by 3DCenter Translation Team

   Alternative multisampling methods and their problems

Matrox created its own anti-aliasing solution for Parhelia and called it "Fragment Anti-Aliasing". A colour compression is implemented, Z fillrate and bandwith are saved as well. The problem of the unsmoothed intersection edges was already mentioned, but on the other hand the smoothing quality on outer polygon edges is quite good, since a 4x4 subpixel mask is used. This is a bit over the top though, because with better positioning, almost equal quality could have been achieved with a lot less subpixels. In addition, the memory for anti-aliasing samples is limited (obviously there's no RAM for 16x fullscreen anti-aliasing), and as a consequence of that, in some cases a few polygon edges aren't smoothed at all.

As we mentioned before, since the framebuffer compression saves just bandwith, AA modes with many subpixels require a lot of memory. The Matrox solution works only up to a certain number AA samples per frame, anything above that isn't calculated.

To get this under control, a procedure called Z3 anti-aliasing was invented. Per pixel the colours of up to 3 triangles can be saved. Thus every pixel has 3 "slots" which can save a colour each, along with the information about how many pixels the colour applies to. For more samples, the colours are mixed on-the-fly, but inaccuracies may occur when there are more than 3 subpixels, i.e. when more than three triangles cover some samples of a pixel. This will also lead to problems with handling Z values correctly.

Z3 anti-aliasing doesn't save the Z values per subpixel, only per triangle that covers the pixel. It also uses a Z gradient that can be used to reconstruct the covered subpixels' Z values. But if a fourth triangle touches the pixel, this process hits a limitation. As a result, the colour values on the edges are a bit imprecise, but a lot of memory is saved. In consumer hardware this procedure hasn't been implemented so far.

All the mentioned problems could be solved at once, if Tile Based Deferred Rendering was used. Multisampling saves a lot of texel fillrate, but bandwith also isn't a problem here, because every tile is rendered only once and is final when it's written into the framebuffer. Fast multisampling anti-aliasing with both a lot of subpixels and low memory footprint is practically only possible on a Tile Based Deferred Renderer.

Enough of the theory. Our practical investigation has so far only concerned the GeForce line. What about the Radeon?

   Quality vs. Radeon: Who's the smoothest?

On Radeon 9500 or higher ATi offers multisampling as well. Here's the comparison:

GeForce since 4 Ti on the left, Radeon since 9500 on the right.

Again, AA lines are shown in green, subpixels in red, and texel positions in blue. With 4x anti-aliasing the R300 core uses 4 temporary lines in the triangle setup. The better subpixels distribution also applies to the X axis. For the edges this results in a higher effective resolution:

A more intelligent grid delivers smoother edges.

Although the chosen example isn't the best, it still demonstrates that the more efficient grid delivers smoother colour gradients.

In terms of multisampling anti-aliasing quality the R300 (as well as its successors R350 and R360) are miles ahead of the NV30: while the NV30 can only subdivide the triangle into two temporary lines per final on-screen line, the R300 can split it in up to six. What GeForce3, 4 and FX offer in terms of edge smoothing, has been hoplessly outdated by the R300. And since the R300 core there's an additional mechanism, which increases the smoothing quality even more.

   Gamma correct Downfiltering: What you calculate should be what you see

Actually this isn't a multisampling specific thing, but since in practice this feature is only encoutered along with multisampling anti-aliasing, we still want to mention it.

Generally gamma correction is used, when an image appears too dark. If just the brightness was cranked up on a dark image, black would turn to grey, and the lighter areas would merge, because everything above a certain brightness would become equally white. Dynamic range decreases and ultimately the image would be worse. Thus gamma correction is used. It has the convenient property to not reduce the contrast range, and instead turns the non-linear brightness of regular cathode ray tubes nearly back to linear.

For historic reasons, a lot of graphics were designed for common monitors though, and to linerarise the brightness in such a frame results in a unexpectedly bright picture. Therefore a truly linear brightness response via gamma correction is often undesirable. For anti-aliasing the downfiltering should assume a linear response though: for example, if the procedure creates up to 3 colour graduations, all three should be equally distributed with regard to brightness.

In reality moderatly dark colours are displayed so dark that it's very hard to distinguish them from black. Hence, not all of the colour graduations are visible anymore, which results in a "jaggy" egde. But then again a full correction of the frame with the gamma function would deliver too bright images. That's why it's a good idea to to gamma correct the subpixels before downfiltering, and to readjust the final pixel.

If the inner and outer areas appear to have the same intensities,
your monitor's brightness response is linear.

As can be seen above, averaging two colours can result in unexpected brightness levels. Gamma corrected downfiltering solves this, but there are limits.

The driver needs to know, what the gamma correction setting of the device is, and then estimate which value would be ideal. On some displays, which perform the gamma correction outside the driver, this method can lead to an overcorrection, also resulting in worse visibility of the colour graduations.

We're actually now done with discussing multisampling. But some subtleties aren't obvious until the method is compared to another one. This shall happen on the next page.