ATi & nVidia in driver "optimizations" race

July 26, 2003 / by Leonidas / page 9 of 9

   nVidia Optimizations, Addition

In the course of the discussion of a first version of this article within the internal part of the 3DCenter Forum concerned Demirug (programmer of the D3D AF-Tester) again more exactly the application-specific optimization of Unreal Tournament 2003. nVidia uses the filename "ut2003.exe" to apply the special optimizations (can easily be proofed by renaming other applications, results in higher performance). To prove the image quality the D3D AF-Tester should be renamed to "ut2003.exe" to show the nVidia optimizations.

But there was a "technical" problem which showed quite unexpected another "optimization" of nVidia. Renaming the actual version 1.1 of the D3D AF-Tester (which was used in the first version of this article) caused nothing - results stayed the same. Only after Demirug presented a new version of the D3D AF-Tester, this effect disappeared and it could be used to evaluate the image quality of Unreal Tournament 2003 after renaming the executable to "ut2003.exe".

It is very obvious that nVidia has integrated a small security token which prohibits the D3D AF-Tester program to show the Unreal Tournament 2003 optimization just by renaming the tool. The only reason for doing this is nVidia don´t want the hardware reviewers to prove the bad filter quality under Unreal Tournament 2003 with the D3D AF-Tester (which is btw a tool recommended by nVidia).

On an interesting sidenote, Demirug came up with a brand-new version of the D3D AF-Tester which let him get the following results in the three nVidia filtering modes under Unreal Tournament 2003:

Quality filter:
• The trilinear filtering on the stage 0 used only for approximately half of all pixels.
• The trilinear filtering on stage 1-7 is only a completely narrow volume between the MipMaps.
• Textures on the stage 0 get the maximum level at anisotropic filtering.
• Textures on stage 1-7 get maximally a level at anisotropic filtering of 2.

Performance filter
• Trilinear filtering is missing still, gets only a bilineare filtering.
• Textures on the stage 0 get the maximum level at anisotropic filtering.
• Textures on stage 1-7 get maximally a level at anisotropic filtering of 2.

High Performance filter
• Trilinear filtering is missing still, gets only a bilineare filtering.
• Textures on the stage 0 get the maximum level at anisotropic filtering - however only with the filtering, which is used anyway during the attitude (in principle angle dependence with this filter mode).
• Textures on stage 1-7 get no anisotropic filtering.

This observation applies only to Unreal Tournament 2003. In all other applications (after present level of knowledge!) the nVidia driver behaves "normally" and implements for all texture stages the computation of the trilinear anisotropic filter in such a way, as this was requested by control panel or tweak tool.

   Conclusion

The following can now be summarized:

• nVidia uses in 3DMark2001 and 3DMark03 application specific optimizations in the drivers 44.03 to 44.90, which under first benchmark approx. 16 percent and under second benchmark approx. 62 percent (!) performance advantage under 8x anisotropic filter bring in. Whether these application specific optimizations bring disadvantages in the image quality with itself was not tested, since application specific optimizations do not have anyway absolutely to be in theoretical benchmarks.

• nVidia uses further under Unreal Tournament 2003 a application specific optimization in the drivers 44.03 to 44.90, which brings a performance advantage in the flybys about approx. 57 percent (!) under 8x anisotropic filter. To exactly determine how much of these 57% were reached only with loss of image quality and how much were reached with identical screen output is beyond our possibilites and must remain an open question. It was clearly shown, however, that nVidia at least uses a pseudo trilinear filter particularly under Unreal Tournament 2003, as well as partial MipMaps shifts to the rear in the anisotropic filter and in some cases the full anisotropic filter did not work. Further it could be proven that this is not the normal condition of the nVidia drivers, since these filter cleanly trilinear anisotropic in all other applications. Thus it is considered proven that the application specific optimization of Unreal Tournament 2003 brings a definitely worse image quality with itself, even if the difference is quite small.

• ATi used a application specific optimization, which brings in approx. 2 percent performance advantage for this benchmark under 16x anisotropic filter under 3DMark03 up to the driver 03.4. In the drivers 03.5 and 03.6 this optimization is no longer verifiable, so we can rest this case.

• ATi uses a application specific optimization in the drivers 03.4 to 03.6, which brings in approx. 4 percent performance advantage under 16x anisotropic filter under 3DMark2001. Wheter this application specific optimization uses degradations in terms of image quality was not tested, since we consider any optimization for a synthetic benchmark an attempt to deceive the public.

• ATi used further a general optimization under the anisotropic filter for all Direct3D applications in the drivers 03.2 to 03.6, which yields an approximately 20% performance advantage under 16x anisotropic filter in the flyby benchmarks of Unreal Tournament 2003 while producing no mentionable effect in some of the other benchmarks. In other benchmarks however as well do not fasten as at all. ATi filters the base texture when anisotropic filtering only trilinear, any further textures (stages) however only bilinear. Disadvantages in the image quality apart from this bilinear/trilinear filter-mixture could not be proven. Nevertheless, the question as to whether this general optimization of the anisotropic filter does not represent an unallowed optimization in itself has to remain unanswered, since ATi's postulated "trilinear filtering" is not achieved. Also, nVidia offers here - except for the shown execption in Unreal Tournament 2003 - a normal trilinear anisotropy for all texture stages.

Generally it can be recommend not to use 3DMark for benchmarking because of the determined application specific optimizations on the part of ATi and nVidia. And, for the future in general, keep a keen eye on for the delivered image quality - especially with new drivers - and rather double-check on your screen output instead of leaving it all up to the GPU vendors.

So we are in an very tricky situation in Unreal Tournament 2003. Surely nVidia uses an unallowed optimization - this, however, degrades the static image quality only marginally, but replaced "only" the trilinear filter by a pseudo trilinear. Furthermore, this is an unallowed application specific optimization, but on the other hand this is also - roughly compared - the normal condition with ATi und anisotropic filtering, as they are replacing the trilinear filter with a bilinear/trilinear-combo by default.

The situation in Unreal Tournament 2003 is nearly the same for nVidia and ATi under anisotropic filter since both use no complete trilinear filter. Theoretically we could go on without remarking that this specific optimization is an unallowed one at nVidia and a partical unallowed at ATI. Ironically, both chip manufacturers have de-optmized the image quality so far, that they find themselves facing each other at knee-level again. nVidia has "created" an unallowed optimization which looks not worse than the ATi normal condition ;-).

As a result of this comparison, however, some other problem arose: Because without anisotropic filter ATi filters normally (purely trilinear) under Unreal Tournament 2003, while nVidia uses the pseudo trilinear filter. But other benchmarks show an disadvantage for nVidia in image quality. Because while nVidia filters outside from Unreal Tournament 2003 everywhere clearly trilinear at anisotropic filtering, there are the observed bilinear/trilinear mix at ATi in all applications. If one regards however this latter comparison as generally fair, then also the application specific optimization is not unallowed Under unreal Tournament 2003 any more, since it does in the long run only the same.

The whole thing reminds a gordian knot, which you can cut but not untie. Surely it would be the simplest to set the bilinear/trilinear anisotropic filtering of the ATi driver as unallowed optimization and to use only the pure trilinear filter offered by tools like rTool. Then we would have weapons equally (with exception Unreal Tournament 2003). But we have also to consider that most of the users take the control panel for switching the anisotropic filter and not any externat tools like rTool or aTuner.

If one makes the benchmarks only with such external tools he works past the control panel and so past to most of the users. Because of that we cannot say how we will benchmark in our upcoming high end graphics cards roudup. Maybe this desicion will come up in the discussion to this article.

Surely, it would be desirable if both graphics chip developers left the quality modes in their drivers untouched and any optimizations whether application specific or general would be build into specially modes. Nobody would likely be argueing against optimizations, which have hardly a visible quality loss and yield a performance advantage between 20 and 57 percent. If we should find such extra modes in future drivers, even only for individual games usable, we would bench this probably additionally to the pure quality mode. Only the quality modes should not use such optimizations, if both chip developers want to assume the role of outriders of image.

 

Added on August 14 2003:

Judging by a quick test, the 45.23 and 45.24 driver sets just released by nVidia don't change anything that would affect this article. The specific optimizations for Unreal Tournament 2003 we noted are without any doubt still in these drivers. The sole difference is that it's impossible - again - to reproduce Unreal Tournament 2003's filtering quality by renaming D3D AF-Tester to "ut2003.exe".

While discussing the article (Thx @ BlackBirdSR) we discovered a reasonable explanation why some benchmarks score higher on ATi cards with full trilinear filtering (set with rTool) than with the regular bilinear/trilinear filtering achieved through the control panel settings: The latter forces its filtering settings on applications, regardless of what they try to select, while rTool will only deliver full trilinear if applications don't try to select something else. In a nutshell, rTool will still allow bilinear filtering if an application explicitly requests it - the way it should be.

If however the filtering is controlled by the ATi control panel it will ignore specific application requests for bilinear and trilinear filters and always deliver the bilinear/trilinear filtering mix we've shown. Usually this leads to lower benchmark scores when using rTool because this way there's full trilinear filtering most of the time - but for applications that do request other filters this situation can reverse, as we've seen. Among these applications are both of the 3DMarks and Counter-Strike - these were exactly the ones where the rTool benchmarks finally outscored those controlled through the control panel.

Added on August 18 2003:

Regarding ATi's anisotropic filtering optimizations, we should add that in Unreal Tournament 2003, the game we selected for our tests, these can be circumvented without resorting to third party tools. To do so, anisotropic filtering must be disabled in ATi's control panel and set in the game's ini file (UT2003.ini, LevelOfAnisotropy). This approach has its caveats: First, it's common practice to control anisotropy through the control panel or tweak tools, not in games. This also allows playing with one AF level for all games. Gamers wishing to play everything at 8xAF, it would be an inconvenience to turn it off in the control panel before playing Unreal Tournament 2003 and to reenable it for other games.

Second, and more significant: ATi optimizes anisotropic filtering for all Direct3D based games. Even though there's a way around that for Unreal Tournament 2003, this won't improve the situation for the vast majority of Direct3D based games, where there simply aren't any anisotropy controls. This workaround for one game can't be regarded as a general solution for ATi anisotropy optimizations.