For example, mplayer spit out the following:
X11 error: BadAlloc (insufficient resources for operation)?,?% 0 0
Similarly, xawtv couldn't set up an overlay window due to the lack of the xv extension thus becoming completely unusable.
Based on a suggestion from someone I don't remember, I decided yesterday to replace my XFree86 4.5.0 installation with X.Org 6.9 hoping that its nv driver could work better. Guess what, it didn't.
But after the installation, and just out of curiosity, I started looking at nv's sources to see if I could discover the problem. I was hopeless to find a solution but I had to try.
First of all, I grepped for BadAlloc in the nv directory, something that quickly drove me to the NVAllocateOverlayMemory function. Based on the name, this seemed like a logic place for the failure. I added some debugging messages, reinstalled the driver and effectively this function was being called and returned a null pointer.
Some more printf's told me that the problem was being raised by xf86AllocateOffscreenLinear. "Ouch... looks as if the driver cannot allocate enough video memory for such a big resolution... may be difficult to fix", I though. Nevertheless, I continued my quest inspecting this other function and many other code in xf86fbman.c. Along the process, I discovered a minor, unrelated bug probably caused due to a pasto. Overall, the code is quite confusing if you are X-clueless, just as I am.
After a couple of hours or so I was looking at the localAllocateOffscreenLinear function. I saw that the first call to AllocateLinear was returning a null pointer, which made me think that the problem was there, continuing the inspection in that direction. That lead to nowhere.
At last, and tired of try & error cycles, I returned to the NVAllocateOverlayMemory function. I saw there that calls to xf86AllocateOffscreenLinear function were passing a 32 value, which seemed like a bpp. "Hmm... if I decrease it to, say, 24, it may need less memory and it might work." And indeed it did! That little change enabled xv again in my machine.
But the rationale behind the change was wrong. It happens to be that that parameter is not a bpp; it is a granularity. Therefore I assume that the less its value, the better. Some other greps "confirmed" this, as other drivers such as radeon are using a value of 16 (or even less) for that call.
Some time later and with xv working properly, I came back to the localAllocateOffscreenLinear function. This made me see that it was always falling back to the second case (below the "NOPE, ALLOCATING AREA" message). "Stupid me; I should have looked there before." This is part of the function:
if(gran && ((gran > pitch) || (pitch % gran))) {
/* we can't match the specified alignment
with XY allocations */
xfree(link);
return NULL;
}gran is the granularity passed in by the driver and pitch seems to be the screen's horizontal resolution. So, doing some numbers we see that pitch % gran = 1680 % 32 = 16 != 0. Voila! There it is, the real problem. Well, I think this is the problem, but I may be wrong.After all, it looks like as if the problem was a simple bug rather than something related to the card's memory. You can follow this bug for more information as well as to retrieve the proposed patch.
Edit (March 6, 21:38): It turns out my patch was not really correct. For more information just check out the bug's audit trail.
