[pvrusb2] xawtv fix [was: firmware problems it seems]

[Mike Isely]

On Mon, 10 Apr 2006, Mike Isely wrote:

> On Mon, 10 Apr 2006, xavier.gnata at free.fr wrote:
>
>> Hi,
>>
>> It is a very stange bug indeed.
>> Can you figure out why xawtv fails with this 0x01 ?
>> It should be fine to know which xawtv call is involved in this problem.
>> Is it a "sanity check" performed by xawtv and not by the other players?
>> I'm going to investigate...
>
> You're only looking at the entrance to the rabbit hole, Alice.  I spent a
> few hours last night crawling into that hole and have learned additional
> clues.  I don't have time right now for the full explanation, but I can
> tell you this: (1) The latest xawtv snapshot (mid-March 2006) exhibits the
> same behavior, (2) There really _is_ a difference in the mpeg stream data
> (yet mplayer continues to work), and (3) according to the ivtv
> documentation for that cx23416 command, there should not even be a second
> argument provided.  (Note that call you see has 2 real arguments; of the 4
> function parameters there, 1 is the command, the second is the number of
> arguments, and then the last 2 are the actual arguments.)
>
>   -Mike

Here's some more info about what is going on here with this problem...

The cx23416 chip in the device is the central part that does mpeg encoding 
of the video & audio data.  This is arguably probably the most complex 
part of the device (and I might add, probably the major source of heat 
emissions).  It is configured through a series of software commands that 
are sent to it during initialization and whenever streaming is started. 
Unfortunately this part is NDA-shrouded so open source folks don't really 
have a complete authoritative spec for how this part plus its firmware 
behaves.  The ivtv project has learned quite a bit about it, and you can 
see what has been learned by examining the file fw-encoder-api.txt in the 
kernel source tree under Documentation/video4linux/cx2341x/.  (Actually 
that might not be in the kernel tree yet - I haven't checked - but it is 
checked into the V4L repository.)

The commands that the pvrusb2 driver sends to this part were originally 
reverse-engineered from snooped USB stream data involving the Windows 
driver.  When I took this driver over, the original source code just had a 
sequence of calls that basically parroted back the bits found from that 
USB snoop data.  One of the things I did was to re-map those calls using 
the symbolic names from the ivtv driver and then subsequently with 
symbolic names from the afore-mentioned document.  Unfortunately the 
mapping isn't perfect; there are some things that the pvrusb2 driver is 
sending that aren't in that document or otherwise disagree with it, but 
I'm sending it that way anyway since that is what the Windows driver is 
doing.

Fast forward now to about a month ago when I was in the midst of getting 
the new 24xxx model series hardware working (which I might add is still an 
unfinished effort).  Using another snoop log gathered from _that_ Windows 
driver, I found several differences in the sequence of commands being 
sent.  One of them was for the command CX2341X_ENC_SET_OUTPUT_PORT, which 
apparently has to do with telling the cx23416 how/where to emit its data 
stream.  According to the fw-encoder-api.txt document, this command has a 
single integer argument which specifies the output pathway.  But in the 
pvrusb2 case, we're sending two integer arguments, and in the case of the 
new hardware, that second argument has changed.  When I discovered this, I 
modified the driver to update that second argument accordingly and sure 
enough that single change suddenly got video streaming working on the new 
hardware.  The very next thing I did was to test the same change on the 
old hardware.  I used mplayer to verify the result and after seeing that 
it worked, I committed that change as a common change (i.e. not dependant 
on hardware type) to the driver.

It was after that point that xawtv broke.  Sunday night I walked back 
through my svn commits and found that the specific change which busted 
xawtv was in fact this change.  I reverted that one change and xawtv 
started working again.  Unfortunately this leaves a major issue: The 
change is definitely needed for the new hardware but it breaks xawtv. 
And of course mplayer mysteriously is not sensitive to this change.  And 
just to deepen the mystery, the specific change has to do with a magic 
second argument to a command that is only known to take one argument - 
leaving therefore no clue at all as to what this means.  Oh, and the 
command has to do with an internal hardware configuration so why should 
xawtv care?

See what I mean about the rabbit hole here?

Most of my time Sunday night was spent trying to understand how xawtv 
could be sensitive to this.  I sprinkled some debug code into xawtv's mpeg 
parser and found the reason: When xawtv initializes streaming, the first 
thing it apparently does is grab a big chunk of mpeg data and then 
proceeds to search it for various packet types.  I'm guessing that it is 
trying to discover details about the mpeg stream in order to configure 
itself.  If a searched-for packet type is not found, then xawtv gives up. 
And that's what was happening.  It was looking for a packet of type 0xc0, 
and when that CX2341X_ENC_SET_OUTPUT_PORT magic second argument was set 
for working with new hardware, then that packet type was missing from the 
stream.  I have saved mpeg stream snippets in both scenarios and best I 
can tell is that the type really is missing.  This of course begs the 
question that if data is really missing, then how is mplayer working?  I 
don't have an answer for that - and I even went back and carefully 
inspected mplayer verbose log output in an attempt to see if it might be 
hiding a clue as to how it is working.  But there was no difference.

So that's where things stand right now.  I can undo the change and get 
xawtv working again, but that will break the new hardware.  And even with 
this xawtv-breaking change in the driver, mplayer (and I think MythTV) 
still don't have any problem with the stream.  So there must be another 
answer to this.

I'm still digging further on this.  Anyone with mpeg stream format 
experience is welcome to suggest something here.  In the mean time I'm 
going to try to teach myself some more info about mpeg standards...

   -Mike


-- 
                         |         Mike Isely          |     PGP fingerprint
      Spammers Die!!     |                             | 03 54 43 4D 75 E5 CC 92
                         |   isely @ pobox (dot) com   | 71 16 01 E2 B5 F5 C1 E8
                         |                             |