$Id: eeprom.html 1134 2006-04-29 20:22:35Z isely $
Mike Isely <isely at pobox dot com>
This page describes issues involving the pvrusb2 driver's use of the tveeprom module that is both a part of Vidoe4Linux and ivtv. Unfortunately these two implementations are somewhat divergent and can cause problems when used with the pvrusb2 driver distribution.
If you find you are having problems that involve tveeprom.ko or are experiencing problems involving the correct determination of your PVR USB2 device's configuration, then you might be getting victimized by issues with tveeprom. You probably should read everything below before asking any questions...
You can find the main pvrusb2 driver page on the web at http://www.isely.net/pvrusb2/pvrusb2.html.
The PVR USB2 device contains within it a block of data that helps drivers determine how best to control the device. This information, typically held inside a I2C-accessed PROM device, needs to be retrieved by the driver during initialization. Information contained therein can include any of:
In the case of the pvrusb2 driver, we're specifically interested in the tuner type, set of support video standards (there may be more than one), and the device's serial number. We can probably live without the serial number or supported video standards (just assume them all), but we really really need that tuner type. Hauppauge seems to like changing out tuners within a product without telling anyone, and we have to be able to recognize and handle that situation.
It turns out that with apparently all Hauppauge TV tuner products, a common data format is used within all embedded PROMs. Thus it becomes possible to write a parser for that data a single time and have it be useful to many different drivers. That is indeed the case within Video4Linux; it provides a module known as tveeprom.ko which parses Hauppauge PROM data. So, for the pvrusb2 driver to retrieve data from the device's PROM, it's just a simple matter of using tveeprom.ko to do the work, right? Well read on...
The tveeprom.ko module (we'll just call it tveeprom from here on out) is designed to operate as a typical I2C client driver within Video4Linux. When loaded into the kernel, it will register itself as a chip driver, and as I2C buses are registered it will check each bus for the presence of a PROM and if found it will attach itself to that bus and make available its command interface for any interested parties. When asked via that command interface, tveeprom will read in the contents of the associated PROM, parse the contents, and hand back the results to the caller in a structure.
The tveeprom module, in an ideal world, is really doing 3 things here:
In an ideal world, those three things will always work perfectly and will never need to change...
But things are not this simple. A number of factors seriously complicate the operation of this module. First and most aggravating is that incompatible variations of this module exist in v4l and ivtv - how do we use tveeprom in pvrusb2 if we're not sure which module has been loaded? Another problem is that while tveeprom does a great job of parsing the PROM contents, it doesn't do so well at detecting the PROM nor correctly reading the area that needs to be parsed. A number of issues like this conspire to make it very difficult to reliably use this module.
For starters, there is more than one tveeprom floating around in Linux - there's one version provided by v4l and another version provided by the out-of-tree ivtv driver. While the two versions may have a common ancestor, they are forked code and have recently diverged. (Yes, I know that there is an effort underway to bring this back together, but that doesn't change what I've recently observed.)
The pvrusb2 driver does not really want to get into the business of implementing a third variation of this module, nor does it want to re-implement what has already been done elsewhere. Thus the pvrusb2 driver is designed to (attempt to) work with either variant of tveeprom. Being able to stay compatible with both variants is proving to be quite a challenge.
Note: the tveeprom.ko built as part of pvrusb2 is not another variant. It is an exact mirror of tveeprom.c taken from a recent snapshot of ivtv. I have hit situations in the past where the v4l implementation of tveeprom causes problems (e.g. not correctly mapping a particular tuner type). Rather than require everyone to then also find and install ivtv, the pvrusb2 driver snapshot contains copies of a few modules (like tveeprom) from ivtv. These files are unmodified, and kept up-to-date whenever possible.
Earlier it was mentioned that tveeprom really does 3 things, detection, retrieval, and parsing. That third thing is exported outside of tveeprom with a direct subroutine entry point. Its prototype is basically this:
void tveeprom_hauppauge_analog(struct tveeprom *tvee,unsigned char *eeprom_data);
(Starting with the v4l variant kernel 2.6.14, this function has changed to expect an additional struct i2c_client pointer as the first argument, with the other arguments shifted down. The implementation uses this pointer to emit additional information when printing to the log.)
By calling this entry point directly rather than using the I2C command interface, it becomes up to the user to detect the PROM and read the appropriate contents. The results are returned inside of the tveeprom structure, which provides more information than would otherwise be possible with the I2C command interface method. Using this direct call, we can retrieve the device's stored serial number, which can't be had any other way.
For all further discussion, we'll refer to the I2C command interface way of retrieving data as the "indirect method" and this direct call means of retrieving data as the "direct method". (Note: as of 20051231, the indirect method is completely broken right now so don't even bother trying it...)
There is however a major downside to the direct method: It is an unstable API. Though both v4l and ivtv variations of tveeprom provide this method, they use different and incompatible definitions for struct tveeprom. This makes it very difficult to compile a driver that can use either variation of tveeprom - which struct tveeprom definition do you use?
The pvrusb2 driver, not wanting to foreclose any choices, attempts to support both indirect and direct methods using either variation of tveeprom available. This does not work well in every combination. In fact right now, the indirect method does not even work at all for your author here (and as of the 20051231 snapshot the indirect method is currently broken anyway), and the direct method requires an ugly, disgusting, sickening hack in order to operate correctly.
The choice of indirect versus direct method is determined by the PVR2_EEPROM_INDIRECT macro (or PVRUSB2_EEPROM_DIRECT_CALL macro for snapshots older than 20051113), which is used within pvrusb2-eeprom.c. Based on this macro, one of two versions of the function pvr2_eeprom_analyze() are compiled into the pvrusb2 driver. As shipped, the direct method is selected by default (since it is the method that actually works for all known cases). If you want to try the indirect method, you'll need to edit pvrusb2-eeprom.c to enable or disable the macro as appropriate, and then rebuild the driver.
Actually, post 20051113, the above is not quite exactly true. There are now three versions of pvr2_eeprom_analyze() implemented. The direct method now has two implementations. They are selected through the use of the PVR2_EEPROM_IVTV macro. Basically, the extra version comes into play when this macro is not defined; it doesn't do any of the hackish junk described below. It instead assumes that it is always always working with a V4L-provided tveeprom.ko, and is intended that this non-hacked version will find its way into a V4L resident version of this driver. For the sake of everything discussed below, consider that PVR2_EEPROM_IVTV is defined.
Probably the best way to describe the problems here is just to walk through the choices and see where that leads us. Have your painkiller ready...
Clearly, the ideal method should be the ideal choice, and that method appears to be the indirect method. Why go to all the work to detect and read the PROM when the tveeprom module is clearly designed to do this for you? It's supported by both tveeprom variants and should work the same in both cases. This has in fact been implemented in the function pvr2_eeprom_analyze() in pvrusb2-eeprom.c within the driver but left disabled. Here's what should happen if you enable it:
What really happens unfortunately is quite a bit different...
At step 2, tveeprom fails to detect the PROM. This is a problem on my PVR USB2 at least. An investigation of the cause suggests that the PROM in my device simply won't ack an I2C write of zero length (this is the type of operation which the I2C core uses to detect connected devices). My knowledge of I2C says that the device actually has to ack before the transfer length is known so this should be impossible. But there it is. My suspicion is that the "proxy" I2C implementation that we operate in the device's 8051 controller may be causing a problem here, but changing that firmware is out of the scope of this driver, so I have to accept that we can't detect the device. This problem can however be worked around by providing options to the tveeprom module to force recognition of the PROM part. Specifying for example "force=-1,80" will get past this problem.
At step 3, reading of the entire PROM fails because tveeprom tries to read 256 bytes in one single operation - but the I2C adapter proxy implementation over USB to the 8051 can't transfer that much data at a time. I've worked around this in the I2C adapter driver within the pvrusb2 driver by fragmenting oversized transfers into smaller pieces. However this really isn't clean since the fragments become separate transfers on the bus, and that might cause other problems. But for this purpose fragmenting the transfers gets around the problem.
At step 3, parsing of the entire PROM's contents fails because it appears that in the PVR USB2 device's PROM data, only the upper 128 bytes contain valid data. Unfortunately, the parsing algorithm in tveeprom just gives up as soon as it sees what it thinks is corrupted data. There is no work-around for this. Since tveeprom in this method is controlling the actual PROM reading operation, there is no way to limit what it reads.
At step 4, the data provided by the official command interface doesn't actually provide all of the data. There is no way to use this interface to retrieve the device's serial number.
I have seen two reports on the pvrusb2 mailing list that suggest that detection probably does work for some people. Perhaps it depends on the particular PROM chip that Hauppauge has used on a particular production run.
I have also modified the driver to not require a device serial number - sysfs needed it to generate a class path but if there is no serial number then a unit number will be used instead.
I have also modified the driver to handle large I2C transfers by fragmenting such large blocks into smaller transfers.
Given all of the above, then the only issue really standing in the way of using the "indirect" method here is that of accidentally trying to parse the lower 128 bytes of the PROM. I suppose it's conceivable that there are PVR USB2 devices out there which might have fully parseable PROM contents. If you think you have such a device, then you can probably try enabling the indirect method in pvrusb2-eeprom.c, rebuilding, and everything should work.
Otherwise, the indirect method just isn't an option for this driver.
By the way, if you see messages like this in your system log:
tveeprom 2-0050: Encountered bad packet header [c2]. Corrupt or not a Hauppauge eeprom.
yet the device works just fine, then you're probably seeing tveeprom attempting the indirect method on its own and failing. That's a strong hint that the pvrusb2 driver will stop correctly reading the PROM if you try to switch to the indirect method.
It should be pretty clear from the preceding section that using the normal I2C command interface "indirect" method to retrieve device configuration data just isn't going to work for the pvrusb2 driver. It's a shame too, because the indirect method is a stable interface and it "works" the same way regardless of which tveeprom variant has actually been loaded into the kernel.
The showstopper problem with the indirect method is that it is reading too much of the PROM. In the case of the PVR USB2, only the upper 128 bytes seem to contain useful data. Thus we have to resort to the "direct" method for data retrieval.
Unfortunately the direct method has its own problems. Unlike the indirect method, the direct method's interface is not stable. Currently (as of kernel 2.6.14 and ivtv 0.4.0) that critical struct tveeprom, which defines all the fields we can retrieve, is radically different between the two variants of the tveeprom module. In fact, in the ivtv variant, the struct is completely private to the tveeprom.c module source file.
In fact, if your kernel is built with CONFIG_MODVERSIONS enabled, the above mentioned interface incompatibility causes an nastier problem, which you'll know you have if you see a message like "pvrusb2 disagrees about version of symbol tveeprom_hauppauge_analog" in your system log. Rather than repeat the explanation for that here, please see the FAQ item that discusses this issue.
Another incompatible change was made to tveeprom_hauppauge_analog() in kernel 2.6.14 - an argument was added. There is a new first argument to the function which is the i2c_client structure that had been used to retrieve the PROM contents. (All that tveeprom_hauppauge_analog() uses this for is to enhance the printk's going to the kernel log.) Fortunately, ivtv 0.4.0's version of this function will also expect this argument if compiled under kernel 2.6.14 so the impact of that change is lessened somewhat.
The pvrusb2 driver's implementation for the direct method (also implemented in the function pvr2_eeprom_analyze() in pvrusb2-eeprom.c) doesn't have to detect the PROM since it knows it has to be there. It will also only read the upper 128 bytes of PROM data, thus avoiding the issue that kills the indirect method. But that so far is just piker stuff....
If the pvrusb2 driver is compiled for kernel 2.6.14, then pvr2_eeprom_analyze() will assume the need for that extra argument, and temporarily fake up an i2_client structure so that tveeprom_hauppauge_analog() in v4l will continue to work properly (the ivtv tveeprom implementation doesn't care).
That leaves just one problem, the worst problem: the conflicting tveeprom structure defintion. We really want to be able to support either tveeprom variant, and we really want that to work at run time not compile time. Based on whichever tveeprom is loaded into the kernel, we need to somehow know which "tveeprom" struct definition to use in order to communicate with that module's data parsing code. After a lot of thought, I came up with something completely sick, twisted, and disgusting. But it works (well, only when CONFIG_MODVERSIONS is not enabled in the kernel). This is what is implemented in pvr2_eeprom_analyze() in pvrusb2-eeprom.c when the default direct method is compiled:
Logically what is going on here is that since we don't know which structure definition to use, we'll just use them both. They are both being simultaneously passed in to tveeprom_hauppauge_analog() as members of a union. Before making that function call, we seed the data structure so we can tell what's been changed afterwards. By noting the changes afterwards, we can make a reasonable guess as to which tveeprom definition tveeprom_hauppauge_analog() has assumed and then act on the data accordingly.
This is ugly, repulsive, and revolting. However I see no other way short of telling everyone that either (1) this driver can't be used concurrently with ivtv, or (2) one must replace tveeprom.c in the kernel (or ivtv) with ivtv's version (or the kernel's version) before anything is going to work. (And of course as said earlier, even this hack won't work if you have CONFIG_MODVERSIONS enabled in your kernel.)
Clearly the tveeprom module is useful to drivers outside of ivtv, and to that end, there really only should be a single official version of it in the Video4Linux core within the kernel sources. Not having an official version causes the implementation to diverge over time, which is the principle cause of that disgusting hack described above with the direct method.
Even with a single official tveeprom version, the (apparent) official indirect API is insufficient. I'm not sure what the right answer here is. Certainly the indirect API should include commands to retrieve other bits of data like the device serial number. But even with that in place, device detection can be difficult, and the module is going to have to deal with situations where apparently not all of the PROM contents contains parseable data.
If it is not possible to make the indirect interface work for everyone, then the direct interface needs to be stabilized.
Hopefully everything stated here should help you to understand the issues behind reading of the PVR USB2 device's PROM and thus maybe hopefully will help you to help me work out any further issues that come up in this area of the driver...
Mike Isely