[pvrusb2] Ability to fully reset a PVRUSB2 Device
Mike Isely
isely at isely.net
Sun Oct 13 18:15:39 CDT 2019
Diego:
I was *finally* able to reproduce the precise kernel oops you got. I
had to load the exact same Ubuntu kernel you are using and the test had
to run specifically against an HVR-1950. The older (simpler) device I
had been trying won't fail. But with that said, I got your exact call
trace.
Now that I see the signature, I immediately tested again using a 5.2.13
kernel.org vanilla kernel that is larded full of printk() statements in
the driver, again on an HVR-1950. And it blew chunks again. The
signature wasn't precisely the same (stack trace is slightly different)
but it's close enough that I believe it's the same root cause.
Now the real digging starts.
Note: This is ignoring the sysfs tear-down collision I had mentioned
earlier (which, interestingly didn't happen this time, probably because
this oops stopped the tear-down before it got that far). This is also
with the external userspace I2C access disabled so I can keep that
source of log noise out of the way, for now. So there's really 3 issues
here. Trying to focus on the one that is burning you specifically.
If it turns out that what I'm seeing in the 5.2.13 kernel is actually
different, well then that just means there are 4 problems :-( But right
now I'm betting it's the same so that's the avenue I'm going to chase.
If I run aground, then I'm going to backtrack to that specific Ubuntu
kernel and rebuild it with all my debug code added and other config
tweaks to help with chasing the problem.
-Mike
On Wed, 9 Oct 2019, Diego Rivera wrote:
> Ok so it turns out it was much easier than expected ☺
> I have a VM now which "houses" both pvrusb2 devices, so if I have to bounce anything it's now just a
> VM. I also have scripts in the host that make it easy to detect when those devices are acting up,
> and bounce the VM accordingly. My next step may be to find a computer-controllable relay of some
> sort that I can power on/off the HVR-1950's when I detect they've died, for automatic recovery.
> But that's a project I have to leave for the weekend as I've neglected work too much already 😄
> Cheers!
>
> On Tue, 2019-10-08 at 21:59 -0600, Diego Rivera wrote:
> > In the mean time, this weekend I'm going to try something that it just occurred to me was within
> > my reach: use KVM to encapsulate each device. Thus, if they bork, I only restart a "small"(ish) VM
> > instead of the whole box. I just need to figure out a way to block the host from starting up the
> > devices. Blacklisting pvrusb2 didn't seem to work, so I may have to simply remove the module
> > altogether.
> > But I'll try that in the weekend.
> > Cheers!--
> >
> >
> >
> > Diego Rivera
> >
> > On Tue, 2019-10-08 at 22:49 -0500, Mike Isely wrote:
> > > This is proving to be a multi-faceted issue.
> > > First, there's the infinite attempts at using the I2C interface from userspace when the device
> > > is unplugged. This is happening from outside the kernel and I need a means to permanently shut
> > > that up - killing the daemon sourcing this is only a workaround. But that's a sideshow.
> > > The issue I spent several days chasing involved the apparent chain of kernel oopses that happen
> > > when the device is unplugged. Turns out this is because when the driver is disconnecting itself
> > > from sysfs it's getting errors because something else outside of the driver (apparently) did all
> > > the disconnects already. This is really not right, as the driver take care to undo what it
> > > previously did when tearing down and somebody is not playing nice here. There's one oops logged
> > > for every sysfs endpoint being removed thus the reason for the cluster. Another clue here is
> > > that I'm also seeing similar failures from other V4L chip-level drivers outside of the pvrusb2
> > > driver (which are connected to the pvrusb2 driver) triggering the same problem in their own
> > > sandboxes. And a few days ago going through my old e-mail I found a discussion thread about
> > > this - last Feb/Mar with people there equally puzzled. The last post there seemed to pin the
> > > blame on the pvrusb2 driver, but I think right now the pvrusb2 driver is more of a victim than a
> > > culprit. In any case I have to find a way to stop that behavior and haven't found it yet.
> > > Unfortunately I don't think that's actually the problem you hit. That whole thing was a decoy
> > > for me. Having burned through that I believe you're hitting something else - which I haven't
> > > been able to reproduce yet. But I have the logs you sent and will be reviewing that again now
> > > in light of the above. Also, I was testing at the time with older hardware which doesn't have
> > > all the bells and whistles of the HVR-1950 - no DVB side - which meant less of the driver was
> > > involved. I had done that thinking I was already reproducing your issue and was trying to
> > > simplify the scenario. Suspecting otherwise now, I've switched to an HVR-1950 and have gone
> > > back to trying to reliably reproduce the issue.
> > > That was about 2 weeks ago. Then I got my chain yanked in 3 other directions. But I'm almost
> > > dug out of all that and will be circling back to this issue, hopefully this weekend, which is
> > > looking to be comparatively quiet.
> > > Please DO keep pinging me.
> > > -Mike
> > >
> > > On Sun, 6 Oct 2019, Diego Rivera wrote:
> > > > Hey, Mike!Any luck? Did my dumps help you out at all? Just poking to see if you had the chance
> > > > to continueworking on this.Cheers!
> > > > On Sun, 2019-09-22 at 15:04 -0500, Mike Isely wrote:
> > > > > Thank you! -MikeOn Sun, 22 Sep 2019, Diego Rivera wrote:
> > > > > > As requested--*Diego Rivera*
> > > > > > On Sun, Sep 22, 2019 at 1:53 PM Mike Isely <isely at isely.net> wrote:
> > > > > > > Ugh. This is line-wrapped to hell-and-back. Can you resend that as afile
> > > > > > > attachment? -MikeOn Sun, 22 Sep 2019, Diego Rivera wrote:
> > > > > > > > This is what kern.log shows when I hot-unplug/hot-poweroff one of thedevices:Sep 22
> > > > > > > > 13:36:05 tvserver kernel: [ 156.265825] usb 1-4: USB disconnect,device number 8Sep22
> > > > > > > > 13:36:05 tvserver kernel: [ 156.266059] pvrusb2: Device beingrendered inoperableSep
> > > > > > > > 2213:36:05 tvserver kernel: [ 156.266162] BUG: unable to handlekernel NULL
> > > > > > > > pointerdereference at 0000000000000520Sep 22 13:36:05 tvserver kernel: [ 156.266299]
> > > > > > > > #PF error:[normal kernelread fault]Sep 22 13:36:05 tvserver kernel: [ 156.266376] PGD
> > > > > > > > 0 P4D 0Sep 2213:36:05 tvserver kernel: [ 156.266424] Oops: 0000 [#1] SMP PTISep 22
> > > > > > > > 13:36:05 tvserverkernel: [ 156.266485] CPU: 0 PID: 2190 Comm:pvrusb2-context Not
> > > > > > > > tainted 5.0.0-29-generic#31-UbuntuSep 22 13:36:05 tvserver kernel: [ 156.266610]
> > > > > > > > Hardware name: To Be
> > > > > > > Filled
> > > > > > > > By O.E.M. To Be Filled By O.E.M./Q1900-ITX, BIOS P1.70 03/31/2016Sep 22 13:36:05
> > > > > > > > tvserverkernel: [ 156.266770] RIP:0010:pvr2_v4l2_internal_check+0x47/0x70
> > > > > > > > [pvrusb2]Sep 22 13:36:05tvserver kernel: [ 156.266867] Code: 2f e4 ff ff 48 8b
> > > > > > > 7b
> > > > > > > > 40 e8 26 e4 ff ff 48 8b 43 38 48 8b 90 20 05 00 00 48 05 20 05 00 00 48
> > > > > > > 39
> > > > > > > > d0 74 03 5b 5d c3 48 8b 43 40 <48> 8b 90 20 05 00 00 48 05 20 05 00 00 4839 d0 75 e7
> > > > > > > > 48 89df e8Sep 22 13:36:05 tvserver kernel: [ 156.267140] RSP:
> > > > > > > 0018:ffffb4f3c262fea0
> > > > > > > > EFLAGS: 00010246Sep 22 13:36:05 tvserver kernel: [ 156.267223] RAX: 0000000000000000
> > > > > > > RBX:
> > > > > > > > ffff9112efad8ba0 RCX: 0000000000000000Sep 22 13:36:05 tvserver kernel: [ 156.267331]
> > > > > > > > RDX:ffff9112ee80cd20
> > > > > > > RSI:
> > > > > > > > 0000000000000000 RDI: 0000000000000000Sep 22 13:36:05 tvserver kernel: [ 156.267439]
> > > > > > > > RBP:ffffb4f3c262fea8
> > > > > > > R08:
> > > > > > > > 0000000000000000 R09: ffff9112ed60c618Sep 22 13:36:05 tvserver kernel: [ 156.267546]
> > > > > > > > R10:000000000000f000
> > > > > > > R11:
> > > > > > > > 0000002462016bed R12: ffff9112ef474000Sep 22 13:36:05 tvserver kernel: [ 156.267653]
> > > > > > > > R13:ffffffffc108ba90
> > > > > > > R14:
> > > > > > > > 0000000000000000 R15: ffff9112f36ed700Sep 22 13:36:05 tvserver kernel: [ 156.267761]
> > > > > > > > FS:
> > > > > > > 0000000000000000(0000)
> > > > > > > > GS:ffff9112f8200000(0000) knlGS:0000000000000000Sep 22 13:36:05 tvserver
> > > > > > > > kernel:[ 156.267880] CS: 0010 DS: 0000 ES:
> > > > > > > 0000
> > > > > > > > CR0: 0000000080050033Sep 22 13:36:05 tvserver kernel: [ 156.267968] CR2:
> > > > > > > > 0000000000000520
> > > > > > > CR3:
> > > > > > > > 000000014820e000 CR4: 00000000001006f0Sep 22 13:36:05 tvserver kernel: [ 156.268074]
> > > > > > > > CallTrace:Sep 22 13:36:05 tvserver kernel: [ 156.268136]
> > > > > > > > pvr2_context_thread_func+0xc4/0x2b0[pvrusb2]Sep 22 13:36:05 tvserver kernel:
> > > > > > > > [ 156.268227] ? wait_woken+0x80/0x80Sep 2213:36:05 tvserver kernel:
> > > > > > > > [ 156.268290] kthread+0x120/0x140Sep 22 13:36:05 tvserverkernel:
> > > > > > > > [ 156.268362] ?pvr2_context_destroy+0xc0/0xc0 [pvrusb2]Sep 22 13:36:05
> > > > > > > > tvserverkernel: [ 156.268449] ?__kthread_parkme+0x70/0x70Sep 22 13:36:05 tvserver
> > > > > > > > kernel:[ 156.268518] ret_from_fork+0x35/0x40Sep 22 13:36:05 tvserver kernel:
> > > > > > > > [ 156.268578]Modules linked in:
> > > > > > > s5h1411
> > > > > > > > tda18271 tda8290 tuner cx25840 pvrusb2 tveeprom cx2341x dvb_corev4l2_common videodev
> > > > > > > > mediaveth xt_nat ipt_MASQUERADE xfrm_user xfrm_algobr_netfilter bridge stp llc
> > > > > > > > xt_recentipt_REJECT nf_reject_ipv4 xt_limitxt_comment xt_multiport xt_conntrack
> > > > > > > > xt_hashlimitxt_addrtype xt_markiptable_mangle xt_tcpudp xt_CT iptable_raw
> > > > > > > > nfnetlink_logxt_NFLOGnf_log_ipv4 nf_log_common xt_LOG nf_conntrack_sane
> > > > > > > > nf_conntrack_netlinknfnetlinknf_nat_tftp nf_nat_snmp_basic nf_conntrack_snmp
> > > > > > > > nf_nat_sipnf_nat_pptp nf_nat_irc nf_nat_h323nf_nat_ftp nf_nat_amandanf_conntrack_tftp
> > > > > > > > nf_conntrack_sip nf_conntrack_pptp
> > > > > > > nf_conntrack_proto_gre
> > > > > > > > nf_conntrack_netbios_ns nf_conntrack_broadcast
> > > > > > > > nf_conntrack_ircnf_conntrack_h323nf_conntrack_ftp ts_kmp nf_conntrack_amanda
> > > > > > > > iptable_natnf_nat_ipv4 nf_nat nf_conntracknf_defrag_ipv6 nf_defrag_ipv4
> > > > > > > > arc4iptable_filter bpfilter md4 cmac nls_utf8 cifs ccm fscacheaufs
> > > > > > > > overlaynls_iso8859_1 xfs libcrc32c snd_hdmi_lpe_audio snd_pcmSep 22 13:36:05
> > > > > > > > tvserverkernel: [ 156.268655] snd_seq_midisnd_seq_midi_event snd_rawmidi snd_seq
> > > > > > > > snd_seq_devicesnd_timer sndsoundcore intel_rapl intel_soc_dts_thermal
> > > > > > > > intel_soc_dts_iosfintel_powerclampcoretemp kvm_intel punit_atom_debug i915 joydev
> > > > > > > > kvmgtcrct10dif_pclmul input_leds vfio_mdevmdev crc32_pclmul
> > > > > > > > vfio_iommu_type1ghash_clmulni_intel cryptd vfio intel_cstate kvm
> > > > > > > > irqbypassdrm_kms_helperdrm hci_uart i2c_algo_bit fb_sys_fops btqca mei_txe
> > > > > > > > syscopyareabtrtlsysfillrect mei sysimgblt btbcm btintel bluetooth ecdh_generic
> > > > > > > rfkill_gpio
> > > > > > > > mac_hid sch_fq_codel ip_tables x_tables autofs4
> > > > > > > > hid_logitech_hidpphid_logitech_djhid_generic usbhid r8169 ahci lpc_ich i2c_i801
> > > > > > > > libahcirealtek i2c_hid video hidSep 2213:36:05 tvserver kernel: [ 156.270831] CR2:
> > > > > > > > 0000000000000520Sep 22 13:36:05 tvserverkernel: [ 156.270891] ---[ end
> > > > > > > > trace5d13378174849ef9 ]---Sep 22 13:36:05 tvserver kernel:[ 156.270988]
> > > > > > > > RIP:0010:pvr2_v4l2_internal_check+0x47/0x70 [pvrusb2]Sep 22 13:36:05 tvserverkernel:
> > > > > > > > [ 156.271089] Code: 2f e4 ff ff 48 8b
> > > > > > > 7b
> > > > > > > > 40 e8 26 e4 ff ff 48 8b 43 38 48 8b 90 20 05 00 00 48 05 20 05 00 00 48
> > > > > > > 39
> > > > > > > > d0 74 03 5b 5d c3 48 8b 43 40 <48> 8b 90 20 05 00 00 48 05 20 05 00 00 4839 d0 75 e7
> > > > > > > > 48 89df e8Sep 22 13:36:05 tvserver kernel: [ 156.271363] RSP:
> > > > > > > 0018:ffffb4f3c262fea0
> > > > > > > > EFLAGS: 00010246Sep 22 13:36:05 tvserver kernel: [ 156.271447] RAX: 0000000000000000
> > > > > > > RBX:
> > > > > > > > ffff9112efad8ba0 RCX: 0000000000000000Sep 22 13:36:05 tvserver kernel: [ 156.271556]
> > > > > > > > RDX:ffff9112ee80cd20
> > > > > > > RSI:
> > > > > > > > 0000000000000000 RDI: 0000000000000000Sep 22 13:36:05 tvserver kernel: [ 156.271665]
> > > > > > > > RBP:ffffb4f3c262fea8
> > > > > > > R08:
> > > > > > > > 0000000000000000 R09: ffff9112ed60c618Sep 22 13:36:05 tvserver kernel: [ 156.271773]
> > > > > > > > R10:000000000000f000
> > > > > > > R11:
> > > > > > > > 0000002462016bed R12: ffff9112ef474000Sep 22 13:36:05 tvserver kernel: [ 156.271882]
> > > > > > > > R13:ffffffffc108ba90
> > > > > > > R14:
> > > > > > > > 0000000000000000 R15: ffff9112f36ed700Sep 22 13:36:05 tvserver kernel: [ 156.271990]
> > > > > > > > FS:
> > > > > > > 0000000000000000(0000)
> > > > > > > > GS:ffff9112f8200000(0000) knlGS:0000000000000000Sep 22 13:36:05 tvserver
> > > > > > > > kernel:[ 156.272111] CS: 0010 DS: 0000 ES:
> > > > > > > 0000
> > > > > > > > CR0: 0000000080050033Sep 22 13:36:05 tvserver kernel: [ 156.272201] CR2:
> > > > > > > > 0000000000000520
> > > > > > > CR3:
> > > > > > > > 000000014820e000 CR4: 00000000001006f0Sep 22 13:36:10 tvserver kernel: [ 161.084276]
> > > > > > > > usb 1-4: new high-speed
> > > > > > > USB
> > > > > > > > device number 9 using xhci_hcdSep 22 13:36:10 tvserver kernel: [ 161.236211] usb 1-4:
> > > > > > > > NewUSB devicefound, idVendor=2040, idProduct=7501, bcdDevice= 8.00Sep 22 13:36:10
> > > > > > > > tvserverkernel: [ 161.236349] usb 1-4: New USB devicestrings: Mfr=1, Product=2,
> > > > > > > > SerialNumber=3Sep22 13:36:10 tvserver kernel: [ 161.236458] usb 1-4: Product:
> > > > > > > > WinTVSep 22 13:36:10 tvserverkernel: [ 161.236516] usb 1-4: Manufacturer:HauppaugeSep
> > > > > > > > 22 13:36:10 tvserver kernel:[ 161.236584] usb 1-4: SerialNumber:7300-00-F080EDCFSep
> > > > > > > > 22 13:36:10 tvserver kernel:[ 161.239374] pvrusb2: Hardwaredescription: WinTV HVR-
> > > > > > > > 1950 Model 751xxCheers!--*Diego Rivera*
> > > > > > > > <
> > > > > > > https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail&utm_term=icon
> > > > > > > > Virus-free.www.avast.com<
> > > > > > > https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail&utm_term=link
> > > > > > > > <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>On Sun, Sep 22, 2019 at 12:42 PM Mike Isely <
> > > > > > > > isely at isely.net> wrote:
> > > > > > > > > On Sun, 22 Sep 2019, Mike Isely wrote:
> > > > > > > > > > On Sun, 14 Apr 2019, Diego Rivera wrote:
> > > > > > > > > > > Guinea pig #1 ready, sir! 😂--Diego Rivera
> > > > > > > > > >
> > > > > > > > > > Diego:Going back over this thread and comparing my recent notes, there's agood
> > > > > > > > > > experiment I'dlike you to try: Get the hardware into a statewhere you get the
> > > > > > > > > > "Attempted to executecontrol transfer when device
> > > > > > > not
> > > > > > > > > > ok" infinite log spew. Once you've confirmed the scenario again,
> > > > > > > reboot
> > > > > > > > > > the host and then rename the ir-kbd-i2c.ko module to something whichdisables
> > > > > > > > > > it. Youcan find this module in the following path:/lib/modules/`uname
> > > > > > > > > > -r`/krtnrl/drivers/media/i2c/
> > > > > > > > >
> > > > > > > > > Typo correction:/lib/modules/`uname -r`/kernel/drivers/media/i2c/(fingers in wrong
> > > > > > > > > position on keyboard, apparently)
> > > > > > > > > > A good thing to do would be to just add "-disabled" to the end of thefile
> > > > > > > > > > name. Thenrun "depmod -a" to rebuild the module dependencies(should take a few
> > > > > > > > > > seconds) and nowthe ir-kbd-i2c module will bedisabled. On the off-chance that it
> > > > > > > > > > has already beenloaded, also
> > > > > > > run
> > > > > > > > > > "modprobe -r ir_kbd_ic2" (or just reboot again). NOW, run that samescenario where
> > > > > > > > > > youget the log spew as mentioned above. Is that
> > > > > > > still
> > > > > > > > > > happening? Also, if it isn't still happening, does "modprobe -rpvrusb2" still
> > > > > > > > > > getstuck?The reason I ask is because that's what I am seeing here. Thatir-kbd-i2c
> > > > > > > > > > here is thesource of the endless stream of failing I2Crequests into the pvrusb2
> > > > > > > > > > driver. I want tomake sure we're looking
> > > > > > > at
> > > > > > > > > > the same bug. I've got roughly 3 misbehaviors on my plate right now.This is one
> > > > > > > > > > ofthem.There was an earlier mention of a kernel panic when trying to remove
> > > > > > > the
> > > > > > > > > > pvrusb2 driver from the system. While I am seeing kernel oopses fromthis - due to
> > > > > > > > > > sysfsdoing something unexpected - it is not panicing.So I have not yet seen that
> > > > > > > > > > specificproblem. I'd like to know whatexact kernel was being run (distro / uname
> > > > > > > > > > -r output /.config wouldhelp too). -Mike--Mike Iselyisely @ isely (dot) netPGP:
> > > > > > > > > > 03 54 43 4D 75 E5 CC 92 71 16 01 E2 B5 F5
> > > > > > > > > > C1E8_______________________________________________pvrusb2 mailing
> > > > > > > > > > listpvrusb2 at isely.net
> > > > > > > > > > http://www.isely.net/cgi-bin/mailman/listinfo/pvrusb2
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > --Mike Iselyisely @ isely (dot) netPGP: 03 54 43 4D 75 E5 CC 92 71 16 01 E2 B5 F5
> > > > > > > > > C1E8_______________________________________________pvrusb2 mailing
> > > > > > > > > listpvrusb2 at isely.net
> > > > > > > > > http://www.isely.net/cgi-bin/mailman/listinfo/pvrusb2
> > > > > > > > >
> > > > > > > > _______________________________________________pvrusb2 mailing listpvrusb2 at isely.net
> > > > > > > > http://www.isely.net/cgi-bin/mailman/listinfo/pvrusb2
> > > > > > > >
> > > > > > >
> > > > > > > --Mike Iselyisely @ isely (dot) netPGP: 03 54 43 4D 75 E5 CC 92 71 16 01 E2 B5 F5
> > > > > > > C1E8_______________________________________________pvrusb2 mailing listpvrusb2 at isely.net
> > > > > > > http://www.isely.net/cgi-bin/mailman/listinfo/pvrusb2
> > > > > > >
>
--
Mike Isely
isely @ isely (dot) net
PGP: 03 54 43 4D 75 E5 CC 92 71 16 01 E2 B5 F5 C1 E8
More information about the pvrusb2
mailing list