PyBoard USB mass storage bug?

[vext01]

Hi,

I've noticed that unmounting a pyboard from an OpenBSD host sometimes hangs.

To reproduce:

1) mount /dev/sd1i /vol/usb
2) sudo touch /vol/usb/somefile
3) sudo umount /vol/usb

If you omit step 2, the device will unmount fine.

I had a prod around in the OpenBSD kernel and turned on some debugging flags and I have an idea as to why we see the hang:

If we look in 'lsusb -v', we see that the micropython usb mass storage device is advertising that it supports SCSI:

Code: Select all

Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass          239 Miscellaneous Device
  bDeviceSubClass         2 ?
  bDeviceProtocol         1 Interface Association
  bMaxPacketSize0        64
  idVendor           0xf055 
  idProduct          0x9800 
  bcdDevice            2.00
  iManufacturer           1 Micro Python
  iProduct                2 Pyboard Virtual Comm Port in FS Mode
  iSerial                 3 000000000011
  bNumConfigurations      1
  Configuration Descriptor:
...
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           2
      bInterfaceClass         8 Mass Storage
      bInterfaceSubClass      6 SCSI
      bInterfaceProtocol     80 Bulk-Only
      iInterface              0 
...

As a result OpenBSD talks SCSI to the pyboard.

Here is the debugging output of a hanging unmount:

Code: Select all

Apr 26 22:35:28 wilfred /bsd: sd1(umass0:1:0): xs  (0xffffff039eca6b48): flg(0x100)sc_link(0xffff800000a59900)retr(0x4)timo(0x186a0)data(0x0)res(0x0)err(0x0)bp(0x0)
Apr 26 22:35:28 wilfred /bsd: sd1(umass0:1:0): cmd (0xffffff039eca6bb3): 35,0,0,0,0,0,0,0,0,0-[0 bytes]
Apr 26 22:35:28 wilfred /bsd: umass0: umass_scsi_cmd: at 1430084128.770897: 1:0 xs=0xffffff039eca6b48 cmd=0x35 datalen=0 (quirks=0x400e, poll=0)
Apr 26 22:35:28 wilfred /bsd: umass_scsi_cmd: async dir=0, cmdlen=10 datalen=0
Apr 26 22:35:28 wilfred /bsd: umass0: umass_bbb_transfer cmd=0x35
Apr 26 22:35:28 wilfred /bsd: umass0: CBW 78: cmdlen=10 (0x35000000000000000000), data = 0 bytes, dir = out
Apr 26 22:35:28 wilfred /bsd: umass0: start xfer buffer=0xffff800000a57694 buflen=31 flags=0x0 timeout=105000
Apr 26 22:35:28 wilfred /bsd: umass0: Handling BBB state 1 (BBB CBW), xfer=0xffffff041cfa7630, NORMAL_COMPLETION
Apr 26 22:35:28 wilfred /bsd: umass0: no data phase
Apr 26 22:35:28 wilfred /bsd: umass0: start xfer buffer=0xffff800000a576b3 buflen=13 flags=0x0 timeout=105000
Apr 26 22:35:28 wilfred /bsd: umass0: Handling BBB state 4 (BBB CSW, 1st attempt), xfer=0xffffff041cfa7948, STALLED
Apr 26 22:35:28 wilfred /bsd: umass0: Failed to read CSW, STALLED, retrying
Apr 26 22:35:28 wilfred /bsd: umass0: Clear endpoint 0x81 stall
Apr 26 22:35:28 wilfred /bsd: umass0: start ctrl xfer buffer=0x0 buflen=0 flags=0x0
Apr 26 22:35:28 wilfred /bsd: umass0: Handling BBB state 5 (BBB CSW bulk-in clear stall), xfer=0xffffff041cfa7b58, NORMAL_COMPLETION
Apr 26 22:35:28 wilfred /bsd: umass0: start xfer buffer=0xffff800000a576b3 buflen=13 flags=0x0 timeout=105000

This shows a SCSI 0x35 command stalling and hanging. This command is SYNCHRONIZE_CACHE command.

If I add this scsi quirk to the OpenBSD kernel, the pyboard no longer hangs on unmount:

Code: Select all

Index: scsiconf.c
===================================================================
RCS file: /home/edd/cvsync/src/sys/scsi/scsiconf.c,v
retrieving revision 1.191
diff -u -p -r1.191 scsiconf.c
--- scsiconf.c  14 Mar 2015 03:38:52 -0000      1.191
+++ scsiconf.c  27 Apr 2015 16:51:14 -0000
@@ -674,6 +674,10 @@ const struct scsi_quirk_inquiry_pattern 
          "CD-ROM  CDR-N16", "", "1.25"},        ADEV_NOCAPACITY}, /* Sanyo */
         {{T_CDROM, T_REMOV,
          "UJDCD8730", "", "1.14"},              ADEV_NODOORLOCK}, /* Acer */
+
+       /* micropython */
+       {{T_DIRECT, T_REMOV,
+        "uPy", "microSD Flash", "1.00"},         SDEV_NOSYNCCACHE},
 };

The quirk tells the kernel that the device does not support SCSI SYNCHRONIZE_CACHE.

I was then able to copy files and unmount etc. I had a couple of cases of corrupted files, but since I did a factory reset of the pyboard, I have not seen that again. Could be another issue if it comes back...

Is it possible that the pyboard does not support SYNCHRONIZE_CACHE? Can this be fixed in the pyboard firmware? Whilst we could commit that quirk to OpenBSD, we would rather not.

(not ruling out a bug in the OpenBSD SCSI layer at this point)

Thanks!

[dhylands]

I think that this is the list of supported SCSI commands:
https://github.com/micropython/micropyt ... #L122-L176

and I don't see SYNCHRONIZE_CACHE listed.

Due to micropython and the host both accessing the storage at the same time, its always possible to have some corruption (especially if micropython writes to the filesystem while it's mounted on the host).

[vext01]

Right, and this is ST micro library code that is ends up in the pyboard firmware?

We could fix it, right? But ideally the bug report needs to go upstream to ST micro...

[dhylands]

We've made patches to these files before. There was an issue wih the way the StartStop command was being sent on the Mac and we had to make changes. The history seems to be lost due to a file directory change.

Reporting upstream is ideal, although personally, I'm not sure how we would even go about that. And we certainly don't want to wait for the change (that could take years).

As long as it doesn't break other platforms, the general sentiment is patches welcome.

[Damien]

Yes, we can add the SYNCHRONIZE_CACHE command ourselves, as long as it doesn't break existing functionality.

[vext01]

Ok, great.

So my next question is, should this command actually do anything? Do you have any cache that we would like to synchronise? Or should the command be a NOP?

Cheers

[dhylands]

Internal flash definitely has a cache. I don't recall whether the physical sdcards do or not.

The proper way would be to add a function to the fops structure:
https://github.com/micropython/micropyt ... .h#L49-L60

and register functions here:
https://github.com/micropython/micropyt ... #L177-L188

that performs the sync. You'll probably call storage_flush()
https://github.com/micropython/micropyt ... #L175-L177

[vext01]

Thanks for the pointers. Once I find some time, I will try hacking these changes.