Electron (USB)Serial seems to be dropping bytes? (SOLVED: GNU screen is the problem)

Troubleshooting
1

Hi folks – I’m trying to get an Adafruit VC0706 camera working with a Particle Electron. This camera works with the official Windows-based tool but I cannot retrieve a valid image off of it via the electron. My setup is basically the camera is connected to Serial1, the electron receives those bytes and prints them to Serial, which is connected to my computer where I save them in a file. The JPEGs always have some sort of error preventing them from being read, usually a bogus DQT value of some sort.

Before the photo is downloaded, a message is printed to the Serial console that says “wait to fetch 47044 byte image …” or so. I noticed that the actual data received is about 300-400 bytes shorter than its stated length.

The image data is read in 32-byte chunks from Serial1 and then printed to Serial with Serial.write(). As a debugging measure, I started to print the value returned from Serial.write() (which is the number of bytes it claims to have written) alongside each data chunk, so that my file looks like this:
(data chunk)WRITTEN: 32(data chunk)WRITTEN: 32(data chunk)WRITTEN: 32 …

Serial.write() prints that it wrote 32 bytes every single time. That is often accurate, but many times the actual size of the data chunk is lower, usually by 1 byte, sometimes as much as 4 bytes. I’m using a hex editor so non-printable characters are still counted. An example file is visible at https://github.com/durkie/durkie.github.io/blob/master/assets/files/screenlog.txt

This doesn’t feel like an issue with the camera since Serial.write() seems to be claiming to write more than it actually did and that seems unrelated to anything happening on Serial1. For what it’s worth, I’m connected to the camera at 38400 baud and the Electron is connected to my computer at 115200, but it seems to happen at any baud rate between Electron and computer: 9200 and 2400 baud both give the same errors and same frequency of errors. The baud rate of the camera is not easily changed.

I’m observing this behavior with the camera connected to either Serial1, Serial4 or Serial5. The cellular radio is off and the camera/electron connection is via 6" long jumper wires. The USB cable is stock. SYSTEM_THREAD(ENABLED) doesn’t seem to make a difference (not that I really know anything about this mode), nor does changing the read/write size from 32 bytes to 16. This is also an error I’ve observed with several other similar types of camera modules but the Adafruit is the only one I have verified to actually work with its windows setup though.

I’m using the library here for interfacing with the camera: https://github.com/adafruit/Adafruit-VC0706-Serial-Camera-Library – the code I’m using is basically the same as in examples/Snapshot.ino, except I’m printing to serial instead of writing image data to an SD card.

Any ideas on things to try? I feel like I’m running out of levers to pull.

2

Okay, and further weirdness: I’m now having the camera print what it thinks its read buffer is so that I can compare with what is actually written to the computer’s serial port. So my serial port output looks like this:

(data chunk of raw hex)(text representation of that data: "0xCE 0x1B 0x33 0xFE", etc)WRITTEN 32

So I can compare the data chunk, and if it’s less than 32 bytes, I can look through the text representation of the camera read buffer and find the missing bytes.

In all cases I’ve found so far, the camera has a read buffer that is a full 32 bytes, and in all situations where bytes have been dropped, the missing byte is either 0x0F or 0x16. So there’s something weird about transmitting just those values apparently…

3

As one measure, you could try to run in a non-cloud connected mode, to see whether or not the issue goes away.

Can you show your read/write protion of the code?

BTW, for USB Serial the baudrate is irrelevant as its always the USB transfer speed the reports are sent back and forth.

4

Would non-cloud connected mode be something beyond Cellular.off()? That’s what I’m running right now.

The read/write portion is below. The cam.frameLength() and cam.readPicture() functions are in the Adafruit library and are a bunch of low-level commands to the camera. I haven’t modified these.

uint16_t jpglen = cam.frameLength();
    Serial.print("wait to fetch ");
    Serial.print(jpglen, DEC);
    Serial.println(" byte image ...");
    uint8_t *buffer;
    while(jpglen != 0){
         Serial.print("WRITTEN: ");
         Serial.println(jpglen);
         uint8_t bytesToRead = min(32, jpglen);
         buffer = cam.readPicture(bytesToRead);    
         uint16_t written = Serial.write(buffer, bytesToRead);
         jpglen -= written;
    }
5

Okay! I actually just figured this out: apparently the problem is GNU screen as a serial console program: it won’t transmit 0x0F (and maybe 0x16) for some reason. Doing cat /dev/cu.usbmodem621 | tee image.jpg works fine and produces useable images.

Argh. I spent so many hours on this… Trying to transmit an image over serial as a means of debugging and saving data just created a big problem in the end…:neutral_face:

1 Like