Issues related to MQTT Audio Bluemix JSON TCP on Photon

Firmware
1

A while back I posted a query regarding large data sets via MQTT. The data sets are related to transferring Audio data back and forth between Bluemix and Photon. I just throw away the audio data for now… What I am trying is if I can possibly do 16KB chunks of data at a time . This 16KB transfer is possible and verified between Bluemix and an app running on my laptop (with lots of memory etc). The challenge is to make the same or close to same on Particle.

Here is what I have found so far:

  1. Limit on MQTT MTU : MQTT_MAX_PACKET_SIZE 255
    When data is sent from Blue mix down to the photon, an unsigned char of value “255” gets sent down as ASCII char 2, followed by Ascii char 5 followed by ascii char 5 – leading to bloat on data size. So if you set packet size to 255, you can really expect to get ~1KB including the MQTT overhead.
    This of course kills the photon – as the extra data may trash the memory – then you see the RED SOS blinking.
    Should you dare to change the MQTT size from 255 to say 2048, so as to allow for ~512 byte packets (512 packets translate to approx 512 times 3 plus overhead ~= 2kB), then you run into the next problem:

  2. The Maximum MTU in TCP client appears to be 1500 Bytes… or this is what I understand from this post: TCP_Tx_Rates . So bottom line you are screwed if you try to change anything in MQTT lib. Iff Particle lib can manage TCP.write() of greater than 1.5KB packet size then a 512 Byte MQTT may be possible. However as one of the posters in the above link pointed out, many servers may only TxRx 1.5KB maximum MTUs.

  3. Another issue is the StaticJsonBuffer runs into problem for size-of-buffer > 4KB. Or at least my attempt on photon basically killed the device (had to re-dfu – painfully!)

Given the above issues (or what I perceive as issues – pl let me know if there is a way to fix it in the app code), here are my questions:

A. Can the Particle team do a wrapper around both TCP.write and TCP.read to handle larger data MTUs, by breaking them down to managable chunks – and manage them as part of the TCP library, So a MQTT lib fix may include at least 8KB chunk transfers…? Or I am barking up a totally wrong tree ?

B. Can the Particle team look into the StaticJsonBuffer<> construct in the SPARKJSON library and allow for larger memory buffers ( within the device limits of course)?

C. Has anyone been able to TxRx MTUs > 255 bytes in MQTT without having to “manage-chunks” of data?.