I am using the cloud string variable to transmit 622 bytes of binary data. On encode I make sure I am only using byte values 1-255 to not compromise the string. My question is now how to decode the JSON string back to the binary data. This are some of the JSON encoded return messages:
After doing some searching it seems that the byte string is being encoded into (most likely) UTF-8, can only handle values up to 128 for a single byte.
From Python 2.7.9 docs:
UTF-8 uses the following rules:
1) If the code point is <128, it’s represented by the corresponding byte value.
2) If the code point is between 128 and 0x7ff, it’s turned into two byte values between 128 and 255.
3) Code points >0x7ff are turned into three- or four-byte sequences, where each byte of the sequence is between 128 and 255.
not sure how to apply this and it does not match what I get below. Is there a lib function doing this? Here is another sample:
I now generated a string with 255 characters 0x01- 0xff and requested it back from the cloud. Very strange results, here are the real values reaching the socket:
all values above 127 are lost and made into the sequence (239)(191)(189). This is really bad because I wanted to use this to transmit binary data.
Can this be fixed somehow?
You should encode your binary data before sending it using only printable ASCII characters. There are a number of schemes for doing this but base64 encoding is probably the most common. That way the other side of your connection should have an easy time converting.
@bko: thanks for confirming that the string cloud function only transmits the lower 7-bit ASCII table. Using base64 is in my case not the ideal option because it uses only 6-bits. I will encode my data into 0x01-0x7f then.
Hello,
When you send a String through the spark cloud it gets encoded to UTF-8. This encoding only works for byte values between 0x00 and 0x7f (or 127 in decimal). When it encodes a value higher than 127, it turns it into multiple bytes to conform to the standard.
If possible, I would highly suggest you use a TCP server or TCP Client (depending on what you need). These are really easy to setup, are much quicker than using the cloud, and do not encode anything; Which means you can use any byte values you see fit.
Unfortunalety all values higher than 127 are mapped to the same sequence (239)(191)(189) as written further up in my post. TCP is only an option for the local network and will not work in the WAN. I need a solution that works outside the local network as well. It is kludgy but I am now using 7 bit encoding and can live with that for now.