How to publish extended (8bit) ASCII

Firmware
1

Hi,
I need some help publishing 8bit ASCII data.
Here’s what I do:

String Text = "123ÄÖÜäöü123";
Particle.publish("Info",Text, 300, PRIVATE);

Works fine.
Received by webook as
event=Info&data=123%C3%84%C3%96%C3%9C%C3%A4%C3%B6%C3%BC123

char TEXT[] = {49,50,51,196,214,220,228,246,252,49,50,51,0};
String TextC = String((char*)TEXT);
Particle.publish("Info",TextC, 300, PRIVATE);

Does not work. Received as
event=Info&data=123%EF%BF%BD%EF%BF%BD%EF%BF%BD%EF%BF%BD%EF%BF%BD%EF%BF%BD123

Is there a simple way to accomplish this?

Thanks!
Arne

2

Yes there is a way - you need to encode as base64, 3 8bit bytes get converted into 4 6bit characters that can be transferred via the publish as an ascii character 0 to . At the receiving end you need to convert them back. There are libraries to help you do this search for base64 or Base64RK.

2 Likes
3

The issue here is that you assume the ASCII values for the German Umlaute to map to the same binary values as in the German version of the extended ASCII table.
But as you can see in the working output, they don’t.

These special characters are in fact a combination of two bytes, e.g. Ä maps to 0xC3 0x84 not single byte 0xC4 (196 decimal).

3 Likes
4

Yes, what ScruffR said. Specifically, the string encoding is UTF-8 (Unicode, 8-bit) not ISO-8859-1 (Latin 1). So characters beyond 7-bit ASCII are represented as two (or more) bytes. All of the high-bit set characters in ISO-8859-1 map to two byte sequences beginning with 0xC2 or 0xC3.

It’s not possible to directly send 8-bit characters or binary data because the data string must be valid UTF-8 sequences, and many sequences of 8-bit bytes are not valid UTF-8.

3 Likes
5

Thanks.
So I just added a UTF8 conversion and it works.

char TEXT[] = {49,50,51,196,214,220,228,246,252,49,50,51,0};
Particle.publish("Info",ToUTF8(TextC), 300, PRIVATE);

Excellent.
Thanks for the support!

1 Like
6

Just a word on your Particle.publish() statement: You have set the TTL to 300 seconds but you are aware that this parameter currently (if ever) doesn’t do anything at all?
There actually is no TTL for Particle events. They are fire and forget and have to be caught there and then.