cermak
June 18, 2017, 5:07pm
1
This port is about 90% done. I am soliciting any helpers at this point.
I am using the mbedTLS/TlsTclClient library as a template. The main goal of this is to get https://things.ubidots.com working. I have tried a myrad of compile combinations with the old and the development version of mbedTLS. It will connect to many other https sites but not the Ubidots site. Plain unix curl works fine so, these libraries should work as well.
I just wanted to give a shout out that there is a workable port on Github, if you want to send me patches or pull requests, that is fine. I am at the point where I need to do the byte by byte comparison of what the Linux client is doing vs this port and fixing things up from there.
The upshot is the firmware footprint is smaller (50-60k so far). The mbedTLS is right up against the edge of the firmware limit (~120k) using the development tree.
I am just porting the axtls client at this time. There is a server component.
src library
More later… the hacking continues…
attempting to compile firmware
downloading binary from: /v1/binaries/5945f1d65a446b1d25f08ccf
saving to: tmp.bin
Memory use:
text data bss dec hex filename
47892 184 1816 49892 c2e4 /workspace/target/workspace.elf
Compile succeeded.
Saved firmware to: /Users/cermak/Particle/projects/axtls/src/tmp.bin
Including:
tmp.bin
attempting to flash firmware to your device testPhoton
Flash device OK: Update started
Polling for available serial device...
Opening serial monitor for com port: "/dev/cu.usbmodem1421"
Serial monitor opened successfully:
0001142624 [comm.sparkprotocol] INFO: chunk
0001142624 [comm.sparkprotocol] INFO: chunk idx=91 crc=1 fast=1 updating=1
0001142627 [comm.sparkprotocol] INFO: chunk
0001142628 [comm.sparkprotocol] INFO: chunk idx=92 crc=1 fast=1 updating=1
0001142631 [comm.sparkprotocol] INFO: chunk
0001142631 [comm.sparkprotocol] INFO: chunk idx=93 crc=1 fast=1 updating=1
0001142633 [comm.sparkprotocol] WARN: update done: received, has missing chunks 0
0001142634 [comm.sparkprotocol] INFO: update done: all done!
0001142662 [system] INFO: Send spark/device/ota_result event
Serial connection closed. Attempting to reconnect...
Serial monitor opened successfully:
0000008277 [system] INFO: ARM_WLAN_WD 2
0000008277 [hal.wlan] INFO: Bringing WiFi interface up with DHCP
0000008738 [system] INFO: CLR_WLAN_WD 1, DHCP success
0000008739 [system] INFO: Cloud: connecting
0000008739 [system] INFO: Read Server Address = type:1,domain:device.spark.io
0000008840 [system] INFO: Resolved host device.spark.io to 54.175.114.62
0000009060 [system] INFO: connected to cloud 54.175.114.62:5683
0000009060 [system] INFO: Cloud socket connected
0000009060 [system] INFO: Starting handshake: presense_announce=1
0000009061 [comm.sparkprotocol.handshake] INFO: Started: Receive nonce
0000009248 [comm.sparkprotocol.handshake] INFO: Encrypting handshake nonce
0000009295 [comm.sparkprotocol.handshake] INFO: Sending encrypted nonce
0000009295 [comm.sparkprotocol.handshake] INFO: Receive key
0000009498 [comm.sparkprotocol.handshake] INFO: Setting key
0000009687 [comm.sparkprotocol.handshake] INFO: Sending HELLO message
0000009687 [comm.sparkprotocol.handshake] INFO: Receiving HELLO response
0000009812 [comm.sparkprotocol.handshake] INFO: Completed
0000009812 [system] INFO: Send spark/hardware/max_binary event
0000009813 [system] INFO: spark/hardware/ota_chunk_size event
0000009813 [system] INFO: Send spark/device/last_reset event
0000009814 [system] INFO: Send subscriptions
0000009814 [comm.sparkprotocol] INFO: Sending TIME request
0000009815 [system] INFO: Cloud connected
0000009816 [axtls] INFO: init()
0000009816 [axtls] INFO: setup() done.
0000010133 [comm.sparkprotocol] INFO: Received TIME response: 1497756140
0000011099 [comm.sparkprotocol] INFO: Sending A describe message
0000011243 [comm.sparkprotocol] INFO: Sending S describe message
0000020245 [axtls] INFO: Trying to connect
0000020245 [axtls] INFO: begin connect()
0000020246 [axtls] INFO: host:jupyter.lccllc.info port:4443
0000020251 [axtls] INFO: init() ssl_ctx(0x20007658) send_Tls(0x80a2b8d) recv_Tls(0x80a2b31)
0000020252 [axtls] INFO: Assigning i/o pathways ssl_ctx(0x20007658) f_send(0x80a2b8d) f_recv(0x80a2b31)
0000020252 [axtls] INFO: connect() try 1
0000020483 [axtls] INFO: ssl_client_new()
0000020484 [axtls] INFO: ssl->version:0x33
0000020484 [axtls] INFO: do_client_connect()
0000020484 [axtls] INFO: send_client_hello() ssl(0x20008b10)
0000020484 [axtls] INFO: send_packet()
0000020485 [axtls] INFO: send_raw_packet() length=67 msg_length=67
0000020485 [axtls] INFO: SOCKET_WRITE ssl(0x20008b10) t:72 s:0
0000020485 [axtls] INFO: sendParticle ssl(0x20008b10) ssl_ctx(0x20007658) f_send(0x80a2b8d)
0000020486 [axtls] INFO: Wanted to send 72 bytes, sent 72 bytes
0000020486 [axtls] INFO: Handshake error: -1
1 Like
timx
June 18, 2017, 8:54pm
2
Do you have SNI implemented?
If not then that might account for being able to connect to some sites but not others.
Ref: https://en.wikipedia.org/wiki/Server_Name_Indication
cermak
June 18, 2017, 11:34pm
3
Yup. As well as SSL_SNI. Which requires X509_CRT_PARSE_C which is required by a lot of things
/**
* \def MBEDTLS_SSL_SERVER_NAME_INDICATION
*
* Enable support for RFC 6066 server name indication (SNI) in SSL.
*
* Requires: MBEDTLS_X509_CRT_PARSE_C
*
* Comment this macro to disable support for server name indication in SSL
*/
#define MBEDTLS_SSL_SERVER_NAME_INDICATION
cermak
June 19, 2017, 2:08am
4
I think we are 99% there, just have to do some cleanup and test it with the Ubidot REST API.
Nice footprint:
Memory use:
text data bss dec hex filename
54964 188 2108 57260 dfac /workspace/target/workspace.elf
Compile succeeded.
Saved firmware to: /Users/cermak/Particle/projects/axtls/src/tmp.bin
Including:
tmp.bin
0000021824 [axtls] INFO: Trying to connect
0000021825 [axtls] INFO: begin connect()
0000021825 [axtls] INFO: host:things.ubidots.com port:443
0000021831 [axtls] INFO: init() ssl_ctx(0x20007780) send_Tls(0x80a4049) recv_Tls(0x80a3f71)
0000021831 [axtls] INFO: Assigning i/o pathways ssl_ctx(0x20007780) f_send(0x80a4049) f_recv(0x80a3f71)
0000021832 [axtls] INFO: connect() try 1
0000022077 [axtls] INFO: ssl_client_new()
0000022078 [axtls] INFO: ssl->version:0x33
0000022078 [axtls] INFO: do_client_connect()
0000022078 [axtls] INFO: send_client_hello() ssl(0x20008c38)
0000022079 [axtls] INFO: send_packet()
0000022079 [axtls] INFO: send_raw_packet() length=67 msg_length=67
0000022079 [axtls] INFO: sending 72 bytes
0000022401 [axtls] INFO: recvParticle ssl(0x20008c38) ssl_ctx(0x20007780) f_recv(0x80a4049)
0000022401 [axtls] INFO: Want 40 byte(s)
0000022403 [axtls] INFO: ssl(0x20008c38) ssl_ctx(0x20007780) sock(0x20000548)
0000022403 [axtls] INFO: sock->connected():1 sock->available():49
0000022404 [axtls] INFO: Got 40 byte(s)
0000022404 [axtls] INFO: received 40 bytes
0000022404 [axtls] INFO: 4b 98 cd da 9f 0c f9 7f : 90 ed 43 4a 12 44 4e 6f
0000022405 [axtls] INFO: 73 7a 28 ea a4 aa 6e 7b : 4c 7d 87 dd e0 c9 02 44
0000022405 [axtls] INFO: a7 87 af c3 34 5b b4 42 :
0000022412 [axtls] INFO: app.ubidots.com
0000022412 [axtls] INFO: things.ubidots.com
0000022412 [axtls] INFO: things6.ubidots.com
0000022418 [axtls] INFO: recvParticle ssl(0x20008c38) ssl_ctx(0x20007780) f_recv(0x80a4049)
0000022419 [axtls] INFO: Want 5 byte(s)
0000022420 [axtls] INFO: ssl(0x20008c38) ssl_ctx(0x20007780) sock(0x20000548)
0000022421 [axtls] INFO: sock->connected():1 sock->available():9
0000022422 [axtls] INFO: Got 5 byte(s)
0000022422 [axtls] INFO: received 5 bytes
0000022422 [axtls] INFO: 16 03 03 00 04
0000022422 [axtls] INFO: recvParticle ssl(0x20008c38) ssl_ctx(0x20007780) f_recv(0x80a4049)
0000022423 [axtls] INFO: Want 4 byte(s)
0000022424 [axtls] INFO: ssl(0x20008c38) ssl_ctx(0x20007780) sock(0x20000548)
0000022425 [axtls] INFO: sock->connected():1 sock->available():4
0000022492 [axtls] INFO: sendParticle ssl(0x20008c38) ssl_ctx(0x20007780) f_send(0x80a4049)
0000022492 [axtls] INFO: ssl(0x20008c38) ssl_ctx(0x20007780) sock(0x20000548)
0000022493 [axtls] INFO: sock->connected():1 sock->available():0
0000022494 [axtls] INFO: Wanted to send 267 bytes, sent 267 bytes
0000022494 [axtls] INFO: send_raw_packet() length=1 msg_length=1
0000022692 [axtls] INFO: Got 80 byte(s)
0000022692 [axtls] INFO: received 80 bytes
0000022694 [axtls] INFO: d5 85 ae 50 d1 ed 63 ae : 3a d8 f2 82 0c 66 e3 62
0000022694 [axtls] INFO: 0d 7b 77 74 1a 70 29 bb : 71 42 f9 a5 76 b8 b1 7e
0000022695 [axtls] INFO: decrypted
0000022695 [axtls] INFO: 14 00 00 0c 8e 8b 25 79 : 7b de 02 f7 ad 33 30 c3
1 Like
@rickkas7 , this might be worth looking at for Matt or Julien?
cermak
June 19, 2017, 7:01am
6
@ScruffR - Thanks! I think we got it. I was overrunning the logger buffer which is why I didn’t initially see the full response.
Is there a good recommended library name? The project name is axTLS . It doesn’t have the full suite of (de/en)cryption which is why it is smaller.
I will try to follow these guidelines as I pull this stuff together.
0000018721 [axtls] INFO: web>HTTP/1.1 200 OK
0000018722 [axtls] INFO: web>Server: nginx
0000018724 [axtls] INFO: web>Date: Mon, 19 Jun 2017 06:16:37 GMT
0000018726 [axtls] INFO: web>Content-Type: application/json
0000018727 [axtls] INFO: web>Transfer-Encoding: chunked
0000018728 [axtls] INFO: web>Connection: keep-alive
0000018729 [axtls] INFO: web>Vary: Accept-Encoding
0000018730 [axtls] INFO: web>Vary: Accept, Cookie
0000018732 [axtls] INFO: web>Allow: GET, POST, HEAD, OPTIONS
0000018733 [axtls] INFO: web>23f
0000018757 [axtls] INFO: web>{"count": 1, "next": null, "previous": null, "results": [{"id": "....2268", "owner": "http://things.ubidots.com/api/v1.6/users/26315", "la~
cermak
June 26, 2017, 7:37pm
7
The client port is coming along. It needs to be stabilized. If I wait for the initial return response and then close the connection, the process is pretty clean. However, if I leave the connection open and the server closes the connection, it sends the Photon into a tail spin (communication loop; one red flash, etc) – it will disconnect from the cloud and go in circles… see quoted text below.
I am not quite sure how to keep the network stack stable? Is the network stack sensitive?
I’d take suggestions on ways to make the code better or questions.
Should I work towards getting the response expected and close the connection before the server disconnects? It would seem that I should be able to detect the remote closure and keep things sane? @ScruffR , can you point me in the right direction or give me some areas to investigate and/or avoid.
EDIT: May need to review this post for ideas.
For those that would like to try and work with the client and send suggestions, that would be great. The repo working towards a stable port is:
https://github.com/jr3cermak/particle/libraries/axtls
0000003186 [axtls] INFO: web></body></html>
0000003698 [comm.sparkprotocol] INFO: Sending A describe message
0000003698 [comm.sparkprotocol] WARN: received ERROR CoAPMessage
0000003698 [system] WARN: Communication loop error, closing cloud socket
0000003800 [system] INFO: Cloud: connecting
0000003800 [system] INFO: Read Server Address = type:1,domain:device.spark.io
0000003815 [system] INFO: Resolved host device.spark.io to 54.90.239.114
0000003831 [system] INFO: connected to cloud 54.90.239.114:5683
0000003831 [system] INFO: Cloud socket connected
0000003831 [system] INFO: Starting handshake: presense_announce=1
0000003831 [comm.sparkprotocol.handshake] INFO: Started: Receive nonce
0000003863 [comm.sparkprotocol.handshake] INFO: Encrypting handshake nonce
0000003864 [comm.sparkprotocol.handshake] ERROR: RSA encrypt error -1087
0000003864 [system] WARN: Cloud handshake failed, code=-1087
0000004115 [system] INFO: Cloud: disconnecting
0000004115 [system] INFO: Cloud: disconnected
0000004923 [system] INFO: Cloud: connecting
0000004923 [system] INFO: Read Server Address = type:1,domain:device.spark.io
0000004952 [system] INFO: Resolved host device.spark.io to 54.90.239.114
0000004969 [system] INFO: connected to cloud 54.90.239.114:5683
0000004969 [system] INFO: Cloud socket connected
0000004969 [system] INFO: Starting handshake: presense_announce=1
0000004970 [comm.sparkprotocol.handshake] INFO: Started: Receive nonce
0000004995 [comm.sparkprotocol.handshake] INFO: Encrypting handshake nonce
0000004995 [comm.sparkprotocol.handshake] ERROR: RSA encrypt error -1087
0000004996 [system] WARN: Cloud handshake failed, code=-1087
0000005246 [system] INFO: Cloud: disconnecting
0000005246 [system] INFO: Cloud: disconnected
0000006054 [system] INFO: Cloud: connecting
0000006054 [system] INFO: Read Server Address = type:1,domain:device.spark.io
0000006093 [system] INFO: Resolved host device.spark.io to 54.90.239.114
0000006124 [system] INFO: connected to cloud 54.90.239.114:5683
0000006124 [system] INFO: Cloud socket connected
0000006125 [system] INFO: Starting handshake: presense_announce=1
0000006125 [comm.sparkprotocol.handshake] INFO: Started: Receive nonce
0000006156 [comm.sparkprotocol.handshake] INFO: Encrypting handshake nonce
0000006156 [comm.sparkprotocol.handshake] ERROR: RSA encrypt error -1087
0000006157 [system] WARN: Cloud handshake failed, code=-1087
I’d rather pass this on to @rickkas7 and he might loop in some Particle devs too.
cermak
June 28, 2017, 4:11pm
9
I think I found one bug. I am using an unallocated pointer in the TCLClient.read(buffer, len). It is working for smaller strings, but a larger string causes a hard fault. This could be the source of the odd network stack behavior since I am tromping on memory.
ret = sock->read(in_data, in_len); where in_data is unallocated. I beleive the documentation says this should be an allocated buffer.
int read(uint8_t *buffer, size_t size) reads all readily available bytes up to size from the server the client is connected to into the provided buffer. (This does not pass back a new pointer to a new buffer.)
I’ve been looking over the Webduino port. It uses a char by char reading method. The axTLS port currently uses block reading. The block reading would be ok for the SSL/TLS handshake. We can switch to char by char for reading the header and content. I like how the Webduino port works. Have to see if we can use the header/content parser for a server response back to a client as well as a server parsing a client request.
cermak
July 3, 2017, 2:01am
10
Trying to harness the v2 library format is a bit elusive.
If I follow this structure: particle/libraries:
$ cat library.properties
# Fill in information about your library then remove # from the start of lines
# https://docs.particle.io/guide/tools-and-features/libraries/#library-properties-fields
name=axtls
version=0.0.1
author=Rob Cermak
license=mixed, Creative Commons 4 if not specified
sentence=axTLS library
paragraph=A port of axTLS from http://axtls.sourceforge.net/
url=https://github.com/jr3cermak/particle/libraries/axtls
repository=https://github.com/jr3cermak/particle.git
testClient has:
testClient:
total 32
-rw-r--r-- 1 cermak staff 204 Jun 24 13:39 Private.h
-rw-r--r-- 1 cermak staff 3690 Jun 27 21:37 axtls_config.h
-rw-r--r-- 1 cermak staff 5033 Jun 27 14:00 testClient.ino
What I was hoping the compile would pull everything from the testClient/ and src/ directory. I can’t achieve this using any of the following scenarios unless I specify everything on the command line.
Within testClient, result #1 is:
$ particle compile photon
Compiling code for photon
Including:
axtls_config.h
Private.h
testClient.ino
attempting to compile firmware
Compile failed. Exiting.
Result #2 , gets closer, but misses the two local files:
$ particle compile photon testClient.ino
Compiling code for photon
Including:
testClient.ino
../../library.properties
../../src/aes.cpp
../../src/axtls.cpp
../../src/asn1.cpp
../../src/axtls.h
../../src/bigint.cpp
../../src/bigint.h
../../src/bigint_impl.h
../../src/byteorder.h
../../src/cert.h
../../src/crypto.h
../../src/crypto_misc.cpp
../../src/crypto_misc.h
../../src/debugging.cpp
../../src/debugging.h
../../src/gen_cert.cpp
../../src/hmac.cpp
../../src/loader.cpp
../../src/md5.cpp
../../src/openssl.cpp
../../src/os_int.h
../../src/os_port.cpp
../../src/os_port.h
../../src/p12.cpp
../../src/private_key.h
../../src/rc4.cpp
../../src/rsa.cpp
../../src/sha1.cpp
../../src/sha256.cpp
../../src/sha384.cpp
../../src/sha512.cpp
../../src/ssl.h
../../src/tls1.h
../../src/tls1_clnt.cpp
../../src/tls1_svr.cpp
../../src/tls1.cpp
../../src/version.h
../../src/x509.cpp
attempting to compile firmware
Try #3:
$ particle compile photon testClient.ino Private.h axtls_config.h
Compiling code for photon
Including:
testClient.ino
Private.h
axtls_config.h
attempting to compile firmware
Any hints on how to solve this? Need any more info?
$ particle --version
1.23.0
To build an embedded example of a library you’d use this syntax
particle compile <targetPlatform> examples/<exampleName>
Executing this command from the position where the library.properties file lives.
cermak
July 3, 2017, 11:55am
12
@ScruffR - thank you! Still didn’t work. Here are two tries:
$ ls
LICENSE bin examples output
README.md doc library.properties src
RobPro:axtls cermak$ pwd
/Users/cermak/Software/particle/libraries/axtls
Try #1:
$ particle compile photon examples/testClient
Compiling code for photon
Including:
examples/testClient/axtls_config.h
examples/testClient/Private.h
examples/testClient/testClient.ino
attempting to compile firmware
Compile failed. Exiting.
Processing testClient.ino
Try #2: just for grins
$ particle compile photon examples/testClient/
Compiling code for photon
Including:
examples/testClient/axtls_config.h
examples/testClient/Private.h
examples/testClient/testClient.ino
attempting to compile firmware
Compile failed. Exiting.
Processing testClient.ino
make -C ../modules/photon/user-part all
make[1]: Entering directory '/firmware/modules/photon/user-part'
make -C ../../../user
make[2]: Entering directory '/firmware/user'
Building cpp file: testClient.cpp
Invoking: ARM GCC CPP Compiler
mkdir -p ../build/target/user/platform-6-m
arm-none-eabi-gcc -DSTM32_DEVICE -DSTM32F2XX -DPLATFORM_THREADING=1 -DPLATFORM_ID=6 -DPLATFORM_NAME=photon -DUSBD_VID_SPARK=0x2B04 -DUSBD_PID_DFU=0xD006 -DUSBD_PID_CDC=0xC006 -DSPARK_PLATFORM -g3 -gdwarf-2 -Os -mcpu=cortex-m3 -mthumb -DINCLUDE_PLATFORM=1 -DPRODUCT_ID=6 -DPRODUCT_FIRMWARE_VERSION=65535 -DUSE_STDPERIPH_DRIVER -DDFU_BUILD_ENABLE -DSYSTEM_VERSION_STRING=0.6.2 -DRELEASE_BUILD -I./inc -I../wiring/inc -I../system/inc -I../services/inc -I../communication/src -I../hal/inc -I../hal/shared -I../hal/src/photon -I../hal/src/stm32f2xx -I../hal/src/stm32 -I../hal/src/photon/api -I../platform/shared/inc -I../platform/MCU/STM32F2xx/STM32_USB_Host_Driver/inc -I../platform/MCU/STM32F2xx/STM32_StdPeriph_Driver/inc -I../platform/MCU/STM32F2xx/STM32_USB_OTG_Driver/inc -I../platform/MCU/STM32F2xx/STM32_USB_Device_Driver/inc -I../platform/MCU/STM32F2xx/SPARK_Firmware_Driver/inc -I../platform/MCU/shared/STM32/inc -I../platform/MCU/STM32F2xx/CMSIS/Include -I../platform/MCU/STM32F2xx/CMSIS/Device/ST/Include -I../dynalib/inc -I -I./libraries -I -I -I -I -I. -MD -MP -MF ../build/target/user/platform-6-mtestClient.o.d -ffunction-sections -fdata-sections -Wall -Wno-switch -Wno-error=deprecated-declarations -fmessage-length=0 -fno-strict-aliasing -DSPARK=1 -DPARTICLE=1 -DSTART_DFU_FLASHER_SERIAL_SPEED=14400 -DSTART_YMODEM_FLASHER_SERIAL_SPEED=28800 -DSPARK_PLATFORM_NET=BCM9WCDUSI09 -fno-builtin-malloc -fno-builtin-free -fno-builtin-realloc -DLOG_INCLUDE_SOURCE_INFO=1 -DPARTICLE_USER_MODULE -DUSE_THREADING=0 -DUSE_SPI=SPI -DUSE_CS=A2 -DUSE_SPI=SPI -DUSE_CS=A2 -DUSE_THREADING=0 -DUSER_FIRMWARE_IMAGE_SIZE=0x20000 -DUSER_FIRMWARE_IMAGE_LOCATION=0x80A0000 -DMODULAR_FIRMWARE=1 -DMODULE_VERSION=4 -DMODULE_FUNCTION=5 -DMODULE_INDEX=1 -DMODULE_DEPENDENCY=4,2,108 -D_WINSOCK_H -D_GNU_SOURCE -DLOG_MODULE_CATEGORY="\"app\"" -fno-exceptions -fno-rtti -fcheck-new -std=gnu++11 -c -o ../build/target/user/platform-6-mtestClient.o testClient.cpp
testClient.ino:24:19: fatal error: axtls.h: No such file or directory
#include "axtls.h"
^
compilation terminated.
../build/module.mk:267: recipe for target '../build/target/user/platform-6-mtestClient.o' failed
make[2]: Leaving directory '/firmware/user'
make[2]: *** [../build/target/user/platform-6-mtestClient.o] Error 1
../../../build/recurse.mk:11: recipe for target 'user' failed
make[1]: Leaving directory '/firmware/modules/photon/user-part'
make[1]: *** [user] Error 2
../build/recurse.mk:11: recipe for target 'modules/photon/user-part' failed
make: *** [modules/photon/user-part] Error 2
UMD
April 11, 2021, 12:15am
13
@cernak did you ever finish this library? It looks like it has a small footprint which would . be great!..
lol, its the same thing