TlsTcpClient library on WebIDE published


#21

I’m going to stop looking at this until the issue in this other thread is resolved, as it may be affecting my testing.


#22

@timx Right, I think maybe RPi cloud implementation or problems.
@developer_bt , I think maybe your problem nothing with this lib. re-try later.


#23

@hirotakaster
I had succeeded to solve the problem with the Electron :smile:
The problem is in instance for library. It should be instantiated outside the loop.
I also noticed that does not work with included SYSTEM_THREAD (ENABLED);

The code is this:

#include "Particle.h"
#include "application.h"
#include "TlsTcpClient/TlsTcpClient.h"

STARTUP(cellular_credentials_set("internet", "internet", "t-mobile", NULL));

ApplicationWatchdog wd(60000, System.reset);

// 
// This example connect to the Let's Encrypt HTTPS server.
// Let's Encrypt ROOT Ca PEM file is here ( https://letsencrypt.org/certificates/ )
// If you want to use other Root CA, check your server administrator or own Root CA pem.
//
#define LET_ENCRYPT_CA_PEM                                              \
"-----BEGIN CERTIFICATE----- \r\n"                                      \
"MIIFazCCA1OgAwIBAgIRAIIQz7DSQONZRGPgu2OCiwAwDQYJKoZIhvcNAQELBQAw\r\n"  \
"TzELMAkGA1UEBhMCVVMxKTAnBgNVBAoTIEludGVybmV0IFNlY3VyaXR5IFJlc2Vh\r\n"  \
"cmNoIEdyb3VwMRUwEwYDVQQDEwxJU1JHIFJvb3QgWDEwHhcNMTUwNjA0MTEwNDM4\r\n"  \
"WhcNMzUwNjA0MTEwNDM4WjBPMQswCQYDVQQGEwJVUzEpMCcGA1UEChMgSW50ZXJu\r\n"  \
"ZXQgU2VjdXJpdHkgUmVzZWFyY2ggR3JvdXAxFTATBgNVBAMTDElTUkcgUm9vdCBY\r\n"  \
"MTCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBAK3oJHP0FDfzm54rVygc\r\n"  \
"h77ct984kIxuPOZXoHj3dcKi/vVqbvYATyjb3miGbESTtrFj/RQSa78f0uoxmyF+\r\n"  \
"0TM8ukj13Xnfs7j/EvEhmkvBioZxaUpmZmyPfjxwv60pIgbz5MDmgK7iS4+3mX6U\r\n"  \
"A5/TR5d8mUgjU+g4rk8Kb4Mu0UlXjIB0ttov0DiNewNwIRt18jA8+o+u3dpjq+sW\r\n"  \
"T8KOEUt+zwvo/7V3LvSye0rgTBIlDHCNAymg4VMk7BPZ7hm/ELNKjD+Jo2FR3qyH\r\n"  \
"B5T0Y3HsLuJvW5iB4YlcNHlsdu87kGJ55tukmi8mxdAQ4Q7e2RCOFvu396j3x+UC\r\n"  \
"B5iPNgiV5+I3lg02dZ77DnKxHZu8A/lJBdiB3QW0KtZB6awBdpUKD9jf1b0SHzUv\r\n"  \
"KBds0pjBqAlkd25HN7rOrFleaJ1/ctaJxQZBKT5ZPt0m9STJEadao0xAH0ahmbWn\r\n"  \
"OlFuhjuefXKnEgV4We0+UXgVCwOPjdAvBbI+e0ocS3MFEvzG6uBQE3xDk3SzynTn\r\n"  \
"jh8BCNAw1FtxNrQHusEwMFxIt4I7mKZ9YIqioymCzLq9gwQbooMDQaHWBfEbwrbw\r\n"  \
"qHyGO0aoSCqI3Haadr8faqU9GY/rOPNk3sgrDQoo//fb4hVC1CLQJ13hef4Y53CI\r\n"  \
"rU7m2Ys6xt0nUW7/vGT1M0NPAgMBAAGjQjBAMA4GA1UdDwEB/wQEAwIBBjAPBgNV\r\n"  \
"HRMBAf8EBTADAQH/MB0GA1UdDgQWBBR5tFnme7bl5AFzgAiIyBpY9umbbjANBgkq\r\n"  \
"hkiG9w0BAQsFAAOCAgEAVR9YqbyyqFDQDLHYGmkgJykIrGF1XIpu+ILlaS/V9lZL\r\n"  \
"ubhzEFnTIZd+50xx+7LSYK05qAvqFyFWhfFQDlnrzuBZ6brJFe+GnY+EgPbk6ZGQ\r\n"  \
"3BebYhtF8GaV0nxvwuo77x/Py9auJ/GpsMiu/X1+mvoiBOv/2X/qkSsisRcOj/KK\r\n"  \
"NFtY2PwByVS5uCbMiogziUwthDyC3+6WVwW6LLv3xLfHTjuCvjHIInNzktHCgKQ5\r\n"  \
"ORAzI4JMPJ+GslWYHb4phowim57iaztXOoJwTdwJx4nLCgdNbOhdjsnvzqvHu7Ur\r\n"  \
"TkXWStAmzOVyyghqpZXjFaH3pO3JLF+l+/+sKAIuvtd7u+Nxe5AW0wdeRlN8NwdC\r\n"  \
"jNPElpzVmbUq4JUagEiuTDkHzsxHpFKVK7q4+63SM1N95R1NbdWhscdCb+ZAJzVc\r\n"  \
"oyi3B43njTOQ5yOf+1CceWxG1bQVs5ZufpsMljq4Ui0/1lvh+wjChP4kqKOJ2qxq\r\n"  \
"4RgqsahDYVvTH9w7jXbyLeiNdd8XM2w9U/t7y0Ff/9yi0GE44Za4rF2LN9d11TPA\r\n"  \
"mRGunUHBcnWEvgJBQl9nJEiU0Zsnvgc/ubhPgXRR4Xq37Z0j4r7g1SgEEzwxA57d\r\n"  \
"emyPxgcYxn/eR44/KJ4EBs+lVDR3veyJm+kXQ99b21/+jh5Xos1AnX5iItreGCc=\r\n"  \
"-----END CERTIFICATE----- \r\n"

const char letencryptCaPem[] = LET_ENCRYPT_CA_PEM;

TlsTcpClient client;

void setup() {
    Particle.keepAlive(45);
    Serial.begin(9600);

}

void loop() {
    unsigned char buff[256];

    // setup Root CA pem.
    client.init(letencryptCaPem, sizeof(letencryptCaPem));
    
    // connect HTTPS server.
    client.connect("www.hirotakaster.com", 443);
    
    // Send request to HTTPS web server.
    int len = sprintf((char *)buff, "GET /robots.txt HTTP/1.0\r\nHost: www.hirotakaster.com\r\nContent-Length: 0\r\n\r\n");
    client.write(buff, len );

    // GET HTTPS response.
    memset(buff, 0, sizeof(buff));
    while(1) {
        // check response is available.
        if (!client.available()) {
            delay(100);
        } else {
            // read renponse.
            int ret = client.read(buff, sizeof(buff) - 1);
            if (ret > 0) {
                Serial.println((char *)buff);
                break;
            }
        }
    };
    delay(10000);
}

It does not work very stable sometimes reboot but it is important there is a progress :smile:


#24

@developer_bt

Did you get it working against www.hirotakaster.com or another website?

@hirotakaster

I think the example application code need something in setup() to pause it until the correct time has been set. Something like:

while (Time.now() < 1234567890) {
    Serial.println("Waiting for time sync");
    delay(1000);
}

HTTPS is time-sensitive and having the date set as 1969-12-01 isn’t going to help :slight_smile:


#25

@developer_bt , thank you for test. case of SYSTEM_THREAD(ENABLED) , particle TCPClient network function is not works well in the first place, so if you need SYSTEM_THREAD wait for particle new firmware update.


#26

I have got it working. I like it :smile:

I’ve tested it against https://api.particle.io/device. Without anything after “/device” it returns a 404 and a few lines of json so it is quite good for testing in anticipation of using it with a more complete url in the future. I replaced the test root cert in the example program with the root cert for api.particle.io.

I am getting the following with 0.5.2 on my Photon:

text data bss dec hex filename
95156 244 9744 105144 19ab8 /workspace/target/workspace.elf

@hirotakaster

In TlsTcpClient::init, I commented out the call to mbedtls_ssl_free(&ssl) just before mbedtls_ssl_init(&ssl). It fails (red SOS, 1 flash) with it in. Is that because it is trying to free something that has not been initialised?


#27

@timx, thank you for your test. Now I update to 0.1.8 for ssl config/sample and your pointed. It maybe works stable.


#28

Hello @hirotakaster and @timx.
Today I tried the latest version 0.1.8 with www.hirotakaster.com and Electron 2G and I can say it works great :smile:
In the example I use even SYSTEM_THREAD (ENABLED).
For delay (5000) cycle repeats approximately every 13 seconds which is great for GPRS communication.
Perfectly completed work @hirotakaster :smile: Great work from your side. This requires a lot of Particle users.
Normally it would be good to do optimization but this is already usable.
Now it remains to adjust http library for TLS support.
I hope that the development will continue :smile:
The example is this:

// This #include statement was automatically added by the Particle IDE.
#include "TlsTcpClient/TlsTcpClient.h"

#include "Particle.h"
#include "application.h"

STARTUP(cellular_credentials_set("internet", "internet", "t-mobile", NULL));
SYSTEM_THREAD(ENABLED);
SYSTEM_MODE(SEMI_AUTOMATIC);
ApplicationWatchdog wd(60000, System.reset);

// 
// This example connect to the Let's Encrypt HTTPS server.
// Let's Encrypt ROOT Ca PEM file is here ( https://letsencrypt.org/certificates/ )
// If you want to use other Root CA, check your server administrator or own Root CA pem.
//
#define LET_ENCRYPT_CA_PEM                                              \
"-----BEGIN CERTIFICATE----- \r\n"                                      \
"MIIFazCCA1OgAwIBAgIRAIIQz7DSQONZRGPgu2OCiwAwDQYJKoZIhvcNAQELBQAw\r\n"  \
"TzELMAkGA1UEBhMCVVMxKTAnBgNVBAoTIEludGVybmV0IFNlY3VyaXR5IFJlc2Vh\r\n"  \
"cmNoIEdyb3VwMRUwEwYDVQQDEwxJU1JHIFJvb3QgWDEwHhcNMTUwNjA0MTEwNDM4\r\n"  \
"WhcNMzUwNjA0MTEwNDM4WjBPMQswCQYDVQQGEwJVUzEpMCcGA1UEChMgSW50ZXJu\r\n"  \
"ZXQgU2VjdXJpdHkgUmVzZWFyY2ggR3JvdXAxFTATBgNVBAMTDElTUkcgUm9vdCBY\r\n"  \
"MTCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBAK3oJHP0FDfzm54rVygc\r\n"  \
"h77ct984kIxuPOZXoHj3dcKi/vVqbvYATyjb3miGbESTtrFj/RQSa78f0uoxmyF+\r\n"  \
"0TM8ukj13Xnfs7j/EvEhmkvBioZxaUpmZmyPfjxwv60pIgbz5MDmgK7iS4+3mX6U\r\n"  \
"A5/TR5d8mUgjU+g4rk8Kb4Mu0UlXjIB0ttov0DiNewNwIRt18jA8+o+u3dpjq+sW\r\n"  \
"T8KOEUt+zwvo/7V3LvSye0rgTBIlDHCNAymg4VMk7BPZ7hm/ELNKjD+Jo2FR3qyH\r\n"  \
"B5T0Y3HsLuJvW5iB4YlcNHlsdu87kGJ55tukmi8mxdAQ4Q7e2RCOFvu396j3x+UC\r\n"  \
"B5iPNgiV5+I3lg02dZ77DnKxHZu8A/lJBdiB3QW0KtZB6awBdpUKD9jf1b0SHzUv\r\n"  \
"KBds0pjBqAlkd25HN7rOrFleaJ1/ctaJxQZBKT5ZPt0m9STJEadao0xAH0ahmbWn\r\n"  \
"OlFuhjuefXKnEgV4We0+UXgVCwOPjdAvBbI+e0ocS3MFEvzG6uBQE3xDk3SzynTn\r\n"  \
"jh8BCNAw1FtxNrQHusEwMFxIt4I7mKZ9YIqioymCzLq9gwQbooMDQaHWBfEbwrbw\r\n"  \
"qHyGO0aoSCqI3Haadr8faqU9GY/rOPNk3sgrDQoo//fb4hVC1CLQJ13hef4Y53CI\r\n"  \
"rU7m2Ys6xt0nUW7/vGT1M0NPAgMBAAGjQjBAMA4GA1UdDwEB/wQEAwIBBjAPBgNV\r\n"  \
"HRMBAf8EBTADAQH/MB0GA1UdDgQWBBR5tFnme7bl5AFzgAiIyBpY9umbbjANBgkq\r\n"  \
"hkiG9w0BAQsFAAOCAgEAVR9YqbyyqFDQDLHYGmkgJykIrGF1XIpu+ILlaS/V9lZL\r\n"  \
"ubhzEFnTIZd+50xx+7LSYK05qAvqFyFWhfFQDlnrzuBZ6brJFe+GnY+EgPbk6ZGQ\r\n"  \
"3BebYhtF8GaV0nxvwuo77x/Py9auJ/GpsMiu/X1+mvoiBOv/2X/qkSsisRcOj/KK\r\n"  \
"NFtY2PwByVS5uCbMiogziUwthDyC3+6WVwW6LLv3xLfHTjuCvjHIInNzktHCgKQ5\r\n"  \
"ORAzI4JMPJ+GslWYHb4phowim57iaztXOoJwTdwJx4nLCgdNbOhdjsnvzqvHu7Ur\r\n"  \
"TkXWStAmzOVyyghqpZXjFaH3pO3JLF+l+/+sKAIuvtd7u+Nxe5AW0wdeRlN8NwdC\r\n"  \
"jNPElpzVmbUq4JUagEiuTDkHzsxHpFKVK7q4+63SM1N95R1NbdWhscdCb+ZAJzVc\r\n"  \
"oyi3B43njTOQ5yOf+1CceWxG1bQVs5ZufpsMljq4Ui0/1lvh+wjChP4kqKOJ2qxq\r\n"  \
"4RgqsahDYVvTH9w7jXbyLeiNdd8XM2w9U/t7y0Ff/9yi0GE44Za4rF2LN9d11TPA\r\n"  \
"mRGunUHBcnWEvgJBQl9nJEiU0Zsnvgc/ubhPgXRR4Xq37Z0j4r7g1SgEEzwxA57d\r\n"  \
"emyPxgcYxn/eR44/KJ4EBs+lVDR3veyJm+kXQ99b21/+jh5Xos1AnX5iItreGCc=\r\n"  \
"-----END CERTIFICATE----- \r\n"

const char letencryptCaPem[] = LET_ENCRYPT_CA_PEM;
#define ONE_DAY_MILLIS (24 * 60 * 60 * 1000)
unsigned long lastSync = millis();


void setup() {
    
    Particle.keepAlive(45);
    Serial.begin(9600);
    
    Particle.connect();  // for semi automatic mode
    
     // need a Particle time sync for X509 certificates verify.
    if (millis() - lastSync > ONE_DAY_MILLIS) {
        Particle.syncTime();
        lastSync = millis();
    }
    Serial.print(Time.timeStr());

}

void loop() {
    
  if (Particle.connected())
     { 
   
    TlsTcpClient client;
   
    unsigned char buff[256];

    // setup Root CA pem.
    client.init(letencryptCaPem, sizeof(letencryptCaPem));

    // connect HTTPS server.
    client.connect("www.hirotakaster.com", 443);

    // Send request to HTTPS web server.
    int len = sprintf((char *)buff, "GET /robots.txt HTTP/1.0\r\nHost: www.hirotakaster.com\r\nContent-Length: 0\r\n\r\n");
    client.write(buff, len );

    // GET HTTPS response.
    memset(buff, 0, sizeof(buff));
    while(1) {
        // check response is available.
        if (!client.available()) {
            delay(100);
        } else {
            // read renponse.
            int ret = client.read(buff, sizeof(buff) - 1);
            if (ret > 0) {
                  Serial.println((char *)buff);
                  break;
            
                }
           }
     
     };  
    delay(5000);
    
    }
}

#29

@hirotakaster
I noticed that in the new version the instance must be in the Loop, while the in old version should be a defined outside the Loop. Otherwise it does not work.
Correctly it should be done only once outside the Loop to be defined in another library?


#30

@developer_bt , which version and code are you pointing?


#31

@hirotakaster

I have tried 0.1.8 and it is working well. Thank you for making the changes.

I have suggestions for 2 enhancements that I think would be useful.

(1). Server Name Indication (SNI) https://en.wikipedia.org/wiki/Server_Name_Indication

Some websites, especially cloud based ones, need this to be set.

I see that you have set MBEDTLS_SSL_SERVER_NAME_INDICATION, but you also need to call mbedtls_ssl_set_hostname.

You can check this using Wireshark. The hostname should be sent in the TLS ClientHello message.

(2). Check that the certificate is valid after calling handshake.

You need to call mbedtls_ssl_get_verify_result. Can you make it optional? Some people will definitely want to verify the certificate and some people will definitely NOT want to verify it :slight_smile:


#32

@hirotakaster
To work with electron in version 0.1.8 the instance for library must be in the Loop, while the in old version (0.1.7) must be outside the Loop.


#33

@developer_bt, sorry, I don’t know what do you point. maybe update miss.


#34

@timx , yeah, thank you suggestions. I know SNI and check the certificate after handshake is needed. This 0.1 version is basically check for TLS encryption and working flow, so Now I was postponed detailed implementation and error handling etc :smile: I will update others and welcome your pull-request to github.


States of HTTPS libraries/firmware & cloud services
#35

Now I update TlsTcpClient library to 0.1.15(latest mbedTls 2.5.1).

And shape the library size.
92024byte -> 73756byte


Particle REST APİ
#36

Thank you @hirotakaster
I solved my problem sending post message through https to a server using particle electron board using your library.


#37

Hello, @hirotakaster
I have been using the library for post message to a server however it is very slow to me.
Everytime electron tries to connect to a server with
client.connect(“my.server.address”, 443);
it takes about 4~6 seconds to establish connection and another 6~8 seconds to finish its post.
I am currently using 3rd party sim. Do you think this might cause the problem?
Is there a way to save time for posting?
Thanks,

Kyu


#38

I think it takes time about the networking connection on SIM(2G or 3G).
It’s better that you check the application speed on SIM and WiFi network.