New library - Papertrail

[barakwei]

Hi
I’ve always been frustrated by the fact the logging is only available using Serial. This means you have to be connected to the platform to view live logs.
Sure, you can have a makeshift logging using variables, but that’s hardly a solution. I came accross a cool service called Papertrail that manages logs in the cloud and decided to write a client to Particle plaforms.

It uses plain-text UDP packet to send syslog-formatted log entries to Papertrail, then you can view/filter/search the logs in your favorite browser.
Since 0.6.0 we have the great logging mechanism so I decided to write this library as a log handler, so it should be easy to adopt in existing apps.
Build
Github

Would appreciate any feedback
Barak

[ctmorrison]

Hi @barakwei, This looks like it would be very handy. I tried to compile it to test on an Electron and got a message which suggests this is for the Photon only. Is that by design and, if so, will there be a version for Electrons?

Here’s the error message:

papertrail/papertrail.cpp:11:26: error: 'WiFi' was not declared in this scope
     IPAddress remoteIP = WiFi.resolve(m_host);

[EDIT] I was able to get it to compile for a Photon. However, what I’m seeing at the papertrail site indicates there’s a problem:

Dec 23 12:02:23 Particle PapertrailSimpleDemo:  [system] INFO: Cloud: connecting
Dec 23 12:02:28 Particle PapertrailSimpleDemo:  [system] ERROR: connection failed to 169.46.82.178:5683, code=-2
Dec 23 12:02:28 Particle PapertrailSimpleDemo:  [system] WARN: Cloud socket connection failed: -2
Dec 23 12:02:28 Particle PapertrailSimpleDemo:  [system] WARN: Internet available, Cloud not reachable!
Dec 23 12:02:29 Particle PapertrailSimpleDemo:  [system] INFO: Cloud: connecting
Dec 23 12:02:29 Particle PapertrailSimpleDemo:  [system] INFO: Resolved host device.spark.io to 169.46.82.181
Dec 23 12:02:34 Particle PapertrailSimpleDemo:  [system] ERROR: connection failed to 169.46.82.181:5683, code=-2
Dec 23 12:02:34 Particle PapertrailSimpleDemo:  [system] WARN: Cloud socket connection failed: -2
Dec 23 12:02:34 Particle PapertrailSimpleDemo:  [system] WARN: Resetting WLAN due to 2 failed connect attempts
Dec 23 12:02:34 Particle PapertrailSimpleDemo:  [system] WARN: Internet available, Cloud not reachable!
Dec 23 12:02:34 Particle PapertrailSimpleDemo:  [system] WARN: Resetting WLAN due to SPARK_WLAN_RESET

[harrisonhjones]

Is your device having a hard time connecting? (that’s what the logs suggest). Was it having this issue before papertrail?

[barakwei]

@harrisonhjones Is there an API to resolve host to IP in Electron?

[rickkas7]

This is the code that I use to resolve DNS on the Electron:

#include "Particle.h"

boolean checked = false;

IPAddress electronResolve(const char *hostname); // forward declaration

const char *testHostname = "www.google.com";

void setup() {
	Serial.begin(9600);
}

void loop() {
	if (!checked) {
		checked = true;

		IPAddress result = electronResolve(testHostname);
		Serial.printlnf("looking up %s got %s", testHostname, result.toString().c_str());
	}
}


static int electronResolveCallback(int type, const char* buf, int len, IPAddress *pResult) {

	// AT+UDNSRN=0,"www.google.com"
	// +UDNSRN: "216.239.59.147"
	// OK
	if (type == TYPE_PLUS && pResult != NULL) {
		char *mutableCopy = (char *) malloc(len + 1);
		if (mutableCopy != NULL) {
			strncpy(mutableCopy, buf, len);
			mutableCopy[len] = 0;

			char *quoted = strtok(mutableCopy, "\"");
			if (quoted != NULL) {
				quoted = strtok(NULL, "\"");
				if (quoted != NULL) {
					int addr[4];
					if (sscanf(quoted, "%u.%u.%u.%u", &addr[0], &addr[1], &addr[2], &addr[3]) == 4) {
						*pResult = IPAddress(addr[0], addr[1], addr[2], addr[3]);
					}
				}
			}

			free(mutableCopy);
		}
	}

	return WAIT;
}

IPAddress electronResolve(const char *hostname) {
	IPAddress result;

	Cellular.command(electronResolveCallback, &result, 10000, "AT+UDNSRN=0,\"%s\"\r\n", hostname);

	return result;
}

[ctmorrison]

Hi @harrisonhjones, well…with the example program that @barakwei provided and if I put his library and code in my own small program, the Photon does have trouble connecting for some yet to be determined reason. If I remove the code from my program, it connects quickly, as normal.

[barakwei]

thanks @rickkas7! I
I have a version of the library with Electron support in a branch if anyone wants to make sure it actually works since I don’t have an Electron.
@ctmorrison - Hope you got it working, I have no idea how the library can cause that.

[harrisonhjones]

Thanks for the awesome library. Just FYI, there is an outstanding request in the firmware repo for a Cellular.resolve() function. I’m going to see if I can get any movement on it.

Also, thinking about adding a section on “community log handlers” to the docs. Would you mind if I added yours?

[barakwei]

That’ll be great. I’m guessing we’ll need something similar for raspberry pi as well.

Feel free to mention the library in the docs or anywhere else.

[micahwedemeyer]

Thanks for the library! I’m having some trouble where not all my log statements are showing up, but pushing logs to PT is already a huge help with debugging.

[barakwei]

I’m glad you find it useful

I saw that after you lose connection to the cloud, you no longer receive events. I intend to fix this of course. I didn’t see random events not showing on papertrail. Which one do you see?

[micahwedemeyer]

I’m finding that log messages just never make it to Papertrail. For example, I have 3 log messages in my setup() function. On startup, the last one shows up in Papertrail about 25% of the time. It would be easier to debug if it never happened, but since it happens sometimes and not others, that makes it very strange.

I’m trying to troubleshoot some kind of freeze-up that happens in my firmware. I had hoped the UDP/Papertrail logging would help, and it has a little, but since I know that a lot of log messages are being lost, I can’t really trust it.

One thought I had was to cache the address that Wifi.resolve() gets back for the Papertrail host. I worry that DNS resolution might take a long time and/or cause UDP packets to back up. That’s just a wild guess though.

[hl68fx]

Do you have SYSTEM_THREAD(ENABLED); in your code?

[barakwei]

The master branch in the git repo includes the IP address cache along with other stuff. I intend to upload this after some more testing, you can give it a try.
Also try to check to return value of m_udp.sendPacket.

[samsonthebrave]

THANK YOU! @barakwei this is exactly what I need. I’m going to be including ParticlePapertrail in all of my projects. I’ll be sure to send you any thoughts/issues that arise for me, but so far so good.

[thrmttnw]

I am new to the platform. Thank you for this - a great step on Photon (with fw 0.6.1).

Unfortunately the same code does not run on Electron with 0.6.1, (and triggers a bug in the IDE, see below). The device breaths green or breaths cyan running the code below.

When breathing cyan, nothing gets sent to papertrail, the particle.variable is not registered, and the devices never starts blinking the blue LED. So the papertrail init seems blocking.

(Insert your own paper trail dem account credentials.)

// This #include statement was automatically added by the Particle IDE.
#include <papertrail.h>


// Slow 1fps blink blue board LED on 0.6.1 Photon and Electron
int led = D7; // blue board LED
int fps = 1000;

/*
SerialLogHandler logHandler(LOG_LEVEL_WARN, { // Logging level for non-application messages
    { "app", LOG_LEVEL_ALL } // Logging level for application messages
});
*/
//
PapertrailLogHandler papertailHandler("logsX.papertrailapp.com", XXXXX, "PapertrailSimpleDemo");

void setup() {
    Particle.variable("fps(ms)", fps);
    pinMode(led, OUTPUT);
    Log.trace("Starting, fps(ms): ", fps);
}

void loop() {
    digitalWrite(led, HIGH);
    delay(fps);
    digitalWrite(led, LOW);
    delay(fps);
}

It also triggers a wrong indication in the IDE:

When breathing cyan the device is listed as online in the console but variables does not show (“no variables registered”).

If I flash to the device breathing cyan via the IDE, the IDE indicates flashing is working fine but the device does not actually get flashed. Nothing shows up in the console log if active during the flashing.

So the device is probably online but can not be interacted with due to blocking code, so the “flash succesful” message is wrong:

followed by

This is on Mac OS 10.12.4 (16E195) with Safari Version 10.1 (12603.1.30.0.34)

I would appreciate to know if others have the lib running on Electron, thanks. It is a great debug option to have!

[barakwei]

Hi @thrmttnw
The git repo also has an issue on this matter. Unfortunately I’m still waiting for someone with an electron to debug this.

I’d really appreciate if anyone would help in this matter
Barak (still waiting for my Electron to arrive)

[thrmttnw]

I looked in to it briefly, adding the library code in two files to modify with the above example.

• Replacing Cellular.resolve() with a manually resolved IP made no difference

• Commenting out udp.begin() and udp.send() and just ret=1 in both cases, the rest of the code runs fine!

• I tried to only do udp.begin() if particle.connected(), but it still blocks led blinking

It’s hard to send UDP packets without .begin() and I can’t go down that rabbit hole.

The change log for Electron states another difference between Photon and Electron:
https://docs.particle.io/reference/changelog/#udp-messaging

and “you can always switch protocols for your device in firmware” would be the next thing I would try, if I could find out how to do it …

[thrmttnw]

I also tried putting in delay(20) just before udp.begin(). Then the device connects, breaths cyan, and starts blinking the blue LED after downloading the binary, but a minute later or so, the device stops blinking the blue LED, still breathing cyan.

Pressing the reset button does not start LED blinking again. Only flashing the same binary again results in the blue LED blinking again, until about a minute later it stops.

So udp.begin seems to be causing a problem at system level.

[thrmttnw]

Seems like udp might be broken in 0.6.1

Help!? UDP send in LogHandler hangs electron w/ 0.6.1

Hi peeps, I’m trying to create a LogHandler that ships data out UDP, similar to the PaperTrail loghandler at https://github.com/barakwei/ParticlePapertrail. (note – their example also hangs for me) I can normally send UDP just fine, but if my class extends the LogHandler class, it hangs when I send. I whipped up a tiny example to illustrate. I’m using an electron with 0.6.1, and can see in logs that it’s booting successfully, connecting to cloud, etc. Anyone have any ideas? Thanks! - Henry …

What prevents the library from running on 0.6.0?