Pi SSH no longer responds after installing & setup of agent


#1

@jvanier, after doing this three times, I now realize that I am not the problem.

I installed newest version of Raspbian on a Pi2 (with a Redbear IoT Hat) and did a sudo apt-get update/sudo apt-get upgrade to brings things up to date. I then install the particle-agent as per instructions and run through setup to register the Pi to my account. So far so good. I then run sudo service particle-agent status and status is reported back as expected. I then run it again, and this is where things go south. Now, the Pi responds to the command in a minute or so! Trying to open another SSH session doesn’t get a login prompt for a minute either.

The status I get is:

pi@raspberrypi:~ $ sudo service particle-agent status
● particle-agent.service - LSB: Manages Particle firmware
   Loaded: loaded (/etc/init.d/particle-agent)
   Active: active (running) since Fri 2016-12-02 13:55:13 EST; 9min ago
  Process: 4616 ExecStop=/etc/init.d/particle-agent stop (code=exited, status=0/SUCCESS)
  Process: 4628 ExecStart=/etc/init.d/particle-agent start (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/particle-agent.service
           ├─4638 /usr/bin/ruby /usr/bin/particle-agent-service --daemon --pi...
           └─4647 /var/lib/particle/devices/7ab705654cee244cacfd1db4/firmware...

Dec 02 13:55:13 raspberrypi particle-agent[4628]: Starting Particle Agent: pa...
Dec 02 13:55:13 raspberrypi systemd[1]: Started LSB: Manages Particle firmware.
Hint: Some lines were ellipsized, use -l to show in full.

Running top is very sluggish, updating every few seconds. The firmware.bin process only accounts for 5.5% of the CPU time. When I run sudo service particle-agen stop, it takes a minute to stop and the system still took a long time to respond UNTIL I ran sudo service particle-agent status and got:

pi@raspberrypi:~ $ sudo service particle-agent status
● particle-agent.service - LSB: Manages Particle firmware
   Loaded: loaded (/etc/init.d/particle-agent)
   Active: inactive (dead) since Fri 2016-12-02 14:09:31 EST; 2min 49s ago
  Process: 4713 ExecStop=/etc/init.d/particle-agent stop (code=exited, status=0/SUCCESS)
  Process: 4628 ExecStart=/etc/init.d/particle-agent start (code=exited, status=0/SUCCESS)

Dec 02 13:55:13 raspberrypi particle-agent[4628]: Starting Particle Agent: pa...
Dec 02 13:55:13 raspberrypi systemd[1]: Started LSB: Manages Particle firmware.
Dec 02 14:09:30 raspberrypi systemd[1]: Stopping LSB: Manages Particle firmw....
Dec 02 14:09:31 raspberrypi particle-agent[4713]: Stopping Particle Agent: pa...
Dec 02 14:09:31 raspberrypi systemd[1]: Stopped LSB: Manages Particle firmware.
Hint: Some lines were ellipsized, use -l to show in full.

Running sudo reboot seems to just hang. Any thoughts?


Raspberry Pi Zero w/IoT Hat (No Connectivity)
#2

Thank for reporting this. One recent change I’ve made is set all IO pins to INPUT when the firmware starts. This may have unexpected side effects.

See I2c not working for similar sluggishness reported when I2C pins were set as INPUT.

I’ll need to investigate on a Pi 2.


#3

I created an issue for this:


#4

@jvanier, I wonder if Redbear IoT hat may be the issue since it sits on the IO expander.


#5

Would like to know as ordered a few pHat’s for some zero’s in anticipation, would be terrible if they affected it in that way.


#6

@jvanier, if I turn off the Pi, remove the Redbear IoT Hat and power back up, all problems go away. So the agent is screwing up the IO lines used by the hat. This board is popular so finding a fix is important IMO :wink:


#7

Thanks for investigating this.


#8

This may be an unrelated issue, but this thread was the only pertinent discussion I could find regarding firmware.bin CPU usage.

I found that on a completely fresh Raspbian install, after installing the particle-agent and booting without network connectivity “firmware.bin” uses 100% of the CPU. This is on a Pi Zero W, so there isn’t much CPU to go around. I was running the example code from the docs that pulls CPU temperature modified to wait 30 seconds instead of broadcasting every second.

I was able to reign in the CPU runaway by issuing sudo service particle-agen stop Once I got it connected to a WiFi network I ran sudo service particle-agen start and the CPU stabilized to a reasonable level (2%-6%).

I’m using the latest drop of Raspbian from November. I haven’t done much on it so far as I burned it out in order to play with Particle integration. The only other factor that might be at play is that I started the image on a Zero, moved it to a Pi 3 to perform the installation, and moved it back to the Zero. I’ve found that older versions had issues running on a Zero if that is not the first hardware it sees on boot.


#9

The Pi firmware doesn’t handle not having a network connection well. It aggressively tries to reconnect to the cloud as you saw. It’s just something we haven’t had time to fix.