Gesture detection using Particle Core and SparkFun Gesture Sensor

https://github.com/krvarma/Particle-Core-Gesture

Here is a project using Particle Core and SparkFun Gesture Sensor. This is s 3 in 1 sensor which detects Ambient Light and Color, Proximity detection and touchless gesture. SparkFun has created one Arduino library for this sensor which can be used on Particle Core with no modification (only one header files change).

Overview

This project use an Image Slider, which can slide left or right using touchless gestures. When you place your hand over the sensor and move right to left, the sensor will detect as LEFT gesture and the image slider will display next image. Similarly when you move from left to right, sensor will detect RIGHT gesture and image slider will display previous image.

Screenshot

enter image description here

enter image description here

Demo Video

11 Likes

Really neat, whatcha going to use it for next?

1 Like

May be a gesture controlled robot :smile:

1 Like

Hi!

I got that same sensor!!

Any changes you made to the demo code??

@frlobo, couple of small changes, changed the headers, publish event.

Excellent! Thanks for the info!

hi @krvarma. What header and publish changes did you make to the demo code?

@vnalla, nothing much, just removed headers specific to Arduino and added the publish code

if(strlen(szEvent) > 0){
    Serial.println(szEvent);
        
    publishEvent(szEvent);
}

.
.
.

void publishEvent(char* szEvent){
    Spark.publish("GESTURE-EVENT", szEvent, 60, PRIVATE);
}

@krvarma Hey I have this gesture sensor also and I’m trying to use it with the Photon.

I used your .CPP & .H files from the Git link for this project.

I’m using the example codes that Spark Fun provides to testing the various features of this sensor. So first I’m trying the Gesture detection .ino code but I can’t get it to compile. I removed the wiring.h reference and changed the interrupt pin to D3.

Can you get the Gesture Test code provided here to load on the Photon?

    /****************************************************************
GestureTest.ino
APDS-9960 RGB and Gesture Sensor
Shawn Hymel @ SparkFun Electronics
May 30, 2014
https://github.com/sparkfun/APDS-9960_RGB_and_Gesture_Sensor
Tests the gesture sensing abilities of the APDS-9960. Configures
APDS-9960 over I2C and waits for gesture events. Calculates the
direction of the swipe (up, down, left, right) and displays it
on a serial console. 
To perform a NEAR gesture, hold your hand
far above the sensor and move it close to the sensor (within 2
inches). Hold your hand there for at least 1 second and move it
away.
To perform a FAR gesture, hold your hand within 2 inches of the
sensor for at least 1 second and then move it above (out of
range) of the sensor.
Hardware Connections:
IMPORTANT: The APDS-9960 can only accept 3.3V!
 
 Arduino Pin  APDS-9960 Board  Function
 
 3.3V         VCC              Power
 GND          GND              Ground
 A4           SDA              I2C Data
 A5           SCL              I2C Clock
 2            INT              Interrupt
Resources:
Include Wire.h and SparkFun_APDS-9960.h
Development environment specifics:
Written in Arduino 1.0.5
Tested with SparkFun Arduino Pro Mini 3.3V
This code is beerware; if you see me (or any other SparkFun 
employee) at the local, and you've found our code helpful, please
buy us a round!
Distributed as-is; no warranty is given.
****************************************************************/


#include "SparkFun_APDS9960.h"

// Pins
#define APDS9960_INT    D3  // Needs to be an interrupt pin

// Constants

// Global Variables
SparkFun_APDS9960 apds = SparkFun_APDS9960();
int isr_flag = 0;

void setup() {

  // Set interrupt pin as input
  pinMode(APDS9960_INT, INPUT);

  // Initialize Serial port
  Serial.begin(9600);
  Serial.println();
  Serial.println(F("--------------------------------"));
  Serial.println(F("SparkFun APDS-9960 - GestureTest"));
  Serial.println(F("--------------------------------"));
  
  // Initialize interrupt service routine
  attachInterrupt(0, interruptRoutine, FALLING);

  // Initialize APDS-9960 (configure I2C and initial values)
  if ( apds.init() ) {
    Serial.println(F("APDS-9960 initialization complete"));
  } else {
    Serial.println(F("Something went wrong during APDS-9960 init!"));
  }
  
  // Start running the APDS-9960 gesture sensor engine
  if ( apds.enableGestureSensor(true) ) {
    Serial.println(F("Gesture sensor is now running"));
  } else {
    Serial.println(F("Something went wrong during gesture sensor init!"));
  }
}

void loop() {
  if( isr_flag == 1 ) {
    detachInterrupt(0);
    handleGesture();
    isr_flag = 0;
    attachInterrupt(0, interruptRoutine, FALLING);
  }
}

void interruptRoutine() {
  isr_flag = 1;
}

void handleGesture() {
    if ( apds.isGestureAvailable() ) {
    switch ( apds.readGesture() ) {
      case DIR_UP:
        Serial.println("UP");
        break;
      case DIR_DOWN:
        Serial.println("DOWN");
        break;
      case DIR_LEFT:
        Serial.println("LEFT");
        break;
      case DIR_RIGHT:
        Serial.println("RIGHT");
        break;
      case DIR_NEAR:
        Serial.println("NEAR");
        break;
      case DIR_FAR:
        Serial.println("FAR");
        break;
      default:
        Serial.println("NONE");
    }
  }
}

I’m getting lots the following compile errors:

This sensor looks to be pretty dam cool.

@frlobo Did you ever get this working on the Photon?

Try adding #include "application.h"

You also should use the correct I2C pins on the Photon (D0/D1).

And as there is a parallel thread I’ll repeat it should be

volatile int isr_flag = 0;

Photon led glows Cyan [Does not breathe Cyan]

@ScruffR Yea Buddy :wink:

I had to add the Application.H to the .ino + .h files to get it to compile.

It compiles fine. Now I need to see if it runs OK.

Thank you so much.

@ScruffR, @RWB, the "application.h" is already included in the "SparkFun_APDS9960.h". I compiled and flashed the code without modification.

Yea I was trying both your and Sparkfuns .H and .CPP files since I was getting errors. That’s why I was having issues.

1 Like

Providing @RWB had used your lib and not the original, which might be the case judging by the error messages :wink:


A bit too late :wink:

1 Like

@ScruffR I see your providing this tip:

volatile int isr_flag = 0;

I checked out the thread you linked to but instead of guessing I’m getting what your telling me to do I’m going to ask you for clarification.

Do I just add this volatile int isr_flag = 0 To the .ino file or do I replace the current interrupt with it?

Interrupts are something I have not played with much yet.

@RWB, This may not matter for your use case but just in case it might - the SparkFun library does not give up control as long as the sensor is being interacted with, which means it will effectively block your other code from running during a gesture:

This would matter mainly if you want to accomplish other things in the background that can’t afford to be blocked for too long since a gesture’s length depends on how agile your user is, or if you wish to interpret a protracted gesture as a “hold”.

We’ve made modifications to make it nonblocking though they haven’t been extensively tested outside our little organization, but if it interests you we can definitely share by posting our code up on Github. The default settings also leave a lot to be desired in terms of responsiveness so we’ve tweaked them significantly.

@krvarma I think I might have fried my Gesture Board by supplying it with 5v on accident so I have another one ordered up to replace it. W

When I connect the sensor to the Photon it keeps the breathing blue led from breathing and essentially freezes the Photon up. If it disconnect the Gesture Sensor the Photon will start breathing again most of the time.

The Gesture sensor is not supposed to stop the Photon from breathing blue right? Or is it stopping all other code from running like @indraastra is speaking about.

@indraastra Yes I would love to see your code that allows other actions to happen when no gestures are being received.

If you’re running in AUTOMATIC mode, it should be impossible for you to mess with the Photon’s connection to the cloud. Most likely you did mess something up with the sensor board if that’s the case.

@indraastra Yea if 5v will ruin the sensor then its toasted. I have a new one ordered up so I’ll come back and confirm if that was the problem or not.

So are you saying that if I want to run this Sparkfun demo code for this sensor because of the interrupt I’ll need to use code similar to yours to keep the loop from being blocked the while time the sensor is sitting idle?

Not at all, so by all means, please use the library for as long as it suits your purposes! When the sensor is idle, there’s no issue because there’s no interrupts firing and therefore no processGesture being called. That’s why I said “as long as the sensor is being interacted with”, so for example if you held your hand in front of the sensor for 10 seconds, your main loop would be blocked for those 10 seconds because it would be processing gestures in the library’s loop (which I linked to) the entire time. You may not have to worry about that situation if you expect your gesturers to be well-behaved and not execute this most basic of DOS attacks on your firmware :stuck_out_tongue: