Advice RE replacing Stitch, Event data to BigQuery

Cloud Integrations
1

Hi All,

I am hoping for some advice. We have been successfully using stitch web hooks to transfer particle event data into BigQuery. Unfortunately we will be moving away from using stitch for pricing reasons.

I’m wondering if anyone has any advice about what we should use to replace Stitch, should I push data into Google pub/sub and then somehow transfer it into BigQuery?

Thanks in advance and apologies if this has been covered elsewhere. I had a look, but couldn’t find anything particular.

2

I would use the Google Pub/Sub integration. Then from Google Pub/Sub pick up the event with either a Google cloud function, or using AppEngine. Both methods work and they can use the Google APIs to write the data to BigQuery.

This tutorial may help; it shows both methods for writing to other datastores, but does not specifically use BigQuery. But you just swap out using Cloud Datastore or Firebase APIs for the BigQuery API.

https://github.com/rickkas7/google_cloud_tutorial

1 Like
3

Thanks Rick, I’ll give the above a go and see what happens.