Spark.publish is popular!
Well, that didn’t take long! After rolling out Spark.publish today, the message counts quickly spiked and consumed a large chunk of server resources. We found some users publishing 2,000-4,000 messages a minute (probably by accident)!
We expected a flood of events, but it’s important everyone gets a chance to publish events.
A few hundred cores monopolizing the event stream wouldn’t be a good user experience. For that reason, (for now…) we think setting a maximum rate of 60 events a minute per core, with a burst rate of up to 4 per second, should allow for almost all use cases.
If you need to move more data / more quickly, it might make more sense to do a direct TCP / UDP socket instead of publishing an event. Please feel free to weigh in! We will continue to improve and optimize things, so this limit may change down the road.