Is your B-SoM in the evaluation board or on your own custom board? In either case, my guess is that the power manager is causing issues for you.
You cannot use I2C slave mode on primary I2C (D0/D1, Wire) on boards with the PMIC/Fuel Gauge connected to primary I2C and enabled. This includes the bsom and b5som. It does not include the Boron, Tracker, and E404X that use secondary I2C (Wire1) for the PMIC.
In order to use I2C slave on primary I2C on the bsom and b5som you need to disable power management by disabling the SystemPowerFeature::PMIC_DETECTIONfeature. The reason is that I2C multi-master is not supported, and there's no way to stop the power manager from writing to the I2C bus at the same time as the other I2C master. Also it will change the settings of the I2C port, which probably is why slave mode stops working.
I think this should disable power management. You only really need to do it once because the setting is saved in the configuration flash.
void setup() {
SystemPowerConfiguration conf;
System.setPowerConfiguration(conf);
}