Consistent timing for SPI bus

Here is my basic setup. I am using the photon2 to program an AT28C256 EEPROM. Given its address and data lines I am using two MCP27S17 GPIO extenders that I can communicate to via SPI rather than the traditional ones most use which is slower I2C based.

When programming the EEPROM I need to take advantage of the page write capability so that it doesn't take 20 minutes to program. Haha. Basically I need to setup the address and data lines, then toggle the WE pin and then move on to the next address and data and repeat the process such that these writes don't take longer than 150 microseconds between each write. A page is 64 bytes long. Once I have written 64 bytes (the lower 7 address lines) I pause for slightly more than 10 milliseconds to wait for the page write to occur.

Note that any delay of greater than 150 microseconds between writes will trigger the page write process. So when I am doing my 64 byte chunks my timing is critical that I get all my wok done in under 150 microseconds.

Seems like something a MCU that is close to bare metal should handle. However I am running into difficulty as there appears to be random spots where the timing from one byte write to the next is delayed wich will trigger the page write too soon.

I am using a DLA to look at the timing and here are some screen shots showing what I am seeing. Note the top row is just a GPIO I am toggling to measure the total time to write all 64 bytes, the middle row is the actual SCK line of the SPI bus which is running at 12Mhz, and the 3rd row is the WE line to the EEPROM being toggled.

The first shot shows a capture and the all look really good

However, as you zoom in you can see that certain page writes will trigger early.

Here is the one page write with a timing delay.

And here is another that is perfect.

Regarding my code. I use

SYSTEM_MODE(MANUAL)
SYSTEM_THREAD(DISABLED)

Normally I will be getting data to burn to EEPROM via BLE and I won't do the burning process until all is received, however, in this test case, I am not enabling any BLE or Serial.

I'm just in this loop

void EEPROM_AT28C256::programEEPROM() {
  digitalWriteFast(ROM_PROG_L, LOW); // take control of the EEPROM
  driveAddressBus();
  driveDataBus();
  digitalWrite(OE_L, HIGH);
  digitalWrite(WE_L, HIGH);
  uint16_t address = 0;
  for (uint16_t pageCount = 0; pageCount < sizeof(data) / 64; pageCount++) {
    address = pageCount * 64;
    digitalWriteFast(BE_L, LOW);
    for (int i = address; i < address + 64; i++) {
      setAddress(i);
      setData(data[address]);
      digitalWriteFast(WE_L, LOW);
      digitalWriteFast(WE_L, HIGH);
    }
    digitalWriteFast(BE_L, HIGH);
    delay(20); // 
  }  
  digitalWriteFast(ROM_PROG_L, HIGH); // Give control back to 6502
}

Here is my code that actually uses the SPI bus.

void EEPROM_AT28C256::writeRegister(uint8_t cs, uint8_t reg, uint8_t value) {
  SPI.beginTransaction();
  digitalWriteFast(cs, LOW); // Select the MCP23S17
  SPI.transfer(WRITE_CMD);  // Send write command
  SPI.transfer(reg);       // Send register address
  SPI.transfer(value);     // Send value
  digitalWriteFast(cs, HIGH); // Deselect the MCP23S17
  SPI.endTransaction();
};

My tendency is to want to disable all interrupts when doing the inner loop page write, but I believe that would break the SPI interface from working properly.

Any thoughts or hints on how I can get this to be consistent?

Hey Kevin!

Thanks for the detailed question. I've spoken with the engineering team and they are currently working on SPI fixes for P2. It should be fixed in 5.6.1.

Let me know if you have any other questions!

2 Likes

Hooray! Thats great to know. I have been planning to do a retry if a page write fails but I will for go that development knowing that this fix is coming.

2 Likes

Hi there! DeviceOS 5.7.0 has been released with the fixes discusses above. Could you please test and let us know the results?

1 Like

Thanks Kevin!

u bet. I'll do this today.

1 Like

Well, unfortunately it looks like it still has this issue. I first took my code and ran it again against 5.6. The issue duplicated as expected. I then upgraded the Photon 2 to 5.7, did a clean of both os and application and reflashed both. The issue still seems to occur. Here are my traces.

First screen shot shows how you visually see easily when it happens. Second and third shots are just diving in to more detail.


Here is the first zoom in.

And finally in enough to see the exact time when the gap occurs.
![Screenshot 2024-01-26 at 11.55.44 AM|690x445]
(upload://eMxSd0bfb4va1zdQ7n2e9iRf1B1.png)

I'm happy to post my code if that helps. Note I am running just with the Photon on a breadboard without any other devices attached. This particular test is programming a EEPROM and does not need to read anything, it just blindly writes 64 bytes.

@no1089 Hi Chris. Just got off a zoom call with @Colleen and that you guys may be having trouble duplicating the issue. Here is my code. It is not perfectly reduced to just the bare minimum as it comes out of a bigger project but it will definitely reproduce the issue on a Photon 2 with no other hardware attached.

#include "Particle.h"

SYSTEM_MODE(MANUAL);
SYSTEM_THREAD(DISABLED);

const byte IODIRA = 0x00; 
const byte IODIRB = 0x01;
const byte GPIOA = 0x12;
const byte GPIOB = 0x13;
const byte WRITE_CMD = 0x40; // Write command (0b0100 A2A1A0 0)
const byte READ_CMD = 0x41;  // Read command (0b0100 A2A1A0 1)

// Control the Address & Data Lines
#define DATA_SEL_L D0
#define ADDR_SEL_L D1
#define MCP_RST_L S4
#define BE_L D4

// Controlling the EEPROM
#define ROM_PROG_L D6
#define OE_L D3
#define WE_L D5

void writeRegister(uint8_t cs, byte reg, byte value) {
  SPI.beginTransaction();
  digitalWriteFast(cs, LOW); // Select the MCP23S17
  SPI.transfer(WRITE_CMD);  // Send write command
  SPI.transfer(reg);       // Send register address
  SPI.transfer(value);     // Send value
  digitalWriteFast(cs, HIGH); // Deselect the MCP23S17
  SPI.endTransaction();
}

void driveAddressBus() {
  writeRegister(ADDR_SEL_L, IODIRA, 0x00);
  writeRegister(ADDR_SEL_L, IODIRB, 0x00);
}

void driveDataBus() {
  writeRegister(DATA_SEL_L, IODIRB, 0x00);
}

void floatAddressBus() {
  writeRegister(ADDR_SEL_L, IODIRA, 0xFF);
  writeRegister(ADDR_SEL_L, IODIRB, 0xFF);
}

void floatDataBus() {
  writeRegister(DATA_SEL_L, IODIRB, 0xFF);
}

// never reading the address
void setAddress(uint16_t value) {
  uint8_t highByte = (value >> 8) & 0xFF;
  uint8_t lowByte = value & 0xFF;

  writeRegister(ADDR_SEL_L, GPIOA, highByte);
  writeRegister(ADDR_SEL_L, GPIOB, lowByte);
}

void setData(uint8_t value) {
  uint8_t lowByte = value & 0xFF;

  writeRegister(DATA_SEL_L, IODIRB, 0x00);
  writeRegister(DATA_SEL_L, GPIOB, lowByte);
}
uint8_t getData() {
//  SPI.beginTransaction(SPISettings(10000000, MSBFIRST, SPI_MODE0));
  SPI.beginTransaction();
  digitalWrite(DATA_SEL_L, LOW); // Select the MCP23S17

  SPI.transfer(READ_CMD);  // Send read command
  SPI.transfer(GPIOB);     // Send GPIOB register address
  byte value = SPI.transfer(0x00); // Read the value

  digitalWrite(DATA_SEL_L, HIGH); // Deselect the MCP23S17
  SPI.endTransaction();

  return value;
}

void resetMCP() {
  pinMode(MCP_RST_L, OUTPUT);
  digitalWrite(MCP_RST_L, HIGH);
  digitalWrite(MCP_RST_L, LOW);
  digitalWrite(MCP_RST_L, HIGH);
}

void writeByteToEEPROM(uint16_t address, uint8_t value) {
  digitalWrite(OE_L, HIGH);
  digitalWrite(WE_L, HIGH);
  driveAddressBus();
  driveDataBus();
  setAddress(address);
  setData(value);
  digitalWrite(WE_L, LOW);
  delay(10);
  digitalWrite(WE_L, HIGH);
  delay(200);
}

uint8_t readFromEEPROM(uint16_t address) {
  digitalWriteFast(OE_L, HIGH);
  driveAddressBus();
  floatDataBus(); // Meaning we can read it when set by EEPROM
  setAddress(address);
  digitalWrite(OE_L, LOW); // assert OE from EEPROM
  delay(100);
  uint8_t result = getData();
  digitalWrite(OE_L, HIGH);
  return result;
}

void setup() {
  pinMode(OE_L, OUTPUT);
  digitalWriteFast(OE_L, HIGH);
  pinMode(WE_L, OUTPUT);
  digitalWriteFast(WE_L, HIGH);
  pinMode(ROM_PROG_L, OUTPUT);
  digitalWriteFast(ROM_PROG_L, HIGH);

  resetMCP();
  pinMode(ADDR_SEL_L, OUTPUT);
  pinMode(DATA_SEL_L, OUTPUT);
  pinMode(BE_L, OUTPUT);

  SPI.begin();
  SPI.setBitOrder(MSBFIRST);
  SPI.setDataMode(SPI_MODE0);
  SPI.setClockDivider(SPI_CLOCK_DIV8);
}

void loop() {
  delay(1);
  digitalWriteFast(D3, LOW);
  for (uint8_t i = 0; i < 64; i++) {
    setAddress(i);
    setData(i | 0x80);
    digitalWriteFast(WE_L, LOW);
    digitalWriteFast(WE_L, HIGH);
  }
  digitalWriteFast(D3, HIGH);
}
1 Like

I'm sorry for the late reply - I'm putting this on the bench today and will discuss the findings with engineering on Friday.

Hey @iitgrad I know I am late to this thread and what I am going to say will not fix the issue you are experiencing with inconsistent SPI timing on the Photon 2.

Is there something in particular that made you select the AT28C256 vs using a 256Kb SPI Serial EEPROM? Parallel chips have much stricter timing restrictions vs their serial counterparts and seems like you could save some BOM costs by going directly to a serial EEPROM.

1 Like

@iitgrad This took a little longer than expected.
I put the Photon two on the Logic Analyser, and played around with the SPI configuration a bit.
It appears that removing the code below results in a clean output:

  SPI.setBitOrder(MSBFIRST);
  SPI.setDataMode(SPI_MODE0);
  SPI.setClockDivider(SPI_CLOCK_DIV8);

You will also notice Loop completes much faster without the extra SPI config.

I'll put this to engineering to discuss on Friday.


EDIT - upon further inspection I don't think this is the issue. Without the clock devision the transaction takes 45ms instead of the 5ms it should. I'll dig deeper.

This definitely relates to the SPI clock speed though - the clock is stable when running without the divider, but has ~90 microsecond gaps that also affect all the other pins.

@no1089 Hooray, you duplicated. Step 1 complete. :slight_smile:

I had a chat with engineering about this earlier today.

  1. I've replicated on an Argon, so it's not isolated to the Photon 2. The Photon 2 actually performs much better (Guess that 200MHz is worth something!)

  2. The issue will be formally raised with engineering on Friday - the cause is likely an interrupt from the system thread. We attempted to isolate the thread so it won't be interrupted, but it did not work.

  3. (and this is the tricky one) Engineering raised concerns about doing this kind of tightly timed operation on Particle hardware - especially once you throw in other sources of interrupts in the mix (such as servicing connectivity). As Erik mentioned above, a true SPI EEPROM would likely perform better, or if you are locked to this choice, an intermediate controller would do a good job of ensuring there are no timing issues.

1 Like

Yes, I don't have that luxury as I am using this to interface to specific hardware that requires this EEPROM.

Are we saying that this interrupt may be an issue even if I turn off Serial. When I actually do this programming task I am not doing nor do I need any communication at all. Plus I have SYSTEM_THREAD(DISABLED)

Yes, it appears there is an interrupt that's firing somewhere inside DeviceOS - despite disabled threads, DeviceOS still needs to run and do it's thing.

The concern however is that tightly controlled loops like this are not great when run on a device that needs to service other threads - and once this interrupt is figured out - could mean the operation need to be done offline to avoid being interrupted by DeviceOS managing connectivity for example.

The issue will be raised at Triage today and I'll get back to you with feedback.

@iitgrad , After an extensive discussion, we've determined that this is not a specific implementation of SPI we can support -- SPI itself still works as spec, and this is not a direct use of the peripheral.
It was the determination of our engineering team that the constraints imposed by the design are not really feasible for a device running RTOS code.

The recommendation remains to use a Serial based EEPROM, or an interposer chip if that's not possible.

The P2 also exposes a Pseudo EEPROM on flash if it's the only device that needs to access the memory. (Wear levelling is done by DeviceOS, and you can't reasonably wear out the flash)

Ok, thanks @no1089 for spending some time on this. Luckily it mostly doesn't get interrupted and I have implemented a read-back on each page. If the read-back doesn't match what was written, I write the page again which usually always passes. The reason the other EEPROM solutions don't work is because I am using the Photon 2's in a 6502 computing class which requires the parallel EEPROM. Rather than removing the EEPROM everytime a program change is made, and giving each student a EEPROM programmer (or the chaos of sharing) we simply send the packets over serial or BLE to the P2 and it programs the EEPROM right on the development base board. No chips being removed and replaced over and over. Again, avoiding a recipe for disaster in the class. Haha.

4 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.