# \[SOLVED\] Exceeding Vector array at 4097th value

**URL:** <https://community.particle.io/t/solved-exceeding-vector-array-at-4097th-value/24735>\
**Category:** Troubleshooting\
**Created:** [July 30, 2016, 9:56pm UTC](https://community.particle.io/t/solved-exceeding-vector-array-at-4097th-value/24735 "2016-07-30T21:56:59Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![icedMocha](https://sea2.discourse-cdn.com/flex026/user_avatar/community.particle.io/icedmocha/32/18854_2.png) [@icedMocha](https://community.particle.io/u/icedMocha)\
**Post date:** [July 30, 2016, 9:56pm UTC](https://community.particle.io/t/solved-exceeding-vector-array-at-4097th-value/24735/1 "2016-07-30T21:56:59Z")

</div>

Hi All. I’m playing with vectors for the first time on my Photon and I’ve encountered a problem. Whenever I go beyond 4096 values (12 bit #) in the array, I get garbage data.

```
#include <vector>
#include <numeric>

int sensorPin1 = A0, arraySizeMax, arrayCapacity;
std::vector<int> calibrationArray;

void setup() {
    pinMode(sensorPin1, INPUT);
    arraySizeMax = 4095;
    calibrate(sensorPin1);
    arraySizeMax = 4096;
    calibrate(sensorPin1);
}
    
int calibrate(int pinToCalibrate) {
    while (calibrationArray.size() <= arraySizeMax) {
        calibrationArray.push_back(analogRead(pinToCalibrate));
    }
    Particle.publish("Array Capacity", String(calibrationArray.capacity()), PRIVATE);
    delay(1000);
    Particle.publish("Initial Value", String(calibrationArray.front()), PRIVATE);
    delay(1000);
    Particle.publish("Final Value", String(calibrationArray.back()), PRIVATE);
    delay(1000);
    calibrationArray.clear();
}

void loop() {
}

```

#Results:  
Array Capacity 4096  
Initial Value 503  
Final Value 16392  
Array Capacity 8192 \<—looks like additional memory has been allocated but…  
Initial Value 537001984 \<—garbage  
Final Value 0 \<—garbage

---

<div class="post-metadata">

**Author:** ![icedMocha](https://sea2.discourse-cdn.com/flex026/user_avatar/community.particle.io/icedmocha/32/18854_2.png) [@icedMocha](https://community.particle.io/u/icedMocha)\
**Post date:** [August 1, 2016, 3:14am UTC](https://community.particle.io/t/solved-exceeding-vector-array-at-4097th-value/24735/2 "2016-08-01T03:14:57Z")

</div>

I’ve cleaned up the code a little to make the issue clearer. Could really use help on this one, I’m not getting anywhere.

---

<div class="post-metadata">

**Author:** ![BulldogLowell](https://sea2.discourse-cdn.com/flex026/user_avatar/community.particle.io/bulldoglowell/32/4305_2.png) [@BulldogLowell](https://community.particle.io/u/BulldogLowell)\
**Post date:** [August 1, 2016, 3:24am UTC](https://community.particle.io/t/solved-exceeding-vector-array-at-4097th-value/24735/3 "2016-08-01T03:24:55Z")

</div>

what happens if you don’t call clear() and just add another element the 2nd call to the sampling function?

---

<div class="post-metadata">

**Author:** ![bko](https://sea2.discourse-cdn.com/flex026/user_avatar/community.particle.io/bko/32/20704_2.png) [@bko](https://community.particle.io/u/bko)\
**Post date:** [August 1, 2016, 3:45am UTC](https://community.particle.io/t/solved-exceeding-vector-array-at-4097th-value/24735/4 "2016-08-01T03:45:05Z")

</div>

Hi @icedMocha

I don’t know what size std::vector will preallocate on this platform, but it probably is not what you wanted anyway. Why trust the memory management to be perfect on an embedded processor? I would preallocate the std::vector to the size I needed. If you need lots of dynamic memory allocation for 16kbyte chunks (out of a total of 128kbytes of RAM so each one is 1/8 of total memory) this is likely to be not the best platform for your needs.

You can preallocate the vector size by adding a size argument to constructor or by calling the reserve method later. Either way will likely save you some heartache.

---

<div class="post-metadata">

**Author:** ![icedMocha](https://sea2.discourse-cdn.com/flex026/user_avatar/community.particle.io/icedmocha/32/18854_2.png) [@icedMocha](https://community.particle.io/u/icedMocha)\
**Post date:** [August 1, 2016, 12:48pm UTC](https://community.particle.io/t/solved-exceeding-vector-array-at-4097th-value/24735/5 "2016-08-01T12:48:28Z")

</div>

> [@BulldogLowell](#):
>
> what happens if you don't call clear() and just add another element the 2nd call to the sampling function?

No change.

---

<div class="post-metadata">

**Author:** ![icedMocha](https://sea2.discourse-cdn.com/flex026/user_avatar/community.particle.io/icedmocha/32/18854_2.png) [@icedMocha](https://community.particle.io/u/icedMocha)\
**Post date:** [August 1, 2016, 1:00pm UTC](https://community.particle.io/t/solved-exceeding-vector-array-at-4097th-value/24735/6 "2016-08-01T13:00:41Z")

</div>

> [@bko](#):
>
> Hi @icedMocha
> 
> I don't know what size std::vector will preallocate on this platform, but it probably is not what you wanted anyway. Why trust the memory management to be perfect on an embedded processor? I would preallocate the std::vector to the size I needed. If you need lots of dynamic memory allocation for 16kbyte chunks (out of a total of 128kbytes of RAM so each one is 1/8 of total memory) this is likely to be not the best platform for your needs.
> 
> You can preallocate the vector size by adding a size argument to constructor or by calling the reserve method later. Either way will likely save you some heartache.

I added `calibrationArray.reserve(8192);` above the while statement. Looks that fixed the problem. Thank you!

As far as why to trust the memory management of an embedded processor, I guess I'd ask why not? (Honest question) This problem is forcing me to understand more about memory management than I have before so I'd be interested to know more.

I believe the way std::vector allocated memory is through exponential growth, so each time I hit a limit it reserves double the memory, or something approximating that. I suppose I could control that more tightly through the reserve function. I was previously calling the callibration.capacity function to make sure I had enough space, but I guess the value that returns has nothing to do with the actual capacity of the hardware.

---

<div class="post-metadata">

**Author:** ![bko](https://sea2.discourse-cdn.com/flex026/user_avatar/community.particle.io/bko/32/20704_2.png) [@bko](https://community.particle.io/u/bko)\
**Post date:** [August 1, 2016, 1:10pm UTC](https://community.particle.io/t/solved-exceeding-vector-array-at-4097th-value/24735/7 "2016-08-01T13:10:01Z")

</div>

Hi @icedMocha

Memory is a scarce resource on any small platform like this and managing it closely when dealing with objects that are large proportion of the total available memory is often required. If you don’t want to understand the limitations of memory here, you will need to stick to smaller objects in memory. Your first object is 1/8 of the total RAM and the second time through, I suspect you took up 1/4 of the available memory. That memory is shared with the networking code and RTOS, so you will have problems if you take too much. If you were working with 100 elements vectors, I would not have necessarily suggested preallocation, but if you really do know the size or have a max size, then preallocation often really helps.

The exponential growth (doubling) on a push that has no room works great on big computers like PCs with lots of physical RAM and virtual memory too. But on a smaller processor like Particle’s it can run wild and take too much.

---

<div class="post-metadata">

**Author:** ![ScruffR](https://sea2.discourse-cdn.com/flex026/user_avatar/community.particle.io/scruffr/32/6952_2.png) [@ScruffR](https://community.particle.io/u/ScruffR)\
**Post date:** [August 1, 2016, 1:32pm UTC](https://community.particle.io/t/solved-exceeding-vector-array-at-4097th-value/24735/8 "2016-08-01T13:32:40Z")

</div>

An additional point might be, that the free space you get reported does not mean that this space is available as one big chunk, but might be scattered due to fragmentation all over the place. So you could fit that many small elements into the free gaps, but not one single big one.  
If there was a function that reported the biggest chunk of mem available, this woukd be the figure to check against.

Hence, for µCs, you _need_ to care about memory management, due to the limitations.

---

<div class="post-metadata">

**Author:** ![icedMocha](https://sea2.discourse-cdn.com/flex026/user_avatar/community.particle.io/icedmocha/32/18854_2.png) [@icedMocha](https://community.particle.io/u/icedMocha)\
**Post date:** [August 1, 2016, 3:14pm UTC](https://community.particle.io/t/solved-exceeding-vector-array-at-4097th-value/24735/9 "2016-08-01T15:14:13Z")

</div>

> [@ScruffR](#):
>
> An additional point might be, that the free space you get reported does not mean that this space is available as one big chunk, but might be scattered due to fragmentation all over the place. So you could fit that many small elements into the free gaps, but not one single big one.  
> If there was a function that reported the biggest chunk of mem available, this woukd be the figure to check against.
> 
> Hence, for µCs, you _need_ to care about memory management, due to the limitations.

So is it correct to say that the correct memory addresses won't necessarily be referenced if the array isn't stored in one big chunk? I had just assumed the system would add some kind of marker for where to jump for the next piece of data. (Why isn't my mysterious magical box working??? 😖 )

---

<div class="post-metadata">

**Author:** ![ScruffR](https://sea2.discourse-cdn.com/flex026/user_avatar/community.particle.io/scruffr/32/6952_2.png) [@ScruffR](https://community.particle.io/u/ScruffR)\
**Post date:** [August 1, 2016, 3:22pm UTC](https://community.particle.io/t/solved-exceeding-vector-array-at-4097th-value/24735/10 "2016-08-01T15:22:56Z")

</div>

That’s what a `vector` is: a contiguous storage  
[http://www.cplusplus.com/reference/vector/vector/](http://www.cplusplus.com/reference/vector/vector/)

To have objects that can be scattered, you could use _lists_, _sets_ or _maps_