I have not looked at this in detail but as a clue for @BDub the numbers that @WebDust21 gets (~3890) can be explained by the 10 sample average code getting 9 samples at 4095 and 1 sample at 2048 (half scale).
We saw some similar problems in the distant past on the Core and the @satishgn updated the averaging code to interact better with the DMA channel. So I would be looking for a timing problem of that type. As @BDub said, turning off the averaging would help debug the problem but will add noise of plus or minus several counts.
Because of the way the switched-capacitor input works (and the diagram in that app note is simplified from what I have seen in the past) the input impedance is not returned to ground at each measurement cycle but to some pre-charged level in the switched capacitor bank. Essentially each bit in the SAR computation has a different input impedance relative to ground. You should plan on driving the input with a source impedance of less than 20k ohms for good results. A lot of folks have had trouble with these inputs and certain sensors, but a good op-amp buffer always fixes the impedance problems in my experience.