Things like tomatoes and bananas, I think, are picked green and they ripen as they're shipped. This means they get fewer nutrients from the soil as they're not absorbing them after they're picked. I think that, and the minimum amount of necessary nutrients to make them grow are factors - because soil naturally depletes quickly, so they just use whatever fertiliser they need for that crop, which is less nutrient dense than the original soil would have been.