You're absolutely right Sarah! The nature of the feedback has very important implications for what the system ends up learning. If you train them off of 1 month returns, you'll probably get a trader. User 3 year returns and you might get something closer to an investor. In fact, perhaps the real point is to not train them just based on returns but with some kind of weighting based on whether the returns were bad because of fundamental reasons or because of completely unpredictable events (the Kobe earthquake).

Perhaps the reason there are fewer good long term investors is because of the long feedback cycles.

The good thing about training machines is that the learning becomes transferable and can accumulate learning for far longer than people.