I would say it is about careful observation and dumb luck. The dumb luck is where you stumble on some odd behavior. Careful observation allows you to determine WHY it happened, what other situations it could happen, natural consequences of it, and how it can be used to save time.
For example, when making my
Batman Begins TAS, I noticed that when you threw a bat star in the air there was a frame where Batman went into standing motion. At first it just seemed like an odd sprite animation flaw but then it occurred to me that the game probably thinks he is on the ground for that frame. I tried jumping on that frame and sure enough he jumped! Cool double jumping. But then it occurred to me I could throw a bat star during that 2nd jump which means I could jump again, and then again, etc. Infinite jumping. Now there is a LOT of uses for that!
However, it didn't stop there. I tried this out to get to crazy places in the game to see what happens. If you go over a ceiling you will fall inside it. The game treats walls as open on top. Once you are in, it is if there is not all at all. This led to even more shortcuts.
The moral of the story is to have try lots of things and use careful observation and creativity to find what can be done with your discoveries.
Also many tricks are common on many games, pages such as
Common tricks can be quite helpful.