Glitches
During my investigation, I not only found the described movement tech that allows you to move faster compared to the naive way, but also looked for ways to beat the game faster altogether.
As it turns out, it is actually possible to clip into a barn in such a way to both count them for your score and have it repel you backwards so you can score it again.
The setup is fairly simple: whether you score or hit the barn only depends on your vertical position, and which direction you are repelled in only depends on your horizontal position.
By entering the barn at the top-most valid height in the first frame, it scores the barn, and by then moving up to collide with it on the second frame while not moving into the barn too much, it repels you backwards.
The balance it tricky to get right, and it's not possible while going full speed, but it can be repeated with the same barn multiple times.
In order to make the second frame collision easier without needing to move vertically inside the barn, you can actually use the difficulty switches yet again.
The left difficulty switch lowers the clearance of the barns, so doing that at the second frame inside the barn guarantees a collision without needing to move vertically.
Doing this 9 times on the first barn of the stage allows beating the game significantly faster than otherwise possible.