So here's the way I see it. All points considered in this topic, I think we can best compare this to, believe it or not, save data corruption.
When you corrupt save data, what you're effectively doing is interrupting the game's access to the save file partway through the process, generating bad data. With an FMV swap, you're interrupting access to the disc partway through the process, generating bad data. User "rog" brought up a good point: that depending on the game, its programming will respond to the swap differently. Not every game responds as favorably as FF9 does to disc swapping. I know it works in most cases in FF7, allowing you to sub in a shorter one (with a few freezes in certain situations), but in others it'll just lock up.
With this in mind, and with DarkKobold's idea that it could potentially be used in interesting and impressive ways to destroy games in ways we only thought possible in the early console days, I'd say the best thing to do is not deny ourselves of that possibility. But when it comes to FMV skipping, I guess we can compare it to
the other controversial debate right now - effectively a way to cut time off an already long run, but not something that should be depended upon alone for saving time. Saying "but we can make an encode that skips the FMVs themselves" isn't really productive/the same for many reasons.
In short, I'd say allow disc swapping mid-game at unintended points, and add a tag to such runs that says the run does as such. In the event of severe corruption via disc swapping, it'd be a different category.
With that in mind, the only issue left is which discs can be swapped in. The best thing to do there IMO is only allow discs belonging to the game itself, as well as a lack of a disc. Otherwise, a run that could switch in an altered disc that just tells the game to trigger the ending or something would technically be allowed.
On other notes, a scratched disc would be best compared to a bad ROM, and thus would be out of the question. Crooked cartridge is a whole different beast, but really isn't comparable to either of these. At best guess, I would say it's like temporarily denying access to certain parts of the program, forcing what's left of it to deal with the situations at hand. Basically like RAM corruption and/or temporary ROM corruption. It's a whole different debate, whatever the case.