It sounds like they don't care much for each other...If they are physically fighting, then you absolutely cannot leave them out together unsupervised, obviously, but you also need to remember that it takes only a split-second, literally, while you're sitting and watching them closely together, for one of them to lose an eye, a toe, or suffer a fatal head wound, or a wound deep enough for them to bleed to death...
As long as you do have them away from their cages (it's good that you thought of this) and they are still actually fighting, unfortunately it sounds like they aren't going to be very compatible...It's not unheard of that two birds learn to "tolerate" each other over time, however that isn't usually the case...So it's your responsibility to do what is best for them, and keep them apart to prevent something tragic happening...If you keep their cages in the same room then it's possible that they'll learn to get along, or at least not fight, but I wouldn't count on that happening at all...
I know this situation is tough, I have 3 birds that get along great, and then a fourth that doesn't get along with anyone....Oh, and a Ringneck Dove who is now the supreme ruler of my house...So I have to portion my time between the 3 and then the other, and that can be trying...But it's a part of owning multiple birds, same as buying birds to breed them, there is never any guarantee that they will even get along at all, let alone bond closely or mate...