You were in the space with him, so even if he hadn't touched the cage with his mouth, there is serious risk *if the bird is actively sick or even just a carrier*--the fact that your bird touched the cage with his mouth makes it even more risky, but just entering the home was risky. Again, you won't know the extent of the risk unless you know what is wrong with the other bird specifically, as certain viruses float in the air on invisible dander etc. That is why super fancy boarding facilities have their own air systems for each enclosure.
It's important to remember that even a bird in perfect health with non-watery poop etc (perfect feathers, no symptoms at all) can spread deadly disease (most common when it comes to PDD, ABV and PBFD but not limited to), which is why they shouldn't be around others unless it is absolutely essential. Some of the scariest diseases are extremely common in terms of asymptomatic spread, and yours could be a carrier too, so this isn't just about your bird's safety, but also about the safety of other birds.
Going to a place like a vet is even risky but it's far less risky than entering the home/air space where a bird has been living because their HVAC etc is going to be pumping particles around the house