Well if his first day in the cage he most likely just needs some time to settle in. You can also try placing perches and toys in a way that he can climb and feel more secure while he investigates his new home.
When I got my lovie I bought her and the cage that she was in at the bird store, so she was pretty much settled in a matter of hours.
Give him/her time. He is probably nervous about his new environment. It could take at a few weeks for it to get used to the new home. They are very intelligent, so moving can be rather stressful for them.