Brian, don't get too frustrated or disappointed. I've found that some toys can freak a bird out...it could be the size, the color, or just something about it that they don't like. Chipper freaked out with me before, when I tried to put a big toy in his cage. So now what I do is take him out of the cage first and hold him and pet him, or just let him perch somewhere else in the room for a few minutes while I put the toy in his cage. Then I put him back in his cage. If you try to put the new toy in there while the bird is still in the cage, he might think some kind of monster is attacking him and he'll panic. But if you put the bird in the cage with the toy already there, he'll just see it as some kind of weird new thing that he has to get used to. And after a little while, he'll accept it.
Birds do hold a grudge, but not for that long. When I tried to give Chipper medicine when he was sick, back in December, he got REALLY mad at me. For a while, he wouldn't even let me pet him. He would make noises and then bite me hard. I was afraid that he was never going to trust me again and he wasn't going to be friendly anymore. But I got his trust back. I bribed him with spray millet, and got him to eat it from my hand and talked to him softly. Then he let me pet him. It took a little while, but eventually Chipper got over it and we were friends again.
Don't worry, and don't give up. Just be patient and gentle, and try to bribe him with treats. Try to get him to eat millet from your hand (but don't pet him yet). He's just nervous and a little upset with you right now. But he'll get over it. Just keep thinking that. He's just nervous right now, but he will be your friend again. Be patient and stay positive. Animals can sense your mood, especially birds, so you want to stay calm and positive. If you're nervous or afraid, he'll be nervous too. So just stay calm. He'll be friendly again, it just might take a little while.