If he is breaking it into bits, it just depends..Is it hard plastic, or rubber? or silicone? Are the chunks sharp?
If I do get kids toys for my bird, I usually try to look for the non-gummy kids that are BPA free, but I am always cautious because Max's dad (cockatoo youtube guy) has found all sorts of random stuff in kids toys that Max destroyed--- like metal screws inside of seemingly solid wood children's building blocks. Basically, safe for a kid may mean SAFER for a bird, but babies still don't have teeth and some of the paints etc that are safe for them wouldn't necessarily be safe for a bird.
The insides are not intended to be exposed in most cases too...so sometimes there is unexpected stuff inside of them (glues, metals, choking hazards etc).
It also depends on if he is possibly ingesting any of it, and that will be partly to do with your bird, but also partly do with the material it's made of.
I doubt any damage has been done at all, I'm sure he's fine, but I probably wouldn't leave it in there without thinking through the type of plastic, his tendencies, choking hazards, sharp edges etc etc.
Hard to say without seeing it and knowing more about the toy and your bird himself.