Sheetrock, drywall, gypsum or whatever name you might know it by is a calcium product that can actually be fed to companion birds...it is basically plaster of paris.....
If it has been painted, no, it would not be safe to eat.....
Windowsills and other moldings can be made from pine, fir, poplar and any of many other woods and without really knowing which specific wood a molding is made from it is difficult to honestly say yes or no on your safety question, bu if it is a light colored, clear grain wood, it is probably alright that the bird redesign the molding, though painted surfaces would not be safe chewing items.....
Unless your bird is going to be sanding patches for you, I don't think you'll have any appreciable dust to worry about...it would be equal to powdered calcium you might use for the bird.....