If it's a hard set, plaster like compound, you can sand, but a coarse paper helps (100 grit) and you do have to be careful not to scuff up the cardboard face of the plasterboard. I've used an orbital, but having a vacuum cleaner attached is a good idea unless you like cleaning the rest of the room.
If it's a flexible filler, use a sharp knife to vee cut it out and re-do. Cornice cement is a good, solid, quick setting first coat (ever-so-slightly underfill) and when that has set, top with top coat or other easy to sand filler. Try not to sand too much, as it damages the board and you end up having to coat it again to get rid of the fluffy cardboard bits!
The idea way is to smooth it out with the trowel, let it set, then just lightly scrape off any rough bits and paint! (at least, that's what the seasoned pros do!)
See Rod's site
How to plaster, plastering tips, plastering cracks, holes, for all the gory details