A plain bandanna will work as long as it's the right color and doesn'thave any sort of printed design no printed roses, Harley bandannas or anything like that , but just using a fairly large piece of solidcolorful cloth works better.
The good thing about gaffs is that they give you a lot more room to work with; if your penis is on the smaller side, you may not even have to fully tuck in order to get a flat look.