How dangerous is it to leave a facial fracture untreated?

I was out clbbing got in a ifght and got a facial fracture the doctors wanted me to follow up but i look fine but i can tell something is wrong under the skin but i only went to the doctor cause i had areally bad cut on the back of my head from his friends who jumped me anyways i said my face feels kind of squishy right there and sure enough I have a facial fracture but Im not even going back cause i feel fine no bumps or cuts on my face. my friend who got jumped really bad said all they say is dont smile or yawn or anything for a month and i doubt ill need a cast.