Current craniomaxillofacial (CMF) surgical planning techniques that rely on cephalometric analysis work reasonably well in correcting mild jaw deformities, but are inadequate for complex CMF defects (e.g., trauma). Instead of relying on population-averaged measurements, patient-specific reference anatomy is needed for high-accuracy planning of complex reconstructive surgery. Ideally, the pre-traumatic CT scan of a patient can be used to construct a reference shape model for surgical planning. Unfortunately, such pre-traumatic CT scan usually does not exist. Therefore, the purpose of this study is to estimate a patient-specific reference bony shape model for planning the surgical correction of post-traumatic CMF defects. Our hypothesis is that the patient’s portrait photographs, taken prior to the trauma, can be utilized to restore the “normal” bony shape in areas with traumatic defects. Our approach involves three steps. First, we reconstruct the patient’s 3D facial surface from pre-traumatic 2D portrait photographs. Second, we generate an initial estimate of the patient’s normal bony shape via sparse representation based on a database of paired facial and bony shape models. Third, we refine the initial estimate of the bony shape by registering it to the post-traumatic bony shape. The deformable registration is regularized by a statistical shape model (SSM) constructed from the bony surfaces of normal subjects. Experimental results show that the patient’s abnormal facial bony structure can be recovered using our method with higher accuracy, and the estimated reference shape model is considered clinically acceptable by an experienced CMF surgeon. Our approach causes a paradigm shift in the planning of reconstructive surgeries. Instead of relying on linear and angular measurements and imagination, a patient-specific reference bony shape model can be estimated for accurate planning of reconstructive surgeries for patients with post-traumatic CMF defects.