Objective: This paper proposes a novel method to localize origins of premature ventricular contractions (PVCs) from 12-lead electrocardiography (ECG) using convolutional neural network (CNN) and a realistic computer heart model. Methods: The proposed method consists of two CNNs (Segment CNN and Epi-Endo CNN) to classify among ventricular sources from 25 segments and from epicardium (Epi) or endocardium (Endo). The inputs are the full time courses and the first half of QRS complexes of 12-lead ECG, respectively. After registering the ventricle computer model with an individual patient’s heart, the training datasets were generated by multiplying ventricular current dipoles derived from single pacing at various locations with patient-specific lead field. The origins of PVC are localized by calculating the weighted center of gravity of classification returned by the CNNs. A number of computer simulations were conducted to evaluate the proposed method under a variety of noise levels and heart registration errors. Furthermore, the proposed method was evaluated on 90 PVC beats from 9 human patients with PVCs and compared against ablation outcome in the same patients. Results: The computer simulation evaluation returned relatively high accuracies for Segment CNN (~78%) and Epi-Endo CNN (~90%). Clinical testing in 9 PVC patients resulted an averaged localization error of 11 mm. Conclusion: Our simulation and clinical evaluation results demonstrate the capability and merits of the proposed CNN-based method for localization of PVC. Significance: This work suggests a new approach for cardiac source localization of origin of arrhythmias using only the 12-lead ECG by means of CNN, and may have important applications for future real-time monitoring and localizing origins of cardiac arrhythmias guiding ablation treatment.