Cardiac diffusion tensor imaging (CDTI) is a non-contrast-enhanced, non-invasive technique that shows great promise in assessing myofiber orientation and organization, thus offering insight to the pathophysiological fiber dynamics of various cardiovascular diseases. However, it is yet to be clinically practical due to the prolonged acquisition time. The objective of this work is to propose a novel framework to accelerate CDTI by jointly exploiting the spatial transform sparsity and the low-rankness of diffusion-weighted images, along with a phase correction procedure to compensate for the phase inconsistency across diffusion directions (thereby enhancing the low-rank property and facilitating undersampling). The proposed method was evaluated both ex vivo on six heart cadavers extracted from heart failure patients and in vivo on seven free-breathing hypertrophic cardiomyopathy patients. Compared to using either single constraint alone, the proposed method preserves helix angle features as well as the transmural continuum across the myocardium more accurately, allowing acceleration factors of 6x ex vivo and 3x in vivo. In addition, the proposed method provides accurate mean diffusivity measurements across the myocardium with acceleration factors of 16x ex vivo and 4x in vivo. For both CDTI metrics, the proposed method yields significantly lower bias and higher intraclass correlation coefficients against fully sampled gold standards. We conclude that the combination of the phase-corrected low-rank image model and the sparsity constraint facilitates acceleration for ex vivo and in vivo CDTI, providing improved reconstruction accuracy compared to using either single constraint alone. The proposed method has the potential to achieve higher reduction factors in practice while preserving features of myocardial fiber structures and measurements of tissue diffusion properties, which may allow greater spatial coverage and/or higher spatial resolution in the future.