Recent advances in machine learning and deep learning allow the researchers to develop the robust computer-aided diagnosis (CAD) tools for classification of brain lesions. Feature extraction is an essential step in any machine learning scheme. Time-frequency analysis methods provide localized information that makes them more attractive for image classification applications. Owing to the advantages of two-dimensional discrete orthonormal Stockwell transform (2D DOST), we propose to use it to extract the efficient features from brain MRIs and obtain the feature matrix. Since there are some irrelevant features, two-directional two-dimensional principal component analysis ((2D)2PCA) is used to reduce the dimension of the feature matrix. Finally, convolutional neural networks (CNNs) are designed and trained for MRI classification. Simulation results indicate that the proposed CAD tool outperforms the recently introduced ones and can efficiently diagnose the MRI scans.