The problem of the classification for binary motor imagery EEG has been widely studied and great achievement has been made. However, it is difficult to get good recognition rate for multi-class motor imagery EEG due to its low signal-to-noise ratio(SNR). In order to improve the classification accuracy of multi-class motor imagery EEG, an EEG recognition method based on channel frequency selection is proposed. First, the original EEG signals are filtered by different frequency bands, and the corresponding band power can be calculated. Then the separability information of each frequency band is obtained by using the Fisher distance. Several bands with the maximum Fisher distance in each channel are selected for filtering. Finally, the feature vector of the filtered EEG signal is extracted by one-versus-one CSP(OVO-CSP) and classified by support vector machine(SVM). The public dataset of four-class motor imagery EEG is applied to evaluate this method. The results indicate that the classification accuracy and the Kappa coefficient achieved by the proposed method can reach 86.85% and 0.825 respectively, remarkably higher than the traditional method using a broad band. Therefore, the frequency bands associated with motor imagery can be effectively selected by this method, which can improve the recognition performance for multi-class motor imagery EEG significantly.