Root Nyquist (M) filters have applications in both single carrier and multicarrier communication systems. A method of designing such filters with a wide range of capabilities can be found in [1]. The following MATLAB functions have been developed based on [1].

rNyquistM0.m
rNyquistMG.m

To effectively use these programs, please refer to [1].  

[1] Behrouz Farhang-Beroujeny , "A Square-Root Nyquist (M) Filter Design for Digital Communication Systems", Submitted to IEEE Transactions on Signal Processing, May 2006. [PDF]