I recommend that you read the Matlab help on writing functions.
Here is an example of how you can create the function to do what you want. Write this as a script, and save it with the name myFunction.m (should match the name on the right hand side of line 1).
Here is an example of two calls to the funciton. The first call results in a warning, because one of the conditions is not met. The second call works fine.
>> m1=1; a1=2; b1=3; k1=4;
Error: When m<>0, 1+k*(m-a)/b must be positive.
>> m1=4; a1=3; b1=2; k1=1;
Here is the function you asked about. I assume from your definition that beta must be >0. Therefore I check for this, and if beta is not >0, an error message is issued, and the function returns the value -1. Obviously you could change this.
disp('Error: b must be >0.');
disp('Error: When m<>0, 1+k*(m-a)/b must be positive.');