I suggest that you test for unexpected results within your objective function code and handle it appropriately. For example, if you get an Inf or NaN as a result of evaluating your objective function, you can try returning a large, finite value instead.
The real problem is when you have this kind of result at the initial point x0. In that case your optimization should halt immediately and ask for a better initial value.
MATLAB mathematical toolbox documentation