I tried to circumvent the ctrl modofier in such a way, that I look into the event's previous value and delete/save as necessary. Now, there is also always one option selected
I modified the callback as follows:
function selectionChanged(src,evt)
    prevVal=evt.PreviousValue;
    currVal=src.Value;
    if strcmp(fig.SelectionType,'normal') % to preserve ctrl behavior
        isThere=ismember(prevVal,currVal);
        if any(isThere)
            prevVal=prevVal(~isThere);
        else
            prevVal=[prevVal currVal];
        end
        src.Value=prevVal;
    end
end
please note that with the current state, the bahavior with ctrl left click changes and does not work as intended anymore.
EDIT: edited code to properly preserve ctrl click behavior
