reverse your mask (so the big black circle becomes white) then calculate the convex hull of that using (bwconvhull). This gives you the entire circle.
Then mask your original mask with the convex hull of your circle, this would set the outer perimeter to black.
Then use your method to get the boundary again.
I downloaded the image you posted and something like this will happen
I=imread('~/Desktop/lateralview.jpg');
I=double(rgb2gray(I))/255;
Mask=logical(round(I));
bigCircleMask=bwconvhull(~Mask);
newMask=bwmorph(and(Mask, bigCircleMask),'majority');
This is the mask of big circle.
This is the new mask of the nail.
Now use your own algorithm to detect the edges of the nail.
If you want you can go one step further and fit a circle using imfindcircle of matlab to get a better circular mask of the bigCircle.