How exactly did you make these plots? PLease explain your sampling protocol and calculations in more detail. That will help identify the source of variability.
The horizontal spacing of the data points in the plots suggets logarithmically spaced frequencies, which should be accounted for by the description of the sampling protocol and calculations.
What is plotted on the x-axis, including units? Frequency in Hz? I recommend scaling your embedded figures (plots) so they don't extend far beyond the typical viewing window. The last plot shows x= 0 to 2. Is there a factor of 10^4 that was cut off?
If you take a constant number of samples at a constant sampling rate, for each frequency on the signal genrator, then you are likely to get more variation in the rms value at low frequencies, because you are likely to have a non-integer number of cycles, and the fractional cycle (which leads to variability in the rms estimate) will be a bigger fraction of the total complete cycles, at low frequencies. This would explain why 100-500 Hz is more variable than 500 Hz - 20 kHz, but would not explain why 10-50 Hz is less variable than 100-500 Hz.
The variation in rms value at different frequencies could be due to (a) variations in the sine wave amplitude as frequency changes, or (b) frequency-dependent random noise. Option (b) means that the sine wave amplitude is 0.200000 V at all frequencies, but there's also a small Gaussian noise component, whose power depends on the frequency setting of the signal generator.
Even at the worst frequencies, the variation in rms value is quite is small. Are the variations reproducible functions of frequency? In other words, if you rerun your experiment 10 times, are the "bad" (i.e. more variable rms value) frequency ranges the same, every time? I'm asking because maybe you are observing time-dependent random fluctuations. If that is true, then the "bad" frequency ranges may be different on different passes.