Here is what is happening. When you save a screenshot larger than the current render view resolution, ParaView needs to scale up text so that it looks proportional to the other elements in the image if you have the Font Scaling property in the Save Screenshot Options dialog set to “Scale fonts proportionally”. Now, this scaling is restricted to integral scale factors, so if you increase the resolution in each dimension by 2, 3, 4 times, or some other integral, the font scaling looks as expected. But if you increase the resolution by 1.1 times, for instance, the fonts will appear too large.
A workaround is to set the render view size in ParaView to some integral factor smaller than the desired output image resolution using the menu item View -> Preview -> Custom … Then save the image at the desired resolution. The fonts will be proportional to the rest of the image.
Why aren’t we able to scale fonts by an arbitrary scale factor? I believe there are historical reasons I am not familiar with, but this integral scale factor is scattered throughout VTK. It may be a fair amount of work to change it to an arbitrary scale factor, but it is worth looking into.