Multiple downloads

Dear,

first of all, my apologizes if my English isn't well-written. I am not a
native speaker but I tried my best.

as I was reading about the picture element combined with the srcset
attribute, I was excited.
Because this time we could have responsive images without the burden of
doing all calculations myself.

However,
the reason I want to use multiple files is performance.
I don't want my users to download a 5MB picture when the biggest they can
display would contain only 1MB

As far as I can understand the specification, this problem is still not
solved by using the picture element combined with the srcset and sizes
attributes.
Because what would happen if a viewport is resized (due to someone simply
resizing their browser window, or an orientation change on mobile).

Would this mean a new file has to be downloaded for displaying if the new
area where the image can be displayed reaches a certain (bigger) threshold?
Let me illustrate with an example.
A mobile phone with screen size 480x800px navigates to a website. The
medium sized image gets downloaded because the phone is held in portrait
mode.
Once the user rotates his phone and the orientation changes, the browser
recalculated and discovered a bigger image could be used.
Will this bigger image be downloaded as well? Or will the other image be
stretched out? Or even something else I couldn't think of?

Regards

Thomas