Thanks for the suggestions, I managed to fix the problem through NVidia Control Panel!
For anyone else with the same problem, here's how I solved it:
In NVidia Control Panel, on the left side there is a section called "Adjust Desktop Size and Position". Inside there should be a setting called "Perform scaling on: ...", which was set to "Display" in my case. After setting it to GPU and running the game in fullscreen it now properly scales to full height.
Now I only wish I could preserve the sharpness of the image.
Perhaps there's a setting for that somewhere too...
EDIT: The term I was looking for is "Integer Scaling", and it seems neither NVidia nor AMD support it right now, so I'm out of luck here.