Here is what I did to solve it. I opened up the eleksmaker plugin in notepad++, and changed the orientation_scale for millimeters to 1 (from 3.5433...). Didn't fix the inches option, but I stick with mm anyway.

I now get the correct scale automatically. No idea why its like this on my setup, but it works now. Newest inkscape, latest linked eleksmaker plugin.