Newer cameras include a position sensor and add a flag to the photo (in the EXIF data) to indicate whether the photo needs to be rotated. Some, but not all, computer programs read that flag and automatically rotate the photo. It's massively annoying. Check the manual on your camera, you may be able to disable that flag. Consider getting an EXIF editor to delete it from all your existing photos too. Anything that needs rotating, rotate it for real. Otherwise your photo will look different depending which program you're using to view it.
(Thumbnails are actually stored separately in the image file, not calculated fresh each time. Some programs will read the rotation flag when displaying full-size images but not for thumbnails. )