Every time you browse the panorama it always starts with the same view direction and zoom settings. Those settings are set automatically by the system at the moment your image is uploaded. You can always update these settings using the 'Snapshot' feature.
Snapshot also sets the preview image which is shown on the panorama shortcut.
To update a 'Snapshot':
1) Open your tour;
2) Switch to the edit mode;
3) Switch to the panorama you'd like to set the Snapshot for;
4) Set the view direction and zooming you'd like to use as default for this panorama;
5) Click ‘Capture Snapshot’ option in the thumbnail menu;
Now the snapshot is updated and your panorama will always start from it.