I think Right-click will move the viewport (dragstart / dragend events in Javascript) and context menu only opens on immediate release (no dragstart fired). Don't know whether that's a feature of C++ or the libs you use, but a very low level approach could also be to decide "onmouseup" (again, Javascript terms as I'm not familiar with the C++ equivalent) how much the mouse has moved and if it's below some threshold (10 pixels?) it's considered a right-click to open the context menu, anything else moves the viewport.