The Old Norse form of the name was Vargøy. The first element is vargr which means "wolf" and the last element is øy which means "island". The first element was later replaced (around 1500) with varða which means "cairn".

Vardø is the easternmost town in Norway, east of Saint Petersburg, Kiev and Istanbul. The eastern part of Finnmark is in the same time zone as the rest of the country, even if it is more than an hour at odds with daylight hours.

The port of Vardø, on the Barents Sea, remains ice-free all year round thanks to the effect of the warm North Atlantic drift. Vardø is usually referred to as Norway's only mainland town in the Arctic climate zone, although this is not strictly correct since the town is located on an island about 2 kilometres (1 mi) off the northeastern coast of the Varanger Peninsula. In July, the 24-hr average temperature is only 9.1 °C (48 °F), while the January average is a modest −5.1 °C (23 °F)