1. Finland was allied with Germany, that's correct, but Finland never concidered themselves a member of the axis.
2. The alliance with Germany was mainly to prevent another invasion from the USSR, which all was fucked up due to Operation Barbarossa, and Stalin's second attempt of invading Finland (of course USSR had no reason to believ they were not in was versus Finland since there were german troops in Lappland, Finland however did not concidered themselves in war with USSR before the Russians attacked).
3. Finland's enemy and fear lied East, and the protection of the Eastern Border was the most important. Finland could not join the Allies, since the Allies was in no position to help defend Finland from the Russians.