Japanese is a difficult language to learn and I have failed miserably after three years of casual study. However, it is easy to survive and retreat from everyday situations with only a few words and phrases. Let's begin:
すみません = Su-mi-ma-sen = Excuse me
This word can be used when you're interupting someone, getting someone's attention, thanking them for their help or just apoligising for your existence. At restaurants you should use this with a clear and high pitch to get attention. An amusing cartoon about being lost in translation is a Japanese lad at a restaurant abroad shouting "EXCUSE MEEEE" to the waiter.
ご免なさい = Go-men Na-sai = I beg your pardon
This differs from the above in that it's more of a physical apology for walking in someone's way and the like. It is best observed when a waitress drops a used chopstick when collecting your plates. The frantic head bowing and gomengomengomennnaasaai is something else.
ありがとうございます = Ari-ga-toe Go-zai-mass = Thank you
Most people already know Arigato or even the full Domo Arigato Gozaimasu. However, I very rarely hear Japanese people use the later version. I tend to say Arigato when I would just say "Cheers" in British English but I've been told I should used Domo for that. If you mix and match any with Sumimasen for all conversation then you should be fine.
はい = Hai = Yes
Myself and most Japanese people probably utter this word the most. It is important to let the speaker know every second that you are listening, you understand them and everything is alright. No need to worry hai. Hai... hai.... hai... hai....uuuuhhhh..... hai.
分かりました = Wa-ka-ri-mash-ta = I understand
This one is essential when one is ending a conversation. If you do not say it then the speaker might worry that you have not understood everything and they will have to repeat themselves a few times. If you are foreign then they will keep talking to you until you say it. If you are Japanese then you will need to schedule another meeting to discuss the problem.
分かりません = Wa-ka-ri-ma-sen = I don't understand
I have only heard a Japanese person use this when I try to teach them English. I only use this myself when I do not understand their Japanese. Outside of language learning I would assume that it was a big no-no to use this in normal day life. You would shame the person, be labelled a fool and need to schedule a meeting. For the Japanese student this should be used with Sumimasen. It'll get you through any tough questioning.
There you go. The first things I thought of are all essential to everyday conversation and the importance of understanding one another. However, more often than not these will be used by the foreigner to apoligise and show that they don't have a clue what is going on. Good luck.