As Aurifex pointed out, I was wrong when I said that. It can also be vicini domum, it depends on the context(s) in which you want to use it. If it is "we were/he is, etc next door", it's vicini domi, but if it is "we went next door" or so, it's vicini domum.
Actually, instead of "the neighbour's house", why not simply proximae domi/proximam domum? Literally "the nearest house".