It happens to me too. Especially when I cross my neighboor on the street or on our common balcony. He really annoys me, and his girlfriend too. Mostly because when I moved in, we had to lift upstairs my sofa and it was too long to just open the door and pass it trough it. The only thing he had to do was opening his door so we would have enough place to make the sofa enter in my flat. My roomate and I knocked on this door, called him. He must have been so scared that we ask him to help us lifting stuff I guess (we never EVER ask him anything). He didn't answer so we had to lift the sofa by the balcony (on the 2nd floor) with USB cables.

When we finished, I look in his window fastly and he opened the light, sitting on his couch and doing nothing. Argh.
Well, since then, I don't say really hostile stuff when I cross him but it's all in the tone that comes in a more agressive way than I intend to. Last week, I told him: "If you guys don't eat the tomatoes that grow on MY side of the balcony, I'm going to fucking EAT them, ok?". He looked at me strangely. Anyway, I know what you mean Vulgaris.
Sorry for the long post.
Pics of said tomatoes :