I hate it when I defeat an invader....blue guy, red guy, whatever and I immediately get hate mail calling me a noob or a ganker or whatever, nailing me for whatever garb I am wearing and whatever weapon I am using. If you are that weak and cannot take defeat, then dont invade dammit. Getting killed is part of the game. I have been killed thousands of times on there and through practice have just gotten better. Make a invasion build dammit people if your going to invade!
From what i know, it is pretty hard to noob in this game unless you somehow spam the other player with arrows/magic so you shouldn't worry to much, there probably just very confident and don't like being defeated.