A single instability doesn't ruin an item, unless it's the Foul Lich-magic, you might as well keep throwing on the encrusts until you first hit an instability and then stop there. The item will be significantly more powerful and the "bad" effect with only occur 5% of the time, and'll be pretty minor even then, bar one unlikely case.
I don't have a 100% understanding of the instability, but I believe that slightly unstable only means that there's only 1 of them on the item, not the % chance of it occuring. My item got "EXTREMELY unstable" once I had 3 instabilities on it. I may be completely wrong, of course.
There was a bug where unstable weapons would proc 100% of the time. But I believe that was fixed in the last patch. Slightly unstable items proc 5% of the time and have only one instability effect on them. Increasing levels of instability increases both the number of procs and the chance of them going off.
So, do instabilities on armor check for procs on a countered hit? E.g. do I have to worry about instabilities goin off if I have maxed Counter Chance? Or does a counter not count as a "hit" in that case?
Well, a counterattack counts as an attack, and if that attack hits it can proc "on-hit" abilities, so. My guess is "probably", but I'd love to see some confirmation about that one way or another.
Yes, that's a good point, a counter attack shouldn't proc an armour instability, but could well proc any instabilities on your weapons.