I do downvote. As per the downvote button labels, whenever I feel:
- "This question does not show any research effort; it is unclear and not useful"
- "This answer is not useful"
As per these notes, I see it as specific to the question and not a judgement on the user who wrote it.
I always try to leave a comment, and I downvote anything I flag for deletion (unless I forget).
I don't really worry about the demotivating effect on new users, after all I think I'm one of the heaviest downvoters and even then I vote about 10 up to every 1 down.
And as Farray says in a comment on another answer "...any new viewers are likely to up-vote the improved post. Up-votes are +10 whereas down-votes are -2. If a down-vote causes an edit that later results in an up-vote, the net effect is heavily weighted towards positive net rep."