Do you think it's rude not to leave comments behind?

I need to get perspective on this one. Now that I'm checking my site meter every hour (ok, more like every ten minutes) I see people coming for a visit and traipsing around, but on some days, the ratio of comments is less than 1 in 50! Is it me or should people take the time to at least post a short note when they've seen or read something they enjoyed? If you're in the "yes, should comment" category, of course you can say so here, but feel free to go over here http://fromsmilerwithlove.blogspot.com/ to let me know as well. If you're in the "no, why should we leave comments?" category, I just want to know: but why?