I use filters, but only because all the pipes I've ever owned use them. By that I mean all two of the Dr. Grabow Savoys came with and were designed for filters. I tried going commando, without the filter, and did not like the taste as much, and since I couldn't just run a cleaner through the stem to the bowl, (due to the filter space,) I just caved and started using filters.
Now, I clean my pipes once a week, and they get a new filter twice daily. It really smooths out the smoke for me, and keeps me from having bits of ash come up on my teeth. Also, I see what they trap after a few bowls, and would rather have it on some paper tube than in my mouth.
-Just my $.02...