I think it's only natural that your body enjoys different sensations. Otherwise everything would just be made out of one material and there wouldn't be a need to be such a variety in sex toys. Same with food, some things taste spicy, some bitter, some sweet etc, and our mouth enjoys some of them more than others.
I tend to steer clear of any toys that are made of rubber. These toys are porous, and can harbour bacteria, as the material will have tiny microscopic holes. Cheaper made rubber toys can also be hazardous to health as they could leech chemicals into you. They also need to be stored completely seperately.
TPE i'm still on the shelf about. I have a few TENGA products that are made of TPE type material (the same goes with Fleshlights) - these are body safe but are still porous so need to be cleaned and maintained properly to prolong usage (in fact, they are best used with condoms) - but if you are the only one using them, then so long as you clean and dry them thoroughly, you'll be OK.
Glass/Steel/Aluminium/ABS Plastic/Silicone are all non-porous and totally body safe and can be sterilised safely and easily. So I generally tend to lean more towards product like that.
In terms of sensations, Glass and Steel feel best for penetrative play, as the added weight enhances the sensations - and you can use absolutely ANY lube you want.