OK, so as someone who blogs about sex toys, let's just cover a few bits and pieces about materials and safety.
So the first important thing to remember is that the sex toy industry is not regulated. So this means that almost anyone could market a sex toy and make it with whatever material they want to - hence why on EVERY single sex toy you'll buy, you'll see a warning about 'Novelty Use Only'.
With that said, there are various materials that toys can be made out of and each of them have their pros and cons. Ultimately it is up to you to decide what you use.
Jelly Rubber
With things like Jelly Rubber, they are porous (which basically means there are tiny pores in the material that can absorb everything - bodily fluids/bacteria etc) and as such can not be properly sanitised. It is important that these sorts of toys are NOT shared (and do not switch between anal/vaginal use). These toys often have added chemicals in them to make them softer and more flexible (and is often why they have that 'smell') which in some people, can cause a reaction/irritation. You could potenitally use a condom with a toy, but that is by no means a fail safe.
You should only ever use a water-based lube with this material and should be stored completely seperately from any other toys (especially if they are made of the same material) as with prolonged contact, the material will break down and you will soon find a melted puddle where your toy used to be. The same applies to things like PVC, 'Skin-Safe' Rubber, TPE/TPR etc
When it comers to things like Masturbators - TPE/TPR and the other usual suspects like UR3/FantaFlesh etc - are generally considered slightly 'safer' as these are not being inserted into the body. As above, use with only water-based lube and store seperately.
Silicone
Silicone is generally considered one of the safer materials as it is generally non-porous (slightly innaccurate as Silicone will still have pores, however these are significantly smaller, and therefore cannot harbour bodily fluids and bacteria). Silicone can be sanitised by bleaching it in a 10% bleach solution, boiled, put in the dishwasher or just washed with water and toy cleaner (do not boil/dishwasher any toys that have electrical parts - even if waterproof) - just to be on the safe side.
You can switch between anal/vaginal usage (provided they are sanitised in between uses). You should use a water-based lube or a silicone-safe hybrid lube. Dependent on the quality of the silicone, I have used silicone lubes before without issue - always best to conduct a patch test before using to make sure. Silicone generally has a soft, silky feel and is personally my preferred choice for insertable toys. Silicone toys can be safely stored with other toys.
Glass
Properly annodised glass is also completely body-safe and non-porous. You can use it with almost any type of lubricant and again, can be sanitished by bleaching/boiling/dishwasher etc. They can also be stored safely with other toys and are surprisingly sturdy - they are usually made from boroscilicate glass (the same material your pyrex dishes are made from) so they hold on to temperature very well.
Metal (Stainless Steel/Aluminium)
Another non-porous material that can be sanitised the same way as glass and silicone. Metal toys can be very intense (especially the heavier stainless steel toys) and are again great for sensory play. You can also use them with almost any type of lube (which ever works for you) and again can be stored with other toys.
Wood
There are fewer toys made from wood, but they do exist. Make sure that the wood is properly finished and lacquered and again, they are non-porous.
At the end of the day, you make the decisions - due to most of my toys being used anally, then I completely avoid anything made from Jelly/PVC etc.