Sometimes yes, sometimes no. It depends on what equipment local recyclers have available. The bottle caps and the bottle are made of different materials which presents a problem when it comes to recycling. The cap is a hard plastic, either high density polyethylene (HDPE), or polypropylene, while the bottle is made of polyethylene terephthalate (PET). These plastics have very different melting points, 260oC for PET and 140-150oC for HDPE and polypropylene. For recycling, ultimately the materials have to be melted and this means they have to be separated because the lower-melting components would decompose by the time the higher melting point is reached. Once separated, both can be easily recycled. The bottles can be recycled into fiber for clothing, comforter fill, carpets or tennis ball fuzz, while HDPE and polypropylene can be made into rakes, brooms, ice scrapers and ropes.
Current technology is available to process bottles with caps attached. These are ground into flakes and then introduced into a flotation tank. PET sinks, while polypropylene and HDPE float and can be skimmed off. They are then independently melted and recycled into a variety of products. Whether the required equipment is available or not depends on whether the recycler has a market for the recycled caps. If the separation machinery is not available, then putting the caps into the recycling bin creates a problem. The only way to know, is to seek information from your local recycler and find out if caps should be placed back on the bottles or discarded in the trash. In any case, there is no question about the bottles. These should always be placed in the recycling bin.