If you want to do cold water paddling, the first things to get are a number of very quick and reliable re-entry methods. Have them so well-practiced you can do them blindfolded and half asleep, because that’s how it might be when you need it.
Survival suits are about exactly that - survival. Note they are not called “Comfort Suits”. Invest in a proper dry suit meant for kayaking. Preferably one with breathable fabric but many will agree that “breathable” is highly exaggerated. If you don’t overheat and sweat easily then this criterion will be less important to you. Layering under the suit is critical. The suit itself offers almost NO INSULATION VALUE when you’re in the water, it just keeps the insulation dry.
You need to be dressed to be in the water for as long as you foresee being there in a worst case scenario. This could mean anything from a confident reentry to bobbing in the water waiting for someone in a passing boat to see you because you forgot your cell phone and don’t carry any other signaling devices (not a great idea) Swim in your gear in a safe environment with someone else present before you trust it to save your life while paddling. Build up to longer and longer times, in colder and colder water. Don’t just jump in beside an iceberg for your first try. Your hands are likely to be the first things to go numb and become useless; hence the emphasis on FAST and reliable reentry skills. Make sure you are well practiced both in the conditions you will paddle in, and in the gear, because the insulating layers and dry suit absolutely will limit your mobility and coordination if you’re dressed properly for very cold water.
“Cold” is a relative thing too. I paddle year round so I’m speaking mostly of water that’s around 2°C. It has a funny way of turning solid much lower than that. Generally I limit my paddling to days where the air temperature is above -5°C and the wind not too severe. Much colder and things just ice up and it’s generally unpleasant. If your intentions are more toward “shoulder season” paddling, everything still applies but not quite to the same extent. Late fall paddling is moderately safer than spring paddling because the water takes longer to cool down. Likewise in the spring the water stays cold long after the days have become warm.