My only guess is that you get the feeling of less stability because of a lack of lean with pressure in the sculling and bracing sweeps, not because of the lean itself.
As an example, if you were walking on a tight wire or the edge of a 2X4 you can feel as if you would fall because you try to stay upright (in perfect balance at the 12:00 position.) But if there was a rail out to the side where you must lean over to reach it, but when you do reach it you can lean all your body weight on that rail the action of leaning on it is what gives you the stability.
So unless I misunderstand, I think you are trying to stay upright and “in balance” as if you were walking, but a sea kayak is supposed to be edged and leaned---- with the paddle giving you the support in those dynamic leans and the control also being from the use of your thighs bracing inside the cockpit.
Many kayaks don’t feel perfectly upright when you sit in them on flat water and that’s what’s called primary stability, but the secondary stability is the important feature, and such hulls need to be edged and leaned to feel stable.
I hope I am not barking up the wrong tree here…but from what you wrote it sounds like that feeling is BECAUSE of too little lean, and not bracing with the paddle.
One kayak I had for a short time (too long and could not store it, so I sold it to a friend) was an 18 foot long P&H that was only 19" wide at the beam. It was really fast and if set on edge I could turn it nearly as well as my 17 foot Chatham. It forced me to quit trying to keep bolt-upright inside the cockpit. But in very little time I found I could lean or edge it to a point I had the spray skirt under water at the cockpit’s 3:00 and 9:00 positions by about 4 inches of flowing water, yet when I would use the paddle with a climbing angle I could easily stay in control. It was said to be a tippy boat, but I learned to use that tippiness instead of trying to fight it. Using the thigh braces and the paddle together made is easy to handle, even in pretty high waves.
So…I may be missing what you are saying and if I am I apologize, ---- but it sounds like you are trying to stay totally upright in a kayak that is designed to be edged and leaned. Like walking on the edge of a 2X4, if you lean your weight on the paddle with the top edge of that paddle always climbing and you use your thigh inside the cockpit, I believe in very little time you’ll get a feel for the balance of power being pressed down on the paddle in opposition to the hull angle of what would otherwise be a capsize.
In my first 3 months of kayaking I learned very well how to do a wet exit and reentry because I knew (correctly) that I was going to capsize many times if I were to learn the use or a kayak in the way it was supposed to be used. So I just made up my mind to do what I saw others do in the videos and if I capsize in the learning to do what I saw the experts do in the videos I’d just capsize. With a wet suit on and also my PFD I just got super comfortable going into the water and I’d capsize a lot pushing my limits of control. Kind of like riding a bike for the first time, but far less painful. When I do a wet exit (later a roll) it never hurt me like falling on the ground when you fall learning to ride a bike. Capsizing when learning a new kayak is nothing to fear.
Anyway… no need to go to the extreme limits of control if you don’t want to, but if you do press your limits and capsize it is not painful at all. If alone I used a paddle float many time and if I was with a partner I got to a point I could do a team rescue in very little time.
I like to learn all these drills in about 30 inches to 3 feet of water so when I tip over I simply lift the bow up to drain the cockpit, flip it upright again and jump back in.
Committing to the idea of learning how to lean and edge with sculling braces and sweeps was a fast and actually pretty easy set of skills to learn. Getting wet is a given.
Hypothetically lets say it takes 100 capsizes to learn the balance point in a boat with a certain paddle. If you do them all in a morning you learn the limits of the hull angle. If you do them all over 10 years you learn the limits of the hull angle. But it takes the number of attempts that it takes. I just got wet, and I learned in very short times So for me, I just assumed I had to learn by doing, and I did what it took to learn. I did a lot of short swims, but inside of just a few months I was able to edge all of my touring kayaks in tight turns when the hull was on the side to about 75 degrees and now I can turn my kayaks quickly and with not all that much effort by using my thighs inside the cockpit and my paddles to sweep in the water with a climbing blade angle, either moving forward or backward in those sweeps. Leaning what keeps me in control, not what causes me to loose control.
I hope that helps.
