As a vegetarian for almost 40 years (and a practicing herbalist since 1992) I have to point out that eliminating soy doesn’t eliminate phytoestrogens. Phytoestrogens, which include isoflavones, are found in many foods - legumes, plums, grape skins (and wine!), many vegetables and grains.
From the NIH (emphasis mine): “ Evidence from preclinical studies suggests these compounds may have an effect on hormones and health, although the results of human trials are unclear. The effects of dietary phytoestrogens depend on the exposure (phytoestrogen type, matrix, concentration, and bioavailability), ethnicity, hormone levels (related to age, sex, and physiological condition), and health status of the consumer.” (Effects of Dietary Phytoestrogens on Hormones throughout a Human Lifespan: A Review - PMC).
In other words, some people shouldn’t eat soy and over consuming it may be problematic.
Phytic acid, known as PA, exists in beans, legumes, nuts and oil seeds. It also has benefits: “Particularly, PA exhibits advantageous effects for human beings, including scavenging free radicals, prevention of type 2 diabetes, anti-inflammatory properties and anti-cancer attributes……”
(https://www.sciencedirect.com/science/article/abs/pii/S0924224423003163).
It would seem that eating a variety of foods, as tolerated by the individual, in moderate amounts will provide good nourishment for the body.
This is a long winded way of saying what Paracelsus said far better: “ All things are poisons, for there is nothing without poisonous qualities. It is only the dose which makes a thing poison.”.