My experience as a kid growing up with a cabin on a lake would lend me to suggest an aluminum Grumman canoe. 16’ to 18’. As children we would beat the hell out of that Grumman. We would tip it on purpose so we could hang out underneath the upside down canoe. We dragged it to the water over rocks… Four or five of us would pack into in it and go paddling. It’s not a great canoe but it was perfect for what we did. Fishing, loading it up to camp on one of the islands. I was able to carry it to the water as an 11 year old, so no reason two 60+ year old adults can’t handle one. As far as durability, it was approaching 40 years old when it was stolen. We were totally bummed out when that happened. It’s the sort of canoe that you can hand down to your children and they can hand down to their children. No worries about leaving it outside in the winter or in the sunlght. Best of all you can find them used for under $400 most of the time.
If you are totally averse to the idea of an aluminum canoe, I would look for a used Royalex canoe. Something the kids can beat to death but still is reasonably light and paddles well. Forget high end kevlar and composites. for that purpose. I also own a kevlar/carbon Swift Keewaydin 16, but it would be the last canoe I would choose for your intended application. When kids are involved I don’t want to worry about the inevitible dings and scratchs that kids will cause. Kids don’t think 'Wow this canoe cost $2,000, I’d better be careful" They want to play, they will punish it. Perhaps if you can find a used Royalex Mad River Explorer, Old Town Discovery or Penobscot, Nova Craft PAL or Bob Special or a Wenonah Spirit II. Any of those will work perfectly. Great all around, “do all” canoes, that in Royalex can be beaten to death.