If the surface that you want to remove the vinyl from is not too deformed, you might be able to slice the outer vinyl layer off using a sharp tool such as a wood chisel or thin-bladed paint scraper with a sharpened edge. Sometimes this can be easily done without gouging the ABS substrate but if the surface is rippled or you need to remove the vinyl from a concave dent, you probably won’t be able to do it.
You can take the vinyl layer off by sanding but it can be pretty laborious. I have applied fiberglass over Royalex surfaces from which I did not entirely remove the vinyl using G Flex epoxy, and I have not had any delaminate. There was always some exposed ABS, however, and I always thoroughly sanded both the exposed ABS substrate and the remaining vinyl and then thoroughly cleaned it before applying epoxy.
After sanding, I usually wash the surface with detergent soap and water then rinse thoroughly with water. I then go over the surface with a solvent, either denatured alcohol or acetone or often both. Be cautious using acetone on the ABS as it will melt it on prolonged contact. I would not use it on any surface that has exposed foam core. Going over the ABS surface with an acetone rag is fine since the solvent flashes off so quickly but don’t let any liquid acetone pool on the surface.
If you read the directions that came with the G Flex, you will see that West Systems suggests flame oxidizing the Royalex surface as an “optional” step before applying G Flex. I have done repairs with and without flame oxidation and have not had any failures either way, but the latest adhesion data that West has published for G Flex and Royalex suggest bonds of modestly greater tensile strength after flame oxidation, so you might want to do this. You will need an inexpensive, hand-held propane torch to do it.
I find it is best to do the flame treatment in a semi-dark environment so that you can see the tip of the inner blue cone of the flame clearly. You want to touch the very tip of that blue cone to the ABS surface and move along fairly quickly. Avoid overheating the Royalex as you can melt the foam core more easily than you might expect. The G Flex instructions have some pretty explicit directions on how to do this. If you have washed the surface with alcohol or acetone make sure it has completely flashed off before doing this or you will have a “canoe flambe”. It is best to apply your epoxy within a half hour of flame treatment as the oxidizing effect is temporary.
I too would use plain weave fabric rather than mat. Mat is easy to use on curved surfaces because the omni-directional fibers lay down nicely without pleats. But the fibers are much shorter than for woven fabrics and are not as strong. I have not had any great difficulty getting plain weave fiberglass cloth of 6 ounce/square yard weight to lay down along a curved stem if you are patient and work the cloth carefully and gently until the epoxy starts to kick. A plain weave fabric will lay down over a curved stem if you cut the cloth on the bias so that the fibers of the waft and warp run at 45 degree angles to the keel line. My suggestion would be to use 6 ounce plain weave S 'glass.
I have used G Flex (unthickened) to wet out plain weave 6 ounce E 'glass, S 'glass, and 5 ounce aramid many times without difficulty but I only try to wet out one layer of cloth at a time and I avoid working in cool temperatures. I don’t have any personal experience using it on mat but since the fiber density of mat is usually much greater it could pose a problem. Some satin weave fabrics (especially chemically treated S 'glass) can also be problematical.
The viscosity of G Flex is greater than conventional epoxy at most working temperatures so it takes longer for fiberglass fibers to take it up. I find it is helpful to apply a thin coat of epoxy to the hull surface before laying the cloth over it. Be patient and let the 'glass take up the epoxy in its own good time. G Flex tends to have a pretty long pot life so for something the size of a skid plate you have plenty of time.
If you must work in cooler temperatures you can speed the wet out process by warming the hull a bit by placing an electric space heater underneath the inverted hull, or wafting a hair drier or heat gun over the surface first. You can also gently warm the mixed epoxy before applying it although you can overheat it, and warming it will reduce pot life some.
If you want a thicker plate than a single layer of 6oz 'glass provides you can apply a second, concentrically smaller piece over the first while the epoxy of the first layer is still green. Once the epoxy is cured you can feather the patch edges by sanding. I don’t mind sanding fiberglass all that much but some people hate it. If you want a smooth patch but don’t want to sand, you can use “peel ply” or mold release fabric over your wet epoxy.
The thickened G Flex should work fine for filling and fairing small chips, divots, and dents. I would do this to smooth any surface you intend to apply 'glass to so it lays flat. Just like using wood putty on a wood surface, this usually requires multiple applications to build up the area so that is can be sanded fair and flush with the adjacent hull. The G Flex can be mixed up by eye in very small quantities and you can apply another layer while the prior one is still green. Before applying any cloth or epoxy over thoroughly cured G Flex, make sure you wash it first to remove any amine blush.
I have not used anything other than spray paint on Royalex hull bottoms, and I have usually used Krylon Fusion for this purpose. I find it is pretty easy to mask off a 3 or 4 inch water line and spray the entire bottom. Yes it does scratch, but it is not hard to remask along the same line and respray the bottom when it gets ugly. I usually mix graphite powder into the epoxy I use for skid plates as the graphite mixed into the epoxy will result in the plate scratching black, and the contrast looks pretty good if you take care and mask along the boundaries of your plate.
I have heard of people using stuff like truck bed liner on their canoe bottoms and they seemed happy with it, but I have no personal experience.