For products with variants, you can add a variant picker block.
This block is not rendered when the product has no variants so it won't slow down your pages.
A key feature of the variant picker is the option to add a size chart. Use a regular store page to link into this block.