HI Rory,
Maybe don't overthink it; you probably don't want to go down that rabbithole.
For instance I tested one Render, Saved it as:
JPG with 90% Quality (default in Blender): just under 700KiB.
JPG with 100% Quality: 3MiB.
PNG with 15% Compression (default in Blender): just over 9MiB.
That's just looking at the size, not the resulting quality. Or what happens when you increase the Compression of a PNG.
And the resulting quality and size will also depend on the image.
It really depends on what you're using the image for, what format to use.
I believe the lossless and lossy things are for when you're doing post production on the images. I think you may not notice the difference if you're looking at the image, where it does make a big difference is when you're managing the black and white levels, the saturation, the ping highlights, etc. Because you have much more data in the image to go to the extremes. But the overall picture when you look at them side by side, the difference will probably be unnoticeable.
When I asked this question I thought I was posting it under 'Fundamentals of Texturing - The Basics of Textures: What File for Specific Textures' so don't know what happened there.
The general message of the video felt like "PNG good, JPG bad!" so not going down too much of a rabbit hole, just questioning this. In my general experience you can get quite good looking JPGs and file sizes if you don't use too much of an aggressive compression. Have other people found this especially for textures?
Actually,
apparently, PNG's are hugely over-rated and some say, should be avoided at all times (see for instance: https://skientia.co/cgi/image-formats )
In short: EXR has a higher dynamic range and results in smaller files:

To answer your question, yes you can use jpeg. Texture artist avoid them, because every time you edit it you lose data. Also they don't have an alpha channel. As for materialist and shader artist they use them all the time. In the world of 3D, no one cares what format you use. They only care if the end result looks good... or bad if that's what is needed for the project 😀
Thanks for everyone's replies - there's great things to consider there. I'm surprised by the quality and size of EXR files, I've never used them and thought they were solely used for HDRI images. Sounds like it's just important to keep your eyes open. Thanks again.