This setting has gone. fxRender draws a typeset equation onto the worksheet, and that is now the only thing the render buttons do.
The dropdown offered four outputs — an equation image, Excel-like text, LaTeX and MathML — but three of them wrote source into a cell rather than drawing anything, which meant Ref, Name and Value appeared to do nothing to the sheet. A setting that silently changes what a button does is a setting worth removing.
They were never really about what goes on the sheet; they were ways of getting the equation out of Excel. That is what the controls beside the preview are for, and they do it without changing what the render buttons do:
See copying and saving an equation, which covers all four and what each is good for.
The equation is typeset on your own machine, as a vector, and then drawn onto the sheet as a PNG. Excel's add-in interface accepts only PNG and JPEG pictures, so the vector is rasterised on the way in — at three times its display size, which is what keeps it sharp when the sheet is zoomed or shown on a high-resolution screen.
The Windows add-in also offered EMF, and a choice of font. Neither is available to an add-in running inside Excel's web view: EMF is a Windows-only picture format, and the equation is set in whichever mathematics font the machine already has — Cambria Math where Excel can reach it, otherwise STIX Two Math.