1.

To do this you need to Use Models that are ^MoE, 'Mixture of Experts '

for example download,

^MoE models naming convention is added text after param size in name, in this case A3B comes after 35billion parameters, stands for active 3billion paramaters,

which i believe are params offloaded to your system ram, giving your gpu ram more to run the model, keeping the more important stuff on graphics card

You know if you have ^MoE on model when loading it, it will show,

'Number of layers for which to force MoE weights onto CPU'

The trick, max out

Context Length & GPU Offload, and get 'Number of layers as low as you can to function well

I don't quite get this, but i'll mess with more on old macbook