To use any opensource LLM locally there's an open-source tool in the market which is both free and effective.

Go to Ollama and download the setup file. https://ollama.com/

After downloading the Ollama install it and go the Models page.

Click on the model you want Iam using gemma.

ollama run gemma

Tun this command and the model will download it's about 5GB.

After that you are good to go