How to run Free Chat AI on your PC

Lord Nil · Post by **Lord Nil** » Sat Aug 26, 2023 2:37 pm

Hello everyone

Recently, it has become possible to install and run free, uncensored, and good quality Chat AI's on your own PC.
There are a few options, but one of them is to try the 1-click installer here:
https://github.com/oobabooga/text-generation-webui

There are many Youtube videos showing how to install and use the system. ( basic computer skills needed )
After you install and run the program, you can select a good chat model from HuggingFace.
LLama2 models are out that have been trained to be uncensored.
you can look at the models provided by this guy:
https://huggingface.co/TheBloke
(7B models with GGML compression can fit on most 8-12gb GPUS, look for models trained to chat, and uncensored )

I am currently playing with a 3GB model that only takes up 5GB of GPU memory, making it able to run on most cards.
https://huggingface.co/TheBloke/orca_mini_3B-GGML
Setting up the prompt may be tricky, as you have to tell the model what girl character you want.
One system has all this set up for you with premade characters ( + pics )
https://github.com/TavernAI/TavernAI

The future of AI should be free, opensource, and running on your local machine
There is also something similar for image generation if you want to generate your own hot girls.
I am sure this is posted already, but I thought it may help some people
Currently the 7B models are pretty good for normal chat, but not ChatGPT4 level.

mr_vic · Post by **mr_vic** » Sat Sep 02, 2023 10:29 pm

Thank you, this is very informative.
I am currently using BotLibre with my doll.
But your suggestion is next level.

Samgg69 · Post by **Samgg69** » Sun Sep 03, 2023 2:38 am

yes sir!!

I have been using oobabooga and Tavern for a while too, and when you get it to work it's crazy. I have a bit more powerful GPU so I have been using WizardLM 13B, Pygmalion 13B and Llama 2 13B AI Models. The "B" is for Billion parameters which usually means how powerful/complex the AI, but usually need more GPU power. The parameters can go high, I think GPT 4 has like 1.7 Trillion, but you got to pay for GPT 4 =(