Faraday.dev is a free software that you can download and install on your PC that allows you to run open source LLMs. It comes with some ready-made AI models that you can download and plug into the app and use then. It provides a chat interface like ChatGPT where you can ask questions and it will return AI generated responses.
For now, it comes with a few models that you can download and load into the software and then proceed with the interactive chat. However, to run it on local PC, you need to have a PC with good hardware configuration. At lease 8 GB of RAM is required to run the lowest model. Other than that, it offers you a nice chat where you can enter text prompt and then AI will reply.
How to Run Open Source LLMs on your PC with this Free Software?
You can download the desktop app of Faraday from here. It’s quite heavy in size and so after downloading, you install it and run it. The main interface will now appear which looks as shown above, and it has list of supported models.
So, the first thing you have to do is select a model load it. If you 8 GB of memory, then start with smaller one. Just wait for a few seconds while it is downloading the model.
Once the model has been downloaded, select it from the interface and then it will take you to the chat interface. From here, you can now ask it any question. You can keep asking it questions and it will keep giving you the responses.
You can create multiple conversions for different topics and then ask the questions accordingly. The accuracy of the responses depends on the model that you have selected. The smaller models may generate the wrong answers on specific subject such as coding. This is the expected behavior and for better results, you need better hardware so that you can use more powerful models.
Closing thoughts:
If you are looking for a free software to run the already available open source LLMs then you are the right place. Just download and install Faraday.dev desktop app then try the different open-source models to chat with. You can use them to generate social media captions, answer generic questions, and do much more.
