If I think about Llma, I think about uncensored. not that I ever used one, but there were not many use cases for censored llama when others were so much better at other things
Hmm? For me it's still the best model for general use. That I've tried at least. Gemma and phi I didn't like so much. Qwen is chinese and I've had bad experiences with those (reverting to Chinese when they get confused).
Not for knowledge, but I combine it with searches of course. None of the small models are good at knowledge.
So you are using local models for general use? For me when it’s not ultra sensitive information, I don‘t want a just good enough LLM so I use the API of a proper large one.
By the way, I usually use the AI for filtering, summarisation etc. Not for facts (like "what would be the best model of CPU I could buy" or "what is the capital of Estonia". When I do that I will link it to web searches anyway. Small models are indeed pretty useless for asking facts from. But big models can't be relied on either. They still hallucinate a lot. So I tend to combine it with web searches so I also have the references. But in general the whole fact thing for me is not a big usecase for AI.
I understand your take, however my main use case is for the LLM to find subtle connections and biases in text or analyse nuances, so I figure more parms do make adifference in my case.
Also in my experience it does also make a difference in summarization and filtering tasks, as a larger param one possibly knows better what is important and relevant in a text
I can recommend to try a comparison if you can (like with OpenWebUI, you can simply make it run the same prompts through multiple models). If you have access to local models of course. Though you could also use some models like llama on groq.
My usecases are different from yours of course but I don't really see significant difference in result quality in most cases.
For me good enough is usually good enough (and pretty excellent). Unless I ask for something really complex, then I use perplexity in research mode.
But I have everything in OpenWebUI so I can choose with a touch of a button. Sometimes I wonder whether GPT could have done better and I try it and it's usually not significantly better than llama 3.1 8b
