Cohere’s command-r models are trained for exactly this type of task. The real struggle is finding a way to feed relevant sources into the model. There are plenty of projects that have attempted it but few can do more than pulling the first few search results.
Koboldcpp should allow you to run much larger models with a little bit of ram offloading. There’s a fork that supports rocm for AMD cards: https://github.com/YellowRoseCx/koboldcpp-rocm
Make sure to use quantized models for the best performace, q4k_M being the standard.
That isn’t neccesarily true, though for now there’s no way to tell since they’ve yet to release their code. If the timeline is anything like their last paper it will be out around a month after publication, which will be Nov 20th.
There have been similar papers for confusing image classification models, not sure how successful they’ve been IRL.
I’ve used the tplink ones that they’re using and they’ve been pretty solid. I can’t say how they’d fare in a 24/7 setup though since they’re not really intended for that.
I sort of understand rounding outside edges for aesthetics since there’s nothing lost and it might be easier as a target for resizing, but inside corners are just stupid. You’re arbitrarily cutting corners out of content for no good reason.
LLMs only predict the next token. Sometimes those predictions are correct, sometimes they’re incorrect. Larger models trained on a greater number of examples make better predictions, but they are always just predictions. This is why incorrect responses often sound plausable even if logically they don’t make sense.
Fixing hallucinations is more about decreasing inaccuracies rather than fixing an actual problem with the model itself.
I’d guess the 3 key staff members leaving all at once without notice had something to do with it.