Yeah, initially it was CPU only, but now it’s GPU accelerated. So, as you can see here, I’m not connected to the internet. And it’s actually very performant now. As you can see, this is like five tokens per second.
Yeah, initially it was CPU only, but now it’s GPU accelerated. So, as you can see here, I’m not connected to the internet. And it’s actually very performant now. As you can see, this is like five tokens per second.