Show HN: An immutable local inference appliance with routing capability

Hey HNI’ve been working on a little side project after spending a bunch of time customising my inference stack at home.I figured it’d be nice to have something relatively portable, easily maintainable and configurable through a UI.It’s early days, there is built support for Vulkan, HIP and CUDA but I’ve only really tested it on AMD (that’s what I’ve got). So more hardware exposure would be awesome along with feedback and the like. At the moment there is no commercial model. Just wanting to get eyes on it.Thanks

Comments URL: https://news.ycombinator.com/item?id=50409129

Points: 1

# Comments: 0