Mirai Labs builds the pieces needed to run models on a device rather than in a data centre: an inference engine, model conversion, a macOS application and a command line tool, with cloud inference available alongside.
Shipping conversion as well as a runtime is what makes it practical. Getting a model to run on-device is rarely a runtime problem; it is a conversion and quantisation problem, and that is the step teams get stuck on.
Its pricing page returns a 404. Android support is listed as coming rather than available, and on-device inference is ultimately bounded by the hardware in someone’s hand, which is the trade accepted in exchange for data never leaving it.
