Mesh LLM 是一个基于 iroh 网络构建的分布式人工智能计算系统，旨在通过聚合多台设备的 GPU 资源，将本地或分散的计算能力整合为一个兼容 OpenAI API 的统一接口，从而降低使用大语言模型的门槛和成本 [1]。该系统允许用户利用现有硬件运行大型语言模型，无需购买昂贵的高端 GPU[1]。
在请求处理方面，Mesh LLM 支持三种模式：直接在本地节点运行、将路由至已加载模型的节点执行，或将模型拆分到多台机器上进行流水线处理 [1]。其底层架构基于 iroh 协议，利用 QUIC 技术实现节点间的 NAT 穿透和直接连接，整个过程无需中央服务器介入 [1]。软件安装包体积约为 18 MB，并提供 localhost:9337/v1 接口以兼容标准 OpenAI 客户端 [1]。
Mesh LLM is a distributed artificial intelligence computing system built upon the iroh network [1]. The project aims to lower the barrier and cost of using large language models by aggregating GPU resources from multiple devices into a unified interface compatible with the OpenAI API [1]. This approach allows users to run large language models on existing hardware without purchasing expensive high-end GPUs [1].
The system supports three distinct methods for handling requests: execution locally, routing queries to nodes where models are already loaded, or splitting models across multiple machines for pipeline processing [1]. Underlying this architecture is the iroh protocol, which utilizes QUIC to enable direct connections and NAT traversal between nodes without requiring a central server [1]. The software installation package occupies approximately 18 MB of space and provides an interface at localhost:9337/v1 that adheres to standard OpenAI client specifications [1].