Together AI on Nov. 19 introduced Together Storage Accelerator, a distributed cache for its GPU clusters that the company says is included at no additional cost.
The add-on is designed to replace repeated downloads from object storage and Hugging Face Hub with reads from cache on the cluster’s NVMe drives, according to Together AI’s announcement1.
Performance claims
Together said the cache is available now on Together GPU Clusters. Customers can enable it from cluster settings, but the company’s documentation2 says the feature may not appear for every organization; customers whose console does not show it are instructed to contact their account team.
In a vendor-run test on Oct. 28, a single-stream read of a 2-gigabyte object from Cloudflare R2 took 12.7 to 15 seconds without the cache and 0.64 to 0.72 seconds for a warm cached read, Together reported. The announcement includes no independent test results.
Operational details
TSA works with S3-compatible storage and Hugging Face Hub. The add-on handles reads; uploads, multipart uploads and listings still go to the origin. The cache can return old data if an object is overwritten at the same storage key. Together warns that reads spanning an overwrite can contain a mix of old and new data.
Together said customers can use the cache with existing storage rather than migrating data to its own T4 service, which remains in limited preview.