You can run Git on object storage if you re-make packfiles

一句话概要

Git packfiles were designed for mmap and local disk, so pulling one object out of a bucket means guessing at a byte range. I wrote a new format instead.

原文开头节选

Skip to main content Contents It sure seems that a bunch of companies are trying to ship a git product of some kind as of late. Wonder why that is.

Either way, I’m building a Git server backed by object storage as an open-source project . It sounded simple enough to start: Git looks like a filesystem, so let’s use a filesystem as a translation layer on top of object storage to make Git speak object storage. This model worked… ok, I guess? But it didn’t work for real-world size repositories, so I needed a different approach. Git stores everything in Objects , so why not store those as objects in Tigris?

Turns out Git packfiles and how they intersected with my (admittedly somewhat terrible) filesystem shim were the main reason why it was slow. I ended up having to invent my own packfile format with a columnar store that’s object storage native. This is the fruit of all of my performance analysis, metrics annotations, and more Texas-style distributed systems work than you can make your k8s cluster shake sticks at.

(以上为原文节选,完整内容见下方”原文来源”)

这条动态今日登上 Hacker News 首页(133 赞 / 30 评论,来源 tigrisdata.com)。技术雷达每日自动聚合 AI 工程、后端架构、DevOps 方向的前沿动态;相关工程落地可浏览下方的相关服务与延伸阅读,或直接与我们团队交流。