Every developer who’s touched ARKit knows the feeling. It just works. You point your phone at a room, and within seconds it’s mapping walls, estimating depth, tracking your position with eerie precision. It’s the kind of polished, frictionless experience that makes you quietly resent Apple’s closed ecosystem — because you know you could never build something this good yourself.
Or could you?
A developer named Pablo Vela Gomez just posted something on Hacker News that stopped me in my tracks. He’s rebuilding ARKit’s visual-inertial odometry (VIO) and depth estimation from scratch — in open source. No Apple frameworks. No proprietary APIs. Just raw engineering and a very specific strategy.
The biggest lie in tech is that proprietary advantage lives in the algorithm. It doesn’t. It lives in the data that trained it.
Here’s what makes Pablo’s approach so interesting: he’s not starting with code. He’s starting with data. Specifically, ARKitScenes — Apple’s own publicly released dataset of thousands of indoor scenes captured with LiDAR-equipped devices. He’s using Apple’s data to build a system that could eventually make Apple’s closed ecosystem irrelevant.
That’s not just clever. That’s a declaration of war.
Think about what ARKit actually does. Visual-inertial odometry fuses camera frames with IMU data to track where a device is in 3D space, in real time, with sub-centimeter accuracy. Depth estimation infers how far away surfaces are. Together, they’re the backbone of any serious AR experience. Apple has spent years tuning this for their specific hardware stack — the A-series chips, the LiDAR scanner, the neural engine, the whole tightly controlled silicon-to-software pipeline.
Most developers assume that hardware-software integration is the unbreakable moat. If you’re not running on Apple’s silicon, you can’t match Apple’s performance. Game over.
Pablo sees it differently.
Hardware optimization is the final mile. But you can’t optimize what you haven’t trained — and you can’t train without the right data.
His roadmap is almost deceptively simple: curate the right dataset first, build the VIO and depth models on top of it, then worry about on-device speed later. The stretch goal — matching ARKit’s real-time performance on open-source constraints — is explicitly acknowledged as hard. Maybe impossible. But the accuracy? That’s achievable, if you have the data.
And that’s the insight that should make every AR developer sit up straight.
We’ve been conditioned to believe that the magic of systems like ARKit lives in secret algorithms locked behind Cupertino’s walls. But algorithms are increasingly commoditized. The papers are public. The architectures are known. What separates a great VIO system from a mediocre one isn’t the math — it’s the thousands of hours of real-world scene data that taught the system how to handle edge cases: glass surfaces, low light, repetitive textures, fast motion.
The moat was never the code. The moat was always the dataset. And Apple gave theirs away.
ARKitScenes was released as a research dataset — 5,000 scenes, depth maps, mesh annotations, the works. Apple probably released it to accelerate academic research and burnish their developer relations image. They likely never imagined someone would use it to build a direct competitor to their crown jewel AR framework.
That’s the beautiful, brutal irony of open data. Once you release it, you can’t control what people build with it.
Now, let’s be clear about the challenges. Pablo himself admits that matching ARKit’s on-device speed is the stretch goal — not the baseline. Apple’s advantage in hardware-specific optimization is real. Their VIO pipeline is tuned at the silicon level, with custom accelerators and tightly integrated sensor fusion. An open-source implementation running on heterogeneous Android hardware or even iOS outside of Apple’s framework won’t have those advantages.
But here’s what matters: accuracy is the foundation. If you can match ARKit’s accuracy — even if you’re slower — you’ve built something that works. Speed is an engineering problem. Accuracy is a data problem. And data problems are solvable when the data exists.
Speed is a constraint. Accuracy is a moat. Solve the moat first, and the constraints become someone else’s problem.
For the AR developer community, this is more than a cool project. It’s a roadmap. It says: you don’t need Apple’s permission to build world-class AR. You need the right dataset, a clear architecture, and the willingness to grind through the unglamorous work of data curation that most people skip because it’s not sexy.
Everyone wants to write the clever algorithm. Almost nobody wants to spend three months cleaning scene data. That’s why most open-source AR projects stall out at demo quality and never reach production.
Pablo gets it. The data-first approach isn’t just a technical choice — it’s a strategic one. It’s the difference between building a toy and building a weapon.
If this project works, the implications go beyond AR. It’s proof that the proprietary advantage of big tech platforms — the thing we all fear is unassailable — is more fragile than they want us to believe. The algorithms are known. The data is increasingly available. The hardware is commoditizing. What’s left is execution.
The giants built their moats with data they collected from the world. The world can take it back.
Pablo’s project is still early. The stretch goal might not be reached. The on-device speed gap might prove too wide. But the strategy — data first, accuracy before speed, open-source from day one — is the right one. And if it works, it won’t just be a win for AR developers. It’ll be a blueprint for dismantling proprietary advantage in every domain where the data already exists and the algorithms are already known.
Which is, frankly, most of them.
FAQ
Q: Can an open-source VIO system actually match ARKit's accuracy?
A: On accuracy alone, yes — ARKitScenes provides enough real-world scene data to train models that handle the same edge cases Apple's system handles. The real question is whether that accuracy holds up in real-time on-device without Apple's silicon-level optimizations.
Q: What does this mean for AR developers outside Apple's ecosystem?
A: It means the path to production-grade AR doesn't require Apple's permission or hardware. If the data-first approach works, developers on Android and other platforms get a viable alternative to ARKit — something that's been missing for years.
Q: Isn't Apple's hardware-software integration still an unbreakable advantage?
A: For speed, probably. For accuracy, no. The contrarian bet is that accuracy is the real product — speed is just optimization. If you nail accuracy with open data, the speed problem becomes an engineering challenge, not a research one. And engineering challenges get solved.