Databricks made Zerobus Ingest generally available earlier this year. It is a direct write path into Delta, (the table format behind Databricks): an application sends records straight to a table, with no queue or streaming system in between. 

Getting live data into a lakehouse normally means running a stack of middleware, typically a Kafka cluster, a schema registry, and a fleet of connectors. A data engineering team has to size it, pay for it, and keep it running. But now Zerobus removes that layer: an application connects, sends rows, and they appear in Delta within seconds.

MetaRouter now supports Zerobus Ingest as a destination. Web, mobile browser, and mobile app events collected by your own first-party pipeline are written straight into Delta tables you own, in your cloud, using your own service principal.

“This was a no-brainer for us. We are already bullish about MetaRouter and Databricks as best-in-class technologies for getting customer data into Databricks in way that supports real-time streaming applications, but the additional required infrastructure, whether Kafka or a cloud messaging technology, was an annoying burden. We were able to stand up Zerobus so quickly and we’re off to the races!"

Murray William
Lovelytics - Digital Marketing Transformation, Solutions Architect

‍

Zerobus was built for machines

Zerobus was designed for a kind of data that is much simpler than customer data. Machine data arrives already knowing what it is: the source is obvious, every record means the same thing, and no privacy law governs whether it can be stored. Customer data is none of those things, so a tool built for machine data does not cover it. 

Customer data has three problems that machine data doesn’t have. 

• Identity: the same person uses a phone and a laptop, logged in and logged out, and those sessions have to be stitched into one customer. 
• Consent: whether an event can be stored at all depends on what the person selected on a consent banner seconds earlier.
• Definition: a purchase has to mean the same thing across every team recording it, which requires the definition to be set once and applied everywhere. 

Zerobus only removed the middleware, but it didn’t remove the work that has to happen before the write.

‍

What MetaRouter does before the write

Identity: MetaRouter resolves identity at the point of collection, so the row arrives with the customer already attached. If it happens later in a batch job, the events table is not usable on its own and every query has to pull in a second table to work out who did what.

Consent: MetaRouter enforces consent server-side before the write, so a record blocked by consent never reaches the table. Deleting a bad row afterwards does not undo the damage, because other tables have usually read from it by then. 

Definition: Zerobus checks each record against the target table and rejects anything with a field that has no matching column. That check is worth having, but it only looks at the shape of the record. A record can have the right columns and still record the wrong thing as a purchase. MetaRouter applies event definitions once, in the pipeline, so every application writing into the same tables uses the same definition of a purchase or a session. 

Databricks recommends organizing data in three stages: raw lands in bronze, gets conformed into silver, gets shaped into gold. Much of the work in that first hop is repair. Deduplicating identity, reconciling event names, working out which rows were never allowed to be there. MetaRouter does that work before the write, so records arriving in bronze are already resolved, already governed, and already speak one vocabulary.

How it works

1. Collect. Your first-party tag collects web, mobile browser, and mobile app events from your own domain and sends them to your MetaRouter ingestion endpoint. No vendor JavaScript on the page.
2. Resolve. Identity is resolved at the point of collection, so anonymous and known activity is stitched before the row is written.
3. Enforce. Consent categories are enforced server-side, inside your infrastructure, so a record blocked by consent never reaches the table. Consent state travels with the records that do land, so Unity Catalog row filters and column masks can govern who reads what downstream.
4. Map. Core fields map to typed columns. Context, properties, and traits pass through as semi-structured columns, so adding a property upstream does not require a schema migration downstream.
5. Deliver. Events stream to your Zerobus endpoint using OAuth credentials tied to a service principal you control, into the Unity Catalog catalog, schema, and table you name. Ordering holds per stream, and delivery is visible in the MetaRouter monitoring dashboard.

‍

What changes for the business

The point of real-time data in the lakehouse is that people can act on it while it still matters. Marketing can build an audience out of what somebody did ten seconds ago and send it to an ad platform or a retail media network while that person is still shopping. Risk teams can score a session for fraud or bot traffic in under 50ms, before the transaction completes, instead of explaining it in a report the following week. And because consent arrives with the data rather than being tracked alongside it, compliance is a decision made once at the front door instead of a deletion project discovered months later.

The same logic runs in reverse when the data is wrong. Every team downstream inherits whatever definitions turned up in the table, and models are the least forgiving of all, because a model does not raise an error when its training data disagrees with itself about what a session is. It gets quietly worse, and the people relying on it have no reason to suspect the collection layer.

‍

Get started

The Databricks Zerobus integration is available now. Read the Databricks Zerobus documentation to set it up.

https://www.databricks.com/product/data-engineering/lakeflow-connect/zerobus-ingest

MetaRouter is the first-mile Customer Context Infrastructure that captures identity, behavior, and consent — in real time, inside the enterprise's own cloud, before the moment is lost. Trusted by global B2C enterprises including The Home Depot, Costco, Dicks Sporting Goods, and the world's largest Retail Media Networks. For more information visit: www.metarouter.io.