Multi-tenant data processing for a super-app

A super-app is several businesses sharing one set of systems. Rides, food delivery, payments and logistics each have their own teams, their own data, and their own view of what "a user" means — running on infrastructure that was usually built for whichever business existed first.

We worked with a Southeast Asian mobility platform at exactly that stage. The engagement began as system and data discovery: understanding what was already running before proposing anything, so that later machine learning and analytics work would be built on an accurate picture rather than an assumed one.

Discovery is the phase clients are most often tempted to skip, and the one that most reliably prevents an expensive wrong turn.

Multi-tenant processing, with governance attached

The central workstream was multi-tenant data processing with data governance — and the two halves belong together.

Multi-tenancy in a data platform means many teams sharing infrastructure without interfering with each other: one team's heavy job must not starve another's, and one team's data must not be casually readable by everyone. Solve only the resource half and you have built an efficient way to leak data across business lines.

Governance is what makes multi-tenancy safe rather than merely economical: who owns which dataset, who may read it, how that is enforced by the platform rather than by convention, and how you answer an auditor asking who accessed what. For a business handling payments alongside rides, that is not optional.

Internationalising a scheduling system

A second workstream covered internationalisation of an internal scheduling system.

Internationalisation sounds like translation and rarely is. In a system that schedules real-world activity it means timezones — including the ones with half-hour offsets and the ones that change on dates nobody expects — date formats, currencies, and every assumption baked in when the system served a single market.

The interesting part of this work is that it is almost entirely archaeological. The assumptions are not documented; they are implicit in code written when a single country was the whole world. Our first action was to read the existing implementation and work through the design proposal before writing anything, because the risk is not writing new code — it is missing an assumption that only surfaces during a daylight-saving transition in a market nobody tested.

Spark and Beam as runtime engines

The platform ran its own in-house data processing framework. We added support for Spark and Beam as runtime engines beneath it.

This is a pattern worth naming: rather than migrating everyone off a bespoke platform onto an open-source engine — disruptive, and it discards genuine domain logic — you keep the platform's interface and swap what executes underneath.

Teams keep the tooling and abstractions they know. The platform gains a mature, well-supported execution layer and stops carrying the maintenance burden of a proprietary engine. Migration becomes a runtime configuration rather than a rewrite, and it can proceed job by job instead of as one high-risk cutover.

Infrastructure and data lake work

Two further workstreams covered data infrastructure migration and expansion and a data lake migration — the unglamorous foundation the rest depends on, and generally the work that determines whether the interesting projects are possible at all.

What we would take from this

Discovery earns its fee. A month spent understanding what actually runs — not what the architecture diagram claims — is cheaper than a quarter spent building against a wrong assumption.

Multi-tenancy and governance are one problem. Treating them separately produces a platform that is efficient and unsafe.

Swap the engine, keep the interface. When an in-house platform has outgrown its execution layer, replacing what runs underneath is almost always less disruptive than moving everyone onto something new — and it lets the migration happen incrementally.

Internationalisation is archaeology. Budget for reading and understanding rather than for writing. The hard part is finding the assumptions, not replacing them.