How to choose among data management companies

Choose data management companies by problem, not by ranking

Match the supplier type to your problem. Buy a platform when your sources are few and clean. Hire an engineering partner when they are messy. Then shortlist on three things: which layers a supplier covers, how they handle a sample of your own data, and who will do the work.

Data management is the work of collecting, storing, cleaning, securing and documenting data so people and systems can rely on it. Data management companies sell tools, services or both for that work.

Suppliers fall into four types

The suppliers sort into four groups, and each sells something different:

• Platform vendors such as Snowflake, Databricks, Informatica and Collibra sell software.
• Cloud providers such as AWS and Microsoft sell the same capabilities as services inside their clouds.
• Large consultancies such as IBM and Slalom sell strategy and programme delivery.
• Engineering partners build the pipelines and models themselves and hand over something you can run.

Most "top 10" lists mix all four groups into one ranking, which is why they are hard to use. A software licence and a delivery team are not interchangeable, and comparing them on one scale produces noise.

Data moves through five layers

Every data management setup has the same five layers. A supplier usually owns two or three of them well, and the gaps between them are where projects stall.

• Sources: your apps, databases and spreadsheets.
• Ingestion: copying data out of those sources on a schedule or as events happen.
• Storage: a warehouse or lake, such as Snowflake or Amazon S3.
• Transformation: reshaping raw records into something consistent, often with a tool like dbt.
• Access: permissions, a record of what each dataset means, and the dashboards or APIs people use.

AWS Lake Formation and Microsoft Purview are two vendor examples of the access and governance layer.

Ask any supplier which layers they cover and who covers the rest. The answer tells you more than their feature list.

A platform fits teams with clean, few sources

Buy a platform when you have a small number of well-behaved sources, an in-house team that can operate the tooling, and a governance problem more than a plumbing problem.

A typical fit is a company with a main product database, a billing system and a CRM. Each has a stable schema and an owner who answers messages.

In that situation a warehouse plus a governance tool covers most needs, and a licence is often cheaper than building the same capability from scratch. The vendor's roadmap also becomes your roadmap, which is a benefit if you do not want to maintain infrastructure.

The failure mode is assuming the platform cleans your data. It stores and labels what you give it. If the inputs are inconsistent, you now have inconsistent data in a more expensive place.

An engineering partner fits messy, many-source estates

Hire an engineering partner when the hard part is getting data out of old, inconsistent or undocumented systems. That work is custom by nature, and no licence removes it.

In our experience on client projects, the pattern repeats: the source systems were built for operations, not reporting. Order records, inventory counts and customer IDs rarely agree between systems, and nobody has written down which one is right.

Someone has to decide which system wins when they disagree, and write that rule down in code. That is data engineering, and it is slow, specific work that depends on people who have seen these estates before.

If that describes you, look at data engineering services built around your actual sources. A partner that starts by demoing a dashboard has skipped the part that decides whether it works.

Two costs suppliers rarely quote

Platform spend has two parts that sales calls underplay: usage that grows with your data, and the people time to keep pipelines working. Neither appears on the first quote.

Many warehouses bill on consumption. Snowflake, for example, meters compute in credits separate from storage, so a badly written query or an always-on dashboard raises the bill without anyone deciding to spend more.

The second cost is upkeep. Sources change their formats, APIs get deprecated, and a pipeline that worked in March breaks in August.

Our own work is scoped per project after a discovery call, so we do not publish rates. Whoever you pick, ask for the commercial shape. Options include a fixed-scope build, such as our Moonship 30-day MVP, ongoing support, or fractional CTO leadership that oversees several suppliers. Then ask what happens to cost if your data doubles.

Shortlist suppliers by what they own

Shortlist on three things: which layers they cover, whether they have worked with sources like yours, and who will actually do the work. Brand size and analyst rankings matter less than all three.

Here is how the four supplier types compare on the points that tend to decide an outcome:

• Platform vendor: strongest at storage, governance and scale. Weak spot: it does not fix upstream mess. Fits when you have few clean sources and a capable team.
• Cloud provider: strongest at integration inside one cloud. Weak spot: lock-in and many separate services. Fits when you are already committed to that cloud.
• Large consultancy: strongest at programme management and regulated sectors. Weak spot: juniors often do the build and timelines run long. Fits multi-year enterprise rollouts.
• Engineering partner: strongest at custom pipelines and messy sources. Weak spot: it needs clear ownership on your side. Fits when sources are complex and you want a working result.

No type wins everywhere. Many buyers combine two, for example a platform for storage and a partner to build what feeds it.

Make suppliers touch your data before you sign

The most reliable test is a paid or free sample: give a supplier an export of one real source and ask them to produce a clean, documented dataset from it. A slide deck proves nothing about your data, while a sample shows how they handle duplicates, missing values and the field that means two things.

Check the output with someone who knows the business. A finance lead can spot in minutes that revenue is double-counted, which no demo will reveal.

Also read what the supplier writes down. Good teams document the rules they applied, such as which customer record survives a merge. If they cannot explain it, you will not be able to maintain it.

Ask the questions vendors dislike

Four questions separate suppliers who have delivered from those who have only sold. Ask all of them, in writing, before a contract.

• Who exactly will work on this, and are they employees or subcontractors?
• What happens to cost if our data volume doubles?
• How do we export everything, including transformation logic, if we leave?
• What did you do the last time a source system changed its format without warning?

The last one is the most revealing. A practitioner answers with a specific incident and a fix. A salesperson answers with a process diagram.

Also ask for references with similar sources, not similar logos. A reference from a company with one tidy database says little about your twelve legacy systems.

Privacy and compliance also bite late, so raise them now. The NIST Privacy Framework is a useful checklist for deciding what you should store at all.

Governance first or pipelines first is contested

Practitioners disagree on sequencing. One camp says define ownership, definitions and access rules first, because tooling without them just automates confusion. Tools like Collibra exist for this.

The other camp says build one working pipeline for one real question first, then govern what exists. Their argument is that governance written in the abstract goes stale before any data flows.

Each approach has a typical failure, though we are describing what we have seen rather than citing a study. Governance-first projects can stall in committees for months. Pipelines-first projects can produce a useful report on top of definitions nobody agreed, which then get disputed.

Our view is a narrow middle: agree the five or six business terms that the first report depends on, build that report, then widen. The sequence depends on whether your pain is trust in numbers or access to them.

Most failures are about ownership, not technology

Data projects fail mostly because nobody on the client side owns the answer when two systems disagree. The technology usually works fine.

Common versions: the person who understands the source system left, the finance team and the sales team define "customer" differently, or a pipeline runs but nobody checks its output. Silent failures are worse than loud ones, because decisions get made on wrong numbers.

Automated checks help. Tools like dbt tests can assert that IDs are unique and amounts are not null on every run, so a broken feed shows up as a failed check instead of a wrong dashboard.

Plan for machine learning use from day one

If you expect to build ML or AI features on this data, say so at selection time. Analytics-ready data and model-ready data are not the same thing.

Reports tolerate a day of delay and a few late-arriving rows. Models trained on shifting data need versioned datasets, so you can reproduce what a model saw last month.

That is a different design decision at the storage and transformation layers, and retrofitting it is painful. Our MLOps work deals with exactly that handover from pipelines to models.

If you are still on-premise, the move to a cloud warehouse is its own project. Scope cloud migration separately so the two do not blur.

Start with the workflows that break, not the vendors

Before you shortlist anyone, list the three or four reports or processes that currently depend on someone manually fixing data. Those are your real requirements, and they differ from any vendor's feature list.

Hand each shortlisted supplier the same sample export and the same list of broken reports. Whoever asks the best questions about that export before quoting is probably the one who has done it before.

Our broken workflow assessment produces that list for you: where data gets copied by hand, which numbers people distrust, and which fix is a pipeline versus a policy. It is the best first step, and it will tell you plainly whether you need a platform, a partner, or neither.