Before diving into building your data platform on Azure, it’s worth hitting pause and focusing on the design of cloud resources. Taking the time to plan key components upfront can save you from costly rework down the line. A well-thought-out foundation doesn’t just help you avoid technical debt – it sets the stage for scalability, stronger governance, and better security from day one. Whether you’re building for a single team or an entire organization, making smart design choices early on will make it much easier to grow, maintain, and trust your platform over time.

In this article, we highlight five fundamental areas we recommend thinking through before starting any development work. We understand that tight deadlines, project pressure, or limited team availability can sometimes tempt you to dive straight into coding, but trust us – you’ll thank yourselves later when you see how much faster, smoother, and higher quality the work progresses in a well-prepared environment.

Landing Zones & Network Architecture

Let’s get this out of the way – yes, if you’re building a data platform, you will need some understanding of Cloud Infrastructure and Networking. You don’t have to be an expert on Microsoft’s Cloud Adoption Framework (CAF) or capable of designing a full enterprise network topology, but you do need to understand the fundamentals and be able to communicate effectively with the teams responsible for these areas.

From our experience, when it comes to setting up the platform, you’ll usually find yourself in one of two scenarios:

The mature setup – where the organization already has well-established cloud environment standards and Landing Zone patterns. Your task here is to align with those rules and build within their constraints.
The greenfield challenge – where those foundations are missing or loosely defined, and as part of your project, you’ll need to help establish at least the basics in collaboration with the client.

Whichever situation you’re in, there are a few things you’re likely to face:

Planning for environment segregation across Azure Subscriptions, including its impact on access control, networking, and governance
Applying environment-specific policies, often through Azure Policy
Budget planning and cost control strategies
The extensive topic of network architecture

That last one, networking, can easily take up a significant chunk of your project timeline – especially when you start discussing Private Link, VNet Peering, DNS resolution, or firewalls. These aren’t just technical footnotes; they’re key decisions that impact how secure, scalable, and maintainable your platform will be.

So what happens if you skip this part? In the worst case, you’ll find yourself – after weeks or even months of work – having to go back and revisit foundational decisions because the network traffic just isn’t flowing the way it should, or because security constraints block your access in production. That means delays, rework, and unexpected costs – all of which could have been avoided with a few early conversations and a bit of architectural planning.

Sounds interesting? Below are a few recommended links, and now let’s move on to IAM.

Identity & Access Management (IAM)

In our platform design, Microsoft Entra ID (formerly Azure Active Directory) should serve as the central hub for identity and access management. Before diving into resource implementation, it’s worth taking the time to carefully think through a few foundational assumptions. These early decisions can save significant effort later on – especially considering that rolling back access model changes in a live production environment is almost always far more time-consuming and complex than getting it right from the start.

Here are a few recommendations based on our experience:

Use domain-based identities wherever possible, keeping Microsoft Entra as the single source of truth for managing users and groups.
Avoid assigning permissions directly to individual users. Instead, define and manage access via security groups – especially when defining ownership..
Design a clear hierarchy of functional and non-functional groups, aligned with team responsibilities (e.g., platform, data engineering, data science).
Use Privileged Identity Management (PIM) for elevated permissions such as Contributor or Owner roles, to add an extra layer of control and visibility.
Follow the principle of least privilege. In particular, create narrowly scoped service principals, each tailored to a specific use case or pipeline, with only the permissions it truly needs.

A well-thought-out IAM strategy not only reduces long-term risk but also improves clarity, governance, and scalability.

Conventions

Wherever possible, it’s essential to define and enforce standards and conventions across your cloud project. This ensures consistency, improves collaboration, and simplifies automation and governance.

For example, consistent naming conventions for cloud resources help teams quickly understand the purpose, environment, and ownership of each resource, which is critical for effective management and collaboration. When naming patterns are predictable and standardized, it’s much easier to locate resources, interpret logs, apply policies, and ensure resources are correctly grouped and monitored. This becomes increasingly important as the environment scales and different teams – such as operations, security, and finance – interact with the same infrastructure. A well-designed naming convention can also help prevent misconfigurations, reduce the risk of human error, and support automation by making it easier to script against resource names in a consistent way.

Well-defined tagging conventions – such as tags for Environment, CostCenter, or Owner – are critical for cost management, accountability, and operational reporting. Tags provide structured metadata that can be used to group and analyze resources across subscriptions and regions, regardless of how they are named or where they reside. This enables teams to track spending per project, team, or business unit, and to identify unused or underutilized resources. Tags also play a key role in enforcing governance policies, enabling automation (e.g., auto-shutdown of non-prod environments), and ensuring clear ownership – which is especially important for incident response and support. Without a tagging strategy in place, managing a growing Azure environment quickly becomes chaotic and inefficient.

Equally important are coding standards – whether for Infrastructure as Code, notebooks, or data processing scripts – helps maintain consistent code quality, reduces onboarding time for new team members, and supports code reuse across environments and projects. Standards around structure, naming, documentation, and formatting lead to clearer, more reliable code that’s easier to review, debug, and extend. These conventions are not just a matter of preference – they are foundational to building a scalable and maintainable platform. To help enforce these standards, teams can leverage tools like pre-commit hooks, which automatically check for issues such as formatting errors, security risks, or missing documentation before code is committed. Combined with linters, formatters (such as black, flake8, or terraform fmt), and CI/CD pipeline checks, these tools serve as automated guardrails that encourage clean, consistent, and production-ready code from day one.

If you’re just about to define your conventions, we recommend starting with the following:

Infrastructure as Code

The days of manually provisioning cloud resources through the Azure portal are long gone. In modern data platform projects – just like in application development – Infrastructure as Code is not a luxury, it’s the standard. It enables teams to define, version, and automate the provisioning of infrastructure in a way that’s consistent, repeatable, and scalable.

IaC is especially critical in multi-environment setups, where the same platform components must be deployed across development, testing, staging, and production environments with minimal drift. Without IaC, maintaining consistency between environments becomes error-prone and time-consuming, which directly affects the reliability and trust in your data workflows.

Nothing accelerates platform development quite like well-designed, reusable IaC modules. These modules act as building blocks for your architecture, allowing teams to quickly spin up secure and compliant infrastructure without reinventing the wheel each time. A robust module library also encourages best practices, simplifies onboarding, and allows for faster experimentation and iteration.

Choosing the right tools for the job is equally important. Terraform is a popular choice in the Azure ecosystem due to its flexibility, strong community support, and wide adoption. However, alternatives like Bicep or Pulumi may be better suited for teams looking for tighter Azure integration or leveraging familiar programming languages.

Finally, adopting Infrastructure as Code is not just about tooling – it’s a mindset. It pushes teams toward automation, testing, and documentation by default, which ultimately leads to higher platform quality, faster delivery cycles, and better collaboration between data and DevOps teams.

Preferred tools? We’re big fans of Terraform! Mainly because of its cloud-agnostic approach and rich ecosystem of providers, including support for technologies like Databricks and Snowflake. For larger deployments, we recommend using Terragrunt and Atlantis to manage things more smoothly.

CI/CD and Environment Strategy

No modern data platform is complete without a well-defined development and deployment lifecycle. CI/CD practices, once reserved for application development, have now become essential in the world of data engineering and analytics as well. It all starts with clearly defined environments – dev, test, and prod – each isolated from the others to allow for safe testing and predictable releases. Your branching strategy should reflect this structure (e.g., trunk-based, GitFlow, etc.), supported by automated release pipelines.

As mentioned earlier, Infrastructure as Code tools like Terraform play a key role in this process, ensuring that environments are consistent, reproducible, and version-controlled. But CI/CD doesn’t stop at infrastructure. You should also think about how to automate the deployment of notebooks, workflows (e.g., jobs or pipelines), libraries, and even permissions. Whether you’re using native tools like Databricks CLI, Asset Bundles, or custom scripts, automating these pieces ensures quality, repeatability, and faster delivery across environments.

And just like with the other areas we’ve discussed, your deployment pipelines should be in place before anyone pushes the first line of code to shared environments. This ensures that from day one, you’re promoting a standardized, reliable development process, avoiding conflicts, manual fixes, and unnecessary redeployments.

Remember – DevOps from Day One! It’s not just a slogan; it’s a mindset that helps you scale with confidence and clarity.

Closing Thoughts about Data Platform

Defining the principles and addressing the challenges we’ve discussed above can feel like a massive task – which is exactly why it’s so often postponed to later phases of the project. And if we’re lucky, those pieces eventually get delivered.

But how can we tackle them early, without blocking the project for months?

Simple: find someone who’s done it before. When building a data platform, we don’t need to reinvent the wheel all the time. In majority of the cases, real business specificity, innovation, and differentiation emerge at the level of data engineering, modeling, and analytics – not in setting up the underlying platform. The platform is just a tool. So let’s lean on proven patterns, use solutions that work, and find someone who knows how to put the pieces together. Then trust them to do it right – and let the rest of the team focus on creating real value. From day one.

If you’d like to hear how we built an end-to-end Data Platform in just 8 weeks, join our webinar on May 13th here.

Key Areas to Plan Before Building Your Azure Data Platform