JobCubby JobCubby
Sign up
DATAECONOMY

Senior Data Engineer - Databricks

DATAECONOMY Raleigh, North Carolina, United States

Information Technology & Services · 201-500 employees

Sep 10
data-engineer Senior (5-10 yrs) Full-time United States
Create a free account to apply — email only, no card. You can also save this posting or score it against your profile with AI.

About the role

Build and maintain scalable, production-grade ETL/ELT pipelines using Databricks and modern data lakehouse architectures. Implement best practices for job orchestration, monitoring, and data quality across Bronze, Silver, and Gold layers.

What they look for

Databricks PySpark Spark SQL Delta Lake Unity Catalog ETL/ELT Medallion Architecture Python SQL Data Modeling CI/CD Git AWS Azure GCP Data Engineering

Requirements

Requires 7-10+ years of experience in data engineering with strong proficiency in Python, PySpark, and SQL. Candidates must have hands-on experience with Databricks ecosystem tools including Delta Lake, Unity Catalog, and cloud platforms like AWS, Azure, or GCP.

Full description

DATAECONOMY is one of the fastest-growing Data & Analytics company with global presence. We are well-differentiated and are known for our Thought leadership, out-of-the-box products, cutting-edge solutions, accelerators, innovative use cases, and cost-effective service offerings.

We offer products and solutions in Cloud, Data Engineering, Data Governance, AI/ML, DevOps and Blockchain to large corporates across the globe. Strategic Partners with AWS, Collibra, cloudera, neo4j, DataRobot, Global IDs, tableau, MuleSoft and Talend.

Senior/Lead Data Engineer — Databricks

Rutherford, NJ/ Jersey City, NJ

Full-time

  • Build scalable, production-grade ETL/ELT pipelines using Databricks (PySpark, Spark SQL, Delta Live Tables, Workflows).
  • Ingest structured, semi-structured, and streaming data into Bronze, Silver, and Gold layers.
  • Develop optimized transformations, data quality rules, and reusable framework components.
  • Implement best practices for job orchestration, monitoring, alerting, and automation.
  • Hands-on experience: Spark, Delta Lake, Workflows, Unity Catalog.
  • Strong SQL programming and performance tuning skills.
  • Experience with cloud environments (AWS/Azure/GCP).
  • Experience with modern data lakehouse concepts and distributed systems.
  • Strong understanding of Lakeflow Connect, LSDP/Lakehouse, Medallion Architecture, Data Validations, Genie, and Agent Bricks/RAG use cases.
  • Should be able to explain these concepts using real project examples and architecture decisions.
  • Knowledge of medallion architecture, DLT and unity catalog within Databricks.

Requirements

  • Strong Python (PySpark) and SQL programming
  • Databricks — Spark, Delta Lake, Workflows, Unity Catalog
  • ETL/ELT pipeline development — Medallion Architecture (Bronze/Silver/Gold)
  • Delta Live Tables, Auto-Loader, Structured Streaming
  • Data modeling — dimensional (star/snowflake), normalization/denormalization
  • CI/CD, Git, job orchestration
  • Cloud experience — AWS, Azure, or GCP
  • 7–10+ years in data engineering
  • Knowledge of medallion architecture, DLT and unity catalog within Databricks.

Nice-to-Have Skills

  • Lakeflow Connect, LSDP/Lakehouse, Genie, Agent Bricks/RAG use cases
  • Data governance, metadata management, Unity Catalog advanced features
  • Airflow, dbt, or similar orchestration tools
  • Data security, compliance, and access models
  • Cost optimization and performance tuning in cloud environments
  • Corporate/enterprise data warehousing background

Similar roles