RedShift Data Engineer

Job Overview

We are seeking an experienced Redshift Data Engineer with 8+ years of experience in data engineering and strong hands-on expertise with Amazon Redshift. The ideal candidate will be responsible for designing, developing, optimizing, and maintaining scalable data platforms and ETL/ELT pipelines on AWS.

Key Responsibilities

• Design and develop scalable data pipelines and ETL/ELT workflows using AWS services.

• Develop and optimize complex SQL queries, stored procedures, views, and data models in Amazon Redshift.

• Perform Redshift performance tuning, query optimization, distribution/sort key optimization, and workload management.

• Build and maintain data ingestion pipelines from multiple sources into Amazon Redshift.

• Work with AWS Glue, S3, Lambda, Step Functions, and CloudWatch for data integration and orchestration.

• Develop robust data transformation processes using Python and SQL.

• Implement data quality, validation, reconciliation, and monitoring frameworks.

• Design and maintain dimensional data models, including fact and dimension tables.

• Work with large datasets and optimize data processing for performance and cost.

• Implement incremental data loading and CDC (Change Data Capture) strategies.

• Troubleshoot data pipeline failures, performance issues, and production incidents.

• Collaborate with Data Architects, Business Analysts, Developers, and other stakeholders.

• Follow best practices for data security, governance, access control, and AWS IAM.

• Participate in code reviews, technical design discussions, and Agile delivery processes.

Mandatory Skills

• 8+ years of experience in Data Engineering

• Strong hands-on experience with Amazon Redshift

• Advanced SQL skills

• Strong Python programming/scripting experience

• Experience developing ETL/ELT pipelines

• Strong experience with AWS

• Hands-on experience with Amazon S3

• Experience with AWS Glue

• Experience with data warehousing and dimensional data modeling

• Strong knowledge of Redshift performance tuning

• Experience with large-scale data processing

• Experience with CI/CD and Git

• Good understanding of data quality and data governance

Good to Have

• Experience with AWS Lambda, Step Functions, EMR, Athena, and CloudWatch

• Experience with Apache Spark / PySpark

• Experience with Airflow

• Knowledge of Kafka / streaming data

• Experience with CDC tools such as AWS DMS, Debezium, or equivalent

• Experience with Terraform / Infrastructure as Code

• Exposure to Databricks, Snowflake, or other cloud data platforms

• Experience working in Agile/Scrum environments

Technical Skills

Cloud: AWS, S3, Glue, Lambda, Step Functions, CloudWatch

Database: Amazon Redshift, PostgreSQL

Programming: Python, SQL

ETL/ELT: AWS Glue, Airflow, custom Python pipelines

Big Data: Spark, PySpark

DevOps: Git, CI/CD, Terraform

Data Modeling: Star Schema, Snowflake Schema, Dimensional Modeling

Streaming: Kafka – desirable