Senior Data Engineer Consultant

Fully On Site

Full time

San Francisco Bay California United States

Consulting

On Demand Talent

About This Role

RGP is seeking a highly experienced Senior Data Engineer Consultant to design, build, and optimize scalable data platforms and pipelines supporting analytics, business intelligence, and AI initiatives. This role will work across data engineering, cloud platforms, data architecture, and AI-enabled workflows, partnering closely with technical and business stakeholders to deliver reliable, high-performing data solutions.

The ideal candidate brings 10+ years of data engineering experience, strong hands-on expertise with Python, SQL, GCP, BigQuery, PySpark, Airflow, and streaming technologies, along with experience leading data engineering teams and complex projects from requirements through implementation and open to onsite project work.

Duration: 6-12 months

Pay Rate Range: $90/hr-$110/hr

What You Will Work On

Key Responsibilities

Data Engineering & Architecture

Design and develop scalable, enterprise-grade data pipelines and data platforms.

Build batch and real-time data processing solutions using PySpark, Dataflow, Kafka, Pub/Sub, and Apache Airflow/GCP Composer.

Develop data models, dimensional models, and analytical data layers to support reporting, BI, and AI use cases.

Design solutions across structured and unstructured data environments, including BigQuery, GCS, Hive, HDFS, and Parquet.

Gather technical requirements and translate business needs into scalable data solutions.

Cloud & Data Platform

Develop and optimize cloud-based data solutions within Google Cloud Platform (GCP).

Build and maintain BigQuery data environments and migrate legacy data platforms into cloud architectures.

Optimize data processing, storage, and query performance for high-volume datasets.

Develop solutions capable of processing millions or billions of records while maintaining reliability and scalability.

AI & Data Engineering

Integrate modern AI tooling into data engineering workflows to improve development, automation, data quality, and pipeline management.

Support data products and analytical solutions that enable AI/ML model measurement and business insights.

Identify opportunities to use AI to improve engineering productivity, pipeline monitoring, data quality, and remediation processes.

Experience with tools such as OpenAI Codex, Claude Code, GitHub Copilot, or similar AI development technologies is highly desirable.

Data Quality, Performance & Reliability

Identify and resolve pipeline performance, scalability, and data quality issues.

Implement best practices around data validation, observability, resiliency, and monitoring.

Optimize ETL/ELT processes and analytical queries to improve processing time and system performance.

Establish standards for data architecture, modeling, performance, and pipeline development.

Leadership & Project Delivery

Lead small to medium-sized data engineering projects and technical initiatives.

Provide technical leadership and mentorship to other data engineers.

Partner with business stakeholders, architects, analysts, data scientists, and engineering teams.

Participate in requirements gathering, solution design, technical documentation, development, testing, and deployment.

Develop project plans and provide technical guidance to ensure successful delivery.

What You Will Bring

Required Qualifications

10+ years of experience in Data Engineering, Data Warehousing, or related disciplines.

Strong hands-on experience with Python and SQL.

Strong experience with GCP and BigQuery.

Experience with PySpark, Apache Airflow/GCP Composer, Dataflow, Pub/Sub, or Kafka.

Strong understanding of dimensional modeling, data warehousing, ETL/ELT, and data architecture.

Experience working with high-volume data environments and performance optimization.

Experience gathering requirements and delivering data solutions end-to-end.

Experience leading technical projects or small-to-medium-sized engineering teams.

Strong communication skills with the ability to work with both technical and business stakeholders.

Preferred Qualifications

Google Cloud Professional Data Engineer certification.

Experience supporting AI/ML data products or AI-enabled workflows.

Experience with Looker, QlikView, SAP BusinessObjects, or other BI platforms.

Experience with Oracle/Exadata and legacy data warehouse modernization.

Experience with CI/CD tools such as Git, TeamCity, or similar platforms.

Experience working in Agile/Scrum environments.

Experience developing automated data quality, observability, or pipeline governance frameworks.

Core Technology Stack

Cloud: GCP, BigQuery, GCS

Programming: Python, SQL, PL/SQL, Shell

Data Engineering: PySpark, Dataflow, Airflow/GCP Composer

Streaming: Kafka, Pub/Sub

Data Platforms: Hive, HDFS, Parquet, Oracle Exadata

BI: Looker, QlikView, SAP BusinessObjects

AI: Claude Code, OpenAI Codex, GitHub Copilot

DevOps/Tools: Git, TeamCity, Jira

What You Can Expect

• An inspirational place for you to do your best work, be engaged in meaningful ways, and continually develop the skills, competencies and qualities that set our team apart. 

• Compensation commensurate with your qualifications, experience, and other factors including geographic location, market and operational factors. 

• Total Rewards include: Medical, Dental, Vision, Life Insurance, Disability Insurance, 401(k) Savings Plan, Employee Stock Purchase Plan, Professional Development Program, Paid Time Off and Paid Sick Time (in geographies where legally required).

What We Do

At RGP, we're creating a future where businesses produce their best work without constraints. We've built a global network of over 2,600 experts across four regions, providing a comprehensive suite of solutions across on-demand talent, next-generation consulting, and outsourced services to support organizations at every stage of their growth journey. Trusted by Fortune 100 companies and emerging disruptors alike, we challenge conventional ways of working, drive growth, and pave the way for long-term success through bold innovation and fearless collaboration.

Our values guide everything we do and strengthen our commitment to people. By combining smart processes, human-centered design, and advanced technology, we celebrate our team's excellence and ensure we grow together. We believe in the power of continuous learning and development to drive both individual and organizational success. It’s time to rethink how work gets done. Dare to Work Differently® with RGP.

RGP is proud to be an Equal Opportunity Employer and committed to creating an inclusive environment for all employees. We do not discriminate on the basis of race, color, religion, national origin, gender, pregnancy, sexual orientation, gender identity, age, physical or mental disability, genetic information, veteran status, or any other legally protected trait and encourage all applicants to apply.