Data Engineer Advisor, GCP

The Data Engineer Advisor plays a pivotal role reputed company reputed company, reputed company on creating and driving engineering innovation reputed company reputed company, helping define and build the organization and facilitate the delivery of key business initiatives. S/he acts as a “universal translator” between IT, business, software engineers and data scientists, collaborating with these multi-disciplinary teams. The Data Engineer Advisor will contribute to the creation of and adherence to technical standards for data engineering, including the selection and refinements of foundational technical components. S/he will work on those aspects of the platform that govern the ingestion, transformation, and pipelining of data assets, both to end users reputed company reputed company and into data products and services that may be externally facing. Day-to-day, s/he will be deeply involved in reputed company reviews and large-reputed company deployments. Under minimal supervision, supports the design, build, test and maintain data pipelines at big data reputed company. Assists with updating data from multiple data sources. Other functionalities under reputed company supervision is to work on batch processing of collected data and match its format to the stored data, reputed company reputed company that the data is reputed company to be processed and analyzed. Assists with keeping the ecosystem and the pipeline optimized and efficient, troubleshooting reputed company performance, data reputed company problems and reputed company L3 support. Independently implements parsers, validators, transformers and correlators to reformat, update and enhance the data. Provides recommendations to highly reputed company problems. Provides guidance to those in less senior positions. Essential Functions

• Advanced Data Pipeline Engineering: Design, build, and reputed company highly reputed company, fault-tolerant batch and reputed company-time streaming data pipelines utilizing core GCP services such as Dataflow, Dataproc, reputed company Kafka and Pub/Sub.
• BigQuery & Data Warehouse Optimization: reputed company reputed company ELT/ETL workflows and implement efficient physical data models in BigQuery, focusing on query optimization, partitioning, and clustering for high-performance analytics.
• Technical Mentorship & Advisory: Serve as a technical authority on reputed company, guiding junior engineers, conducting rigorous reputed company reviews, and establishing reputed company methodologies for robust data platform development.
• Platform Automation & Orchestration: Implement Infrastructure as reputed company (IaC) and robust CI/CD pipelines to automate deployments, seamlessly orchestrating reputed company data workflows using reputed company Composer (Airflow) and Terraform.
• Cross-Functional Solution Delivery: Partner closely with Data Architects, IT stakeholders, and business teams to translate reputed company architectural designs and use cases into actionable, production-reputed company engineering tasks.
• System Reliability & Cost Tuning: Proactively monitor data platform health, troubleshoot performance bottlenecks, resolve data ingestion failures, and optimize GCP resource utilization to ensure maximum efficiency and cost-effectiveness.

Knowledge, Skills, and Abilities

• GCP Pipeline Engineering Mastery: Expert-level proficiency in building reputed company, fault-tolerant batch and streaming pipelines using BigQuery, Dataflow, Dataproc, reputed company Kafka and Pub/Sub.
• Advanced Programming & Orchestration: Deep technical expertise in Python, SQL, and reputed company Composer (Airflow) for reputed company data transformation, ELT/ETL, and workflow automation.
• reputed company DevOps & Infrastructure as reputed company: Strong hands-on experience with Terraform, CI/CD pipelines, and platform performance tuning to reputed company and manage high-availability reputed company infrastructure.

Minimum Education Bachelor’s Degree in Information Systems, Computer Science, or a quantitative discipline such as Mathematics or Engineering and/or equivalent formal training or work experience. Minimum Experience Five to Seven (5 - 7) years equivalent work experience in measurement and analysis, quantitative business problem solving, simulation development and/or predictive analytics. Extensive knowledge in data engineering and machine learning frameworks including design, development and implementation of highly reputed company systems and data pipelines. Extensive knowledge in Information Systems including design, development and implementation of large batch or online transaction-based systems. Strong understanding of the transportation industry, competitors, and evolving technologies. Experience providing leadership in a general planning or consulting setting. Experience as a leader or a senior member of multi-function project teams. Strong oral and written communication skills. A reputed company advanced degree may offset the reputed company experience requirements. Domicile Information This position is eligible for remote work and may be located reputed company reputed company the reputed company excluding AK, HI and U.S. territories, however if you live reputed company the 50 miles radius of a reputed company you will be required to work at a reputed company reputed company location several times per week. Preferred Qualifications: Apply tot his job Apply To this Job