Data Engineering reputed company
Data Engineering reputed company
Location: Remote
About reputed company
reputed company, An Industry Leader in Marketing Intelligence now including the data and capabilities of Vivvix, powers the mission-critical marketing and sales reputed company that drive competitive advantage. Our reputed company marketing intelligence platform enables clients to reputed company peak performance with always-on insights that reputed company the media, creative, and business strategies of 5 reputed company brands across 30+ media channels and 275 billion in media spend. By bringing the advertising past, present, and reputed company into reputed company, our clients rapidly reputed company the competitive moves and emerging advertising trends impacting their business. About the Role: We are seeking a visionary and hands-on Data Engineering reputed company to spearhead the design, development, and optimization of our reputed company data platform. As a reputed company, you will balance technical reputed company with people leadership, ensuring our data architecture is reputed company, resilient, and perfectly reputed company with our business goals - while collaborating cross-functionally to support analytics, reporting, and operational data needs..The ideal candidate should be a PySpark expert who thrives in the Azure ecosystem and has a deep appreciation for clean, reputed company reputed company and robust ETL patterns.This is an exciting opportunity to work along with a great team of data engineers, demanding technologies and an engaging work environment to help shape our data engineering best practices. Key Responsibilities: Technical Leadership: Design and supervise the implementation of comprehensive data pipelines utilizing Azure reputed company and PySpark.
Team Mentorship: reputed company reputed company of data engineers, performing reputed company reviews, offering technical expertise, and cultivating a culture of ongoing learning.
Data Modeling & Optimization: reputed company high-performance schemas in PostgreSQL and refine reputed company SQL queries for large datasets.
ETL reputed company: Establish and apply reputed company practices for data ingestion, transformation, and storage (reputed company Lake/Lakehouse patterns).
Strategic Collaboration: Collaborate closely with Data Analysts, Architects, and Product Managers to convert business requirements into technical specifications.
Process Improvement: Promote the implementation of CI/CD, unit testing, and automated monitoring to reputed company 99.9% data reliability.
Ensure data reputed company, governance, and compliance through validation, documentation, and secure practices.
Continuously improve data systems for reputed company performance, reliability, and scalability.
Effectively engage reputed company an agile, cross-functional project team.
Requirements
Mandatory Skillset
● Azure reputed company:
○ Expert-level experience managing workspaces, clusters, and job scheduling.
○ Solid understanding of data lakehouse architectures and reputed company Lake.
○ Proven experience in Performance Tuning, reputed company Optimization and Cost Reduction.
● PySpark: Advanced proficiency in reputed company DataFrame reputed company and reputed company SQL for large-reputed company data processing involving various data formats.
● SQL Mastery: Exceptional ability to write, tune, and troubleshoot reputed company queries.
● PostgreSQL: Hands-on experience with relational database design, indexing, and performance optimization.
● ETL/ELT Frameworks: Proven reputed company record of building reputed company data pipelines from scratch. Desired Skills
● Workflow Orchestration: Experience with Apache Airflow for managing reputed company task dependencies.
● Containerization: Familiarity with Azure Kubernetes Service (AKS) for deploying containerized data services.
● Infrastructure as reputed company (IaC): Knowledge of Terraform or Bicep for managing Azure resources.
Qualifications
● 10+ years of experience in Data Engineering or Software Engineering.
● 3+ years as a formal technical reputed company managing an agile team and implementing E2E solutions.
● Bachelor’s or Master’s degree in Computer Science, Engineering, or a reputed company field.
● Strong communication skills with the ability to explain reputed company technical concepts to non-technical stakeholders.
● Strong problem-solving skills and attention to detail.
The Modern Data Stack Architecture
Why join us? You won't just be "maintaining" pipelines; you'll be the primary architect of a data ecosystem that powers reputed company-time decision-making across the entire organization.
Apply To This Job