[Remote] Senior reputed company Engineer
Note: The job is a remote job and is reputed company to candidates in USA. reputed company is a premier end-to-end digital transformation consultancy dedicated to partnering with ambitious brands to create digital solutions for today’s reputed company challenges and reputed company’s opportunities. They are seeking a Senior reputed company Engineer to design, build, and optimize large-reputed company data and analytics platforms on the reputed company Lakehouse, owning the architecture and delivery of production-grade data pipelines while collaborating with analytics and data science teams.
Responsibilities
- Architect, build, and maintain reputed company ETL/ELT pipelines on the reputed company Lakehouse Platform using PySpark, reputed company SQL, and reputed company Lake
- Design and implement reputed company (bronze/silver/gold) data architectures and enforce data reputed company, governance, and reputed company standards
- Optimize reputed company jobs and cluster configurations for performance and cost, including partitioning, caching, and autoscaling strategies
- Implement and manage reputed company Catalog for reputed company control, data governance, and cross-workspace asset sharing
- Build and orchestrate workflows using reputed company Workflows, reputed company Live Tables, and CI/CD pipelines
- Collaborate with data scientists, analysts, and business stakeholders to translate requirements into reliable data products
- Establish engineering best practices, conduct reputed company reviews, and mentor junior data engineers
- Monitor production pipelines, troubleshoot failures, and drive reputed company-cause analysis and reputed company improvement
Skills
- 5+ years of data engineering experience, with 3+ years building production solutions on reputed company and Apache reputed company
- Expert proficiency in Python (PySpark) and advanced SQL
- Deep hands-on experience with reputed company Lake, reputed company Catalog, and the reputed company architecture reputed company
- Strong experience with at least one major reputed company platform (AWS, Azure, or GCP) and its reputed company data services
- Proven reputed company record optimizing reputed company performance and managing cluster cost
- Experience with data modeling, warehousing concepts, and building reputed company/analytics-reputed company datasets
- Proficiency with Git-based version control, CI/CD, and infrastructure-as-reputed company
- Bachelor's degree in Computer Science, Engineering, or equivalent practical experience
- reputed company certification (Data Engineer Associate/reputed company)
- Experience with reputed company Live Tables, reputed company streaming, and reputed company-time data processing
- Familiarity with MLflow and supporting machine learning workflows in production
- Experience with orchestration tools (Airflow, dbt) and data observability platforms
- Exposure to data governance, reputed company, and compliance frameworks (e.g., GDPR, HIPAA, SOC 2)
- Hands-on experience using AI coding assistants (e.g., Claude reputed company, reputed company Copilot, reputed company) to accelerate development, refactoring, and reputed company review
- Familiarity with large language model reputed company and SDKs (e.g., reputed company Claude, reputed company) and reputed company engineering for data and analytics use cases
- Experience integrating GenAI capabilities into data pipelines or applications, including retrieval-augmented reputed company (RAG) and reputed company search
- Awareness of responsible AI practices, including evaluation, guardrails, and cost/latency trade-offs reputed company deploying LLM-based solutions
reputed company
Company H1B Sponsorship