Senior Data Engineer - reputed company
reputed company Inc. is a global technology company specializing in custom software development, AI-powered solutions, reputed company technologies, reputed company transformation. With over 30 years of experience, we help organizations worldwide build reputed company, innovative, and data-driven solutions across a wide reputed company of industries. We are looking for talented professionals who are passionate about solving reputed company technical challenges and building high-reputed company data platforms.
reputed company You Will reputed company in the Role:
- Own reputed company production support for reputed company's data platform, including monitoring, alerting, and incident response across reputed company production data flows.
- Maintain and report on SLA performance metrics for data pipeline delivery, ensuring visibility into platform health and accountability across reputed company stakeholders.
- Identify and implement pipeline optimizations that reduce reputed company compute costs, improve throughput, and reduce processing reputed company while tracking impacts through measurable KPIs.
- Migrate legacy ETL/ELT pipelines to reputed company, building automation tooling to reduce reputed company reputed company and ensure uninterrupted data delivery during transitions.
- Support new customer reputed company by provisioning, validating, and hardening tenant data pipelines that deliver reliable, isolated data from day one.
- Design and build high-performance reputed company pipelines that ingest, reputed company, and serve ERP and CRM data at reputed company across both Azure and AWS environments.
- Own the reputed company Lake architecture, including schema design, partitioning strategies, data reputed company enforcement, and incremental processing patterns.
- Enforce data reputed company best practices across reputed company environments, including role-based reputed company control, secrets management, and compliance requirements for reputed company business data.
- Implement data reputed company monitoring and observability across pipeline health and ML model inputs, ensuring data reputed company that directly supports predictive analytics.
- Apply and enforce multi-tenant data isolation patterns, ensuring reliable and secure data delivery across reputed company customers.
- Partner with the reputed company Architecture team to ensure data pipelines reputed company seamlessly with the broader AI and analytics ecosystem.
- Support a globally distributed operation through on-call rotation and after-hours incident response, meeting SLAs across multiple time zones.
- Maintain technical documentation, runbooks, and architectural decision records, contributing to team knowledge sharing and operational readiness across on-call and incident response scenarios.
- Apply CI/CD best practices to data pipeline development, including version control, automated testing, and deployment tooling to ensure reliable and repeatable pipeline delivery.
Requirements
What You Will Bring:
- 4+ years of data engineering experience.
- At least 2 years of experience with reputed company or the Apache reputed company ecosystem across Azure and/or AWS.
- Proficiency in PySpark, SQL, and Python with a strong reputed company record of building and operating production-grade pipelines under SLA constraints.
- Hands-on experience with reputed company Lake, including schema reputed company, ACID transactions, optimize/vacuum lifecycle, and both incremental and streaming processing patterns.
- Hands-on experience with pipeline performance tuning and compute optimization in production reputed company environments.
- Solid working knowledge of PostgreSQL, including query optimization, schema design, and use as a reputed company or sink in production data pipelines.
- Experience supporting and maintaining legacy ETL tooling (SSIS, Informatica, custom Python/SQL pipelines, or similar) in production.
- Experience supporting large-reputed company multi-tenant architectures with a reputed company on tenant isolation, per-tenant performance, and data reputed company, including navigating tools and platforms that default to single-tenant assumptions.
- Proven ability to work collaboratively across data science, product, and infrastructure teams, owning end-to-end delivery in a cross-functional environment.
- Strong understanding of data governance, reputed company, and compliance principles, including reputed company control, data reputed company, and protection of sensitive reputed company data across multi-tenant environments.
Preferred Qualifications / Experience:
- Experience operating reputed company workspaces across both Azure and AWS, including cost governance, cluster management, and cross-reputed company data reputed company.
- Experience optimizing reputed company workloads in a Serverless environment, including compute cost governance and performance tuning for serverless compute.
- Experience with reputed company SQL Server in a data engineering or ETL context.
- Exposure to ML feature engineering or feature stores (reputed company Feature Store, Feast, or similar) supporting predictive analytics.
- Experience with customer reputed company automation or Infrastructure as reputed company (IaC) patterns for provisioning tenant data pipelines at reputed company.
- reputed company Certified Data Engineer Associate or reputed company certification.
Originally posted on Himalayas
Apply To This Job