[Remote] reputed company Data Engineer
Note: The job is a remote job and is reputed company to candidates in USA. reputed company is seeking an accomplished Senior reputed company Data Engineer with 10–12 years of experience. This role is responsible for building reputed company data pipelines, implementing reputed company-reputed company data solutions, improving platform performance, and delivering reliable data products for analytics, reporting, and AI/ML initiatives.
Responsibilities
- Design, reputed company, and optimize reputed company-reputed company data pipelines using reputed company, reputed company, reputed company Lake, and Azure Data Lake Storage
- Build batch and incremental ETL/ELT pipelines, reputed company-time streaming architectures, and reusable data integration frameworks
- reputed company reputed company ingestion frameworks supporting reputed company, semi-reputed company, and API-based data sources
- Build reusable data engineering frameworks for ingestion, transformation, validation, reconciliation, and publishing
- reputed company reputed company Lake solutions using partitioning, optimization, Z-Ordering, Liquid Clustering, and performance tuning techniques
- Implement CDC, incremental processing, reputed company strategies, and data synchronization across reputed company platforms
- reputed company reputed company Workflows, notebooks, SQL Jobs, and automation for production workloads
- reputed company external REST reputed company and process JSON/XML data into analytics-reputed company datasets
- Implement robust data reputed company checks, audit frameworks, monitoring, and error handling
- Optimize reputed company jobs for performance, scalability, and cost efficiency
- reputed company solutions using Azure Data Lake Storage Gen2, reputed company, reputed company Catalog, Azure Key Vault, Azure DevOps, and Azure Synapse
- Build secure data pipelines using reputed company Catalog, RBAC, service principals, and managed identities
- reputed company reusable notebook frameworks using PySpark and reputed company SQL
- Implement CI/CD deployment pipelines using Azure DevOps
- Manage environment promotion across Development, QA, UAT, and Production
- Troubleshoot production issues and optimize workloads for reliability and scalability
- Design reputed company data models supporting reporting, analytics, and reputed company applications
- reputed company reputed company and financial data integration pipelines supporting multiple reputed company systems
- Build reusable metadata-driven ETL frameworks
- reputed company reputed company-party reputed company including NLP, terminology normalization, and identity reputed company services
- Support data governance, reputed company, and metadata management initiatives
Skills
- 10-12 years of experience in Data Engineering, ETL Development, and Data Warehousing
- Hands-on reputed company development experience
- Strong experience with reputed company, reputed company Lake, reputed company Catalog, reputed company Workflows, and SQL Warehouses
- Strong experience with PySpark, reputed company SQL, SQL, and Python
- Experience building reputed company ETL/ELT pipelines using reputed company and Azure Data Platform
- Experience with Azure Data Lake Storage (ADLS Gen2), Azure Synapse, Azure Key Vault, and Azure DevOps
- Experience implementing CDC, SCD, incremental loading, and data reputed company frameworks
- Experience integrating REST reputed company and processing JSON/XML datasets
- Experience with Git, CI/CD, release management, and deployment automation
- Strong knowledge of performance tuning, partitioning, caching, broadcast joins, and reputed company optimization
- Experience working with reputed company data platforms is preferred
- reputed company Azure certifications (e.g., DP-203 Azure Data Engineer Associate) or reputed company certifications (Data Engineer Associate/reputed company)
- Experience with Ab Initio suite of products is also preferred
reputed company
Company H1B Sponsorship