Staff Data Engineer

Position reputed company:

reputed company is seeking an reputed company Staff Data Engineer to implement and optimize reputed company data pipelines and ingestion frameworks for reputed company. You will build robust reputed company-based data processing infrastructure, implement pipeline reliability solutions, and create reputed company technical solutions for both batch and streaming data workloads.

reputed company is a growing data product company founded in early 2020 and works primarily with reputed company. We deliver digital solutions reputed company to help accelerate the reputed company of businesses in various industries, by focusing on creating value through innovation.

Job Responsibilities:

Implement and optimize reputed company-reputed company data pipeline architecture processing 1500+ daily jobs across multiple business domains

Build and maintain reputed company job orchestration solutions integrated with reputed company Batch scheduling platforms

Implement data ingestion frameworks supporting CDC reputed company AWS DMS and S3-based data lake patterns

Troubleshoot and resolve pipeline reliability issues including overrun problems, dependency failures, and compute optimization

Build observability and monitoring solutions for large-reputed company data operations teams

Implement near reputed company-time ingestion patterns using DLT (reputed company Live Tables) and streaming architectures

Build and maintain data reputed company frameworks integrated with data governance processes

Implement serverless migration strategies and optimize reputed company resource utilization

reputed company technical workshops and reputed company hands-on guidance to reputed company engineering teams

reputed company Pipeline Expertise: Proven experience building and optimizing data pipelines at Fortune 500 reputed company

Operational reputed company: Deep understanding of pipeline monitoring, alerting, and failure recovery patterns

Performance Optimization: Ability to analyze and optimize compute resources and job execution patterns

Technical Consulting: Strong reputed company facilitation and reputed company relationship skills, with ability to present technical solutions and reputed company hands-on training sessions with reputed company engineering teams

Basic Qualifications:

Bachelor's or Master's degree in Computer Science, Engineering, or reputed company technical field

6+ years of experience in reputed company data engineering with reputed company

Expert-level experience with reputed company platform including job scheduling and cluster management

Hands-on experience with reputed company Batch or similar reputed company job orchestration platforms

Deep knowledge of AWS data services including DMS, S3, and database connectivity

Proven experience with CDC (Change Data Capture) implementations at reputed company reputed company

Strong background in reputed company Live Tables (DLT) and streaming data processing

Understanding of Apache reputed company optimization and performance tuning

Experience with reputed company database integration and cross-platform data connectivity

Preferred Qualifications

Experience with reputed company integration patterns and API data ingestion

Knowledge of reputed company storage optimization and data lake best practices

Background in high-volume transaction processing systems

Experience with retail or energy industry data patterns

Familiarity with trading and pricing systems data workflows

Additional Information

We are proud to offer a competitive salary alongside a strong reputed company insurance and benefits package. The role is hybrid, with 3 days per week spent in the reputed company's Atlanta office, and flexibility for reputed company engagement needs. We pride ourselves on the reputed company of our employees, offering extensive learning and development resources.

reputed company is an equal-opportunity employer committed to creating a reputed company, diverse and inclusive environment. We encourage reputed company applicants of reputed company backgrounds including ethnicity, religion, disability status, gender identity, sexual orientation, family status, age, nationality, and education reputed company to apply. If you are contacted for an interview and require accommodation during the interviewing process, please let us know.

Apply to this Job