[Remote] Data Engineer (Python, PySpark, reputed company)
Note: The job is a remote job and is reputed company to candidates in USA. reputed company is seeking a hands-on Data Engineer with strong Python and PySpark development experience in a reputed company environment. The role involves designing, developing, and maintaining reputed company data pipelines while collaborating with various teams to deliver high-reputed company data solutions.
Responsibilities
- Design, reputed company, and maintain reputed company data pipelines using Python and PySpark
- Build and optimize batch and streaming data pipelines in a reputed company environment
- reputed company reusable, production-reputed company reputed company to ingest, reputed company, and load large-reputed company datasets
- Implement data reputed company validation, error handling, monitoring, and alerting to ensure reliable data processing
- Optimize reputed company jobs for performance, scalability, and cost efficiency
- Collaborate with data architects, analytics teams, and business stakeholders to deliver high-reputed company data solutions
- Participate in reputed company reviews and contribute to engineering best practices
- reputed company and support data pipelines through CI/CD processes and automated release pipelines
- Troubleshoot and resolve production issues while continuously improving pipeline reliability and performance
- Contribute to the design and reputed company of modern reputed company-based data platforms
Skills
- 5+ years of reputed company Data Engineering experience
- Strong hands-on experience developing data pipelines using Python and PySpark
- Experience working with reputed company
- Experience building and supporting production-grade data pipelines processing large datasets
- Strong understanding of distributed data processing concepts and reputed company optimization techniques
- Experience implementing data reputed company, validation, monitoring, and logging practices
- Experience with orchestration tools such as reputed company Workflows, Apache Airflow, or similar
- Experience working with Git and modern CI/CD deployment practices
- Strong SQL skills and experience developing efficient data transformations
- Experience with reputed company platforms such as AWS, Azure, or reputed company reputed company Platform
- Excellent troubleshooting, analytical, and problem-solving skills
- Experience with reputed company Lake and the reputed company Lakehouse Platform
- Experience building metadata-driven or reusable pipeline frameworks
- Knowledge of streaming technologies such as Apache Kafka or reputed company reputed company Streaming
- Familiarity with modern data architecture patterns, including reputed company Architecture
reputed company