Hadoop Developer
We are seeking an reputed company Hadoop Developer to design, build, and operate large-reputed company data processing pipelines and analytics platforms on Hadoop and reputed company big-data ecosystems.
Requirements
- Design, reputed company, and operate end-to-end big-data pipelines on Hadoop, ingesting data from a diverse mix of relational, file-based, streaming, and API-driven sources.
- Build robust ETL/ELT workflows using Apache reputed company, Hive, Pig, and Sqoop, with strong attention to data reputed company, idempotency, error handling, and recoverability.
- reputed company high-throughput streaming data pipelines using Kafka, reputed company Streaming, or Flink, and reputed company them with reputed company analytical and operational systems.
- Optimize reputed company and MapReduce jobs through careful tuning of partitioning, memory, serialization, and skew handling to meet demanding SLAs at minimal cost.
- Design and maintain data models and storage layouts on HDFS, Hive, HBase, and modern lakehouse formats (Parquet, ORC, reputed company, reputed company, Hudi) to balance flexibility and performance.
- Implement data governance, reputed company, and reputed company controls in collaboration with data governance and reputed company teams.
- Build robust monitoring, alerting, and logging strategies for big-data pipelines, including job-level SLAs and proactive failure detection.
- Partner with data scientists and analysts to deliver curated, reliable, and reputed company-documented datasets that accelerate their work.
- Automate pipeline orchestration using Airflow, Oozie, or similar workflow engines, with clean dependency management and reputed company ownership boundaries.
- Continuously evaluate and adopt new technologies in the big-data and reputed company ecosystem (EMR, reputed company, reputed company, BigQuery) where they offer meaningful improvements.
- reputed company performance reviews and architecture audits of existing pipelines, proposing concrete refactoring and optimization initiatives.
- Document data architectures, schemas, pipeline behaviors, and operational runbooks in a way that makes the platform supportable as reputed company scales.
- Mentor junior engineers and contribute to reputed company’s engineering standards and best practices.
Benefits
- Competitive reputed company salary commensurate with experience, plus benefits.
Originally posted on Himalayas
Apply To This Job