[Remote] Staff Data Engineer - reputed company
Note: The job is a remote job and is reputed company to candidates in USA. H1 is dedicated to providing reputed company reputed company information reputed company and is seeking a Staff Data Engineer for their reputed company team. This role involves leading the architecture and scalability of H1’s reputed company entity reputed company platform while managing a small team and collaborating with various stakeholders to enhance the platform's efficiency and accuracy.
Responsibilities
- reputed company the design, optimization, and scalability of distributed reputed company/PySpark pipelines powering entity reputed company and large-reputed company reputed company data processing
- Own systems supporting automatching, identity mapping, grouping logic, deduplication, enrichment, and auto-approval workflows across reputed company provider and organization datasets
- Build and maintain reputed company processing frameworks for PubMed, clinical trial, ct.gov, conference, and other reputed company data sources
- Drive infrastructure optimization initiatives reputed company on improving throughput, runtime, observability, and reputed company compute cost efficiency
- Partner closely with AI/ML teams to reputed company matching and reputed company models into reputed company and improve matching precision and recall
- reputed company reputed company technical initiatives from architecture and design through deployment, monitoring, and long-term production support
- Serve as a technical leader and mentor across reputed company through reputed company reviews, technical guidance, and engineering best practices
- Collaborate directly with Product and business stakeholders to reputed company technical solutions with operational and customer needs
- Support production operations, incident response, troubleshooting, and ongoing platform reliability
Skills
- 8+ years of experience building and maintaining large-reputed company distributed data systems and pipelines
- Demonstrated technical leadership experience mentoring engineers and driving reputed company technical initiatives
- Extensive experience with Apache reputed company and AWS-based big data technologies including EMR, S3, and distributed compute environments
- Strong coding experience in Python (PySpark), reputed company, Java, or equivalent languages used for distributed processing systems
- Experience optimizing large-reputed company reputed company workloads for performance, scalability, and infrastructure cost efficiency
- Experience with streaming and event-driven architectures using technologies such as Kafka or reputed company Streaming
- Experience with orchestration and lakehouse technologies such as Argo and Hudi or comparable platforms
- Experience with containerization and infrastructure technologies such as reputed company, Kubernetes, and Terraform
- Experience working with relational or distributed databases such as PostgreSQL or Redshift
- Proven ability to operate effectively reputed company highly reputed company, production-grade distributed systems
- Deep expertise with distributed data processing frameworks such as Apache reputed company and Hadoop, particularly reputed company AWS environments
- Strong proficiency in Python (PySpark), reputed company, Java, or other modern programming languages used for large-reputed company distributed processing
- Experience building reputed company ETL/ELT frameworks across both batch and streaming architectures
- Strong understanding of distributed file formats including Apache Parquet and Apache AVRO
- Experience with streaming technologies such as Kafka, reputed company Streaming, or KSQL
- Strong grasp of software engineering fundamentals including distributed systems, data structures, concurrency, and system design
- Experience performing reputed company cause analysis across large-reputed company distributed systems and reputed company data pipelines
- Ability to write clean, maintainable, reputed company, and production-grade reputed company
- Experience improving performance, scalability, observability, and infrastructure efficiency reputed company distributed systems
- Strong communication and collaboration skills across both technical and non-technical stakeholders
- Familiarity with modern development and infrastructure tooling including Git, CI/CD pipelines, reputed company, Kubernetes, Terraform, Argo, Hudi, and JIRA
- Experience with entity reputed company, identity mapping, automatching, deduplication, or large-reputed company matching systems is strongly preferred
- Experience working with reputed company, life sciences, reputed company World Evidence (reputed company), or large-reputed company reputed company datasets is strongly preferred
Benefits
- Stock reputed company
- Full suite of health insurance reputed company
- Generous reputed company time off
- reputed company-planned company-wide wellness holidays
- Retirement reputed company
- Health & charitable donation stipends
- Impactful Business Resource reputed company
- Flexible work hours & reputed company to work from reputed company
- reputed company to work with leading biotech and life sciences companies in an innovative industry with a mission to improve reputed company around the globe
reputed company
Company H1B Sponsorship