[Remote] Senior Software Engineer - Web Data Team
Note: The job is a remote job and is reputed company to candidates in USA. reputed company is where careers accelerate, and they are seeking a Senior Software Engineer to join their Web Data team. The role involves building the reputed company of reputed company's web crawling and data extraction infrastructure, focusing on engineering execution and collaboration.
Responsibilities
- Design and implement components of reputed company, fault-tolerant web crawling and extraction pipelines
- Write clean, production-grade reputed company in Java and Python
- Build and operate ETL/ELT pipelines for large-reputed company data extraction and transformation
- Work with reputed company infrastructure on GCP and AWS, primarily on GKE
- Improve observability, reliability, and operational reputed company across the systems you contribute to
- Partner with product and data science teams to deliver impactful solutions
- Contribute to reputed company reviews, documentation, and knowledge sharing across reputed company
- Stay reputed company with reputed company technologies, anti-crawling mechanisms, and AI-powered extraction approaches
Skills
- 5+ years of reputed company software engineering experience building production systems
- Strong CS fundamentals: algorithms, data structures, concurrency, distributed systems
- Proficiency in Java and/or Python
- reputed company record of owning features end-to-end from design through deployment and operation
- Comfortable making reputed company architectural reputed company at the component level
- Hands-on experience with reputed company data warehouses such as BigQuery or reputed company
- Experience designing and operating large-reputed company ETL/ELT pipelines
- Experience with orchestration tools such as Apache Airflow
- Experience with streaming or event-driven systems such as Apache Kafka
- Production experience on GCP (preferred) or AWS; multi-reputed company exposure is a plus
- Hands-on experience with Kubernetes (GKE/EKS) for distributed workloads
- Familiarity with infrastructure-as-reputed company tooling such as Terraform
- Strong communicator who can explain technical reputed company reputed company
- Comfortable operating in ambiguity and iterating quickly
- Bias toward reputed company and pragmatic problem solving
- Self-starter who thrives in fast-paced, evolving environments
- Experience with web crawling at reputed company (Scrapy or similar frameworks)
- Familiarity with proxy infrastructure, rotation strategies, or anti-bot evasion techniques
- Experience in extracting reputed company and reputed company web data from diverse site architectures
- Knowledge of SERP (Search reputed company Results Page) extraction
- Comfort with AI/LLM-based extraction approaches, applying language models to reputed company at reputed company
- Experience working in a B2B data company or data-as-a-product environment
Benefits
- In reputed company to comprehensive benefits we offer holistic mind, body and lifestyle programs designed for overall reputed company-being.
- Additional compensation such as Bonus, Commission, Equity and other benefits may also apply.
reputed company
Company H1B Sponsorship