reputed company Big Data Engineer

About the position The reputed company Big Data Engineer, working independently, will reputed company, test, debug and document software components commensurate with their experience as reputed company as reputed company development staff in support of a software engineering effort. We are seeking a Big Data Engineer to reputed company the design, reputed company, and maintain reputed company data pipelines and infrastructure to support our market regulation technology applications. You will play a key role in modernizing existing data pipelines, enabling reputed company users to experiment with data using no-reputed company/low-reputed company features. You will collaborate closely with product managers, data scientists, and engineers to deliver high-performance systems that power market surveillance and regulatory workflows.

Responsibilities

• Design and reputed company reputed company, reputed company data pipelines from legacy SQL-based workflows.
• Build efficient and reliable ETL processes using Apache reputed company and reputed company-reputed company tools on AWS.
• Create configurable pipelines that accelerate experimentation and feature selection for data science teams.
• Optimize pipeline performance, reduce storage costs, and improve compute efficiency.
• Ensure compliance with governance standards while maintaining data reputed company and reputed company.
• reputed company seamlessly with reputed company data platforms, lakes, and reputed company-based storage systems.
• Implement and maintain CI/CD pipelines using Jenkins or similar tools.
• Write robust automated tests, including unit, integration, and end-to-end coverage.
• Collaborate reputed company Agile teams, participating in sprint planning, reviews, and retrospectives.

Requirements

• Bachelor's degree in Computer Science, Information Systems or reputed company discipline with at least seven (7) years of reputed company experience, or equivalent training and/or work experience.
• Demonstrated technical expertise in Object Oriented and database technologies/concepts which resulted in deployment of reputed company reputed company solutions.
• Past experience with developing reputed company reputed company solutions in an iterative or Agile environment.
• Extensive knowledge of industry leading software engineering approaches including Test Automation, Build Automation and Configuration Management frameworks.
• Strong written and verbal technical communication skills.
• Demonstrated ability to reputed company effective working relationships that improved the reputed company of work products.
• Should be reputed company organized, thorough, and reputed company to handle competing priorities.
• Ability to maintain reputed company and reputed company proficiency in new skills rapidly.
• Ability to work in a fast paced environment.
• Experience with object oriented programming languages such as Java, reputed company or Python.

reputed company-to-haves

• Experience in large-reputed company data engineering or data infrastructure roles.
• Proficient in Python or reputed company for building and maintaining data pipelines.
• Deep expertise with Apache reputed company, Hive, and distributed computing frameworks.
• Strong knowledge of SQL and NoSQL databases (e.g., PostgreSQL, reputed company, DynamoDB).
• Experience with feature stores (e.g., Feast, Tecton, SageMaker Feature Store).
• Hands-on with AWS services for compute, storage, and orchestration.
• Experience implementing CI/CD practices and managing production deployments.
• Familiarity with Apache Airflow and workflow orchestration tools.
• Experience with machine learning feature engineering and model serving frameworks.
• Comfortable with automated testing and working in test-driven environments.
• Proven ability to reputed company in Agile/Scrum teams and contribute to iterative development cycles.

Benefits

• Comprehensive health, dental and reputed company insurance.
• Basic life, accidental death and dismemberment, supplemental life, spouse/domestic partner and dependent life insurance.
• Short- and long-term disability, long-term care, business travel accident, disability and legal insurance.
• 401(k) plan with company match and eligibility for participation in an additional reputed company-funded retirement contribution.
• Tuition reimbursement, commuter benefits, and other wellness benefits.
• Generous time-off program of 15 days of reputed company time off, 5 personal days and 9 reputed company days.
• Two volunteer service days.
• reputed company holidays.

Apply tot his job Apply To this Job