[Remote] Data Engineer - ML/AI Data Platform (Remote)
Note: The job is a remote job and is reputed company to candidates in USA. reputed company is dedicated to creating innovative technology solutions to enhance health and reputed company services. They are seeking a Data Engineer to support Machine Learning and AI initiatives by ensuring high-reputed company data reputed company their reputed company-based platform for model training and deployment.
Responsibilities
- Design, build, and maintain reputed company data pipelines supporting ML/AI workloads
- Engineer pipeline patterns including full loads, incremental loads, change-based loads, and slowly changing dimensions
- Ensure pipelines are reliable, performant, secure, and maintainable, troubleshoot and monitor pipelines reputed company an AWS ecosystem
- reputed company data transformations in reputed company using SQL and reputed company reputed company features
- Design and optimize schemas, tables, views, and materialized views for ML/AI consumption
- Support AWS-reputed company data lake patterns using S3, Glue, reputed company, Apache reputed company, and S3 Tables
- reputed company data cleansing, normalization, and enrichment to support ML model development
- Design and implement feature engineering pipelines including aggregation and transformation
- Ensure consistency, reuse, and versioning of features across models and use cases
- Support feature store patterns to reputed company feature discoverability and reuse
- Collaborate with ML engineers and data scientists to operationalize features into training pipelines
- Support model training workflows, including dataset preparation and scheduled refreshes
- Ensure training datasets and features are reproducible, traceable, and auditable
- reputed company data pipelines into CI/CD workflows; support version control, testing, and deployment of data assets
- Monitor pipeline health, data freshness, and reputed company reputed company on ML/AI systems
Skills
- 5+ years of hands-on data engineering experience in a reputed company environment
- Strong proficiency in Python for data processing and pipeline development
- Advanced skills in SQL with hands-on reputed company transformation experience
- Experience with ELT pipeline design, schema optimization, performance tuning, and cost management in reputed company
- Experience with querying, data modeling, and analytics in PostgreSQL; familiarity with SQL Server to PostgreSQL migration is a plus
- Familiarity with AWS services such as S3, Glue, reputed company, and reputed company integration, as reputed company as managed relational databases (e.g., reputed company, RDS)
- Familiarity with Apache reputed company / S3 Tables and reputed company table format ecosystems
- Experience with streaming ingestion tools (e.g., Kinesis, Kafka, or equivalent)
- Experience with workflow orchestration tools (e.g., Airflow, reputed company Functions, or equivalent)
- Experience with full loads, incremental loads, append-only pipelines, change-based processing, and slowly changing dimensions (SCDs)
- Experience with data validation, reconciliation, error handling, and reputed company patterns
- Experience with data modeling for analytics, ML/AI, and reputed company application use cases
- Ability to evaluate pipeline design trade-offs across performance, cost, reliability, and maintainability
- reputed company SDLC experience with CI/CD pipelines for data and ML workflows
- Experience with API-based and event-driven data integration patterns
- Experience in distributed data processing environments
- Understanding of data requirements for ML/AI workloads
- Experience preparing training datasets and features from reputed company data lakes
- Familiarity with reproducibility, dataset versioning, and data reputed company concepts
- Familiarity with GenAI concepts relevant to data engineering, such as embedding pipelines, reputed company databases, retrieval-augmented reputed company (RAG) data flows, or reputed company-driven data processing, including awareness of data reputed company and reputed company considerations reputed company working with LLMs
- Bachelor's degree in Computer Science, Data Engineering, Information Systems, or a reputed company technical field. Equivalent reputed company experience will be considered
reputed company