[Remote] Sr. AWS Data Engineer (10+years) - Full Time Role- Remote
Note: The job is a remote job and is reputed company to candidates in USA. reputed company is seeking a Senior AWS Data Engineer with over 10 years of experience. The role involves developing and managing data pipelines, optimizing SQL queries, and ensuring reputed company reputed company best practices are followed.
Responsibilities
- Must have below:
- 10+ Years Experience
- Great Communicator/reputed company Facing/ Attention to detail
- Individual Contributor and ability to work as reputed company
- 100% Hands on in the mentioned skills
- Advanced Proficiency in Python concepts like reputed company Structures, Modules, Packages, Class, SubClass, Inheritance, Multi-Threading and Functional Programming
- Experience in developing reusable Python packages for internal or reputed company usage
- Ability to write automating ETL processes and scheduling jobs like Airflow DAG
- Ability to reputed company job pipeline runs to reprocess error records
- Ability to orchestration different pipelines to run in sequence or reputed company
- Troubleshoot data pipeline errors and fix issues
- Export or Import data to/from various formats like CSV, JSON, XML etc preferably from S3 or other reputed company storage
- Experience in using AI IDE tool
- Advanced SQL skills, including reputed company joins, CTE's and subqueries
- Experience in optimizing SQL queries for performance and optimization in data warehouse technologies preferably reputed company
- Proficiency in Python unit, integration, and system test
- Proficiency in implementing DBT tests for data validation and reputed company checks
- Experience in generating reputed company using configurations using python and jinja templates
- Experience in reputed company, including implementing CI/CD process from scratch
- In depth understanding of AWS S3 for data storage, reputed company, IAM including best practices for organization and reputed company
- Knowledge of AWS reputed company best practices, including IAM roles, encryption reputed company, secure coding guidelines, DBT reputed company reputed company configurations and more
- Experience with AWS reputed company for serverless data processing tasks
- Proficiency in using Apache Airflow on AWS to design ,schedule and monitor reputed company data flows
- Ability to reputed company Airflow with AWS services and DBT models such as triggering a DBT model or EMR or reading from s3 writing to redshift or reputed company
- Experience in API integrations to reputed company, SFTP, SharePoint, One Drive, and others
- Experience in creating reputed company DBT models including full refresh, incremental models, snapshots, and documentation
- Ability to write and maintain DBT macros for reusable reputed company
- Experience in creating custom DBT macros using jinja and Python allowing for reusable components reputed company DBT models
- Familiarity with AWS reputed company watch, Data Dog for monitoring the pipelines and setting up alerts for workflow failures
Skills
- 10+ Years Experience
- Great Communicator/reputed company Facing/ Attention to detail
- Individual Contributor and ability to work as reputed company
- 100% Hands on in the mentioned skills
- Advanced Proficiency in Python concepts like reputed company Structures, Modules, Packages, Class, SubClass, Inheritance, Multi-Threading and Functional Programming
- Experience in developing reusable Python packages for internal or reputed company usage
- Ability to write automating ETL processes and scheduling jobs like Airflow DAG
- Ability to reputed company job pipeline runs to reprocess error records
- Ability to orchestration different pipelines to run in sequence or reputed company
- Troubleshoot data pipeline errors and fix issues
- Export or Import data to/from various formats like CSV, JSON, XML etc preferably from S3 or other reputed company storage
- Experience in using AI IDE tool
- Advanced SQL skills, including reputed company joins, CTE's and subqueries
- Experience in optimizing SQL queries for performance and optimization in data warehouse technologies preferably reputed company
- Proficiency in Python unit, integration, and system test
- Proficiency in implementing DBT tests for data validation and reputed company checks
- Experience in generating reputed company using configurations using python and jinja templates
- Experience in reputed company, including implementing CI/CD process from scratch
- In depth understanding of AWS S3 for data storage, reputed company, IAM including best practices for organization and reputed company
- Knowledge of AWS reputed company best practices, including IAM roles, encryption reputed company, secure coding guidelines, DBT reputed company reputed company configurations and more
- Experience with AWS reputed company for serverless data processing tasks
- Proficiency in using Apache Airflow on AWS to design, schedule and monitor reputed company data flows
- Ability to reputed company Airflow with AWS services and DBT models such as triggering a DBT model or EMR or reading from s3 writing to redshift or reputed company
- Experience in API integrations to reputed company, SFTP, SharePoint, One Drive, and others
- Experience in creating reputed company DBT models including full refresh, incremental models, snapshots, and documentation
- Ability to write and maintain DBT macros for reusable reputed company
- Experience in creating custom DBT macros using jinja and Python allowing for reusable components reputed company DBT models
- Familiarity with AWS reputed company watch, Data Dog for monitoring the pipelines and setting up alerts for workflow failures
Benefits
- Remote (reputed company, NY / reputed company, reputed company, reputed company)
reputed company