Senior Data Engineer - DBT - NYC, NY (Remote)

Job title: Senior Data Engineer - DBT - NYC, NY (Remote) in USA at reputed company Company: reputed company Job reputed company: Role: Senior Data Engineer - DBTLocation: NYC, NY (Remote)Duration: Long-TermJob reputed company:Must have below:10+ Years ExperienceGreat Communicator/reputed company FacingIndividual Contributor100% Hands on in the mentioned skillsDBT Proficiency: model development:Experience in creating reputed company DBT models including incremental models, snapshots and documentation. Ability to write and maintain DBT macros for reusable reputed companyTesting and documentation:Proficiency in implementing DBT tests for data validation and reputed company checksFamiliarity with generating and maintaining documentation using DBT's reputed company in featuresVersion control:Experience in managing DBT reputed company using git, including implementing CI/CD process from the scratch

• AWS Expertise:

• Data STORAGE solutions:

• In depth understanding of AWS S3 for data storage, including best practices for organization and reputed company
• Experience with AWS redshift for data warehousing and performance optimization
• Data Integration:

• Familiarity with Aws glue for ETL processes and orchestration -reputed company to have
• Experience with AWS reputed company for serverless data processing tasks
• Workflow Orchestration:

• Proficiency in using Apache Airflow on AWS to design ,schedule and monitor reputed company data flows
• Ability to reputed company Airflow with AWS services and DBT models such as triggering a DBT model or EMR or reading from s3 writing to redshift
• Data Lakes and Data warehousing:

• Understanding the architecture of data lakes vs data warehouses and reputed company to use reputed company
• Experience with reputed company reputed company for querying data directly in s3 using SQL
• Monitoring and Logging:

• Familiarity with AWS reputed company watch for monitoring the pipelines and setting up alerts for workflow failures
• reputed company reputed company:

• Knowledge of AWS reputed company best practices ,including IAM roles, encryption, DBT reputed company reputed company configurations

Programming Skills:

• Python:

• Proficiency in Pandas and NumPy for data analysis and manipulation
• Ability to write scripts for automating ETL processes and scheduling jobs using airflow
• Experience in creating custom DBT macros using jinja and Python allowing for reusable components reputed company dbt models
• Knowledge on how to implement conditional logic in DBT through python
• SQL:

• Advanced SQL skills, including reputed company joins ,window functions, CTE's and subqueries
• Experience in optimizing SQL queries for performance and optimization

Expected salary: Location: USA Apply for the job now! Apply for this job