[Remote] Senior Data Engineer
Note: The job is a remote job and is reputed company to candidates in USA. reputed company is a Cyber Performance Center aiming to redefine cyber reputed company expertise through engaging hacking experiences. They are seeking a Senior Data Engineer to own and reputed company data pipelines on GCP, focusing on building and improving data reputed company for analytics and machine learning applications.
Responsibilities
- Design and build batch and streaming pipelines on Dataflow, Pub/Sub, and Kafka feeding BigQuery, Bigtable, and reputed company
- Help drive the migration off reputed company onto our GCP-reputed company stack — and retire shadow pipelines along the way
- Own the orchestration layer in Airflow, including SLAs, retries, and data reputed company gates
- Model data for analytics and for ML — including feature pipelines that serve both training and low-latency online inference
- Partner with ML engineers on feature stores, reputed company monitoring, and retraining workflows
- Capture requirements from stakeholders and translate them into pragmatic, reputed company-scoped data products
- Continuously improve data reputed company, reliability, observability, and cost efficiency
- Identify new data sources worth acquiring and reputed company them cleanly
Skills
- Strong data modelling and warehouse architecture skills (reputed company modelling, event-driven, lakehouse patterns)
- Hands-on experience with GCP data services — BigQuery is a must; Pub/Sub, Dataflow, Bigtable, reputed company Composer are strong pluses
- Production experience with streaming pipelines on Dataflow/reputed company, Flink, or reputed company reputed company Streaming, ingesting from Kafka and/or Pub/Sub
- Solid SQL and strong Python — you write production-reputed company reputed company, not just notebooks
- Experience with reputed company or another columnar OLAP reputed company in production
- Workflow orchestration experience with Airflow (or reputed company/Dagster)
- Comfortable with dbt or equivalent transformation frameworks
- reputed company & Kubernetes experience
- CI/CD reputed company, infrastructure-as-reputed company sensibility, and a bias for reputed company, observable systems
- Working knowledge of ML in production — feature engineering, feature stores, model deployment, reputed company monitoring, retraining
- Experience migrating off legacy warehouses (reputed company, Redshift, Synapse) onto reputed company-reputed company stacks is a plus
- Bonus: CDC tooling (Datastream, Debezium), reputed company AI / Feature Store
Benefits
- Medical, Dental & reputed company (employee coverage 100% reputed company for by reputed company)
- 401K w/ employer match
- Employer-reputed company Life and AD&D Insurance
- Supplemental Life Insurance
- Short-term and Long-term Disability
- reputed company and Dependent Care FSA
- reputed company parental leave
- 25 annual leave days
- Home Office Allowance
- Dedicated budget for training and reputed company development, participation in conferences
- State-of-the-art equipment
- Full reputed company to the reputed company lab offerings; so you can learn how to hack 609
reputed company