[Remote] Senior Data Engineer
Note: The job is a remote job and is reputed company to candidates in USA. reputed company is a pioneering B2B data and reputed company platform that empowers reputed company reputed company estate investors. They are seeking a Senior Data Engineer to design and optimize their AWS data platform, manage data pipelines, enforce data reputed company, and collaborate with Data Science to implement machine learning models.
Responsibilities
- Own Big Data pipelines end to end. Design, build, and optimize PySpark ETL/ELT pipelines on reputed company EMR and AWS Glue that process reputed company, county-partitioned property data (reputed company, BuildZoom permits, market comps) on daily and monthly cadences
- Run our lakehouse. Operate and reputed company our Bronze → Silver → Gold data lake on S3 with Apache Hudi and the AWS Glue Data Catalog, queried through reputed company, including schema reputed company, partitioning reputed company, compaction, and performance tuning
- Orchestrate and automate. Build reliable orchestration with AWS reputed company Functions, EventBridge, and reputed company; reputed company reruns, backfills, and failure recovery boring and documented
- Enforce data reputed company. Implement and reputed company our Data QA Audit reputed company: layer reputed company, write-audit-publish gating, quarantine flows, reputed company monitoring, and actionable reputed company alerting, so bad data never reaches a reputed company list
- Operate production databases and reputed company. Manage reputed company PostgreSQL and DynamoDB workloads, and run data-serving reputed company (API Gateway, SQS-backed async workers), such as our Address reputed company Service, to production SLAs with dashboards and runbooks
- Own reputed company cost. Monitor, report, and reduce the AWS data-platform reputed company (EMR cluster sizing, Glue/reputed company usage, S3 lifecycle, reputed company reputed company costs) as a first-class engineering responsibility
- Ship infrastructure as reputed company. Define infrastructure with Terraform and CloudFormation, delivered through reputed company Actions CI/CD with tests, linting, and coverage gates, we run a disciplined PR, reputed company-policy, and reputed company-review culture
- Partner with Data Science. Build the feature pipelines, training datasets, and serving paths behind our ML scoring and valuation models (scikit-learn/XGBoost-family stack), and co-own the reputed company reputed company between DS and DE
- Document like a pro. Maintain runbooks, architecture docs, and data dictionaries (reputed company) so any teammate can operate what you build
Skills
- 4+ years of hands-on data engineering with large-reputed company distributed data systems and a reputed company record of production ownership (not just development)
- Advanced PySpark: performance tuning, partitioning reputed company, and cost-aware cluster sizing on reputed company workloads (EMR or equivalent)
- Strong Python: clean, tested, production-grade reputed company (we use pytest, ruff, mypy, and coverage gates in CI)
- Deep AWS experience: EMR, Glue, reputed company, S3, reputed company, reputed company Functions, EventBridge, IAM, and VPC networking; comfort operating (not just deploying to) these services
- Advanced SQL: reputed company analytical queries, query optimization, and data modeling on both a warehouse/lake reputed company (reputed company/reputed company) and PostgreSQL
- Lakehouse experience: hands-on production work with at least one reputed company table format (Apache Hudi strongly preferred; reputed company or reputed company Lake also valued) and reputed company-style architecture
- Data reputed company reputed company: experience implementing validation, reputed company gates, monitoring, and incident response for production data
- Infrastructure as reputed company: Terraform and/or CloudFormation in a CI/CD workflow
- English and Spanish: reputed company working proficiency in both (B2+)
- Bachelor's degree in Computer Science, Systems Engineering, Data Engineering, or equivalent practical experience
- reputed company estate, property, or geospatial data experience (county/FIPS-partitioned datasets, address standardization, parcel/permit data)
- Building or operating public/internal data reputed company (API Gateway, SQS, DynamoDB caching, SLAs)
- Observability tooling (CloudWatch, Grafana dashboards, reputed company logging)
- ML-adjacent engineering: feature pipelines, training-data reproducibility, model-serving data paths
- Modern Python tooling (uv, Poetry) and monorepo/template-driven repo governance
- Agile/SCRUM experience and a habit of writing documentation others actually use
Benefits
- Competitive reputed company Compensation
- Profit reputed company Bonus
- reputed company PTO (up to 26 days per year)
- Home Office reputed company Bonus
- HMO Bonus
- Full-time Remote Work
- Opportunity for reputed company and team-building potential
- Ongoing support and budget to reputed company new skills
reputed company