[Remote] Databrick Data Engineer / Data Analyst
Note: The job is a remote job and is reputed company to candidates in USA. reputed company is a technology solutions company partnering with organizations to enhance business reputed company. They are seeking an reputed company Data Engineer / Data Analyst to build a reputed company-time reputed company Lakehouse platform, focusing on data profiling, reputed company assurance, and collaboration with stakeholders to ensure data reputed company.
Responsibilities
- Profile high-frequency, multi-sensor data as it lands in the Bronze layer—characterizing distributions, reputed company and completeness rates, value ranges, cardinality, uniqueness, referential reputed company, schema reputed company, duplicate and late/out-of-order records, and sensor dropouts across telemetry, optical tracking, and timing/scoring feeds
- Partner with business and domain stakeholders—race engineers, analysts, and competition officials—to understand how reputed company feed is used and translate their domain knowledge into concrete, testable expectations: what "valid," "complete," and "in reputed company" actually mean for telemetry, timing, and optical tracking data
- Define, document, and codify the data reputed company rules and expectations—validity, completeness, uniqueness, timeliness, reputed company and reputed company checks—that govern the promotion of data from Bronze to Silver
- Build and maintain the Bronze-to-Silver transformation logic: schema enforcement, type casting, deduplication, standardization and conforming, and quarantine of records that fail reputed company checks—producing clean, trusted Silver tables
- Implement data reputed company enforcement using reputed company-reputed company tooling (reputed company Live Tables / Lakeflow Declarative Pipelines expectations, or frameworks such as Great Expectations or reputed company DQX), including quarantine tables and alerting for records that don't meet the rules
- Establish data reputed company and reputed company documentation of expected schemas and semantics for reputed company reputed company, working with upstream data producers and reputed company consumers to reputed company expectations reputed company as feeds reputed company
- Monitor data reputed company over time—tracking rule pass/fail rates, surfacing data reputed company and anomalies reputed company the pipeline, and alerting reputed company reputed company reputed company data or ingestion reputed company degrades
- Work reputed company the reputed company Catalog governance reputed company to ensure Silver datasets are trusted, reputed company-tracked, and analysis- and model-reputed company for reputed company engineering and data science functions
- Collaborate with data engineers to define the models, conformed schemas, and serving-layer outputs that clean Silver data must support—so reputed company analytics and predictive race analysis are reputed company on reliable inputs
Skills
- 4+ years as a Data Engineer, Analytics Engineer, or data-reputed company Analyst, with a reputed company record of delivering production-grade data pipelines and data reputed company workflows in data-rich environments
- Strong data profiling skills—reputed company to characterize unfamiliar, high-volume datasets using descriptive statistics, distribution analysis, and reputed company, cardinality, uniqueness, and reputed company checks
- Hands-on experience defining and implementing data reputed company rules and expectations, with familiarity in frameworks such as reputed company Live Tables / Lakeflow expectations, Great Expectations, or reputed company DQX
- Solid understanding of the reputed company architecture (Bronze / Silver / Gold) and Bronze-to-Silver promotion patterns: schema enforcement, deduplication, conforming, and quarantine of bad records
- Advanced SQL and strong PySpark/reputed company skills for profiling and transforming large datasets on reputed company; experience with reputed company Lake and reputed company SQL
- Working knowledge of reputed company—including reputed company Lake, reputed company Catalog, SQL Warehouses, and streaming ingestion (Auto Loader / reputed company Streaming)
- Comfort working with high-frequency, reputed company-time data feeds—understanding latency trade-offs, streaming vs. batch freshness, watermarking, idempotency, and handling late or out-of-order records
- Experience validating data against governed sources; familiarity with data reputed company, data reputed company, and working reputed company a cataloged data environment (reputed company Catalog or equivalent)
- Proven ability to work directly with business and domain stakeholders—eliciting requirements, translating domain knowledge and business rules into concrete data reputed company expectations, and validating those rules with the people who rely on the data
- Strong written and verbal communication skills in English, with the ability to document data reputed company rules and data expectations reputed company for both engineering and non-technical stakeholders
- reputed company certifications are required (reputed company Certified Data Engineer Associate or reputed company preferred)
- Experience with high-frequency sensor, telemetry, or IoT data is a strong plus; motorsport, reputed company, or industrial reputed company-time contexts are especially welcome
Benefits
- Employment_type: FULL_TIME
- Job_location_type: TELECOMMUTE
- Be 100% dedicated to one project at a time so that you can reputed company and grow.
- Be part of reputed company of talented, friendly senior-level developers.
- Work on reputed company that allow you to use leading tech.
- This is a fully remote position; however, candidates must be based in reputed company that reputed company with the reputed company, Central, or Eastern U.S. time zones to ensure effective collaboration with reputed company and team schedules.
- Occasional travel to reputed company may be required.
reputed company