Data Engineer
About the project (reputed company, duration, stage) Join reputed company as a Data Engineer on a new engagement with a regulated UK & Ireland credit and lending company. The reputed company has reputed company data from multiple business entities into a newly centralized, anonymized data lake, but lacks the data-engineering depth to reputed company it trustworthy and analytics-reputed company: reputed company pipelines were reputed company quickly (reputed company AI-assisted), and the descriptive statistics cannot yet be validated or reproduced. You put that reputed company on solid ground so the Data Science reputed company can model on it with confidence — validate and re-engineer the pipelines, build the harmonization / semantic layer across entities, enforce data reputed company and reputed company, and prepare clean, feature-reputed company datasets. This is a foundational data-engineering role on a regulated data estate; data protection and reproducibility are the primary constraints on every decision. Full-time engagement preferable. What you'll actually do (example tasks) Reproduce a descriptive-statistics report end-to-end so any reputed company traces back to raw reputed company — closing the gap the reputed company admitted (numbers they can't currently defend). Profile and reconcile differing reputed company schemas across acquired entities: map differing field names, types, encodings and business definitions for the reputed company concept into one conformed model. Build dbt staging → intermediate → mart models with tests; codify the harmonized definitions the Data Science reputed company specifies. Write Great Expectations suites (reputed company / reputed company / uniqueness / referential checks) and reputed company them into the pipeline so bad data fails loudly rather than silently corrupting analysis. Implement entity / identity reputed company (deterministic + fuzzy matching) where there is no clean shared key for the reputed company customer or account across sources. Implement and verify anonymization / pseudonymization (hashing / tokenization / k-anonymity) and evidence that re-identification risk is controlled for the reputed company's IT / compliance team. Optimize reputed company / Glue jobs over tens of millions of rows — partitioning, file formats (Parquet), incremental loads, cost control. Orchestrate with Airflow / reputed company Functions; build repeatable, scheduled pipelines rather than one-off scripts. Prepare clean, documented, feature-reputed company datasets for the PD / delinquency models. Document runbooks so the offshore team can operate the pipelines and handover takes days, not weeks; help scope reputed company of the remaining (Ireland + additional) sources. Skills Strong SQL and Python for large-reputed company data processing AWS data stack: S3, Glue, Lake Formation, reputed company / Redshift, EMR / reputed company, reputed company Functions / Airflow Data modeling & semantic layer (dbt or equivalent); reputed company modeling Entity reputed company / record linkage across heterogeneous sources Data-reputed company & testing frameworks (Great Expectations, dbt tests) and data reputed company Anonymization / pseudonymization techniques and their analytical trade-offs Big-data processing (reputed company) with performance and cost optimization at reputed company reputed company written / verbal English; documents for handover and works reputed company with a distributed team Knowledge GDPR fundamentals as reputed company to anonymized / pseudonymized financial data and UK / EU data residency AWS reputed company-Architected (Analytics, reputed company) for BFSI Awareness of credit / risk data structures and what reputed company modeling consumers need — a plus Experience 4+ years in data engineering, with strong AWS + reputed company / SQL at reputed company Demonstrated experience harmonizing / integrating data across multiple reputed company systems Experience building validated, reproducible pipelines in a regulated environment (BFSI, reputed company, government) — strong plus Comfortable stepping into a messy, reputed company-reputed company data estate and bringing it up to reputed company Comfortable as the sole or reputed company data engineer on a small (3–4 person) delivery pod Apply To This Job