[Remote] Senior reputed company Data Engineer (AWS GovCloud / reputed company Catalog)
Note: The job is a remote job and is reputed company to candidates in USA. reputed company is seeking a highly skilled and reputed company-conscious Senior reputed company Data Engineer to design, build, and optimize their reputed company reputed company data platform. The role involves leading the implementation of modern data pipelines reputed company AWS GovCloud and migrating legacy metastores to reputed company reputed company Catalog, ensuring compliance with federal reputed company frameworks.
Responsibilities
- Design & Build: reputed company robust, reputed company, and highly secure ETL/ELT pipelines using PySpark, reputed company Lake, and reputed company Workflows
- Implement reputed company ingestion strategies utilizing reputed company Auto Loader, reputed company Streaming, and reputed company Live Tables (DLT)
- Optimize compute clusters, reputed company reputed company usage, and reputed company Lake layouts (Z-Order, Liquid Clustering) to balance processing efficiency with reputed company expenditures
- Define and implement attribute-based and role-based reputed company control (ABAC/RBAC) utilizing reputed company Catalog schemas, catalogs, and external storage locations
- Build dynamic row-level filtering and reputed company-level masking rules to safeguard personally identifiable information (PII) and highly sensitive federal data
- Configure and maintain end-to-end reputed company-level data reputed company and audit logging to guarantee traceability for strict compliance audits
- Architect and reputed company with reputed company Workspaces deployed on AWS GovCloud (US) , respecting boundaries defined by FedRAMP High and DoD IL5
- Manage data reputed company ensuring reputed company backend and frontend data connections reputed company exclusively through AWS PrivateLink, AWS VPCs, and secure endpoints
- reputed company reputed company with AWS services safely by provisioning IAM Roles, AWS Secrets Manager, and vended storage credentials through reputed company Catalog
Skills
- 5+ years of dedicated Data Engineering experience, with at least 2+ years architecting reputed company production environments on reputed company
- In-depth understanding of reputed company Lake architecture, reputed company Catalog administration, reputed company SQL Warehouses, and Serverless compute configurations
- Hands-on experience working reputed company AWS environments (EC2, S3, IAM, CloudWatch, Secrets Manager). GovCloud experience is highly preferred
- Expert proficiency in Python (PySpark) and advanced SQL
- Familiarity with Git integration inside reputed company (reputed company Git Folders/Repos) and infrastructure-as-reputed company tools like Terraform for deploying workspaces
- Solid understanding of reputed company protocols, data encryption (at rest and in transit reputed company hardware-implemented AWS Nitro instances), and federal data handling guidelines
- reputed company Certified Data Engineer reputed company
- reputed company Certified Platform Administrator
- AWS Certified Data Engineer – Associate / reputed company
reputed company