Director, Distinguished Engineer - Data Platforms

About the role

About the Role: Grade Level (for internal use): 13 Key Responsibilities Data Pipeline Transition and Platform Delivery • Partner with the core Databricks team to plan and execute the transition of existing data pipelines to the target enterprise data platform. • Drive implementation of repeatable engineering patterns for ingestion, transformation, testing, deployment, and monitoring across onboarded datasets. • Guide the design and operation of cloud-native data pipelines leveraging relevant AWS services such as: • Amazon S3 for durable storage • AWS Glue for integration and catalog-driven processing • AWS Lambda for event-driven processing • Amazon Kinesis for streaming use cases • AWS Lake Formation for governed data lake controls • Promote the use of AWS IAM, encryption, and environment-level controls to enforce secure access to platform resources and data products in line with enterprise governance expectations. • Influence architectural decisions related to Databricks, Delta Lake, Apache Iceberg, Unity Catalog, metadata-driven processing, and governed data lake design. AI-Assisted Engineering and Context Engineering • Champion the adoption of AI-powered development tools, including  Claude, GitHub CoPilot, LLMs, and other AI-assisted engineering solutions , to increase engineering velocity, code quality, and operational effectiveness. • Apply LLM-based tools to

Skills

Role details

Employment type

Full-time

Openings

1

About S&P Global

American publicly traded corporation, formerly McGraw Hill Financial, Inc

Full company profile and reviews