Loading...
USM Jobs / Data Tech Lead – Azure & Databricks Platforms
High Contract

JB061706 - Data Tech Lead – Azure & Dat Apply

  • Start Date:
    Interview Types
  • Skills Azure Data Engineeri..
    Visa Types Green Card, US Citiz..
Job Title: Data Tech Lead – Azure & Databricks Platforms
Location: Houston, Texas 77002
Duration: 06 Months Contract
Only on W2
Client: SHELL Energy
Rate: $90-95/Hr on W2
 
Job Description:
As a Data Tech Lead, you will provide end-to-end leadership across architecture, engineering, governance, and capability development.
You will:
·         Deliver sustainable and innovative solutions to unlock value from data.
·         Lead the architecture, design, and implementation of on-premises/cloud-based application development and Data solutions.
·         Engage cross-functional teams to support project stakeholders in delivering data scope and operational excellence.
·         Drive governance, metadata management, Data Modelling, cost optimization, high-volume performance tuning,g and agility in delivery.
·         Engage effectively with senior leadership, resolve conflicts proactively, and represent the data organization with confidence and clarity.
 
Key Responsibilities
Leadership & Strategy
·         Drive the architecture, design, and delivery of Data ecosystems for a futuristic data platform.
·         Define data product roadmaps and drive development of data solutionsusing thee approved tech stack and related technologies.
·         Align data engineering initiatives with long-term business and technology strategies.
·         Provide technical leadership to ensure adherence to best practices, standards, and quality expectations.
 
Technical Oversight
·         Lead data modelling and architecture using tools such as SAP PowerDesigner and WizDesign.
·         Develop and optimize intelligent, AI-augmented data pipelines across ADF, ADLS, Databricks, DLT, and Structured Streaming to enable high-performance data delivery.
·         OLTP schema design, normalization, indexing strategy, Performance tuning for transactional systems
·         Transaction management, concurrency, deadlocks
·         PostgreSQL, MySQL, SQL Server expertise (depending on Shell standards)
·         Optimize Databricks workloads with a strong focus on performance and cost efficiency.
·         Implement robust security, governance, lineage, and metadata controls using Unity Catalog.
·         Troubleshoot pipeline issues, ensure high data quality, and enforce SLA compliance.
 
Governance & Quality Assurance
·         Establish data governance frameworks covering security, privacy, quality, and lifecycle management.
·         Monitor pipeline performance and lead root cause analysis for data and process issues.
·         Drive CI/CD and touchless change management frameworks to enable seamless deployments.
 
4. Stakeholder Management
·         Collaborate closely with PMs, BAs, Data Scientists, Architects, and business stakeholders.
·         Translate complex technical concepts into business terms for executive audiences.
·         Resolve conflicts and ensure alignment across strategic priorities.
·         Stakeholder management & executive communication
 
Required Skills & Experience
Technical Expertise
·         12–17 years of experience in data engineering, data architecture, or analytics platforms.
Deep expertise in:
·         Azure Data Engineering (ADF, ADLS, Databricks, DLT), AZURE Function, SQL, REST/FAST API, Logic Apps
·         Databricks Delta Live Tables & Structured Streaming
·         Unity Catalog governance and lineage
·         Strong hands-on skills in SQL, Python, Spark/Scala.
·         Strong experience in data modeling, data architecture, and metadata management.
·         CI/CD for application delivery (Azure DevOps/GitHub Actions)
·         Containerization (Docker), Kubernetes basics
·         API gateway, reverse proxies, load balancing
·         Solution Data Architecture,  Data Modeling tools like SAP PowerDesigner, Biz Design
 
AI & Future-Ready Skills
Working knowledge of AI/ML, GenAI, Agentic AI, and applying AI for data engineering automation.
Ability to envision and build future-ready, scalable data ecosystems.
 
Soft Skills / Leadership Competencies
·         Proven ability to lead mid-sized to large technical teams.
·         Strong decision-making capabilities backed by data.
·         Exceptional communication and executive presentation skills.
·         Conflict management and stakeholder alignment experience.
·         High accountability, ownership, and ability to drive independent outcomes.
·         Adaptability and continuous learning mindset.
 
Nice to Have
·         Experience with Kafka, Hadoop, Spark Streaming.
·         Experience with workflow tools like Airflow, Azkaban, Luigi.
·         Knowledge of Power BI/QlikSense visualizations.
·         Certifications in Azure Data Engineer, Databricks, AI/ML, or Data Science.
·         Experience with GitHub/Azure DevOps, CI/CD, Agile/Kanban methodologies.
·         Understanding of Freight Domain like Voyage Management, Chartering, Freight Pricing, Demurrage, Laytime, Port calls, Vessel scheduling
·         Integration with RightAngle, Endur, Veson IMOS, or internal Shell Freight systems
·         Knowledge of T&S business processes