Data Engineer - Databricks

Tata Consultancy Services•Bengaluru, Karnataka, India

fulltime

Information Technology

5+ years

TCS is Hiring Data Engineer - Databricks For Bangalore, Chennai and Hyderabad location

Experience : 5 - 8 years

Mode of interview : Virtual

Location : Bangalore, Chennai, Hyderabad

JOB DESCRIPTION:

Must have:

Strong experience in programming languages such as Python, PySpark, or Scala for data engineering tasks.
Design, develop, and maintain scalable and robust data pipelines on Databricks
Hands-on experience with cloud technologies, expertise in AWS/Azure data engineering services to handle large-scale data processing and real-time data streams.
In-depth knowledge of data warehousing solutions for building and managing data warehouses.
Proficiency in designing, developing, and maintaining ETL (Extract, Transform, Load) processes using cloud services, ADF, Glue etc

Good to have:

Strong understanding of data modeling concepts and techniques to create efficient and scalable data models.
Experience with version control systems such as Git for code management and collaboration.
Knowledge of data governance, data quality standards, and data security practices to ensure compliance and protection of sensitive information

Responsibilities:

Design, develop, and maintain scalable data pipelines for extracting, transforming, and loading data from various sources to ensure seamless data flow and accessibility.
Collaborate with cross-functional teams to integrate data from multiple disparate sources, ensuring consistency, accuracy, and reliability of data.
Optimize data processing workflows and storage solutions for performance, scalability, and cost-efficiency
Implement data quality checks and validation processes to ensure the accuracy, completeness, and consistency of data throughout the data lifecycle.
Monitor the performance of data pipelines and infrastructure, identifying and resolving issues to maintain system stability and reliability.