Senior Data Engineer - ETL Tools

We are looking for an experienced and innovative Senior Data Engineer to design, develop, and optimize scalable data systems and infrastructure. The ideal candidate will play a key role in building robust data pipelines, ensuring the availability and quality of data, and supporting advanced analytics and machine learning initiatives. This role requires a blend of technical expertise, leadership skills, and a strong commitment to best practices in data engineering. The core responsibilities for the job include the following : Data Architecture and System Design : - Design, implement, and maintain large-scale, distributed data platforms, including data lakes and warehouses. - Develop scalable and secure data architectures to meet current and future business needs. ETL/ELT Pipeline Development : - Build, monitor, and optimize data pipelines to efficiently ingest, transform, and store data from various sources. - Automate repetitive data engineering tasks to enhance productivity and consistency. Collaboration and Stakeholder Engagement: - Work closely with data scientists, analysts, and business stakeholders to understand data requirements and deliver solutions. - Collaborate with software engineers to integrate data systems with applications and services. Performance Optimization: - Ensure high performance, scalability, and availability of data infrastructure. - Optimize queries, workflows, and storage solutions for efficiency and cost-effectiveness. Data Governance and Security: - Implement and maintain data quality, governance, and security best practices. - Enforce data standards and compliance with regulations (e.g , GDPR, HIPAA). Mentorship and Leadership : - Provide technical guidance and mentorship to junior engineers and team members. - Lead technical projects and contribute to strategic decision-making related to data systems. Innovation and Continuous Learning : - Stay updated on emerging technologies and trends in data engineering. - Evaluate and recommend new tools, frameworks, and methodologies to improve processes and systems. Requirements : - Bachelor's or master's degree in computer science, Engineering, Information Systems, or a related field. - 4 years of experience in data engineering, big data systems, or a related field. - Proven experience in architecting and managing large-scale data solutions. Technical Skills : Programming : - Proficiency in Python, Java, or Scala for data processing. Data Frameworks : - Expertise in Apache Spark, Hadoop, Kafka, or Flink. Cloud Platforms: Hands-on experience with AWS (e.g , S3 Redshift, EMR), Azure, or Google Cloud (e.g , BigQuery, Dataflow). Data Warehousing: Strong knowledge of Snowflake, Redshift, BigQuery, or similar platforms. ETL Tools: Experience with Apache Airflow, DBT, Informatica, or similar tools. Database Management: Advanced skills in SQL and NoSQL databases (e.g , PostgreSQL, Cassandra, MongoDB). DevOps: Familiarity with CI/CD pipelines, Docker, Kubernetes, and infrastructure as code (e.g , Terraform) (ref:hirist.tech)

Location: hyderabad, IN

Posted Date: 5/2/2025