AWS Data Engineer Manager - Tableau/Power BI

AWS Data Engineer - Manager Company: Talent Worx (Representing a Big 4 in India) Location: Bengaluru, Karnataka, India (Also open to Coimbatore, Delhi NCR, Mumbai) Experience: 8 to 12 years Job Summary : Talent Worx is seeking a highly skilled and experienced AWS Data Engineer Manager to join our team, representing a prominent Big 4 firm in India. In this role, you will lead the design, implementation, and maintenance of robust data pipelines and analytics solutions on AWS. You will be responsible for ensuring the scalability, performance, and reliability of our data infrastructure, while also mentoring and guiding junior engineers. Key Responsibilities : - Design and implement scalable, high-performance data pipelines using various AWS services. - Optimize and automate ETL/ELT processes using AWS Glue, EMR, and Lambda. - Build and maintain data lakes using Amazon S3 and Delta Lake. - Implement data governance and security best practices for data lakes. - Develop and manage analytics solutions using Amazon Athena and Redshift. - Design and implement data warehousing solutions for reporting and analysis. - Design and implement database solutions using Amazon Aurora, RDS, and DynamoDB. - Optimize database performance and ensure data integrity. - Develop and implement serverless workflows using AWS Step Functions for data processing and orchestration. - Write efficient and maintainable code using Python/PySpark and SQL/PostgreSQL. - Optimize code for performance and scalability. - Ensure data quality, security, and compliance with industry standards and regulations. - Implement data validation and monitoring processes. - Collaborate with data scientists and analysts to understand their data needs and provide support. - Work with cross-functional teams to deliver data-driven solutions. - Optimize data architecture for performance, cost-efficiency, and scalability. - Troubleshoot and resolve data pipeline and infrastructure issues. - Mentor and guide junior data engineers. - Lead and manage data engineering projects. - Drive best practices in data engineering. Technical Skills : - AWS Glue, EMR, Lambda, Athena, Redshift, S3, Aurora, RDS, DynamoDB, Step Functions. - Hadoop, Spark, Delta Lake. Programming Languages : - Python, PySpark. Databases : - SQL, PostgreSQL, NoSQL databases. Data Warehousing & Analytics : - Data warehousing concepts, data modeling, and analytics tools. ETL/ELT Processes : - Experience in designing and implementing ETL/ELT processes. Data Lake Architecture : - Understanding of data lake architectures and best practices. Version Control : Git (GitHub, GitLab, Bitbucket). Agile Methodologies : Experience working in Agile/Scrum environments. Required Qualifications : Education : - Master's degree in computer science, Information Technology, or a related field. Experience : - 8 to 12 years of experience as a Data Engineer, with at least 60% of experience focused on AWS. AWS Expertise : - Strong proficiency in AWS data services (Glue, EMR, Lambda, Athena, Redshift, S3). Data Lake Experience : - Experience with data lake technologies, particularly Delta Lake. Database Expertise : - Expertise in database systems (Aurora, RDS, DynamoDB, PostgreSQL). - Programming Proficiency: Proficiency in Python and PySpark programming. SQL Skills : - Strong SQL skills and experience with PostgreSQL. Preferred Qualifications : - AWS certifications (AWS Certified Data Analytics - Specialty). - Experience with data governance and security frameworks. - Experience with data visualization tools (e.g., Tableau, Power BI). - Experience with infrastructure as code (IaC) tools like Terraform or CloudFormation. Benefits : - Opportunity to work with one of the Big 4 consulting firms in India. - Exposure to large-scale data engineering projects and diverse industry sectors. - Competitive compensation and benefits package. - Professional development and growth opportunities. - Dynamic and collaborative work environment. (ref:hirist.tech)

Location: in, IN

Posted Date: 5/1/2025