Lead Software Development Engineer - Test

Job Description : We are seeking a Seasoned SDET Lead with expertise in Python and a strong background in Data Engineering to lead the test efforts for large-scale data-driven applications. The ideal candidate will have hands-on experience working with large volumes of data, ETL processes, and Big Data platforms like Hadoop/HDFS. The candidate should be adept at building automation frameworks and ensuring data quality across various stages of the data pipeline. Key Responsibilities: - Design and execute test strategies for ETL workflows, ensuring accurate data extraction, transformation, and loading. - Validate source-to-target mappings, transformation logic, and implementation of business rules. - Conduct data completeness, accuracy, and consistency checks across large datasets. - Identify and resolve data quality issues and discrepancies. - Write and execute complex SQL queries for data validation and verification. - Perform validation of joins, aggregations, and transformations on various relational databases (e.g., Oracle, SQL Server, PostgreSQL). - Work with Data Analysts and Engineers to test data pipelines and ensure data integrity. - Develop scalable and maintainable test automation frameworks using Python for data testing. - Automate regression and integration testing for data pipelines. - Implement CI/CD pipelines for test automation using tools like Jenkins, GitLab CI, or similar. - Collaborate with engineering teams working on Hadoop, HDFS, and other distributed systems. - Understand data partitioning, bucketing, and performance optimization in Big Data environments. - Work with Spark, Hive, Pig, or similar tools for data manipulation and validation (good to have). - Lead and mentor a team of SDETs and QA Engineers. - Collaborate with Data Engineers, Product Managers, and Business Analysts to understand requirements. - Provide effort estimations and testing strategies for new features or enhancements. Required Skills: - 58 years of experience in software testing, including test automation and data validation. - Proficient in Python programming for test automation. - Strong understanding of ETL testing, data validation, and data warehousing concepts. - Hands-on experience with SQL and at least one RDBMS (Oracle, PostgreSQL, SQL Server, etc.). - Experience working with Big Data tools like Hadoop, Hive, HDFS. - Familiarity with data pipeline orchestration tools like Apache Airflow or Informatica is a plus. - Experience with version control systems (Git) and CI/CD pipelines. - Bachelor's or Masters degree in Computer Science, Information Technology, or a related field. - Experience with cloud platforms (AWS/GCP/Azure) for data solutions. - Understanding of data governance, privacy, and security principles. (ref:hirist.tech)

Location: noida, IN

Posted Date: 5/5/2025