Job Description

We are looking for an experienced Senior Azure Data Engineer & Test Engineer to design, develop, test, and maintain data products and systems. This role requires full stack ownership, focusing on creating production-ready code, ensuring scalability and performance, and adhering to best practices in software reliability. The candidate will work with a range of technologies including PySpark, ADF, Synapse, and Azure Data Lake, and will contribute to data engineering, ingestion, and testing data pipelines. Key Responsibilities: Product Development and Maintenance: Design, develop, test, operate, and maintain data products, ensuring high performance, scalability, and reliability. Take full stack ownership by consistently writing production-ready and testable code. Design Optimization: Consistently create optimal designs, adhering to architectural best practices. Consider scalability, reliability, and performance of systems when defining technical designs. Failure Analysis and Improvement: Perform analysis on system failures, propose design changes, and recommend operational improvements. Make sound design and coding decisions with customer experience in mind. Code Review and Standards: Take feedback from code reviews and apply necessary changes to meet team standards. Conduct code reviews to provide guidance on engineering best practices and compliance with development procedures. Compliance and Security: Ensure all aspects of product development follow compliance and security best practices. Software Reliability Engineering: Exhibit a relentless focus on software reliability engineering standards, integrating them into the development process. Embrace emerging technology opportunities and contribute to best practices in support of the companys technology transformation. Contribute to a culture of innovation, collaboration, and continuous improvement. Task Review and Prioritization: Review tasks critically and ensure they are appropriately prioritized and sized for incremental delivery. Anticipate and communicate blockers and delays proactively before they require escalation. Documentation and Support: Document all work, provide pipeline demos, and offer support for UAT and production deployment. Solution Architecture and Data Analysis: Provide solution architecture support and assist in data analysis when required. Required Qualifications: Experience: 5+ years of experience in data engineering, particularly with Azure technologies. Proficient in building and testing data pipelines with tech stacks like PySpark, Azure Data Factory (ADF), Synapse, Azure Data Lake, and SQL Server. Technical Skills: Strong experience with data engineering, ingestion, and pipeline testing. Familiarity with DevOps practices and tools for CI/CD. Hands-on experience with Kubernetes, Azure Service Fabric, and Delta Lake is a plus. Knowledge of data integration tools like Informatica and SSIS. Additional Skills: Strong problem-solving skills and the ability to deliver solutions under pressure. Ability to work with teams to document processes, deliver pipeline demos, and provide UAT and production support. Preferred Qualifications: Experience with Azure Data Services (e.g., Azure Synapse, Data Lake) and big data technologies like Delta Lake. Familiarity with cloud-native architectures and services on Microsoft Azure. Background in building and deploying scalable data solutions within an enterprise environment. Certifications (if any): Microsoft Certified: Azure Data Engineer Associate or similar certifications are preferred, but not required. #J-18808-Ljbffr Compunnel

Job Tags

Similar Jobs