Kavtech Solutions (Private) Ltd.Senior Data Engineer
Mar. 2022 - Mar. 2025Lahore, Punjab, Pakistan■ Spearheaded the design and implementation of comprehensive data solutions on the Azure cloud platform, covering data ingestion, processing, storage, and visualization. ■ Led the migration of Microsoft Dynamics 365 NAV on-premises server data to Azure Data Lake Gen2, utilizing Python and PySpark for efficient data extraction, transformation, and loading. Streamlined the process by ensuring data integrity, scalability, and alignment with organizational cloud strategies ■ Developed and optimized robust ETL pipelines using Azure Data Factory and Azure Databricks, enhancing data integration speed by 25%. ■ Leveraged Python and PySpark to engineer scalable and high-performance data workflows tailored to complex business needs. ■ Applied advanced data modeling techniques to create efficient, well-structured data pipelines and ensure consistency across databases. ■ Leveraged Azure Data Lake Storage for storing large volumes of structured and semi-structured data, enabling scalable and cost-effective solutions. ■ Championed data governance practices, ensuring data quality and compliance with organizational policies and regulations. ■ Transformed static Excel reports into automated Power BI dashboards, achieving a 60% performance improvement and enhancing data accessibility. ■ Conducted comprehensive data analysis, uncovering patterns and insights. ■ Automated data validation and testing processes with Selenium and Python, ensuring the reliability of data across systems. ■ Extensively utilized Azure services such as External tables and Delta Tables for efficient data management and processing. ■ Created dynamic, interactive dashboards using Power BI and Tableau, driving a 20% improvement in decision-making effectiveness. ■ Collaborated with cross-functional teams, meeting diverse data requirements with a 95% satisfaction rate. ■ Provided mentorship and technical guidance to junior team members, fostering a culture of learning and professional growth.