AstraZenecaSenior Engineer
May. 2018Chennai Area, IndiaAs Senior Data Engineer, orchestrated the transformation logic using DBT in Snowflake, streamlining data processes and ensuring high-quality outputs for stakeholders; developed and maintained custom DBT macros and models, setting the standard for data engineering workflows. Spearheaded the optimization of data storage solutions, selecting between HDFS and cloud-based options, and integrated AWS Kinesis and Lambda; led the design and automation of ETL workflows using Apache Spark and Airflow, enhancing the reliability and efficiency of data pipelines. In a leadership role, mentored the data engineering team on DBT best practices, fostering a culture of excellence; led the migration to cloud-based data integration, designed ETLs using Apache Spark and Airflow, and managed a team of 5 data engineers, collaborating with management to build high-performing analytical models.