Cognizant SoftvisionData Engineer
Apr. 2011 - Aug. 2021Cluj-NapocaData Engineer responsible for ETL processes used by one of our clients. As main responsibilities:
▪ collaborate with business and process owners to understand business issues
▪ create and maintain optimal data pipeline architecture
▪ perform quality tests on ingested data
Solutions and tools used: Azure Data Lake Storage, Databricks, PySpark, Airflow, Python, SQL, Talend Other ETL related projects: EDW on Azure Cloud SQL for a US based financial company. Important features implemented: ▪ the ETL process using Airflow, Python and SQL ▪ connectors to multiple data sources (FTP, MSSQL, MongoDB, CosmosDB, SalesForce, GA, 3rd party APIs, web scraping) in Python ▪ ODS implementation based on Azure Data Lake data ▪ data cleansing for Data Science Team using Python and Pandas
▪ perform unit & integration tests ETL for online clothing store using GCS and Google BigQuery ▪ worked closed with Product and Mobile team to get needed metrics ▪ create BigQuery jobs for custom data exporting ▪ create BigQuery scripts for ad-hoc data reporting ▪ handle unstructured data ▪ data cleansing and export to a 3rd party system Other responsibilities: ▪ peer code review, project guidance, mentoring, technology related presentation