Keep in touch with meI'm using Intch to connect with new people. Use this link to open chat with me via Intch app
Work Background
Senior Data Engineer
DATAECONOMYSenior Data Engineer
Sep. 2022India, PuneLed end-to-end modernization of legacy DB2/Mainframe systems, implementing Delta Lake with Medallion Architecture to build high-performance Apache Spark pipelines on Databricks. Designed and developed scalable ETL solutions using Azure Synapse Analytics, ADF, Azure Logic Apps, Databricks, PySpark, and Python to ingest, transform, and load large data volumes into Delta Lake and Hive metastore. Migrated and optimized banking pipelines from DataStage to Databricks, Snowflake, and AWS Redshift, improving reliability, scalability, and cost efficiency. Built and tuned Snowflake data models with clustering, partitioning, and materialized views to accelerate query performance and analytics for business stakeholders. Developed Python-based data validation frameworks and utility scripts to automate data quality checks, reconciliation, and ETL orchestration, reducing manual intervention by 40%. Owned the migration of legacy WebFocus reports and ETL workloads to Power BI and Dataiku workflows, enabling advanced reporting, cloud-native transformations, and self-serve analytics. Architected Power BI data models using star and snowflake schemas to deliver interactive dashboards with embedded capabilities and secure Row-Level Security. Spearheaded migration of SSIS pipelines to Azure Databricks, ADF, and Synapse, optimizing large-scale data loads and reducing ETL runtime significantly. Collaborated with stakeholders to define requirements, implement automated data-quality checks, and set up monitoring alerts for pipeline failures using Python and Databricks jobs. Authored comprehensive documentation and runbooks for onboarding and troubleshooting, recognised by senior leadership for clarity and completeness. Created automation runbooks and orchestrated jobs using Control-M and Autosys, streamlining ETL operations across hybrid environments. Developed PoCs to validate business cases such as Power BI embedding, real-time reporting, and Python-based ETL utilities. Followed CI/CD best practices by leveraging Git, Bitbucket, and Azure DevOps for version control, testing, and automated deployment pipelines.
Member of Technical Staff
Mindstix Software LabsMember of Technical Staff
Jun. 2019 - Aug. 2022India, PuneCollaborating closely with senior executives, delivering data-driven insights to inform and shape business strategy. Designed and developed scalable ETL pipelines using Azure Databricks, ADF, Azure Synapse Analytics, and PySpark to ingest and process high-volume data from diverse sources. Transformed raw datasets into structured Delta Lake and Hive metastore tables using Databricks, enabling efficient querying and downstream analytics. Implemented automation flows with Power Automate and Azure Logic Apps to reduce/remove manual dependencies. Implementing row-level security and query parameters-based access control to enable selective access to analytics reports using Power BI and Tableau. Establishing robust data pipelines to facilitate seamless data ingestion, storage, visualization, and analytics processes. Designing and developing star and snowflake schemas for high-performance data models. Took initiative in creating training resources to support the training of new team members. Collaborated with cross‑functional stakeholders to define requirements, implement data‑quality checks, and set up monitoring alerts for pipeline failures. Authored documentation and runbooks for onboarding and troubleshooting; recognised by senior leadership for clarity and completeness. Followed CI/CD best practices by leveraging Git, Bitbucket, and Azure DevOps for version control and deployment automation.
Intch is a Professional Networking App for the Future of Work
300k+ people
130+ countries
AI matching
Get Intch if you'd like to
📝
Get Recommended to New Customers
👋
Connect with Like-Minded People
💸
Raise Capital
🚀
Find a side project
😎
Reach Top Niche Experts
⚡
Find Jobs on a Hidden Market
💼
Find a Perfect Co-Founder
🙌
Find a Trusted Mentor
🤝
Hire Top Global Professionals
See more people like Vivek on Intch
IT
1023999 people
15
Business process lead @ Tata consultancy Services
212
AI Marketing Automation and Digital Strategist @ GV Shri
22
Founder @ Accession
ITBI Analyst
83174 people
212
AI Marketing Automation and Digital Strategist @ GV Shri
18
Founder & CEO @ GVM Technologies LLC
15
BI Developer , Qlik Sense Data Architect @ Betsson Group