Independent / Freelance ConsultantSenior Python Azure/AWS Snowflake Power BI Data Engineer
Nov. 2013 - Dec. 2024LondonCollege Intranet
Develop an intranet for the college with a service-oriented architecture to migrate data from existing CRM and process data from inquiries to student onboarding – extract data from SAP BW (CDC) into Azure, ADF to staging, model data transformations using dbt, SQL model on snowflake, python-based transformations using data bricks, Power BI for reporting.
Assist in the implementation of a full DevOps process to build, test and deploy code using CI\CD and integrate package sharing using Azure Artifacts. Travel Portal (Startup)
Creation of logical and physical models to build a highly scalable database to support users, vendor and transaction data in the enterprise application.
Collaborate with cross-functional teams to architect a portal for spiritual travel (gaining a multi-million-$ investment) by sharing expertise and building a continuous learning culture.
Implement optimized data ingestion pipelines to load data using REST API and from files, Kafka and other internal databases, AWS - Kinesis for data ingestion, S3 cold storage, Lambda and Glue for data processing, DMS and Glue for data migration, Snowflake to manage all user/vendor transactions, Athena and Quick Sight for querying and visualizations.