APPLITECH SOLUTION PVT LTDData Engineer
Nov. 2019Develop an automated data lake solution on AWS using Python and services like S3, Glue, and Redshift, enabling seamless data ingestion, processing, and querying. The pipeline ingests data from various data sources, performs transformations, and loads the processed data into Redshift data warehouse for analysis. Position Data Engineer Responsibilities Design and configure the data lake architecture Develop ETL workflows using AWS Glue crawler, Glue jobs, handle data ingestion and transformation Load the transformed data into Redshift for further analysis and reporting. Ensure data quality and integrity. Technical Skills Python, SQL, AWS, Pyspark, AWS Glue, Redshift, Apache Airflow