dataautomation 2023.09 - 2023.11
Public Parking Lot Big Data Analysis Project (Seoul City)
Data Engineer
Participated in a project to analyze optimal locations utilizing Seoul City's big data (parking lots, commercial districts, population movement). Constructed a data processing ETL pipeline and an API for ML training using Python and AWS services, and visualized the analysis results on a GIS-based map.
담당 역할
- • Data Engineer: Responsible for the technical design of the overall data pipeline and analysis models, and built and operated an AWS-based automated data collection and processing pipeline.
성과
- • Advanced to the finals and won the Excellence Award at the Seoul City Big Data Contest
- • Realized pipeline efficiency by optimizing large-scale data processing processes.
기술 환경
OS: AWS-based Cloud Environment / DB: NoSQL (AWS DynamoDB) / Languages: Python, SQL / Cloud: AWS Lambda, AWS API Gateway, AWS S3, AWS DynamoDB / Tools: QGIS, VSCode, DBeaver
Tech Stack
Python SQL AWS Lambda AWS S3 AWS DynamoDB QGIS